What’s The Difference Between Client-Side & Server-Side A/B Testing?

Client-side and server-side testing tools are both powerful and beneficial, but they both have limitations that make it difficult to determine when to use which method. This post explores those limitations, and how using client-side and server-side testing together can enable you to run more powerful tests.

Client-side and server-side (aka full-stack tests) simply describe where the tests are run: client-side tests run in a user’s web browser and server-side tests run on the server. But while changing where a test runs may seem trivial, the impact is actually quite substantial. 

To understand why, you need to understand the process to load a webpage:

1. The user enters a URL into their web browser and a request for the page is sent to a server.
2. The server receives this request and immediately begins assembling the page. This assembly may include retrieving various files, requesting data from databases or other local sources that are needed for the page, maybe even sending additional requests to other servers for remote data, and any number of other things required for the page.
3. When the server is done assembling everything, it “serves” the result back to the user in the form of a webpage. 
4. Once received, the client computer begins to load the webpage. This includes making additional network requests to load additional javascript files, stylesheets, font files, tools like tag managers and analytics and rendering the visible elements on the page for the user to view and interact with.

Traditionally, client-side tests are run on or after step #4, with the testing tool and test code loaded with the other assets we mentioned. However, there are a couple of issues that arise from loading testing tools and test code at that step in the process.

The first issue with client-side testing is flicker.

As the testing tool is being loaded, a network request is made to the testing tool servers to run the testing tool itself. The page continues to load while it waits for the response and for the tool to load. 

Depending on which tool you’re using, once it is loaded, another network request is usually made to get any test code that needs to run on this page. Again, the page continues to load while this request is made. 

Page load is a perpetual race to see who can get there first—the testing tool or the visible elements on the page.

If these network requests take too long, the elements on the page may start to become visible. Once the test code reaches the page and executes, the visible elements are changed, resulting in a flicker from default to variation.

Page load is a perpetual race to see who can get there first—the testing tool or the visible elements on the page.

The second-most-common challenge with client-side testing derives from the limited amount of data that’s available at this point.

In step #2, the server only knows to load the data required by the default page. So when step #4 comes around, and the testing tool begins to load, there is limited data available for the test code to use. 

There are ways of getting additional data to the page, but this usually means: using static data in the test, rather than dynamic data, which may not be ideal or even possible; having to make yet another network request and increasing the risk of flickering; or, having to get your developer or IT team to add the necessary data to the default page which can significantly delay timelines. 

Finally, the last major challenge with client-side testing is the inability to test across different platforms. 

When a test is run in the user’s web browser, that data is stored locally–meaning if the user switches to another device, that previously saved data would not be available, which creates a disjointed experience for your users and could potentially taint your test results.

Additionally, if you wanted to test on both desktop device and a mobile app, you would not be able to use client-side testing for this.

Referring back to the process of loading a webpage, you now know that client-side testing runs during step #4. Server-side testing, alternatively, runs during step #2. 

With server-side testing, before anything is sent to the user, the server calls the testing tool, checks if there are any tests that should run on this page, executes the test code and then sends the page to the user. Neat, right?

Testing on the server-side solves for many of the challenges that come with client-side testing: Flicker is no longer an issue because the test has already executed before the page even reaches the user’s computer. If the test requires additional data, the server can be coded to include it in the webpage response. Finally, if you want to run a test across your website and your mobile app, server-side testing allows for this because the server acts as a central point for both platforms. 

Of course, while server-side testing is pretty great, it does have some limitations.

Server-side testing requires changing the default codebase (that’s the code that lives on your server) and does not come with a WYSIWYG editor. This means you absolutely need dev and/or IT support for even the most basic of tests.

Changes to server-side code are permanent, opposed to the temporary nature of client-side code that executes on top of the default code. This requires additional work to be done after the test—regardless of if the test wins or loses.

Permanent changes to the default codebase are higher risk. If a test needs to be turned off for some reason, you can’t simply deactivate a campaign, like you would in client-side testing. 

Server-side tests also take more work to complete, so timelines are not as fast-paced as in client-side testing. 

And finally, data collection on the server-side is not as granular as it is on client-side.

The server does not have access to individual DOM elements like CTAs or hyperlinks the way that client-side tests do. This means if your test requires tracking clicks on elements or some other form of user engagement, the server would be hard-pressed to track this.

How to determine when to test on the client-side and when to test on the server-side

By now, you’re probably scratching your head trying to figure out which one of these technologies you should use. They’re both powerful and hugely beneficial, but they also have their own downsides. Fortunately, your answer doesn’t have to be one or the other. In fact, we recommend using both technologies in your testing program.

You wouldn’t use a hammer to change your car tire just because you want to use that hammer, would you? The same thinking needs to be applied to these technologies. Just because you want to use server-side testing, doesn’t mean that it’s the right choice for the test you have in mind. 

Use Server Side Testing When You Need To:

• Run a test across different platforms and mediums, like website, mobile app and email
• Dynamically display different data to users
• Test significant changes to your product, including algorithms, back-end logic, and underlying features.

Use Client-Side Testing When:

• All the data and elements you need are already on the page
• You need to track user engagement with a DOM element to determine a winner
• The test can be easily created using a WYSIWYG editor

Test Yourself: Server-Side or Client-Side?

TEST CASE #1

The Challenge: increase signups for your monthly newsletter. 

Considerations: The majority of users that reach the sign-up page complete the submission, so the challenge is getting more people to this page.

The Plan: Test the homepage CTA’s (location, color, and copy) that redirects users to the sign-up page.

KPIs: CTA clicks and successful sign-ups across all variations.

Did you guess client-side? That’s what we would recommend!

The data and elements needed are already on the page, engagement with a DOM element is needed to be tracked, and if desired, this test could be created using a WYSIWYG editor so that no dev or IT support is needed! For these reasons, client-side would provide the most benefits for this test.

TEST CASE #2

The Challenge: You run a gaming website where users can buy and download their favorite games. You want to see if offering a discount to users who download a specific game from the site and create a new account in the game (rather than on the website), will increase the number of new accounts.  

Considerations: You only want to offer this discount to users who downloaded the game from the website, and not users who download it from some other source. Currently, you are able to track all account creations from the game, but you are not able to identify where the user downloaded it from.

The Plan: Test offering a limited time discount to users who download a specific game from the site and create a new account in the game rather than on the website.

KPIs: Number of New, In-Game Accounts Created Using Discount

Was your answer server-side? That’s ours too!

As mentioned earlier, client-side testing cannot run across different platforms. So in this case, the user is bucketed into a variation on the website, but there is no way for the game to know this once it’s downloaded and installed. So when this user creates a new account in the game, there is no way of knowing that the user was in a variation on the website via client-side.

However, with server-side testing, the server is able to act as a central point between platforms and share data between them.

In this test, we would assign the user a unique ID as they’re bucketed into a variation on the website and store it on the server. When they click to download the game, that ID can be passed along with the request, which could then be embedded into the game’s installation files. Once it’s installed on the user’s device, any requests made to the server from the game can include this ID, and we are now able to link the web activity and the game activity to each other via the server.

As you can see, both server-side testing and client-side testing are hugely valuable when used effectively.

While server-side testing is newer, it’s shouldn’t necessarily be viewed as better. We recommend using both of these technologies within your testing program and letting your each individual test requirements determine which is the right tool for the job.