Here’s What To Do If You’re Drowning In A/B Test Requests

You’ve done the work to get your A/B testing program off the ground: you’ve secured buy-in by communicating all the ways A/B testing can benefit your business; you’ve purchased and implemented a testing tool, and you’ve trained everyone on its capabilities. Now comes the hard part: logistics.

Seeing your colleagues get excited about A/B testing is one of the most fulfilling parts of starting a website optimization program. But when requests for tests come flooding in like ballots on election day, how do you manage them?

The answer? An intake form.

In order to scale properly, it’s critical for your testing program to establish an intake process to manage your testing roadmap. An intake form will help you organize your requests, and determine not only which tests to execute, but also how much of your testing team’s budget, time, and energy should be spent towards executing each test in the queue.

There are several ways to create an intake form, but it depends on what’s most efficient for your organization. At Brooks Bell, we’ve seen clients use a variety of formats for testing intake forms including email templates, Excel or Word documents, Google Sheets and Jira / Confluence.

Download Your Free A/B Test Intake Form Template

No matter which format you choose, it’s important that the form and the broader intake process accomplishes two things: it should enable your stakeholders to submit requests efficiently and also enable you to easily organize requests and respond in a timely manner. This is critical as a timely response ensures your stakeholders’ excitement around testing doesn’t diminish and that they don’t lose trust in the process.

Here are some fields that should be on your A/B test intake form:

Test Name

A short name to identify the test request. We suggest using this format to keep everything consistent: Page Name – Test Idea (ex: Homepage – Hero Test)

Description

A short description of the test. This will help you estimate the level of effort for the test or raise an early red flag if the request is not feasible through the testing software.

Page URL

Provide the testing page URL(s). With this information, you and your team can confirm that the testing tool exists on the page and whether there is enough traffic to the page to run a test. Seeing the page(s) will also enable your developers to provide a more accurate estimate around level of effort (LOE).

Device

Are you testing an experience for desktop users, mobile users or tablet users? Or perhaps your test will be launched across all devices. This is often something that someone submitting a website test request may not consider, but an important distinction as it affects your estimated LOE as well as calculating sample size and duration.

Audience

Is this test targeted to all audiences, or is it targeted towards a particular segment? Possible segments could include new or returning users, those that came from a specific campaign, users that have signed up for an account or made a purchase, and/or those that have subscribed to your email newsletter.

Goal / Primary Metric

What’s the primary objective of the test? To increase revenue, engagement with the page, click through rate or perhaps something else. Many times, we find that submitters may not have the correct answer or may not know the answer to this question, so it’s important that you and your team have an opportunity to provide feedback as well.

Secondary Metrics

Are there other metrics that the requester would like to measure? Depending on how your testing software is configured, this may require your team to set up custom metrics before launching the test.

Often times, we hear stories of stakeholders asking “how did the users perform against [insert metric]?” Addressing this in advance helps you from having the awkward conversation that the test wasn’t set up to capture that metric.

Hypothesis

With any test, you need a hypothesis, a prediction you create prior to running a website test. It states clearly what is being changed and what you believe the outcome will be.

At Brooks Bell, we also like to include a few Whypotheses™– a term we coined for statements that address possible reasons why a test would win, and what that could say about your target audience.

For example, a hypothesis would be “changing product imagery to lifestyle-focused imagery will lead to an increase in AOV when purchasing from the product page.” In this case, one associated Whypothesis could be: “If this test wins, we may learn that customers prefer lifestyle imagery over product imagery because of concerns around how the product fits.”

Whypotheses are a part of our Insight Altitude framework for using A/B testing as a means of learning big picture insights about your customers.

Desired Launch & End Date

This is an optional field but can help you identify if there are any time constraints to this test. For example: does the test need to launch before a certain campaign goes live on the page? Does the stakeholder need to see the test results before a certain date? Is there a code freeze period? Knowing these details ahead of time will help you prioritize the test request.

Other Information

Another optional field, but important to include so that the submitter can provide any extra information or necessary context to support their request.

Intake forms can vary in length. The above form is ideal for companies that are just getting started with website testing because it maximizes testing momentum by limiting the number of fields. However, as your organization becomes more familiar with optimization, you should add additional fields–such as a field for the stakeholder to attach any supporting files, user research, voice of the customer data, previous test information, or outside data resources.