How Do You Prove the Value of Experimentation to Executives?

For a testing program to be as successful as possible, it must be backed by an executive — a leader in the company who believes in the power of testing and champions the cause.

Getting executive buy-in, however, remains a struggle for many testing teams. So what can you do to turn skeptics into supporters? Brooks Bell experts share their advice.

Our senior director of experimentation strategy suggests showcasing more than the traditional metrics when you want to gain executive buy-in.

One is loss aversion, the idea that the pain of losing is greater than the joy of gaining.

“This is the basic principle of ‘look what I saved the company from doing,’” Suzi says. “By ruling out an experience via testing, you save the development resources that would have been spent just rolling it out — as well as the revenue that would have been lost as a result of the change.”

The other metric is testing potential. If you have a lot of traffic and conversions, you should be doing a lot of testing. Communicate to executives that you’re trying to use every opportunity to increase your engagement, signups, logins, and so on.

”Showing what your testing potential is now and measuring against it moving forward could be a great way to illustrate how you’re growing the program,” she says.

The goal may be to create tests that win, but executives need to understand that experimentation isn’t solely about winning, our senior director of optimization consulting says.

Learning from flat tests and losses — and not losing momentum when you encounter a series of them — is just as important as creating tests that get big lifts.

It’s also a good idea to remind executives why you test in the first place and what you’re driving toward.

“It’s not just about money in the end — it’s about the customer and the customer experience,” she says. So don’t test only for conversion; include tests that measure engagement.

Traditional marketers are used to an old-school model, says our director of optimization consulting – analytics. “They get two pitches from an ad agency and they decide on one,” Dave says. And the decision usually lies with the HIPPO.

In the testing and optimization world, a cohesive plan built on data analysis often yields higher value than a spaghetti-on-the wall approach. At the same time, you need to be cautious when communicating the potential value of a win.

For example, your test results may show 95 percent statistical significance. In that case, “we’re 95 percent confident in a positive result, we’re not 95 percent confident in a specific result or value,” he says.

Our VP of data science and analytics echoes Dave’s thoughts that a traditional marketing mindset can be a hurdle.

For example, executives may be used to spending more money getting customers to the site rather than testing them while they’re there. “The whole idea of conversion rate optimization is that getting them to their doorstep is only step one,” Reid says.

Helping executives shift their focus may be key to getting their support. “You need to show that testing is a new way to answer questions and solve problems,” he says.

Testing can be a great morale-builder, says our senior optimization engineer, so consider that angle when you’re trying to get executive buy-in.

“It provides a spark for internal teams, giving them an outlet to try new things,” Jeremy says. “It can help them feel like they have a voice at the company.”

Executives certainly have a stake in this: After all, if you can build a culture around that mindset — one that values everyone’s input — you can create a healthy company. Once you plant that seed, employees will proactively look for opportunities to test and make the company better.

“A lot of people have to work pretty hard to get that spark of innovation — it’s forced. A healthy testing culture can bring that about organically,” he says.

How do you prove the value of your testing program to executives? Share your thoughts in the comments below.