Reduce the Risk of Adding New Website Features with a Painted Door Test

When it comes to managing a website, it seems there’s always a new tool, technique, or feature that can be applied to improve the performance and experience. Take, for example, live chat widgets. These low-commitment communication tools have become increasingly popular and, reportedly, contribute to sizeable increases in conversions—sometimes 20 percent or more. It sounds great, but live chat is expensive to implement and manage—both in terms of development and maintenance. Double-digit increases in conversion could justify this cost but only if your customers actually take advantage of it. You could take a risk, arguing the opportunity was too promising to miss. Or, you could try a low-cost alternative to gauge the potential value of such a tool, minimizing the risk.

Read more: How does testing fit into agile development cycles?

One way to implement a low-cost assessment of a new feature is through a “painted door” test. The idea is simple: Instead of building a complete solution or feature, you build the suggestion of such a feature and measure how many people try to utilize it. Imagine two rooms that share a wall but are only connected via a long hallway. People may grow tired of the long walk and start asking for a more convenient door between the shared wall—but would they use it? Instead of sawing a hole between the rooms, you could paint a door on the wall with a doorknob screwed in place. With this fake door on the wall, you could sit in either room and count the number of people that try to open it—if there’s sufficient interest, it’s time to break out the saws.

In the case of the live chat feature, you could create a link or CTA on the relevant page. In one variation, the label would say, “Chat with a live representative.” In a second variation, it would say, “Have a representative contact you.” For the sake of this test, both variations would lead to a general contact page. The result, however, will be valuable. First, it will provide an indication of whether customers are interested in connecting with a representative at all. In addition, it will suggest whether there’s enough interest in live chat to justify implementing it or whether an email and phone number capture form might do the trick.

Live chat, of course, is just one application. Perhaps you are considering adding more content to your site; testing the addition of a blog link in your navigation—pointing, perhaps, to an existing newsroom or a single test post—might indicate whether users have an interest in such content. If you have a service, you could create CTAs that suggest pages for pricing, features, a demo, or contacting a representative to asses which is most important to prospective users—even if all these CTAs point to the same, general page.

Obviously, creating a painted door test is, in itself, not without risk. Creating the promise of a feature and failing to deliver fully on this promise can create a disorienting experience—and could even lead to a negative brand impression. Acknowledging this, it’s important to take great care in the design of painted door tests. Ensure that the result of an action produces some feedback. In addition, make sure that there is some way for a user to accomplish the desired task—in the live chat example above, they were directed to a contact page with information that could lead to a live representative, albeit not through a chat widget. Finally, consider limiting painted door tests to a small percentage of users, traffic volume permitting.

Whether you want to implement a new feature, new content area, or new direction in your site design, a painted door test provides a simple, low-cost way to assess the potential value of big ideas. Any organization with limited development or creative resources—and this is, no doubt, most organizations—can use this method to reduce the risk of bold changes and learn a bit more about users and customers, too.