Do we start developing a new service? Are we going to create an app to engage customers? Do we want to improve the old product or develop a new one? To arrive at choices, organizations are increasingly using data and experiments. Assumptions are tested and the insights gained are used to guide choices. The focus of these experimentation programs is often on growth, namely generating as many additional transactions as possible. However, using experimentation and the value it delivers can be applied much more broadly and is a common thread within all your marketing and business development activities.

In this article, I want to show in what different ways experiment-driven work can deliver value within any organization. We look beyond the universal desire to grow and focus on different business strategies and their corresponding validation methods. How can you use experiments strategically and what steps are important in doing so?

Value Strategies

One way to look at experimentation, growth and strategy is with The ‘value strategy’ model of Treacy and Wiersema. By now a somewhat older model, but it provides an interesting perspective on the value strategy of different organizations. In their model, Treacy and Wiersema indicate that teams and organizations choose different strategies to focus on, but one strategy always prevails. They distinguish between three different strategies:

• Product Leadership: the product is central. The organization excels in this area because it provides ‘the best’ product on the market.
• Customer Intimacy: the customer (experience) is central. The organization focuses on long-term customer relationships.
• Operational Excellence: the organization seeks to optimize operational processes. This enables the organization to minimize costs (or growth).

Different value strategy, different experimentation

For each of these strategies, conducting experiments helps you make good, fast choices that lead to new products, long-term customer relationships and low costs. But for each strategy, the focus is slightly different and the steps you go through within the experimentation program differ.

Product Leadership

If the goal is to bring new products and propositions to market and be innovative or even disruptive, then follow Product Leadership. For this strategy, it is best to follow the following roadmap:

1. Quantitative and qualitative data. 

• Do a data quality check so you know the data is correct.
• Collect as much data as possible from your online channels to analyze how your customers behave.
• Supplement this with information from customer feedback.

2. Market data and trend analysis. Through (SWOT) analysis and competitive analysis, you find out where the opportunities are in the market and where the market may be moving. This allows you to determine, based on your own strengths and weaknesses, which trends you want to hitch a ride on and which you want to develop further.

3. Rapid Assumption Testing. List all the ideas you have and try to figure out the assumptions underlying those ideas. Keep asking the question: Why would you want to launch this new product? Once you have the assumptions in order, see what ways you can validate those assumptions (instead of putting all the time and effort into developing something new without knowing if it will work). Learn more about Rapid Assumption Testing can be found here Finding.

4. Validate with MVT (multivariate). Once you have validated interesting assumptions, you want to quickly learn how to market your new proposition. A/B testing is too specific for this. Use multivariate testing to quickly figure out the best move.

Customer Intimacy

Customer Intimacy focuses on the customer and the customer experience. The organization focuses on the long-term relationship with the customer. What roadmap do you use to really learn about your customers' behavior?

1. Data quality check.

2. Lots of quantitative data supported by qualitative data. You want to learn from behavior, but behavior is very complex. Therefore, try to extract as many insights as possible from your data in combination with customer feedback (usability tests, surveys, et cetera). Keep in mind that what people say, can be very different from what they do.

3. Focus on behavioral sciences. To learn from your data and experiments, you need to understand consumer behavior. In this way, you will learn better and better what will work for your customers and gradually begin to recognize patterns of behavior that you can capitalize on.

4. A/B testing. An A/B test is the best way to find out what effect manipulation has on behavior. You want to set up multiple experiments per hypothesis to find out the effect of your hypothesis and what behavior takes place.

Operational Excellence

With Operational Excellence, you focus entirely on cost savings. You want to be as efficient and effective as possible. Experiments help by quickly testing whether and in what form it works. That way you know what features, funnels, designs, technology and application to build. The moment you don't do this, you don't know if it works and you may end up spending a lot of time, energy and money on digital development that visitors and customers don't need.

To achieve Operational Excellence, follow the following roadmap:

1. Qualitative check.

2. Data and impact analysis. To figure out how to offer your customers a better and more efficient product, you need to know exactly where the gaps are currently: where is the drop-off, why do people drop out at a specific place, where is the best place to put your energy to increase your conversion rate? Where can you be as efficient and effective as possible?

3. Lots of testing. You want to learn quickly how to improve the organization. That means a lot of experimentation based on test ideas that come from the business. This can be A/B testing, but also multivariate testing. Finally, it doesn't matter if you can prove a hypothesis: it's all about the extra euros you can add to the bottom line or the costs you didn't incur.

Conclusion

With each of the strategies, you want to be able to make quick and informed choices. To arrive at these choices, it is very important that the data are correct, tools are implemented correctly and the design of your experiments is correct. In each of the steps, you go through the phases of: research, assumptions, testing and do/don't do.

Treacy and Wiersema's value strategies provide guidance for making better decisions from different strategic decisions (product innovation, customer knowledge and operational forward thinking) in different ways with experimentation.

Of course, the strategies may overlap with each other and you may see this reflected in your experiments, but the important thing is that you experiment across the board and choose the way of experimentation that best suits your organization.

Read how we handle change management

One of Online Dialogue's clients is Amsterdam-based Travix. Travix is the parent company of a large number of Online Travel Agents, such as Cheaptickets, Vliegwinkel, Flugladen, Vayama and BudgetAir. Besides increasing conversion rates, the cooperation between Online Dialogue and Travix is mainly focused on getting CRO - or rather Evidence Based Growth - into the genes. The ultimate goal is for all of Travix to be data- and evidence-driven.

Read Travix's customer case study