Last Thursday, the 47th edition of Dialogue Thursday (DiDo) took place. DiDo is the knowledge event for professionals in conversion optimization, product development and data-driven working. This edition revolved around effective collaboration with product teams. In this article you can read the most important takeaways from the presentations of experts from Booking.com, VodafoneZiggo and Online Dialogue.

Talk 1: Building an experiment-driven game plan for your product - Dennis Nederlof (Online Dialogue)

Dennis Nederlof, conversion manager at Online Dialogue, kicked off the afternoon with a practical talk on how to adopt a structured experimentation framework as a product team. He drew a parallel between rugby and product development: just as a scrum-half in rugby manages the team and monitors strategy, plays a product owner a crucial role in connecting development and business.

The Opportunity Solution Tree (OST) framework.

A good experimentation process begins with a clear hypothesis and a structured framework. In his presentation, Dennis talked about the Opportunity Solution Tree (OST), developed by Teresa Torres. This model helps teams base their roadmap on user insights and experiments rather than assumptions.

This framework consists of three core components:

• Outcome: What is the desired business goal or KPI?
• Opportunities: What customer needs or frictions need to be resolved?
• Solutions: Which hypothesis-driven solutions can address these problems?

From output to outcome: getting the right focus through experimentation

Many product owners are still being judged on output (features shipping) rather than outcome (impact on KPIs). But a list of delivered features says nothing about the actual impact on conversion and customer value.

According to Dennis, there are three main areas of concern:

• Stakeholder buy-in. A strong experimentation strategy and clear KPIs reduce internal resistance.
• Use data in prioritization. 21% of product owners rarely speak to users. By using both qualitative and quantitative data, you can optimize in a more targeted way.
• Iterative work accelerates the learning process. With Minimum Viable Tests (MVTs). you can validate faster and minimize risk.

Core message: Do you want to be successful in product development? Focus on validation, not assumptions.

Talk 2: How to leverage the power of developers to supercharge your CRO program - Maite Bemelman (VodafoneZiggo)

Maite Bemelman, CRO specialist and analyst at VodafoneZiggo, discussed how developers can become your strategic partners in CRO. Many CRO teams think primarily in conversion rates, while developers value code quality, performance and scalability. She talked about how to bridge these disparate priorities. 

How to make experiments relevant to developers

Linking experiments to technical priorities can help teams strengthen collaboration. Maite gave the example of a backend developer who complained about a cumbersome, manually maintained notification in a dashboard. From a UX perspective, the notification was also unnecessary and conversion-reducing.

Instead of intervening immediately, a A/B testing with guardrailmetrics set up to measure the impact. In this way, the various disciplines were able to convince internal stakeholders of the adaptation. 

And the results:

• Higher CTR, with no negative impact on conversions.
• Improved site speed, which benefited both UX and SEO.
• More developer confidence in CRO testing through a collaborative approach.

Developers as allies in experimentation

Maite emphasized that developers are not only performers of experiments, but can actually play an active role in a culture of experimentation. A good example of this is the redesign of the Ziggo Dashboard Widgets.

In this case, the product owner was reluctant to make changes while developers identified technical inefficiencies. Through an experiment-driven approach, the product owner was convinced of the added value.

The result: an improved UX, a 10% increase in average order value and fewer internal searches.

Core message: Developers can become your greatest allies if you link experiments to their priorities: scalability, performance and quality of code.

Talk 3: 20 years of experimentation at Booking.com - Jorden Lentze (Booking)

Jorden Lentze gave an insight into 20 years of experimentation at Booking.com. What began as a means of managing stakeholder expectations grew into a core strategy for product development, CRO and innovation.

The evolution of experimentation at Booking.com

• More than 1,000 experiments are running simultaneously.
• 2,200 employees launch experiments weekly, 750 daily.
• It took 2.5 years and 114 failed tests before the first successful experiment went live.

The nine principles of a culture of experimentation

Booking.com uses nine core principles to make experimentation successful:

1. Mistakes are allowed, as long as you learn from them.
2. No HiPPOs: decisions are based on data, not opinions.
3. 100% transparency in experiment data.
4. Anyone can conduct experiments, not just analysts.
5. Experimentation is social: teams can follow, like and comment on experiments.
6. Learn quickly, and don't assume that results are reproducible.
7. Test everything from small UI changes to internal processes.
8. Small, autonomous teams work more efficiently and faster.
9. Give teams the freedom to make their own decisions.

Better measurements, better experiments

Booking.com improved their metrics by reducing noise and increasing reliability:

• 15% less measurement noise, making experiments 50% faster could be completed.
• Predicted Net Customers metric reduced false positives with 6% and false negatives with 8%.

Small changes, big impact

Sometimes small changes have big consequences. A UI test in which only one icon was changed caused a loss of sales large enough to fund a team for a year. This underscores the importance of careful testing and validation.

Key insights from Jorden's talk:

• A successful experimentation program requires innovation on multiple fronts: tooling, methodology, culture and decision-making.
• Decentralized decision-making accelerates development and increases the speed and quality of experimentation.
• Better measurement leads to better decisions: continuous improvement of measurement methods increases the reliability of experiment results.

Conclusion: validation encourages innovation

The speakers at DiDo #47 demonstrated that a strong culture of experimentation leads to better products, better decisions and a faster innovation cycle.

The next edition of DiDo will take place on June 12, 2025.