March 5, 2026
Why experimentation is becoming an operating model for smart organizations
A conversation with Valentin Radu, founder of Omniconvert, on experimentation as an operating model, AI and sustainable digital growth. Read more
On Thursday, June 14, Online Dialogue hosted the 27th Dialogue Thursday (DiDo). Especially for the 27th edition, we have invited none other than Niels Reijmer of the Beehive and Ellen Tonkens of wehkamp invited to share with us their experiences and knowledge in the field of CRO. The ultimate candidates for a successful, high velocity CRO program. The day started with an introduction by Tom van den Berg:
Digital developments have made us increasingly capable of validating all assumptions about our users and customers. Fast-growing organizations such as Booking.com, Facebook, Amazon and Google are already convinced: the more and faster you validate, the faster you grow. Tom van den Berg shared three tips to establish a validation culture within an organization.
Always establish an Overall Evaluation Criteria (OEC) before you start experimenting. By agreeing in advance what the main goal is, you avoid unnecessary discussions after an experiment. For example, when a variant scores better on conversions, but returns also increase. One option is to set up a formula that takes the different goals into account. Another possibility is to take one main metric and a number of guardrail metrics. If the main metric is positive and the guardrail metrics do not fall below a certain threshold then the experiment can be deemed a success.
We often see that effective and efficient working are seen separately. On the one hand, working methods such as scrum, sigma and kanban are introduced. The focus is mainly on efficiency: delivering something in the shortest possible time. This often does not consider the returns or ROI (revenue/cost) that these methods deliver. By combining optimization and a working method like scrum, you combine efficiency and effectiveness. That way, you know exactly what your adjustments/additions deliver and avoid implementations of elements that might have a negative effect on your conversions.
We regularly see that when people think about growing within the CRO program, they mainly look at increasing the quality per experiment. But which is easier? Double the quantity or double the quality? As the calculation above shows, the cost of an experiment increases faster than the value an experiment will add. For this reason, before we start experimenting with all our clients, we do a bandwidth calculation in which we can see exactly how much can be experimented with, but also how many opportunities the organization will miss once they do not use the entire bandwidth. In short: don't bullshit, but test.
By combining data from all touchpoints, you can create an overall picture of the visitor. But how do you convert all that data into relevant information for the visitor? And how do you continuously tell the same story? In his presentation, Niels Reijmer shared how Bijenkorf creates segments and applies personalization to the website, emails and other online marketing channels.
In recent years, Bijenkorf's online sales have grown rapidly. To sustain this growth in the coming years, CRO is very important. Bijenkorf has been doing Conversion Optimization since 2012 and over the years they have grown from a conversion manager who did everything himself, to an organization in which all product teams are responsible for their own optimization work. To take their CRO program to the next level, they have been working hard over the past few months to take the next step.
Bijenkorf has a huge number of products making filtering very difficult. One solution to still show customers relevant products (without flooding them with products that are not interesting to them at all) is personalization. But where do you start?
To keep it simple, they started with brand-based product segmentation: you cluster all the brands that are often sold together and you give names to the clusters. To test whether the clusters are actually valuable, they set up email campaigns to A/B test them. Gradually, in this way, they added and removed more and more attributes when they did not have a positive effect.
After testing some attributes in the mail, it was time to move over to the website. Unfortunately this took a little longer than planned, but after some trial and error it was possible to segment within the web shop as well! By continuously testing and segmenting, Bijenkorf had learned an enormous amount about their customers, so it was time to move on to personalization. By processing all the information they had gathered into a neural network, they can now recommend new products of interest to a visitor based on previously purchased products.
In short, if you want to personalize, follow these steps:
And in the end, it only comes down to one thing: just get started.
Since last year, wehkamp has begun a transition, experimenting more and more at an ever-increasing pace. How do you set up such an experimentation process, how do you scale it up and what growing pains are involved? In her presentation, Ellen Tonkens shared how she is managing experimentation at wehkamp and getting the process ready to scale up.
Like Bijenkorf, wehkamp works from product teams that collaborate on CRO with the product owners (and possibly a ninja). Each team has its own North Star and all experiments revolve around that North Star. During the sprints, there are fixed moments scheduled for experiments that are largely conducted under the direction of the product owner.
In addition to the product teams, there are customer experience, experimentation and interaction design teams that work as a kind of staff department over all the teams. The experimentation team has weekly meetings with the product owners to make sure everything is going well and everything is being measured properly. During shop performance meetings, all teams report the impact of their experiments and then an estimate is made of the yield from A/B testing which is reported to management.
Over the years, they have grown considerably and scaled up quickly in their CRO program, but even this was not without setbacks. Ellen Tonkens shared her key learnings:
To take the level of their CRO program up another notch, wehkamp wants to further scale up the number of experiments they do. In this process, however, the experimentation team must not become a bottleneck for the other teams within the organization, so they are making the move to a Center of Excellence: the process will be smoothed and standardized, the tests must be well documented, and the focus shifts from execution to training.
To get everything done, wehkamp has set a number of goals for the coming period:
De Bijenkorf and wehkamp are growing rapidly and the conversion optimization / personalization program is an important part of this. Within both companies there is a data-driven culture in which teams are given the freedom to validate ideas. This validation ensures that the ideas that go live actually add value for visitors and the company.
Want to learn more about the 27th Dialogue Thursday? Watch the compilation video for an overview of the afternoon, click on the links below for the slides of all the presentations or sign up for our newsletter To keep up to date with all our events!
Nederlands
English