On Thursday, March 5, 2020, we organized the Dialogue Thursday (DiDo) for the 32nd time. For more than ten years our customers and relations have been coming together during the DiDo to share knowledge and experiences. This time the speakers and visitors addressed the question: ‘How do you set up a culture and process of validation?’.

Speakers from Hallmark and ANWB answered questions such as: What does the optimal CRO process look like? How do you set up a validation culture? What or do you need for this? And how do you get the organization on board? In addition, Tom van den Berg talked about his experiences with Airtable.

How do you use Airtable to speed up your process and get more learnings from your CRO program?

At Online Dialogue, we have been working a lot with Airtable as a project management tool since last year. Tom talked about his experience with Airtable, how Airtable provides additional insights into learnings and how to speed up the process with Airtable.

At Online Dialogue, we use a huge number of tools to capture and analyze experiments. Therefore, we wanted to reduce the number of tools by bringing multiple functions together in one tool.

Tom shared during his presentation how we are using Airtable to speed up the CRO process on the one hand (database and Kanban are combined instead of being spread across different tools) and on the other hand also get more learnings and insights from all the experiments done.

In addition to the benefit of saving time working with Airtable, there is also less risk of errors because you don't have to share data between different tools. The input of test ideas, prioritization and planning all run through Airtable. And right now we are also analyzing tests in Airtable to save even more time.

The basic version of Airtable is free. But if you want to add many users or large files, you need the paid version. In addition, the paid version also includes a dashboard feature, which gives an overview of all your results.

How do you set up a successful CRO program within a year?

The second speaker at the DiDo was Lisanne van de Weijgert. She works as a conversion manager at Hallmark. When Lisanne started at Hallmark a year ago, testing had never been done. But great strides have been made in a year in the area of CRO. On average, more than one test per week has been launched, and that includes great winners that are also implemented quickly. In addition, the organization has been enthused about CRO and ideas are being submitted from different departments. 

The CRO virus

Lisanne begins her presentation with current events; “Last week, the word coronavirus was mentioned in more than 3,000 media mentions a day.” Lisanne uses the metaphor of the virus: just like a virus, people get infected, pass it on to others and build resistance. Lisanne takes us through how she has dealt with this resistance over the past year at Hallmark.

The plan was to work using a structured process at Hallmark, but the reality is very different. According to Lisanne, within CRO we tend to focus very much on the ‘what’ and ‘how’ (golden circle by Simon Sinek). These are all kinds of substantive reasons, while this is often not the right method to convince colleagues within your organization. Here, above all, try listening to motivations of your colleagues and what goals they have and how CRO can connect to them. The ‘why’ from both sides is much more important and will get you into a conversation and explore how you can help each other. So in a conversation, don't go into what you are testing but say why you are testing.

Who do you need to infect? 

If you want CRO to have wider support within the organization and you want more ‘fans,’ look for the early adopters for CRO within the department or organization. These are the people who embrace CRO, find it interesting and ultimately can help you get it further into your organization. If you have to do this alone, it takes a lot more time and energy. With some early adopters it can spread like an oil slick and you make strides faster. This early adopters finding at Hallmark succeeded in part by creating a (test) idea box; in addition to ideas, this is how you discover who the early adopters are.

To create a validation culture, Lisanne says there are a number of conditions; 

• Early adopters and faith from above.
• Long-term focus and a lot of patience.
• An environment where you are allowed to fail, but when you fail quickly show that you are learning.
• Gain confidence from the organization, but also in yourself.

Key takeaways from Lisanne's presentation are:

• Start with the ‘why’ and only then with ‘how’ and ‘what.
• Divide your energy smartly and focus on people who are already enthusiastic.
• Embrace resistance and invite people to express this resistance in order to together moving forward.

The growth of an optimization and validation-driven culture within ANWB

Four and a half years ago, Michaël Hamelinck started as a Web analyst at ANWB. During his presentation, he discussed setting up experimentation over the years. What were the challenges of recent years and how were they tackled?

When Michael started at ANWB, he had a bright outlook; after all, ANWB has enough traffic and conversions for dozens of experiments a month. But there were also challenges; ANWB only did one experiment a year. And there was no development and UX capacity, a drawn-out process for putting an experiment live and a culture of putting adjustments live without validating.

Quantity

To address these challenges, Michael started to focus more on validation: extracting substantiation from data and speeding up the testing process (fewer steps and stakeholders). With success, test volume increased dramatically. Unfortunately, only one in eight tests was a winner. The quantity was there, but the quality lagged behind. So ANWB set to work on the question, ‘How do we increase the chances of a winning experiment?’.

Quality

Soon Michael discovered that although hypotheses were being used, the rationale was lacking and there were no clear KPIs. It was all done on gut feeling. To convince people to substantiate their ideas, Michael recommends showing the consequences of working on gut feeling.

Resources to back up your testing ideas include heatmaps, recordings, the digital assistant, previous tests, surveys and data from analytics.

Support

Michael found out that teams did not keep validating if the Web analysts were not there. The main reason for this was that teams didn't know how to test themselves. So Michaël started researching what stakeholders need to test themselves. The answer: structure and standardization. To create this, the 5V model From Online Dialogue.

To roll out your CRO process to multiple teams, it can be very helpful to standardize the processes and make them available to everyone in one place. Onboarding new teams, for example product teams, becomes a lot easier this way.

Goals for the future

Now, as team lead web analytics and conversion optimization, he and his growing team facilitate an increasing number of experiments within ANWB. There is a stable and automated experimentation program, development and UX capacity is available, and more and more ideas are being substantiated and validated.

The points Michael wants to work on in the coming years to expand the validation culture are: training colleagues, automation of qualitative research and that everything is validated and substantiated.

Key takeaways from Michael's presentation are:

• First, make flying hours of experimentation and show that gut feeling should always be tested;
• Work with reasoned hypotheses using clear KPIs;
• Emphasize the gains of experimentation, but more importantly, the losses of not experimenting;
• Standardization of a process helps in its adoption;
• Do research on internal stakeholders, where are their needs?

The 33rd Dialogue Thursday will be on June 11, 2020

We organize three DiDo's per year and the next edition is scheduled for June 11, 2020. Sign up for our newsletter If you want to keep up to date with all our events!

We always enjoy getting input for our events. So do you have an interesting topic you would like to hear about during the DiDo? Or do you have another good idea on how to fill the DiDo? Take contact with us!

Want to learn even more about conversion optimization? Check out our trainings.

Photos were also taken during the event. View them here.