Join our free event Dialogue Thursday on November 13. Sign up now!
What began as small-scale experimentation has evolved into an organization-wide CRO approach.
With training, inspiration sessions and internal guidance, CRO is firmly embedded in the culture and way of working.
The joint efforts led to several award nominations as well as the win of an Experimentation Culture Award.
In 2017, the collaboration between VodafoneZiggo (then just Ziggo) and Online Dialogue began. At the time, Ziggo was looking for a party that could help them with their next step In conversion optimization. Their CRO program at the time consisted mainly of testing by gut feeling, but that has changed tremendously over the past 4 years.
“Everything was still in baby shoes.” says Farshad Soleymani (Lead CRO at VodafoneZiggo) about those early days. There was still plenty of room to grow into more maturity. “I had already read a lot about CRO and I thought I knew it all, but it turned out there was still plenty to learn.”. To get that knowledge within the organization, Online Dialogue was hooked up, for more structure and a clear process.
Thanks to the FACT & ACT model of Online Dialogue slowly but surely a disciplinary working method emerged, in which different specialists work together, such as developers, designers, data analysts and copywriters.
Step 1 in the FACT & ACT model is to perform a technical check. This establishes whether the available data is actually correct and whether everything is set up correctly in the analytics program.
Annemarie Klaassen (Lead CRO Analyst at VodafoneZiggo) remembers that the first step was to run an A/A test to see if the technology was up to par. However, this turned out not to be the case. “There were suddenly 40% fewer visitors, winners turned out not to be winners at all and results turned out to be completely the other way around. We then spent six months trying to solve this”.
Then a new phase dawned where they could really get down to business, and it became a matter of convincing the teams and bringing them into the new process of setting up good A/B testing. Farshad: “At the time, Online Dialogue was in the lead on that, but the goal has always been to eventually be able to do it yourself. It really had to be learning-by-doing”.
The new way of working landed better in some departments than others. Convincing teams about the importance of online optimization and a data-driven mindset is an ongoing process even to this day. Farshad: “My role then was for mainly: how can we make the importance of CRO greater throughout the organization?”. This role became especially important because of the merger of Ziggo and Vodafone, both with different (business) cultures and ways of working.
What really helped was the arrival of Annemarie at VodafoneZiggo in the role of CRO lead. The big advantage was that she knew the vision and working method of Online Dialogue well and was able to further anchor it within VodafoneZiggo.
Online Dialogue's contribution then consisted mainly of thinking along about the part psychology plays in A/B testing, drafting ideas and hypotheses, and making sure the process within the various teams got going.
In 2017, they started with two CRO teams. Over the years, the importance of CRO has been proven and there are currently 20 teams actively working on CRO.
In the past year, the focus of the cooperation between VodafoneZiggo and Online Dialogue was mainly on training and educating (new) employees. Young, inexperienced analysts were recruited who needed to be trained in the VodafoneZiggo-CRO way of working. The goal was to further embed this way of working throughout the organization, focusing on the specific Online Dialogue vision of CRO: using knowledge in the field of psychology and web analytics in multidisciplinary teams.
To get the teams ‘on board,’ Online Dialogue organized inspiration sessions that answered questions such as: why is CRO important? Why do we want to experiment? And what is the point of experimenting?
There was also a focus on building a good hypothesis and people could come up with their own test ideas to increase engagement. In addition, there was a training for CRO specialists to be able to build simple tests themselves. In it they learned, for example, the basics of CSS and HTML. The Online Dialogue's Behavioral Online Optimization Method (OD's BOOM!) has been used to teach people the importance of a structured way of working and applying sound psychological knowledge within CRO.
Within the current structure of VodafoneZiggo, Annemarie and Farshad are in a Center of Excellence placed, in which they play the role of a sort of internal consultant. In that role, they help the CRO specialists who have questions. These are mostly young people with relatively little experience, whom they can help well with advice on analyses, processes and structures.
Asking about whether VodafoneZiggo has managed to stand on its own two feet now, Farshad answers confidently: “We definitely succeeded! If you look at the last four years, we have done a lot of knowledge transfer. CRO is really embedded in the organization.” Annemarie adds: “Actually, we have now become a bit of the Online Dialogue within our own organization!”.
The past few years have been about mastering a good process. Annemarie: “We now know how it should be according to theory, now the challenge becomes to keep that up.”. The CRO specialists must strike a good balance in performing the analyses, while also keeping an eye on the process and keeping others on their team hooked. “CRO should not be something we do on the side, but everything we deliver should be validated.”.
When asked what they would advise other organizations that also want to establish a CRO program, Farshad replied: “Especially the approach of working right from the beginning with an agency that was experienced helped us a lot. That provided a kind of quality assurance that you miss when you start on your own.”.
He also found it very interesting to get a handle on psychology, but even more important was the value of a good process. The importance of a good hypothesis, doing research, the relevance of correct data and connecting all those facets to the process of conversion optimization. “In doing so, working with an agency is very valuable.”, finds Farshad. “Working with a desk provides a kick-start, says Annemarie. “It prevents mistakes you might otherwise have made. If you want to make quick steps, it's helpful to have an agency to help with that. You don't have to come up with everything yourself, but you can work with a validated process.”
Looking back, Farshad thinks that when partnering with an agency, you should especially consider the development path your company is in. “It's very important to determine how to properly land a new process. It really is change management.”.
“It's great when an agency comes up with a wonderful process, but you have to take into account the stage the organization is in and adjust your pace accordingly.” adds Annemarie.
What Farshad especially wants to mention is that he is immensely proud of the team that is there now. “Our CRO graduates are all tops! It is really super amazing how quickly they have mastered the CRO profession. And the fact that this has resulted in 4 nominations at the Dutch CRO Awards - with 1 winner and winning an Experimentation Culture Awards for our Center of Excellence, is surely the best proof that we are doing very well!”
Farshad concludes by complimenting the entire Online Dialogue team. “I have always found the cooperation to be very pleasant. It feels like the people at Online Dialogue are also colleagues that you enjoy working with and click with.”
Of course Online Dialogue is sorry to see the partnership with VodafoneZiggo come to an end, but what prevails is a feeling of pride. Proud of what we have achieved together over the past four years.
Nederlands Nederlands
English