Beerwulf has been experiencing explosive growth in recent years and came to us in the summer of 2019 asking if we could help map the behavior of their visitors. They wanted to understand their visitors better, but did not have a psychologist on staff themselves. And besides, partly due to their explosive growth, they could use a fresh pair of eyes. We then created a behavioral analysis created for them. To give you an inside look at how Beerwulf works and show you our process, we asked Lars Harmsen (head of user experience at Beerwulf) some questions.

CRO at Beerwulf

You've been with Beerwulf for about 3 years now. What development has Beerwulf gone through in terms of CRO during that time?

Lars: “When I started at Beerwulf in September 2017, the team was small. We were still about 12-13 beer lovers at the time, yet there was already a spot for a CRO specialist. So the will to experiment and make data-driven decisions was there early on. My job was to set up a CRO program and continuously represent the voice of the customer within Beerwulf. Over the years, we continued to grow as an organization: in size and revenue, but also in terms of professionalizing our way of working. The organization became more complex, increasing the importance of making data-driven decisions. We expanded the CRO capacity little by little, until we ‘officially’ created a UX team at the end of last year. Meanwhile, the UX team has become indispensable at Beerwulf and we support the teams with continuous research, experiments and insights.”

What do you want to accomplish with your team in the coming years?

Lars: “Our ultimate goal as a team, of course, is to make all of our visitors happy. Regardless of the reason you visit our website: we want to make it as fun and easy as possible for you to reach your goal. To achieve this goal, it is important, as a team and as Beerwulf, to understand the different user journeys and always put our visitors at the center of everything we do. Our internal goal as a UX team is therefore also still the same as in 2017: to continuously represent the voice of the customer within Beerwulf. And in addition, making sure we use data to make the right decisions for our visitors.”

What does the rest of the organization know about CRO? And how do you involve them?

Lars: “Over the past few years, we have shared hundreds of analyses, insights, usability tests and experiment outcomes within Beerwulf. Therefore, it is almost impossible not to be aware of CRO within Beerwulf. We also provide training, give presentations and have a competition where colleagues can gamble on A/B test outcomes (with candy bars & trophies for the winners).”

What misconceptions do you often have to clear up with colleagues when it comes to CRO?

Lars: “In fact, the most common misunderstandings always have to do with the reliability and/or validity of data. When is an observation an insight? What sample size do I need to draw a conclusion? Why can't we test everything? And what role does confirmation bias play?”

Behavior Analysis by Online Dialogue

What was the reason for having a behavioral analysis developed by Online Dialogue?

Lars: “On the one hand, we were eager to explain the behavior of our visitors with the help of behavioral psychologists. We want to understand our visitors better, but don't employ a psychologist ourselves. The expert perspective of an outside behavioral psychologist helped us recognize patterns and link them to existing and proven theories.

On the other hand, we felt that, thanks in part to our explosive growth, we were losing track. After two years, you get a kind of banner-blindness for your own Web site. Things that surprise others make sense to you, and vice versa. Working with Online Dialogue allowed us to take a fresh look at our website again and create a good situational picture.”

How was the collaboration with Online Dialogue?

Lars: “The collaboration with Online Dialogue went really well! It really felt like we had some new colleagues there. In the initial phase we were completely vetted, just as we had hoped. Online Dialogue asked us all about our growth, our customers, our data and our goals. All with the aim of painting the best possible picture of our situation. We constantly discussed our priorities with each other to achieve the best possible end result. And oh yeah, of course we also had the occasional beer together.... 😉 ”

How did you start working with the behavioral analysis internally?

Lars: “We used the outcomes of the behavioral analysis to restructure our experimentation roadmap. We now think from behavior and motivation, linking insights and experiments to overarching psychological theories. In addition, there were learnings that we could take throughout the organization. For example, we now know that the biggest differences in our visitors” behavior are not visible at the country level, but at the product level."

What were the successes and challenges of the project?

Lars: “The biggest success is that we have made a lasting change in the mindset of our team. We still put our visitors first, but now focus even more on the behavior those visitors exhibit and how we can recognize behavior in our various data tools. In addition, everyone within Beerwulf now talks almost daily about reducing cognitive load, increasing fluency and using nudges.”

What would you advise other organizations that also want to do a behavioral analysis?

Lars: “Make an overview of everything you know about your customers and use as much as possible to do so proprietary data sources from your organization. Can you still see the forest for the trees? Then hire a behavioral psychologist and try to make the link between the data you have and behavioral science, in order to better understand the behavior of your visitors. If, like us, you can't see the forest for the trees when looking at all the collected data and/or don't have a psychologist as a colleague, it's high time to get around the table with Online Dialogue!”

You won a CRO award last year. What do peers have to do to win one too 🙂 .

Lars: “Get everyone in the organization involved in CRO and A/B testing! The chance of a winning experiment with impact on your entire organization becomes much higher when you use data from different angles. By adding value not only for the customer, but also for other teams within your organization (in our case, customer service & logistics), you ensure that there is more and more support for CRO and you can really influence the mindset of your company. And in the end, that might even earn you a CRO award! 😉 “