“You're the specialist, right?” “Can't you just tell me right now what 10 things I need to change on the site to get more transactions?” Unfortunately, we still get this question on a regular basis. We see the gap between CRO as a solution and CRO as a method widening all the time and expect it to continue to grow by 2024.

CRO as a solution
“I have fewer transactions and CRO promises me guaranteed 15% uplift if I implement these 10 changes.”

CRO as a method
“I know why I have fewer transactions and am constantly optimizing with various teams to solve these issues.”

In previous years, there was a larger middle ground in companies working from solution to method. With the sunset of Optimize, the transition to GA4 and the use of ChatGPT, companies were forced to take a critical look at their CRO program. As a result, fortunately, many organizations have grown incredibly in their CRO maturity. Unfortunately, we still see that some companies have chosen the easy way out.

Want to know what you need to focus on in 2024 to end up on the right side of the CRO maturity gap? We explain it using these 4 disciplines within CRO.

• Strategy
• Psychology
• Data
• UX

Strategy

The sunset of Google Optimize has caused quite a stir over the past year. Companies were forced to think about the position of the experimentation program within the organization. Suddenly questions such as, which teams (and matching skillset) should use the new tool? How do we want CRO in the organization in 2 to 5 years? What does the tech stack look like that it needs to interface with? What is the ROI of the CRO program now and how does it compare to the cost of the new tooling?

Companies took this as an opportunity to take a hard look at their existing CRO strategy and plot how mature their organization actually is in terms of data-driven work. Take a look at the maturity model below. Can you plot your own organization on the 5 components?

Fortunately, there are a lot of companies that have taken this situation seriously and are choosing the right long-term path. We will give you some tips to continue this path in 2024 and grow your organization in maturity.

• Company Culture:
  Invest in developing a culture of failure. Consider a monthly fuck-up-friday where management also shows that it makes mistakes. Work actively to share knowledge between teams and tailor content to the receiving audience. Culture is something that quickly gets snowed under in the daily experimentation business. It helps tremendously to set aside dedicated time for this.
• Strategy & Scope
  Examine how the KPIs and metrics of different departments relate to each other. Together with branding, marketing, IT (and other relevant departments), lay out the KPI framework and work together on the north star metric (NSM). You can think of the NSM as the most important overarching metric for your organization. This metric is based on the value you add for your customers. For example, number of minutes of engagement, monthly active users or growth in revenue. Make sure the team is measured on both commercial and non-commercial goals. Ultimately, it's about how the customer is actually best served.
• Team & Expertise
  Actively invest in your employees' soft skills. Content training is always good, of course, but what's really going to make a difference are your colleagues' consulting skills. Think presentation training, NLP courses, sales training, feedback calls, etc. Read more about the skillset of the modern CRO specialist in this blog.
• Data & Tech stack
  With changes in the data landscape, GDPR and AI automations, the skills of an analyst are changing tremendously. The role of Web analyst is changing to one of data scientist. More on this under the Data heading.
• Process & Methodology
  More and more companies are getting into product discovery. Keep it up! Just make sure that product discovery and product delivery stay connected and the feedback loop is continuous. You don't want delivery and discovery to both take on separate lives.

For those companies that have chosen the easy way out and still have lists of
“15 winning tests that always work” want to see...success in 2024 😉 CRO has become so mature by now that you're not going to make a difference with this thought. You can either bury your head in the sand, or get started with CRO in an approachable but data-driven way. All you have to do is look at the data from your own visitors and don't let promising quick wins fool you. In doing so, you will learn nothing about the behavior of your own target audience and you will continue to shoot with hail, hoping that it will hit.

... Just a quick note: we send out a newsletter every three weeks that includes the latest blogs, team updates and, of course, news about the offerings in our academy. Click here to subscribe.

Newsletter sign up

Psychology

Psychologists are incredibly valuable in the CRO industry. They are indispensable if you want to increase the quality of your testing program (and thus the ROI). That's not surprising: after all, you're trying to change people's behavior. You use experiments to learn what works and what doesn't work to solve your customers‘ problems. This is a wide-ranging process that delivers a lot of value, but it can't just be achieved with shortcuts and ’psychology hacks. You really have to be willing to think deeply about what you are really helping your target audience with instead of haphazardly implementing solutions.

Unfortunately, current trends make this increase in the quality of your CRO program incredibly difficult. In fact, most automations operate output-driven rather than problem-solving. Take a look at the example below:

Visitors taking the elevator in an apartment building find the elevator incredibly slow. 

What would you do to solve this problem? With today's product delivery mindset, we are quick to say, “We need to make the elevator faster!” A logical, but also potentially costly solution.

What would a behavioral expert do? Psychology says: what if you distract people, making them stop thinking about the speed of the elevator? You can do this by hanging a mirror. Why? People like to be with themselves, so if you hang a mirror in the elevator, the time in the elevator feels a lot less long to people and you don't have to invest in a new elevator.

ChatGPT

This digital mastermind is fantastic fun to play with. It helps you in writing texts, summarizing articles, doing keyword research, writing and checking code. It brings you a lot of shortcuts. But we shouldn't expect AI to give us the solution ready-made if we don't keep giving the right input ourselves.

Indeed, ChatGPT is a language-based AI model that uses deep learning algorithms and natural language processing (NLP) to understand your queries and generate a matching response. The deep learning and intelligence aspect of the tool actually mimics what is already happening in our brains, but a lot faster. It is not a database full of information, but it reproduces learned language patterns. The tool doesn't think, “What do I know about confirmation bias?” The tool figures out “What do expressions about confirmation bias normally look like?”. The tool itself does not know what is right or wrong and is not trained after 2021. You can imagine that if you yourself have been receiving misinformation for years, you then logically start drawing wrong conclusions yourself. Garbage in, garbage out, both for your own brain and for AI models.

Therefore, use it as a source of inspiration and feedback, but not as a source of truthful information. Cross-reference the tool's answers with other sources. The danger is also that the tool gives you the answer in such persuasive language that you also easily believe it is reliable. That's why they've now added the caption: ChatGPT can make mistakes. verify important information.

Personalization

Most personalization processes revolve around sending the right message at the right time to the right audience. It often ignores the actual problem you are trying to solve. Why do people drop out? What have we learned from these large-scale multivariate tests? Do we really understand the target group better or are we just trying to alternate 15 messages and let data determine “what works better”?

As humans, we are incredibly inclined to think in terms of solutions. But we need to keep the focus on the problem statement even in personalization processes.

Consider Einstein's quote:

If I had an hour to solve a problem and my life depended on it,

I would use the first 55 minutes determining the proper questions to ask.

For example, by combining personalization as product delivery pathways with product discovery, you keep the right focus on the problem.

Lists of best practices

Stakeholders want to see results quickly. Competition needs to be tracked or companies themselves lack the proper internal expertise to properly research online behavior. A list of psychological tactics then seems like the solution. After all, don't these always work? Or even better, “Can't you just tell me which changes are going to work anyway?” If we knew everything in advance...

In doing so, you ignore the complexity of human behavior, the nuances of individual businesses and the dynamic nature of the digital landscape.

Instead of best practices, take a holistic approach. Make use of psychological principles and theories that allow you to gain deeper insight into user behavior by taking into account various cognitive, emotional and behavioral factors that influence the target audience's decision-making.

Data

On the data front, we also see that the landscape has become much more complex. The transition from Universal Analytics to GA4 has had a lot of impact, especially on the skills that analysts need to have. This change in skills comes primarily from these 2 areas:

BigQuery and the advent of GA4

Universal Analytics (UA) was accessible and understandable to everyone. For example, a marketer or communications specialist could easily create their own Year-on-Year comparisons with custom metrics.

If you compare UA and GA4, you can see that GA4 has a very different approach. GA4 reports are mainly for quick data checks. Once you want to create custom analyses, you will have to use the Explore function in GA4. The next step is the raw dataset and then you will end up with BigQuery.

BigQuery is the data warehouse of the Google Cloud Platform where you can work with large datasets containing the raw data. It uses Structured Query Language (SQL) and the name says it all, as a user you will have to write your own SQL queries for proper data selection.

BigQuery doesn't just affect an analyst's required skills. Because of BigQuery's pricing model, you have to pay a small fee for each query, and this can quickly add up when using a large data set. In addition, writing SQL is a time-consuming activity, and partly because of this, as an analyst you have to think much more critically about which analyses you want to make and why. This requires you as an analyst to think strategically and advisoryly and to be more critical of the requests coming in from product owners or conversion managers. In 2024, invest in the development of these soft skills as well as the hard skills.

AI

The shift from an executive to a consulting role for data analysts is all the more underscored by the advent of AI. AI tools are certainly going to support the analyst in terms of analysis, processes and insights. Consider ChatGPT's supporting role in writing SQL queries to name but one example.

Although AI is capable of performing routine tasks, human interpretation and context remain critical. Computers cannot understand emotional nuances and fall short in complex conversations. In addition, it remains necessary to keep a close eye on the output of a Large Language Model (LLM) such as ChatGTP. Several studies have now shown that the accuracy of results has dropped significantly in recent months.

In addition, the role of a data analyst encompasses more than just analysis. Analysts also help implement and develop data skills in others. As such, asking the right questions and making decisions based on data will continue to be a task for skilled analysts.

UX

As the landscape becomes more complex and companies opt more for CRO as a method, we also see it happening at the UX level. Like analysts, the role of UX'ers is getting a lot broader. UX'ers are making more use of efficient AI tooling, focusing more and more on UX research and advising on big topics like digital accessibility.

AI

You can't open the Internet or see an article about the cleverness of AI. In terms of UX, we see more and more useful applications in this in tools like Canva and Photoshop for creating and editing photos. We also see many AI applications in the area of personalization. These applications make the life of the UX'er easier, but as with all other disciplines within CRO, AI will not replace UX'ers. In fact, they will focus on broader and more strategic topics in the coming year.

Continuous user research

Product discovery is growing significantly in the marketplace. Companies that invest in product discovery will more often conduct qualitative UX research and make it part of their sprints. The proportion of qualitative research, and thus the role of UX, will become much more leading in ongoing CRO optimizations.

In particular, doing continuous ongoing qualitative research will grow. Continuous user research also called. The role of UX designer will increasingly grow into UX researcher thanks to this (pre)validation phase.

UX researchers lend themselves ideally to the purposes below:

Establishing good data-driven hypotheses:
In the trend article from last year we already described how difficult it is to extract appropriate learnings from experiments. This can often be traced to poorly written hypotheses that are not based on data.

UX research lends itself very well to identifying customer problems, motivations and behaviors. You can do this using such tools as usability tests, user interviews and focus groups. The results of these surveys can then be used well as input for hypotheses for, for example A/B testing. If you incorporate this rationale into your hypotheses, after your experiment you can more easily incorporate the results and learnings of why the outcome is the way it is.

• UX research as a validation method:
  When you think of user research, you often think of user interviews. But there are so many other types of UX research methods! Try underpinning your research question with 5 second tests, first click tests and prototype tests, for example. By supplementing all these insights with data and A/B testing, you'll arrive at a super thorough optimization strategy.

Digital accessibility

The last topic we don't want to leave unsaid is going to play an incredibly big role in the field of UX next year. This is because in June 2025, the European Accessibility Act (EAA) will come into effect. Not only do you have a moral obligation to ensure that everyone can use your websites and apps, you will also miss out on conversions if your digital domain is not accessible to everyone. In the Netherlands, there are approximately 4 million people with disabilities. 

To ensure that your digital domain complies with the guidelines established in this law, you as a company will have to take steps in this area in the coming year. And this is not going to be easy. To help you get started, here's a accessibility checklist which you can set as a desktop background, for example.

Source: https://www.onlinedialogue.nl/artikelen/toegankelijkheid-checklist/

In conclusion

2023 has been something of a CRO reset, so to speak. The CRO landscape changed a lot with Optimize's sunset, the transition to GA4 and ChatGPT's shortcuts. Companies were forced to either opt for shortcuts and quick wins (CRO as solution), or take their CRO programs seriously and choose the right long-term strategy (CRO as method). In 2024, the gap between CRO as a solution and CRO as a method is widening. Therefore, invest in quality and in training CRO specialists so that they can handle their broader tasks.

We hope that with this blog we have completely updated you on the trends and developments we foresee in the coming year. And we look forward to hearing your additions! How will you get started with CRO in 2024? Let us know in the comments.

You might also find this interesting: