You know the drill, you're at a birthday party and someone asks what exactly do you do? You answer the question with ‘CRO specialist’ (oid), then you used to get “what?”. But it's been noticing more and more lately that more and more people outside the field, have knowledge (or at least know of its existence) of CRO and experimentation. The field is growing, and so is the maturity of organizations.

Whereas a few years ago the question focused more on how to set up a good experimentation process, now it's about the maturity of the company. To what extent does the company embrace a data-driven way of working? Is there a culture of experimentation or failure? Are hypotheses and learnings shared appropriately?

In this article, we take a closer look at the trends of 4 disciplines within the CRO world:

• Strategy
• Psychology
• Data
• UX

CRO Trends strategy

CRO for Product Discovery

We are increasingly seeing a shift toward Product Discovery. This means that market demand is less focused on helping us run an A/B test program with the goal of optimizing what has already been built. Market demand is focusing more and more on pre-validation: how can we, before we build, gather data and validate whether something works or not? We call this Product Discovery. This shift in demand shows that organizations are becoming more mature and that CRO is really becoming part of the various business processes.

© Dottie Schrok,Productboard, 2022

Embedding experimentation into process product teams

The aforementioned trend ‘CRO for Product Discovery’ can also be viewed more broadly. Indeed, what we are increasingly seeing is that experimentation is becoming part of the entire process of product teams. In both the Discovery and Delivery phases, there is plenty of experimentation and research being done. This close collaboration ensures that larger tests can be done and winners implemented quickly. Many organizations are not here yet, but we do expect increasing demand for this in 2023. Wondering how best to approach this? Then read this article.

More organizations are working toward a Center of Excellence

The growth in maturity is also reflected in the number of companies working toward a Center of Excellence (CoE). To scale up experimentation to an organization-wide method, a CoE must be established. This CoE is responsible for making experimentation accessible and user-friendly throughout the organization. This CoE has its own budget and team that ensures each team can experiment (source).

CRO Trends 2023: psychology

Results are not learnings
Organizations are increasingly A/B testing. Which is a very good thing. The more we can substantiate with data (from gut to substantiated), the fewer assumptions are taken for true. Yet we still don't get enough out of the tests that are done. Unfortunately, we still often see the following conclusions reflected in test reports:
“This experiment has a significant uplift of 2.5% so we learned that the variant works better.”

Error! You still don't know in this way why your variant works better. And how you can extend this to get even more results from your test. Or to test the same principle in other channels.

Many people find it difficult to go beyond just describing outcomes (e.g., Uplift in conversion) in reporting. Therefore, it is helpful to use the following mnemonic: good reporting includes the past, present and future.

With these 3 elements, make sure you test the bigger picture in 2023 instead of separate, haphazard tests.

1 - The Past: Hypothesis & rationale.

When preparing a good report, it is crucial that you have prepared a good hypothesis in advance. Without a hypothesis, it becomes very difficult to draw conclusions afterwards and continue testing accordingly. When drafting your hypothesis, keep in mind that you properly support your hypothesis with different sources of information.

Use the following checklist:

• Does your report describe what you expect?
• Is the metric mentioned on which you will judge your experiment?
• Does it describe what changes were made to the variant? (including images)
• Is the rationale for your expectation described?
• What sources do you use as substantiation?
• How does the current experiment build on previous findings?

2 - The present: results & learnings

This section is about the actual outcomes of your experiment. People tend to forget a losing test as quickly as possible and move on to the next test idea quickly. This is a real shame. It is precisely from losing tests that you can learn a lot, if you know how. Again, it is important to make a feedback to your hypothesis. Highlight here how the results you found match, or differ from, the expectations you had. If your findings don't match what you expected, this is a great opportunity to dive deeper and learn. You can do this by closely examining unexpected outcomes. At what layer did things go wrong? Did it interact at all with the element you had modified? Also look closely at alternative explanations. Suppose you put the element in a different position on the page, would it work better? If the information displayed was not helpful, what information are visitors looking for?

For the present, use the following checklist:

• Is the outcome described?
• How is the outcome related to your hypothesis?
• What are possible explanations for an unexpected outcome?
  • Tip: Take another critical look at the designs
  • Tip: Look at different segments/devices
  • Tip: Check out other metrics
• How can you test alternative explanations?

3 - The future: recommendations and follow-up tests

In this last section, you describe what the next steps are. What do you recommend? Should the variant be implemented or not? And how will you build on the findings? What are your follow-up tests? Can you make the effect of the winner even greater by testing on other pages or channels? How can you turn your winner's learning into a follow-up test to prove otherwise? There is always a follow-up to your reporting.

Handle the following checklist for the future:

• Do you want to implement the variant or not?
• If so, how can you continue testing on the same direction?
• If no, what test ideas follow from your alternative explanations?

By applying past, present and future in your reporting, you're going to get the most out of your learnings. After all, this way you're testing with an overarching strategy and you're no longer testing haphazardly.

CRO Trends 2023: data

In the area of data & analytics, we build on last year's trends & developments. We name 3 of them in this article:

GDPR and the use of Google Analytics

This is an ongoing issue, as many DPAs (Data Protection Authorities) will issue rulings on the use of Google Analytics. The Dutch DPA (Autoriteit Persoonsgegevens) has not yet made an official ruling on this. What may be a plausible option is that the DPA, like the Danish DPA, will not determine whether Google Analytics will be banned, but will only concern itself with testing whether companies comply with the law. Of course, it is not for nothing that Google is pushing harder toward GA4. After all, a number of privacy problems are overcome with this.

In order to be compliant, it is especially important that you provide the user with clear and proper information regarding his/her personal data at the time you collect it. What can you do in the meantime?

- Make sure your cookie banner is on the right consent and information
- Check that your privacy & cookie statement contains the right information
- Be critical of what (personal) data you really need
- Be critical in your tooling. Don't use something anymore? Take it off the website.

Unsure about what applies to your situation as a business? Seek advice from a lawyer. Think of it this way: suppose the DPA comes knocking on your door one day, you can at least show what active steps you have taken. That will help!

From Universal Analytics to GA4

Hopefully it has not escaped anyone's notice that a major change is coming to Google Analytics in 2023. The transition to GA4 should be completed by July 2023. A large portion of users already have GA4 running (and if not: get on it asap!), but the majority of the industry is still working in the GA3 interface. Partly because GA4 has not yet contained all the functionalities that were standard in GA3. Also because the interface really does work in a different way. But Krista Seiden, among others, talks about the many advantages of GA4. In particular, the BigQuery connection will provide analysts with new skills. Expect extra training and getting used to it before it will work as smoothly as GA3.

Server side tagging & testing

More and more companies are moving to server side tagging and testing. The big advantage is the ease of connecting various data streams on the backend and the positive impact on site performance. With A/B testing, it is often attractive to dive right into the content, whereas improving speed can really do wonders for your conversion rate. Sometimes it pays to spend one or more sprints on site performance instead of doing multiple A/B tests! Apart from the A/B testing process, of course, it also does a lot of good for your SEO.

One disadvantage is that, of course, you no longer see what is being measured. This makes QA and analysis more complicated. You have to think longer in advance about which events you all need and whether this is all measured server-side by default.

Server-side tagging is also not the solution to all data privacy issues with Google Analytics. Just because you as a user can't see what is being measured doesn't mean you can suddenly measure everything without permission. Server-side tags must also comply with privacy requirements!

CRO Trends 2023: UX

UX research post covid

One trend in UX research is remote research. For a very long time, Covid prevented us from conducting physical usabiltiy tests in our UX lab. However, we continued to conduct research, but remotely. Tools like Lookback allowed us to continue to learn about our users, but remotely. The big advantage of this turned out to be that respondents then participated in the research in their own familiar environment, which in turn provided very good insights. Every disadvantage has its advantage. We expect remote usability research to continue to be used a lot post covid.

Cross-device UX

Cross-device UX is about visitors using two or more different devices in their customer journey. We are constantly switching devices. The most classic example is that visitors orient on mobile and buy on desktop. This black-and-white picture is no longer of our time. Visitors orient on mobile while on the train, look at the product again at home on desktop and put it in their shopping cart, then convert on mobile and consult customer service via their laptop at work.

We expect cross-device UX to become an important skill for designers by 2023. The most common example of this is being able to copy text from your iPhone to your MacBook, but there are many other capabilities that companies are taking smart advantage of. How do we ensure visitors have the same experience across all devices? Can we make sure visitors can easily copy their shopping basket from desktop to mobile and vice versa?

In conclusion
We hope that with this blog we have completely updated you on the trends and developments we foresee in the coming year. And we look forward to your additions! How will you get started with CRO in 2023? Let us know in the comments.

This article was also published as a Trend article at our friends at Frankwatching.