During and after the corona crisis, the online field is undergoing a huge change. For many companies, conversion rates have either dropped dramatically or increased. But what does this mean for CRO? We asked a number of specialists in the field.

In this article, Karl Gilis (founder of the Belgian AGconsult), Lucia van den Brink (CRO specialist at NU.co.uk) and Ton Wesseling (founder of Online Dialogue) include questions such as: what can you test during this period? Is the data still reliable? And what opportunities does it offer?

What can you do test during this period? And are the data and learnings reliable?

Karl: “If you can go through the entire cycle from test set up to winner implementation in two to four days, A/B testing is still somewhat justifiable from a business perspective. Especially as long as we are in a situation where just about everything totally changes every other week. In Belgium, this is particularly the ‘business cycle’ with which the government announces or extends measures. And with which the behavior of many people changes.

Long-term learnings you're rarely going to get out of it. Unless the situation has no impact on your organization (seems strange to me), if you think this is going to be the new normal or if you make your check-out easier, for example, but for that you shouldn't have waited for a crisis.”

Lucia: “NU.nl has always had a fantastic reach, but during corona that reach got even bigger. We had a record 9.017 million unique visitors in March. That shows how important NU.nl is for the provision of information in the Netherlands. New audiences are finding us, also because of innovations in our work; editors translated important corona information into different languages such as Arabic, Polish and Turkish, making us accessible to people who don't speak Dutch. 

It may be that all records are being broken, but I still had to run A/B tests. Like on the sports section. There I saw a drop of 20% in traffic in the first few days, as all events and sports games were canceled. It no longer seemed sensible to test there, because the change in news meant the behavior there was completely different.

To learnings, I look at each test: could the new behavior cause the outcome to be different than ’normal’? If so, as with the tests on the sports section, I turn off the tests. If not, I hope that my method keeps the learnings I find reliable. My method is as follows: I test a hypothesis in multiple ways and often retest six months later, hoping to make the correct findings and also rule out false positives and negatives.”

Ton: “Of course you can conduct experiments during this period! In fact, I would rather say: actually accelerate the gathering of ‘evidence’ during this period (after all, CRO is also more than just testing). The world has changed because of Covid-19 and not with a temporary shock. This is going to take months in terms of adjusted external conditions. You want to examine what matters to your digital visitors and what doesn't. How can you be the partner they need now?

Why would the data and learnings not be reliable now? Then neither would they have been before Covid-19. The data from properly conducted experiments are as reliable as they were before. The main question now is how long they are tenable. What period are we in now (mid-April)? The shock, temporary behavior, the new normal or is there no new normal and we are back to original behavior? That's why you also want to accelerate in gathering ‘evidence’; whoever switches faster wins over the competition.”

Can we compare data from this period in different years? Or is that actually not possible?

Karl: “No, I find that a little strange anyway, because a lot can change in a year. There are too many external factors. And now that's completely off the mark (this is probably a fairly Belgian expression;)). 

In many web shops, we even see that the normal seasonal data is completely shaken up. My basic attitude: look at yesterday's behavior and, based on that, adjust your website today. In a manner of speaking. Acting fast is the message for most organizations.”

Lucia: “This is a unique period of time. Both online and offline. Of course, you can compare the data from this period to last year's period to see the difference, but you have to keep in mind that this is now corona data.”

Ton: “You can always compare. The question is for what purpose you do it. In any case, I think analytics and revenue data comparison is a great method to work with your trends from immediately before the Covid-19 crisis to determine the main (behavioral) effects of the new situation. From revenue impact to behavioral change.”

Do you need to deviate from your standard KPIs? If so, which ones are important in this day and age?

Karl: “If you had mostly online things as KPIs, I think you could keep them in most cases. Especially if it's about sales or leads. If you also have KPIs around support, you may need to thoroughly reevaluate those. Asking people at all costs to handle online inquiries, without human interaction, doesn't seem like a good idea right now. Human warmth and contact are important in this day and age. And will have a nice effect in the long run. ‘Be kind. Be human’ is an important one for us.

The same is true in other areas, by the way. This is the time to start live chat if you don't already have it. So add that as a KPI and see what the impact is on sales and conversion.”

Lucia: “No, we mainly steer towards increasing the number of pages per session. In other words, keeping people longer. That main KPI has remained the same.”

Ton: “Of course, that very much depends on what your KPIs are and what industry your business is in. Can you still practice your business? Partly? Or not at all? A pivot (temporary or otherwise) of your business naturally creates new KPIs. Without a pivot, your current KPIs will be fine - service experiments will still be judged on the service KPIs and sales experiments on the sales KPIs. However, budgets and targets will change significantly and be largely abandoned when demand within a company has completely changed, even without a pivot.”

What is the impact of corona on companies that are not yet doing much with online? Does it also provide opportunities? 

Karl: “This is why we became entrepreneurs, isn't it? That's how I feel about it myself. Trying things, adjusting and seeing what works. And I see that in many small businesses where that ‘self-employment spirit’ is present.

In Belgium, I see two different types of larger companies:

• On the one hand, companies that suddenly can do things that we never got through to them. Like direct delivery without store pickup, free shipping, a decent returns policy, time to man a chat, etc. Too bad it takes a crisis to convince them, though. And I especially hope they get something out of that when the situation normalizes a bit.
• On the other hand, companies that stay on the ropes. And especially keep saying that business is not going. I don't really get that: there are brands among them that are gigantic and are now letting new or small players who do try and dare eat the cheese off their bread. Now is really the time to look at how you can best help your customers.” 

Lucia: “Definitely. The greengrocer on the corner of my street now delivers at home by letting customers order through Facebook, Michelin-starred restaurants deliver food to their homes, and I also see a lot of new courses coming along that you can now take online. Book launches, events and concerts are suddenly being held online as well, sometimes even for free. What strikes me is that most people like a stage better than a webcam;). In any case, you see companies seizing opportunities wherever possible.” 

Ton: “Yes indeed. I myself cannot wait for the end of this crisis. Not only because of health, but also to experience what all has started up. I have seen very nice developments within the companies I was active in and the markets I worked in during the bursting of the Internet bubble in 2000 and the banking crisis in 2008/2009. In addition, in the Netherlands there is now even the luxury of the government continuing to pay staff in part in the event of significant drops in turnover! If that doesn't stimulate innovation... 

Now is the time to invest time in further digitization, automation of work and new services and products. Everything you can recoup later once the market picks up (and government support has already fallen away). But perhaps real disruption will arise only when everyone is just sitting at home without attractive wages. As a company, however, I wouldn't wait for that, because that's starting over. Take your chance now to run faster than normal, even if you just grew in sales during this crisis.”

How does corona ensure that consumer behavior changes? Even after the crisis?

Karl: “Things we see now:

• Big increase in mobile traffic. Also to buy. My guess is that some of that jump will remain after the crisis.
• A large group of people are now being forced to do things online for the first time. You see that a lot now in larger drop-outs (percentage-wise) in a check-out. Or the questions to support. Time to put some effort into making your website really friction-free. And nice and intuitive. A little more human.
• And many people are now buying things online that they have not yet ventured into. A garden set, new furniture, paint, a lawn mower, etc. Most websites that sell such things are incredibly bad. No wonder these things were mostly bought offline. But if you make a difference there now, that remains. 
• Amid all the irrational behavior, we are seeing an increase in more reasoned buying. Comparing more products, being more price-conscious and sustainable. Support that behavior. Whether this continues depends on the economic wounds this crisis leaves behind.

Many orders require a slightly longer than normal wait. And look: we survive that too. So hopefully we can live with that in the future. So that supply can be more coordinated and ecologically responsible. Because that side effect of corona is clean air and less traffic jams. Fewer delivery trucks because your package doesn't have to get to you before 10 a.m. tomorrow may also be in the future.”

Lucia: “Everyone is out of their regular rhythm. As a result, now is a great time to create new habits. Shopping online, working from home in a non-optimal workplace, having food delivered to your home, cutting your own hair and figuring out a workout yourself instead of going to the gym. These are habits that I see developing in myself, and who knows, keep a lasting impact on my behavior. Online, too, now is the opportunity to instill new habits in your visitors.

What about working from home? Will remote jobs soon be widely accepted? Or will the norm soon be 1 day in the office and the rest of the days at home? I am curious and would not mind, because I am so much more productive. As a CRO specialist, I benefit a lot from so-called ‘Deep Work,’ analyzing data and building tests. We'll see!”

Ton: “I am not a consumer behavior expert. As an inhabitant of this planet, I hope that people continue to make more conscious choices (which I think I'm observing now) and that people continue to appreciate what we had before the crisis in terms of freedom of movement when we have it again, rather than only half thinking about it. However, that we have to get five more outbreaks like this before this really happens I also want to believe. After all, the behavioral experts around me expect that once everyone has been vaccinated, people will soon go back to business as usual.”

What is going to permanently change in the digital field within organizations?

Karl: “More openness to working from home. And hopefully also a much deeper understanding of what is customer-centric and that online really is crucial to your business. The companies that have fairly mastered these two things are also the ones that are now thriving the most (unless their business is mandatorily closed or deals with masses of people). Other companies I wish this understanding as well.”

Lucia: “I think many organizations now realize how much can actually be done online. It amazed me how quickly schools were able to rig online curriculums! Because there is now a need to look outside the box, that is happening. Who knows, we may remember this for the future.”

Ton: “On the one hand, it is a final lifeline for the companies that were actually already no longer future-proof to become so in an accelerated manner. And on the other hand, it is an accelerated shake-out. Also of the many startups with the plan to get funding especially as fast as possible. The only thing that I see changing permanently is that digital has become normal and that digital transformation projects are no longer needed. If those are still needed after the crisis, it will be too late.”

Dialogue Thursday: CRO during corona

At Online Dialogue, we continue to continuously experiment and validate with our clients. In these separate times of corona, it is even more important to validate whether our new ideas really add value. 

Want to know how other companies handle this and spar with their CRO managers? Then sign up for our 33rd Dialogue Thursday.