When starting a Web site optimization process, people like to know when the process is complete. After all, with most products or services, you agree to deliver a product or service after a certain period of time. Therefore, a frequently asked question is: When did you finish optimizing? 

The answer to that is very simple: never! In this article, I explain why, as well as how important context is on behavior and the sustainability of your A/B test insights.

Influence of context on behavior

Behavior occurs because you are motivated to do something. For example: you eat because you are hungry. This is the simplest form to explain how behavior comes about. But, motivation is not the only thing that determines behavior. In fact, context greatly influences behavior. You know the drill: you're at a party with a drink in your hand at a bar table with snacks. Chances are you eat from the appetizers without feeling hungry. The context, in this case the table with snacks, makes you eat (or, in other words; exhibit behavior).

But what do these snacks have to do with optimization? All sorts of things! But before I talk more about that, I want to go back to the basics; why do we actually optimize?

We optimize websites to ensure that visitors can achieve their goals. We make it easier to buy a bed, book a hotel room or take out a mobile subscription the moment visitors come to the website with the goal of doing so. By A/B testing, we find out what makes visitors come closer to their goal. We establish behavioral models, which we test and adjust with A/B testing. Based on this, we get better and better insight into how behavior can be influenced on this specific (place on the) website. And that is a very important point; how behavior on the website can be influenced.

How do you respond to context?

What we don't include in behavioral models is context. So anything that happens outside the website, for example, the weather. But also whether there is an economic crisis, whether there are political tensions. Because these things also affect behavior. Think about how much money people are willing to spend, how much risk they take and to what extent they are able to make a conscious choice. But also consider smaller things like a person's personal context. Is someone viewing your website at night with full attention, or is they cooking dinner in the meantime, or listening to a meeting with half an ear? This all plays into it. 

See, and that's where the bite comes back. The fact that there is context. Or in other words; the snacks that are on the table, means that behavior is influenced. The snacks cause you to eat at the party. And, the fact that there is an economic crisis causes people to spend less money. Similarly, if someone is sitting on the couch with kids running around, it causes someone to pay less attention to the website.

There is always context

You have no control over what goes on outside your website. But on your website, you do! What we're trying to do is set up the Web site to be easy and accessible enough that someone in all of these settings can use it well. 

An economic crisis may cause people to spend less money, but it does not mean that people need a new bed less. People will still want to make this purchase. By A/B testing, you find out what influence you can have on your visitor's behavior, despite the context. This knowledge is extremely valuable and most likely the basis of your optimization process. After all, there is always context.

Best before...

Unfortunately, this knowledge has an expiration date. This is because the context is changeable. And, so does your visitor's behavior. The weather changes, a fad disappears, economic crisis turns into economic boom and political tensions disappear. The longer you are experimenting, the more likely the context will change. The tricky thing is that these kinds of changes often happen very gradually. A/B testing makes spotting such a change easier. You notice that behavior changes and your A/B testing turns out differently than expected. 

So you see that during an economic crisis, less money is spent. The moment we recover from the economic crisis, you will see that your visitors will start to show different behavior. And that your website, optimized for the economic crisis, no longer performs as before. Fortunately, by A/B testing you will find this out sooner, allowing you to intervene in time. You can learn what your visitor does need (to buy that bed, book a hotel room or take out a mobile subscription). 

Never done with A/B testing

In short, behavior is subject to all kinds of influences on a website and outside. On a Web site we can control them, but outside we cannot. Because context is changeable, so is behavior. Continuous A/B testing ensures that you are always aware of change. So you are never done with A/B testing!

Tips to extend the shelf life of your knowledge:

1. Be aware that changing context causes behavior to change. In a dashboard, keep track of the baseline. If numbers start getting out of sync, it may be an indication that context is changing. Then it's time to try a different direction with your A/B testing.
2. React to a changing event! Are you selling airline tickets and people suddenly stop flying? Do a data analysis to see what happens on your website. Last year we performed corona analyses at our clients to see what effect this drastic event had on visitor behavior.
3. Do regular retesting or back-testing to see if the effect still exists. For example, we experienced that the biggest winner ever became the biggest loser ever five years later. This was a very clear indication to change course.
4. In short: always stay A/B testing 🙂