What is a culture of validation?

Validating something means verifying that it is correct or “true.” A validation culture is an environment where changes are implemented only when there is conclusive evidence that they have a positive impact on key outcome values (e.g., KPIs). Thus, in organizations with a validation culture, the impact of a specific change is first examined in order to assess whether it is desirable. This can involve small things (e.g. adjusting a specific button on a specific website page), but also very large things (completely redesigning organization-wide processes; redesigning your entire website).

It may sound logical to find out if something is working well before you change something, but in practice this is far from always the case. Think, for example, of a new manager who wants to do “everything differently” so that processes are suddenly rearranged. Or a new functionality that is implemented without knowing whether customers will use it. In organizations with a validation culture, these things don't happen much, if at all. In these organizations, large and small changes are tried and tested first and only widely rolled out when there is sufficient evidence that it actually works well. There are good reasons to move toward a validation culture.

Why is a culture of validation important?

Human behavior is fickle. Therefore, it can be difficult to properly assess the impact of a change on human behavior. This is probably not what you wanted to hear. It can be difficult to accept that your well-argued plan, which involved blood sweat and tears, may have a negative impact. Moreover, this applies not only to “ordinary people,” but also to professionals. A recent mega-study (Milkman et al., 2021), for example, showed that even experts were unable to estimate in advance what would be a successful method of changing human behavior.

How well can you assess what works?

In these mega studies, 30 scientists had set up dozens of different digital interventions with the goal of getting more people to the gym. These included things like (a) sending a message to remind people to go to the gym, or (b) getting a small amount of money for each gym visit, or (c) a small token reward (saving points), or (d) getting a bonus for gym attendance after a missed workout. Which of these four methods above do you think was most successful in getting people to the gym more often? Read at the very bottom of this blog to see if you got it right. Can you make a better guess than an expert?

Estimating what works is too difficult even for experts

Besides insights into successful interventions, something else interesting came out of the results. Namely, the researchers had asked people in advance what they thought would be the most successful intervention. They had not just asked people, but professionals (consultants specialized in applying behavioral science) and scientists (professors at leading universities). And what turned out? Even these experts were unable to estimate which interventions were the most successful. Or in other words, even people who have made it their profession to change behavior cannot -without trying it out- necessarily estimate the best way to do so.

This shows that it is essential to validate ideas before implementing them. This is especially true for those ideas where you want to change human behavior, such as the behavior of visitors on your websites, of your employees in the office, or of customers or staff in your stores. This is because in the worst case, the impact is negative, and you implement something that causes you to move further away from the goals you want to achieve.

How do you get a culture of validation?

So validating plans before they are implemented is an important means of achieving business goals. What does it take to establish a culture of validation?

Normalize failure

First, it is important to (dare to) failure. If you know that many of the ideas you come up with are not successful then it means that many of your plans will never see the light of day. This can be painful to realize, and this is one reason why people sometimes prefer to conceal mistakes rather than name them (read more in this blog about Failure and Radiance). Therefore, a change in thinking is needed. Failure is an essential part of success. It is necessary in order to grow. You can train this way of thinking by practicing more sharing and discussing errors. This can be done, for example, through a specific time when sharing mistakes is the focus. At our office, we do this through a monthly “fuck-up Friday” where employees can share what went wrong, what they learned from it, and what others might learn from it.

... Just a quick note: we send out a newsletter every three weeks that includes the latest blogs, team updates and, of course, news about the offerings in our academy. Click here to subscribe.

Newsletter sign up

Experiment

Teach yourself to try things out before you decide on them. Want to start using a new salutation form in emails? Then send out different versions of emails with different salutation forms and keep track of the responses to the emails. Want to get employees to make healthier choices in the cafeteria? Then try out different setups of the buffet and track how many healthy products are sold in each setup. Are you responsible for your company's Web site? Start A/B testing changes. Want to make a change in your operations? Announce it as a test. Try different strategies at different branches and track the results. With each change, big and small, ask yourself how to test it. What should this change have an effect on? And be sure to keep track of this so that when it's over, you can indeed determine whether your experiment had the expected effects.

Keep it playful

Humans are curious creatures. They like to discover and try things. Of doing things for fun. Therefore, one of the most powerful ways to get people to move is to ask something of them that is “fun” (Ryan & Deci, 2000). Make sure there are plenty of ways to practice new skills in a fun and approachable way in addition to regular work. For example, consider the “marshmallow challenge” to get people to practice experimenting. In this exercise, people in groups must try to build the highest possible tower out of dry strands of spaghetti, with the only end goal being that a marshmallow can remain on top. Experience shows that groups that experiment a lot (i.e., continuously trying out whether the marsmallow can still stay on top) get the farthest.

Conclusion

In a culture of validation, the positive contribution of a plan is proven before it is implemented. This is important because it is difficult to estimate what kind of change will have a positive effect. Even experts can't do this well. You can achieve a validation culture by normalizing failure, trying out more, and doing it in a fun and approachable way.

Answer the question at the beginning of the article:
the most successful intervention was giving a bonus for first visit after a missed workout

References
Milkman, K.L., Gromet, D., Ho, H. et al. (2021). Megastudies improve the impact of applied behavioral science. Nature, 600, 478-483. https://doi-org.proxy.library.uu.nl/10.1038/s41586-021-04128-4

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78. https://selfdeterminationtheory.org/SDT/documents/2000_RyanDeci_SDT.pdf