Key lessons from the expert meet-up ‘Privacy: Threat or Friend?’

Developments to protect consumer privacy are rapidly following one another. Social awareness about privacy and the value of data is increasing. Technical measures (such as Intelligent Tracking Prevention) and legal measures (such as GDPR) make it harder to track Internet users. All indications are that this social movement will only continue.

Is this an existential threat to the world of online experimentation? Or should we instead welcome these developments? 

These questions are of great interest to us and our clients. So we organized a private meet-up with experts on privacy and online experimentation, from both the public and private sectors. To ensure that everyone could speak completely freely, we did not write down who said what. However, we would like to share the most interesting lessons learned from this important discussion with you in this blog.

What are legal and technical developments in privacy going to mean for online experimentation and CRO?

The experts present distinguished between personal data and aggregated (or aggregated) data. The privacy discussion is mainly about protecting personal data. New legislation is also aimed against the unnecessary storage and use of citizens' personal data. 

Lesson 1:

Privacy developments will cause us to work more with aggregated data. You have enough segmented- and aggregated- data to do a CRO experiment. 

One of the experts explains that you only need to know what condition people are in (variant A or B seen) and whether these people have converted. Personal information is not needed and therefore does not need to be collected. 

According to another expert, we're currently in an interim period with some ambiguities in terms of laws and regulations about what information you're allowed to store and what you're not. His estimate is that we will be through this within two years. He expects that new technical solutions will then make it possible to continue experimenting in a way that meets the standards set for user privacy. 

How do people actually feel about participating in online experiments? 

Previously I wrote about the A/B effect: people have an intuitive resistance to experimentation. Even in the most prosocial environments, for example in a hospital where a choice must be made between drug A or drug B, people generally prefer ‘the boss’ (the doctor in this case) to decide, rather than a controlled A/B test.

Therefore, should there be no more experiments? 

Lesson 2:

Here one of the experts disagrees: “No more experimentation means we can never change anything about the Internet and it will forever remain as it is now.” Here's the thing: If a company makes a change to their website and they find that it causes their sales to drop, they will reverse the change. Viewed this way, any change to a Web site can be seen as an (uncontrolled) experiment. 

We also discussed the ethical side: are experiments by themselves good or bad? Participants agreed that in the end, what matters is what you do with it and which metric you optimize for. “What we really need to talk about is companies that manipulate (no matter how) and how to prevent it.” 

This leaves the question: what (and who) then determines whether an experiment is good or bad? According to one participant, it is abundantly clear: “there will always have to be a balance between consumer protection and a healthy degree of business.” How do you achieve that balance? This question is big enough to fill years of meet-ups but two points immediately come to mind:

• An experiment should not be manipulative. 
• You have to be transparent about what you do.

Should companies be fully transparent about all experiments they conduct? 

This question also generates an interesting discussion. Participants agree that it is not desirable to bother people with information about every online experiment you do.

Lesson 3:

Complete transparency is not the solution because people simply do not have the time and energy to give their consent to every experiment. Who is going to read all those pages of conditions?

Giving permission for everything takes (too) much time and energy, but the freedom to make your own decisions must be guarded. So how do you protect people's autonomy? One solution is to make it easier for everyone to exercise their right to ‘opt out’: to be able to indicate that you do not want data collected about you and that you will not participate in online experiments.  

This topic raises questions among experts: If you don't want data collected about you, can you still expect websites and products to work well? Or is it like going to the doctor and asking for a diagnosis without giving him or her permission to do research? And suppose as a company you honestly and transparently communicate that you can't help your users unless they share their data with you, is that honest and transparent, or aggressive manipulation to get them to share their data? Or both?

What can you do to make sure you are using online experiments for good?

In closing, one participant points out that companies who want to be sure they are using their online experiments for good can consult the (Nobel Prize-winning) persuasion expert Richard Thaler. According to Thaler, three principles guide the creation of ‘nudges for good:’

• All nudging should be transparent and never misleading.
• It should be as easy as possible to opt out of the nudge, preferably with as little as one mouse click.
• There should be good reason to believe that the behavior being encouraged will improve the welfare of those being nudged.

Would you like to participate in one of our meetups next time? Leave your details using the form below and we'll keep you posted about it.

Do you have your own contribution or perspective to add to this important discussion? Let me know.

Stay up to date on upcoming meetups? Leave your information below.