We are looking for a data analyst! Check the job posting.

The impact of Intelligent Tracking Prevention (ITP) on A/B testing

What do the changes in ITP mean for optimization?

In recent years, a number of changes have been made to the Safari browser to protect the privacy of Web site visitors. These changes make it increasingly difficult for users to be tracked when they visit multiple websites. This so-called cross domain tracking makes it possible, for example, for parties such as Facebook to track the steps you take online in order to then show you ‘relevant’ ads based on your online behavior.

Changes to first- and third-party cookies

In June 2017, the first version of ITP (Intelligent Tracking Prevention) in web browser Safari went live. In this version, 1.0, the main focus is on completely blocking third-party cookies. A third-party cookie is a Web cookie, which is placed to identify an Internet user on one or more Web sites. This change has had a lot of impact on the ad market, as it makes it harder to build profiles, do proper conversion tracking and makes retargeting more difficult. 

March 2019

A new change will follow in March 2019: ITP 2.1. This version also affects first-party cookies. Examples include preserving a user's login credentials and preserving certain data such as products in the shopping cart across sessions. First-party cookies can be set both from the server and from the client (through javascript); ITP 2.1 focuses specifically on the use of client-side cookies. A/B testing and analytics tools use these client-side first-party cookies because they ensure that a user is recognized across sessions and that a user continues to see the same variant of an A/B test over and over again. 

June 2019

In ITP 2.1, the validity of these first-party cookies is limited to seven days. If a visitor comes back to the website after seven days, this visitor is considered new. This creates a new distribution in your A/B test, so you can't be sure if the user has seen the same variation each time. This also affects your Analytics statistics. The user is thus seen as new after seven days, significantly increasing the number of unique users. 

ITP 2.2, the version that went live in June 2019, further restricts the use of first-party cookies. When a user comes from a website characterized by Safari as a cross-site tracking domain (such as Facebook, Google, etc.) and the url contains a query parameter or hash fragment, the cookie duration is shortened to one day instead of seven days. 

What will the future bring?

We are already facing version 2.1 and 2.2 after ITP version 2.3, and it will not stop there. So it's high time to take a closer look at the implications of ITP for A/B testing and possible ways to deal with these implications.

In addition, other browsers are also placing restrictions on first- and third-party cookies. Clearcode has written an overview article on the handling of cookies by different browsers.

Impact of Intelligent Tracking Prevention on analytics data 

At Online Dialogue, we work for different clients in different markets and thus have a good understanding of the impact of ITP on analytics data. 

We looked at twelve Google Analytics accounts to see the effect on the number of new users within Safari, compared to the other browsers. Here we expected a significant increase after the introduction of both version 2.1 and 2.2. In fact, users are tagged as new visitors after seven or one day. 

The graph below shows that this is indeed the case. The percentage of new users has increased significantly since ITP 2.1 and 2.2 went live. We see this increase especially in companies where the number of returning users is already large. In the rest of the browsers, we do not see this increase, but instead see a slight decrease.

Intelligent Tracking Prevention ITP

What does this increase mean? 

This increase shows that ITP has a big impact on the quality of your data. After all, users who were previously still recognized across sessions are now seen as unique users. In addition, this user is redistributed within an A/B test if he returns to the website later than seven days or even one day. This pollutes your A/B test data making it more difficult to find an effect. Your conversion attribution model will also suffer from this. After all, it becomes more difficult to link the user to the original source if the time between visits is longer than one or seven days. 

Organizations using analytics and A/B test results to gain insight into their customer's behavior will need to look for solutions because of the changes in ITP. Do you have questions about this or need help? Then please contact with us. We are happy to think with you!