During Superweek 2023 there were again dozens of prominent speakers (including Krista Seiden, Simo Ahava, Matt Gershoff, Jim Gordon and of course our own Ton Wesseling) to share their knowledge to the more than 150 attendees. After a 5-day stay in a hotel in the middle of nowhere in Hungary, one thing became clear: web/data analytics is a field that is far from finished. Data Insight Experts George Onderdijk and Reinier Koolmees were there and are happy to share their knowledge gained with you.

Superweek 2023: “In search of Analytics” new Wild West"

The world of analytics has matured, despite all the changes in recent times. Does this make it more boring and less interesting or more fun and easier and can we now look for the new Wild West of Analytics?

Jim Gordon, Search Discovery

Jim Gordon of Search Discovery knew exactly where to put his finger. 15 years ago, the field of analytics was a virgin territory, where everyone just did something hoping to do it right. Today, with the migration to Web-based analytics, we are more structured, the boundaries are clearer, and the playing field is pretty much completely defined. And sometimes that's pretty boring. So the underlying question is: How can we keep it fun for ourselves? We could think about innovating within the existing boundaries, but it would be much more interesting to start again with a blank canvas.

How do we deal with the increasing number of bots in our data?

We might think about switching from the idea of excluding bots from the data to only including actual users in our data. This is because it is becoming increasingly difficult to recognize bots as technology gets smarter and smarter and they can behave more and more like a real user.

David Hermann, Digitec Galaxus AG

To solve this, thought could be given toward a unique identifier for all users indicating that they are actually a user and not a bot. Lukas Oldenburg and David Hermann gave some more hands-on tips for battling bots with the mantra “Analyze. Suffer. Exclude. Prevent.” They addressed that you can proactively and reactively exclude bots from your data and that both are necessary. From the data you already have, you can start to recognize certain traits that bots have in almost 100% of cases.

Think about certain screen resolutions, browsers, as well as the country where the visit comes from. With these properties in hand, you can create a profile that a ‘real user’ should meet. This can then be used with each visit to the Web site to evaluate whether it is a bot. This makes it possible to allow more and more real users and keep more and more bots out of the data.

Google Analytics 4: 5 stages of grievance

Without Google Analytics 4 no Superweek 2023. So it was a theme that kept coming back somewhere in different forms and in different personal talks and presentations. GA4 is new for everyone and therefore you can learn a lot from each other, for example how to set up GA4 or how to use it.

For example, how do you migrate more than 3,000 events to GA4? Alejandro Zielinsky elaborated on this question in his talk. Using “5 stages of grievance,” he discussed the challenges he had with GA4 at HelloFresh, but also the opportunities he saw. Indeed, the transition to GA4 is a perfect time to reorganize and further standardize accounts and events.

Alejandro Zielinsky, HelloFresh

For example, instead of using 1 property with multiple streams, use 1 property with only 1 stream for each country/branded website. In addition, manage a roll-up property where the data from all websites comes together (only for GA360 clients). This format avoids issues with user/session reporting and also ensures that you don't run into GA4's new limits for audiences (max. 100) and goals (max. 30).

Events and their parameters in GA4 have a limited number of characters they can contain. Therefore, information in existing events cannot simply be merged into a new GA4 event. Tip: Do not delete or duplicate old events, but use the information to classify new fields. Use the familiar format for events, turn them into new default fields and name them gaEventCategory, gaEventAction and gaEventLabel. That way you can reuse many events more easily and in a way you are also compatible with GA3.

According to Alejandro, the best way to manage all new GA4 events is to create an online application where product owners and analysts can easily manage GA tracking events. This way you get a better overview of who is responsible for events, as well as more insight into their usage (information can in turn be used as instruction for developers).

Roberto Guiotto and Matteo Zambon from Tag Manager Italia took a completely different tack. Their story about GA4 was not a story about mourning, but a story about love. In “That's GAmore!” recounted their problematic but ultimately satisfying romance with GA4. Love at first sight? No, definitely not.

Roberto Guiotto and Matteo Zambon of Tag Manager Italia

In 2 years, the relationship with GA4 went through several phases: from not understanding each other, wanting to help each other right to the moment they would live happily ever after together 🙂

You can think of GA4 as a great partner, but the road to it is bumpy and full of obstacles that you must first overcome together. Some takeaways: tracking should be thought out and organized beforehand. Use tools to better streamline your own workload. As an example, Roberto showed a homemade tool created to manage Custom Dimensions and metrics in GA4 externally via a Google sheet. Finally the tip that the digital analytics community is an important and endless source of knowledge to exchange information with and help you further on your way with GA4. Make use of it!

>> Want to get started with GA4 but don't know how? Click here for training GA4. In two mornings, we'll get you up-to-speed in GA4. 

Surprising was the discussion afterwards about privacy & first party data. Is the use of GA4 and services like Stape.io even fully GDPR-proof? According to Roberto in Italy, yes. We think not. Using GA4 instead of Universal Analytics does not automatically comply with GDPR legislation. In addition, sharing user information with external services is, in our opinion, also something you have to ask permission for. Roberto's argument here was that you don't ask permission from the hosting provider either.

Superweek 2023: CRO & experimentation

In the area of CRO & experimentation, there were also new developments to hear for the analytics community. Ton Wesseling asked analysts for help on how the analyst can add value. He did the same at Superweek 2 years ago, then with a deep dive on topics such as false positives, false negatives, m-type errors, SRM and business case calculations. His call now was for analysts to better support marketing and product teams in order to improve the velocity and winner's rate of the experimentation program.

Our Ton Wesseling, founder of Online Dialogue

In about 75% of cases, changes are still put live without first validating them using an A/B test. Despite this observation, Ton indicated that change can indeed be seen. In many organizations, marketing teams in particular were experimenting. These organizations are moving more and more toward product validation in the product teams. This may increase test velocity but it does not always improve quality. In fact, it results in fewer true positives.

Ton his statement was very apt: “Validation is not a solution to avoid risk, validation is your invitation to take more risk.” The solution is also not to really just validate everything all the time, but the trick is to use the method to gain insights about the customer journey. An analyst's approach should be to first understand the customer journey, know where can be optimized and then come up with good, promising proposals.

Good tips here: as an analyst, first go buy a product on your own website, as well as the competitor's. Use tools like the Optimizely plugin and Visual Ping to see what competitors are testing and how much of it is actually implemented. In short, get more insight into the product, company, customers and the like. Segment the data and see how certain segments behave, deploying surveys or tools like Usability Hub. In addition, analyze the success of your used testing strategies to come up with new insights for experiments. Which strategy works and which does not? In short, as an analyst, look beyond the analytics data and use the knowledge to help teams grow further on 2 axes: in the number of experiments and in increasing the win rate.

Data use in CRO

Keeping in mind the always important GDPR, privacy and personal data, there was also a lot of talk about how we collect data, why we collect data, but most importantly what we do with this data. Matt Gershoff of Conductrics elaborated on how (as he put it, after being unable to see the forest for the trees) we can separate the trees from the forest in the world of Big Data, Machine Learning (ML) and Artificial Intelligence (AI). His main conclusion: see analytics as the frontline of the business in terms of listening to customers/users and be their advocate to senior executives regarding what needs to change.

Matt Gershoff, Conductrics

... Just a quick note: we send out a newsletter every three weeks that includes the latest blogs, team updates and, of course, news about the offerings in our academy. Click here to subscribe.

Newsletter sign up

In doing so, we as analysts must take clear and deliberate action based on the data. Experimentation helps us do this and ultimately allows us to make explicit and considered choices keeping in mind what is important to the user, but also how they feel while using our websites. In order to do this properly, it can be important to not only look at what we see in the data in terms of how often an event is done, but also to link this to what the users in a test group say in this regard, through targeted surveys within the control and test group of an A/B test.

We returned to the Netherlands with lots of energy and brimming with new analytics ideas and would love to apply them to your organization!