We live in an era where the most valuable resource is no longer oil, but rather data. The main food source for technological development is data, data and more data. The time-honored adage “To measure is to know.”, taken from quotes by a physical scientist in the 19th century, has by now become a cliché.

Even among Web analysts, measuring everything measurable is highly regarded. Google Tag Manager guru Simo Ahava even promotes it in his mantra “Context is King! Measure Everything!”. Yet, in my opinion, measuring more is not always better. Wanting to measure everything brings not only costs but also other disadvantages and pitfalls that are often overlooked in the unbridled measurement drive.

Why do we want to measure everything?

First, we have the need to measure everything because, as outlined above, we live in an age of data. All around us, data collection and processing is being lauded to the skies and if you don't do this, you are missing out on value and will be left behind as an organization. More and more decisions, no matter how small, we want to back up with data. We look up to large parties that 41 shades of blue can be A/B tested.

Second, we are “better be safe than sorry.” Because next year, what if someone wants to know what effect this summer campaign had on the landing page scroll rate and you don't have that information available?

And finally, it gives satisfaction to create something. Something that otherwise would not have existed. Information about click behavior, for example, that would otherwise have disappeared forever. Through your efforts, you are at the cradle of brand new data, and ensure that it is preserved forever. Besides, it's free anyway. So why not use that big event-store and 100 dimensions?

Pitfalls of measuring more & more

1. Cost

Data collection seems free at a time when the actual cost of storage per Gigabyte is a fraction of the cost of yesteryear. But with the exponentially growing number of possible data points, the collection and long-term storage of additional data will always remain a cost, either directly or indirectly.

More importantly; setting up additional measurements takes time, sometimes a lot of time. Hours that a technical Web analyst or developer might have better spent on other tasks. A trade-off must be made. How about using the hours that go into collecting additional data for in-depth behavioral data analysis (admittedly with a slightly smaller data set) or bug fixing? Which yields the most?

With the increasing complexity of data collection and also rapidly iterating development cycles, maintaining good data collection also takes more and more time. How many times has it happened that certain measurements are ‘broken’ by a new release?

2. Privacy

A second issue that is at odds with highly detailed data collection and the analytical capabilities afforded by the volume of data is privacy. Few average visitors will be aware that companies are able to precisely recreate their specific visit, down to every mouse and scroll movement. But consumer awareness and scrutiny of data collection and use has now gained momentum. Ethical questions are emerging; for example, to what extent do you want to hand over sensitive detailed behavioral data of your own potential customers to third-party analytics companies?

3. More data ≠ more insight

Data-driven decisions are generally seen as the better type of decision. It is easier to engage in a discussion with a designer's UX proposal acting on intuition than the same proposal using argumentation from observed data.

Yet there are numerous pitfalls that must be avoided to ensure that insights from data are actually close to the truth. In addition, more data can sometimes increase rather than decrease the risk of these pitfalls.

Too much data creates noise and distracts you from the main goal. As an analyst, you would be stupid not to include all available data in the analysis, wouldn't you? But as you continue to analyze all sorts of different trends and movements, you lose the overview. And in order to come up with a coherent story, your subconscious starts to make all sorts of rationalizations and connections - influenced in part by your own hidden assumptions and judgments. The more data you have at your disposal, the more likely you are to find (“cherry-pick”) data that support your own beliefs.

Not to mention the statistical hazards of many different data points and outcomes with respect to significance.

More data feeds curiosity, and conversely, curiosity feeds the desire for even more data. But although curiosity and insight are an extension of each other, there is a danger that an abundance of data will keep you stuck in the ‘curiosity’ phase. As a result, you lose the overview and time for reflection needed to actually arrive at actionable insights.

Then just don't measure anything?

No, of course not. But do be critical of the data you collect. See if it still fits within the measurement plan and the goals you are driving.

And the next time you want to add additional data to your implementation, take a moment and make a conscious decision: is measuring this additional data point worth the effort and cost? Do I reasonably expect that it can contribute to real actionable insight? Can things really start to change on my website based on (insights from) this data?

But if it's purely to feed your curiosity, and nothing more than that. Then just don't do it for a change.