Despite providing an enormous amount of valuable information, we see that bandwidth calculations are still far too underutilized in CRO. If you want to run a mature and evidence driven CRO program then calculating your bandwidth is indispensable. Together with my colleagues, I have listed for you why it is important to do a bandwidth calculation before you start A/B testing (in the first phase of the CRO process).

Why are you doing a bandwidth calculation?

Bandwidth is basically your test potential calculated over the entire year. The calculation gives you a lot of information about how your site is set up, how your visitors use the site throughout the year, and thus how best to organize your testing. The bandwidth calculation answers the following questions:

Which page creates the highest impact?

First, you find out which pages are appropriate to test on and which pages will have the most impact. A bandwidth calculation is a data-supported way to determine on which pages it is desirable to test (in most cases, these are where the most money can be made).

What effect should be achieved per page?

In addition to ranking the pages by impact, you also get insight into the minimum required impact per page. This is a good measure of the size of the change in the variant. Is the minimum required impact for a page low? Then you can recognize a behavioral change with certainty with a small adjustment. Is the minimum required effect high? Then you need to make a large adjustment in the variant to see a behavioral difference in the data with certainty.

Which tests are given priority?

Within large companies, requests frequently come from other departments to A/B test certain components on a site. Although it's great when the entire company gets started with CRO and provides input for testing, a lack of knowledge often causes questions to be irrelevant. The bandwidth calculator helps prioritize which pages you want to test on. It substantiates why one idea is rejected and the others are not.

Am I getting everything out of my tests?

In addition, the bandwidth calculation also shows whether you are using the full testing potential. This is because you can compare the number of tests you have realized against the bandwidth (the maximum number of tests you can perform) and set goals based on that.

How long will my test be live?

Finally, the bandwidth calculation for each page type shows how many weeks a test should last and the uplift you should achieve. These page types are ranked by additional potential revenue per page type.

A bandwidth calculation avoids a lot of pitfalls

By calculating your bandwidth you can avoid a lot of errors that could otherwise cause problems in achieving your business case or improving your CRO program in general. For example, without knowing what the minimum impact is at a given page you run the risk of not having enough uplift and conversions and thus not having enough data at the end of your test. Also, a test that is too short can result in too few visitors and data, making your test results inconclusive (they have too low a power). Too little data makes it impossible to identify a variant as a winner or loser.

The prerequisite for a successful testing program

Like a good technical check, bandwidth calculations are a prerequisite before you start testing. First you want to know that the data is correct and that everything is measured so that you can properly map the behavior. Then you start calculating on which pages, for how long and at what (minimum) uplift you should test. This allows you to rank page types based on additional potential sales. It is the basis for (data-driven) testing and therefore, in our opinion, indispensable in a successful CRO program.

This article was published on Aug. 3 at Webanalists.com