2. Mutually exclusive tests
This is the tactic we’ll be covering below. By running mutually exclusive tests, you’ll be able to better eliminate noise and bias from your results and have (at least in theory) more reliable and untainted data. The potential downside is that traffic volumes will be lower and may slow down testing as you wait for adequate traffic.
3. Combine multiple tests into one as a MVT
The last option is to combine the tests into one MVT, multi-page test. This sort of tactic will give you the best read on how the two tests relate to one another and what the optimal experience is in relation to both tests. Again, you’ll be sacrificing speed of testing, as the multiple combinations will require higher traffic volumes.
With some context and understanding of the options above, let’s look at how to set up mutually exclusive tests in Optimize.
Set up your first experiment. For example, let’s say we’re hiding an element on the homepage. Create the experiment and make the change in the variant. Add a second variation that will serve as the true ‘control’. Add a bit of javascript to both variations that sets a cookie “Homepage Test” = true, and adjust the traffic allocation to split traffic evenly between the two cookied variants.
Next, create your second test that you’d like to be mutually exclusive. In this case, let’s set up a test on a specific collection page. Make the adjustment in the variant and then, under Audience Targeting, select ‘First-party cookie’ and configure the ‘Homepage Test’ cookie as a new cookie.
Add audience targeting when ‘Homepage Test’ does not equal true.
Once live, if a user is enrolled in the homepage test, they will be excluded from the collection page test, keeping cross-contamination at a minimum.