This is a core concept
ion enables you to split test (A/B test) any number of Creatives (landing pages) on any of your ion URLs. You will create a URL (an ion Traffic Source) and associate any Creatives you want to test to the URL. Then ion will distribute the incoming traffic across those Creatives in your test group.
This article covers the step-by-step to launching an A/B test.
Before you begin
- You should be familiar with (1) creating landing experiences in ion and (2) how to open a Traffic Source (going live). We lightly touch on these concepts below, but if you aren't already familiar with this, check out the Publishing section and the Going Live section in this support site to get familiar with those concepts.
- It's a best practice to start with a test plan — know what you want to test and why. If you are new to testing, you can download a copy of ion’s “Guide to Online Testing” for more information on formulating your test strategy.
Let’s Get Started!
1. Engage Your Creatives
To begin A/B testing, you will need to have two or more engaged creatives within a single campaign. Once you have your creatives ready, setting up the actual A/B test only takes about one minute! It’s that easy, just follow these instructions for opening a traffic source and splitting your traffic across your creative variations.
- Navigate to the creative management screen for each creative and click the drop-down in the top right corner labeled “MODE”
- Select “Engaged” from the drop-down menu and then click the blue refresh button immediately to the right
- The pages within this creative should now be highlighted in green
2. Open A Traffic Source
Once your creatives are engaged, you can now open a traffic source (a unique URL) that will split traffic among your engaged creatives. Opening a traffic source both takes your creative live by setting up a unique URL, as well as gives you the opportunity to split traffic across two or more creatives in an A/B test.
- Navigate to the campaign management screen
- Click the orange “traffic sources” tab just above your list of creatives
- Click the green plus sign on the “traffic sources” tab to open a new traffic source
- Select the subdomain you would like to use with the URL you are creating
- Add a unique identifier for this page in the text box immediately to the right of the subdomain drop-down menu
- Choose how you would like to track visitors
- This will show in reporting how traffic is driven into this URL
- Choose a category and a sub-category
- If there is not a sub-category that you would like to use, you can add a custom sub-category by clicking the button immediately to the right
- Select one of the creatives you would like to receive traffic from this URL
- Once selected, the thumbnail of the creative you chose should be highlighted in green
- Click Save
3. Split Your Traffic
After creating your traffic source, you will be taken to the traffic source management screen. From here, you will be able to set up your newly created URL to split traffic between multiple engaged creatives.
- To the left of each creative in the traffic source management screen, you will notice sliders for each creative
- These are weighted sliders that will split traffic based on the weighted sliders
- To split traffic, move the slider to the right for a creative that is not set to receive traffic from your URL
- Note: Only creatives highlighted in green are set to receive traffic from your URL
- If your sliders are all in the same position, then traffic will split evenly across the green creatives
- You can have your traffic split in any way you would like
- The percentage of traffic for each creative is represented in a pie graph labeled “Traffic allocation” to the right of the list of creatives
- Once your sliders have been set as you would like, any traffic that lands on this URL will be split as specified by your sliders
- Behind your single traffic source URL, you can route the traffic across as many creative variations as you wish
ion allows you to automatically optimize, by sending traffic to your winning creative once statistical significance is reached, or you can manually monitor your test results without auto-optimization.
- In the top right corner of the traffic source management screen, you will see a button displaying the word “automatic” just below the word “OPTIMIZATION”
- By default, ion will automatically find the better performing creative and remove the lesser performing creative from the traffic split
- ion determines which creative is performing better based on a statistical confidence interval that is set in the “Configure Site” screen
- You can find more information on statistical confidence in the "Statistical Confidence" section below.
- If you would like to manually control when a creative is removed from the test, simply click the button below “OPTIMIZATION” and it will switch over to “manual”
The ion platform can compare the conversion results of different Creatives and determine which one will be more successful based on a predetermined confidence level. Within your ion console you can define this level between 80% and 99% confidence at your preference.
To set your preferred confidence level, hover over the “Settings” tab in your ion console, and select “Configure Site” from the drop down. At the bottom of the page, you will see a drop down menu labeled, “Confidence.” This is where you can select the level of confidence you would like to achieve before declaring a winner of a given test between creatives.
- Selecting a higher confidence level may require more respondents (traffic) to achieve confidence and therefore takes longer to achieve than lower confidence levels.
- Selecting a higher confidence level will result in more accurate test results (a higher level of confidence means the margin of error is smaller on the results).
You can view your test confidence in the "traffic source performance" tab for any of your traffic sources—check out the "creative summary" gauge for the margin of error, and check out the "confidence interval" to view the test results at both 80% & 95% confidence.