How does OpenWrap A/B testing work?
A/B testing is a method of comparing two versions of something to see which one performs better. OpenWrap A/B testing allows you to test two profile configurations and compare the results. You can use this data to optimize the profile settings.
Group A = This is the control group. Results for this traffic are based on the profile version as is.
Group B = This is the test group. Results for this traffic are based on the modified profile.
Types of A/B testing
OpenWrap supports two types of A/B tests:
Test a different auction timeout on a fraction of traffic to see the effect on revenue and latency.
Test the overall effect of adding or removing bidding partners. Note that the test partner path defaults to the control partner path.
Set up an A/B test
Only one A/B test can be performed at at time. If you want to perform more than one test, create a new profile version for each one.
- Create an OpenWrap profile as you normally would.
- Enable A/B Testing. You can edit the profile version to disable the test at anytime.
- Select the Test Group Size and Test Type.
- Enter the test group criteria. The control and test configurations must be different otherwise the system will not let you save the profile.
- Access the results in the Profile Details page.
Sample results page: