Shipping the winning variation
Overview
This topic explains how to choose and ship a winning variation for a completed experiment.
LaunchDarkly offers both Bayesian and frequentist statistics as analysis methods for experiments.
The winning variation in Bayesian experiments
For experiments using Bayesian statistics, the winning variation is typically the treatment with the highest probability of being the best among those that show a significant high probability of beating the control. If a Bayesian experiment has collected enough data to determine a winning variation, and the winning variation is not the control, then the winning variation is highlighted in green on the Results tab.
If all treatment variations have a significantly low probability of beating the control, then the control is considered the winning variation. To learn more, read Decision making with Bayesian statistics.
The winning variation in frequentist experiments
For experiments that use frequentist statistics, the winning variation for an experiment is the variation that has the highlighted p-value. You should only ship a winning variation after the experiment has reached its needed sample size.
Ship the winning variation
To stop an experiment and ship the winning variation:
- Navigate to the experiment’s Results tab.
- Click Stop.
- Select the winning variation.
- Enter a Reason for stopping.
- Click Stop experiment.
Certain Bayesian experiments will display a Ship it button when the experiment has enough data to determine a winning variation. You can also use this option to stop the experiment and ship the winning variation.