Traffic reshuffling

This topic explains how LaunchDarkly assigns or reassigns contexts to flag variations when you make changes to an experiment. When you make certain edits to an experiment, such as adding additional metrics or increasing traffic, LaunchDarkly creates a new iteration of that experiment.

When LaunchDarkly creates a new iteration, you can either allow or disable traffic reshuffling:

Allow traffic reshuffling: This option lets LaunchDarkly assign contexts to new variations when you start a new experiment iteration. For example, if LaunchDarkly assigns user context Anna to the “blue button” variation in the first iteration of an experiment, LaunchDarkly reassigns Anna to the “red button” variation of the experiment in the next iteration. This option is useful for when you want to keep the number of contexts evenly allocated between different variations.
Disable traffic reshuffling: This option forces LaunchDarkly to keep contexts assigned to their original variation across iterations. For example, if LaunchDarkly assigns user context Anna to the “blue button” variation in the first iteration of an experiment, LaunchDarkly will always assign Anna to the “blue button” variation in subsequent iterations. This option is useful when you want to keep user experience consistent for the full duration of an experiment. This is the default setting when you create an experiment.

Allow or disable reshuffling

You can decide whether to allow or disable reshuffling when you create an experiment.

To allow or disable traffic reshuffling for a new or existing experiment, navigate to the experiment’s Design tab. In the “Audience allocation” section, check or uncheck the Disable reshuffling box. By default, LaunchDarkly does not reshuffle experiment traffic when you start a new iteration.

This setting only preserves assignments when you increase traffic. For example, increasing traffic from 5% to 20%, then from 20% to 60%, preserves existing assignments. If you decrease traffic and then increase it again, such as from 20% to 5%, then from 5% to 60%, LaunchDarkly may reassign contexts to different variations.

Traffic will be assigned to new variations if you stop an experiment

Checking the Disable reshuffling box prevents traffic reshuffling only if you make changes to an experiment using the experiment’s Edit design button. If you use the Stop button instead, LaunchDarkly will always reshuffle traffic into new variations when you start a new iteration, whether or not the Disable reshuffling box is checked.

Example of an experiment with traffic reshuffling disabled

In this example, you are running an experiment with traffic reshuffling disabled.

You add an experiment to a flag with three variations of Blue, Green, and Orange. You include 30% of total contexts in the experiment, with approximately 10% of traffic assigned to each variation.

The remaining 70% of traffic receives the control variation of Blue. The control traffic is not part of the experiment nor its analysis.

Next, you increase your experiment traffic from 30% to 60%. This creates a new iteration of the experiment. In the new iteration, the 30% that were already receiving variations Blue, Green, and Orange continue to receive those variations, but are no longer included in the experiment nor its analysis. New traffic is used for the additional 30% allocated to the experiment.

Here is a visualization of the traffic allocation in the first and second iterations:

An experiment audience increasing from iteration to iteration.