This topic explains how to include specific groups of contexts in an experiment audience using audience allocation.
You have the option to only include a subset of contexts in your experiments, which gives you accurate experiment results more quickly. This subset of contexts is called your “experiment audience.”
LaunchDarkly determines audience membership when an SDK evaluates the flag’s targeting rule for a context and LaunchDarkly receives the corresponding evaluation event. To learn more, read Experimentation and metric events.
For example, imagine you plan to test alternate copy for your checkout button. You target all Canadian and US contexts with the true variation for the button, which shows the new, alternate copy, but you only want to run an experiment measuring click conversions for end users in the United States.
To accomplish this, you would select the targeting rule on the flag’s Targeting tab that affects US-based contexts and de-select the rule that targets contexts in Canada. This limits the end users who evaluate the flag to only those who are based in the United States.
You may want to refine your experiment audience for any of the following reasons:
You determine the initial experiment audience when you create a new experiment. You must include at least two variations in the experiment for the experiment to be valid. To learn more, read Creating experiments.
You can run an experiment on a flag’s default rule, or you can create a custom experiment audience by selecting a specific flag targeting rule to include in your experiment. In either case, the context kind that the flag rule targets should match the randomization unit of your experiment. We recommend running the experiment on a rule that targets a subsection of your contexts rather than the default rule. This helps ensure consistent experiment results.
A flag targeting rule can target by any context attribute you collect. To learn how, read Target with flags.
We strongly recommend using flag targeting rules to limit experiment audiences, rather than limiting experiment audiences directly in your app code. If you limit experiment audiences in your app code, it can cause a sample ratio mismatch and unreliable experiment results.
When you build your experiment, you can allocate all or a percentage of the traffic that matches the flag targeting rule you choose. Audience allocation gives you flexibility when selecting your experiment audience and ensures accurate experiment results. LaunchDarkly analyzes only contexts that you choose to be part of the experiment.
If you decide to increase or decrease the number of contexts in an experiment, LaunchDarkly will create a new iteration of your experiment. To learn more, read Start experiment iterations.
LaunchDarkly automatically performs checks on the allocation, to make sure that actual traffic matches the allocation you set. To learn more, read Sample ratio mismatch.
If you change the amount of traffic in an experiment and start a new iteration, some of the contexts in the experiment may begin receiving different variations:
To learn how to change the audience for a running experiment, read Change experiment audiences.