Creating guarded rollouts

Overview

This topic explains how to create a guarded rollout. Guarded rollouts let you progressively increase traffic to a flag or AI Config variation over time while monitoring chosen metrics for regressions. If LaunchDarkly detects a regression, it can pause the release and send you a notification.

What happens when a guarded or progressive rollout starts or stops

Guarded rollouts and progressive rollouts use randomized traffic allocation to determine which contexts receive a new variation. Before you create a rollout, understand how context targeting behaves when stopping and restarting the rollout.

If you stop a guarded rollout or progressive rollout and later create a new one on the same flag or AI Config variation, LaunchDarkly may serve the variation to a different set of contexts than in the original rollout. This happens because each new guarded or progressive rollout independently assigns contexts to receive the variation, based on the selected randomization unit.

This behavior is different from flag percentage rollouts, which retain the same set of contexts when you stop and restart them. The same contexts continue to receive the variation, provided the flag configuration and context kind remain unchanged.

Prerequisites

To create a guarded rollout, you should first complete the following prerequisites:

Choose a context kind

You must choose a context kind for the rollout to gradually ramp up traffic to the new flag or AI Config variation.

If this is your first guarded rollout, read Setting up contexts for guarded rollouts to learn more about best practices for setting up and choosing contexts.

Create metrics

Before you begin a guarded rollout, choose one or more existing metrics or create new ones to monitor. To learn how, read Creating and managing metrics.

Determine regression thresholds

For each metric you use in the guarded rollout, you can use the LaunchDarkly default regression threshold or set a custom one:

• The Default option uses LaunchDarkly’s standard regression threshold. This option is appropriate for most guarded rollouts.
• The Set custom thresholds option lets you define how much underperformance you’re willing to tolerate in the new variation. LaunchDarkly employs a Bayesian statistical model to assess the probability of a new variation performing worse than the original variation. This model applies to all metrics, whether they use average or percentile analysis methods.

To learn more, read Regression thresholds for guarded rollouts.

Rollout duration

LaunchDarkly evaluates how the treatment variation is performing over a specified time period. You can use the default duration or set a custom schedule. If the variation isn’t exposed to enough contexts during this period, LaunchDarkly will roll back the change.

Create guarded rollouts on flags and AI Configs

You can create a guarded rollout on a flag or AI Config:

• When you toggle the flag or AI Config on and begin serving a new variation
• If the flag or AI Config is already toggled on, when you make a change to a variation

You can create a guarded rollout on any targeting rule, as long as there are no other guarded rollouts, progressive rollouts, or experiments running on the flag or AI Config, and the flag is not a migration flag.

Follow the steps below to set up a guarded rollout.

Open the rollout setup

1. Navigate to the Targeting tab for the flag or AI Config.
2. Find the targeting rule for which you want to create a guarded rollout and click Edit.
3. From the Serve menu, select Guarded rollout. The guarded rollout setup options appear.
4. Next to Guarded rollout setup, click Health warnings to check for any issues with your flag, AI Config, or metric that could prevent LaunchDarkly from monitoring the health of your rollout. To learn more, read Health checks for guarded rollouts.

Configure rollout details

5. From the Target variation menu, select the variation to gradually roll out.
6. Choose a context kind to determine how to assign traffic to variations. This context kind is called a randomization unit.
7. Select one or more Metrics to monitor.
8. Under Regression detection and behavior, define how much worse the target variation can perform compared to the original variation before LaunchDarkly detects a regression.
   • Check Automatic rollback if you want LaunchDarkly to automatically revert your change when it detects a regression, in addition to sending a notification. LaunchDarkly always rolls back a release automatically if it detects a sample ratio mismatch (SRM) or if the release has not been exposed to enough contexts by the end of the rollout period.
9. Select a Rollout duration. You can use the default duration or set a custom schedule. LaunchDarkly automatically increases the percentage of contexts receiving the new variation over the course of the rollout duration until it reaches 100%.
10. Click Review and save.

(Optional) Schedule and request approval

1. Click Request approval. The “Save changes” dialog appears.
2. Review and confirm the changes displayed in the dialog.
3. Enter a Reason to provide context for reviewers. If your organization uses a third-party workflow management system for approvals, there may be additional fields you are required to complete. To learn more, read Configuring approvals for an environment.
4. Choose one or more reviewers from the Reviewers menu.
   • Members and teams with permission to approve or decline requests appear in the top section of the menu.
   • Members and teams that can only view approval requests appear in the bottom section. If you add one of these members or teams, they will be notified but cannot approve the request.
5. To schedule the rollout start time:
   1. Click Schedule + in the Save changes dialog.
   2. Select a date, time, and timezone.
6. In the Comment field, add a comment describing the changes.
7. Click Save changes.

Review rollout status

After saving, the rule displays the percentage of contexts that have been exposed to the new variation.

The top of the Targeting tab for the flag or AI Config displays the current rollout status. To view more details about the rollout, click the Monitoring tab. To learn more, read Managing guarded rollouts.

If LaunchDarkly detects a regression, it sends an email, an in-app notification, and, if configured, a Slack, Microsoft Teams, or PagerDuty notification. To learn how to set up these integrations, read:

• Setting up Slack accounts and permissions
• Setting up the Microsoft Teams integration
• PagerDuty for guarded rollouts

How metrics are displayed

In a guarded rollout, each metric appears in its own tile. Each tile contains a relative difference chart.

Each metric tile includes:

• Header: Identifies the metric. Shows the metric name and the current relative difference estimate. You can hover over the metric name to view its definition and description, the environment and time when it was last seen, and a link to the metric details page.
• Chart: Shows the relative difference between treatment and control over time. The line is the relative difference as a percentage, with a shaded grey area for confidence interval and a dotted line for the regression threshold. If the entire credible interval goes above the threshold line for lower-is-better metrics, or below the threshold line for higher-is-better metrics, LaunchDarkly calls a regression. When this happens, the tile highlights the regression, and if automatic rollback is enabled, LaunchDarkly reverts the rollout.
• Footer: Summarizes the most recent results. Shows the latest values for both treatment and control.

On the Targeting tab, metric tiles appear in up to three columns in a single row. If you have more metrics than fit in one row, click Show all metrics to expand the view. Hovering over a chart shows a pop-up with the relative difference between treatment and control at that time.

On the Monitoring tab, each metric appears on its own tab. Hovering over the chart shows control and treatment estimates, confidence interval, and exposures at that point in time.