Creating guarded rollouts

Overview

This topic explains how to create a guarded rollout. Guarded rollouts progressively increase traffic to a flag or AI Config variation while monitoring selected metrics for regressions. If LaunchDarkly detects a regression, it can pause the release and send you a notification. This helps your team reduce release risk and respond quickly if performance degrades.

What happens when a guarded or progressive rollout starts or stops

Guarded rollouts and progressive rollouts use randomized traffic allocation to determine which contexts receive a new variation. Before you create a rollout, understand how context targeting behaves when stopping and restarting the rollout.

If you stop a guarded rollout or progressive rollout and later create a new one on the same flag or AI Config variation, LaunchDarkly may serve the variation to a different set of contexts than in the original rollout. This happens because each new guarded or progressive rollout independently assigns contexts to receive the variation, based on the selected randomization unit.

When you stop a progressive rollout, you must choose a specific variation to serve to all traffic for that targeting rule. After you save, LaunchDarkly updates the rule to serve the selected variation to every context that matches the rule.

This behavior is different from flag percentage rollouts, which retain the same set of contexts when you stop and restart them. The same contexts continue to receive the variation, provided the flag configuration and context kind remain unchanged.

Prerequisites

To create a guarded rollout, you should first complete the following prerequisites:

Choose a context kind

You must choose a context kind for the rollout to gradually ramp up traffic to the new flag or AI Config variation.

If this is your first guarded rollout, read Setting up contexts for guarded rollouts to learn more about best practices for setting up and choosing contexts.

Create metrics

Choose one or more existing metrics or create new ones to monitor during the rollout. To learn how, read Creating and managing metrics.

Understand regression detection

Release Guardian measures metric changes using absolute difference. Sequential testing determines whether the observed absolute difference between the new and original variations is statistically significant.

Rollout duration

LaunchDarkly monitors how the new variation is performing over a specified time period.

Each step of a guarded rollout requires the new variation to be served to a minimum number of contexts before LaunchDarkly can advance to the next step. If this minimum requirement is not met, LaunchDarkly automatically extends the current step to allow more contexts to receive the variation. If the requirement is still not met after the extension period, LaunchDarkly can roll back the change.

Create guarded rollouts on flags and AI Configs

You can create a guarded rollout on a flag or AI Config:

• When you toggle the flag or AI Config on and begin serving a new variation
• If the flag or AI Config is already toggled on, when you make a change to a variation

You can create a guarded rollout on any targeting rule, as long as there are no other guarded rollouts, progressive rollouts, or experiments running on the flag or AI Config, and the flag is not a migration flag.

Follow the steps below to set up a guarded rollout.

Create a guarded rollout on a rule

To create a guarded rollout, first open the flag or AI Config you want to release, then configure the rollout settings on its Targeting tab.

1. Select the flag or AI Config you want to release, then navigate to its Targeting tab.

2. Find the targeting rule for which you want to create a guarded rollout and click Edit.

3. From the Serve menu, select Guarded rollout. The guarded rollout setup options appear.

4. Click Health warnings to check for any issues with your flag, AI Config, or metric that could prevent LaunchDarkly from monitoring the health of your rollout. To learn more, read Health checks for guarded rollouts.

5. Use the Original variation and Target variation menus to select variations to use in this rollout.

6. Select one or more Metrics to monitor.

7. For each metric, check the Automatic rollback box if you want LaunchDarkly to automatically revert your change when it detects a regression in addition to sending a notification.

   LaunchDarkly can automatically roll back a guarded rollout if it detects a sample ratio mismatch (SRM) at any time during the rollout, or if the new variation does not reach the required minimum number of contexts by the end of the rollout.

8. Choose a context kind to determine how to assign traffic to variations. This context kind is called a randomization unit.

9. Select a Rollout duration. You can use the default duration or set a custom schedule. LaunchDarkly automatically increases the percentage of contexts receiving the new variation over the course of the rollout duration until it reaches 100%.

(Optional) Schedule and request approval

If your workflow requires review or scheduled changes, you can schedule the rollout and request approval before it starts.

1. Click Review and save. The “Save changes” dialog appears.
2. In the bottom left corner, click the calendar icon.
3. Select a Date, Time, and Timezone.
4. Click Update.
5. (Optional) Request reviewers to approve your changes:
   1. If the Reviewer(s) menu is not visible, click the > icon.
   2. Use the “Select reviewers” dropdown menu to select one or more reviewers. You can enter text in the “Select reviewers” box to filter your choices.
      • Members and teams with permission to approve or decline approval requests appear in the top section of the menu.
      • Members and teams that can only view approval requests appear in the bottom section of the menu. If you add a member or team from this list, they will be notified of the approval request, but cannot approve it.
   3. Enter a Comment to provide context for reviewers.
      • If your organization uses a third-party workflow management system for approvals, there may be additional fields you are required to complete. To learn more, read Configuring approvals for an environment.
   4. Click Request approval.
6. Click Review and save.

Review rollout status

After saving, the rule displays the percentage of contexts currently receiving the new variation.

The top of the Targeting tab for the flag or AI Config displays the current rollout status. To view more details about the rollout, click the Monitoring tab. To learn more, read Managing guarded rollouts.

If LaunchDarkly detects a regression, it can send an email, an in-app notification, and, if configured, a Slack, Microsoft Teams, or PagerDuty notification. To learn how to set up these integrations, read:

• Setting up Slack accounts and permissions
• Setting up the Microsoft Teams integration
• PagerDuty for guarded rollouts

How metrics are displayed

In a guarded rollout, each metric appears in its own tile. Each tile contains a difference chart that shows the absolute difference between the new and original variations.

Each metric tile includes:

• Header: Identifies the metric and shows the current difference estimate. Hover over the metric name to view its definition, description, environment, and last-seen timestamp.
• Chart: Shows the absolute difference between the new and original variations over time. The dark grey line represents the observed difference. The shaded grey area represents the confidence interval. If sequential testing determines that the observed absolute difference represents a statistically significant negative impact on a monitored metric, LaunchDarkly identifies a regression. When this occurs, the tile highlights the regression, and if automatic rollback is enabled, LaunchDarkly rolls back the release.
• Footer: Summarizes the most recent results and displays the latest values for both the new and original variations.

On the Targeting tab, metric tiles appear in up to three columns in a single row.

On the Monitoring tab, each metric appears on its own tab. Hovering over the chart shows new and original variation estimates, the confidence interval, and the number of contexts served at that point in time.