Managing guarded rollouts

Overview

This topic explains how to manage guarded rollouts.

Monitor a guarded rollout

You can monitor a guarded rollout on a flag’s Monitoring tab. The Monitoring tab displays the monitoring progression, how many contexts have been exposed to the new variation, and how each variation is performing against the control variation:

From the Monitoring tab, you can:

• Use the date range picker to view results over a selected period of time
• Hover on a graph to view more details about each metric
• Manually roll back the release
• Dismiss a regression alert and continue with the rollout
• Stop monitoring the rollout early

Monitoring charts

The charts on the Monitoring tab displays the following information for each metric you included in the guarded rollout:

• Metric: the metric name
• Graph: a chart showing the metric’s performance over time
• Treatment estimate: the point estimate of the metric for the treatment variation
• Diff. from control: the relative difference from the control variation
• Auto rollback: whether you enabled auto-rollback for that metric in the case of a regression

In LaunchDarkly, hover over the metric performance graph to view more information about that metric and any regressions detected:

Treatment estimates

“Treatment estimates” are point estimates of the metric value for the treatment variation. A point estimate uses sample data to calculate a single value that serves as the best approximation of the metric’s true value.

For metrics that use a percentile analysis method, such as latency at the 99th percentile, the “Treatment estimate” column displays the estimated percentile value for contexts that received the treatment variation.

For metrics that use the average analysis method, such as average response time or conversion rate, the “Treatment estimate” column displays the average metric value for contexts that received the treatment variation.

Relative difference from control

The relative difference from the control variation measures how much a metric in the treatment variation differs from the control variation, expressed as a proportion of the control’s estimated value. LaunchDarkly calculates this by taking the difference between the treatment variation’s estimated value and the control variation’s estimated value, then dividing that difference by the control variation’s estimated value.

Roll back releases

To manually roll back a release after LaunchDarkly has detected a regression:

1. Navigate to the flag’s Targeting or Monitoring tab.
2. Click Roll back. The “Stop rollout early” dialog appears.
3. Choose which Variation to serve to all contexts after you stop monitoring. The field defaults to the control variation.
4. Click Stop.

If you are using a guarded rollout on a prerequisite flag and you roll back the change, LaunchDarkly will not also roll back any changes on the dependent flags. You must roll back changes on dependent flags separately.

Automatic rollback behavior

LaunchDarkly automatically rolls back a guarded rollout in these scenarios:

• If LaunchDarkly detects a regression at any time during the rollout, and you have automatic rollback enabled.
• If LaunchDarkly detects a sample ratio mismatch (SRM) any time during the rollout. LaunchDarkly will roll back a release with an SRM whether or not you have automatic rollback enabled. To learn how to fix SRMs, read Understanding sample ratios.
• If too few contexts see the new variation by the end of the rollout, LaunchDarkly cannot detect a regression. In this case, it rolls back the release, even if automatic rollback is off. To improve results, try the rollout again with a longer duration.

When LaunchDarkly automatically rolls back a rollout, it sends an email and, if you use the LaunchDarkly Slack or Microsoft Teams app, a Slack or Teams notification.

If you are using a guarded rollout on a prerequisite flag and LaunchDarkly automatically rolls back the change, it will not also roll back any changes on the dependent flags. You must roll back changes on dependent flags separately.

Dismiss regression alerts

If LaunchDarkly detects a regression, but you want to continue with the release, you can dismiss the regression alert.

To dismiss an alert:

1. Navigate to the flag’s Targeting or Monitoring tab.
2. Click Dismiss alert. The “Dismiss alert” dialog appears.
3. Click Dismiss alert.

If LaunchDarkly detects another regression, you will receive another alert. If LaunchDarkly does not detect any additional regressions, the release will continue.

Stop monitoring early

To stop monitoring before the monitoring window is over:

1. Navigate to the flag’s Targeting or Monitoring tab.
2. Click Stop monitoring. The “Stop monitoring” dialog appears.
3. Choose which Variation to serve to all contexts after you stop monitoring. The field defaults to the control variation.
4. Click Stop.