Managing guarded rollouts

Overview

This topic explains how to manage guarded rollouts.

Monitor a guarded rollout

You can monitor a guarded rollout on a flag’s Monitoring tab. The Monitoring tab displays the monitoring progression, how many contexts have been exposed to the new variation, and how each variation is performing against the control variation:

From the Monitoring tab, you can:

• Use the date range picker to view results over a selected period of time
• Hover on a graph to view more details about each metric
• Manually roll back the release
• Dismiss a regression alert and continue with the rollout
• Stop monitoring the rollout early

Monitoring charts

The charts on the Monitoring tab displays the following information for each metric you included in the guarded rollout:

• Metric: the metric name
• Graph: a chart showing the metric’s performance over time
• Treatment estimate: the point estimate of the metric for the treatment variation
• Diff. from control: the relative difference from the control variation
• Auto rollback: whether you enabled auto-rollback for that metric in the case of a regression

Hover over the metric performance graph to view more information about that metric and any regressions detected:

Treatment estimates

“Treatment estimates” are point estimates of the metric value for the treatment variation. A point estimate uses sample data to calculate a single value to serve as a “best guess” for the metric’s true value.

For example, if a metric is measuring latency time using the 99th percentile analysis method, the “Treatment estimate” column displays the estimated 99th percentile of latency time for all contexts who received the treatment variation.

Or, if a metric is measuring average response time for a request, the “Treatment estimate” column displays the average response time for all contexts who received the treatment variation.

Relative difference from control

The relative difference from the control variation measures how much a metric in the treatment variation differs from the control variation, expressed as a proportion of the control’s estimated value. LaunchDarkly calculates this by taking the difference between the treatment variation’s estimated value and the control variation’s estimated value, then dividing that difference by the control variation’s estimated value.

Roll back releases

To manually roll back a release after LaunchDarkly has detected a regression:

1. Navigate to the flag’s Targeting or Monitoring tab.
2. Click Roll back. The “Stop rollout early” dialog appears.
3. Choose which Variation to serve to all contexts after you stop monitoring. The field defaults to the control variation.
4. Click Stop.

If you are using a guarded rollout on a prerequisite flag and you roll back the change, LaunchDarkly will not also roll back any changes on the dependent flags. You must roll back changes on dependent flags separately.

Automatic rollback behavior

LaunchDarkly automatically rolls back a guarded rollout in these scenarios:

• If LaunchDarkly detects a regression at any time during the rollout, and you have automatic rollback enabled.
• If LaunchDarkly detects a sample ratio mismatch (SRM) any time during the rollout. LaunchDarkly will roll back a release with an SRM whether or not you have automatic rollback enabled. To learn how to fix SRMs, read Understanding sample ratios.
• If, by the end of the rollout period, too few contexts were exposed to the new variation for LaunchDarkly to determine whether or not a regression occurred. LaunchDarkly will roll back a release with insufficient exposures whether or not you have automatic rollback enabled. In this case, we recommend you retry the guarded rollout over a longer duration to collect more data.

When LaunchDarkly automatically rolls back a rollout, it sends an email and, if you use the LaunchDarkly Slack or Microsoft Teams app, a Slack or Teams notification.

If you are using a guarded rollout on a prerequisite flag and LaunchDarkly automatically rolls back the change, it will not also roll back any changes on the dependent flags. You must roll back changes on dependent flags separately.

Dismiss regression alerts

If LaunchDarkly detects a regression, but you want to continue with the release, you can dismiss the regression alert.

To dismiss an alert:

1. Navigate to the flag’s Targeting or Monitoring tab.
2. Click Dismiss alert. The “Dismiss alert” dialog appears.
3. Click Dismiss alert.

If LaunchDarkly detects another regression, you will receive another alert. If LaunchDarkly does not detect any additional regressions, the release will continue.

Stop monitoring early

To stop monitoring before the monitoring window is over:

1. Navigate to the flag’s Targeting or Monitoring tab.
2. Click Stop monitoring. The “Stop monitoring” dialog appears.
3. Choose which Variation to serve to all contexts after you stop monitoring. The field defaults to the control variation.
4. Click Stop.