Skip to main content
Auto-classification is available to customers on select plansAuto-classification is only available to customers on certain plans. To learn more, read about our pricing or contact Sales.
Agent skill availableIf you use an AI coding agent, you can use the LaunchDarkly should-flag-change agent skill to decide whether a change needs a flag before you open a pull request. It reads the diff and the surrounding code, then recommends whether the change needs a flag. It is read-only and never creates or changes flags.To install the skill, run npx skills add https://github.com/launchdarkly/ai-tooling --skill should-flag-change.To learn more, read LaunchDarkly agent skills.
This topic explains how Release Autopilot reviews each pull request and recommends whether the change should be released behind a feature flag. Release Autopilot makes this recommendation through auto-classification. When a pull request is ready for review, auto-classification reads the code changes in the pull request and the surrounding code. It classifies the change into one of four outcomes, then posts a comment on the pull request with its recommendation. The comment appears within a few minutes. The outcome tells you whether to release the change behind a feature flag. A feature flag lets you release a change gradually and roll it back without a redeploy. Not every change needs a flag. Auto-classification applies the same criteria to each pull request to identify the ones that do. To learn about the ways LaunchDarkly can release a change gradually, read Releasing features with LaunchDarkly. Use auto-classification to:
  • Identify changes that need a flag before they merge.
  • Skip flags on changes that do not need them.
  • Find an existing flag to reuse instead of creating a duplicate.
  • Apply the same flagging criteria across your mapped repositories.
Auto-classification does not create flagsAuto-classification recommends an outcome and explains its reasoning. You decide whether to create or reuse a flag.

Prerequisites

You need the following to receive auto-classification recommendations:
  • A repository that you have mapped to a LaunchDarkly project.
  • Auto-flagging or auto-releasing turned on for the repository. Auto-classification does not have its own setting. To learn how, read Turn on capabilities for a project.
  • A pull request that you have marked ready for review, or a comment that says @launchdarkly plan on a draft pull request.
If you do not receive a recommendation, read Troubleshoot when no comment appears.

What triggers auto-classification

Auto-classification classifies a change when a pull request opens or reopens, or when you mark the pull request as ready for review. It does not classify a draft pull request automatically. This keeps recommendations scoped to code that the author considers complete. To classify a draft pull request, comment @launchdarkly plan on it. Auto-classification skips the pull requests that Release Autopilot creates for automated flag cleanup. These pull requests exist to remove flags that are no longer needed, not to add flags.

Classification outcomes

Auto-classification assigns each pull request one of four outcomes. The rule behind every outcome is to flag the change when it benefits from its own gradual rollout or rollback. To apply the rule, auto-classification checks the following:
  • Which code calls the changed code.
  • Whether the change is a behavior change, which means it alters what production does for users.
  • Whether a flag already protects the change.
The four possible outcomes are:

New flag suggested

Auto-classification suggests a new flag when the rule applies to the change and the change is not already covered by an existing flag. Auto-classification suggests a new flag for changes such as:
  • A new user-facing path, such as an endpoint, screen, or flow.
  • A change to the behavior of a live production path, such as authentication, payments, permissions, data writes, pricing, or rate limits.
  • A phased migration, such as a dual write, a backfill, or a cutover between an old and a new implementation.
  • A change to a performance-sensitive path, where a fast rollback without a redeploy is valuable.
  • A change to a critical subsystem that many other parts of the code depend on.
  • A change that must not go live before a different feature is released.

Existing flag reused

Auto-classification recommends reusing an existing flag when the change is part of a larger feature that is not yet released and that an existing flag already controls. The flag must also address the risk of the change. Reusing the flag avoids creating a duplicate flag for the same feature. When the change is a separate release with its own rollout or ownership, auto-classification suggests a new flag instead.

Already flagged

Auto-classification reports that a change is already flagged when a flag already protects it in one of the following ways:
  • The pull request adds the change behind a flag check.
  • The changed code sits inside an enclosing flag that is off everywhere or still rolling out.
An enclosing flag is a flag check that wraps the changed code, such as a route guard, a wrapping component, or a conditional branch. Auto-classification identifies the flag and, for an enclosing flag, its rollout state. An enclosing flag does not protect a change when it is fully launched or when it permanently controls configuration or entitlements. In those cases, auto-classification evaluates the change as it would any other change and is likely to suggest a new flag.

Not suited

Auto-classification reports that a change is not suited to a flag when there is nothing to roll out or roll back. Auto-classification does not suggest a flag for the following changes when users cannot observe a difference:
  • Refactors that preserve behavior, such as renaming, extracting, moving, or reformatting code.
  • Test-only changes.
  • Documentation and comment changes.
  • Build, continuous integration, and tooling configuration changes that do not affect runtime behavior.
  • Dependency bumps with no behavior change at the places where your code uses the dependency.
  • Mechanical or code-generated changes that preserve behavior.
Auto-classification can treat a small, additive, or low-risk change as not suited even when users can see it. For example, a new button might not be classified as needing a flag. You can re-run the classification task to ask for a flag by replying @launchdarkly replan and including an explanation of why the change should be flagged.
Some refactors and dependency bumps change behaviorAuto-classification treats a refactor that also changes business logic or an API contract as a behavior change. It also treats a dependency bump that changes runtime behavior on a live path as a behavior change. In these cases, auto-classification is likely to suggest a new flag. Missing a risky change costs more than suggesting a flag that the change did not need.

Example outcomes

These examples show typical outcomes for different pull requests. To learn how auto-classification defines each outcome, read Classification outcomes. Auto-classification weighs the details of each change. A similar pull request can receive a different outcome. The last three examples show cases where the details of the change determine the outcome.

New public endpoint

A pull request adds a new public endpoint, and no flag check exists nearby. Auto-classification suggests a new flag. The endpoint is a new user-facing path that no flag protects.

Dual write to a new data store

A pull request adds a dual write to a new data store, and the old data store stays in use. Auto-classification suggests a new flag. The change is a phased migration, where a flag controls the rollout and provides a rollback.

Change to an authentication check

A pull request changes the behavior of an authentication check that every route depends on. Auto-classification suggests a new flag. The change alters a live production path that many callers depend on.

Change to a default sort order

A pull request changes the default sort order of a table. Auto-classification suggests a new flag. Users experience the change without opting in, even when the change is one line.

New route in an unreleased feature

A pull request adds a new API route in one file. Another file in the same pull request checks a flag that controls the same feature, and the feature is not yet released. Auto-classification recommends reusing the existing flag. The flag already controls the same unreleased feature and safely covers the risk of the change.

Code inside a flag that is off everywhere

A pull request adds code to a function that sits inside a route guard, and the guard checks a flag that is off everywhere. Auto-classification reports the change as already flagged. An enclosing flag that is off everywhere already protects the change.

Code inside a flag that is still rolling out

A pull request changes code inside a flag check, and the flag is still rolling out. Auto-classification reports the change as already flagged. The flag already protects the change.

Update to test fixtures and mocks

A pull request updates test fixtures and mocks and changes nothing else. Auto-classification reports the change as not suited. Test-only changes do not ship to users.

Rename that changes no behavior

A pull request renames internal functions and changes no behavior. Auto-classification reports the change as not suited. A rename that preserves behavior leaves nothing to roll out or roll back.

Code inside a fully launched flag

A pull request adds new behavior inside a flag check, and the flag is fully launched. Auto-classification is likely to suggest a new flag. A fully launched flag no longer protects the change. Auto-classification evaluates the change as it would any other change.

Dependency bump that changes defaults

A pull request bumps a dependency to a major version that changes defaults on a live path. Auto-classification is likely to suggest a new flag. The change alters runtime behavior on a live path.

Rename that changes a return value

A pull request renames a function and also changes what the function returns. Auto-classification is likely to suggest a new flag. The change is not a pure refactor. Auto-classification treats it as a behavior change.

Use the recommendation

After auto-classification posts its comment on your pull request, review the outcome and the reasons for it. The reasons point to the specific files and behaviors that led to the outcome.
  • When the outcome is new flag suggested, create the flag, wrap the change in a flag check, and release the change behind the flag.
  • When the outcome is existing flag reused, release the change behind the flag that auto-classification recommends.
  • When the outcome is already flagged or not suited, continue with your normal review.

Troubleshoot when no comment appears

A pull request that does not receive a comment from auto-classification has one of the following causes. To learn what triggers auto-classification, read What triggers auto-classification.

The repository is not mapped

Auto-classification classifies pull requests only in repositories that you map to a LaunchDarkly project. Map the repository to the project that holds its flags.

Auto-flagging and auto-releasing are off

Auto-classification does not classify pull requests in a repository where auto-flagging and auto-releasing are both off. Turn on auto-flagging or auto-releasing for the repository. To learn how, read Turn on capabilities for a project.

The pull request is still a draft

Auto-classification classifies a pull request after you mark it ready for review. To start the classification, mark the draft pull request as ready for review.

The pull request is a cleanup pull request

Auto-classification skips the pull requests that Release Autopilot creates for automated flag cleanup. No comment is expected on these pull requests.