Introducing Release Autopilot: Automated release control for AI-built code — Read the announcement

Skip to:
BlogExperimentation
Automate experimentation with LaunchDarkly, AWS DevOps Agent, and Kiro

Nov 17, 2026

Automate experimentation with LaunchDarkly, AWS DevOps Agent, and Kiro

With AWS and LaunchDarkly, AI agents plan, safely ship, and measure experiments against goals your team defines.

AI has collapsed the cost of writing software. The volume of change heading toward production is already higher than most teams can review or test before it ships, and experimentation is one of the first things to fall behind. That’s because each experiment carries overhead. Someone has to define the feature flag strategy, wire up the implementation, configure the metrics, watch the rollout, and interpret the results. When a metric regresses, someone has to notice and act. And without a clear record of which change caused which outcome, the next hypothesis is a guess.

So teams run a handful of experiments a quarter when the business could benefit from dozens. The learning loop that should compound stalls instead.

That's changing. In September, AWS and LaunchDarkly published a reference architecture that closes the gap between defining a goal and reaching it. A team states a business outcome, such as "increase add-to-cart rate by 10%," and AI agents handle the rest: planning the experiment, implementing the change, shipping it safely, measuring the impact, and feeding what they learn into the next attempt. Humans set the goal and the guardrails, with approvals at any step you choose. 

You can try it today through the AWS Marketplace.

How the loop works

The architecture connects three layers, each with a distinct job:

  • Kiro turns goals into specs and specs into production-ready code, adding feature flags to every change that needs one. 
  • AWS DevOps Agent orchestrates the workflow and reviews every update for release readiness before it merges, acting as an always-on quality gate.
  • LaunchDarkly is the runtime control layer that closes the loop. Agent-generated changes ship progressively behind a feature flag, success metrics determine whether they scale or roll back automatically, and Guarded Releases helps ensure no single iteration reaches full production without validation.

The cycle runs in three phases. In the Plan phase, the agent generates a hypothesis informed by prior outcomes, creates a feature flag, implements the change, and validates it through a release readiness review before merge. In the Prove phase, an experiment tests the change on a small slice of traffic until the data picks a winner, and a Guarded Release then ramps the winning variation while LaunchDarkly monitors operational guardrails like error rate. In the Iterate phase, the agent records the outcome and feeds it into the next hypothesis, so the system never loses context between cycles.

The layers are complementary. AWS DevOps Agent raises the quality of what ships, while LaunchDarkly controls the exposure of what's released through progressive rollouts, metric-driven monitoring, and automatic rollback when a guardrail is breached.

What happens when something goes wrong?

Letting agents ship code is only as safe as your ability to turn individual changes off. The reason this architecture works for production, and the reason we built it with AWS, is that control is enforced at runtime, where the change actually lives.

Every experiment ships behind a feature flag from the start, so anything can be turned off without a redeploy. Every change passes a release readiness review before merge. Every rollout is a Guarded Release, which means LaunchDarkly watches operational guardrails like error rate in real time and reverts the flag automatically the moment a regression appears. No alert triage, no waiting on a human to notice, no 2 AM page.

The constraints are yours to set. You decide which metrics gate a rollout, which repositories and branches agents can touch, and where human approval is required. The agent retries failed validations a few times, then stops and asks for help rather than pushing through. If you want a person reviewing every merge, the system supports that. As trust builds, you automate more.

What it looked like in practice

The AWS team ran the reference implementation against a demo storefront with a goal of lifting add-to-cart conversion. Two moments from that run are worth the attention of anyone evaluating this model.

First, the safety net fired. During one rollout ramp, an error-rate spike crossed the guardrail threshold. LaunchDarkly detected the regression, halted the rollout, and reverted the flag to its prior state within seconds. No human intervened, and no redeployment ran. The agent recorded the rollback, adjusted its approach, and proposed a revised implementation on the next cycle.

Second, the loop converged. That revised attempt passed review, won its experiment, and rolled out to full traffic with no regressions, lifting add-to-cart conversion from 20.1% to 37.9% in the demo environment. The two earlier cosmetic experiments matter, too. Both came back inconclusive, and the system recorded them that way instead of shipping changes that didn't move the metric. 

The same runtime control applies to AI features as well as UI experiments. Poka, a connected worker platform serving more than one million frontline workers, builds its AI on Amazon Bedrock and runs it through LaunchDarkly. It decides per customer, at runtime, which model, prompt, and parameters each request uses, with progressive rollouts and automatic rollback if metrics drop. Since launch, Poka has handled more than 60,000 AI conversions and 50,000 agent conversations.

Why this matters now

This is the gap AI opened. The old solution—slowing down and adding more gates before deploy—costs you the velocity AI was supposed to deliver. The other old solution—shipping fast and firefighting in production—costs you incidents and trust.

The teams winning with AI are increasingly taking a third path: control that lives in production. Put the changes that need control behind feature flags, control their exposure with precision, observe performance at the change level, and let the system act automatically when something goes wrong. After that control layer exists, agents can safely run more of the loop, from building and releasing to observing and iterating, while engineers set goals and define guardrails instead of watching releases.

AWS gives teams the stack to build and run software and AI at production scale. LaunchDarkly is the runtime control layer that makes agent-driven development safer, so teams can ship everything AWS makes possible without more incidents. It’s built to sit in your production path: More than 70 trillion flags evaluated daily, backed by a 99.99% uptime SLA. The experimentation architecture is one concrete example of what that combination looks like when the loop runs end to end.

Get started

The reference architecture is available to implement today, and the technical walkthrough, including the full orchestration skill, is on the AWS DevOps Blog.

The fastest way to try it is through AWS Marketplace. Start a free 30-day trial of LaunchDarkly, pay as you go with billing consolidated through AWS, and draw down your existing AWS spend commitments. Experiments are available on every LaunchDarkly plan, so you can run the loop manually from day one and add Guarded Releases when you're ready to automate the safety net.

Flaming pointer icon
Get started in minutes.

Learn how to scale seamlessly with LaunchDarkly as your project grows.

Letter in envelope icon
Sign up for our newsletter.

Get all the content, tips, and news you can use.

By supplying my contact information, I authorize LaunchDarkly to contact me with personalized marketing communications about our products and services. See our Privacy Policy for more details, or Opt-Out at any time.