Introducing Release Autopilot: Automated release control for AI-built code — Read the announcement

Skip to:
BlogAI Agents
AI agents in production: What changes when real users arrive

Nov 16, 2026

AI agents in production: What changes when real users arrive

AI agents in production drift once real users arrive, even after passing pre-launch checks. Learn how to catch agent drift and roll back bad behavior fast.

Kelvin Yap
Senior Product Marketing Manager

Key takeaways

  • Passing a pre-launch check does not always keep an AI agent stable in production, because model updates, retrieval changes, and real user inputs can shift its behavior.
  • Agent drift can arrive without a deploy, so detecting it requires a scored production baseline and a trend line.
  • Managing prompts, models, and tools as runtime configuration can allow teams to roll back a misbehaving agent without a redeploy.

The same prompt can produce many different responses because agent behavior is probabilistic, so passing a pre-launch check does not mean an agent configuration will stay stable in production. Once real users introduce unfamiliar inputs, context, and usage patterns, performance can degrade in ways a test suite never captured. Without continuous evaluation and monitoring, those regressions can easily go undetected.

Production is not a milestone an AI agent clears once; it is an ongoing control problem where the real test is how quickly teams can understand and change behavior once the agent is live.

Why do AI agents fail in production?

AI agents fail in production because real users bring inputs tests never covered, models and data change underneath them, and no one is watching every step.

What separates a prototype agent from one in production is that humans are no longer watching every step, and the same input will likely produce a different output each time.

In a demo, a human catches mistakes before they matter. Once an agent starts running unattended on work others depend on, the production system needs its own mechanisms for detecting errors, enforcing permissions, maintaining state, and preserving an auditable record of what happened.

Part of that control problem is understanding where the variability originates. Teams can reduce run-to-run differences by lowering temperature, fixing a seed, or constraining outputs to a schema. But those controls cannot stabilize everything around the model because a provider can update the model, a retrieval-augmented generation (RAG) index can change, or production context can grow beyond what you initially tested.

And since AI is nondeterministic, meaning the same input can produce different outputs across runs, production behavior can shift even when the application code does not.

For example, at Hireology, a recruiting platform that uses AI across hiring workflows, shipping changes once meant waiting on a monthly deployment cycle. That made the correction path itself a constraint, since identifying a problem was only useful if the team could change the agent quickly enough to address it. So deterministic decoding can reduce variation, but it does not guarantee stable behavior in production, which depends on how quickly teams can respond when models, prompts, data, or other parts of the system change.

Teams that reach production tend to tighten constraints rather than loosen them. In the first large-scale, systematic study of AI agents in production, researchers surveyed 306 practitioners and conducted 20 in-depth interviews with agent teams. Among deployed agents, 68% ran for no more than 10 steps before needing human intervention, while 74% relied primarily on human evaluation. Among the interviewed teams, 70% relied on prompting off-the-shelf models instead of fine-tuning.

The Measuring Agents in Production study ranks its core technical performance category (reliability, robustness, and scalability) as the top challenge at 38%, against 3% for governance. The overall trend shows that production teams are prioritizing bounded autonomy, human oversight, and operational control rather than maximum agent freedom.

All of that collides with a slow correction path. When changing agent behavior requires a code change, a container rebuild and redeploy can take several minutes. On a high-traffic endpoint, every minute of that delay means more users get degraded responses before the fix reaches production. Measure that path for your own agent before you need it, because it determines how much damage a bad release does before you can undo it.

What you own once agents serve real users

Moving an agent into production changes more than its audience. Concerns that were limited, manually managed, or easy to overlook during prototyping become ongoing operational responsibilities once real users depend on the system.

Area

In a prototype

Once real users arrive

Orchestration and state

A loop runs until it looks done

Step budgets, ceiling behavior, and memory that outlives the request

Evaluation of agent output

A golden dataset passes before launch

Continuous scoring of responses users are already reading

Observability and tracing

Logs you read when something looks wrong

Every step, tool call, and intermediate state tied to a config version

Cost and latency

A number nobody owns

A per-step budget with attribution

Guardrails and governance

A pre-release review

Permissions, approvals, and audit applied to live behavior changes

Drift after launch

Not yet a category

A scored baseline and a trend line

Orchestration and state

Orchestration gets increasingly complicated when one user request triggers multiple model calls, tool calls, and decision steps. Teams must decide how long an agent is allowed to keep working, what happens when it reaches that limit, and which information should persist between interactions.

Most agents run a planning loop: the model decides the next step, calls a tool, reads the result, and decides whether to continue. Each pass costs tokens and time, which is why step budgets and a defined ceiling matter.

In practice, that means production agents need a defined ceiling on how many steps they can take before stopping or escalating. Teams also have to decide in advance what happens at that ceiling: return the best answer available, ask the user for more information, or try another approach.

Agent memory that persists beyond a single request creates additional failure modes. Retrieved information can become stale, cached tool results can outlive their usefulness, and a summarization step can accidentally drop an important constraint. Teams need rules for what information persists, how long it remains valid, and what causes it to be refreshed or discarded.

All of that information also competes for limited context-window space. Context engineering is not simply about supplying more information, but also about deciding which information the agent needs for the current task and what should be left out.

Evaluation of agent output

Agent evaluation should shift from a gate you pass to a measurement you run continuously, scoring responses that real users are already interacting with.

Automated scoring does the bulk of the work, but with human assistance. Among the 20 teams in the Measuring Agents in Production study, every one using LLM-as-a-judge also employs human verification. The lesson is not to treat the judge as an authority, but as a sampling instrument that routes low scores, near-threshold scores, and disagreements between a human and the judge to a person. That keeps a human-in-the-loop where judgment is irreplaceable without putting one on every response.

Keep in mind that a judge from the same model family tends to favor output patterns like its own, a bias researchers call self-preference. A judge from a different family will give a more independent read, but cross-family judging adds a second provider dependency with its own rate limits, per-token spend, and eval-loop latency.

Across those same 20 teams, the distinct reliability requirement is continuous correctness verification during both development and runtime. Instead, evaluation centers on whether responses are correct and high quality. For agents, availability alone is the wrong axis because the system can be fully up and still returning wrong answers.

How do you monitor AI agents in production?

A trace has to cover the whole agent run, not the request, because the answer can be right while the path to it was broken.

Request-level monitoring can tell you that an agent successfully returned a response in 4.2 seconds, but it can’t tell you whether the agent took the right steps to produce it. It can't tell you the agent chose the wrong tool twice, retried a failed lookup, and recovered with a plausible guess, a hallucination that looks like a correct answer. End-to-end tracing should capture every step, instance of tool calling, and intermediate state, with the configuration version that produced them. That gives teams a way to trace a drop in quality back to the specific part of the workflow that changed.

With LaunchDarkly LLM observability, each agent run is traced step by step and tied to the configuration variation that produced it, so you can trace a drop in quality back to the prompt, model, or tool change behind it.

For multi-agent systems, performance data belongs on the workflow itself, with latency, tool calls, and invocation counts visible for each node rather than as disconnected traces to correlate afterward. A node here is one individual agent in the workflow. Correlating traces by hand during an incident can turn a quick diagnosis into a lengthy one.

Cost and latency

In production, cost and latency become operational variables that teams need to measure and control at the step or agent level.

Latency rarely blocks deployment outright. In the Measuring Agents in Production study mentioned previously, only 14.8% of deployed-agent teams identify it as a critical blocker, while 59.3% call it suboptimal but sufficient to ship. That makes it easy for performance to degrade gradually without triggering an obvious failure. Track tokens, cost, and latency per step or per node, then set a budget at that granularity, because an agent with an acceptable average can still produce a single runaway request that costs many times the median interaction.

Guardrails and governance

Governance becomes a runtime requirement rather than a pre-release gate, especially as regulations such as the EU AI Act take effect, because the behavior being governed changes after release. Start with the inventory. The Cloud Security Alliance survey of enterprise practitioners found that 82% of organizations discovered previously unknown AI agents in their infrastructure in the past year, with 65% reporting an agent-related incident in the past 12 months. An agent that is not inventoried cannot be consistently monitored, permissioned, or governed.

Autonomy isn't a set-it-and-forget-it feature either. The same Cloud Security Alliance study found that while 53% let agents act autonomously on low-risk work while routing higher-risk actions to a person, 24% keep a human in the loop for most tasks, and only 13% report fully autonomous operation. The most useful agents seek the widest access to private data, so least-privilege scoping, sandboxed execution, and layered containment now sit in the non-negotiable category. Teams also need clear rules for who may change a prompt or swap a model in production, with an approval path and audit record for every change.

“AI agent security and governance encompass an interconnected system spanning visibility, lifecycle management, policy, and monitoring. While foundational controls are in place, gaps in consistency and end-of-life management remain. As agents gain greater autonomy, governance must evolve into a more unified, operational model that can sustain control at scale,” said Hillary Baron, AVP of Research, Cloud Security Alliance.

Drift detection after launch

Drift is the one failure mode that arrives without a deploy, which is why it needs a detection strategy rather than a release checklist.

A discrete change has a cause that can be determined, such as someone editing a prompt, a provider shipping a model update, or a retrieval index being rebuilt.

Drift has no such event, and quality erodes across hours or days as context grows or inputs deviate from the ones you evaluated against. Detecting it requires a scored baseline and a trend, not a threshold on a single response.

Establish a production baseline, then measure quality over time so you can distinguish a sustained decline from normal response-to-response variation. Tie those scores to the configuration in use so that when quality shifts, you can determine whether the cause was a configuration change or something in the surrounding production environment.

LaunchDarkly supports this through AgentControl’s monitoring and evaluation capabilities. Online evals can score sampled production responses for measures such as accuracy, relevance, and toxicity, creating quality metrics that can be tracked over time. For agent-based, multi-step configurations, LaunchDarkly also supports judge models and performance monitoring through agent graphs and the AI SDK. An agent graph maps each agent in a workflow as a node, so latency, tool calls, and scores attach to the step that produced them.

The goal is to detect a meaningful trend early enough to act on it. Once quality falls outside the acceptable range, runtime controls can limit exposure, switch configurations, or roll back a change rather than waiting for users to surface the problem first.

What production-grade agentic architecture controls

A production agentic architecture separates four things prototypes collapse together: the agent's configuration, the code that calls the model, the trace that records what happened, and the control surface that changes behavior without a deploy.

Take a travel assistant covering where to eat, where to stay, and what to do. An orchestrator agent routes each query to a specialist with its own tools, and a summarizer compiles the response. Here is what a request traverses, and where control attaches:

  1. The request enters the orchestrator. It resolves configuration at runtime rather than from the deployed bundle. Prompt, model, and tool bindings arrive as a versioned variation, one named combination you can swap or roll back as a unit, so routing changes without shipping the service.
  2. The orchestrator selects a specialist. Each node is its own configuration with its own tool bindings, often exposed through the Model Context Protocol (MCP), an open standard for connecting agents to tools and data, and each edge defines how output passes onward. A configuration can appear in multiple workflows, so an improvement to the lodging agent carries over to other workflows that use it.
  3. The specialist calls the model provider directly. Your application calls the provider with its own credentials, using the configuration the LaunchDarkly AI SDK resolved, rather than routing inference through a third party.
  4. Every step emits a trace. Latency, token cost, tool calls, and the configuration version in effect attach to the node that produced them, so the record maps the workflow instead of piling up logs to correlate.
  5. The summarizer compiles the response, and it gets scored. Those scores land on the configuration version the trace references, which is what lets a quality change be attributed to a specific decision.

A production workflow also needs controls around that path.

Put timeouts and retry limits around every model and tool call, and use idempotency keys for actions that must not run twice. Checkpoint longer workflows so they can resume after a failure without repeating completed work.

Run tools with scoped credentials in a sandbox, require human approval for high-risk actions, and define a fallback for every unavailable model or tool. Persist only the state the agent needs, with a clear retention policy, and use queues and concurrency limits to keep demand from overwhelming the system. Finally, feed Online Eval scores back into the control layer so a sustained quality decline can pause a rollout, reroute traffic, or restore the last known good configuration.

Framework choice is still unsettled. Among deployed-agent teams in the Measuring Agents in Production survey, 60.7% use third-party agentic frameworks, yet among the 20 teams studied in depth, 85% built entirely in-house with direct model API calls.

Spotify's background coding agent, Honk, shows what the in-house approach can look like at scale. Spotify built its own lightweight tooling around model calls, evaluation, tracing, and agent switching. The system has since generated more than 1,500 pull requests merged into production, according to Spotify Engineering, with a time savings of 60% to 90% on migrations.

Whichever way you build, keep configuration out of the deployment artifact, keep the trace attached to the configuration version, and keep a control surface that can change behavior without a build.

How runtime control changes prompts, models, and tools

LaunchDarkly is the runtime control layer for agents in production. AgentControl is the part of it that governs agent behavior directly. Prompts, models, and tool configurations are managed at runtime, so changes typically propagate in under 200ms without touching CI/CD.

Because applications resolve configuration at runtime, an updated configuration can take effect on the next evaluation within an ongoing conversation. That speed comes from the delivery network underneath, the same one that serves more than 70 trillion flag evaluations daily.

The lifecycle around that control surface runs in both directions:

  • Offline Evals benchmark prompt and model variants against curated datasets before anything ships. With AgentControl, you can now run offline evaluations from a script or a CI job.
  • Experiments compare agent variants on live traffic and measure which performs better on the metrics you choose.
  • Guarded rollouts release a new agent version progressively, with automatic rollback on quality, cost, or latency signals.
  • Online Evals score production responses continuously, with customizable judges added alongside built-in rubrics for accuracy, relevance, and toxicity.
  • Adaptive Triggers act on what the rest of that lifecycle measures.

Adaptive Triggers turn monitoring into runtime action. A team defines the condition and response in advance, so when quality scores dip below a threshold, the system can escalate to a more capable configuration, and when a provider's error rates breach a limit, it can switch to another. Adaptive triggers can be configured to automatically escalate without human intervention based on the thresholds your team sets.

For agent-mode variations, you can attach custom judges or invoke judges programmatically through the AI SDK and record the resulting evaluation metrics. Operational metrics such as latency, token usage, cost, and evaluation scores can be monitored for both completion and agent modes.

This builds on the progression LaunchDarkly ran for code, where guarded releases arrived in 2024 to tie releases to critical metrics; guarded rollouts apply that mechanism to agent configurations.

Zack Rossman, former Senior Staff Engineer at Veeam, put the operational effect this way: "To deliver best-in-class AI agents to our customers, we need to keep pace with the latest frontier models. AgentControl lets us systematically test and upgrade our agents in production without waiting on a full deployment cycle."

As part of incident planning, inventory every prompt, model, parameter, tool, and routing decision that responders may need to change without a new build.

What to do the moment an agent starts misbehaving

The first decision in an agent incident is containment. Stop the exposure while you diagnose the cause. That ordering isn't intuitive for engineers trained on deterministic systems, but while you read traces, users keep receiving output you already know is wrong.

How you contain it depends on whether a change caused the problem or the agent started misbehaving without anyone touching it.

When a change caused it

  1. Pause the rollout. If the bad behavior arrived with a new variation, stop its advance before it reaches more users. Pausing is low-cost even if you’re wrong about the cause. If the behavior is severe, use a kill switch to turn the configuration off for everyone at once.
  2. Reroute traffic. Use targeting rules to send affected cohorts to a configuration you trust, such as a more capable model, a narrower prompt, or a conservative fallback. This buys diagnosis time without taking the feature away.
  3. Restore the last known good variation. When degradation is broad rather than cohort-specific, revert to what was working. Because these are configuration changes rather than deployments, all three actions typically take effect in milliseconds without a redeploy.

When nothing changed

A provider outage, a model update, or drifting inputs can degrade an agent without a deploy. When that happens, there’s no rollout to stop, and the first sign may come when no one is watching. Adaptive triggers set the response in advance. When a signal crosses a threshold you set, LaunchDarkly can automatically switch the config to a fallback variation and alerts your team.

  1. Choose the signal. Track errors for provider outages, an evaluation score such as accuracy for quality drift, or other metrics, logs, traces, and events.
  2. Set the threshold. Use a fixed number, or an anomaly threshold that fires when the value moves significantly away from its baseline.
  3. Pick the fallback. Switch to another model provider, a cheaper model, or any variation you choose. Built-in templates cover the two most common cases: switching providers when errors appear, and switching variations when accuracy drops.
  4. Route the alert. Send notifications to the team members or Slack channels you pick. The alert also appears on the Alerts page.

Adaptive triggers can bypass approvals, so choose a fallback your team would approve anyway.

Then diagnose and verify

  1. Diagnose from the trace, not the output. Work backward from the traces to the configuration version, then to the prompt, model, or tool binding that changed. Scores and traces reference the same version, which is what makes attribution possible. If a trigger fired, its alert shows the signal over time.
  2. Verify before resuming. Confirm the corrected configuration on a limited cohort before widening it again. If a trigger switched to a fallback, confirm the provider or model has recovered before switching back.

How quickly those changes reach applications depends on how the SDK receives updates. LaunchDarkly server-side SDKs use streaming connections by default, enabling near-immediate updates, and polling configurations receive changes on their next polling cycle.

Marek Poliks, Head of AI at LaunchDarkly, describes the timescale this collapses to: "We can remediate an agent that's misbehaving in production—live, in just milliseconds—without a customer ever knowing the problem even occurred."

Agent failures that aren't contained can become public, with sometimes devastating consequences. For example, a coding agent deleted a live company database against instructions, then misreported what it had done. Gartner expects the pattern to decide adoption, predicting that by the end of 2027, 40% of agentic AI projects will be cancelled, because of escalating costs, unclear business value, or inadequate risk controls.

Containment has a tail end most teams skip, with only 21% holding a formal decommissioning process, according to the Cloud Security Alliance. Put a retirement step in the same runbook, and treat revoking credentials as the final action of any incident that ends in shutting an agent down.

How fast you respond is the metric that matters

The teams that run agents well aren't the ones with the best pre-launch evaluation suites; they're the ones who can change agent behavior faster than it degrades. That shifts the engineering question from whether an agent is ready to ship toward how quickly its behavior can be corrected once it's live, which is a question with a number attached rather than a judgment call.

Take one agent you already run and time the path from noticing bad output to changing what users receive. If that path runs through a pull request, a build, and a deploy, that duration is your real exposure window, and it's the first number to reduce.

FAQs

What is LLM-as-a-judge for AI agents?

LLM-as-a-judge is an evaluation method where a separate language model scores an agent's responses against a rubric, such as accuracy, relevance, or toxicity. It scales review across production traffic, but it works best as a sampling tool that sends low, borderline, or disputed scores to a human reviewer, not as a final authority.

What is the difference between offline and online evals?

Offline evals test prompt and model variants against curated datasets before a change ships, while online evals score sampled responses from real production traffic after it ships. Teams need both: offline evals catch regressions you can predict, and online evals catch problems that only appear once real users send unfamiliar inputs.

How is monitoring AI agents different from traditional application monitoring?

Traditional monitoring checks whether a service is up and fast, while AI agent monitoring checks whether its answers are correct and how it reached them. An agent can return a response in seconds with full availability and still be wrong, so teams trace every step, tool call, and intermediate state alongside quality scores.

How do you decommission an AI agent?

A decommissioning process should remove the agent from production traffic, revoke its credentials and tool access, and retire its configurations so it cannot be reactivated by accident. Record the agent in an inventory first, because an agent no one has inventoried cannot be reliably shut down.

Flaming pointer icon
Get started in minutes.

Learn how to scale seamlessly with LaunchDarkly as your project grows.

Letter in envelope icon
Sign up for our newsletter.

Get all the content, tips, and news you can use.

By supplying my contact information, I authorize LaunchDarkly to contact me with personalized marketing communications about our products and services. See our Privacy Policy for more details, or Opt-Out at any time.