What is an AI software factory?

An AI software factory is a repeatable development system that moves work from specification through testing and release, and keeps control of each change once it reaches production. Agents pick up tasks, build in isolated environments, and check one another's work at each stage. Engineers remain in the loop by defining what the software should do, setting the checks it must pass, and having the final say on what reaches users. Agents handle most of the implementation, but the engineering team is still responsible for what ships. An AI software factory is different from NVIDIA’s AI factory, which refers to the compute infrastructure used to build and run AI.

Engineering responsibility cannot stop at release. A factory contains two kinds of agents: builder agents, which build the product, and product agents, which run inside it. Builder agents produce code changes that engineers can review and gate before release. Product agents produce behavior that can change without a commit when a prompt, model, tool, or data source changes.

So a factory has two kinds of change, and both need the same kind of control: in production, at the change level, without a redeploy. The rest of this article covers how that control works for each.

How does a software factory move work from specification to release?

For builder agents, the difference between one-off coding assistance and a software factory is the repeatable process that carries work from specification through review and release.

A coding assistant can help you finish a single task. A software factory moves that task through a repeatable workflow with checks and documented evidence between each stage. An agent might:

1. Write the code in an isolated environment
2. Run tests
3. Diagnose failures
4. Open a pull request
5. Hand off outputs to the next stage for verification

Those checks must confirm that the software behaves as intended, not only that it looks right.

A menu, for example, can render correctly and still send the wrong request when you select an option. The LaunchDarkly approach to the AI software factory calls for testing that behavior, down to the API parameters, as well as checking what appears on screen. Your team has to define the expected result before an agent can test for it.

The expected result, combined with the evidence that the agent met it, should follow the code into the review process. So keep the requirements and test results alongside the code change, or “diff.” If a reviewer has to dig through an agent's chat history to figure out what it changed and why, you've handed them another job before they can even start reviewing the code.

Many descriptions of an AI software factory stop at release: once a change passes review and is released, production results feed back into the next build. That leaves out the step where most of the risk now lives. Code that passed every check can still break in production, and an agent's behavior can keep changing after release.

Why does code review become the bottleneck in an AI software factory?

Coding agents often produce changes faster than reviewers can assess them. Faros’s 2026 AI Engineering Report, which tracked 22,000 developers across more than 4,000 teams over two years, shows what that does to review. As teams moved from low to high AI adoption, average pull request size grew 51%, and the time a pull request spent in review nearly tripled on average. At the median, it grew more than fivefold. Merges didn’t keep pace. Pull requests merged per developer rose only 16%, and Faros’s most likely explanation is a review bottleneck that’s throttling throughput.

The pressure doesn’t stay in the review queue. Bugs per developer rose 54%, incidents per merged pull request more than tripled, and 31% more pull requests merged without any review. Strong engineering foundations didn’t protect teams, either. Organizations with high pre-AI DevOps Research and Assessment (DORA) scores saw the same decline as everyone else.

The constraint is no longer code velocity. It’s control that lets teams move quickly without sacrificing safety. More pressure on review doesn't mean removing humans from it. Stripe’s minions show how human oversight can remain part of a high-volume agent workflow. In its February 2026 account, Stripe reported more than 1,000 agent-produced pull requests reviewed by humans and merged each week.

Reviews, then, must become more efficient to scale an agent workflow. Before adding more agents to the queue, make each change easier to judge by showing reviewers which requirement a change satisfies and which test or measured result supports it. For a high-risk change, agree on the release criteria before it gets to them.

Review is only half of it. Rolling a change out gradually, watching it, and pulling it back when something breaks are still the right steps. At AI volume, doing them by hand for every change is the step that doesn't scale. And because adding control means adding work, the control often doesn't get added.

How do builder agents differ from product agents?

When an agent writes a feature, its code goes into a repository and through your CI/CD pipeline, which builds, tests, and deploys it. Deploying that code puts it in an environment. Releasing it determines which users experience the feature, so you can deploy a change and still choose when to expose it.

An agent inside your product has more ways to change. It can change when:

• Someone edits an externally managed prompt
• A provider updates a model alias
• A retrieval source starts returning different information
• Tool permissions change what the agent is allowed to do

Any of these actions can impact the agent’s behavior while the deployed code stays the same. For years, you could assume behavior only changed when code changed. That stops holding once the models, prompts, and context a system depends on keep shifting underneath it.

Pinning versions and managing configuration in code can help reduce this uncertainty, but won’t completely eliminate it. External dependencies can still vary after pre-release testing and affect quality, cost, latency, or safety.

Builder agents and product agents can also operate within the same system. An agent-built feature, for instance, may contain an agent whose prompts and model settings change independently. That’s why the factory needs control over both the code it releases and the live configuration the product agent uses.

How do you control exposure of agent-written code?

The point of runtime control is that it can act on its own once you've set the thresholds. The rollout advances or pulls back on your metrics without waiting on a human, unless you want a human gate at that step. Setting it up takes a few decisions. Passing tests give you a reason to begin a release. You still get to decide how many users should see it while you learn how it behaves in production. Start with an internal group or a small share of users, and agree on what would make you expand or stop the rollout. Choose the metrics, the amount of data you'll need, and the variation you'd restore if something regresses.

At LaunchDarkly, we used this approach during our flag-targeting UI rewrite, which started with roughly 66,000 lines of React. Two senior engineers tried an autonomous one-shot approach, then abandoned it and worked through the rewrite in human-guided phases. The planned 22 phases grew to 34, and the team spent roughly $7,000 of a $10,000 inference budget.

The rewrite stayed behind flags, so we could use it internally before releasing it to customers. Customer rollout was still ahead when we published the account. Generating the code was only part of the work; the team still had to check it and decide who should see it.

LaunchDarkly helps you control exposure of flag-wrapped code. You can target a group of users and expand exposure as you gain confidence. If your plan includes guarded rollouts, LaunchDarkly can monitor selected metrics and roll the release back automatically when it detects a regression, with the option to notify you instead. You configure that response once, in advance. Reliable detection needs enough data, so each rollout step serves a minimum number of users before it advances. And because the variations are already deployed, switching which one the flag serves changes exposure without another deployment.

Once the rollout completes, the flags come out: on plans that include code references, they show where each flag still lives in the codebase, and flag archiving keeps temporary release flags from piling up as agents create more of them.

How do you control an AI agent's behavior at runtime?

Suppose a provider updates the model behind an alias your agent uses, and answer quality drops. Your application code hasn't changed, and the tests you ran before release only describe the earlier model's behavior. Without live measurement, the first signal you get is often a customer complaint.

Keep versioned variations of the prompts and model settings your application uses, along with versioned definitions for its tools. That gives you something specific to target to a small audience, then measure and change. Have an alternative ready if the results get worse. Restoring an earlier variation that points to the same updated model alias will still call the updated model.

With AgentControl, LaunchDarkly manages and serves prompts, instructions, and model settings separately from your application code. Your application still calls the model provider with its own credentials, runs the agent, and invokes the tools attached to its configuration.

You can also use online evaluations to score live responses asynchronously at a sampling rate you configure. With traces instrumented and metrics recorded, you can see which variation was in use when results changed, tie the quality shift to the configuration change behind it, and verify the link by rolling that variation back. To compare two variations head to head, run an experiment: LaunchDarkly Experimentation splits traffic between variations of your code or agent configuration and measures the difference against the metrics you choose.

The same guarded rollouts apply to AgentControl configurations. Choose the metrics to monitor, and decide whether a regression rolls the change back automatically or notifies you. Observability tells you the behavior changed. Runtime control lets you change the configuration the application receives, without a redeploy.

Production evidence can also inform future optimization cycles. Agent optimization can generate a new variation and report its scores, latency, token usage, and estimated cost. From there, you can decide whether to promote that variation into another controlled rollout.

Checking your software factory's controls

Before giving agents more autonomy, follow one code change and one agent-configuration change through your factory. For each, ask:

• What shows that the change did what we asked it to do?
• How do we limit exposure while we gather production data?
• Which metric would make us stop or reverse the change?
• Who makes that call, and what can they restore?
• Who can approve a change to a prompt or model, and where is that change recorded?

You build your software factory your own way, with your own agents and pipeline. LaunchDarkly is the runtime control layer for the AI era. It controls code exposure and live agent configuration in production, without a redeploy, and it acts on its own wherever you let it. Move at AI speed. Stay in control.