Every so often, the software industry hits a stretch when, as Jonathan Nolen puts it, “The smartest minds in our industry are thinking not about what we’re building, but how we’re building.” Nolen, Chief Product Officer at LaunchDarkly, has lived through a few of these stretches. On a recent episode of the AWS for Software Companies Podcast, he talked about what this one asks of teams as they build software factories around agents that write code faster than people can read it.
Here are five takeaways for keeping agent-led development fast and safe. The full episode has even more, including why Nolen is ready to retire the phrase “vibe coding.” Give it a listen.
1. Validate at agent speed, before and after release.
“If you’re going to be writing code at agent speed, you have to be able to validate that code at agent speed,” Nolen says. He sees two levers for doing it. “The first lever is investing more in testing, so we have more certainty before it sees production. The other lever is reducing the risk, so that when it sees production you can contain any problem incredibly quickly.”
In practice, he says, the second lever involves controlling how a change reaches customers. “If you can take code that an agent has written, roll it to production, show it to an internal audience, then to synthetic tests, then to a beta cohort … and monitor it at every step, then you can afford to ship more quickly.”
2. Measure valuable software per unit of human attention.
LaunchDarkly judges its own software factory, and the factories its customers are building, by one question: “How much valuable software can you ship per unit of human attention?” The reason, Nolen says, is that “human attention is the part that does not scale. So we have to figure out how to automate as much as we can, recognize when humans are actually needed for judgment, and then bring in the human at that point.”
3. Start your software factory with bug fixes.
According to Nolen, the customers charting the course are building software factories with LaunchDarkly, “and not just LaunchDarkly, of course.” He points to Ramp as “a great role model” for the rest of the industry. The company has published openly about its own factory, where agents write and validate code and tools like LaunchDarkly control it in production. And at one large insurance company, he says the factory is “up and running” on three tools: its deployment platform, its observability tooling, and LaunchDarkly.
He’s clear that no one starts at full scale, noting, “Everybody starts with low-risk things. That’s what we’re doing, as well. We proved the software factory on bug fixes.” From there, teams move up to features, then projects.
Eventually, he says, “We will have humans driving an army of agents creating valuable software change for their customers at a rate that we have not experienced in our entire careers.”
4. Ask two questions about your release process.
When asked what software leaders should consider before adopting a service like LaunchDarkly, Nolen said he’d start with two questions: “When there is a problem with something that you have released, how quickly can you change it? And … at what granularity can you change it?”
If fixing one problem means rolling back a whole release, he says, “You’re holding every team and every feature captive to the slowest and least reliable feature.” With runtime control, you can “turn it off with a push of a button, and that change is propagated to your fleet in less than a second.”
5. Learn by building.
For decades, product teams planned around the cost and time of writing software. Now, Nolen says, that’s “essentially no longer a constraint,” and it changes how teams decide what to build. “We can just build the thing, put it in front of people, put it in production, measure it, and figure out whether it’s working or not.”
The loop gets faster when agents can act on what they learn in production. Nolen describes a next step where those signals “get fed back into the system so the agents can begin to fix their own mistakes or begin to optimize towards a desired outcome.”
Over the next couple of years, he expects more organizations to start running that build-and-measure loop. “The ones who do it faster are going to see compounding benefits, where they are both making better product decisions and shipping them more quickly than their competitors.”
Listen to the full episode, and book a meeting with us at AWS re:Invent. LaunchDarkly is also available on AWS Marketplace.
