This post is part of an ongoing series on how LaunchDarkly engineers are closing the loop of the AI SDLC—and what we're learning along the way.
At LaunchDarkly, we face the same challenge many engineering teams do: going faster without losing control of what reaches customers. That’s why we’re building an AI software factory with LaunchDarkly primitives, and we’re using what we’ve learned to help customers build their own. I decided to push it further by turning my personal side project into a real-world testbed for our internal factory implementation.
Over a few weeks of near-daily feature work, this software factory has created and wired 21 flags for me, and it's changed how I ship.
The app in 90 seconds
The app is an AI baseball analytics tool. You can chat directly with real data, generate structured reports and team reviews, run player analyses, replay games pitch-by-pitch, and use a pitch sequencing tool that answers questions like, "What sequence of pitches should a left-handed pitcher throw to a right-handed batter to induce a ground ball?"
Under the hood, it's a small Docker Compose stack: a FastAPI backend talking to Postgres and Claude (and optionally GPT) over an MCP Postgres server, and a single-page frontend. The data includes Statcast pitch-level data, Retrosheet game logs, Lahman historical stats, and my own Out of the Park simulation exports.
The same stack runs in three places: my laptop, a NAS at home, and a public DigitalOcean VPS with HTTPS and Google login.
How flags are used
The app uses 50 flags for four distinct jobs: feature gates and kill switches, access and data control, runtime behavior configuration, and UI adjustments.
Here are a few representative examples:
enable-bulk-data-managementgates the destructive "flush all" and bulk-delete endpoints; when it's off, those endpoints return 404, and the UI controls disappear.require-loginturns Google auth on or off for the whole site.classic-sidebar-layoutis a full-layout escape hatch. Several string flags override the model's system prompts for each mode (chat, reports, team reviews) so I can adjust model behavior without a redeploy.
Tangible benefits
The clearest wins so far have come from real incidents.
The most dramatic: I did a sweeping redesign that removed the sidebar and moved every tool to the home page. The factory had wrapped it in a classic-sidebar-layout flag. When the new layout shipped with a nasty blank-page bug, rolling back was a single flag flip—no revert, no redeploy. On a public app with real users, that's the difference between "annoying" and "incident."
The factory also quietly handled things I would have forgotten. The bulk-delete and "flush all" features are exactly the kind of destructive operations you don't want live by default on a shared instance. The factory gated them at PR time before I had to think about it. The same pattern held for Google auth and the registration allowlist—both shipped off, then flipped on when seeded. This reduced the risk of the public VPS accepting unintended access during rollout or accidentally locking me out. And because the factory authored the metric events on features like require-login, turning them on came with success and error counters attached from Day 1.
The consistency also compounds over time. The flag, the wiring, the metrics, and the tests arrive together with the PR. For a solo project, that's a real multiplier; for a team, it's consistency you don't have to enforce by hand. And an in-app SDK Status page automatically badges and explains every factory-tagged flag, so I can always distinguish between the factory-authored ones and those I wrote by hand.
Gotchas
Dogfooding means finding the sharp edges.
Dark by default cuts both ways. The factory ships flags off, which is correct for guarded release—but it means after merging, I have to remember to flip the flag on to actually use the feature I just built. A couple of times I deployed and wondered why my feature had "vanished." It was working exactly as designed, just gated. Now it's a habit: Merge, then flip on.
A gated feature can also break an existing flow, not just hide a new one. My most recent feature moved team review generation to a background job. The factory gated it dark by default, as it should have, but my frontend had already swapped the Generate button to call only the new background endpoint. With the flag off, the button hit a 404. The fix was on me: Make the client honor both flag states cleanly, which the flag's own description had already implied. When a new code path replaces the old one, the flag has to switch cleanly between them, not just guard the new arrival.
A smaller thing: A flag that exists in LaunchDarkly but hasn't been wired in the code yet will surface as a mismatch—both sides have to match. This is nonblocking, but it’s worth being aware of.
None of these are dealbreakers. They're the normal texture of an automated release system, and mostly they've been teaching me good guarded release hygiene.
Takeaway
The factory turns "I should really put that behind a flag" into something that is designed to happen on every PR, complete with metrics and tests. On this app, it's produced 19 feature kill switches, saved me a real rollback during a botched redesign, and helped me control access as public deployment expanded from just me to anyone at LaunchDarkly.
Join the waitlist for early access to LaunchDarkly tools for the AI software factory.
