BlogRight arrowRisk Mitigation
Right arrowThe real reason your releases feel like fire drills—and how to change this
Backspace icon
Search iconClose icon

JUN 09 2025

The real reason your releases feel like fire drills—and how to change this

How innovative teams are transforming their release process.

The real reason your releases feel like fire drills—and how to change this featured image

Sign up for our newsletter

Get tips and best practices on feature management, developing great AI apps, running smart experiments, and more.

Subscribe
Subscribe

Imagine this: It's 2 PM on a Thursday. Your phone buzzes with an urgent alert. A critical API route change has triggered cascading failures across the platform. Customer complaints are flooding in. Systems you're responsible for are down.

The next hour unfolds like a familiar, painful ritual:

  • Engineers pulled from important work into an emergency war room
  • Domain experts identify the issue within minutes
  • ...and then the waiting begins
  • 25 minutes for PR review
  • 30 minutes for CI/CD pipeline
  • 20 minutes for deployment
  • 15 minutes for verification

Three hours later, the issue is finally resolved with a fix that took 2 minutes to identify, but nearly 50 engineer-hours to implement.

Now imagine a different scenario. Same alert. Same issue identified. But this time, the operations team toggles a feature flag and within 45 seconds, service is restored.

This isn't fantasy. It's what organizations do every day with LaunchDarkly. And we at LaunchDarkly have also benefited from our own product. I’ll show you how. 

The hidden costs of your current approach

We've grown accustomed to a model where deployment and release are inseparable. Where every change, no matter how small, must follow the full pipeline. This approach is slowing us down in ways that extend far beyond incident response:

  1. Code freezes during critical periods. We halt innovation precisely when it could deliver the most value.
  2. Deployment anxiety. Engineers release less frequently because each deployment carries such risk.
  3. Domain expert dependency. Subject matter experts can identify issues but can't directly address them.
  4. All-or-nothing releases. We can't target specific customer segments or gradually roll out features.

The most insidious cost isn't measured in time or dollars; it's measured in missed opportunities and eroding confidence. Every hour spent in a war room is an hour not spent building the future. Every delayed release is customer value deferred.

A fundamental shift: Beyond configuration-as-code

Configuration-as-code has provided tremendous benefits: version control, code review, and consistent environments. But it was designed for defining structure, not controlling behavior at runtime.

Think of your software system like a modern building:

  • Configuration-as-code is the architectural blueprint, thoughtfully designed, carefully reviewed.
  • Runtime control is the building management system, with immediate adjustments based on current conditions.

When there's a fire in a building, you don't file a request with the architect to redesign the sprinkler system. You pull the alarm that was thoughtfully placed there during construction.

LaunchDarkly gives us runtime controls for our software: the kill switches, circuit breakers, and targeted controls that empower us to respond instantly to changing conditions.

From theory to reality: How others are transforming

Companies similar to ours are already revolutionizing how they work. Let's look at three compelling examples:

Case Study: World Kinect Corporation

Key Highlights:

• Increased releases by 400%, to 640 per month, by standardizing on LaunchDarkly across teams.

• Moved to trunk-based development, improving integration speed and reducing overhead from long-lived branches.

• Rolled out A/B testing and progressive delivery, allowing for safer, controlled experimentation.

“We needed to modernize the way we delivered software to our customers, and the first step was accelerating our deployments safely.”

— Adam Setch, VP of Architecture, World Kinect

Case Study: Coles Digital

Key Highlights:

• Shifted from monthly deployments to multiple per week, aiming for daily deploys with feature flag controls.

• Eliminated coordination meetings—teams now release when ready without cross-team bottlenecks.

• Integrated observability tools (like Sentry) to toggle off failing features instantly, reducing customer response times.

“We got rid of all of those expensive meetings, no coordination meetings, no one’s saying what’s going into a release.”

— Patrik Mihailescu, Senior Engineering Manager, Coles Digital

Case Study: SPS Commerce

Key Highlights:

• Enabled continuous deployment with feature flags, decoupling deploy from release.

• Gave developers confidence to ship anytime—deployments became “more relaxing”, not risky.

These aren't isolated success stories—they represent a fundamental shift in how modern software is delivered. 

How we used LaunchDarkly to enable 3 transformations at LaunchDarkly

LaunchDarkly enables three fundamental transformations for our organization:

  1. Deployment and release transformation.
    We can deploy continuously and release features when they’re ready. This breaks the cycle of infrequent, high-risk deployments and enables true CI/CD.
  2. Incident response transformation.
    We can reduce MTTR from hours to seconds with immediate runtime control. This eliminates the need for war rooms and allows us to fix issues on our schedule, not during emergencies.
  3. Organizational transformation.
    We can empower domain experts to control their features without bottlenecks. This creates a more responsive organization where the right people have the right controls.

Imagine telling this story at your next all-hands: "It's been three months since our last code red. Our domain experts now control their own features. Our engineers are shipping more code with less stress than ever before."

Starting the journey: a practical path forward

This transformation doesn't require a "big bang" approach. We can start small, prove the value of a change, and scale methodically:

Phase 1: Kill Switches (2 weeks)

  • Identify the top 5 recent code red triggers
  • Implement emergency kill switches for each
  • Train the operations team on response procedures

Phase 2: Progressive Delivery (1 month)

  • Integrate with our SDLC process
  • Implement automatic rollback monitoring
  • Enable trunk-based development with feature flags

Phase 3: Domain Expert Empowerment (3 months)

  • Create self-service provisioning templates
  • Train product and domain experts on flag management
  • Develop playbooks for common scenarios

The call to action: From reactive to proactive

Today, I'm asking each of you to make a small commitment that will spark a larger transformation:

  1. Identify the one recurring incident in your area that causes the most pain.
  2. Commit to spending one hour next week learning how LaunchDarkly could address it.

That's it. One pain point, one hour.

From that simple beginning, we'll build a new reality where:

  • Code freezes become unnecessary
  • Deployments happen daily without fear
  • Domain experts control their own destiny
  • Engineers focus on building, not firefighting
  • Incidents are resolved in seconds, not hours

This isn't simply adding another tool to your stack. It's fundamentally changing how you deliver software, shifting from reactive to proactive, from fearful to confident, from disconnected to empowered.

The technology exists, and the path is clear. All that's missing is your decision to take the first step.

Like what you read?
Get a demo
Previous
Next