Reconciling change management and continuous delivery.

Feature management is a class of modern software delivery tools and techniques anchored in feature flags. LaunchDarkly is the industry’s first feature management platform.

Manjari Ranganathan
IT Manager, Hawaiian Airlines

Three core capabilities underpin feature management.

1. Decouple deployments from releases

In basic terms, “deploy” refers to moving a code artifact into a production environment. “Release” refers to making that code available to external audiences. Historically, businesses have been unable to sever deploy from release. Thus, whenever they deployed code, they had no choice but to release it to all users at the same time. The risks of such an arrangement are high.

Feature flags, on the other hand, let you separate code deployments from feature releases. This enables developers to deploy without fear. Moreover, it allays some of the concerns IT operators have about code being shipped to production frequently. And it lets product managers and release managers control the pace at which customers receive new features.

Of note, in its list of 34 practices supporting the “service value chain”, ITIL 4 puts “deployment management” and “release management” into separate categories

Associate VP of Engineering and DevOps
Fortune 500 health insurance company

2. Instantly roll back with kill switches

Another chief benefit of feature flags is they allow you to disable broken code at runtime. That is, you can remotely turn a feature on or off in production without having to push code through your deployment pipeline—or change approval process. When an incident occurs, you can toggle the flag associated with the error and fix the problem in milliseconds. The stress, engineering costs, and bad customer experience all but evaporate.

An Associate VP of Engineering and DevOps at a Fortune 500 health insurance company declared that such “fail-backs” with LaunchDarkly have been a “lifesaver” for his team.

3. Employ fine-grained targeting

Feature management supports targeting across discrete users, environments, geographies, customer sites, and other endpoints. You can create segments based on a host of custom attributes. What’s more, you can leverage percentage rollouts and targeting in tandem. Such capabilities give you tremendous control over how, when, and to whom you ship features. In keeping with ITIL 4, they allow you to progressively deliver new functionality

LaunchDarkly customers have used the three core capabilities of feature management to improve their software delivery performance:

• Deployment and release frequency
  Northern Trust went from 2-3 deployments per month to 21 deployments.
• Mean time to restore service (MTTR)
  It used to take Paramount up to a week to fix bugs, but now it only takes a day.
• Change fail rate
  LaunchDarkly helps Honeycomb maintain a 0.1% change fail rate.

Feature management provides the safeguards to decentralize software change approvals

With feature flags in the picture, IT operators and change managers can feel confident about designating more software changes as “standard”. In the ITIL model, the development team is permitted to approve “standard” changes through peer review. Delegating approvals benefits everyone. Developers avoid delays. Operators get peace of mind about risk mitigation. And relations between the two improve.

Let’s illustrate how this might look in practice.

Imagine you have a developer who wants to deploy a feature to production. First, they wrap the code in a flag, thus barring users from activating the code pathways. This represents the first layer of risk mitigation.

If for whatever reason the feature causes a bug, one person—a developer, digital product owner, IT service operator, SRE, etc.—can turn it off immediately. No prolonged rollback. No fix forward. No emergency code change that must traverse the whole deployment pipeline. The feature flag kill switch thus constitutes a second layer of risk mitigation.

Artie Lee Director of Engineering
Climate LLC, a Bayer company

With Flag Triggers, you can resolve the incident in near real-time. Our platform integrates with popular observability and application performance monitoring (APM) solutions. In cases where error rates exceed a certain threshold, your APM will trigger LaunchDarkly to disable the feature flag associated with the errors. This happens automatically. And it represents yet a third layer of risk mitigation.

You also mitigate risk through targeting. For example, you can safely run canary tests of new features with a small subset of users in production. If something goes wrong, you’ve limited the blast radius to just the canary group.

Feature management embeds change management into automated software release workflows

Feature flags alone can take you a long way toward enabling safe, compliant CI/CD. But at LaunchDarkly, we’ve taken this a step further by automating key parts of the software delivery process. Through Workflow Builder, a part of our Feature Workflows offering, we let you embed change controls, compliance requirements, and quality gates within automated release workflows. Moreover, we let you create custom release templates that you can use repeatedly

Here’s what this looks like in practice.

1. One of your developers builds a substantial feature and wraps it in a LaunchDarkly flag. This change requires approval before production

2. The developer uses Workflow Builder to assign a change approver from among the engineering managers.

3. The developer schedules a future date, time, and target segment for the initial rollout. In this case, your team plans to progressively deliver the feature to six different segments, ending with all users and environments (aka a “ring deployment”).

Each successive segment, or ring, carries more risk. As such, you might make the approval requirements lighter in the early stages of the release and stricter in the later stages. For instance, if Stage 1 entails deploying to just internal developers, then you may decide to forgo approvals. But if in Stage 5, you’re shipping to a geography where your application has previously malfunctioned, then you might require an engineering leader, senior IT operator, or even a CAB to sign off on the change.

In any event, you can easily set all such custom rules with Worfklow Builder.

4. LaunchDarkly automatically initiates Stage 1 of the rollout, deploying the feature to production while only releasing it to internal developers.

5. In this case, let’s assume the team is using LaunchDarkly’s integration with ServiceNow, a top IT service management cloud software provider. As a result, when Stage 1 is initiated, it automatically generates a ServiceNow ticket.

6. In Stage 2, which again has been scheduled in advance, the feature gets released to a small beta group for testing. Developers, IT operators, and product managers monitor the situation.

7. Once the rollout reaches Stage 4, the assigned reviewer will approve the software change, thus, in part, meeting the SOC II “segregation of duties” requirement. The release proceeds as scheduled.

8. If your APM detects an error tied to the feature, it will automatically notify LaunchDarkly, in turn, disabling the feature. Whatever impact the error may have had on the user experience has been greatly diminished.

9. Once the feature clears every checkpoint and performs as intended, it is rolled out to all audiences. Then you become multi-millionaires. Obviously.

Nicholas Goss
Architecture Senior Designer, Cigna

The benefits of automated release workflows

1. Workflows cut down on change management bottlenecks and augment the CAB’s strategic qualities. Change managers can tackle other issues beyond the walls of software development. What’s more, they can better serve as an information beacon that keeps developers across the company aligned. And, frankly, they’re liberated from having to review every infinitesimal software change.

2. Workflows reduce inter-team dependencies and boost developer productivity. In fact, the LaunchDarkly safety net gives organizations the confidence to practice continuous delivery.

3. Workflows nearly eliminate risk from deployments and releases. If a feature causes a bug in production, you can disable it immediately. Moreover, with targeting and percentage rollouts, you can tread cautiously when shipping features.

4. Workflows help ensure regulatory compliance. And they do so while reducing toil and paperwork.

5. Workflows improve collaboration, especially between change and release managers. Traditionally, a clear division lies between change management (a governance process) and release management (a technical implementation and operations process). But when you weave governance into software delivery, you inevitably draw change and release managers into closer alignment.

Here’s how one LaunchDarkly customer benefits from Feature Workflows.

ULINE, a leading distributor of shipping, industrial, and packaging materials to businesses throughout North America, uses Feature Workflows for major digital product releases. They’ll start by, say, releasing a feature to 50% of their SMB customers. Then, after a few days of monitoring, they’ll expand the release to 100% of the SMB segment and 10% of major markets. This gradual rollout continues until, at last, a team lead approves shipping the changes to all customers.

Aaron Jeske
Senior Software Developer, ULINE

Feature management advances platform standardization

More and more enterprises are building internal “platforms” to support safe, efficient software development, delivery, and operations. Here are a few ways in which LaunchDarkly meets the requirements of virtually any platform initiative. And here’s why we encourage platform engineering teams to use LaunchDarkly for standardizing deployments and releases.

Integrate feature flags with developers’ daily lives

API-first

Every feature in LaunchDarkly is API-first, giving you the flexibility to plug feature flags into your platform. Build integrations, export raw data, and write custom scripts for automating feature flag workflows.

SDKs and integrations

LaunchDarkly offers 25+ client-side and server-side SDKs tailored to every major platform. We also integrate with numerous tools you use every day for monitoring, collaboration, data analysis, and beyond.

Control access to LaunchDarkly based on your business needs

Custom Roles

Custom Roles give you control over who can access what in LaunchDarkly. This applies to everything from feature flags and Projects to Environments and Teams. Enforce access policies that meet your exact workflow needs.

For example, only permit a Director of IT Operations to make certain changes to production. Or, allow an engineering manager to alter flags in one microservice while barring them from another service area.

Teams

A Team is simply a group of members within your LaunchDarkly account. An account administrator can grant specific permissions to specific Teams. This makes creating and enforcing access rules more secure, scalable, and efficient.

Track every software change with an audit log

Audit Log

LaunchDarkly provides a full audit log that automatically records all software changes you make involving feature flags. Change managers can present this log to regulators during an audit.

LaunchDarkly’s integrations with Slack and Microsoft Teams also help you maintain compliance from an auditability perspective. You can view, monitor, and control feature flags within your Slack and Microsoft Teams workspaces. Lastly, you can also log changes automatically via LaunchDarkly’s ServiceNow integration.

All of these automated change logs further support enterprise governance and compliance.

Microservices Domain Architect
Fortune 500 energy company

Effortlessly manage technical debt

Code References

Code References in LaunchDarkly allow you to quickly find and remove stale flags from your code. At the same time, you can set up alerts for Slack or Microsoft Teams in which you get reminded to archive old flags. This helps you maintain impeccable code hygiene when using feature flags on a large scale.

With LaunchDarkly, thousands of developers working in different languages and frameworks can follow a standard release pattern—one in which speed, risk mitigation, and compliance are embedded. A number of platform engineering teams are making feature management an essential piece of the software delivery pipeline within their platform.

Director of Enterprise Engineering
Fortune 500 financial services firm