No One Loses $1000 Playing Penny Slots

278
Reducing-Risk-During-a-Downturn

Reduce risk and spending while you increase velocity

We are officially in a recession, and that means that budgets are tightening up all over. Most companies can’t afford to be the corporate equivalent of Vegas high rollers, splashing out big sums on marginal bets. The price of losing a business bet right now might not just be lost cash, but losing the business itself.

User:Pcb21 / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)

New hires, new software, new ways of relating to your users, even new infrastructure are all a bet that your business will gain or save enough money to cover the cost of transition and then some. Our current financial strain doesn’t mean that you can stop making business bets, because businesses must continue investing in their ability to deliver value.

Many large organizations have a development and release process that is equivalent to a big bet. Think of Apple’s yearly announcements about their hardware and software—if they are wrong about that, it will cost a huge amount of money. One way to keep making changes without staking your whole company is to make much, much smaller bets. 

The 2019 State of DevOps report makes it clear that smaller, more frequent releases correlate to both stability and effectiveness.

State-of-DevOps-Report-2019-1
State of DevOps Report, 2019

If your organization releases once a quarter, it’s a significant bet—the release includes all the things your teams have been working on for three months. If that release fails, either technically or because it’s not what the market wanted, you have lost a lot of time and money. It’s like playing $100 slots. It wouldn’t take many losses to become painful.

If your organization is releasing multiple times a day, and each release is a single feature or part of a feature, the failure of one does not mean that any other parts of the release were affected. You may have lost a penny in the slot machine, but it doesn’t affect your overall product or company. It would take many, many wrong bets at these low stakes to lose as much as you could with one wrong high-stakes bet.

How can you start reducing the size of the risks you take in your software development lifecycle?

Using feature management and CI/CD to make smaller bets

Organizations are not releasing slowly for the fun of it—it’s an artifact of the way software used to be developed, and a result of the business needing to present function and interface changes all at once. Building and investing in a CI/CD pipeline allows deployments to happen rapidly without disrupting production or requiring the marketing/training effort of a release.

With feature management, a new set of practices and tools powered by feature flags, you can separate the concept of deployment and release. This frees your teams to work at their maximum speed, without depending on each other for schedules. This is especially useful for distributed teams, which may have trouble coordinating for a large release.

One crucial element of a good continuous deployment pipeline is being able to deploy “broken” code into production without it affecting anything. Use feature flags to identify features that are not ready to operate in production yet. You still get the advantage of testing the integration and code as part of your build process, but you don’t need to be prepared for full operation yet. Once the feature is ready, you can use the flag to canary, or test on a small portion of your audience first, before releasing it to everyone.

Using feature flags to gate your code also forces developers to think about what the smallest unit of value is. A single checkbox might be a user interface item, but it might also be controlling several elements on the back end. Those should be bundled and managed with the user interface control, but each of them could be tested independently.

For example, imagine you had a checkbox that controlled whether a user is using Dark Mode. That would be controlled by one flag. There might also be flags controlling the app background color, the font color, the button appearance, and the button font. Each of those features is reasonably small, small enough to test and merge independently. 

Using LaunchDarkly, you can identify the checkbox flag as a prerequisite flag. The checkbox will only appear in production if the flag rules are enabled for production. The font and color flags will only take effect if the checkbox is visible and selected on.

Each of those flags controls a small part of the “Dark Mode” feature. It’s a small bet, and if something goes wrong with one part of the whole feature, it’s easy to separate out what is happening and where. Breaking out your business value with flags is similar to the microservice pattern—it’s easier to find and remediate breakage if your codebase is not fully intertwined.

Tiny tests

Breaking features into smaller testable units and being able to deploy more often changes your perspective on tests. Instead of testing being a part of development and then a part of the release approval process, testing can be woven into more of the feature lifecycle. Developers will still do the unit and function tests that they do now, but they will also be able to see how a feature behaves in production while it is still fresh in their minds. Instead of having to context switch back to something they wrote weeks or months earlier, they can write it, deploy it so only they can see it, and check that it functions correctly. That moves the risk of errors further left and reduces the cost to fix them. Tiny tests are another way to make your cost-of-development bet smaller.

Smaller bets increase business agility

When a gambler feels that they are on a cold streak, they may stop gambling. When an organization is making software changes, it can’t stop, but it can change focus or emphasis. The easier it is to walk away from a product direction that isn’t working, the less risk there is in trying new directions. Changing a product direction that isn’t working is a lot easier if it doesn’t take major overall changes, but can be achieved through small, checkable iterations. Changing development to smaller bets enables that kind of flexibility.

Conclusion

You may find it useful to think of feature management and Continuous Integration/Continuous Deployment as a way to reduce the risk your business faces by making each business decision smaller and more independent.

Learn more about feature management.