A definitive guide to releasing your best software
Farewell to outages, midnight releases, and customer churn
4. Stop throwing features at the wall to see what sticks
Elite software teams at organizations like Expedia, Booking.com, Meta, Google, and Microsoft continuously run A/B tests and experiments. And they profit handsomely from it. At one time, Microsoft was generating hundreds of millions of dollars of additional revenue every year as a result of large-scale experimentation.[5]
But most organizations struggle to run enough experiments for it to matter. And they seldom get much value from the few experiments they do run. The promise of experimentation eludes them. Why?
One answer is that traditional experimentation tools are isolated from the development workflow. What’s more, they often require advanced knowledge of statistics to operate. Lastly, many tools take too long to yield meaningful results. All of this leads to bottlenecks, fewer experiments, and less actionable data.
"Many firms run no more than a few dozen (experiments per year) that have little impact. If testing is so valuable, why don’t companies do it more?”– Stefan Thomke, Harvard Business School[6]
Maximize the business impact of every feature through experimentation
LaunchDarkly unlocks experimentation for the masses. It lets you continuously measure and improve the business value of digital products with a clear line of sight into both user experience and back-end performance.
LaunchDarkly allows anyone—not just data scientists—to run A/B tests and multivariate experiments with high statistical rigor. It also lets you quickly attach metrics to any feature you intend to ship, learn which variation of that feature has the greatest business impact, and automatically direct users to the winning variation. Finally, unlike traditional experimentation tools that operate in a silo, LaunchDarkly draws feature development and experimentation into a common workflow. Developers and product managers can collaborate within a single platform.
As a part of its offering, LaunchDarkly provides Funnel Experiments, enabling you to:
- Make decisions with higher confidence
Understand user interactions with features and track the effect of experiments on key metrics and funnels over time to confidently ship the winner. - Drive product-led business impact
Measure and test every change with out-of-the-box funnel experiments for a shared view of key business metrics. - Conduct powerful analyses
Combine audience targeting and experiment allocation data to build customized analyses directly into Snowflake data warehouse.
In opening the door to large-scale experimentation, LaunchDarkly enables you to measure and improve your digital products. Having fulfilled the fourth tenet of great software, you’ll be well on your way to building more products that customers love.
Customer success stories: Experimentation and A/B testing
- CCP Games increases median session length—a key business metrics—through self-serve experimentation. Learn about CCP Games’ success.
- Loom went from only shipping experiments once every 2-3 weeks to shipping them every day. Learn about Loom’s success.
- Ritual goes from running 1-2 experiments/month to 5+ after replacing its old experimentation tool. Learn about Ritual’s success.
--
[5] Fabijan, A., Dmitriev, P., Olsson, H.H., and Bosch, J., “The Benefits of Controlled Experimentation at Scale,” 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Vienna Austria, 20217, p. 18-26.
[6] Thomke, Stefan, “Building a Culture of Experimentation,” Harvard Business Review, March-April 2020.
