DORA metrics changed the way a lot of engineering organizations viewed their productivity, but there’s a risk in getting hung up on moving faster just for the sake of it.
In a recent webinar, “Beyond DORA metrics: Maximizing the business value of software delivery,” Chris Condo, Forrester Principal Analyst, joined Ravi Thariyasi, Senior Director of Product Marketing at LaunchDarkly, to discuss how leaders can connect DORA metrics to the results that really matter.
Watch the full recording below and read on for some highlights.
What are DORA metrics?
You’re probably familiar with these, but before we explore more sophisticated gauges of your organization’s success, let’s align on what DevOps Research and Assessment (commonly referred to as DORA) metrics entail. Ravi summarized the four key metrics:
- Deployment frequency: How often changes are deployed to production
- Lead time: The time to commit to deployment in production
- Mean time to resolution: How much time it takes to restore service after an incident
- Change failure rate: The percentage of changes in production that result in a degradation or an outage
“The notion here is that these four simple but really powerful metrics are really associated with higher software delivery performance," Ravi said. "And that making improvements on these can really improve the efficiency and effectiveness of engineering efforts.”
Two stages of DORA metrics adoption
Chris shared that almost every engineering team he encounters now is looking at DORA metrics as a framework for creating a baseline for their engineering performance. And from those gathered metrics, they're then looking to further improve their efficiency.
The two stages that teams seem to advance through are first looking at that baseline and the target numbers for where they want to be (whether that’s number of releases per month or current mean time to resolution), and then reverse engineering how they can reach that desired improvement. At this point, they are comparing their previous and current performance to one another.
Then they look to their peers to benchmark their performance against others in the industry.
At the next level, as Ravi and Chris discuss in the webinar, organizations should not just be improving their efficiency, but really be interrogating whether they’re being more efficient with the right activities.
Beyond DORA metrics: speed != value
Having a way to quantify an engineering organization’s performance was novel for a lot of engineering leaders, but software delivery performance metrics are only useful if they’re supporting overall business objectives.
“So many companies come to me and say, ‘How do we go faster? We've got thousands of items in our backlog, and we need to get them through as quickly as possible," Chris said. "'We're not showing our worth to the company.’ My question always back to them is, ‘How many of those things in your backlog do you think are going to deliver any impact? How many of those things in the backlog do you think are really important?’ A lot of times they just say, ‘We don't know. All we know is our job is to get them done.’”
Chris explains that this disconnect emerges when you have multiple handoffs between units within an organization and an absence of shared business goals.
If each team is being measured by different standards and is not being rewarded for delivering value, you don’t have the right incentives in place.
“I think the conversation is shifting," Chris said, "from straight up, ‘We need to maximize our DORA metrics,’ to, ‘What is the right value for our DORA Metrics to serve our business so that we're actually satisfying the left and the right side? It requires a balanced approach, a more holistic view.”
Chris goes on to describe some ways that this more holistic approach can prompt leaders to evaluate why they are trying something and what they hope to gain from a change. A more experimental approach can help teams understand whether a change they want to make is worth investing time and resources in.
By scoping down to a small variation, deciding up front how they will measure impact, and monitoring how it performs and is received by end users, teams can get closer to results that actually matter to the business.
“This focus on value and experimentation takes software development away from just project-oriented work and brings it into the realm of product-oriented work where results matter and those results are business results," Chris said. "It really gets people thinking along the lines of: how do we manage this as a product versus just a bunch of story points that once they're done, we move on to the next project?”
Chris and Ravi take some time to explore how feature flags and feature management make it much easier to experiment with changes, which you can watch in full here.
What else is in the webinar?
As always, there are a ton of useful takeaways in response to the questions asked by the audience:
- How does the incorporation of early feedback from specific users play into this journey?
- Sometimes the DORA metrics will give you a warped sense of what is going on. For instance, solving a bunch of incidents that have a small MTTR will actually increase the MTTR, even though reliability increased statistics can be misleading. How would you counteract that focus on the wrong kinds of driver?
- How can we get started on an organizational shift towards a product mindset?
- For teams and industries like banking or healthcare, what do you advise in terms of balancing governance while also trying to adopt modern development practices?
Watch the full recording below.
.webp)
