Editor's Note: This post has recently been updated from a previous version.
Technical debt is a metaphor to understand how the tradeoffs we make in software development have long-term consequences.
It is traditional to compare software technical debt to financial debt: both kinds of debt are frequently taken on to achieve a long-term goal, both incur a kind of interest that must be added to the original cost of the loan, and both have a lot written about how to avoid or leverage them.
In theory, technical debt, if well-structured and paid off reliably, does not hurt an organization any more than a mortgage hurts a homebuyer.
In practice, organizations writing software sometimes accidentally take out high-interest loans to make progress that doesn’t pay off. Code debt is not bad in itself, because everything about software engineering is a set of tradeoffs. Technical debt and short-term-only code choices are dangerous when they are silent, unacknowledged, and there is no plan to pay them off.
“Move fast and break things,” sounds bold and visionary, but in the perspective of companies that don’t want to break things, the statement’s fallacies and overlooked nuances become scarier.
So how do we ship software, proceed in production safely, and maintain quality, without compromising speed? What kind of technical debt is important to take on, and what can we avoid, or pay off rapidly?
What is technical debt?
Technical debt is the cost of refactoring or redesigning code tomorrow because of choices made to meet constraints in the delivery timeline. Ward Cunningham describes his origination of the term technical debt in this 2009 video.
This was further elaborated by Martin Fowler as “Prudent Intentional Debt.” Technical debt, like a housing loan, is not inherently bad, but it needs to be part of an organization’s work budget.
I came across the concept when first reading the paper, “Machine Learning: The High Interest Credit Card of Technical Debt,” and learning about the construction of AI systems. Because of the alluring nature and magical promises of ML pipelines, we often ignore their high maintenance behavior, which require extensive attention, time, money, effort, and sometimes emotional anguish. The same drawbacks may apply to distributed systems, streaming infrastructure, or other applications of technologies with entangled architectures and designs.
The accumulation of debt is inevitable modern software development practices. Development teams create it as a byproduct of creating the features that drive business value.
Accumulating and failing to service technical debt is a common operational error of startups, but the longer maintenance is delayed, the more expensive it is to fix. The bad code and hacks that we promised ourselves we’d fix “as soon as we get through this release” becomes something so complicated and interwoven that it requires expensive expertise to even start cleaning up.
Early debt can be almost invisible, but it will always eventually turn into an expensive superfund cleanup scattered across the codebase. Debt isn’t always in the code — sometimes it can be design debt or business need debt.
For example, the US National Weather Service has been working on a message simplification plan since 2011, and each change takes about three years to fully integrate, since the weather service has so many dependencies, many of which don’t need to be updated frequently. Moving away from using all capital letter to describe weather conditions took decades, because the messages had been delivered by telegram and then teletype, neither of which allowed lower-case letters. When the National Weather Services partners moved to more modern delivery methods, the NWS had to go in and refactor their own systems to conform.
In a competitive labor environment, a clean, easy-to-learn codebase and high-quality software make it easier to onboard and retain developers at all levels.
Accrued tech debt will directly increase costs, stifle innovation, and elevate security risks—all factors that decrease and flatline revenue growth.
Some types of technical debt include:
- Hard-coded values, references, or pointers
- Old or deprecated libraries or referenced objects
- Workarounds for problems that have since been solved better
- Features that are no longer used or relevant
- Tests or experiments that are finished or no longer relevant
- Code supporting business goals that no longer apply
- Broken or partially-working pipelines or deployment tools
Consequences of technical debt
Tech debt is not just “messy”. It presents problems for our organizations in the following ways.
Difficulty updating
A codebase with a lot of technical debt is less like a race car and more like a rural bus with a crate of chickens strapped to the top. It will get where you need to go, but it may take longer and have a lot of things that you don’t need. When you need to add a feature, it will require negotiating with everyone already on the bus, because they have good reasons for the packages, libraries, and hacks that they bring along. This complexity also makes automation difficult to build and maintain. Single-use workarounds cause inadvertent roadblocks to simplicity and automation.
A codebase with technical debt is harder to maintain because the architectural documentation frequently reflects the ideal, not the system as it was actually written. The shortcuts and hacks are not reflected in the documentation or design, so someone new to the system may have difficulty determining what’s happening. Legacy code that is clearly documented is much easier to maintain and update than the “move fast” code quality that we often see from teams under feature factory pressure. Stakeholders can identify and prioritize features accurately when they understand the full cost and impact of their request, instead of the idealized “feature-only” expense.
Code simplification
Honeycomb co-founder Charity Majors has famously said that “The best diff is a red diff.” Every time you remove code by refactoring or improving technical debt, you’re making the codebase easier to understand and usually more secure. Although reducing tech debt is not always about deletion, it’s an important part of the process. Refactoring can also mean replacing internally-built solutions with software solutions built and maintained by others. This streamlines the code your team members need to maintain and update.
Security vulnerabilities
A chaotic codebase makes it easier to miss security vulnerabilities. If you don’t have a way to see an accurate code manifest, are you sure that everything in your code is what you want? If you solved a problem by hard-coding in a reference, will it get updated when a security patch is released? Is the quick-and-dirty solution something that will withstand security challenges? T
Technical debt also makes it harder to analyze which parts of your codebase might be causing a vulnerability and harder to correct the problem when you find it. Finally, a chaotic codebase is one where attackers can leverage older and deprecated parts of your system without you even knowing about it.
Stability issues
New features don’t matter if users can’t get to them. The stability of our systems is crucial to continued operations. Technical debt endangers our system stability and uptime because the interaction of untested or unready code can cause unforeseen problems. When there is a problem, troubleshooting it can be more difficult and time consuming, because it is difficult to tell which elements are “live” and contributed to the problem, and which parts of the software are not being used and therefore don’t need to be examined.
Ways to better manage technical debt
Managing technical debt is a multi-pronged responsibility. Leadership needs to prioritize time for technical debt paydown and reward people who do the code janitor work as well as those who create new features and functionality.
Programmers need to keep track of shortcuts they have taken and create plans to go back and revise their initial releases. Working in software involves many skills that are seldom taught in classes, including agile, scrum, and lean methodologies, technical writing, and codebase maintenance. These frameworks enable developers to scale from a solo project to part of a team producing high-quality, interoperable, reliable software products.
So how can you help as an individual?
- Note when you execute something that will need fixing, so you can actually fix it later
- Document and encourage documentation of decisions and trade-offs. Documentation debt is a subtle
- Attend or view technology-specific conferences and forums to start the conversation
- Talk to others who’ve experienced consequences of debt and learn preventive measures
- Start with small, dynamic, and easy changes or habits
What can a development team do to pay down technical debt and help each other work in an efficient, debt-reducing way?
- Designate times when the whole team will stop doing feature work and instead work on paying off technical debt
- Set up pairing practice to examine older parts of the code base
- Refactor-as-you-go when you are writing new features, even if it means slightly reducing velocity or time to market
- Recognize and reward developers and product managers who make the effort to remove technical debt
Best practices are often domain-specific. For a concrete example, here are some steps you can take to manage feature flags recommended in the LD Galaxy Conference by Mark Burry from iPipeline:
- Implement code references around relevant blocks and frequently think about flag removals as part of feature management
- Tag your flags with descriptive labels such as project/team, repo, in progress removal, awaiting third party
- Adhere to an appropriate fitness function for consistent flagging conventions
- Enable visibility of your flags through web hooks and other notifications
At LaunchDarkly, there are a number of forthcoming projects related to controlling technical debt, both on the individual flag level (for developers) or the environment level (for managers). This is one of the reasons why Forrester scored LaunchDarkly the highest for managing feature flag technical debt in the recent report, “The Forrester New Wave™: Feature Management And Experimentation.”
While it is satisfying to write new code, recall the last time you spent days debugging confusing dependencies or agonizing over the inefficiencies of old deprecated functions around the code-base. Solving technical debt as quickly as possible means that your code is higher-quality and requires less painful maintenance.
If you are still not convinced on the importance of addressing technical debt, envision the psychological pain and monotony your future self will endure. Establish guidelines and habits now to chip away at debt as you scale, and consequently reduce your system’s risk of implosion.
How do you measure technical debt?
Of course there are tools to measure technical debt, and each method has strengths and weaknesses, but here are some ways to look at your debt load:
- What is your cycle time? How long does it take you to release a feature?
- How many unintended outages do you have? Does your codebase feel fragile or scary to alter?
- Bug closure rate - how many bugs do you close for every bug that is opened?
Prioritizing debt remediation is sometimes difficult without firm measurements or metrics on how expensive your technical debt service is for your team. However, what is expensive for one team may be simple or unimportant for another team. Your software quality may be crucial to your ability to continue building, or it may be a placeholder for a new system coming in.
For more information on measuring technical debt, try this dev.to article from Alex Omeyer, a tools vendor and technical debt nerd.
What next? Ways to learn more
- Explore Martin Fowler’s writings on technical debt for more examples and illustrations on technical debt and backlogs and how to understand them
- Take a self-paced course on technical debt, such as this one from Mark Heath
- Consider moving to trunk-based development, which makes incurring technical debt more obvious and allows it to be visible to everyone on the development team
- Make a plan for what parts of your code base you want to rework and refactor, alone or with your team. Even reducing the amount of technical debt in the code as you come across it has a cumulative effect on your developer and user experience.
.webp)
