How to Measure Technical Debt

Technical debt adds significant risks, drains resources, increases costs, and harms long-term system performance and scalability. As it grows, it increasingly undermines businesses' ability to adapt to demands and leverage new technologies. It’s a constant source of frustration for IT leaders and security teams.

91% of CTOs believe technical debt is their biggest challenge, and, on average, nearly 70% of organizations view technical debt as having a high impact on their ability to innovate. Technical debt consumes almost one-third of all technology budgets and more than one-fifth of technology professionals’ time, according to research from global consultancy company Protiviti.

Yet, despite the impact of technical debt on budget, resources, and innovation, many organizations fail to manage and measure technical debt across their IT estates. It’s often ignored until it becomes an overwhelming problem, silently consuming resources and stopping progress. The problem is only expected to increase as businesses race to implement AI and other new technologies on top of already debt-ridden IT portfolios. 

“Most organizations don’t know how to measure their technical debt, much less how to actively manage it. As unintuitive as it may appear, measuring and managing technical debt can create a competitive advantage for those who do it right.”
- Dr Ken Knapton, “Measuring And Managing Technical Debt” for Forbes Technology Council 

Given this scenario, it is fortunate that several approaches exist for tracking technical debt as it accumulates, controlling costs, and proactively managing its associated risks before they become out of control.

In this article, we share the key steps organizations can take to simplify technical debt management.

This includes: 

• Understanding the technical debt backlog - Are you acquiring debt faster than you are dealing with it?
• Estimating the costs - What is the financial cost of various debt items or the measurable risk it presents?
• Prioritizing technical debt investments, creating a roadmap, and paving the way for modernizing your IT estate -  What debt can we safely ignore, what can’t we ignore, and what will it cost to address it?

By exposing technical debt, you will be better able to articulate its business impact and work with teams to make decisions about technology investments and digital transformation plans.

For a more detailed overview of technical debt, its causes, and impact, read our:  Introduction to Technical Debt.

Key Steps for Measuring Technical Debt

Diagram 1: An overview of the four key steps to proactively managing technical debt.

As is covered in the overview article linked above, there are many possible definitions for technical debt, so before embarking on managing technical debt, it’s important to have a clear definition of what it means for your organization and instance. It’s also helpful to have the relevant parts of your technology stack documented. This includes documenting application software products and may also include other technology products like database management systems and operating systems.

Learn more about how Enterprise Architecture portfolio management helped an American insurance group resolve technical debt and future-proof their business.

Step 1: Identify and Describe Technical Debt Items

The first step is to look across your IT estate, focus on the biggest problem areas, and start documenting where you have technical debt Items. 

Some common ways to uncover technical debt include:

• Running periodic architectural reviews or application health checks of the most business-critical systems
• Surveying application and technology owners
• Applying application rationalization techniques to undercover duplicate or overlapping applications
• Analyzing service incidents
• Scanning through code libraries
• Reviewing project documentation
• Checking for known vulnerabilities (e.g, by connecting tools like IT-Pedia to your Enterprise Architecture platform)

As you identify technical debt, record each one as a debt item in your technical debt backlog and classify each item based on your chosen model for evaluating system and software product quality. You probably have your own approach to classifying quality based on characteristics like reliability, maintainability, efficiency, security, maintainability, etc.

Three examples of quality models are:

• BS ISO/IEC 25010:2023 
• GOV.UK Service Standard (a widely used model for assessing service quality)
• The Ardoq Quality Model (the diagram below shows the model and the system quality characteristics involved)

Diagram 2: The Ardoq Quality Model

Your technical debt backlog should be stored in a central repository, ideally with your application inventory and risk register, bringing data together in one place for analysis.

Your repository should link technical debt items to the relevant applications. This will enable you to model, visualize, and communicate to others the impact of technical debt on business technologies. Technical debt is often the cause of risks, so you can also connect technical debt to your IT risk register. If you want to strengthen your technical debt assessment and examine the impact of technical debt on the business's ability to deliver on its objective, you can include business processes or capabilities in your analysis.

Diagram 3: An Enterprise Architectural approach to technology debt management enables you to understand the impact of technical debt on your IT estate and its risk profile.

Now that you have a living and current record of your technical debt, the next question is, “How big is that debt?”

2. Assess the Technical Debt Backlog

By assessing technical debt, you will be able to answer questions such as:

• What is the amount of technical debt?
• Which types of technical debt are most prevalent across our estate?
• Where is technical debt having the biggest impact on our business capabilities?
• Which organizational units are most impacted by technical debt? 
• How is technical debt connected to application risk or IT security risk? 

Quantifying Technical Debt

Two well-known methods used to quantify technical debt are the SQALE method and the Gartner method. These have different perspectives on technical debt costs and impact.

The SQALE method uses financial estimates of:

• The cost of living with the debt
• The cost of remediation

The Gartner model uses a 1-5 scale to estimate:

• What is the worst possible impact of the debt?
• What is the likelihood of it happening?
• What is the cost of remediation?

These quality models can be used separately or together as they complement each other. Many people are also proponents of quantifying technical debt based on business criticality or business value. Your organization may choose to develop its own model or leverage a hybrid of these.

It is often challenging to estimate the cost of technical debt and the cost of remediation. We suggest getting a holistic sense of this using two different perspectives:

• What are the costs and risks of not addressing this technical debt?
• What is the cost involved in addressing this technical debt?

Rather than focusing on accurate financial costs, the “t-shirt” sizing approach might be good enough to get a high-level estimate. This estimate can then help you prioritize accumulated technical debt items and create a roadmap to address them.

You could also consider it as, “How many days, weeks, months, or years will it take to fix this debt?” and then multiply the number of days by a daily rate. The result is still an estimate, but formulating the question this way makes it easier to answer. 

As you carry out this assessment, you should collaborate with several departments to gather insights and review information. The surveys built into Ardoq, for example, are effective tools for working together and speeding up assessment processes because they force you to formulate clear and consistent questions, helping everyone understand what you want to achieve.

You can send surveys to software developers, project managers, application owners, and other stakeholders to capture information about the level and impact of technical debt.  Survey information will then be automatically populated to your database, so you can review results instead of manually wrangling data.Diagram 4: Examples of survey questions that can help uncover, classify, and document technical debt

Some key technical debt metrics include:

• Number of assets impacted by live debt
• Technical debt per business capability
• Amount of live debt per quality characteristic
• Live debt trend line
• IT assets impacted by live debt items
• Estimated debt remediation costs

Consolidating and Communicating Assessments

Dashboards are a useful way to summarize assessments and provide ongoing information to key stakeholders.

Diagram 5: Dashboard in Ardoq summarizing the financial impact of technical debt and outstanding debt items

Various types of impact assessments help show the consequences of technical debt and can also surface issues that software architects will find valuable as they rebuild or refactor applications. Impact analysis is also useful for understanding how funds should be allocated to reduce debt items.

Diagram 6: An example of how impact can be modeled in Ardoq 

Find out how capability modeling can be a crucial part of reducing technical debt and optimizing IT spend. 

Step 3: Take Action

With the assessment completed, you now have key facts and figures to prepare a business case and discuss with peers and other stakeholders. 

It’s critical to assign an action to each technical debt item: Address, Plan, Delay, Ignore, or Remediate. For each decision, record the agreed actions including dates, people, and review dates, so that they can be followed up in a timely fashion and the backlog doesn’t grow out of control. Ideally, the technical debt items marked “Address” will be linked to a project or initiative so you can track progress and measure how the backlog is being reduced.

Diagram 7: Example of a report summarizing initiatives that are addressing technical debt

Step 4: Implement and Review

It takes commitment to continuously stay on top of technical debt - but it is critical to keep from drowning in sub-optimization. Make sure to track your remediation initiatives, understand your progress in reducing the backlog, and conduct periodic reviews of critical systems. 

Tracking remediation initiatives enables you to demonstrate your success in tackling debt. As initiatives are completed, debt items can be marked as remediated, removing them from the set of live debt items and reducing your backlog. By tracking the accumulation trend, you can see whether you are remediating debt items faster than you are discovering new ones.

Data-driven software platforms like Ardoq can help you efficiently measure technical debt, optimize your IT portfolio, and manage IT risk by leveraging automation and data-driven visualizations as well as reports and dashboards.

Key Takeaways

• The most important step in tackling technical debt is recognizing it must be managed actively, or it will grow out of control.
• Classify technical debt based on your chosen software quality models and prioritize actions and budgets to identify and remediate debt items causing the most significant issues.
• Encourage everyone working with software applications to document technical debt and use architecture reviews and health checks to update the technical debt backlog 

Take a closer look at how Ardoq’s Technical Debt Solution helps you quickly address key challenges, analyze and assess impact, and track progress more efficiently.