ON-DEMAND PODCAST

Using ML to Assess Technical Debt and Prioritize Application Modernization

Other Recommended Resources

Video Transcript

Oliver White (OW): Good day, everyone, and welcome to this vFunction podcast. I’m Oliver White, and joining me today is our co-founder and chief architect, Ori Saporta. Today, we’re speaking about technical debt, what it is, why it’s a problem for enterprises, how it’s possible to calculate it, and how we can help you get out of debt, well, technical that is. Ori, it’s wonderful to have you on the show today. Before we start, could you just give us a quick 50-word bio about yourself, who you are, and what things do you like, what are your hobbies? And so on.

Ori Saporta (OS): Sure. So as you said, my name is Ori Saporta, I am co-founder and systems architecture in vFunction. My hobbies include playing base, double base. I also like playing basketball. My passion is programming. Ever since I was a kid, I started about 10 years old. And…

OW: In Java, or… [chuckle]

OS: No, no, I don’t think, was there Java then? I think the year Java came out or something. I was a basic. We were basic, whatever. And it led my career path, first Master’s degree in Computer Science and now with vFunction, I think this passion that we all share for software development is really what’s driving us also as a company to help organizations with their… Organization with their architecture. I think it’s nice to do the little things to help organizations go to the cloud, to see people use all the new cool tech that these platforms have to offer, but the real root issue of the thing is that we really wanna help the organizations with their overall efficiency, efficiency, sorry and engineering velocity to really help make people’s lives as developers more fruitful and and also more fun.

OW: So it sounds like technical debt is one of those things that does not make an engineer’s life fruitful and fun. Let’s talk a little bit about technical debt. So this is something you’ve spent a lot of time thinking about, how can a non-engineer understand this? Is it something like credit card debt for enterprise systems, or… How would you describe it to someone who’s not a developer? 

OS: Well, it really is like credit card debt or student loans or mortgages or any other type of monetary debt. The similarities are basically that when you assume debt, then it’s sometimes the right thing to do, but when that debt starts merging you instead the other way around, then you’re in trouble. So if I continue with the credit card analogy, then if you wanna buy a new, let’s say sofa for your home and you need money for that purchase and you take on credit cards debt, then that’s usually reasonable, but if at some point you’re spending half your paycheck, repaying that credit card debt for the sofa, maybe you should have bought a cheaper one.

OW: So your debt has accumulated [chuckle] to an un-manager… Unmanageable degree? 

OS: Yeah, it’s just not reasonable. So the same principle applies to technical debt in software development, so when developer makes designer selection choosing its specific design may not be optimal as a global solution, but it might be the right solution for that point in time when you have a goal to achieve with a specific time frame in mind. But if months later, you’re still paying for that by having all of your development resources taking double what they should because of that decision, then you should do something about that, right? You should either make, spend time to fix that, or… Well, basically, that’s it, or you shouldn’t have made it to begin with, but you can’t really build a time machine at that point, if you could, then that will be the least of your problems.

OW: So we’ll take a quick pause on the technical debt subject for just a second, while I rattle off a few interesting data points that we’ve seen recently. So one of them is from IDG, which is a well-known industry analyst. Their latest CIO report shows that the number one priorities of organizations is to modernize Legacy applications, and in our own research, we’ve seen that the number one goals of modernizing applications is to improve application innovation and increase engineering velocity. These are great, but there’s a flip side to these stats, and unfortunately, what we found in our own research is that 79% of application modernization efforts end in failure at an average cost of 1.5 million and about 16 months of invested work days. So considering these different metrics that we’ve seen pop up in the industry lately, what do you… How does it make sense to think about technical debt in terms of some of these statistics we’re seeing, such as, application innovation being the number one goal of modernization? 

OS: Well, it kinda reminds me of a quote from the Simpsons, by Homer Simpsons. At some point he says, “To alcohol, the cause to and solution of all of life’s problems.”

[laughter]

OS: So this is… Really, there’s a circular thing going on here with technical debt. ‘Cause the reason that your innovation is stunted and an organization would want to make this modernization effort is partially… Not a hundred percent, but partially because of existing technical debt becoming a very big problem, and very resource consuming as part of the development process. But also when you… When the that harmonization project takes place, if the technical debt is not handled head on with the proper planning, it might fail that effort. So it’s in the startup process, but it can potentially also end your process, and not in the way that you wanted it to.

OW: So, it’s a weird chicken and egg scenario. [chuckle] Technical debt is making it harder for you to address and handle… Manage technical debt. [chuckle]

OS: Yeah, I guess it’s a chicken and egg, but the egg catches a dinosaur or something.

OW: So how does technical debt… Accumulating technical debt actually affect your ability to innovate further down the line? If maybe we’re thinking of engineering velocity as being one of the primary starting points, let’s say? 

OS: Well, it all comes down to resources. And resources usually mean time. So everything would take longer and longer than it should. Basically, if you wanna spend two weeks, which is a standard sprint on a feature, and you end up spending three or four weeks on that feature. Then it cascades to everything it followed. So longer development cycles lead to longer test cycles, and longer release cycles, and the end to a longer period of time between when a customer asked for a feature and that customer is seeing that feature in production.

OW: So the… When we’re talking about longer test cycles, it’s because technical debt is preventing you from moving more quickly. And usually, we’re referring to a legacy architecture, monolithic architecture, which has interdependencies and poorly defined domains and so on. And essentially, whenever you need to test or release a new version of this system, your technical debt is slowing you down because you have more things to care about? 

OS: Well, I think it affects you in two separate like attack vectors, with this specific question, with regarding to the test cycles. So first, usually when you’re dealing with application, s with soft pieces of software, with large technical debt, they tend to be bigger applications. And these bigger applications almost by definition have longer test cycles. But even to compound that when you have large technical debt, usually it’s because of a lot of interdependencies between components in your architecture, in your software design. So then, when you make a change to one, you inadvertently may cause changes to some of the others, which would mean that you would need to test even harder to make sure that nothing was affected when you didn’t mean need it to.

OW: Gotcha. So let’s talk about what we can do, about technical debt. I know one thing we’ve heard a lot of… We’ve heard a lot in the past is, “Well, is it possible to manually detect and manage technical debt in a large code base? You know, what can we do on our own? Are there any tools out there that can help us” and so on? 

OS: Well, there are tools, I would first say that you can’t manually go through five or 10 million lines of code to look for technical debts. It will require a very, very large team over a very long period of time, and they will quit on you halfway through.

OW: And this is… This sounds like… This is what’s happened in the results so that we talked about earlier from our research saying that 80% nearly of modernization projects fail. It’s likely because they embarked on a 12-month manual analysis project and event storming that ultimately led nowhere.

OS: I’m sure some of them did. Yeah, absolutely. There are existing tools. And these tools, they usually focus on the very nitty-gritty details, looking for specific examples of anti-patterns in your existing code and pointing those out to you. This is good, right? I’m not… It’s required. It’s important. But the existing tools don’t really deal with the application as a whole. And that is important because when you deal with the application as a whole, you get some sense of urgency, understanding the status of this application, also allowing you to compare it to other applications, other pieces of software in your own company’s portfolio, to understand where you really need to put your resources to manage the technical debt.

OW: Has anyone ever tried to figure out a way to actually calculate technical debt? Is there any research on this at an academic level that can help point us in a direction? 

OS: Yes. Well, there have been multiple attempts at this issue of calculating and measuring technical debt. I think the most famous is the article in search of metric for managing architectural technical debt by Nord et al. It is a very, very good to read. I think it’s a great article, won a lot of awards. So it’s not just me that thinks that.

OW: Yeah, so this is a… This is a 10-year-old paper and it won an award in 2022 for being the most innovative research in this area. Right? 

OS: Exactly. I think the more people talk about technical debt, the more people rediscover this article. And in this article, they do a very important first step or second step and really suggesting a metric on how to measure technical debt. That metric is based on interdependencies between software components. They call it architectural components in the article. And the same kind of principle that we mentioned earlier, and they show how to use that information about interdependencies between architectural components to try and plan and choose between different design alternatives going forward, when to make a decision about this version and next version, and three to four versions down based on this decision.

OW: So this is something like best practices for avoiding technical debt in your next app? 

OS: Oh, not exactly best practice practices, sorry.

OW: Or a metric too. Yeah.

OS: More of a metric to estimate and to help you choose between different alternatives, design alternatives when planning, it doesn’t necessarily have to be your next step, but, but your next feature of an existing app. What it doesn’t do, it doesn’t really tell you how to measure interdependencies or any type of metric on existing application in any state, which is, I think the, real world issue that is kind of the next step after the theoretical issue that they presented so well.

OW: So let’s talk a little bit about how we’ve taken the theoretical side of things and turned it into an actual tool that we can use. And let’s talk a little bit about, what are we measuring when we say technical debt and how do we do that? Could you shed some light into that area? 

OS: Sure. So we’re taking this concept of interdependencies and we are really, really diving deep into it. Well, basically, we are building a dependency graph between all the classes in the system, in the software that we are analyzing, and we are taking this graph and we are looking good at it in kind of multiple facets. We’re looking at it as is. And we’re trying to find classes with a lot of incoming or outgoing edges. These sometimes represent what we call God Classes, just classes that do a lot of things, instead of just one thing, which is what usually a class should do. And then we also do a lot of manipulation on this graph and we try to create communities within it because it’s a very natural thing to happen based on domain. Usually you’ll get different communities of classes. And then we analyze the interdependencies between these communities. Again, looking for kind of the same things, communities with a lot of in degrees and out degrees and a long paths in the graph between communities. So we’re basically, without formally defining what an architectural component is, we are mapping dependencies, interdependencies between architectural components, these classes and these communities in the graph.

OS: Once we have that, we create several metrics that will help the user understand where the technical debt is coming from. So we create… The first metric we created is we refer to as complexity. And complexity is really, we’re trying to speak the language of developer, trying to write new feature, and it’ll be complex to write. If you have a lot more effort or resources, you need to spend to add a new feature.

OW: So how much time is spent actually writing productive code versus untangling, or figuring out a dependency somewhere else and managing the technical debt aspect? 

OS: Precisely. So if you’d be in a, what we call a Greenfield environment, you’re just writing this new feature without all the backlog. Then it’ll take you a specific amount of time. And right now, when you’re doing that in your current environment with a technical debt, it takes you more than that, presumably. And that is the complexity index. The other index we defined is the risk index. And the risk index is basically the unknown unknown, meaning that, when I make a change and I don’t expect other things to change, when I change one class and I don’t expect the behavior of other classes to change, but eventually they do because of very high entanglement, that’s the risk factor. That, I don’t know what to expect. So if the probability of me making changes I don’t expect is low, then the risk would be low. If the probability of me making a change you didn’t expect is high, then the risk would be high. And we kind of combined these two indexes along with other less significant metrics into, kind of an overall debt score and that’s what we share with our users.

OW: So the bottom line is that, when you’re looking at a debt score, it’s the extra work you need to put in per unit of work going towards innovation or new features, etcetera? 

OS: Exactly. And it’s the extra work you know that you’re going put in and it’s the extra work that you don’t know, but you’ll have to put it in to maintain stability of the system.

OW: Could we take a little bit of time to discuss how we used machine learning? This is a question that a lot of people ask us, what aspect of data science is playing a role in the way that we assess technical debt and come up with these metrics? 

OS: Yes, absolutely. So the first thing to note is that these dependency graphs are huge. Alright. And if we have an application with 10,000 classes and a lot of them are interconnected, there are a lot more edges in this graph. It’s not very easy to go over these with straightforward, deterministic algorithms, not to mention manually looking at it. You can’t look at it.

OW: Yeah. [laughter]

OS: We actually have some examples on our… In our environment. You see, you open the dependencies of a single class and the screen goes white with all the edges and…

OW: It’s almost a meme. [chuckle]

OS: Yeah, absolutely. And then if you take that and you wanna really understand patterns, then you have to do that for a lot of applications. We did several dozen applications and that makes it even harder to do manually or with deterministic algorithms. And lastly, we’re not necessarily sure what we’re looking for. We can say… There are specific anti-patterns that really know how to find, but when all of these come together in this very complex graph, that you have these large indegrees and outdegrees and long paths, it becomes something that you can’t just pick one at a time, one anti-patten at a time, you have to look beyond that and that’s exactly where machine learning helps us. Because we can train models based on some assessment or estimation on what’s the level of technical debt in these pieces of software. We did that with the applications that we were looking into with the original developers. And so, we had kind of this training set, and then we used this training set and created models based on that to get features really with telling, not just for specific anti-patterns, but for graphs as a whole, what’s the level of risk of complexity and overall debt.

OW: So it’s these models that we’ve… That we’re employing in vFunction Assessment Hub? So from complexity risk and overall debt, we’ve actually abstracted some of that information into other metrics, in addition to that. Can we talk a little bit about how Assessment Hub looks at things, like cost of innovation, return on investment? The top 10 highest debt classes, is one of my favorite features.

OS: Well, yeah, so this is really a combination of everything we talked about, up till now. So let’s start with top 10 debt classes. This is really where the anti-patterns come in, but we identify these anti-patterns on specific classes and then we estimate what’s the role of this class out of the overall debt, depending on those graph behaviors. And we find really the worst ones and show them to you so you can work on these issues regardless of the modernization project you’re doing or not doing. But really you need to understand these classes are causing you pain. And on the other hand of it, the other side of it, there is the cost of innovation. So cost of innovation is really looking at the overall debt from the monetary perspective and saying, “If we wanna put a dollar into innovation, how many dollars do I have to put into the project?” So, if you want to put a dollar innovation and your technical debt, your overall debt is low, you might need to put a dollar plus 20 cents, then you’re great, you’re in great condition. But if to put $1 into innovation, you have to put $5 in, not so great.

OW: So this… It sounds like this particular part is addressing the business case of prioritizing a modernization project, which is something we hear a lot from our users as being a challenge to actually convince executive team members that we know what we wanna do, we’ve prioritized it, and this sort of thing shows business metrics for that group.

OS: Yes, exactly. We wanna help these organizations make these smart decisions. So we allow you to do two things. First, you can look at your entire portfolio and see where you’re affected the most by technical debt or inflicted the most by technical debt. And then, you also have a much easier time building this business case and you go and say, “Look, we’re spending five times what we should, and if we spend an X amount now to manage this debt and reduce it… ” and we also show how much it could be reduced by some of these actions that were, like, the ten highest of classes. You manage this, you’ll reduce the debt index or the overall spending by so and so, and you can make this case saying, “It’s really… It’s easy money, right? I’ll just spend this money now, but it will save me so much later.”

OW: Well, I think we had a very interesting conversation about technical debt, how it’s creating this circle of despair where technical debt is preventing high engineering velocity which results in low innovation, yet at the same time it’s also the cause of the problems that are making it hard to start. So thank you for this overview of technical debt, what it means, how we can actually calculate it and think of it. And it’s not just a scary word that goes around the board room, but there’s actions that can be taken. Do you have any final comments or suggestions for developers out there before we end our call today? 

OS: Don’t disparage and check us out. We’re coming out with a new assessment as a service offering. We’ll allow you to take a quick look of your… The status of your technical debt in your applications and get started with making the smart decisions.

OW: Alright. Yeah, we’re looking forward to that. Stay tuned for more updates on the upcoming vFunction Assessment Hub as a service. I don’t think we’re 100% sold on what we’re calling it, but so far we’re going with that. And folks, please visit vfunction.com/trial, you can start assessing the technical debt of your own applications now. It’s free for your first app. Go have fun and let us know what you think. So Ori, thanks so much for a great conversation.

OS: Great. Thanks. Alright.