For over 400 years, people have used the adage, “can’t see the forest for the trees,” to indicate that individuals are so focused on parts of a situation that they lose sight of the bigger picture. The same sentiment is often applied to development teams who appear to ignore or deprioritize more significant issues when maintaining and updating applications.
This myopic focus can lead to the accumulation of architectural technical debt, a hidden cost that can impede innovation, compromise system reliability, and increase the likelihood of critical failures. Unchecked technical debt not only hampers the overall health of software applications but can also result in significant financial and reputational losses for organizations.
Understanding the intricate balance between architecture, code quality, scalability, and functionality is crucial. Leveraging tools that prioritize these aspects can help developers maintain a healthy software architecture, prevent technical debt, and ensure the long-term success of their software projects.
What Happens When Technical Debt Grows Unrestrained
In many ways, the forest analogy is an unfair one. Just as a forest can’t exist without trees, software can’t operate efficiently today without modern architecture. Similarly, a thriving forest cannot grow unconstrained without the risk of a devastating forest fire. Software architectures need to be managed, maintained, and modernized, or tech debt accumulates and the software can implode. Take Southwest as an example.
Southwest’s 2022 holiday failure was only the beginning of its technical “glitches” as it tried to compensate for technical debt that grew uncontrolled. In April 2023, Southwest asked the FAA to pause all flights until they could address another glitch that delayed 1,700 flights. The airline’s woes illustrate the challenges of balancing maintaining the forest while addressing the trees. Southwest chose to ignore its growing tech debt in favor of selling more trees.
How can organizations avoid becoming the next Southwest? How do they control architectural and application technical debt? They must incorporate architectural observability and testing tools at every stage of their development and DevOps process to ensure their applications operate smoothly across their enterprises. This includes
- Static source code analysis tools
- Software composition analysis tools
- Static and dynamic application security testing tools (SAST, DAST)
- Architectural observability tools
Ground-Level Coding Tools
Many programmers dislike testing their code. They believe unit testing is a poor use of their time. When teams are pressured to deliver updates, taking time away from development seems counterproductive. However, static code analysis can help identify technical debt at a foundational level. Tailoring these tools can establish rules for consistent coding within an application or across an enterprise.
Related: Why Mastering Architectural Observability is Pivotal to Managing Technical Debt
Static code analysis tools detect bugs and weak coding methods. They can highlight spaghetti, dead, and zombie code. Some even calculate technical debt and help build more comprehensive unit testing. Encouraging developers to use tools improves code quality and reduces costly rework because of errors found later in the delivery pipeline.
Paying Attention to the Environment
Forests begin with a single tree. Saplings planted in the appropriate environment thrive, building a robust forest. Attempting to create a forest in an area that meets the minimal requirements increases the effort needed to sustain growth. Land management spends more time monitoring the environment and compensating for poor growing conditions.
To effectively manage an evolving application, software developers need to evaluate the components used to build an application. Programming languages, frameworks, and third-party libraries impact the final solution. Picking the most appropriate building blocks improves an application’s stability. However, yesterday’s solution may not meet today’s needs.
For example, open-source components may contain security vulnerabilities and lack the support to address them. Programming languages may not have the flexibility to deliver new capabilities efficiently and frameworks may demonstrate weaknesses over time. Unless developers routinely reassess their choices, they may discover an increase in their software’s technical debt.
Monitoring the environment of a growing forest requires automation. Modern technology places sensors among the trees to report changes in the environment. Development teams can deploy automation to evaluate application building blocks.
Software composition analysis tools can detect outdated frameworks or known vulnerabilities in open-source components. They can recommend remediation solutions to continuously reduce technical debt. By automating the process, organizations can focus on new development, knowing that tools are in place to monitor the environment.
Secure by Design
Secure by Design is an initiative to design and develop software that incorporates cybersecurity measures to protect against malicious attacks as early in the process as possible. To help guide developers around the globe, the Cybersecurity and Infrastructure Security Agency and its international partners released a Secure-by-Design publication in April 2023. The document outlines core principles to protect technology, including
- Shifting security responsibilities from the end user to designers and developers.
- Building an organizational structure that prioritizes security as an essential design element.
- Embracing transparency and accountability to ensure vulnerabilities are addressed.
The difficulty with such an approach is developers are not cybersecurity professionals. When companies struggle to fill cybersecurity positions, allocating those limited resources to design and development may not be possible. Their skills are better applied to protecting the enterprise.
Deploying security and vulnerability analysis tools (e.g., SAST and DAST tools) throughout the development life cycle can make secure-by-design feasible. Instead of waiting until the end of a development cycle to run the software through a security expert, have team members use analysis tools early. It’s less costly to address vulnerabilities at the design stage than to rework code once in production.
Managing Growth
Managing a forest means managing growth. Pine trees and Douglas fir saplings grow at different rates. Comparing real-time growth metrics with established benchmarks helps the Forest Service plan. Perhaps the saplings aren’t receiving enough sunlight, or brush is accumulating on the forest floor. Determining the best approach requires visibility.
No one expects that trees will grow according to benchmarks. They plan on viewing the site to understand how the trees are performing. From the data, forest managers make adjustments to their plans. They may rely on monitoring data for information between site visits to decide if an unscheduled look is needed, but nothing can replace onsite visibility.
Although developers can’t make onsite visits to their code, they can deploy tools that can record activity during testing and at runtime. These records may be logs and traces. Performance records can generate a historical log of changes. Observability tools make it possible to digest the volume of recorded data to help DevOps understand deviations quickly.
Architectural observability tools can help measure architectural technical debt that accrues as software architectures drift and evolve. Incorporating architectural observability into the continuous integration and delivery process helps minimize architectural debt introduced sprint by sprint, release by release. Best practices suggest that these tools should be used before and after each software change.
Observing the Architecture
Suppose forest managers followed all the best practices only to discover their forest is three feet over the property line, and the owner wants the trees removed. That’s a significant effort for a mile-long boundary. Unfortunately, the land managers lost sight of the forest while building thriving trees.
For software developers, it’s easy to focus on individual snippets of code or an application, without considering how they impact the entire architecture. For example, the original architecture used a data lake for storage. When adding several new features, developers found the data lake architecture was not responding fast enough to meet their needs. Reporting jobs took hours to complete. They decided to extract the necessary data from the data lake to build a smaller dataset without regard to the myriad of inter dependencies and cross-domain issues for the rest of the application.
Related: Shift Left to Avoid Technical Debt Disasters: The Need for Continuous Modernization
While the application continued to perform, the problem was that other domains and services were also using the data services due to cross-domain pollution and contamination. This created further testing issues, new bugs, release delays, data integrity and security problems.
While the data lake controlled access to its data, other domains that tried to access the new data lake did not. The developers assumed the application authentication was enough. What they failed to realize was some of the extracted data fell under HIPAA and PCI standards.
Without fully understanding the architecture and related dependencies, the developers took a localized view without a more global architectural perspective and were forced to back out their fixes. The resulting product release delays and potential compliance violations demonstrated the need for architectural observability as a guidepost for future updates.
Understanding Software Architecture
Just as trees make a forest, applications make an enterprise. Applications carrying heavy technical debt weaken an enterprise and expose it to vulnerabilities. When a tree is damaged or diseased, it is removed to secure the forest. Yet, many organizations fail to look at the technical debt that accumulates at the architectural level, letting it grow unchecked.
In order to address architectural technical debt, development teams have a range of tools to help manage source code, security, or software composition but need architectural observability tools for their software architecture. Individual programmers already can access existing tools to find bugs, eliminate smelly code, and conduct unit testing. These tools can reduce technical debt before it leaves a developer’s desk but do not address architectural technical debt.
Automated tools can help development teams look for outdated or unsupported components and check for security vulnerabilities throughout the development life cycle. Adding architectural observability capabilities before and after software changes lets teams find, fix, and manage architectural technical debt as part of a continuous modernization process.vFunction’s Architectural Observability Manager allows engineers and architects to view, manage, and track application architecture in both a static and runtime context. It lets organizations manage their software architecture to minimize architectural drift. To avoid being the next Southwest, let us help you see the forest without ignoring the trees.
