5 ways to cut the Gordian knot of technological debt
(Legacy Series 3/3)
My series on legacy systems will show how the adherence to old tech, processes and skills is the major threat for companies and economies. In the wake of breakthrough innovation such as blockchain or AI, legacy-strapped China is vying to dominate global business and finance. In three essays I will examine what policymakers and managers must do to capture the promise of the digital age.
People build lavish houses, drive expensive cars, and jet off to five-star vacations. They project an image of potency and financial health. Yet we know that in many cases their real net worth is not keeping pace with the image they cultivate. Often houses are mortgaged, cars leased, and the vacations paid by credit card debt. With companies’ IT-systems it works exactly the same way. Much of their outside appearance is built on piles of debt, sometimes so large it will never be paid back.
Companies stick a new front-end to their old back-end to pretend to be cutting-edge. While the external software quality appears good (e.g. in an app interface), the internal quality is brittle. In the beginning, internal issues can only be identified by developers, yet they eventually deteriorate the external quality as well. We have seen some prominent examples of what can go wrong in last month’s edition – from system breakdowns in banks and chaos at airlines, to serious hacks and attacks.
Even worse, sometimes corporations themselves believe they have achieved competitive parity with new generation competitors. In the worst case the deficits of the core system show only years or decades later. So front-end facelifts can turn out to be highly perilous.
Intentional debt strategy
In 1992 Ward Cunningham coined the term technical debt, explaining how companies creating their first code accept its imperfections in return for a speedy market entry. Just like financial debt, it enables rapid growth in the beginning, but slows down the debtor later when interest accrues. If ignored, it leads to bankruptcy.
Indeed, in most cases flaws are never ironed out. On the contrary: Companies spend lavishly and accumulate more debt due to liquidity shortages, which in this case is programmers’ time.
Accumulating debt can be a highly successful business strategy. In fact, most blitzscaling start-ups were able to grow quickly enough only because they were willing to accept imperfections in return for a quick start and hypergrowth. The crucial thing is to make design choices that will keep future costs of change manageable and ensure that you use the latest available tech.
Cunningham used technical debt in a very narrow sense, namely to describe old and imperfect code. I suggest to apply it more broadly to indicate the drag suffered by legacy systems in general, whether it be old and defunct code or an entire system that has been replaced by new innovation, say when chip cards replaced mag-stripe cards. Hence, I propose the broader term technological debt.
Integration debt and entropy
Though I will below argue that systems should be overhauled regularly, more frequently new modules (and non-technical innovation too) will need to be patched to existing system, i.e. integrated. Frequently the integrated features or updates get hard-coded, there is functional overlap, or the new module simply bloats the system with hundreds of features that are not needed. This usually happens when a company decides to go with packaged software. It is at this point that integration debt occurs.
Integration debt drives up so-called software entropy. Entropy basically means the inevitable rise of a system’s complexity by evolution. The concept shows how each modification increases the disorder and leaves systems slower and more vulnerable – exactly the opposite of the agility that is at the top of everybody’s agenda.
This principle is a curse for early innovators. Many headline-capturing trail blazers experience this truism first-hand. With infrastructure innovation in particular they invest into a version inferior to that of later emulators. To catch up, they add layer upon layer, thus multiplying technological and integration debt. No industry illustrates this better than finance. Half a century ago banks and insurance companies pumped unprecedented money into mainframes and self-programmed core systems. Today, switching them is likened to exchanging a jet engine 35,000 feet above the ground. Thus, the sought-after agility falls to fintechs, neo-banks, and other market entrants.
Technological debt is inevitable – so what to do?
Every company must start with an analysis of whether innovation threatens its core business, expands it, or allows it to enter new markets. Then the decision is on whether to be a pioneer, a quick emulator, or a laggard, because blind investment in new infrastructure might not be the best choice.
Finally, to answer the question at which part of the enterprise to start, managers must resort to tools such as Gartner’s market clock, tools that show the life cycle of technological assets. They are indispensable devices to estimate how long an investment will last until the underlying technology reaches its next phase and an upgrade of some sort is needed.
Once the direction is set, the dilemma is how radical the switch should be. Reengineering experts have argued for decades whether it is best to migrate a large, mission-critical information system piecemeal or at once. In 1993 researchers from the pioneering DARWIN project at the University of California advocated an incremental approach dubbed chicken little over a cold turkey migration. But piecemeal migration is akin to a game of whack-a-mole. It takes years and adds more complexity. A cold turkey migration, on the other hand, is often sheer impossible.
So better to deploy a dual-speed approach for managing disruptive transformation. The dilemma between cold turkey and chicken little is often one built on the dichotomy of agility vs. stability. Speedy implementation is weighted against operational risk. Thanks to APIs and microservices there is an alternative to scrapping a legacy system overnight. A dual-speed model smooths the migration path by offering a period of coexistence of old and new. Customer-facing systems are put on the fast track, whereas the back-end is swapped with a well-defined migration plan.
Dual-speed migration is far from easy to pull off, but when successful, it can combine the best of two worlds. Managing technical debt is a massive undertaking. To ensure it is successful, there are five corporate tools that can help.
1) In need of a general – introducing the Chief Digital Officer
Half a century ago, when mergers and acquisitions went mainstream, the CFO position was born. Nowadays data and digital are the topics keeping boards occupied. They cry out for a CDO, a Chief Digital Officer. More strategic and business-oriented than the CIO and CTO, this position will put technological legacy topics high up the organization’s hierarchy. Boards need a specialist who understands stodgy systems and how to fight them, how to train people, and how to determine which investment is worth making. Somebody who can undertake ROI calculations that do not stop at the short-term financial impact, but who has a holistic understanding of technology and finance.
Unlike the CIO and CTO, this new position must have P&L responsibility. A CDO is also best placed to make very hard decisions, such as whether and when to overhaul core systems.
2) Report technological debt as a liability on the balance sheet
The degree of legacy is very hard to grasp; for customers, managers, and investors. But often even the IT-staff managing the systems is not fully aware of their state. Yet the technical health of an organization is just as important to its future performance as its financial one. Reporting it on the balance sheet might increase transparency and ensure its prominence on the boards’ agendas.
3) Foster a culture of constant renewal
Some companies are masters in keeping up with the times. Amazon and Twitter are bright examples, but Ant Financial is the company to top it all. The Chinese giant trashes its systems every 3-4 years. It battles bad code, utilizes new tools such as microservices, but also pursues cloud computing, blockchain, and artificial intelligence with a rare zeal. It is prototypical for a new breed of legacy-strapped companies rising in developing economies. If you are interested in how Ant manages this constant flow of renewal, I highly suggest to read the case study by Chris Skinner in his book Digital Human.
4) Give open-source software a thought
Whether it is platforms such as Github or operating systems such as Linux – governments and companies are increasingly opting for open-source software. On the one hand they save on licensing fees, but more importantly, vendor-software is always a risky bet that the provider will keep supporting your legacy system.
5) Deploy software management methods of the future
Since the 1990-ies software tools such as testing automation have been developed to help with renewal. Equally important are new management methods. Lean IT cuts waste by tackling issues in organization and culture.
Agile software development and its main driver DevOps hack time-to-market and slash average code development cycles from 89 to 15 days, according to McKinsey. The innovation of DevOps is also organizational: IT operations and software developers are merged. SCRUM teams help to iteratively solve urgent projects that are too complex to be completely mapped in a project plan.
Finally, intrapreneurship and start-up-like teams within company frameworks promise an agility-boost.
There are also other techniques to fight spaghetti code and architectural brittleness. Companies can continuously slash technical debt by employing techniques such as creating buffers in IT-release in which to improve structural problems and error-sprinkled code. Forlorn components or systems can be rebuilt from scratch.
This article concludes my series on legacy systems, but next month we will continue with a highly linked topic, namely how you can make your corporation fit for the digital age. Yet instead of looking at the IT-systems, I will examine the key non-technical levers that you need to pull. After all, legacy can manifest itself on all corporate levels, whether it is your employee mindset, your customers, or your marketing-strategy.