Most talk about AI coding tools is about building something new. Describe an app, get working code. That is useful, but it is not where most software work happens.
On October 23, NTT announced research technology aimed at the harder case: existing systems where the documents, the code and the tests no longer agree. NTT says about 60% of software development is enhancing, maintaining or redeveloping systems that already exist.
If your business runs on an ERP that has been customized for years, or a custom tool built long ago, this is the part worth paying attention to.
Why AI gets lost in older systems
Long-running systems change a little at a time. A field gets added, a rule changes, a report gets patched. The design documents often stay the same.
NTT’s point is simple. When documentation and code drift apart, developers spend a lot of time working out what to change and what else might break. Generative AI has the same problem. It reads the documents, trusts them, and builds the wrong picture of the system.
Some important rules are not written down at all. Lark, a Y Combinator startup that launched AI agents for distributors this week, notes that deciding what to reorder often depends on rules experienced employees carry in their heads.
I have seen this in ERP projects many times: the spec says one thing, the customization does another, and the person who knows why has moved on.
Matching the documents to the code
NTT’s first technology does traceability analysis. It breaks specifications, design documents and code into units by function, links them together, and flags where they contradict each other or where something is missing.
In a test on a sample application, NTT planted 41 errors in the design documents. A common approach called retrieval-augmented generation (RAG), where the AI searches documents before answering, caught 11 of them, or 27%. NTT’s method caught 33, or 80%.
According to NTT, the difference is that its method checks how the pieces are supposed to relate, instead of only searching for text that looks relevant.
Finding the tests nobody wrote
The second technology maps existing tests against what the system should be checked for, then generates new tests for the gaps. NTT says the weak spots are usually abnormal conditions, boundary values, combinations of conditions, and how functions interact.
In its experiment, the method found 4 of 6 planted bugs. It also found 3 real bugs nobody had planted, in input validation and error messages where test coverage was thin. When generative AI wrote tests directly from the documents, it found none of those bugs.
That last result is the useful one. Asking AI to write some tests is not the same as having a clear method for deciding what needs testing.
It is also early. These are preliminary results on a sample app, and NTT still plans to try the technology on real projects of different sizes.
What this means if you run an older system
You do not need NTT’s tools to act on the lesson. Before you point AI at an existing system:
- Check whether your documentation still matches what the software actually does. If it does not, AI will learn the wrong version.
- Write down the rules that live in people’s heads, like reorder logic, approval limits and pricing exceptions.
- Look at your test coverage, especially error handling and edge cases, before letting AI change anything.
- Treat AI changes to old systems as drafts to review, not finished work.
AI can speed up work on existing software, but only once the system is understandable. For many businesses, that cleanup is the real project.
How up to date is the documentation for the system your business depends on most?

