Most companies asking about AI already have working software. An ERP, a case system, a portal, a warehouse app, years of records inside them. So the real question is rarely whether to build something new. It is about adding AI to existing software: getting something useful into what already runs, without a rebuild and without betting the quarter on it.
That is a narrower and much more tractable problem than “adopting AI”. Here is how we approach it.
Key takeaways
- Start from a decision someone makes repeatedly, not from a model or a tool.
- Four patterns cover most useful work in existing systems: retrieval, classification, drafting, and extraction.
- The hard part is almost never the model. It is your data and the integration around it.
- You almost certainly do not need to train your own model.
- Design for being wrong. Anything that acts without review needs a way to catch mistakes.
- A first feature should be small enough to ship in weeks and measurable enough to judge.
Start with a decision, not a model
The projects that go nowhere start with the technology: someone wants to use AI, and then hunts for somewhere to put it. The projects that work start with a specific repeated decision that a person currently makes by reading something.
Which team should this ticket go to. Is this invoice a duplicate. What did we agree with this customer last time. Which of these 400 applications match the requirement. Each of those is narrow, happens constantly, and has a right answer you can check afterwards. That last property is what makes it buildable, because you can tell whether the feature is working.
If nobody can describe the decision in one sentence, it is too early to build anything.
Four patterns for adding AI to existing software
Retrieval over your own content
Answering questions from your own documents, policies, tickets, or product data. The model supplies the language, your content supplies the facts, and every answer cites where it came from. This is the most common useful pattern in a business that has accumulated a lot of written material, and it does not require changing the system that stores it.
Classification and routing
Reading an incoming item and deciding what it is: which queue, which priority, which category, which team. It replaces a rules engine that has grown to hundreds of conditions nobody dares to touch. It is easy to evaluate because you have history showing where things actually ended up.
Drafting
Producing a first version that a person edits: a reply, a summary of a long thread, a description, a report section. The value is in the minutes saved per item multiplied by volume, and the person stays in control of what goes out.
Extraction
Pulling structured fields out of unstructured documents. Invoices, contracts, specifications, forms. This is the pattern with the clearest financial case in most operations, because the alternative is somebody retyping the same fields all day.
The hard part is your data
Nearly every AI project we have worked on spent more time on data than on models. Not because the data is bad, but because it was stored to run the business rather than to be read by something else.
Records are spread across systems with no shared identifier. The same customer appears three ways. The useful text is in a PDF attachment rather than a field. Access rules exist in the application layer, so pulling data underneath it quietly bypasses permissions people rely on.
None of that is exotic and all of it takes time. Getting the storage and access layer right is normal engineering work, and it is what our database development service handles alongside this kind of project. Budget for it honestly, because it is where the schedule actually goes.
The integration layer
An AI feature that lives in a separate tool people have to remember to open will not get used. The value comes from putting it where the work already happens: in the case screen, in the ticket queue, in the ERP form.
That means a clean interface between your existing application and whatever is doing the reasoning, with sensible behaviour when the model is slow or unavailable. Treat it like any other external dependency, with timeouts, retries, and a fallback that lets people carry on working. This is ordinary API development work, and doing it properly is what makes the difference between a demo and a feature. Where the surrounding application also needs work, that falls under application development services.
What it costs to run
Unlike most features, this one has a per-use cost, and it is worth modelling before you commit. Three things dominate.
- Volume. Cost scales with how much text goes in and out, so a feature used on every record behaves very differently from one used on exceptions.
- Context size. Sending a whole document when a section would do is the most common source of avoidable spend.
- Caching. Repeated work on stable content can often be reused rather than recomputed.
Latency matters too. A person waiting on screen tolerates a second or two. A background job can take a minute. Deciding which one you are building changes the design.
Design for being wrong
These systems are confidently wrong sometimes. That is a property to design around, not a reason to avoid them.
The OWASP Top 10 for LLM Applications is a useful catalogue of how these systems fail in production, and the NIST AI Risk Management Framework is the reference most enterprise reviews are written against. The practical rules are simple. Anything customer-facing gets reviewed before it goes out, at least initially. Anything that writes to a system of record is logged with enough detail to reconstruct what happened. Every feature has a measured accuracy on a held-out set of real cases, and someone owns that number. Confidence is surfaced so people know when to look closer.
If the existing codebase makes that kind of logging and review awkward to add, that is worth knowing before you start. A code audit answers it quickly.
A sensible first project
Pick one decision. Use real data, not a sample. Ship to a small group of people who actually do the work. Measure against what they did before. Give it a few weeks, not a quarter.
The point of the first project is not the feature. It is finding out what your data is really like, how the integration behaves, and whether the people doing the job find it useful. That is what an MVP development approach is for, and it is a far better use of a first budget than a long evaluation. If it works, you will know what the second one should be. Teams that want continuity across several of these usually put a dedicated development team on it rather than starting fresh each time, and our AI development services are structured that way.
Frequently asked questions
Almost certainly not. The common patterns above are built by giving an existing model access to your data at the moment it is asked, not by training. Training becomes relevant in narrow cases with large volumes of labelled examples, and it is a much bigger commitment. Start without it.
It depends entirely on the provider and the plan you are on, and it is a contractual question rather than a technical one. Business tiers from the major providers generally exclude API data from training. Get it in writing, and route anything sensitive accordingly.
Assemble a set of real cases with known correct answers before you build, and measure against it. “Accurate enough” is not a universal threshold, it is whatever beats the current process at acceptable risk. Without that set you are guessing, and so is everyone else in the room.
For one narrow decision on data you can already access, weeks rather than months. If the data has to be consolidated or cleaned first, that work dominates the timeline and should be estimated separately rather than folded into the feature.
Bottom line
Adding AI to software you already run is mostly ordinary engineering: a clearly defined decision, clean access to your own data, a sound integration, and a way to catch mistakes. Choose one repeated decision, prove it on real records, measure it honestly, and expand from there.
If you want to work out which decision in your business is the right first one, get in touch. We will look at what you run today and where it would actually pay off.
