What AI Integration Services Do Not Quote

Search for AI integration services and the results are a directory. Vendor lists, capability grids, logos arranged by industry. What almost none of them describe is the actual work, which is why the quotes that come back are so hard to compare: they are priced against different pictures of the job.

Integration is presented as a decision about which model to use. That decision takes an afternoon and is reversible. The rest of it is plumbing, and the plumbing is where the months go, where the cost sits, and where the thing either survives a year of real traffic or quietly stops being used.

There are four questions underneath any integration. Where the context comes from, who is allowed to see it, what happens when something does not answer, and how a wrong answer gets caught before a customer reads it. A proposal that does not address all four is not cheaper. It has moved those parts to you without saying so.

Where does the context come from

A model knows nothing about your business until something puts your business in front of it. That something is a retrieval step, and it is the part of the system that actually determines answer quality. The model is close to a constant across vendors. Your retrieval is not.

The questions here are unglamorous and decisive. Which systems hold the truth, and which hold a stale copy of it. How quickly a change in the source reaches the index, and what happens to an answer given in the gap. Whether a document that was correct last quarter is still returned today, and whether anything marks it as superseded. Whether the same fact appears in three places with three different values, which is common and which no model can resolve for you.

Most disappointing pilots are retrieval problems wearing a model costume. The usual symptom is an assistant that is right about the general shape of things and wrong about the specific case, which is exactly what happens when the general knowledge comes from training and the specific knowledge never arrived. We wrote up the retrieval design this site runs in a piece on RAG for consulting sites.

Who is allowed to see it

The fastest way to build a working demo is to index everything with one service account that can read everything. The fastest way to create an incident is to ship that.

Permissions have to be enforced at retrieval, not requested of the model in an instruction. A model asked politely not to reveal something will usually comply and cannot be relied on to, because the instruction and the content share one channel and anybody typing into the box is writing in that channel too. The only durable version of this is that a document the user may not read is never fetched into the context in the first place.

That means the identity of the person asking has to travel through the whole chain, and the index has to carry access metadata that stays current when a permission changes upstream. This is ordinary engineering, it is not hard, and it is routinely left out of proposals because it is invisible in a demo where one person is logged in as themselves.

What happens when nothing answers

Model providers fail slowly. A request does not usually error, it hangs, and the difference matters because a hang propagates upward into whatever called it. So the integration needs a defined behaviour for the case where the answer does not arrive: how long it waits, what the user sees, whether the request is retried, and whether a retry can repeat a side effect that already happened.

That last one is the sharp edge. If the step that timed out had already issued a refund, sent an email or written a record, a retry does it twice. Anything that changes state needs an identifier that makes the second attempt a no-op rather than a second action. This is standard practice in payments and it is unevenly applied in agent wiring, where the retry is often added later by somebody fixing a flakiness complaint.

There is also a design answer worth having ready: what the system does when the model is simply unavailable for an hour. A degraded mode that returns search results, or a queue, or an honest message is a decision. Falling over is also a decision, just not one anybody made.

How a wrong answer gets caught

Every deployment produces confidently wrong output eventually. The question that predicts cost is not how often, it is where that output goes next.

Sort the integration by what it may touch. Output a human reads before acting on is one class of risk. Output written into a system of record, sent to a customer, or used to move money is another class entirely, and no accuracy figure makes the difference between those two disappear. Enumerate the write actions before anything else is chosen, and decide for each one whether it may fire unattended, needs a person to confirm, or may only ever be proposed. That axis is worked through in a piece on sorting agents by what they may touch.

Then there is the question of noticing. A model integration degrades silently: no exception, no alert, just answers that are slightly less useful than they were, because a source changed shape or a prompt was edited or the provider shipped a new version. Catching that needs a set of real questions with known good answers, run on a schedule, with somebody who reads the result. It is cheap to build at the start and awkward to retrofit, and it is the line most often absent from a quote.

Context length is a constraint, not a strategy

A recurring shortcut is to skip retrieval and put everything in the prompt, on the grounds that the window is now large enough. It is a real option and it has real costs: every request carries the whole payload, latency scales with it, and relevant material buried in a long context is not attended to as reliably as material that was selected for being relevant.

Treat the window as a budget you are spending rather than a filing cabinet. If you want to see how the current models actually compare on that budget, the context window comparison lists them side by side, and the token counter turns a document you are thinking of pasting into a number before you commit to it.

What to ask a vendor

The proposals worth taking seriously answer these without being pushed:

Which systems will be read, how the index stays current, and what happens to an answer given while it is stale. How the asker's permissions are enforced at retrieval. What the timeout is, what the user sees when it fires, and which actions are protected against a repeated retry. Which actions the system may take without a person, named individually. What the evaluation set is, who reads it, and what threshold triggers a rollback.

None of that is about the model. All of it decides whether the integration is still in use next year, and a provider who has done this before will be relieved to be asked.