AI Document Processing: Where the Human Stays

Every page in this category describes the same pipeline. Capture the document, classify it, extract the fields, validate them, push the result into the system of record. The description is accurate and it is not the hard part, because the pipeline has been buildable for a while now and the models have stopped being the constraint.

What decides whether one of these projects is worth having is narrower and less discussed: which fields a person still looks at, how those fields get chosen, and what you are measuring to know the arrangement is holding. Get that wrong and you have automated the typing and kept all of the risk, which is the most common outcome we are called in to look at.

Sort your documents by how they fail, not by what they are

The usual sorting is by document type: invoices, purchase orders, claims, contracts, onboarding forms. It is the wrong axis for planning the work, because two documents with the same name can behave completely differently depending on who produced them.

The axis that predicts effort is how much the layout varies between instances. A form your own organisation issues is effectively a fixed grid, and extraction from it is close to solved. A supplier invoice is semi structured: the same facts are present every time and they move around the page depending on which supplier sent it. Free text, correspondence and contracts hold the facts in prose, so the question stops being where on the page and becomes which clause, which is a genuinely different job. Handwriting and poor scans sit across all three and set a ceiling nothing in the software can lift.

Sort a pile that way and the project usually shrinks. Most of the volume is in one or two classes, those classes are usually the predictable ones, and the difficult class is often the one with the fewest documents in it and the most judgement attached to each.

A single accuracy number tells you almost nothing

The question every finance or operations lead asks early is how accurate it will be, and a single percentage is the least useful answer available, whoever is quoting it.

Errors are not spread evenly. A model reads a date and a company name more reliably than it reads a total buried in a table with a discount line above it, so an overall figure averages the fields that never go wrong with the ones that decide what you pay. The figure also depends entirely on which documents it was measured against, and a vendor measuring on clean samples is not describing your post room.

What is worth measuring instead is per field, on the fields that move money or create an obligation, against documents drawn from your actual intake including the bad ones. Two numbers matter more than the headline: how often a wrong value is produced, and how often a wrong value survives review and lands in the system of record. Only the second one has ever cost anyone anything.

Confidence routing is the whole design

The useful mental model is not a system that extracts documents. It is a system that sorts documents into ones it is confident about and ones it is not, and the quality of that sorting is what you are buying.

A model that is right most of the time and knows when it is unsure is far more valuable than one that is right slightly more often with no sense of its own uncertainty. The first can be run with a review queue sized to the uncertain cases. The second has to be reviewed in full, which removes the reason for building it.

So the design question is where the threshold sits per field, and that is a business decision rather than a technical one. A delivery date read wrongly is an annoyance somebody catches. A bank account read wrongly is a payment to the wrong place. Those two do not deserve the same threshold and should not share one, and in practice this is where most of the configuration effort in a serious deployment goes.

The validation nobody budgets for catches more than the model

Before reaching for a better model, check the extracted values against the data you already hold. Does this purchase order number exist. Does the supplier name resolve to a supplier you have. Do the line items sum to the stated total. Is this invoice number one you have already paid.

These checks are ordinary software, they are cheap, and they catch a class of error no model improvement reaches, because they are not asking whether the reading was accurate. They are asking whether the result is possible. A total that does not match its own lines is wrong regardless of how confidently it was read, and a system that knows that can route the document to a person instead of passing it on.

This is also the part that makes a review queue tolerable to the people staffing it. A queue that says this document needs checking is tiring. A queue that says this total does not match the lines on the document is a specific question with a quick answer.

Where the cost actually sits

Budgets for this work tend to be written around the model and the integration, and then the running cost turns out to be somewhere else: the review step, in people hours, every day, for as long as the system runs.

That has a design consequence. Anything that lowers review time pays back continuously, which makes the interface the reviewer uses a first class part of the build rather than an internal tool somebody throws together at the end. Showing the document with the extracted value highlighted in place, so a check is a glance rather than a hunt, changes the economics of the whole thing more than a marginally better extraction rate does.

It also means volume is the first question to ask before any of this is justified. A process handling a steady stream of similar documents repays automation. A process handling a trickle of varied ones, each needing a judgement call, mostly repays a better template and a clearer inbox.

What we build first, and what we refuse to build

The first build is classification plus one document class, end to end, including the review queue and the writeback. One narrow slice in production teaches more in a fortnight than a broad pilot does in a quarter, because it surfaces the parts nobody budgets for: the documents that arrive as photographs of screens, the supplier who changed their template, the exception handling nobody specified.

Alongside it, a labelled set drawn from documents your organisation has already processed. Most clients believe they have no ground truth and in fact have years of it, in the form of documents and the values somebody keyed from them. That set is what turns accuracy from a vendor claim into something you can check yourself, and it keeps working as a regression test every time anything in the pipeline changes.

What we will not build is a straight through path on a field that creates a legal or financial obligation without a human in it, on the strength of a good evaluation. The evaluation describes the documents you have seen. The expensive failure is always the document shaped slightly differently from all of them, and that is precisely the one a confidence threshold is for.

If you are earlier than this and still deciding whether to automate a process at all, our automation services piece covers that prior question, and the integration piece covers how a system like this attaches to the software you already run. For the measurement habit underneath all of it, the evaluation piece is the one that matters most here.