Handwritten vendor invoices: why OCR fails and what to do about it

A plumber finishes a job, writes an invoice on a duplicate pad in the cab of his truck, photographs it slightly crooked, and emails it from his phone.

That is a real payable for real work. It is also the document that defeats most AP automation, which then hands it back to you as an exception.

Generic OCR was built for printed text, and handwriting is a different problem

Optical character recognition assumes consistent glyph shapes. Printed text has them. A person writing quickly does not.

Three things break at once. Character shapes vary within the same document, so the same digit looks different twice. Baselines drift, so a line of text is not straight and segmentation guesses wrong. And there is no font to match against, which is the assumption most of the pipeline rests on.

Add photograph conditions to that. Shadow across half the page, a slight rotation, keystone distortion from holding a phone at an angle, and a dashboard as the backdrop.

The result is not a wrong number. It is usually a confidence score too low to act on, which routes the document to a human. That is the correct behavior, and it is also why the pile exists.

The fields that matter are the ones handwriting hurts most

An invoice needs a few things to post: vendor, invoice number, date, property or unit, line items, amount.

Handwriting degrades exactly the high-stakes ones.

Amounts are digits, and digits are where a misread is most expensive. A 1 read as a 7 is a $600 error on a $100 invoice.

Invoice numbers are often absent entirely on a handwritten pad, which is why "missing invoice number" is one of the five error types we report. Not a reading failure. The field genuinely is not there.

Unit numbers get abbreviated in ways only that vendor uses. "Apt 12B" and "12-B" and "unit 12b" are the same unit and three different strings.

Four things that actually improve the outcome

Ranked by how much they help relative to effort.

Read the handwritten ones with the same pipeline, not a separate one. BillRoute reads handwritten field vendor invoices alongside digital ones rather than routing them to a manual queue by document type. Most tools quietly do the second thing, and the giveaway is a "scanned documents" bucket in the interface. There is a worked example of one such page on handwritten invoices: seven amounts written down a carbonless pad, grouped onto the three account codes the approver stamped, posted at $1,060.00.

Learn the vendor, not just the layout. A template registered from one sample of that vendor's pad, with layout variants, does far more than improving general handwriting recognition. Their invoice looks the same every time because it is the same pad.

Accept the missing invoice number as a policy decision. Deciding to generate an internal reference for pad-invoice vendors removes a recurring exception that is not an error. Fighting for a field the vendor does not print is not a solvable problem.

Fix the photograph, not the recognition. Two sentences to a vendor about laying the invoice flat with light behind them removes more errors than any model change. It is also the least likely thing to happen, which is why it is fourth.

What we do not claim

Handwriting extraction is the least reliable path in the pipeline, and it should be.

A printed invoice from a vendor with a billing department extracts at very high confidence. A photographed handwritten pad from a truck does not, and a tool claiming otherwise is either not telling you its confidence threshold or is guessing on amounts, which is worse than an exception.

The right target is not perfect extraction. It is that the handwritten ones fail loudly with the reason attached, rather than passing quietly with a wrong number, and that they stay inside the same queue and audit trail as everything else.

Our 98% automation rate is a portfolio figure across all document types. The handwritten subset sits below that, and the 197 invoices in a representative month's error queue are disproportionately these.

The pricing angle nobody mentions

There is a billing-model wrinkle worth knowing.

A handwritten invoice is one page, so it is cheap under per-page pricing and expensive under per-invoice pricing, since it takes the most work to read.

That means a vendor charging per invoice has a real incentive to read it correctly rather than punt it to your error queue. A vendor charging per page does not. We charge $1 per invoice uploaded, so this is not a neutral observation, but the incentive structure is real regardless of who points it out.

How many handwritten invoices did your team key by hand last month?