AI-OCR for Fax Orders — Turning Paper Orders into Data

Introduction

"Fax, in this day and age?" you might think. But once you work in wholesale import operations, it's no surprise at all that orders from clients still arrive by fax.

Getting clients to abandon fax is far from easy. So the approach I chose was this: keep receiving faxes as they are, and automate everything that happens after they arrive with AI.

This article walks through the overall picture of a system that reads fax order sheets with AI-OCR and registers them automatically as order data.

The Reality of Fax Orders

Fax Is Still Alive in Wholesale

In B2B ordering, fax remains widely used. Retail clients have a long history of ordering by fax, and asking them to change a "we send by fax" workflow simply isn't realistic.

For the sender, fax is a convenient method: you can send handwriting or typed text, and it leaves a transmission record. Asking them to "please switch to a web form" purely for the recipient's convenience rarely moves them. That's the reality on the ground.

The Burden and Errors of Manual Entry

The problem lies in what happens next. Someone looks at the incoming fax and manually transcribes the item code, quantity, client, and requested delivery date into an order management spreadsheet. Even a few minutes per order adds up to substantial time when repeated daily.

Even scarier are transcription errors. A single wrong character in an item code or a misplaced digit in a quantity leads directly to misshipments or inventory discrepancies. It's not that "the person who typed it is at fault" — the very act of humans transcribing is the risk.

Not "Abolish Fax" but "Change What Comes After"

To automate without changing the client's workflow, you only need to replace the internal steps that happen after a fax arrives. Fortunately, modern fax multifunction machines can convert incoming faxes into PDFs and forward them by email.

In other words, a "paper fax" can effectively become "a PDF arriving by email." Once you're here, the rest is a software problem — fetching the email, reading its contents, and writing to the sheet can all be automated.

The Overall Picture

How a Fax Becomes Data

The flow works like this. The fax machine forwards a PDF by email; that email is fetched periodically via the Gmail API. AI (Claude) reads the PDF and turns it into structured data, then writes it into the order sheet according to per-client settings.

Three Building Blocks

This system is built from three main parts. The first is the email intake pipeline — the foundation that monitors the inbox and picks up unprocessed fax PDFs without missing any.

The second is structured extraction with an LLM. From order sheets with wildly varying layouts, it extracts item codes and quantities as data in a fixed format.

The third is routing rules and misread recovery — managing which client's fax goes to which sheet, plus a reprocessing mechanism to fix mistakes quickly when the AI misreads. Each is covered in detail in its own article.

Why an LLM Instead of a Dedicated OCR Service

Robust Against Layout Variation

Traditional OCR services mostly required defining the form layout in advance — "item code goes here, quantity goes there." But fax order sheets use different formats per client, and often come with handwritten notes.

Because an LLM "reads and understands" the document, it can judge from context that "this is the item code, this is the quantity" without any layout definition. The big advantage is that when a new client with a new format appears, you don't have to rebuild a template.

It Can "Read Between the Lines"

Faxes contain expressions a human can grasp but a machine struggles with, like "the usual item, 10 units" or "same terms as last time." An LLM can, to some extent, work these out — for example, "the quantity field is blank, but the body says 10 cases."

Of course it isn't infallible. That's exactly why the extracted results go through mechanical validation — required-field checks and matching against the item master — to back up their reliability.

We Still Keep a Human in the Loop

No matter how much accuracy improves, order intake is a task tied directly to money and trust. In this system, data written by the AI is confirmed by a person before it's finalized.

What matters is that the human's job shifted from "transcribing" to "confirming." Entering from scratch versus reviewing finished data are worlds apart in both effort and speed. To me, automation isn't about "removing people" — it's about "moving people to better work."

Design Lessons from Operating It

Misreads: "Fixable Fast" Beats "Never Happens"

At first, all I thought about was raising reading accuracy. But operating it taught me an obvious truth: you can't eliminate misreads entirely. Faded faxes, quirky abbreviations, and unexpected formats will always come.

So I changed course and built a mechanism where "the moment you notice a misread, you can reprocess it on the spot." A design that finishes recovery in 10 seconds does more for peace of mind on the floor than shaving 0.1% off the misread rate, doesn't it?

Push Per-Client Differences into "Settings"

Each client has a different destination sheet and column format. Hard-coding these means reworking the code every time a client is added.

So I made it possible to identify the client from the sender's fax number or email address, and to manage the destination and format through a "routing settings" admin screen with full CRUD. Adding a new client is done purely by adding a setting — no code changes.

A Place to Try Builds Trust

Another thing that helped was a test UI where you can upload a single PDF and check the OCR result. Because you can try "can this fax be read correctly?" without pushing it to production, it's invaluable for checking new client formats and investigating misreads.

Team members being able to try it themselves and go "ah, so that's how it reads" — this quiet detail is essential to getting people to trust and use an AI system.

Conclusion

I've introduced the overall picture of AI-OCR auto-intake for fax orders.

1. Don't abolish fax: leave the client's workflow alone and automate the internal steps after arrival
2. Read with an LLM: robust against layout variation, structuring data with no template needed
3. Humans move to confirming: from transcribing to confirming, with misreads caught by a "fixable fast" design

Paper culture and automation aren't in conflict — insert "PDF + AI" between them and both can coexist. That's the biggest takeaway from this project.

The details are covered in each article. For the foundation from email receipt to PDF fetching, see "Designing the Intake Pipeline from Email Inbox"; for the reading itself, see "Structured Data Extraction from Forms with LLMs"; and for per-client rules and misread handling, see "Per-Client Routing Rules and Misread Recovery."