Extracting structured data from PDF contracts

Prices, seasons and cancellation terms live in PDF files inside a folder. Finding anything means opening documents one at a time, and a question like "what does this cost in Ramadan" can take minutes, sometimes an hour.

The idea

Watch the document library, open every new file, and pull the values into a table: the price, the period, the tax, the fees, the cancellation terms.

Once that table exists, the question becomes a query and the answer comes back in seconds.

What works and what does not

Structured text files extract cleanly: clear tables, consistent headings.

Scanned images are beyond it, and so are chaotic layouts and nested tables.

The most important rule: fail loudly

The one error worth avoiding at any cost: extracting a wrong value and storing it as if it were right. A wrong price in the database causes far more damage than a missing one.

So the pipeline enforces a single rule: when extraction confidence is low, nothing is stored and the file gets flagged for manual review.

A structure that prevents duplication

Every file gets a fingerprint computed from its content. The same file arriving under a new name is skipped, and a file whose content has changed produces a new fingerprint and goes through processing again.

What I learned about dealing with partners

Once the system was running, I started asking for files in a clearer format up front. A single message to a supplier, requesting a structured table instead of a designed document, saved a month of extraction effort.

In my experience, improving the source has been the simplest way to improve the extraction.

Related reading