Parse PDF documents using Docling with layout analysis on CPU — lab audio overview

Host: Welcome back to GenAI Data Pipelines. This is the course about the plumbing behind production AI systems — the unglamorous but critical layer that moves real-world data into models that actually work. This chapter is called Document Ingestion with Vision-Language Models, and it's about getting messy documents — PDFs, Word files, web pages — into clean text that an AI system can actually reason over. This first skill, turning a PDF into structured text, is something every data team hits the moment they move past toy demos. Expert: Let me paint the picture. Imagine you're on a small data team at a mid-sized insurance company — maybe eight engineers. Leadership wants an AI assistant that can answer questions about policy documents, claims reports, and regulatory filings. You've got a folder with fifty thousand PDFs. Some are clean, modern exports. Some are scanned from the nineteen-nineties. Many have tables — premium schedules, coverage grids, financial summaries — where the meaning lives in the rows and columns, not just the words. If you extract the text naively, you get a jumbled mess. Column headers end up next to random body text. Tables collapse into gibberish. Footnotes interrupt paragraphs mid-sentence. And when you feed that mess into a language model, it hallucinates, because the structure that gave the words meaning is gone. This is the problem that breaks most early AI projects. Getting the document ingestion layer right is the difference between a demo that looks cool and a system your team can actually put in front of customers. Host: So this is the starting point for the chapter — the overview set the stage, and now we get specific. What exactly are we building in this first exercise? Expert: You're going to build a piece of code that takes a PDF and produces a clean, structured version of its contents — the text, the tables, the figures, all labeled and organized so a downstream system knows what each piece is. The tool you'll use is called Docling. Docling is an open-source library, originally from IBM, that acts as a universal document reader. You hand it a PDF, and it hands you back a structured representation where paragraphs are paragraphs, tables are tables, and headings are headings. Now, here's the key idea — the conceptual shift. Most people think of a PDF as text plus images. But a PDF is really a bag of visual marks on a page. There's no built-in notion of "this is a table" or "this is a heading." To recover that structure, you need a model that looks at the page the way a human does — visually — and identifies the regions. That's where the layout model comes in. You'll configure a small vision model called Granite-Docling, with two hundred and fifty-eight million parameters, which is small enough to run on a regular laptop processor with no graphics card required. It scans each page, labels the regions, and Docling stitches the results into a clean structured document. Finally, you'll save information about what you parsed — filename, page count, when it was processed — into a database called PostgreSQL, so your team has a record of what's been ingested. Host: Before people start — what's the part that trips them up on the first try? Expert: The thing that catches people is performance expectations on the processor. This vision model is small by modern standards, but it's still doing real work on every page — looking at pixels, classifying regions, extracting tables. On a laptop without a graphics card, a single multi-page PDF can take tens of seconds to process. People assume something's broken and start killing the process. It's not broken — it's just that vision models are computationally heavy. The tip: start with a short document, maybe two or three pages, to confirm the pipeline works end to end. Once you see clean structured output come back, then try a longer file. And when you're processing many documents, do them in batches rather than one giant run, so you can see progress and recover if something fails partway through. Host: Great. So what walks out the door when this is done, and what comes next? Expert: After this exercise, you'll be able to take any PDF and turn it into clean, structured text where tables, headings, and figures are all properly labeled — running entirely on a normal laptop, no cloud service, no special hardware. That's a real capability. For your team, this is the foundation block that every document-based AI system sits on top of — search over internal documents, question-answering over policy manuals, automated summarization of reports. Without this layer working well, none of those systems work well. In the next exercise, you'll extend what you built here to handle Word documents, slide decks, and web pages, all flowing through the same unified pipeline, so your team can ingest whatever format the business throws at them. Then in the third exercise, you'll compare quality across document types to understand where this approach shines and where you need something heavier. For now, focus on getting one clean PDF through the pipeline. See the structure come out the other side. That moment — when you see a messy document become organized data — is the moment this chapter clicks. Good luck, and thanks for listening.