Build extraction-method adapters producing the unified document model — lab audio overview

Host: Welcome back to the GenBodha lab podcast. Today we're digging into lab 2 of objective 1.4 in the GenAI Data Pipelines course — "Build extraction-method adapters producing the unified document model." This sits in our Document Ingestion with VLMs chapter, and it's the connective tissue between three very different extractors and the rest of your pipeline. Expert: Right, and the framing matters. In lab 1 of this objective, students defined a format-agnostic document schema — typed content blocks for paragraphs, tables, figures, headings, and so on. That schema is just a contract. It doesn't actually do anything until something fills it in. That's lab 2's job. Then lab 3 will take the adapters we build today and validate them against documents processed by all three extraction methods end to end. Host: So lab 1 was the data contract, lab 2 is the translation layer, lab 3 is the cross-method validation. Let's talk about what the student actually builds today. Expert: The student will build three adapter classes — one for Docling, one for vision-language models accessed through OpenAI and Gemini, and one for Google Document AI. Each adapter takes the raw, vendor-specific output from its extractor and produces an instance of the unified document model from lab 1. On top of that, students implement a factory that takes an extraction method name — a string like "docling" or "gemini_vlm" or "google_docai" — and returns the right adapter instance. Host: That factory pattern is the part I want to underline. Why is it worth its own component instead of just an if-else chain in the pipeline code? Expert: Because the consumers of this code shouldn't know or care which extractor produced a document. Imagine your downstream chunker, your embedder, your table-to-markdown converter — they all want to operate on the unified model. If each of them has to branch on extractor type, you've leaked the vendor abstraction into every layer. The factory localizes that decision to one place. Add a fourth extractor next quarter — say, Azure Document Intelligence — and you write one new adapter and register it in the factory. Nothing else changes. Host: Let's ground this in production. Who actually needs this and what breaks without it? Expert: Picture a mid-size insurance company processing claims documents. A data engineering team runs a pipeline that ingests scanned PDFs, faxes, and digital forms. Different document types route to different extractors — Docling for clean digital PDFs because it's cheap and fast, Google Document AI for structured forms with known schemas, and a VLM for messy handwritten or photographed pages. Without an adapter layer, the downstream code — entity extraction, claim routing, fraud scoring — is full of conditional logic checking which extractor ran. When Docling releases a new version that renames a field from "text" to "content", every downstream service breaks at once. The on-call data engineer gets paged at 2 AM, claim processing backs up, the SLA with the claims operations team gets violated, and now the platform team is in a war room rewriting consumer code under pressure. Host: And with the adapter pattern? Expert: You update one adapter file. Run your unit tests. Ship it. The blast radius shrinks from the entire pipeline to a single class. This is the workflow of an ML platform engineer or a senior data engineer who owns ingestion. They live in this code daily — adding new extractors, handling format drift from vendor APIs, and onboarding new document types from business stakeholders. Host: What are the key classes the student should focus on? Expert: There's a base adapter — likely an abstract class with a single method, something like to_unified_document, that takes raw extractor output and returns the unified model from lab 1. Then three concrete subclasses. The Docling adapter walks Docling's hierarchical document tree and maps each node type to a content block. The VLM adapter is interesting because OpenAI and Gemini return free-form structured JSON — the adapter has to parse that, validate it against the schema, and gracefully handle when the model hallucinates a field or skips one. The Document AI adapter pulls from Google's protobuf-based response, walking entities, page anchors, and bounding boxes. Host: That VLM case sounds like the trickiest one. Expert: It is, and that's where one of the key design decisions comes in. When a VLM returns malformed output — a missing required field, a bogus content type — do you raise an exception and fail the document, or do you log a warning and produce a partial document? Both are defensible. Failing loudly catches regressions in your prompt engineering early. Producing partial documents keeps the pipeline flowing and lets human reviewers handle edge cases downstream. The student will face this trade-off explicitly in the lab, and the right answer depends on whether your pipeline is batch or real-time, and whether downstream consumers can tolerate gaps. Host: Any other trade-offs worth flagging? Expert: Yes — adapter granularity. Should bounding box coordinates from Document AI be preserved in the unified model, or normalized away? If you preserve them, downstream rendering is easier but the schema gets heavier. If you drop them, you lose the ability to highlight regions in a UI later. Students will need to think about what their unified model actually promises to consumers. Host: What should students focus on as they work through this? Expert: Three things. First, keep the adapters thin — they translate, they don't transform business logic. Second, the factory should be data-driven, not a giant switch statement; a registry dictionary that maps method names to adapter classes is cleaner and makes lab 3 much easier when you're iterating across all three. Third, write your adapter outputs through the unified model's own validation from lab 1. If the schema rejects your adapter output, that's the bug — not a reason to bypass validation. Host: Perfect. Lab 2 of objective 1.4 — build the adapters, build the factory, and set yourself up for the cross-method validation in lab 3. Good luck. Expert: Have fun with it.