Add fallback chains when primary extraction method fails — lab audio overview

Host: Welcome back to the GenBodha lab podcast. Today we're on the third and final lab of objective one-point-five in the Document Ingestion with VLMs chapter of our GenAI Data Pipelines course. The lab is titled "Add fallback chains when primary extraction method fails," and the project directory is add-fallback-chains-when-primary. This is an intermediate-difficulty lab, and it's where our routing system finally grows up and learns to survive contact with production. Expert: That's the perfect framing. In lab one, students built a classifier that routes documents to the right extraction method — native text parsing for clean PDFs, OCR for scans, a VLM for complex layouts. In lab two, they added cost awareness, so the router stops reflexively sending every document to the most expensive model. Lab three is the reliability layer. The student implements a ResilientRouter that wraps those earlier routing decisions with fallback chains, confidence-based retry logic, and per-method success and failure metrics. Host: Let's ground this in a concrete production scenario. Who actually needs a ResilientRouter? Expert: Picture a mid-size fintech or insurtech company running a document ingestion pipeline — they're processing loan applications, claims forms, medical records, vendor invoices. Tens of thousands of documents per day flowing through a queue. The data engineering team picked Textract as the primary extraction method because it's cheap and fast for the majority of clean documents. But one Tuesday morning, Textract has a regional degradation. Suddenly twelve percent of documents come back with empty fields or confidence scores below 0.3. Host: And without a fallback chain, what happens? Expert: Without a fallback chain, those documents either fail loudly — which floods the dead-letter queue and wakes up the on-call SRE — or worse, they succeed silently with garbage data, and that garbage flows downstream into underwriting models or claims adjudication. The blast radius is ugly. You get paged at 3 AM, you spend the morning replaying a backlog of forty thousand documents, and the compliance team asks uncomfortable questions about data lineage. The ML engineer or platform engineer who owns this pipeline is the one feeling that pain. Host: So what does the ResilientRouter actually do in code? Expert: The student builds a ResilientRouter class that composes on top of the CostAwareRouter from lab two. It holds a fallback chain — an ordered list of extraction methods per document class. For a scanned form, the chain might be Textract first, then AWS OCR as a secondary, then a VLM like GPT-4 Vision as a last resort. The core method, something like extract_with_fallback, invokes the primary method, inspects the returned confidence score, and if it's below a configurable threshold — say 0.7 — or if an exception was raised, it advances to the next method in the chain. Host: What patterns should the student focus on? Expert: Three big ones. First, the chain-of-responsibility pattern — each method gets a shot at the document, and the decision to pass it on is encapsulated. Second, the circuit breaker concept — if Textract has failed ten times in a row, the router should temporarily skip it rather than burning latency on every request. Third, the metrics layer. There's typically a MetricsCollector or a simple dictionary keyed by method name tracking attempts, successes, failures, and average confidence. Those metrics feed dashboards and also drive the circuit breaker logic. Host: Let's talk about the confidence-based retry. That sounds subtle. Expert: It is subtle, and it's where students often trip up. There's a difference between a hard failure — an exception, a timeout, a 500 from the API — and a soft failure where the method returns something but you don't trust it. The ResilientRouter has to handle both. For hard failures, you retry immediately with the next method. For soft failures, you compare the confidence against a threshold. The lab will have students implement both paths and think about when to log versus when to escalate. Host: What trade-off will the student really have to wrestle with? Expert: The big one is latency versus completeness. Every fallback you attempt adds seconds, sometimes tens of seconds if a VLM is at the end of the chain. If you set the confidence threshold too high, you'll cascade through the entire chain for documents that were actually fine, and your p99 latency doubles. Set it too low, and you accept low-quality extractions that should have been retried. The lab asks students to pick a threshold and justify it, and the answer genuinely depends on the document type and downstream tolerance for bad data. Host: Any other design decision worth flagging? Expert: Yes — whether fallbacks should run sequentially or in parallel. Sequential is cheaper because you stop as soon as one succeeds. Parallel is faster but you pay for every method every time. The reference implementation in this lab is sequential, but a thoughtful student will note the trade-off in their comments. And there's a related decision about whether to cache fallback outcomes — if Textract failed on document X, should the router remember that for document X's retries, or start the chain fresh? Host: What should students focus on when they're stuck? Expert: Read the TODO markers in resilient_router dot py carefully. The scaffolding walks you through building the fallback chain configuration, then the extract_with_fallback loop, then the metrics update calls. Don't skip the tests — they exercise the fallback path by deliberately making the primary method raise an exception, so they're your best signal that the chain is actually activating. And pay attention to how the router interacts with the CostAwareRouter from lab two — you're not replacing cost logic, you're layering resilience on top of it. Host: Great framing. This lab turns a clever router into a router you can actually put on-call. Expert: Exactly. When students finish, they'll have built the same pattern that keeps real document pipelines from paging their owners at 3 AM. That's the takeaway. Host: Thanks everyone — good luck with obj_01_5_lab_3.