Create quarantine workflow and contamination report — lab audio overview

Host: Welcome back to GenAI Evaluation, Safety and Governance — the course about making sure the AI systems your team ships actually behave the way you promised they would. You're in the chapter on evaluation dataset curation, which is all about building and maintaining the test sets you use to grade your models. And today's skill — handling test questions that have been contaminated — is something every serious evaluation team has to solve before their numbers can be trusted. Expert: Let me paint the scene. Imagine you work on a mid-size AI team — maybe fifteen engineers — and you've just run your quarterly benchmark. Your new model scores ninety-two percent on your reasoning test. Leadership is thrilled. Then a researcher pings you on Slack: "Hey, I think some of those test questions showed up in the training data the model provider used." Your stomach drops. Because if even ten percent of your test questions leaked into training, your ninety-two is meaningless — the model didn't reason, it remembered. Now you need to answer three questions, fast. Which specific questions are contaminated? What's the real score if you remove them? And can you prove, with evidence, which ones you pulled and why? Without a system for this, you're doing it in a spreadsheet at midnight. With a system, it's a two-minute report. That's what you're building today. Host: So this is the third and final exercise in this objective. In the first two, you built a contamination detector that probes a model to see if it has memorized questions, and you built a scoring method called ROUGE-L that measures text overlap between what the model produces and what the correct answer looks like. Now you're putting it all together. What exactly are we building? Expert: Two things, working as a pair. The first is what we'll call a quarantine system. Think of it like a medical quarantine — when you suspect a patient is contaminated, you don't throw them away, you isolate them in a separate room with a full chart of symptoms and evidence. Same idea here. When your detector from the earlier exercises flags a test question as probably contaminated, the quarantine system pulls that question out of the active test set, moves it to an isolated holding area, and attaches an evidence record — which model flagged it, what the overlap score was, when it happened, and why. Nothing gets deleted. Everything is auditable. The second piece is a reporting system. It reads everything in quarantine and produces aggregate numbers — how contaminated is your dataset overall, broken down per model and per category. So you can say things like "our math questions are eight percent contaminated on one model but thirty percent on another." And finally, you'll expose all of this through a web endpoint — meaning another service, or a dashboard, or a teammate's browser, can ask your system for the latest report over the network and get a clean answer back. The web framework you'll use for that is called FastAPI, a popular Python tool for building web endpoints quickly. Host: Okay, so what's the big conceptual idea here — the thing that makes this click? Expert: The key idea is separation of detection from decision. Your earlier exercises built detectors — they produce signals, scores, probabilities. But a signal is not a verdict. A question might score high on overlap and still be legitimately in your test set, maybe because it's a common phrasing. So you never want your detector to directly delete things. Instead, the detector raises a flag, the quarantine system captures the flag with full evidence, and a human — or a policy — makes the final call about whether to permanently remove it. This is the same pattern security teams use: detect, isolate, investigate, then act. It keeps your evaluation process defensible. When an auditor or a skeptical executive asks "why did you drop these twelve questions from the benchmark," you have a complete paper trail. Host: Before we wrap — what's the tricky part? What trips people up in this exercise? Expert: The trap is treating the evidence record as an afterthought. People focus on the logic — detect, move, done — and store the bare minimum about why something was quarantined. Then three months later, someone asks "why is this question in quarantine?" and the record just says "contamination: true." Useless. So: when you write the piece that captures evidence, be generous. Store which model triggered it, the exact overlap score, the category the question came from, a timestamp, and ideally a short reason string. The second tripwire is in the reporting piece — make sure your per-model and per-category rates are computed against the right denominator. The rate means contaminated questions divided by total questions in that slice, not divided by total quarantined items. Getting that denominator wrong makes your whole report misleading. Host: Great. Bring it home for us — what will the listener be able to do after this, and what comes next? Expert: After this exercise, you'll be able to take a raw evaluation dataset, detect contaminated questions automatically, isolate them with full evidence, and produce a trustworthy contamination report that another service can pull over the network. That's a complete, production-grade pipeline. For your team, this becomes the foundation of defensible benchmarking — the thing you point to when someone asks whether your model scores are real. And because this is the final exercise in the objective, you now have the full picture: detection, scoring, and governance, wired together. You can bring this pattern straight into your team's architecture discussions about how evaluation should work at scale. Thanks for listening, and good luck with the build.