Define Pydantic models for use case scoring criteria, weights, and structured LLM responses — lab audio overview

Host: Welcome back. You're in AI Solution Delivery — a course about the full lifecycle of turning a client's vague AI ambition into a shipped system. This chapter is called AI Use Case Discovery and Data Readiness Assessment. And the skill you're about to practice — designing the data shapes that drive a scoring tool — is the quiet foundation that keeps every downstream system honest. Expert: Let me paint the scene. Imagine you're a senior engineer at a consulting firm, and your team of five is running a two-day discovery workshop with a large insurance client. The client walks in with forty ideas for where they want to use AI — everything from claims triage to marketing copy. Your job, by Friday, is to tell them which five are worth piloting. Now, you could do this in a spreadsheet. Teams have done it that way for years. But spreadsheets fall apart the moment you want consistency across twenty clients, or you want a large language model — the kind of AI that can read and reason about text — to help score use cases automatically. Without a clean, enforced structure behind your scoring, you get chaos. One consultant rates complexity on a scale of one to five, another uses low-medium-high, and the language model you bring in to help? It hallucinates a score of "pretty good." None of it compares. None of it ranks. And the client walks away with a recommendation they can't trust. Host: So that's the problem — inconsistency at every layer, human and machine. This is the first exercise in a three-part sequence, which means we're starting at the foundation. No prior work to build on yet. So what exactly are we building today? Expert: You're building the blueprints. Think of it like this — before a construction crew pours concrete, an architect defines the exact dimensions of every room, every doorway, every load-bearing wall. What you're building today are those architectural specifications, but for data. You're going to define three specific shapes. First, a shape that describes what a scoring criterion looks like — things like business value, technical feasibility, data availability — along with how much each criterion weighs in the final score. Second, a shape that tells the language model exactly what to give back when you ask it to evaluate a use case. And third, a shape for the final scored result — the use case, its scores across every criterion, an overall ranking, and a confidence range showing how certain the model is about its own answer. Now, the tool you'll use to express these shapes is called Pydantic. Pydantic is a Python library — widely used in production AI systems — that lets you declare the exact structure of your data. Field names, types, allowed values, minimums, maximums. And once you've declared that structure, Pydantic enforces it. If something tries to pass in bad data, Pydantic rejects it at the door. You'll pair this with a second library called Instructor, which is a tool that forces a language model's output to match a shape you've defined. So instead of getting back a paragraph of prose that you have to parse with fingers crossed, you get back structured, validated data every single time. Here's the key idea — the conceptual shift. Most beginners think of data models as documentation. They're not. In this system, the models ARE the contract. They're the thing that guarantees your scoring engine, your language model, and your final report are all speaking the same language. Get the shapes right, and everything downstream becomes simple. Get them wrong, and you'll be patching bugs for weeks. Host: That's a really useful framing — the shapes aren't documentation, they're the contract. Before folks dive in, what trips people up on this kind of foundational exercise? Expert: Two things. The first is over-engineering. Because this is the foundation, there's a temptation to add every field you might ever want — alternative names, historical tracking, metadata about who created the criterion. Resist that. Start with exactly what the next exercise will need: the criterion, its weight, a score, a brief justification from the model, a confidence level. You can always extend later. A model that does one thing cleanly beats a model that does ten things vaguely. The second trip-up is being too loose with your value ranges. When you're defining something like a score or a weight, it's tempting to just say "this is a number." But if you don't set a minimum and a maximum — say, a score must be between one and ten, and all the weights across your criteria must add up to one — then a language model can and will hand you back a score of forty-seven, or a set of weights that sum to three. Lock those constraints down at the shape level. That way, when the language model hallucinates — and it will — your data layer catches the error immediately, before it poisons a downstream calculation. Think of these constraints as guardrails on a mountain road. They feel restrictive until the moment they save you. Host: Great advice. So to wrap up — after this exercise, you'll be able to take a messy, human, subjective evaluation process and give it a structured backbone that both consultants and language models can speak into reliably. This is the kind of building block your team can reuse across every client engagement — a shared vocabulary for how you rank AI opportunities. Next up, in exercise two, you'll take these shapes and plug them into the actual scoring engine — the piece that sends a use case to a language model, applies your weights, and produces the final ranked score. Today's the blueprint. Next time, you pour the concrete. Good luck, and thanks for listening.