Configure Pydantic

Introduction

When you call an LLM and the response shape drifts — a missing field, a string where you expected a number, or a usage block that's suddenly absent — your downstream code crashes far from the source, often in production. Untyped LLM I/O is the most common cause of agent bugs that pass demos but surface during real customer interactions. By the end of this lesson you'll be able to declare Pydantic models for LLM responses, configuration, and structured outputs so that invalid data fails fast at the boundary with a clear ValidationError instead of corrupting your application state.

Key Terminology

• Type hint — Python annotation (e.g. name: str) that documents expected types; checked statically by tools like mypy but not enforced at runtime. Foundation for the type-safe APIs we build in this lesson.
• Pydantic model — a BaseModel subclass that turns type hints into runtime validators; parsing a dict into the model raises ValidationError on bad data, so corrupt LLM output never reaches business logic.
• Field constraint — extra validation rules attached via Field(...) (ge, le, gt, min_length, etc.); enforces invariants like 0.0 ≤ temperature ≤ 2.0 that bare type hints can't express.
• ValidationError — exception Pydantic raises when input fails validation; carries a structured list of which fields failed and why, so you can log, retry, or surface a useful message.
• BaseSettings — Pydantic's environment-aware model; reads required values from env vars at construction time and fails the process on missing config instead of None-crashing at first use.

Concepts

A type-safe LLM application has three validation boundaries — response, configuration, and structured output — and each one is a Pydantic model.

Modeling the LLM response

LLM JSON shapes vary between providers and versions. Defining LLMResponse(BaseModel) with nested choices and usage models fixes the shape your code sees. Optional fields use Optional[T] so absent keys parse cleanly rather than raising. Helper methods on the model (e.g. get_content()) keep call sites short and centralize the "what if the LLM returned nothing" handling (see Code Walkthrough).

Constraining configuration

API keys, model names, and sampling parameters all enter the app through environment variables. BaseSettings reads them at startup; Field(ge=0.0, le=2.0) on temperature rejects out-of-range values immediately. The process fails fast on bad config instead of producing garbled completions hours later.

Parsing structured LLM output

When you prompt an LLM to return JSON matching a schema, wrapping the parse in try/except for json.JSONDecodeError and ValidationError separates "model returned non-JSON" from "model returned JSON of the wrong shape." Both are recoverable with different strategies — re-prompt vs. fall back to a smaller model.

Code Walkthrough

Building on the three validation boundaries described in the Concepts section — response, configuration, and structured output — the following module implements all three in a single cohesive example.

Code snippetpython
1from pydantic import BaseModel, Field, ValidationError
2from pydantic_settings import BaseSettings
3from typing import Optional
4import json
5
6class LLMUsage(BaseModel):
7    prompt_tokens: int
8    completion_tokens: int
9    total_tokens: int
10
11class LLMChoice(BaseModel):
12    index: int
13    message: dict[str, str]
14    finish_reason: Optional[str] = None
15
16class LLMResponse(BaseModel):
17    id: str
18    model: str
19    choices: list[LLMChoice]
20    usage: Optional[LLMUsage] = None
21
22    def get_content(self) -> str:
23        if not self.choices:
24            raise ValueError("No choices in response")
25        return self.choices[0].message.get("content", "")
26
27class AppConfig(BaseSettings):
28    gemini_api_key: str = Field(description="Gemini API key")
29    temperature: float = Field(default=0.7, ge=0.0, le=2.0)
30    max_tokens: int = Field(default=1000, gt=0)
31
32class ExtractedEntity(BaseModel):
33    name: str
34    entity_type: str
35    confidence: float = Field(ge=0.0, le=1.0)
36
37class ExtractionResult(BaseModel):
38    entities: list[ExtractedEntity]
39    source_text: str
40
41def parse_extraction(llm_output: str, source_text: str) -> Optional[ExtractionResult]:
42    try:
43        data = json.loads(llm_output)
44        return ExtractionResult(
45            entities=[ExtractedEntity(**e) for e in data.get("entities", [])],
46            source_text=source_text,
47        )
48    except (json.JSONDecodeError, ValidationError) as exc:
49        print(f"Parse failed: {exc}")
50        return None
51
52raw = {
53    "id": "chatcmpl-123",
54    "model": "gemini-2.0-flash",
55    "choices": [
56        {"index": 0, "message": {"role": "assistant", "content": "Hi"}, "finish_reason": "stop"}
57    ],
58    "usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15},
59}
60resp = LLMResponse(**raw)
61print(resp.get_content())
62
63result = parse_extraction(
64    '{"entities": [{"name": "Python", "entity_type": "technology", "confidence": 0.95}]}',
65    "Python is widely used.",
66)
67if result:
68    for e in result.entities:
69        print(f"{e.entity_type}: {e.name} ({e.confidence:.0%})")

LLMResponse covers the response boundary: it defines nested models for choices and usage, marks both as Optional where the API may omit them, and centralizes the "what if choices is empty" guard inside get_content() so call sites stay simple. AppConfig covers the configuration boundary: BaseSettings reads gemini_api_key from the environment at startup, and Field(ge=0.0, le=2.0) on temperature rejects out-of-range values immediately — the process fails fast on bad config rather than producing garbled completions later. parse_extraction covers the structured-output boundary: catching json.JSONDecodeError and ValidationError separately distinguishes "model returned non-JSON" from "model returned JSON of the wrong shape," giving you two distinct recovery paths — re-prompt versus fall back.

You'll know it works when resp.get_content() prints Hi, the extraction loop prints technology: Python (95%), and passing a malformed payload — such as confidence=1.5 in the entity dict or omitting the id key from raw — raises a ValidationError that names the exact field that failed.

Do's and Don'ts

Do's

1. ✓Do define a Pydantic model at every LLM boundary — request, response, config, structured output. The cost is small; the diagnostic clarity when something breaks is large.
2. ✓Do use Field(...) constraints for ranges — ge=0.0, le=2.0 on temperature catches drift before the API call. Constraints are documentation that runs.
3. ✓Do catch ValidationError and JSONDecodeError separately — they imply different recovery (re-prompt vs. retry the parse). Conflating them costs you the signal.

Don'ts

1. ✗Don't pass raw dict objects through your app — every field access is unchecked. Parse into a model at the edge and propagate the typed object.
2. ✗Don't mark every field Optional to silence validation errors — that just moves the crash to a None-deref later. If your logic requires a field, declare it required.
3. ✗Don't validate the same shape in multiple places — one model, owned by the module nearest the boundary, imported everywhere it's needed.