Lab overviews

Build an isolated Python virtual environment programmatically, install a pinned package into it, and verify that the package imports only inside the venv while its interpreter reports a prefix separate from the base install.

Build a key manager that loads LLM provider API keys from environment variables with a secret-file fallback, fails loudly when a required key is missing, redacts keys for safe logging, and uses the loaded key to make an authenticated LLM call.

Async client patterns: exponential backoff, retry loops, and concurrent batching with asyncio, verified deterministically against a scripted transport. Complete the TODO implementations to pass all tests.

Build a client that turns on Anthropic extended thinking and separates the thinking trace from the final answer, then request strict JSON output, parse it into a typed weather report, and reject malformed replies.

Build Pydantic models that represent platform service catalog entries including service metadata, configuration schemas, dependency declarations, and tier classifications for the AI developer platform.

Create a template registry that stores and validates golden path templates, including template versioning, step ordering, and parameter validation for AI workflow automation.

Build a catalog entry validator that uses JSON Schema to validate service configuration parameters, providing detailed error messages for invalid entries and supporting schema evolution.

Build cursor-based pagination for the service catalog API supporting configurable page sizes and stable traversal of large catalogs.

Implement text search across service catalog entries with relevance ranking, prefix matching for autocomplete, and combined search with structured filters.

Implement ETag-based conditional requests for the service catalog, supporting cache validation with If-None-Match and optimistic concurrency control with If-Match headers.

Add Prometheus counter and histogram instrumentation to platform API endpoints, tracking request counts, error rates, and latency distributions.

Create histogram-based latency tracking for provisioning operations with configurable bucket boundaries, percentile computation, and alert threshold evaluation.

Build Grafana dashboard configurations as code, defining panels for request rates, error rates, latency percentiles, and active service counts with PromQL queries.

Define a structured prompt injection taxonomy using Pydantic models. Build enums for injection vectors (direct, indirect, context-manipulation), severity levels with numeric scores, and a DetectionResult model that carries classification metadata for downstream guard chain decisions.

Implement a deterministic injection detector using compiled regex patterns and keyword rules. Build a PatternRule registry, a multi-pattern evaluator that returns the highest-severity match, and a FastAPI endpoint for real-time pattern scanning.

Build an LLM-as-judge classifier that uses LiteLLM to perform semantic injection detection. The judge analyzes user inputs for novel or obfuscated injection attempts that bypass pattern matching, returning structured JSON classification results.

Implement a guard chain orchestrator that runs an ordered set of independent, deterministic injection detectors (a keyword-pattern guard and a structural-anomaly guard) over input text. Each guard has a kind, weight, and required flag. The orchestrator collects per-guard verdicts and combines them w

Add short circuit logic to the guard chain so high-confidence detections immediately block requests without running slower downstream guards. Implement latency budgets and Redis-based caching for repeat queries.

Build a guard result aggregator that computes weighted confidence scores from multiple guard results. Implements configurable decision thresholds with Prometheus metric instrumentation for monitoring aggregation decisions.

Package the injection defense pipeline as a FastAPI microservice with scan, health, and configuration endpoints. Build async request handlers for maximum throughput with proper error handling and response models.

Create a Python-based Helm chart generator that produces Kubernetes YAML manifests for the defense sidecar deployment. Includes Workload Identity configuration for secure GKE authentication without stored credentials.

Configure Horizontal Pod Autoscaler for the defense guard service. Generate HPA YAML with CPU-based scaling, custom metrics for guard chain latency, stabilization windows, and scale-down policies.

Build Pydantic models that define the data structures for use case scoring: criteria definitions with configurable weights, structured response schemas for validated evaluations, and scored use case results. Then implement the deterministic weighting, validation, and ranking logic that turns raw eva

Build the UseCaseScoringEngine class that ranks proposed AI use cases with deterministic weighted multi-criteria scoring. The engine registers weighted criteria, normalizes the weights, computes each use case's weighted-sum aggregate, and produces a ranked best-first list. Pure model-free arithmetic

Build the request-handling layer of a use case scoring API. Implement an evaluate handler for single use case scoring, a batch handler for batch evaluation with ranking, and a criteria handler that exposes normalized scoring weights. Every handler is deterministic: it validates the request and retur

Rebuild lab_1's interview-state-machine on LangGraph's StateGraph (define question_generation/response_capture/follow_up_analysis/insight_extraction nodes with edges + conditional routing over InterviewState) instead of the hand-rolled next_state/advance dispatch; the existing LLM insight-extraction

Implement deterministic structured insight extraction from discovery interview transcripts. Students build an extractor that classifies each transcript utterance into pain points, opportunities, or requirements using keyword-marker rules with a fixed precedence order, then aggregates the insights in

Build a deterministic save-and-resume API for discovery interview sessions. Students implement an in-memory session store and a stage state machine — create a session, record answers, advance through a fixed stage sequence (completing at the final stage), snapshot a session to a plain dict, and rebu

Build a deterministic multi-provider router that maps model names to providers (OpenAI, Gemini, Anthropic), resolves each provider's proxy base URL from configuration, produces a complete routing decision, and computes an ordered fallback chain across configured providers. No LLM is called — routing

Build a deterministic benchmarking aggregator that turns recorded provider probe samples into a comparison: compute each sample's USD cost from a fixed pricing table, aggregate per-provider latency and cost stats, identify the fastest and cheapest provider, and filter providers by a latency budget c

Build a deterministic feasibility analyzer that scores candidate providers (each pre-rated on quality, speed, and cost) with a weighted formula, gates them on compliance and a minimum-score bar, compares two providers head-to-head, and ranks all candidates to recommend the best feasible one. No LLM

Design Jinja2 report templates that aggregate scored assessment components into a formatted Markdown report. Students aggregate components into a weighted overall score, map scores to rating bands, and render one component and the full report deterministically with Jinja2 — no LLM involved, byte-for

Build a report assembly pipeline that merges data from multiple assessment components into a unified discovery report. Implement pipeline stages for data validation, cross-component correlation, conflict resolution, and sequential section assembly using Pydantic models and Jinja2 rendering.

Build a FastAPI-based report generation service that accepts discovery data via REST endpoints, renders reports using Jinja2 templates, converts markdown to HTML, and provides multiple export formats. Implement request validation with Pydantic models and structured error handling for the report gene

Build a Python tool that initializes a Git repository with trunk-based development conventions. Create a standard AI project directory structure (prompts, model-configs, eval-datasets, k8s), configure Git settings for rebase-by-default and autostash, deterministically classify candidate branches aga

Build a Python validator that enforces feature branch naming conventions for AI projects. Validate branch names against patterns like feature/, fix/, config/, infra/, deploy/, eval/. Parse branch names to extract type and description components. Report invalid branches with helpful error messages su

Build a Python tool that manages the complete lifecycle of a feature branch: creation from main, commit tracking, squash merge simulation, and cleanup. Track branch metadata including age, commit count, and files changed. Detect stale branches that exceed the maximum age threshold.

Build a Python tool that deterministically generates and audits pre-commit hook configurations for AI projects. Render .pre-commit-config.yaml with hooks for ruff, mypy, yamllint, detect-secrets, and JSON validation, validate prompt template files against a Pydantic schema, and audit a rendered conf

Build a Python tool that generates Renovate configuration and Kubernetes CronJob manifests for automated dependency management. Create renovate.json with AI-specific package grouping rules. Generate the Kubernetes YAML for deploying Renovate as a GKE CronJob.

Build a Python tool that compares Renovate and Dependabot across multiple dimensions for AI project dependency management. Generate comparison matrices, score each tool on relevant criteria, and produce a recommendation based on project requirements.

Build the Pydantic data models that define the chat request schema and SSE frame formatting. You will create ChatMessage, ChatRequest, and StreamChunk models with field validators for provider names and temperature ranges, plus an SSE frame formatter that outputs W3C-compliant event-stream strings.

Implement the core streaming endpoint at POST /api/v1/chat/stream using an async generator that yields SSE-formatted token strings. You will build the stream_tokens() async generator, wire it into a FastAPI StreamingResponse with media_type text/event-stream, and create the FastAPI application with

Configure CORSMiddleware on the FastAPI application for browser EventSource clients and implement a GET /api/v1/chat/stream/health endpoint that returns the streaming service status. You will also add startup and shutdown lifespan events for graceful resource management.

Build a GeminiStreamAdapter class that initializes google.genai.Client with proxy configuration and defines Pydantic models for streaming chat messages and response chunks.

Implement the stream_chat async generator method with ThinkingConfig toggle that controls Gemini 2.5 Flash extended reasoning budget and streams GenerateContentResponse chunks as unified StreamChunk objects.

Add safety block detection by checking candidate finish_reason against FinishReason.SAFETY, implement role normalization from Gemini 'model' to 'assistant', and emit StreamError objects when content is blocked.

Build the foundational AnthropicStreamAdapter class that wraps anthropic.AsyncAnthropic with proxy-based configuration. Define the ChatMessage, StreamChunk, TokenUsage, and FinishReason models that form the unified SSE interface, and initialize the async client with proper base_url and api_key setti

Build the streaming event dispatch logic for the AnthropicStreamAdapter. Implement the stream_chat async generator that calls client.messages.stream() as an async context manager, iterates over MessageStream events, and dispatches message_start, content_block_delta, and message_stop events into unif

Implement system prompt extraction that separates system-role messages into the Anthropic top-level system parameter, and map Anthropic stop_reason values (end_turn, max_tokens, stop_sequence) to the unified FinishReason enum. Build a message preparation pipeline and a complete stream_chat method th

Build typed Pydantic models for Architecture Decision Records including decision categories, status lifecycle, and FastAPI endpoints for creating and querying ADRs.

Build DecisionOption models with multi-criteria scoring, a WeightedCriteriaMatrix for normalization and ranking, and utility functions for generating comparison reports and validating evidence.

Implement an in-memory ADR repository with save, find, and atomic supersede operations, plus a versioning system that builds supersede chains, finds the current active version, and returns the full decision history timeline.

Build a framework that extracts testable assumptions from ADRs and validates them against production metrics.

Build a staleness detection system that deterministically validates ADR assumptions against observed metrics, tracks consecutive failures per assumption, scores staleness, and raises severity-ranked drift alerts when decisions go stale.

Build a decision effectiveness scorecard that deterministically checks measured outcomes against their targets, aggregates pass rates into category-level statistics, and computes a weighted effectiveness score with a letter grade.

Build a system that tracks the outcomes of architecture decisions over time, labeling them as successful or failed based on measured criteria.

Build a recommendation engine that uses deterministic word-overlap (Jaccard) similarity to suggest relevant past Architecture Decision Records when making new decisions.

Build a system that monitors technology updates and proactively suggests which existing Architecture Decision Records may need review based on rule-based impact analysis.

Build a coverage analysis system that identifies services without required ADRs and generates gap reports.

Build a review workflow system with approval gates, reviewer validation, and category-specific expiry tracking for Architecture Decision Records.

Build a compliance reporting system that aggregates ADR data, coverage, reviews, and staleness into a weighted compliance score with executive summaries and detailed breakdowns.

Use Docling's DocumentConverter to parse PDF documents into structured DoclingDocument representations. Configure the Granite-Docling-258M layout model for CPU-based table and figure detection, extract text with structure preservation, and store parsed metadata in PostgreSQL.

Extend Docling's DocumentConverter to handle DOCX, PPTX, and HTML input formats through a single unified pipeline. Configure format-specific options, batch-process multiple document types, and normalize outputs into a consistent DoclingDocument structure.

Build a quality evaluation framework that compares Docling extraction results across PDF, DOCX, PPTX, and HTML formats. Measure structural fidelity by scoring heading preservation, table extraction accuracy, and content completeness across simple and complex documents.

Design and implement a unified document schema using Pydantic models that normalizes extraction outputs from Docling, VLMs, and Google Document AI into a single format-agnostic representation. Define typed content blocks for text, tables, images, and key-value pairs, along with extraction metadata t

Build adapter classes that convert raw extraction outputs from Docling, VLMs (via OpenAI/Gemini), and Google Document AI into the unified document model. Implement a factory pattern that selects the correct adapter based on the extraction method name, enabling format-agnostic downstream processing.

Process the same document through Docling, VLM, and Google Document AI adapters, then validate output compatibility across methods. Build a cross-validator that compares block structures and computes content overlap scores, and a quality report generator that summarizes extraction agreement and high

Build a document classifier that inspects PDF files to determine the optimal extraction method. Analyze documents for text layer presence, layout complexity, and scanned-page detection to produce a DocumentProfile that drives routing decisions.

Implement an ExtractionRouter that selects extraction methods based on document complexity and cost constraints. Build a cost tracker that estimates per-document API spend and enforces daily budget limits to prevent runaway costs.

Implement a ResilientRouter that extends extraction routing with fallback chains, confidence-based retry logic, and per-method success/failure metrics. When the primary extraction method fails or returns low-confidence results, the router automatically tries the next method in the fallback chain.

Define the core data structures, schemas, and configuration needed for design interview loops covering coding, system design, llm-specific, and evaluation challenges. Design the foundational models that all other components will build upon.

Build a request-handling layer over a team-health analyzer: parse and validate incoming payloads into domain models, run the analysis, return a status-coded JSON response, and register operation handlers for dispatch.

Configure deployment manifests and resources for a career-ladder service: size replicas, CPU, and memory from an environment-tier policy, render the exact Kubernetes Deployment manifest, evaluate operational metrics against thresholds, and derive a deterministic health verdict with recommendations.

Define a Pydantic-based schema for evaluation test cases, including enums for task categories and difficulty levels, and build a schema validator that loads JSONL datasets and reports validation errors with row-level detail.

Build a TestCaseGenerator that creates evaluation test cases spanning five task categories and three difficulty levels using template-based patterns, then analyze the resulting dataset for stratification balance and coverage gaps.

Build a DatasetBuilder class that loads JSONL evaluation datasets, validates entries with Pydantic, computes coverage statistics including category counts, difficulty distribution, and average token counts, then generates a CoverageReporter that formats results as tables and identifies balance issue

Build a ContaminationDetector class that analyzes recorded probe transcripts - the expected answer for each evaluation test case paired with the text a model returned for it - and decides deterministically which cases look contaminated. You implement text normalization, n-gram overlap scoring, per-p

Implement ROUGE-L scoring from scratch using the longest common subsequence algorithm to measure text overlap between model outputs and expected answers, then build a batch scorer that computes contamination rates across an evaluation dataset.

Build a QuarantineWorkflow that takes per-(case, model) contamination signals, isolates burned evaluation test cases with evidence records, and generates an aggregate contamination report with per-model and per-category rates - all with deterministic grouping, filtering, and arithmetic that runs ful

Build a ComplianceTracker over the four EU AI Act enforcement milestones (Feb 2025, Aug 2025, Feb 2026, Aug 2026). Compute the signed days remaining to each deadline relative to an explicit reference date, classify every milestone as complete, overdue, due-soon (within a configurable window), or upc

Build an AutomatedComplianceChecker that registers a verification predicate for each compliance requirement, runs one or all checks against a system-state snapshot, treats a predicate error as a failed check rather than a crash, and aggregates the pass/fail results into a compliance summary with a c

Build a GapReportGenerator that identifies non-compliant and partially compliant requirements, prioritizes them by combining enforcement deadline proximity with risk severity into a single priority score, generates deterministic remediation recommendations, and produces a formatted, ranked gap analy

Build a benchmarker that calls all three LLM provider APIs through proxies, measures latency and token consumption, computes costs from actual usage, and implements weighted scoring and ranking. Complete the TODO implementations to pass all tests.

Build a token economics engine that calculates costs across providers using actual token counts from real API calls, tracks budget consumption, and generates cost optimization suggestions. Complete the TODO implementations to pass all tests.

Build a rate limit controller that tracks API usage across providers using token bucket algorithm, monitors provider health via response timing, and implements smart request routing. Complete the TODO implementations to pass all tests.

Build a simulated Argo Workflow pipeline for prompt CI/CD operations including linting, validation, evaluation with LLM calls, and promotion stages. Implement pipeline orchestration with artifact storage and metrics tracking for GenAI operations.

Build a Python app that renders text from style templates, enforces a word budget, and returns structured, reproducible report responses.

Build a ConfigPipeline that chains generator stages together to source data from Gemini, filter items by predicate, transform data, and collect results through a complete pipeline.

Build a CounterManager class that demonstrates local and global variables. Learn how scope affects variable accessibility and when to use the global keyword versus explicit data flow.

Rebuild the lab around a real FastAPI app: define Pydantic request/response models with Field(examples=...) and json_schema_extra, attach them via response_model and openapi_extra on actual @app.post/@app.get routes, then have the learner assert that app.openapi()['paths'][...]['requestBody'/'respon