Host: Welcome back. You're in GenAI Architecture and Design Patterns — a course about how to actually structure AI systems so they hold up in production, not just in a demo. This chapter is about something called an ADR engine. ADR stands for Architecture Decision Record — it's a short written note that captures a technical choice your team made and why. This specific exercise teaches you how to track whether those past decisions actually worked out.
Expert: Picture this. You're on a platform team at a mid-sized company, maybe two hundred engineers, and over the last eighteen months your group has made dozens of big AI choices. Which language model to use. Whether to host it yourselves or call a cloud service. Whether to feed the model extra information at query time or retrain it on your data. Every one of those decisions got written down somewhere — a document, a wiki page, a message thread. But here's what breaks. Six months later, a new team faces the same choice. They go looking for prior wisdom. They find the decision. What they don't find is the outcome. Did it work? Did latency blow up? Did the cloud bill triple? Did the team quietly rip it out and replace it three months later? Without that feedback loop, every team repeats the same mistakes. Architecture decisions become folklore instead of data. The skill you're about to build is what turns that folklore into evidence.
Host: So this is the starting point for the chapter — the first of three exercises that together build a full decision engine. What exactly are we building in this one?
Expert: You're building a tracker. Think of it as a structured logbook for architecture decisions. Each entry in the logbook describes one decision — what the team picked, when they picked it, and what they were hoping would happen. Then, over time, you attach outcome information to that entry. Things like: did the system actually meet its performance target, did the cost stay within budget, did the team end up keeping the solution or abandoning it. And based on those measurements, each decision gets a label — successful, failed, or still being evaluated. The key idea underneath all of this is something called measured criteria. You are not labeling decisions based on opinion or vibes. You're labeling them based on numbers you committed to up front. So when someone writes down the decision, they also write down what success looks like — for example, responses under two seconds, monthly cost under five thousand dollars, user satisfaction above a certain threshold. Later, when real numbers come in, the system compares them to those original targets and produces the label automatically. That separation — between the decision, the promised criteria, and the observed reality — is the whole mental model you need before you sit down to code.
Host: Before we start — what's the one thing people tend to get wrong on this?
Expert: The trap is making the success criteria too vague. People write things like "fast enough" or "reasonable cost." Those cannot be measured, so the tracker cannot ever produce a verdict. The fix is to force every decision to carry numeric thresholds with clear units. Two seconds, not fast. Five thousand dollars a month, not reasonable. The second smaller trap is forgetting the middle state. A decision that hasn't been measured yet isn't successful and isn't failed — it's pending. Your tracker needs to represent that third state honestly, because a lot of real decisions live there for months.
Host: Great. So after this exercise, what can the listener actually do, and where does the chapter go next?
Expert: After this, you'll be able to take any architecture decision your team makes and wrap it in a structure that automatically tells you, later, whether it panned out. That's a foundational building block — the kind of piece your team can plug into their own documentation workflow so that institutional memory becomes searchable evidence instead of scattered conversations. In the next exercise, you'll extend this. You'll take all those labeled past decisions and build a recommendation engine on top of them. The idea is that when someone is about to make a new decision, the system finds the most similar past decisions — using a math technique called cosine similarity on text embeddings, which is basically a way to measure how close two pieces of writing are in meaning — and surfaces them, along with their outcomes. So instead of starting from scratch, the new team sees: here are four past decisions that look a lot like yours, three succeeded, one failed, here's why. That's how you turn this tracker into a real decision support system. For now, focus on getting the outcome labeling right — clean criteria, honest labels, pending as a first-class state. Everything else in the chapter depends on this foundation. Thanks for listening, and good luck with the build.
Want to go deeper? Explore disciplines with hands-on labs, quizzes, and chapter podcasts.