Implement similarity-based ADR recommendation from prior decisions — lab audio overview

Host: Welcome back to GenAI Architecture and Design Patterns. You're in the chapter on the GenAI Architecture Decision Record Engine — and an Architecture Decision Record, or ADR, is just a short written document that captures a significant technical choice your team made, why you made it, and what the trade-offs were. This discipline matters because GenAI teams are making huge, expensive decisions — which model to use, whether to host it yourself, whether to fine-tune or use retrieval — and those decisions need to be remembered. Expert: Picture this scenario. You're on a platform team at a mid-sized company, maybe fifty engineers building GenAI features. Over eighteen months, your team has made around two hundred architecture decisions. Which large language model to use for summarization. Whether to run models on your own cloud or call a vendor. When to fine-tune versus when to retrieve from a knowledge base. These decisions live in scattered documents, old wiki pages, and in the heads of people who may have left. Now a new team is starting a support-ticket-summarization project. They sit down to make their model choice. And here's what breaks without this skill: they either spend a week digging through old documents, or they just pick something fresh — and three months later they discover another team already tried that exact approach, hit a wall, and documented why. The organization paid for that lesson twice. What you need is a system that, the moment someone starts writing a new decision, automatically surfaces the most relevant past decisions. That's what this exercise builds. Host: In the previous exercise, you built a tracker that stores past architecture decisions and labels each one as a success or a failure based on measured outcomes. So now you have a library of decisions with verdicts attached. The question is: how do we find the right ones at the right moment? What exactly will we build here? Expert: You'll build a recommendation engine. Think of it as a smart librarian sitting next to the engineer. When they start describing a new decision — say, "we need to pick a model for summarizing long legal contracts" — the librarian instantly pulls the three or four most relevant past decisions from the shelf and hands them over. The key idea — and this is the real aha — is that the computer doesn't understand the words the way you and I do. So we convert every decision description into something called a text embedding. An embedding is just a long list of numbers — hundreds of them — that represents the meaning of a piece of text as a point in mathematical space. Texts with similar meanings end up as points close to each other in that space, even if they use completely different words. So a decision about "picking an LLM for contract summarization" and another about "choosing a model for long-document analysis" will land near each other, because they mean similar things. Then we measure closeness using cosine similarity — a standard mathematical way to score how similar two of these number-lists are, giving you a value between zero and one. One means nearly identical meaning; zero means unrelated. You rank past decisions by that score and return the top handful. Host: Before people start, what's the one thing that tends to trip them up on this kind of work? Expert: The tricky part is what you feed into the embedding step. It's tempting to throw the entire decision document in — title, context, options considered, final choice, consequences, the works. But embeddings have a sweet spot. If you include too much, the signal from the actual decision topic gets diluted by boilerplate text, and unrelated decisions start scoring as similar just because they share generic architecture language. If you include too little — say, just the title — you miss the real substance. The practical tip: combine the title and a concise summary of the context and the chosen option. Keep it focused. The other gotcha is remembering to convert new incoming queries using the exact same embedding model as the stored decisions. Mixing models gives you numbers that live in different mathematical spaces, and your similarity scores become meaningless. Same model in, same model out — always. Host: So what will you walk away able to do? Expert: After this, you'll be able to take a library of past technical decisions and build a system that, given any new decision description, instantly returns the most relevant prior ones ranked by meaning-based similarity. That's a foundational capability — the same pattern powers semantic search, duplicate detection, and retrieval for question-answering systems. For your team, this becomes a building block you can bring back to architecture discussions: a way to make sure institutional knowledge actually gets reused instead of rediscovered. In the next exercise, you'll extend this one step further — you'll build a system that watches for new technology announcements and proactively flags which of your existing decisions might need a fresh look, so the knowledge base doesn't just sit there, it actively nudges the team. That's where this three-exercise arc lands. Host: Alright — open the exercise, keep the embedding inputs focused, and remember the same-model rule. Thanks for listening, and good luck.