AI Use Case Discovery & Data Readiness Assessment — chapter audio overview

Podcast Script: AI Use Case Discovery & Data Readiness Assessment Host: Welcome back to the GenBodha Solution Delivery podcast. You're listening to Chapter 1 of 12 in our AI Solution Delivery course, and this one is called "AI Use Case Discovery and Data Readiness Assessment." Now, your organization has invested in this training because AI solution delivery is a core competency for any team building production AI systems. This is the depth that turns an engineer who uses AI tools into one who builds the infrastructure behind them. Picture this scenario. You walk into a client's office on day one of a new engagement. Their marketing team wants automated content generation. Support wants ticket classification. Finance is dreaming about fraud detection. Operations wants predictive maintenance. Everyone is talking at once, everyone wants to go first, and nobody has a defensible way to decide what to actually build. In the last thing most of you studied — agent engineering and LLM customization — you learned how to build the systems. Now the question is, how do you decide which systems to build and whether the client's data can even support them? You'll practice this across five hands-on labs, but first let's build the mental model. We'll walk through five interconnected tools, learn when each one matters, and tie it all together. Let me bring in our expert. Expert: Thanks. Let me set the frame before we dive in. The traditional discovery phase is heavily consultant-driven — senior people interview stakeholders, fill spreadsheets, form opinions, and present recommendations. The problem is that approach doesn't scale, it isn't repeatable, and the conclusions are hard to defend when an executive asks "why this use case and not that one?" What we're building in this chapter is a structured, data-driven replacement. Five components. A scoring engine that ranks AI opportunities. A data readiness profiler that tells you whether the client's data can actually support the work. An interview agent that conducts adaptive discovery conversations. A feasibility analyzer that benchmarks the major LLM providers head-to-head. And a report generator that packages all of this into an executive-ready deliverable. Each one solves a real gap, and — this is the important part — the output of each feeds the next. So by the end of the chapter you have a complete discovery pipeline that turns messy stakeholder wish-lists into prioritized, evidence-backed recommendations your team can stand behind in front of the client's C-suite. Host: Great framing. Let's start with the first piece — use case scoring. When a client dumps twenty possible AI ideas on the table, how do we decide which ones to pursue in a way that doesn't just reflect whoever talked loudest in the room? Expert: Right, this is the problem use case scoring solves. Use case scoring is a structured evaluation method — you assign numerical scores to each potential AI use case across several weighted criteria, and you end up with a ranked list. It matters because without this, organizations default to the loudest stakeholder or the shiniest technology, and neither correlates with actual business value. Here's how it works. We evaluate every use case across four dimensions. First, business impact — how much revenue, cost reduction, or competitive advantage does this deliver? A chatbot that deflects thirty percent of support tickets has measurable impact. A proof-of-concept with no deployment path does not. Second, technical feasibility — can current AI actually solve this? Document summarization, very feasible. Real-time reasoning over streaming sensor data with sub-ten-millisecond latency, significant technical risk. Third, data readiness — does the client have the data assets needed, at the volume and quality required? And fourth, time-to-value — can we deliver measurable results in four to six weeks, or are we signing up for a multi-year research project that never reaches production? Now, here's where the engineering gets interesting. We use a Python library called Instructor. Instructor is a tool that wraps LLM client libraries — it takes an OpenAI or Anthropic or Google call and forces the response into a validated, typed data structure instead of free-form text. Pair that with Pydantic — which is Python's data validation library — and you get something powerful. Instead of the LLM returning a paragraph of prose where you'd have to manually parse out scores, the LLM is forced to return a structured object with a score between zero and ten, a confidence value between zero and one, and a reasoning string. It's validated the moment it arrives. You never have to clean it up. One critical detail — LLM scores are not deterministic. Run the same use case through the engine three times and you get slightly different numbers. So we run multiple evaluation passes and compute confidence intervals. A use case scoring 7.2 plus or minus 0.3 is a much more trustworthy recommendation than one scoring 7.5 plus or minus 2.1. That plus-or-minus range is what transforms this from a one-shot evaluation into a statistically grounded assessment stakeholders can actually base investment decisions on. And the weights themselves are configurable per engagement — a startup weights time-to-value heavily, a regulated enterprise weights data readiness heavily. Same engine, different client contexts. Host: So we've turned a subjective ranking into a repeatable, weighted analysis with confidence ranges. That's the use case half. But a use case is only as good as the data it depends on. Let's talk about data readiness — what exactly are we profiling, and why does it so often blow projects up? Expert: This is the single most underestimated part of any AI engagement. A 2024 industry survey found that sixty-five percent of enterprise AI initiatives are delayed by data quality issues that get discovered after project kickoff. Not before — after. Which means the scope was already signed, the timeline was already promised, and now everybody is renegotiating. Data readiness profiling is how you catch this before committing. We evaluate five dimensions of the client's dataset. Schema completeness — does the data even have the fields we need? A support chatbot needs conversation history with resolution status. If the client's export is missing that, it's a scope issue. Null rate analysis — the percentage of missing values in each column. Forty percent nulls in your product category field is going to wreck a classification model regardless of how good the model is. Cardinality — the distribution of unique values. If a department field has five thousand distinct values in a ten-thousand-row dataset, you've got typos, inconsistent naming, data quality chaos. Volume — fine-tuning needs thousands of labeled examples, retrieval-augmented generation can work with hundreds of documents. And embedding suitability, which asks whether the text is the right shape for vector search. Very short snippets produce poor embeddings. Mixed languages hurt consistency. The profiler scores all of this. Now here's the part that separates a professional delivery engineer from someone who just ships code — personally identifiable information detection. PII is data like names, emails, phone numbers, Social Security numbers, credit card numbers. Clients routinely tell you their data is "already clean." It almost never is. So before any data enters an AI pipeline, you run PII detection. We use an open-source framework from Microsoft called Presidio — think of it as a universal scanner for sensitive data. It combines pattern matching, named entity recognition, and context analysis to flag emails, phone numbers, Social Security numbers, credit cards, names, and more, with a confidence score for each detection. And critically, the findings get classified by severity. Critical findings — Social Security numbers, credit cards, passwords — likely block data usage entirely until redacted. High severity is full names combined with contact info, which creates re-identification risk. Medium is isolated emails or phone numbers. Low is things like generic location data. That severity layer is what lets you talk to the client in business terms instead of handing them a raw entity dump they can't interpret. One rule — never present raw PII findings to non-technical stakeholders. Always classify by severity and business impact first. Host: So data readiness is catching the landmines early and translating them into language the client can act on. Now, both the scoring and the readiness assessment depend on information that comes out of stakeholder interviews. Let's talk about how we scale those conversations — because frankly, interviews are where most discovery either succeeds or quietly falls apart. Expert: Exactly right. Discovery interviews are the richest source of information in any engagement. Surveys and document reviews miss the stuff that actually matters — the unstated assumptions, the political tensions, the gap between what stakeholders claim they need and what they actually need. But here's the catch. Human interviewers get tired. They ask inconsistent follow-ups. They forget to probe when someone says something revealing. And synthesizing insights across fifteen or twenty conversations is genuinely hard work. So we build what we call a discovery interview agent. Now, to build this we use a framework called LangGraph. LangGraph is a library for designing multi-step agent workflows as a directed graph — you define states, you define transitions, and the framework handles memory, branching, and orchestration. Think of it like a flowchart that actually runs. The interview agent has four logical states. Question generation produces a contextual question based on the topic and everything the stakeholder has already said. Response capture records the answer with metadata. Follow-up analysis evaluates each response and decides whether the agent should dig deeper. If the stakeholder mentions "we tried something similar last year but it didn't work," that's a goldmine — the agent automatically generates follow-ups about what was tried, why it failed, what would be different this time. And insight extraction processes the full conversation at the end into structured findings — pain points, opportunities, constraints, and requirements. Each insight carries a confidence score and supporting quotes from the transcript, so every finding traces back to evidence. The magic happens at the cross-interview synthesis layer. When eight out of ten stakeholders independently mention the same pain point, that's convergent validation. When engineering says "our data is clean" and operations says "we spend hours fixing data errors every week," that contradiction is itself an insight — divergent perspectives are often more valuable than consensus because they reveal requirements nobody articulated in isolation. One discipline here — always record raw interview transcripts alongside extracted insights, so interpretations can be checked against source material. And never ignore contradictions. The contradictions are usually where the real requirements live. Host: I love that — the contradictions are the signal, not the noise. Okay, so we have scored use cases, profiled data, synthesized interviews. Now comes a technical question that trips up a lot of teams. Which LLM provider do we actually use for this client? How do we turn that into a data-driven decision instead of a brand preference? Expert: This is one of the most consequential technical calls in any engagement, and it's almost always made wrong. Someone on the architecture team says "we already use Azure, so let's use OpenAI" or "Gemini is cheapest, use that." Neither is an analysis. The feasibility analyzer replaces that guesswork with measurement. Each major provider has distinct strengths. OpenAI's GPT-4o is strong at complex reasoning and tool use but has higher per-token cost. Google's Gemini 2.5 Flash is dramatically cheaper with a very large context window — think a million tokens — which matters for document-heavy work. Anthropic's Claude is strong on analysis and safety-sensitive applications. But those are generalizations. The real question is always, which provider fits this specific use case at this client's scale? To answer that we use another library called LiteLLM. LiteLLM is a unified interface that talks to over a hundred LLM providers through a single consistent API. Instead of learning three different SDKs with three different response formats, you call one function and LiteLLM handles the provider-specific details underneath. The analyzer fires the same prompt at all three providers in parallel — using Python's async capabilities so network variability hits every provider equally — and measures four things. Latency in seconds, token usage, estimated cost based on each provider's pricing, and response quality. Quality is judged by using another LLM as a judge — scoring relevance, completeness, accuracy, and format compliance against use-case-specific criteria. Then we project cost at the client's expected scale. This is where it gets dramatic. For a use case processing ten thousand documents per month, the monthly cost difference between providers can be ten to twenty times. At per-token pricing the numbers look similar. At production volume they are wildly different. The final recommendation reads something like this — Gemini delivers comparable quality, 8.2 out of 10 versus GPT-4o's 8.5, at seventeen times lower cost, twenty-eight dollars a month versus four hundred seventy-five, and fifteen percent lower latency. The 3.6 percent quality gap does not justify the seventeen-times cost premium for this workload. That's a recommendation an executive can actually approve. One rule — never compare providers on latency alone. Cost, quality, and context window all matter for real decisions. And always benchmark with prompts that match the actual use case, not generic test prompts. Host: So now we have scored use cases, profiled data, interview insights, and a provider recommendation with dollar figures. That's a mountain of assessment data. Last section — how do we turn all of that into something a client can actually use to make a decision? Expert: This is where the report generator earns its keep. Raw assessment data does not drive decisions. Executives need a concise summary, technical leads need architectural guidance, project managers need actionable next steps. A good discovery report has five sections. An executive summary — one or two pages that tell a C-level reader how many use cases were evaluated, which scored highest, what the data situation looks like, and what we recommend. Use case rankings with visual indicators, typically a heatmap of business impact against data readiness so the valuable-and-feasible opportunities pop out. Data readiness assessment per dataset for the data engineering team. Provider recommendations with cost projections for the architecture team. And next steps and timeline, which sets up the scoping conversation in Chapter 2. To build this we use Jinja2. Jinja2 is a templating language — it lets you define the structure of a document once, with placeholders for dynamic content, and then inject the assessment data at render time. The big advantage is separation. The report structure lives in the template. The data lives in Pydantic models. You change the template once and every future engagement's report uses the updated format. Before you render anything though, you have to aggregate. The four assessment components — scoring, profiling, interviews, feasibility — each produce their own output. The report generator cross-references them. Interview insights get linked to the use cases they support. Data readiness findings get connected to the datasets each use case needs. Provider recommendations get attached to the use cases they were benchmarked against. Every recommendation in the final report traces to specific evidence. That traceability is what makes discovery reports defensible. And then — this is critical — you run a validation step before delivery. The numbers in the executive summary must match the detailed sections. Every scored use case must appear in the rankings. Every dataset must appear in readiness. Inconsistent reports damage credibility more than a report that ships a day late. And one more thing — the report exports to JSON as well as markdown. That JSON export is the bridge to Chapter 2. Your scoping tools in the next chapter will consume this file directly. Host: Before we wrap up the concepts, give us the production wisdom. If the listener takes only two or three things from this chapter into a real engagement, what must they remember? Expert: Three things. First — data readiness is never binary. Clients want a yes or no answer. The honest answer is almost always "partial readiness with a remediation plan." Train yourself and your stakeholders to think in terms of gaps and mitigation, not pass or fail. Second — always run multiple scoring passes and report confidence intervals. LLM-based evaluation is inherently probabilistic. A single pass is a guess. Three to five passes with an interval is a defensible position. And third — never skip PII detection because the client says their data is already clean. It isn't. It almost never is. PII exposure is the single biggest risk in any data-enters-AI-pipeline situation, and finding it after the fact is career-defining in the wrong direction. On the strict don't list — don't hard-code scoring weights into the engine. Weights are client-specific and must be API-configurable. Don't use a single LLM call to evaluate all use cases at once — batch them individually for accurate scoring. And don't generate reports without the validation step that cross-checks numbers across sections. Host: That's gold. Okay, let's preview what you'll actually build. Five hands-on labs, each mapped to one tool we just walked through. Lab one, you'll build the use case scoring engine with weighted criteria evaluation using Instructor and Pydantic models. Lab two, you'll implement the data readiness profiler that analyzes comma-separated-value and JSON datasets for quality metrics and runs Presidio against text columns to catch PII. Lab three is the discovery interview agent, built on LangGraph with structured question flows and automated follow-ups. Lab four, the provider feasibility analyzer — you'll benchmark use cases across OpenAI, Gemini, and Anthropic with parallel execution using LiteLLM. And lab five, the discovery report generator with Jinja2 templates producing structured markdown. Each lab has its own audio overview that goes deeper on the mechanics. Host: Let's close it out. Three takeaways. You now understand why structured, weighted scoring with confidence intervals beats subjective stakeholder preference every time, and you can defend those scores with evidence. You now understand that data readiness — especially PII exposure — is the single biggest predictor of project success or failure, and you have a systematic way to assess it before you commit to scope. And you now understand how multi-provider feasibility analysis turns provider selection from a brand preference into a measured engineering decision with real cost projections. You have the depth to evaluate AI opportunities for your team's discovery engagements, to explain the trade-offs to executive stakeholders, and to push back intelligently when a client insists their data is ready when it isn't. The chapter quiz will focus on which scoring criteria apply where, how Instructor works with Pydantic to produce structured LLM output, the role of LangGraph state in the interview agent, and Microsoft's Presidio framework for PII detection. Pay special attention to the trade-off between provider cost, quality, and latency — that decision point shows up across multiple questions. Next time, Chapter 2, Solution Scoping and Effort Estimation. The outputs you generate in this chapter — the scored use cases, the readiness findings, the provider recommendations — become the inputs for work breakdown structures, risk assessments, and formal scope documents. Discovery feeds scoping feeds proposal feeds execution, just like real engagements. Catch you in the next one.