Free lesson · GenAI Agent Engineering

Build a unified LLM client interface

Course: GenAI Agent Engineering · Chapter 11 · The LLM Client

Free to read — no subscription required.

Goal

You can define one LLMClient interface for full and streaming replies, write OpenAI, Anthropic and Gemini adapters that hide each provider's differences and return the same response and token-usage shape, and use a factory that checks credentials up front, so callers switch providers by changing one setting.

Lesson walkthrough

When you wire your application directly to OpenAI's SDK and a quarter later need to add Anthropic or Gemini — or fall back to a different provider during an outage — every call site has to change, every test stub multiplies, and a one-line model swap becomes a sprawling refactor. Teams that postpone the abstraction usually end up doing it under pressure, in production, with a deadline.

Skipping it is how you end up with provider lock-in: business logic coupled to one vendor's request shape, response shape, error taxonomy, and streaming contract.

By the end of this lesson you will be able to define a Python Protocol that describes a provider-agnostic LLM client, implement adapters for OpenAI, Anthropic, and Gemini behind that Protocol, and use a factory to swap providers by changing a single string — with zero changes to caller code.

  • LLMClient Protocol — a structural-typing contract declaring generate and generate_stream that every provider adapter must satisfy so callers depend on the interface, not the vendor SDK.
  • Provider adapter — a class (e.g. OpenAIAdapter, AnthropicAdapter, GeminiAdapter) that wraps a vendor SDK and translates between its native request/response format and the unified ChatMessage / ChatResponse types.
  • ChatResponse — the unified response dataclass (content, input_tokens, output_tokens, model, finish_reason) that normalizes vendor-specific usage and stop-reason fields into one shape downstream code can rely on.
  • Factory — the construction seam: takes a provider string plus credentials and returns an object typed as LLMClient, so caller code never names a concrete adapter class and a swap becomes a config change.

The Protocol as the contract

A Protocol defines the surface every adapter must expose: generate(...) -> ChatResponse and generate_stream(...) -> AsyncIterator[str]. Caller code is typed against the Protocol, not against any concrete adapter, so the type checker enforces that all three providers stay interchangeable. Adding a fourth provider tomorrow forces you to do exactly one thing: implement those two methods with the right signatures.

Loading diagram...

Per-provider differences the adapters must absorb

The three SDKs disagree on three things that would otherwise leak into caller code:

  • System prompt placement. OpenAI accepts the system prompt as the first message in the messages array. Anthropic and Gemini take it as a separate parameter outside the message list (Anthropic's top-level system=; Gemini's system_instruction, set inside config=types.GenerateContentConfig(...) together with max_output_tokens and temperature). Adapters strip it from the messages list and route it to the right slot.
  • Response shape. OpenAI returns response.choices[0].message.content and response.usage.prompt_tokens / completion_tokens. Anthropic returns a list of content blocks you concatenate and usage.input_tokens / output_tokens. Gemini returns response.text plus an optional usage_metadata whose fields may be absent. Each adapter funnels these into the same ChatResponse dataclass.
  • Streaming. OpenAI yields chunks with chunk.choices[0].delta.content. Anthropic uses an async with client.messages.stream(...) context manager and stream.text_stream. Gemini uses client.aio.models.generate_content_stream(...) and yields chunks with .text. Each adapter's generate_stream hides this and yields plain strings.

The factory as the swap point

Caller code never instantiates OpenAIAdapter directly — it asks the factory for a client by provider name. That single string is the only thing that changes when you switch providers, so it can come from config, an env var, or a feature flag. The factory also centralises credential validation, so a missing API key fails loudly at construction time, not in the middle of a streaming call (see Code Walkthrough).

The snippet below stitches together every concept from the previous section: the LLMClient Protocol with its shared ChatMessage / ChatResponse types, three adapters that absorb the per-provider differences (system-prompt placement, response shape, streaming idiom), a factory that maps a provider string to the right adapter, and a caller that runs against an LLMClient reference without naming any concrete class. Read the main() at the bottom first — that's the only code your application would actually contain.

Code snippetpython
1from __future__ import annotations 2import asyncio, os 3from dataclasses import dataclass 4from enum import Enum 5from typing import AsyncIterator, List, Literal, Optional, Protocol 6 7from openai import AsyncOpenAI 8from anthropic import AsyncAnthropic 9from google import genai 10from google.genai import types 11 12class MessageRole(str, Enum): 13 SYSTEM = "system" 14 USER = "user" 15 ASSISTANT = "assistant" 16 17@dataclass 18class ChatMessage: 19 role: MessageRole 20 content: str 21 22@dataclass 23class ChatResponse: 24 content: str 25 input_tokens: int 26 output_tokens: int 27 model: str 28 finish_reason: Optional[str] = None 29 30class LLMClient(Protocol): 31 async def generate( 32 self, 33 messages: List[ChatMessage], 34 system_prompt: Optional[str] = None, 35 max_tokens: int = 1000, 36 temperature: float = 0.7, 37 ) -> ChatResponse: ... 38 39 def generate_stream( 40 self, 41 messages: List[ChatMessage], 42 system_prompt: Optional[str] = None, 43 max_tokens: int = 1000, 44 temperature: float = 0.7, 45 ) -> AsyncIterator[str]: ... 46 47class OpenAIAdapter: 48 def __init__(self, client: AsyncOpenAI, model: str = "gpt-4o-mini"): 49 self.client, self.model = client, model 50 51 def _to_openai(self, messages, system_prompt): 52 out = [] 53 if system_prompt: 54 out.append({"role": "system", "content": system_prompt}) 55 for m in messages: 56 out.append({"role": m.role.value, "content": m.content}) 57 return out 58 59 async def generate(self, messages, system_prompt=None, max_tokens=1000, temperature=0.7): 60 r = await self.client.chat.completions.create( 61 model=self.model, 62 messages=self._to_openai(messages, system_prompt), 63 max_tokens=max_tokens, 64 temperature=temperature, 65 ) 66 return ChatResponse( 67 content=r.choices[0].message.content or "", 68 input_tokens=r.usage.prompt_tokens, 69 output_tokens=r.usage.completion_tokens, 70 model=r.model, 71 finish_reason=r.choices[0].finish_reason, 72 ) 73 74 async def generate_stream(self, messages, system_prompt=None, max_tokens=1000, temperature=0.7): 75 stream = await self.client.chat.completions.create( 76 model=self.model, 77 messages=self._to_openai(messages, system_prompt), 78 max_tokens=max_tokens, 79 temperature=temperature, 80 stream=True, 81 ) 82 async for chunk in stream: 83 piece = chunk.choices[0].delta.content 84 if piece: 85 yield piece 86 87class AnthropicAdapter: 88 def __init__(self, client: AsyncAnthropic, model: str = "claude-haiku-4-5-20251001"): 89 self.client, self.model = client, model 90 91 def _to_anthropic(self, messages): 92 return [ 93 {"role": m.role.value, "content": m.content} 94 for m in messages if m.role != MessageRole.SYSTEM 95 ] 96 97 async def generate(self, messages, system_prompt=None, max_tokens=1000, temperature=0.7): 98 r = await self.client.messages.create( 99 model=self.model, 100 max_tokens=max_tokens, 101 system=system_prompt or "", 102 messages=self._to_anthropic(messages), 103 temperature=temperature, 104 ) 105 text = "".join(b.text for b in r.content if hasattr(b, "text")) 106 return ChatResponse( 107 content=text, 108 input_tokens=r.usage.input_tokens, 109 output_tokens=r.usage.output_tokens, 110 model=r.model, 111 finish_reason=r.stop_reason, 112 ) 113 114 async def generate_stream(self, messages, system_prompt=None, max_tokens=1000, temperature=0.7): 115 async with self.client.messages.stream( 116 model=self.model, 117 max_tokens=max_tokens, 118 system=system_prompt or "", 119 messages=self._to_anthropic(messages), 120 temperature=temperature, 121 ) as stream: 122 async for text in stream.text_stream: 123 yield text 124 125class GeminiAdapter: 126 def __init__(self, client: genai.Client, model: str = "gemini-2.5-flash-lite"): 127 self.client, self.model = client, model 128 129 def _to_gemini(self, messages): 130 contents = [] 131 for m in messages: 132 if m.role == MessageRole.SYSTEM: 133 continue 134 role = "user" if m.role == MessageRole.USER else "model" 135 contents.append({"role": role, "parts": [{"text": m.content}]}) 136 return contents 137 138 async def generate(self, messages, system_prompt=None, max_tokens=1000, temperature=0.7): 139 r = await self.client.aio.models.generate_content( 140 model=self.model, 141 contents=self._to_gemini(messages), 142 config=types.GenerateContentConfig( 143 system_instruction=system_prompt, 144 max_output_tokens=max_tokens, 145 temperature=temperature, 146 ), 147 ) 148 meta = getattr(r, "usage_metadata", None) 149 return ChatResponse( 150 content=r.text or "", 151 input_tokens=getattr(meta, "prompt_token_count", 0) or 0, 152 output_tokens=getattr(meta, "candidates_token_count", 0) or 0, 153 model=self.model, 154 finish_reason="stop", 155 ) 156 157 async def generate_stream(self, messages, system_prompt=None, max_tokens=1000, temperature=0.7): 158 stream = await self.client.aio.models.generate_content_stream( 159 model=self.model, 160 contents=self._to_gemini(messages), 161 config=types.GenerateContentConfig( 162 system_instruction=system_prompt, 163 max_output_tokens=max_tokens, 164 temperature=temperature, 165 ), 166 ) 167 async for chunk in stream: 168 if chunk.text: 169 yield chunk.text 170 171ProviderType = Literal["openai", "anthropic", "gemini"] 172 173class LLMClientFactory: 174 def __init__(self, openai_key=None, anthropic_key=None, gemini_key=None): 175 self._openai_key = openai_key 176 self._anthropic_key = anthropic_key 177 self._gemini_key = gemini_key 178 179 def create_client(self, provider: ProviderType, model: Optional[str] = None) -> LLMClient: 180 if provider == "openai": 181 if not self._openai_key: 182 raise ValueError("OPENAI_API_KEY missing") 183 return OpenAIAdapter(AsyncOpenAI(api_key=self._openai_key), model or "gpt-4o-mini") 184 if provider == "anthropic": 185 if not self._anthropic_key: 186 raise ValueError("ANTHROPIC_API_KEY missing") 187 return AnthropicAdapter(AsyncAnthropic(api_key=self._anthropic_key), model or "claude-haiku-4-5-20251001") 188 if provider == "gemini": 189 if not self._gemini_key: 190 raise ValueError("GEMINI_API_KEY missing") 191 return GeminiAdapter(genai.Client(api_key=self._gemini_key), model or "gemini-2.5-flash-lite") 192 raise ValueError(f"Unknown provider: {provider}") 193 194async def main() -> None: 195 factory = LLMClientFactory( 196 openai_key=os.environ.get("OPENAI_API_KEY"), 197 anthropic_key=os.environ.get("ANTHROPIC_API_KEY"), 198 gemini_key=os.environ.get("GEMINI_API_KEY"), 199 ) 200 # Swap providers by changing this one string. Caller code below is identical. 201 client: LLMClient = factory.create_client(os.environ.get("LLM_PROVIDER", "openai")) 202 203 messages = [ChatMessage(MessageRole.USER, "What is functional programming, in one sentence?")] 204 resp = await client.generate(messages, system_prompt="You are concise.", max_tokens=200) 205 print(f"[{resp.model}] {resp.content}") 206 print(f"tokens in/out: {resp.input_tokens}/{resp.output_tokens}") 207 208if __name__ == "__main__": 209 asyncio.run(main())

You'll know it works when you can run LLM_PROVIDER=openai python unified.py, then LLM_PROVIDER=anthropic python unified.py, then LLM_PROVIDER=gemini python unified.py against the same main() and get a one-sentence answer plus non-zero tokens in/out from each — without editing a single line in main().

Having walked through building a unified client interface above, the following Do's and Don'ts distill it into practice.

Do's

  1. ✓Do type caller code against the LLMClient Protocol — never against OpenAIAdapter or any concrete class — so a provider swap is a one-line change at the factory.
  2. ✓Do normalize responses inside the adapter — collapse usage.prompt_tokens, usage.input_tokens, and usage_metadata.prompt_token_count into ChatResponse.input_tokens so downstream code reads one field name.
  3. ✓Do validate credentials at factory construction — raise ValueError for missing keys before the first request, not in the middle of a streaming call.

Don'ts

  1. ✗Don't leak provider-specific kwargs into the unified signature — if Anthropic adds a new sampling knob, expose it via a generic extra: dict rather than widening every adapter's interface.
  2. ✗Don't branch on provider == "openai" in caller code — that's the smell the Protocol exists to eliminate; push the branch down into the factory or a new adapter.
  3. ✗Don't silently swallow per-provider streaming errors — let them propagate so retry/fallback logic above the adapter can decide what to do.

2 hands-on labs come with this lesson — real code, in a cloud IDE. Create a free account to run them. No card.

Free account · no card · straight to the labs

Or get the full path — from