Build a Gemini grounded-generation endpoint with Google Search

Introduction

When you ship a multi-modal GenAI API, hallucinated facts erode user trust faster than any UI bug — a wrong citation can leak into a customer report, a legal filing, or a clinical note, and there is no verification trail to walk it back. This lesson shows how Gemini 2.5 Flash's native Google Search grounding gives every factual claim a verifiable source URL inline, without standing up a vector store or RAG retriever. By the end you'll be able to wire the google-genai SDK's GoogleSearch tool into a FastAPI endpoint, parse grounding_metadata into structured Citation objects, and filter low-confidence sources before they reach the unified response assembly.

Key Terminology

• Grounding chunk: A single source document returned by Google Search, containing a URL and page title — the citation surface your API exposes.
• Grounding support: A mapping from a specific segment of generated text to one or more grounding chunks, paired with a confidence score between 0 and 1.
• GoogleSearch tool: The first-party google.genai.types.GoogleSearch tool passed in GenerateContentConfig.tools that lets Gemini 2.5 Flash issue live search queries mid-inference.

Concepts

Shape of the grounded response block

The grounded generation endpoint returns a GroundedResponse with three top-level fields: answer (the generated text), citations (a list of Citation objects extracted from grounding_chunks), and search_queries (the queries Gemini actually issued, surfaced for observability). Each Citation carries the source URL, page title, the snippet of generated text the grounding_support mapped to that source, and the confidence score from the support entry. Downstream consumers can render citations inline (anchored to the text segment) or as a trailing reference list — the segment-level mapping makes both UX patterns possible without re-parsing the answer.

A key architectural decision is whether to call generate_content synchronously or wrap it in an asyncio.gather with other independent calls. Google Search grounding adds 1–3 seconds of latency because the model issues real search queries mid-inference, so when the caller has other independent work to do, running the grounded call concurrently hides that latency behind the parallel-execution boundary. The endpoint itself stays focused on one job: invoke Gemini with the GoogleSearch tool, parse grounding_metadata into Citation objects, and return the GroundedResponse.

Production hardening considerations

Grounded generation introduces a dependency on Google Search availability. Unlike a self-hosted vector store, you cannot control the search index, its freshness, or its availability. Implement circuit breakers around the generate_content call with fallback to ungrounded generation when Search is degraded. Log the search_queries field to detect query reformulation drift—if Gemini consistently reformulates user queries in unexpected ways, your prompt engineering may need adjustment.

Rate limits for Gemini with grounding are separate from standard generation quotas. The Google Search tool consumes additional quota per request. Monitor your usage via the Google Cloud console and implement client-side rate limiting with exponential backoff. For cost optimization, cache grounded responses for identical queries within a short TTL window (60-300 seconds), since search results change infrequently within that timeframe. Use the query string as the cache key, not the full prompt, to maximize cache hit rates across slight prompt variations.

Finally, validate that grounding metadata is present before assuming citations exist. Gemini may choose not to ground a response if it determines the query does not benefit from search—for example, purely creative or hypothetical prompts. Your parse_grounding_metadata function already handles this with the None check on grounding_metadata, but the calling code should communicate the absence of citations to the client explicitly rather than returning an empty list with no explanation. Add a grounded: bool field to GroundedResponse that signals whether the model actually used search, enabling clients to adjust their citation rendering logic accordingly.

Code Walkthrough

How Gemini native grounding works

Unlike tool-calling patterns where you define custom functions, Gemini's Google Search grounding is a first-party tool built into the model's inference pipeline. When you pass the GoogleSearch tool in your generation config, Gemini autonomously decides when a query benefits from real-time search. The model issues search queries, processes the results, and generates a response that includes grounding_metadata on each candidate. This metadata contains two critical structures: grounding_chunks (the source URLs and titles) and grounding_supports (the specific text segments mapped to those sources with confidence scores).

• Grounding chunk: A single source document returned by Google Search, containing a URL and page title. These are the citations your API will expose.
• Grounding support: A mapping from a specific segment of the generated text to one or more grounding chunks, with a confidence score between 0 and 1.
• Search entry point: An optional rendered HTML snippet that provides a Google Search link for the query, useful for "search more" UX patterns.
• Retrieval query: The actual search query Gemini formulated internally—often a reformulation of the user's original prompt optimized for search retrieval.

When Gemini generates text from its training data alone, users have no way to verify claims. The /api/v1/grounded/generate endpoint solves this by passing tools=[GoogleSearch] directly to generate_content(), enabling Gemini 2.5 Flash to execute live search queries mid-generation. The response pipeline then parses grounding_metadata — extracting grounding_chunks into source URLs and mapping grounding_supports to specific text segments — ultimately returning a GroundedResponse containing Citation objects with verifiable, inline citations.

Loading diagram...

Configuring the google-genai client with grounding

The google-genai SDK (package name google-genai, imported as google.genai) provides the Client class for direct API access and the types module for configuration objects. The critical class for grounding is google.genai.types.GoogleSearch, which you pass in the tools parameter of GenerateContentConfig. Unlike the older google-generativeai package, the new SDK uses an explicit client instantiation pattern that supports both API key and OAuth authentication. The following snippet demonstrates how to initialize the client, configure the grounding tool, and issue a grounded generation request within a FastAPI route handler. Note the use of GenerateContentConfig to bundle the tool declaration with generation parameters like temperature and max_output_tokens.

1from google import genai
2from google.genai import types
3from pydantic import BaseModel, Field
4from fastapi import APIRouter, HTTPException
5
6router = APIRouter(prefix="/api/v1/grounded")
7
8class GroundedRequest(BaseModel):
9    query: str = Field(..., min_length=1, max_length=2000)
10    temperature: float = Field(default=0.3, ge=0.0, le=1.0)
11    max_tokens: int = Field(default=2048, ge=256, le=8192)
12
13class Citation(BaseModel):
14    url: str
15    title: str
16    snippet: str
17    confidence: float
18
19class GroundedResponse(BaseModel):
20    answer: str
21    citations: list[Citation]
22    search_queries: list[str]
23
24def get_genai_client() -> genai.Client:
25    """Initialize google-genai client with API key."""
26    return genai.Client()  # Uses GOOGLE_API_KEY env var
27
28@router.post("/generate", response_model=GroundedResponse)
29async def generate_grounded(request: GroundedRequest):
30    client = get_genai_client()
31    search_tool = types.Tool(google_search=types.GoogleSearch())
32    config = types.GenerateContentConfig(
33        tools=[search_tool],
34        temperature=request.temperature,
35        max_output_tokens=request.max_tokens,
36    )
37    try:
38        response = client.models.generate_content(
39            model="gemini-2.5-flash",
40            contents=request.query,
41            config=config,
42        )
43    except Exception as exc:
44        raise HTTPException(status_code=502, detail=f"Gemini API error: {exc}")
45
46    answer = response.text or ""
47    citations = parse_grounding_metadata(response)
48    queries = extract_search_queries(response)
49    return GroundedResponse(
50        answer=answer, citations=citations, search_queries=queries
51    )

Parsing grounding metadata into structured citations

The raw grounding metadata returned by Gemini requires careful extraction. The response.candidates list contains generation candidates, each with an optional grounding_metadata attribute. Within this metadata, grounding_chunks holds the source references while grounding_supports maps generated text segments to those chunks via index references. The following function, parse_grounding_metadata, iterates through these structures to produce a flat list of Citation objects. The companion function extract_search_queries pulls the reformulated search queries Gemini used internally. This parsing layer is essential because the raw metadata uses index-based references between supports and chunks that must be resolved into self-contained citation objects before they reach the unified response assembly layer.

1from google.genai.types import GenerateContentResponse
2
3def parse_grounding_metadata(response: GenerateContentResponse) -> list[Citation]:
4    """Extract citations from Gemini grounding metadata."""
5    citations: list[Citation] = []
6    if not response.candidates:
7        return citations
8
9    candidate = response.candidates[0]
10    metadata = getattr(candidate, "grounding_metadata", None)
11    if metadata is None:
12        return citations
13
14    chunks = metadata.grounding_chunks or []
15    supports = metadata.grounding_supports or []
16    chunk_lookup = {}
17    for idx, chunk in enumerate(chunks):
18        web = getattr(chunk, "web", None)
19        if web is not None:
20            chunk_lookup[idx] = {"url": web.uri, "title": web.title or ""}
21
22    for support in supports:
23        segment = getattr(support, "segment", None)
24        snippet = segment.text if segment else ""
25        indices = support.grounding_chunk_indices or []
26        scores = support.confidence_scores or []
27        for i, chunk_idx in enumerate(indices):
28            if chunk_idx in chunk_lookup:
29                score = scores[i] if i < len(scores) else 0.0
30                citations.append(Citation(
31                    url=chunk_lookup[chunk_idx]["url"],
32                    title=chunk_lookup[chunk_idx]["title"],
33                    snippet=snippet,
34                    confidence=round(score, 4),
35                ))
36    return citations
37
38def extract_search_queries(response: GenerateContentResponse) -> list[str]:
39    """Pull internal search queries Gemini used for grounding."""
40    queries: list[str] = []
41    if not response.candidates:
42        return queries
43    metadata = getattr(response.candidates[0], "grounding_metadata", None)
44    if metadata is None:
45        return queries
46    for query_obj in (metadata.retrieval_queries or []):
47        queries.append(str(query_obj))
48    return queries
49
50def filter_and_deduplicate(
51    citations: list[Citation],
52    min_confidence: float = 0.7,
53) -> list[Citation]:
54    """Filter low-confidence citations and deduplicate by URL."""
55    seen: dict[str, Citation] = {}
56    for cite in citations:
57        if cite.confidence < min_confidence:
58            continue
59        existing = seen.get(cite.url)
60        if existing is None or cite.confidence > existing.confidence:
61            seen[cite.url] = cite
62    return sorted(seen.values(), key=lambda c: c.confidence, reverse=True)

Confidence filtering and citation deduplication

In production, not every grounding support carries equal weight. A confidence score of 0.95 from a .gov domain is qualitatively different from 0.4 on a forum post. The filter_and_deduplicate helper in the snippet above applies a confidence threshold (default 0.7) before surfacing citations to end users—below this, the model's own assessment suggests the grounding link is tenuous. It also deduplicates by URL while preserving the highest confidence score, preventing citation bloat when the same source backs multiple text segments. Expose min_confidence through GroundedRequest when you need per-request tuning (e.g., stricter thresholds for regulated domains).

Do's and Don'ts

Do's

1. ✓Do pass types.Tool(google_search=types.GoogleSearch()) inside GenerateContentConfig — this is the only way to activate Gemini's first-party grounding pipeline; without it, grounding_metadata is absent from the response and every factual claim in response.text is unverifiable training-data output.
2. ✓Do parse both grounding_chunks and grounding_supports from grounding_metadata — chunks give you the source URL and title, while supports provide the confidence score and the text-segment mapping; assembling Citation objects from both structures is what produces inline, traceable citations rather than a bare list of URLs.
3. ✓Do filter grounding_supports by confidence score before including a Citation in GroundedResponse — low-confidence supports indicate weak source alignment; surfacing them as authoritative citations defeats the purpose of grounding and can still propagate incorrect claims to callers.

Don'ts

1. ✗Don't import from the legacy google-generativeai package when using grounding — the new google-genai SDK (from google import genai) is required for types.GoogleSearch and the explicit genai.Client() instantiation pattern; mixing the two packages produces import conflicts and missing grounding_metadata fields on the response object.
2. ✗Don't treat response.text as verified output if grounding_metadata is empty or absent — when Gemini answers from training data alone (no search queries were issued), the response carries no grounding_chunks, so returning it as a GroundedResponse with an empty citations list silently misrepresents unverified content as grounded output.
3. ✗Don't omit the try/except block around client.models.generate_content() — Google Search grounding involves a live network round-trip inside Gemini's inference pipeline, making transient 502 failures qualitatively more likely than plain generation calls; swallowing the exception causes the FastAPI route to surface an unhandled 500 with no actionable detail for the caller.