Integrate with Langfuse

Introduction

When you deploy an agent to production, understanding what the model actually did — which prompts were sent, how many tokens were consumed, and where latency spiked — becomes critical for debugging and cost control. Without structured traces, failures surface only as vague symptoms and growing bills. Langfuse provides an open-source observability layer that captures this data automatically through a decorator-based API that wraps your existing functions with minimal code changes. By the end of this lesson, you'll be able to instrument a Python agent with Langfuse, view nested traces in the dashboard, and attach token-usage metadata to generation spans.

Key Terminology

• Trace — the root-level record that Langfuse creates when the outermost @observe-decorated function is called; it captures total end-to-end duration, the associated user_id, and any custom metadata set via langfuse_context.update_current_observation.
• Generation span — a child span created by decorating an LLM-calling function with @observe(as_type="generation"); unlike a generic span, it accepts a usage dict that enables per-request token counting and cost calculation in the Langfuse dashboard.
• @observe decorator — the primary Langfuse integration point that wraps a Python function to automatically record its inputs, outputs, and latency as a trace or span, with no manual SDK calls required inside the function body.
• langfuse_context.update_current_observation() — a thread-local context method called from inside a decorated function to attach runtime data — such as user_id, model, input, output, or a usage token breakdown — to the span that is currently being recorded.
• langfuse.flush() — an explicit drain of Langfuse's internal event buffer that must be called before a short-lived process exits; omitting it is the most common reason traces appear missing in the dashboard.
• Span hierarchy — the parent-child nesting between a root trace and its child spans or generations, produced automatically when @observe-decorated functions call each other; Langfuse uses this tree to display per-span timing and cost in a single drill-down view.

Concepts

Why Production Agents Need Structured Observability

Running an LLM agent in production without instrumentation is like running a web server without logs: failures surface only as vague symptoms — slow responses, unexpected outputs, or a growing API bill — with no way to identify which prompt, which model call, or which tool invocation was responsible. Langfuse solves this by capturing a structured record of every function call in the agent's execution path: what inputs went in, what came out, how long each step took, and how many tokens were consumed.

The key advantage over ad-hoc print statements is that Langfuse organizes this data hierarchically, so you can zoom out to see the full agent invocation or zoom into a single generation to inspect the exact prompt sent to the model (see Code Walkthrough).

Traces, Spans, and Generations

Langfuse represents an agent execution as a tree. The trace is the root node, created when the outermost instrumented function is called — in this lesson, chat_with_agent. Every nested call decorated with @observe becomes a child span inside that trace, recording its own start time, duration, and inputs/outputs independently.

A special subtype of child span is the generation: when a function wraps an LLM call and is decorated with as_type="generation", Langfuse treats it differently from a generic span by expecting a usage dictionary containing input, output, and total token counts. This allows the dashboard to compute per-call cost and display token breakdowns that a plain span cannot produce. In this lesson, call_model is the generation; chat_with_agent is the root trace that contains it.

This hierarchy is produced automatically by the nesting of decorated function calls — no explicit parent-ID wiring is required.

Decorator-Driven Instrumentation and Metadata Attachment

The @observe decorator is designed to minimize the instrumentation footprint. Wrapping an existing function with @observe is enough to start capturing its call signature, return value, and wall-clock duration; the function body itself does not need to import or call any Langfuse SDK methods to produce a visible span.

When richer metadata is needed — associating a trace with a specific user, recording the model name, or attaching the token breakdown — langfuse_context.update_current_observation() provides a context-local escape hatch. Because Langfuse uses a thread-local context internally, this call knows which span is currently executing without requiring the developer to pass a span object through every layer of the call stack (see Code Walkthrough for how user_id is attached on the trace level in chat_with_agent while usage is attached on the generation level in call_model).

One operational detail that frequently causes confusion in short-lived scripts: Langfuse batches events before sending them to reduce network overhead. The langfuse.flush() call at the end of the script forces the buffer to drain synchronously. In a long-running server process the buffer drains continuously, but in a script that exits within seconds, skipping flush() silently drops all buffered events and the trace never appears in the dashboard.

Code Walkthrough

Now that you understand how Langfuse traces, spans, and generations map to the lifecycle of an LLM call, the following example wires those concepts directly into runnable code.

The @observe decorator is the central integration point. Wrapping a function with @observe tells Langfuse to automatically create a trace, record the function's inputs and outputs, and measure latency — with no manual SDK calls required inside the function body. For inner LLM calls, passing as_type="generation" unlocks token counting and cost calculation; as_type="span" handles tool calls or any other sub-operation you want to time independently.

Code snippetpython
1import os
2from openai import OpenAI
3from langfuse import Langfuse
4from langfuse.decorators import observe, langfuse_context
5
6langfuse = Langfuse(
7    public_key=os.getenv("LANGFUSE_PUBLIC_KEY"),
8    secret_key=os.getenv("LANGFUSE_SECRET_KEY"),
9    host=os.getenv("LANGFUSE_HOST", "https://cloud.langfuse.com"),
10)
11
12openai_client = OpenAI()
13
14@observe(name="agent-chat")
15def chat_with_agent(user_message: str, user_id: str) -> str:
16    langfuse_context.update_current_observation(
17        metadata={"source": "api"},
18        user_id=user_id,
19    )
20    return call_model(user_message)
21
22@observe(name="llm-call", as_type="generation")
23def call_model(prompt: str) -> str:
24    response = openai_client.chat.completions.create(
25        model="gpt-4o-mini",
26        messages=[{"role": "user", "content": prompt}],
27    )
28    content = response.choices[0].message.content
29    langfuse_context.update_current_observation(
30        model="gpt-4o-mini",
31        input=prompt,
32        output=content,
33        usage={
34            "input": response.usage.prompt_tokens,
35            "output": response.usage.completion_tokens,
36            "total": response.usage.total_tokens,
37            "unit": "TOKENS",
38        },
39    )
40    return content
41
42if __name__ == "__main__":
43    answer = chat_with_agent("What is LLM observability?", user_id="user-123")
44    print(answer)
45    langfuse.flush()

The outer chat_with_agent function is the root trace: Langfuse records its total duration and associates the trace with the user_id supplied to update_current_observation. The inner call_model function is decorated with as_type="generation", which tells Langfuse to record it as a generation child span rather than a generic operation. Attaching the usage dict inside that function enables the per-request token and cost breakdown you can drill into from the Langfuse dashboard. The final langfuse.flush() call ensures all buffered events are sent before the process exits — omitting it in a short-lived script is the most common reason traces appear missing.

Confirm that after running this script, a new trace named agent-chat appears in your Langfuse dashboard, contains a nested llm-call generation child span, and displays accurate prompt and completion token counts matching the OpenAI response.

Do's and Don'ts

Having walked through integrating with Langfuse above, the following Do's and Don'ts distill it into practice.

Do's

1. ✓Do call langfuse.flush() at the end of every short-lived script — Langfuse buffers events asynchronously, and omitting flush() is the most common reason traces appear entirely missing in the dashboard after a script exits.
2. ✓Do use as_type="generation" on the function that wraps your LLM call and attach the usage dict via langfuse_context.update_current_observation — without this, the Langfuse dashboard has no token counts and cannot calculate per-request cost breakdowns; as_type="span" on that same function only records timing, not consumption.
3. ✓Do set user_id on the root trace via langfuse_context.update_current_observation inside the @observe-decorated entry function — attaching it at the outermost trace (e.g., chat_with_agent) propagates the identity to every nested child span, making cross-request debugging and per-user cost attribution possible from the dashboard.

Don'ts

1. ✗Don't place langfuse_context.update_current_observation calls inside a function whose decorator lacks the matching as_type — calling update_current_observation with model, input, output, and usage inside a plain @observe span (not as_type="generation") silently discards the token-counting fields; Langfuse only promotes those fields to cost calculations on generation-typed spans.
2. ✗Don't initialize Langfuse(...) with hardcoded key strings instead of os.getenv("LANGFUSE_PUBLIC_KEY") and os.getenv("LANGFUSE_SECRET_KEY") — the Langfuse client authenticates every event flush with these credentials, so embedding them in source code leaks project access and makes rotating keys require a code change rather than an environment update.
3. ✗Don't omit the name= argument on @observe — without an explicit name, Langfuse falls back to the Python function name as the trace or span label; in the dashboard this produces ambiguous entries like call_model that are indistinguishable across multiple agents, making it impossible to filter or compare specific operations by intent.