Tracing AI agents with OpenTelemetry
How to instrument autonomous agents with OpenTelemetry's GenAI conventions so every reasoning step, tool call and cost is captured in one portable trace.
Definition
Agent tracing records the full execution path of an agent run as a tree of spans — prompts, model calls, tool invocations and sub-agent steps — each with timing, tokens and outcome. OpenTelemetry's GenAI semantic conventions are becoming the de facto standard for representing these spans portably.
When an agent fails, the question is always the same: what did it actually do? Tracing answers it. A trace turns a thirty-second, five-tool run into a readable tree you can inspect step by step.
What a trace captures
A trace is the full tree of one agent run. Each node is a span: a prompt, a model call, a tool invocation, a sub-agent step. A good span records its operation type, inputs and outputs, token counts, cost, latency and status. Read top to bottom, the spans reconstruct exactly how the agent reasoned and acted.
This is the unit of three things at once: debugging (where did it go wrong), costing (which step burned tokens), and audit (what decision led to what action).
| Span attribute | What it records | Used for |
|---|---|---|
| Operation type | Model call, tool call, retrieval, sub-agent | Reading the run as a tree |
| Inputs / outputs | The prompt and result, scoped | Debugging |
| Tokens | Prompt and completion tokens | Costing |
| Cost | Money spent on the step | FinOps and budgets |
| Latency | How long the step took | Performance |
| Status | Success or error | Locating failures |
Why OpenTelemetry’s GenAI conventions?
The important shift of 2025 was convergence on OpenTelemetry’s GenAI semantic conventions. Instead of every platform inventing its own trace format, agent spans now follow a shared schema. Two practical wins:
- Portability. Your traces are not locked to one vendor; any compatible backend can read them.
- Unification. Agent spans sit alongside your existing service traces, so one backend covers the whole stack.
Dedicated agent-observability platforms build value on top — session replay, time-travel debugging, agent dashboards — but the standard is what protects you as tools churn.
Instrumenting an agent
- Wrap the model calls. Emit a span per inference with token and cost attributes.
- Wrap each tool. A span per tool call with inputs, outputs and status — this is where most failures hide.
- Nest sub-agents. Child spans under the parent run preserve the tree.
- Propagate context. Carry the trace context across async steps so the tree stays connected.
- Export async. Sample and export off the hot path to keep overhead negligible.
From traces to operations
Once traces exist, the rest of observability follows: completion rate, cost per task and latency are aggregations over spans; loop detection is a pattern over them; and the audit logs governance expects are your traces made tamper-evident. Start by tracing one agent end to end — everything else builds on it. Terms are in the glossary.
Frequently asked questions
Why OpenTelemetry rather than a vendor SDK?
Because it keeps your traces portable at exactly the moment the tooling around agents is changing fastest. With OpenTelemetry's GenAI semantic conventions, agent spans follow a shared, open schema and land in the same backend as the rest of your services, so you get one view of your whole stack rather than an isolated agent silo. A vendor SDK can certainly capture rich traces, and the good ones emit or read the standard, but building only on a proprietary format means your history and your dashboards are hostage to one supplier's roadmap and pricing. The pragmatic stance is to instrument against the open convention first and treat dedicated platforms as value added on top — session replay, agent dashboards, time-travel debugging — rather than as the foundation. That way you can switch or add tools later without re-instrumenting your agents, which is a real risk in a field where today's leading platform may not be next year's. Portability is cheap to choose now and expensive to retrofit.
What should a span capture for an agent?
Each span should record enough to reconstruct and account for one step of the run, which means six things at minimum: the operation type (model call, tool call, retrieval or sub-agent step), the inputs and outputs at an appropriate level of detail, token counts, monetary cost, latency, and a status that says whether it succeeded. Capturing inputs and outputs needs judgement — you want enough to debug without logging secrets or huge blobs, so scope or redact sensitive fields and truncate where needed. The operation type is what lets a viewer render the run as a readable tree; tokens and cost turn the same span into a FinOps record; latency and status make it a performance and failure signal. Read top to bottom, well-formed spans reconstruct exactly how the agent reasoned and acted, which is why the discipline of attaching these attributes consistently matters more than any particular tool. A span missing tokens cannot be costed; one missing status cannot be triaged.
Does tracing add too much overhead?
In practice no, because tracing is cheap relative to the thing that dominates an agent's latency and cost: model inference. A span is a small amount of structured metadata; an inference is hundreds of milliseconds and real money, so the relative overhead of recording the step is negligible. Two standard techniques keep it that way. Asynchronous export moves the work of shipping spans off the hot path, so your agent does not wait on the telemetry backend, and sampling lets you record full detail on a representative fraction of runs while still counting every run for aggregate metrics. The mistake is to skip tracing to save overhead and then fly blind when an agent loops, fails or overspends — the cost of not seeing what happened dwarfs the cost of recording it. Treat tracing as part of the agent, not an optional add-on, and tune sampling and retention to your volume rather than turning it off.
What is the difference between a trace, a span, and a log?
A span is the record of a single operation — one model call, one tool invocation — with its start time, duration, attributes and status. A trace is the tree of spans that together represent one complete agent run, linked by parent-child relationships so you can see that this tool call happened inside that planning step inside that sub-agent. A log, by contrast, is a discrete timestamped message, useful for ad-hoc detail but flat: a stream of lines without the structure that shows how steps nest and relate. For agents the trace is the unit that matters, because the interesting failures are about sequence and causation — the agent retrieved a poisoned document, then made an unusual tool call — which a tree captures and a log stream obscures. In practice you still emit logs, but you attach them to the relevant span so they inherit its context. Think of spans as the steps, the trace as the whole story, and logs as annotations on individual steps.