OpenTelemetry Tracing for AG2 Agents#

AG2 provides built-in OpenTelemetry instrumentation for tracing multi-agent conversations. This lets you observe agent interactions, LLM calls, tool executions, and more using any OpenTelemetry-compatible backend (Jaeger, Grafana Tempo, Datadog, etc.).

Requirements

Install AG2 with tracing support:

pip install "ag2[openai,tracing]"

For this notebook we use ConsoleSpanExporter so no external backend is needed.

Setting Up OpenTelemetry#

First, configure a TracerProvider with a ConsoleSpanExporter. This prints spans directly to stdout, which is perfect for learning. In production, you would replace this with an OTLP exporter.

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor

resource = Resource.create(attributes={"service.name": "ag2-tracing-notebook"})
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
trace.set_tracer_provider(tracer_provider)

Creating and Instrumenting Agents#

Create agents as usual, then call instrument_llm_wrapper (once, globally) and instrument_agent (per agent) to enable tracing. No changes to your agent code are needed.

from autogen import ConversableAgent, LLMConfig
from autogen.opentelemetry import instrument_agent, instrument_llm_wrapper

llm_config = LLMConfig(api_type="openai", model="gpt-4o-mini")

assistant = ConversableAgent(
    "assistant",
    system_message="You are a helpful assistant. Reply concisely in one or two sentences.",
    llm_config=llm_config,
    human_input_mode="NEVER",
)

user_proxy = ConversableAgent(
    "user_proxy",
    human_input_mode="NEVER",
    llm_config=False,
    max_consecutive_auto_reply=0,
)

# Instrument the LLM wrapper (global, once) and each agent
instrument_llm_wrapper(tracer_provider=tracer_provider)
instrument_agent(assistant, tracer_provider=tracer_provider)
instrument_agent(user_proxy, tracer_provider=tracer_provider)

Two-Agent Chat with Tracing#

Run a simple chat. The console output will show the spans emitted during the conversation. The trace hierarchy looks like:

conversation "user_proxy" -> "assistant"
  └── invoke_agent "assistant"
        └── chat gpt-4o-mini

Key attributes on each span include ag2.span.type, gen_ai.operation.name, gen_ai.agent.name, and token usage.

result = user_proxy.initiate_chat(
    assistant,
    message="What are the three primary colors?",
    max_turns=1,
)

Tracing Tool Execution#

When agents use tools, tool execution spans are automatically captured as children of the agent span. The trace hierarchy expands to:

conversation "tool_user" -> "tool_agent"
  ├── invoke_agent "tool_agent"
  │     └── chat gpt-4o-mini          (LLM decides to call tool)
  ├── invoke_agent "tool_user"
  │     └── execute_tool get_weather   (tool execution)
  └── invoke_agent "tool_agent"
        └── chat gpt-4o-mini          (LLM processes tool result)

from typing import Annotated

def get_weather(city: Annotated[str, "The city to get weather for"]) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is sunny, 72F."

tool_agent = ConversableAgent(
    "tool_agent",
    system_message="Use the get_weather tool to answer weather questions. Reply concisely.",
    llm_config=llm_config,
    human_input_mode="NEVER",
)

tool_user = ConversableAgent(
    "tool_user",
    human_input_mode="NEVER",
    llm_config=False,
    max_consecutive_auto_reply=2,
)

tool_agent.register_for_llm()(get_weather)
tool_user.register_for_execution()(get_weather)

instrument_agent(tool_agent, tracer_provider=tracer_provider)
instrument_agent(tool_user, tracer_provider=tracer_provider)

result = tool_user.initiate_chat(
    tool_agent,
    message="What's the weather in Tokyo?",
    max_turns=2,
)

Tracing Group Chats#

For group chat patterns, use instrument_pattern to automatically instrument all agents in the group, including the GroupChatManager and speaker selection logic.

from autogen.agentchat import initiate_group_chat
from autogen.agentchat.group.patterns.auto import AutoPattern
from autogen.opentelemetry import instrument_pattern

planner = ConversableAgent(
    "planner",
    system_message="You plan tasks. Be concise and reply in one sentence.",
    llm_config=llm_config,
    human_input_mode="NEVER",
)
executor = ConversableAgent(
    "executor",
    system_message="You execute planned tasks. Be concise and reply in one sentence.",
    llm_config=llm_config,
    human_input_mode="NEVER",
)

group_user = ConversableAgent(
    "group_user",
    human_input_mode="NEVER",
    llm_config=False,
)

pattern = AutoPattern(
    initial_agent=planner,
    agents=[planner, executor],
    user_agent=group_user,
    group_manager_args={"llm_config": llm_config},
)

instrument_pattern(pattern, tracer_provider=tracer_provider)

result, context, last_agent = initiate_group_chat(
    pattern=pattern,
    messages="Write a haiku about coding.",
    max_rounds=3,
)

Sending Traces to a Backend#

For production use, replace ConsoleSpanExporter with an OTLP exporter to send traces to Jaeger, Grafana Tempo, Datadog, or any OpenTelemetry-compatible backend.

from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor

exporter = OTLPSpanExporter(endpoint="http://localhost:4317")
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(tracer_provider)

A docker-compose stack with Grafana Tempo and OpenTelemetry Collector is included in the tracing/ directory of the AG2 repo:

cd tracing && docker-compose up -d
# Grafana UI: http://localhost:3333
# OTLP gRPC endpoint: localhost:14317

Summary#

AG2’s OpenTelemetry integration gives you full observability into multi-agent workflows:

instrument_agent traces conversations, agent replies, tool calls, code execution, and human input
instrument_llm_wrapper traces all LLM API calls with token usage and cost
instrument_pattern traces group chats including speaker selection
All spans follow the OpenTelemetry GenAI Semantic Conventions

For more details, see the Tracing documentation.