Metrics#

AG2 includes a MetricsMiddleware that emits Prometheus-compatible counters and histograms for agent turns, LLM calls, tool executions, human input requests, and LLM token usage.

Use metrics when you need operational dashboards and alerts for latency, success rates, error types, and token consumption. Use Telemetry when you need request-level traces and span details.

Installation#

Install the Prometheus metrics extra:

pip install "ag2[metrics]"

If your agent uses OpenAI in the examples below, install both extras:

pip install "ag2[openai,metrics]"

Quick Start#

Create one Prometheus CollectorRegistry, create one MetricsMiddleware for that registry, then reuse the middleware on the agents that should emit into it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import asyncio

from prometheus_client import CollectorRegistry, start_http_server

from ag2 import Agent
from ag2.config import OpenAIConfig
from ag2.middleware import MetricsMiddleware

async def main() -> None:
    registry = CollectorRegistry()
    metrics = MetricsMiddleware(registry=registry)

    start_http_server(8000, registry=registry)

    agent = Agent(
        "assistant",
        prompt="You are a helpful assistant.",
        config=OpenAIConfig(model="gpt-4o-mini"),
        middleware=[metrics],
    )

    reply = await agent.ask("Give me a one sentence project status template.")
    print(reply.body)

if __name__ == "__main__":
    asyncio.run(main())

Run the process, then open http://localhost:8000/metrics or configure Prometheus to scrape that endpoint.

Metrics Reference#

Counters#

Histograms#

Prometheus histograms also expose _bucket, _count, and _sum series for each metric.

Label Reference#

Missing or empty label values are normalized to unknown, except error_type, which is an empty string for successful operations. Token values of zero are not emitted, and missing token fields are not synthesized as zero. Which token_type values appear depends on what the provider reports (for example, thinking only for reasoning models and cache_read_input / cache_creation_input only when prompt caching is active).

Configuration#

MetricsMiddleware accepts:

CollectorRegistry Lifecycle#

Create one MetricsMiddleware per CollectorRegistry, then share that middleware instance across agents that should emit to the same registry.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

from prometheus_client import CollectorRegistry

from ag2 import Agent
from ag2.config import OpenAIConfig
from ag2.middleware import MetricsMiddleware

registry = CollectorRegistry()
metrics = MetricsMiddleware(registry=registry)

assistant = Agent(
    "assistant",
    config=OpenAIConfig(model="gpt-4o-mini"),
    middleware=[metrics],
)

reviewer = Agent(
    "reviewer",
    config=OpenAIConfig(model="gpt-4o-mini"),
    middleware=[metrics],
)

Prometheus Integration#

For a standalone worker or service process, the simplest integration is start_http_server() from prometheus_client:

1
2
3
4
5
6
7
8

from prometheus_client import CollectorRegistry, start_http_server

from ag2.middleware import MetricsMiddleware

registry = CollectorRegistry()
metrics = MetricsMiddleware(registry=registry)

start_http_server(8000, registry=registry)

Prometheus can scrape the endpoint with a job like:

scrape_configs:
  - job_name: ag2
    metrics_path: /metrics
    static_configs:
      - targets:
          - localhost:8000

For ASGI applications, Prometheus Python client can expose the same CollectorRegistry as an ASGI app via make_asgi_app(registry=registry). For example, you can serve it directly with uvicorn:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import uvicorn
from prometheus_client import CollectorRegistry, make_asgi_app

from ag2.middleware import MetricsMiddleware

registry = CollectorRegistry()
metrics = MetricsMiddleware(registry=registry)

app = make_asgi_app(registry=registry)

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

Then scrape http://localhost:8000/. If you prefer the CLI form, expose the same app object from a module and run uvicorn your_module:app.

Grafana Dashboard#

Useful starting panels:

LLM call rate by provider, model, and outcome:

sum by (provider, model, outcome) (rate(ag2_llm_calls_total[5m]))

LLM token rate by token type:

sum by (provider, model, token_type) (rate(ag2_llm_tokens_total[5m]))

95th percentile LLM latency:

histogram_quantile(
  0.95,
  sum by (le, provider, model) (rate(ag2_llm_call_duration_seconds_bucket[5m]))
)

Agent turn success rate:

sum(rate(ag2_agent_turns_total{outcome="success"}[5m]))
/
sum(rate(ag2_agent_turns_total[5m]))

Tool error rate by tool and error type:

sum by (tool, error_type) (rate(ag2_tool_calls_total{outcome="error"}[5m]))

Create separate rows for agent turns, LLM calls, tools, human input, and token usage. Use outcome and error_type as dashboard filters when investigating failures.

Best Practices#

• Keep agent, tool, provider, and model labels low-cardinality. Do not include user IDs, request IDs, session IDs, or other unbounded values in agent or tool names.
• Use stable, descriptive agent names such as support_triage or invoice_reviewer.
• Reuse a single MetricsMiddleware instance per registry across all agents in one process.
• If you combine metrics with RetryMiddleware and want each retry attempt counted as a separate LLM call, register RetryMiddleware before MetricsMiddleware.
• Streaming LLM durations are recorded after the final response arrives, so dashboards update when the streamed call completes.