Skip to content

Backend deep dive

This page expands on the backend of the AG-UI integration: how to secure your endpoint, how tool calls are represented in the protocol, and how to pass per-request context into your tools.

If you haven’t set up an AG-UI endpoint yet, start with the AG-UI overview.

Authentication#

Because AG-UI integration works on top of HTTP SSE endpoints, you can validate headers or tokens before streaming events.

Example: protect /chat with a shared token header.

run_ag_ui_auth.py
from typing import Annotated

from fastapi import FastAPI, Header, HTTPException, Request
from fastapi.responses import JSONResponse, Response, StreamingResponse

from ag2 import Agent
from ag2.ag_ui import AGUIStream, read_run_input
from ag2.config import OpenAICompletionsConfig

agent = Agent(
    name="support_bot",
    prompt="You help users with billing questions.",
    config=OpenAICompletionsConfig(model="gpt-4o-mini"),
)

stream = AGUIStream(agent)
app = FastAPI()

@app.post("/chat")
async def run_agent(
    request: Request,
    token: Annotated[str, Header(..., description="Authentication token")],
) -> Response:
    if token != "1234567890":
        raise HTTPException(status_code=401, detail="Invalid token")
    try:
        message = read_run_input(await request.body())
    except ValueError:
        return JSONResponse({"error": "invalid AG-UI RunAgentInput body"}, status_code=400)
    return StreamingResponse(
        stream.dispatch(message, accept=request.headers.get("accept")),
        media_type="text/event-stream",
    )

The body goes through read_run_input, which strips what a newer client sends that this server does not recognise, and the content type is always text/event-stream (see the overview).

Notes:

  • Do not put secrets (API keys, auth tokens) in the browser bundle. Protect the Next.js runtime route (/api/copilotkit) and forward credentials server-to-server.
  • If you expose a capabilities GET route alongside this POST route, apply the same authentication check there.
  • To give tools the verified user, pass it to dispatch(dependencies=...) rather than variables. The weather sample verifies a signed token in a FastAPI dependency and does exactly that, so the user is never echoed back to the browser or supplied by the client.
  • If you don’t need auth or middleware, add the generated AGUIStream.build_asgi() endpoint as a route, as shown in the basic server example.

Tools context#

You often need to pass per-request context (user ID, org ID, plan, permissions) into tools without baking it into prompts or trusting client-provided text.

AG2 supports this through injected Context and dispatch(variables=...). Your tool reads values from context.variables.

tools_context.py
from ag2 import Agent, Context, tool
from ag2.ag_ui import AGUIStream, RunAgentInput
from ag2.config import OpenAICompletionsConfig

@tool
def get_user_profile(context: Context) -> str:
    user_id = context.variables.get("user_id")
    return f"User profile for user {user_id}"

agent = Agent(
    name="profile_bot",
    prompt="You can look up a user profile when needed.",
    config=OpenAICompletionsConfig(model="gpt-4o-mini"),
    tools=[get_user_profile],
)

stream = AGUIStream(agent)

def dispatch_with_context(message: RunAgentInput, user_id: str):
    return stream.dispatch(message, variables={"user_id": user_id})

Variables are shared state

Variables reach the client too: each STATE_SNAPSHOT carries the client's state and the server's variables together (see Shared state). A user id is fine to share. A token or an API key is not: pass it to dispatch(dependencies=...), which never leaves the server.

Backend tools (Python functions)#

Backend tools are regular Python callables registered on the agent (for example via tools=[...]). When the agent invokes a tool during a run, the AG-UI stream emits tool lifecycle events that a UI can render in real time.

Example backend tool:

backend_tools.py
from ag2 import Agent, tool
from ag2.config import OpenAICompletionsConfig

@tool
def calculate_sum(a: int, b: int) -> int:
    """Adds two numbers and returns the result."""
    return a + b

agent = Agent(
    name="calculator",
    tools=[calculate_sum],
    config=OpenAICompletionsConfig(model="gpt-4o-mini"),
)

A tool may return more than text. An image, a document or structured data reaches a 1.0 client as content parts (see Tool results).

In an AG-UI-compatible UI, you typically render these as:

  • Status updates (tool started / finished)
  • Structured result cards (e.g., “Weather in Tokyo”)
  • Debug panels in development

Frontend tools (UI-driven actions)#

Frontend tools are defined by the UI/client and sent to the agent as part of the run payload (for example, CopilotKit “actions” / GenUI tools). They are useful for:

  • Generative UI (custom cards, lists, buttons rendered from tool calls)
  • HITL (human-in-the-loop) input flows (buttons, forms, confirmations)

In this setup:

  • The frontend advertises available tools in RunAgentInput.tools.
  • The agent can call those tools during the run.
  • The frontend executes/handles the tool and renders the UI.

A run that stops on client tool calls finishes with a success outcome naming the calls it started, in call order, in pendingToolCallIds. A call an earlier run announced is not repeated: the run that paused on a question ended on an interrupt outcome, which carries no pending ids. The client answers them in the next run's messages, and a tool message may carry content parts, not only a string.

This frontend-tool round trip is separate from an AG2 context.input() interrupt. To resume an interrupt, the client must echo its metadata in the resume entry; CopilotKit's useInterrupt().resolve() and .cancel() do not send that metadata. See Asking the human a question.

A frontend tool never replaces a backend tool with the same name: the backend tool is kept and the frontend one is dropped with a warning. Give frontend tools names of their own.

For a production-ready React/Next.js client that supports frontend tools, backend tools, streaming, and shared state, see the CopilotKit UI quickstart and CopilotKit’s docs on:

See also#