Model Configuration#

The AG2 framework provides an explicit, predictable, and type-safe way to configure Large Language Models (LLMs) for your agents. The configuration API is designed to provide a consistent developer experience across different model providers while maintaining strong typing support.

Supported Providers#

AG2 supports multiple LLM providers through dedicated configuration classes. Each provider requires its respective optional dependencies to be installed.

(Note: OpenAIConfig is also available for OpenAI-compatible endpoints).

How to Configure a Model#

Basic Configuration#

To configure a model, import the specific provider's configuration class and initialize it with your desired parameters. The most common parameters are model, api_key, and base_url.

1
2
3
4
5
6
7
8

from autogen.beta.config import OpenAIResponsesConfig

# Configure an OpenAI Responses API model
config = OpenAIResponsesConfig(
    model="gpt-4.1-nano",
    api_key="sk-...",
    streaming=True
)

1
2
3
4
5
6
7
8
9

from autogen.beta.config import OpenAIConfig

# Configure an OpenAI model
config = OpenAIConfig(
    model="gpt-4o-mini",
    api_key="sk-...",
    temperature=0.2,
    streaming=True
)

1
2
3
4
5
6
7
8

from autogen.beta.config import AnthropicConfig

# Configure an Anthropic model
config = AnthropicConfig(
    model="claude-haiku-4-5-20251001",
    api_key="sk-ant-...",
    streaming=True
)

1
2
3
4
5
6
7
8

from autogen.beta.config import GeminiConfig

# Configure a Gemini model
config = GeminiConfig(
    model="gemini-3-flash-preview",
    api_key="...",
    streaming=True
)

1
2
3
4
5
6
7

from autogen.beta.config import OllamaConfig

# Configure an Ollama model
config = OllamaConfig(
    model="qwen3.5:latest",
    streaming=True
)

1
2
3
4
5
6
7
8

from autogen.beta.config import DashScopeConfig

# Configure a DashScope model
config = DashScopeConfig(
    model="qwen-plus",
    api_key="...",
    streaming=True
)

Using Environment Variables#

For security and convenience, you don't need to hardcode your API keys. If api_key is not explicitly provided, the configuration will automatically attempt to load it from your environment variables.

The system looks for provider-specific keys (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY).

1
2
3
4

from autogen.beta.config import OpenAIConfig

# Automatically falls back to OPENAI_API_KEY from the environment
config = OpenAIConfig(model="gpt-5")

Self-Hosted and OpenAI-Compatible Models (vLLM, LM Studio, etc.)#

If you are using a self-hosted model or an API that is compatible with the OpenAI format (such as vLLM, LM Studio, FastChat, or Together AI), you can use the OpenAIConfig class and specify a custom base_url.

1
2
3
4
5
6
7
8
9

from autogen.beta.config import OpenAIConfig

# Configure a vLLM or other OpenAI-compatible endpoint
config = OpenAIConfig(
    model="qwen-3",
    base_url="http://localhost:8000/v1",
    # Some endpoints don't require an API key, but the client expects a non-empty string
    api_key="NotRequired",
)

1
2
3
4
5
6
7
8
9

import httpx
from autogen.beta.config import OpenAIConfig

config = OpenAIConfig(
    model="qwen-3",
    base_url="https://localhost:8000/v1",
    api_key="NotRequired",
    http_client=httpx.AsyncClient(verify=False)
)

Reusing and Overriding Configurations#

Model configurations are immutable. If you need to reuse a configuration for multiple agents with slight variations (e.g., changing the model version or adjusting the temperature), use the .copy() method. This creates a new updated instance without mutating the original configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

from autogen.agent import Agent
from autogen.beta.config import OpenAIConfig

base_config = OpenAIConfig(model="gpt-5")

agent1 = Agent(
    "Assistant",
    # Create a new configuration with updated temperature
    config=base_config.copy(temperature=0.2),
)

agent2 = Agent(
    "AnotherAssistant",
    # Create a new configuration with updated model and temperature
    config=base_config.copy(model="gpt-5-mini", temperature=0.8),
)

Delaying Model Configuration#

In many use cases, you may want to separate the logic of defining your agent (tools, system messages, instructions) from configuring the specific model it uses. This allows you to construct an agent once and dynamically provide the model configuration later during execution.

You can accomplish this by passing the configuration to the .ask() method when interacting with the agent. This is especially useful for applications like web servers where the user might bring their own API key or choose a different model on the fly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

from autogen.agent import Agent
from autogen.beta.config import OpenAIConfig

# Define an agent without an initial model config,
# or with a default one you plan to override later
agent = Agent(
    "Assistant",
    prompt="You are a helpful assistant.",
    # other tools and settings...
)

# Ask the agent, passing the explicit model configuration
response = await agent.ask(
    "Hello!",
    config=OpenAIConfig(
        model="gpt-5",
        api_key="sk-user-specific-key"
    )
)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

from autogen.agent import Agent
from autogen.beta.config import OpenAIConfig

agent = Agent(
    "Assistant",
    config=OpenAIConfig(model="gpt-5"),
)

response = await agent.ask(
    "Hello!",
    # overrides the original model configuration
    config=OpenAIConfig(model="gpt-5-mini")
)