LLM Configuration

Your AG2 agents are likely to need an LLM and you can configure one, or more, for each agent.

AG2's agents can use LLMs through OpenAI, Anthropic, Google, Amazon, Mistral AI, Cerebras, Together AI, and Groq. Locally hosted models can also be used through Ollama, LiteLLM, and LM Studio.

Warning

From version 0.8: The OpenAI package, openai, is not installed by default.

Install AG2 with your preferred model provider(s), for example:

pip install ag2[openai]
pip install ag2[gemini]
pip install ag2[anthropic,cohere,mistral]

Creating an LLM Configuration#

First, create the LLM configuration object with the API type, model, and key if necessary. There are two ways to do this:

Using `config_list` Parameter#

import os
from autogen import LLMConfig

llm_config = LLMConfig(
    config_list=[
        {
            "api_type": "openai",
            "model": "gpt-4o-mini",
            "api_key": os.environ["OPENAI_API_KEY"]
        },
        {
            "api_type": "openai",
            "model": "gpt-4o",
            "api_key": os.environ["OPENAI_API_KEY"]
        }
    ],
)

Using Direct Parameters#

import os
from autogen import LLMConfig

llm_config = LLMConfig(
    api_type="openai",
    model="gpt-4o-mini",
    api_key=os.environ["OPENAI_API_KEY"],
)

Warning

It is important to never hard-code secrets into your code. Use environment variables instead.

Integrating LLM Configuration with Agents#

Once the LLM configuration is created, pass it to your agents:

Passing as a Keyword Argument#

from autogen import ConversableAgent

my_agent = ConversableAgent(
    name="helpful_agent",
    system_message="You are a poetic AI assistant",
    llm_config=llm_config
)

Using a Context Manager#

from autogen import ConversableAgent

with llm_config:
    my_agent = ConversableAgent(
        name="helpful_agent",
        system_message="You are a poetic AI assistant",
    )

Default LLM Provider#

The default LLM provider is OpenAI. If you'd like to use a different provider, see the available providers.

Tip

AG2's LLM configuration allows specifying multiple LLMs for fallback support and filtering them per agent. See the LLM Configuration deep-dive.

Setting Environment Variables#

The examples require the OPENAI_API_KEY environment variable.

macOS / LinuxWindows

bash export OPENAI_API_KEY="YOUR_API_KEY"

bash setx OPENAI_API_KEY "YOUR_API_KEY"