Your AG2 agents are likely to need an LLM and you can configure one, or more, for each agent.

AG2’s agents can use LLMs through OpenAI, Anthropic, Google, Amazon, Mistral AI, Cerebras, Together AI, and Groq. Locally hosted models can also be used through Ollama, LiteLLM, and LM Studio.

First, we define our configuration with the API type, model, and, if necessary, the key.

import os

llm_config = {
  "config_list": [
    {
      "api_type": "openai",
      "model": "gpt-4o-mini",
      "api_key": os.environ["OPENAI_API_KEY"]
    }
  ],
}

It is important to never hard-code secrets into your code, therefore we read the OpenAI API key from an environment variable.

Then, when you create your agents you’ll set your LLM configuration:

my_agent = ConversableAgent(
    name="helpful_agent",
    llm_config=llm_config,
    system_message="You are a poetic AI assistant",
)

The default LLM provider is OpenAI but if you would like to use a different provider, see the available providers.

AG2’s LLM configuration allows you to specify many LLMs for fallback support and the ability to filter them for an agent, see the LLM Configuration deep-dive.

​
Environment variables

The examples in these guides include an LLM configuration for OpenAI’s GPT-4o mini model and will need the OPENAI_API_KEY environment variable set with your OpenAI API key.

Set it in your terminal/command prompt:

export OPENAI_API_KEY="your_api_key_here"
Installing AG2ConversableAgent