Structured output

OpenAI offers functionality for defining a structure of the messages generated by LLMs, AG2 enables this functionality by propagating response_format, in the LLM configuration for your agents, to the underlying client. This is currently only supported by OpenAI.

For more info on structured output, please check here

pip install ag2[openai]

​

Supported clients

AG2 has structured output support for following client providers: - OpenAI (openai) - Anthropic (anthropic) - Google Gemini (google) - Ollama (ollama)

​

Set your API Endpoint

The config_list_from_json function loads a list of configurations from an environment variable or a json file. Structured Output is supported by OpenAI’s models from gpt-4-0613 and gpt-3.5-turbo-0613.

import autogen

config_list = autogen.config_list_from_json(
    "OAI_CONFIG_LIST",
    filter_dict={
        "model": ["gpt-4o", "gpt-4o-mini"],
    },
)

​

Example: math reasoning

Using structured output, we can enforce chain-of-thought reasoning in the model to output an answer in a structured, step-by-step way.

​

Define the reasoning model

First we will define the math reasoning model. This model will indirectly force the LLM to solve the posed math problems iteratively through math reasoning steps.

from pydantic import BaseModel


class Step(BaseModel):
    explanation: str
    output: str


class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

​

Define chat actors

Now we can define the agents that will solve the posed math problem. We will keep this example simple; we will use a UserProxyAgent to input the math problem and an AssistantAgent to solve it.

The AssistantAgent will be constrained to solving the math problem step-by-step by using the MathReasoning response format we defined above.

The response_format is added to the LLM configuration and then this configuration is applied to the agent.

llm_config = {
    "config_list": config_list,
    "cache_seed": 42,
    "response_format": MathReasoning,
}

user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="A human admin.",
    human_input_mode="NEVER",
)

assistant = autogen.AssistantAgent(
    name="Math_solver",
    llm_config=llm_config,  # Response Format is in the configuration
)

​

Start the chat

Let’s now start the chat and prompt the assistant to solve a simple equation. The assistant agent should return a response solving the equation using a step-by-step MathReasoning model.

user_proxy.initiate_chat(assistant, message="how can I solve 8x + 7 = -23", max_turns=1, summary_method="last_msg")

​

Formatting a response

When defining a response_format, you have the flexibility to customize how the output is parsed and presented, making it more user-friendly. To demonstrate this, we’ll add a format method to our MathReasoning model. This method will define the logic for transforming the raw JSON response into a more human-readable and accessible format.

​

Define the reasoning model

Let’s redefine the MathReasoning model to include a format method. This method will allow the underlying client to parse the return value from the LLM into a more human-readable format. If the format method is not defined, the client will default to returning the model’s JSON representation, as demonstrated in the previous example.

from pydantic import BaseModel


class Step(BaseModel):
    explanation: str
    output: str


class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

    def format(self) -> str:
        steps_output = "\n".join(
            f"Step {i + 1}: {step.explanation}\n  Output: {step.output}" for i, step in enumerate(self.steps)
        )
        return f"{steps_output}\n\nFinal Answer: {self.final_answer}"

​

Define chat actors and start the chat

The rest of the process is the same as in the previous example: define the actors and start the chat.

Observe how the Math_solver agent now communicates using the format we have defined in our MathReasoning.format method.

for config in config_list:
    config["response_format"] = MathReasoning

user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="A human admin.",
    human_input_mode="NEVER",
)

assistant = autogen.AssistantAgent(
    name="Math_solver",
    llm_config=llm_config,
)

print(
    user_proxy.initiate_chat(
        assistant, message="how can I solve 8x + 7 = -23", max_turns=1, summary_method="last_msg"
    ).summary
)

Normal function calling still works alongside structured output, so your agent can have a response format while still calling tools.

@user_proxy.register_for_execution()
@assistant.register_for_llm(description="You can use this function call to solve addition")
def add(x: int, y: int) -> int:
    return x + y


user_proxy.initiate_chat(
    assistant, message="solve 3 + 4 by calling appropriate function", max_turns=2, summary_method="last_msg"
)