Skip to content

Structured Outputs: Ensuring Consistent Responses

In our financial compliance system, we've built a workflow that checks transactions, detects duplicates, and generates summary reports. Everything seems to be working well, but then we get a new requirement:

The risk management team needs to integrate our transaction summary reports with their systems, and they need the data in a specific format.

Our summary agent currently produces nice-looking reports, but the output format isn't guaranteed. The agent decides how to format the data based on its system message instructions. This inconsistency makes it impossible for the risk management system to reliably extract the data it needs.

What we need is a way to ensure our summary bot always produces reports in exactly the same structured format, regardless of the specific transactions being processed.

The Need for Structured Outputs#

We're faced with a common challenge in AI systems: how do we get our agents to output information in a consistent, predictable format that other systems can understand?

Think about it like this - our finance bot and summary bot are doing great work analyzing transactions, but if we want to connect them to other business systems, we need something more reliable than hoping they'll format data consistently each time.

This is where structured outputs come in - they provide a way to define the exact structure your agent's responses should follow.

An Analogy for Structured Outputs:

Continuing our hospital emergency room analogy:

  • Think of structured outputs like standardized medical forms used in the hospital
  • Without standardized forms, each doctor might document patient information differently (freeform text)
  • With standardized forms, all required fields must be completed and in the correct format
  • Technicians know exactly where to find vitals, lab results, and medications without guessing
  • Downstream systems (insurance, billing, pharmacy) can reliably process these forms
  • The forms act as guardrails, ensuring all necessary information is captured and consistently structured

Just as standardized medical forms ensure complete and consistent documentation across different healthcare providers, structured outputs ensure your agents produce responses in a predictable format that can be reliably processed by other systems.

From Concept to Implementation#

Implementing structured outputs in AG2 is straightforward using Pydantic models and the response_format parameter:

Note

Not all model providers support structured outputs. For a current list of supported providers, see here.

from pydantic import BaseModel
from autogen import ConversableAgent, LLMConfig

# 1. Define your structured output model with Pydantic
class ResponseModel(BaseModel):
    field1: str
    field2: int
    field3: list[str]

# 2. Create LLM configuration with the structured output model
llm_config = LLMConfig(
    api_type="openai",
    model="gpt-4o-mini",
    response_format=ResponseModel,  # Specify the response format
)

# 3. Create agent with structured output configuration
with llm_config:
    structured_agent = ConversableAgent(
        name="structured_agent",
        system_message="You provide information in a structured format."
    )

With this setup:

  • The agent will always return responses that match the ResponseModel structure
  • All required fields will be present and correctly typed
  • You no longer need formatting instructions in the system message
  • The responses can be easily parsed and processed by other systems

Enhancing Our Financial Compliance System with Structured Outputs#

Now, let's enhance our financial compliance system by adding structured outputs to our summary agent:

Defining the Structured Output Model#

First, let's define our structured output model for transaction audit logs:

from pydantic import BaseModel

# Define structured output for audit logs
class TransactionAuditEntry(BaseModel):
    vendor: str
    amount: float
    memo: str
    status: str
    reason: str

class AuditLogSummary(BaseModel):
    total_transactions: int
    approved_count: int
    rejected_count: int
    transactions: List[TransactionAuditEntry]
    summary_generated_time: str

Creating the Summary Agent with Structured Output#

Next, let's update our summary agent to use this structured output model:

# Configure the LLM for summary bot with structured output
summary_llm_config = LLMConfig(
    api_type="openai",
    model="gpt-4o-mini",
    api_key=os.environ.get("OPENAI_API_KEY"),
    temperature=0.2,
    response_format=AuditLogSummary,  # Using our Pydantic model for structured output
)

# Define the system message for the summary agent
# Note: No formatting instructions needed!
summary_system_message = """
You are a financial summary assistant that generates audit logs.
Analyze the transaction details and their approval status from the conversation.
Include each transaction with its vendor, amount, memo, status and reason.
"""

# Create the summary agent with structured output
with summary_llm_config:
    summary_bot = ConversableAgent(
        name="summary_bot",
        system_message=summary_system_message,
    )

Notice how much simpler the system message is now - we no longer need to include detailed formatting instructions because the structure is enforced by the Pydantic model.

Updating the Termination Condition#

We'll also update our termination condition to recognize the structured output:

def is_termination_msg(x: dict[str, Any]) -> bool:
    content = x.get("content", "")
    try:
        # Try to parse the content as JSON (structured output)
        data = json.loads(content)
        return isinstance(data, dict) and "summary_generated_time" in data
    except:
        # If not JSON, check for text marker (fallback)
        return (content is not None) and "==== SUMMARY GENERATED ====" in content

Using the Structured Output#

After the swarm completes, we can easily get the final message and parse it as JSON. For example:

result, _, _ = initiate_swarm_chat(
    initial_agent=finance_bot,
    agents=[finance_bot, summary_bot, human],
    messages=initial_prompt,
    swarm_manager_args={
        "llm_config": finance_llm_config,
        "is_termination_msg": is_termination_msg,
    },
    after_work=AfterWorkOption.SWARM_MANAGER
)

final_message = result.chat_history[-1]["content"]
structured_data = json.loads(final_message)
print(json.dumps(structured_data, indent=4))

Understanding the Enhanced Workflow#

With structured outputs added to our financial compliance system, our workflow becomes much more reliable:

flowchart TD
    User([User]) --> |"Submits transactions"| Finance[Finance Bot]
    Finance --> |"Processes transactions"| Human[Human Agent]
    Human --> |"Approves/rejects"| Finance
    Finance --> |"Handoff"| Summary[Summary Bot with Structured Output]
    Summary --> |"Generates structured audit log"| Output[["JSON in exact required format"]]
    Output --> RiskSystem[Risk Management System]
    Output --> Dashboard[Finance Dashboard]
    Output --> Database[(Transaction Database)]

    style Finance fill:#e3f2fd,stroke:#1976d2
    style Human fill:#f3e5f5,stroke:#7b1fa2
    style Summary fill:#e8f5e9,stroke:#4caf50
    style Output fill:#fff8e1,stroke:#ffa000
    style RiskSystem fill:#e8f5e9,stroke:#4caf50
    style Dashboard fill:#e0f2f1,stroke:#009688
    style Database fill:#f3e5f5,stroke:#7b1fa2

Complete Code Example#

Here's the complete, ready-to-run code for our financial compliance system with structured outputs:

Complete Code Example 
import os
import random
import json
from typing import Annotated, Any, List, Dict
from datetime import datetime, timedelta
from pydantic import BaseModel

from autogen import ConversableAgent, AfterWorkOption, initiate_swarm_chat, LLMConfig, register_function

# Mock database of previous transactions
def get_previous_transactions() -> list[dict[str, Any]]:
    return [
        {"vendor": "Staples", "amount": 500, "date": "2025-04-15", "memo": "Quarterly supplies"},
        {"vendor": "Acme Corp", "amount": 1500, "date": "2025-04-16", "memo": "NDA services"},
        {"vendor": "Globex", "amount": 12000, "date": "2025-04-17", "memo": "Confidential"},
    ]

# Simple duplicate detection function
def check_duplicate_payment(
    vendor: Annotated[str, "The vendor name"],
    amount: Annotated[float, "The transaction amount"],
    memo: Annotated[str, "The transaction memo"]
) -> dict[str, Any]:
    """Check if a transaction appears to be a duplicate of a recent payment"""
    previous_transactions = get_previous_transactions()

    today = datetime.now()

    for tx in previous_transactions:
        tx_date = datetime.strptime(tx["date"], "%Y-%m-%d")
        date_diff = (today - tx_date).days

        # If vendor, memo and amount match, and transaction is within 7 days
        if (
            tx["vendor"] == vendor and
            tx["memo"] == memo and
            tx["amount"] == amount and
            date_diff <= 7
        ):
            return {
                "is_duplicate": True,
                "reason": f"Duplicate payment to {vendor} for ${amount} on {tx['date']}"
            }

    return {
        "is_duplicate": False,
        "reason": "No recent duplicates found"
    }

# Configure the LLM for finance bot (standard configuration)
finance_llm_config = LLMConfig(
    api_type="openai",
    model="gpt-4o-mini",
    api_key=os.environ.get("OPENAI_API_KEY"),
    temperature=0.2
)

# Define the system message for our finance bot
finance_system_message = """
You are a financial compliance assistant. You will be given a set of transaction descriptions.

For each transaction:
1. First, extract the vendor name, amount, and memo
2. Check if the transaction is a duplicate using the check_duplicate_payment tool
3. If the tool identifies a duplicate, automatically reject the transaction
4. If not a duplicate, continue with normal evaluation:
    - If it seems suspicious (e.g., amount > $10,000, vendor is unusual, memo is vague), ask the human agent for approval
    - Otherwise, approve it automatically

Provide clear explanations for your decisions, especially for duplicates or suspicious transactions.
When all transactions are processed, summarize the results and say "You can type exit to finish".
"""

# Create the agents with respective LLM configurations
with finance_llm_config:
    finance_bot = ConversableAgent(
        name="finance_bot",
        system_message=finance_system_message,
        functions=[check_duplicate_payment],
    )

# Create the human agent for oversight
human = ConversableAgent(
    name="human",
    human_input_mode="ALWAYS",  # Always ask for human input
)

# Define structured output for audit logs
class TransactionAuditEntry(BaseModel):
    vendor: str
    amount: float
    memo: str
    status: str  # "approved" or "rejected"
    reason: str

class AuditLogSummary(BaseModel):
    total_transactions: int
    approved_count: int
    rejected_count: int
    transactions: List[TransactionAuditEntry]
    summary_generated_time: str

# Configure the LLM for summary bot with structured output
summary_llm_config = LLMConfig(
    api_type="openai",
    model="gpt-4o-mini",
    api_key=os.environ.get("OPENAI_API_KEY"),
    temperature=0.2,
    response_format=AuditLogSummary,  # Using the Pydantic model for structured output
)

# Define the system message for the summary agent
# Note: No formatting instructions needed!
summary_system_message = """
You are a financial summary assistant that generates audit logs.
Analyze the transaction details and their approval status from the conversation.
Include each transaction with its vendor, amount, memo, status and reason.
"""

with summary_llm_config:
    summary_bot = ConversableAgent(
        name="summary_bot",
        system_message=summary_system_message,
    )

def is_termination_msg(x: dict[str, Any]) -> bool:
    content = x.get("content", "")
    try:
        # Try to parse the content as JSON (structured output)
        data = json.loads(content)
        return isinstance(data, dict) and "summary_generated_time" in data
    except:
        # If not JSON, check for text marker
        return (content is not None) and "==== SUMMARY GENERATED ====" in content

# Generate new transactions including a duplicate
transactions = [
    "Transaction: $500 to Staples. Memo: Quarterly supplies.",  # Duplicate
    "Transaction: $4000 to Unicorn LLC. Memo: Reimbursement.",
    "Transaction: $12000 to Globex. Memo: Confidential.",  # Duplicate
    "Transaction: $9999 to Initech. Memo: Urgent request."
]

# Format the initial message
initial_prompt = (
    "Please process the following transactions one at a time, checking for duplicates:\n\n" +
    "\n".join([f"{i+1}. {tx}" for i, tx in enumerate(transactions)])
)

result, _, _ = initiate_swarm_chat(
    initial_agent=finance_bot,
    agents=[finance_bot, summary_bot],
    user_agent=human,
    messages=initial_prompt,
    swarm_manager_args={
        "llm_config": finance_llm_config,
        "is_termination_msg": is_termination_msg,
    },
    after_work=AfterWorkOption.SWARM_MANAGER
)

# Pretty-print the transaction_data
final_message = result.chat_history[-1]["content"]
structured_data = json.loads(final_message)
print(json.dumps(structured_data, indent=4))

Example Output#

When you run this code, you'll receive a perfectly structured JSON output (shown here with pretty-printing):

_User (to chat_manager):

Please process the following transactions one at a time, checking for duplicates:

1. Transaction: $500 to Staples. Memo: Quarterly supplies.
2. Transaction: $4000 to Unicorn LLC. Memo: Reimbursement.
3. Transaction: $12000 to Globex. Memo: Confidential.
4. Transaction: $9999 to Initech. Memo: Urgent request.

--------------------------------------------------------------------------------

Next speaker: finance_bot

>>>>>>>> USING AUTO REPLY...
finance_bot (to chat_manager):

***** Suggested tool call (call_DMvIKNT8OKHYMtIp7ZcsJe8F): check_duplicate_payment *****
Arguments:
{"vendor": "Staples", "amount": 500, "memo": "Quarterly supplies"}
****************************************************************************************
***** Suggested tool call (call_cfuSo9etd4F279Vg6BnZKuU1): check_duplicate_payment *****
Arguments:
{"vendor": "Unicorn LLC", "amount": 4000, "memo": "Reimbursement"}
****************************************************************************************
***** Suggested tool call (call_hnK4LLirWc5xPh4YYjBEMVqS): check_duplicate_payment *****
Arguments:
{"vendor": "Globex", "amount": 12000, "memo": "Confidential"}
****************************************************************************************
***** Suggested tool call (call_6mJtVN8AMTHwgXoct9LLzo4A): check_duplicate_payment *****
Arguments:
{"vendor": "Initech", "amount": 9999, "memo": "Urgent request"}
****************************************************************************************

--------------------------------------------------------------------------------

Next speaker: _Swarm_Tool_Executor

>>>>>>>> EXECUTING FUNCTION check_duplicate_payment...
Call ID: call_DMvIKNT8OKHYMtIp7ZcsJe8F
Input arguments: {'vendor': 'Staples', 'amount': 500, 'memo': 'Quarterly supplies'}

>>>>>>>> EXECUTING FUNCTION check_duplicate_payment...
Call ID: call_cfuSo9etd4F279Vg6BnZKuU1
Input arguments: {'vendor': 'Unicorn LLC', 'amount': 4000, 'memo': 'Reimbursement'}

>>>>>>>> EXECUTING FUNCTION check_duplicate_payment...
Call ID: call_hnK4LLirWc5xPh4YYjBEMVqS
Input arguments: {'vendor': 'Globex', 'amount': 12000, 'memo': 'Confidential'}

>>>>>>>> EXECUTING FUNCTION check_duplicate_payment...
Call ID: call_6mJtVN8AMTHwgXoct9LLzo4A
Input arguments: {'vendor': 'Initech', 'amount': 9999, 'memo': 'Urgent request'}
_Swarm_Tool_Executor (to chat_manager):

***** Response from calling tool (call_DMvIKNT8OKHYMtIp7ZcsJe8F) *****
{'is_duplicate': True, 'reason': 'Duplicate payment to Staples for $500.0 on 2025-04-15'}
**********************************************************************

--------------------------------------------------------------------------------
***** Response from calling tool (call_cfuSo9etd4F279Vg6BnZKuU1) *****
{'is_duplicate': False, 'reason': 'No recent duplicates found'}
**********************************************************************

--------------------------------------------------------------------------------
***** Response from calling tool (call_hnK4LLirWc5xPh4YYjBEMVqS) *****
{'is_duplicate': True, 'reason': 'Duplicate payment to Globex for $12000.0 on 2025-04-17'}
**********************************************************************

--------------------------------------------------------------------------------
***** Response from calling tool (call_6mJtVN8AMTHwgXoct9LLzo4A) *****
{'is_duplicate': False, 'reason': 'No recent duplicates found'}
**********************************************************************

--------------------------------------------------------------------------------

Next speaker: finance_bot

>>>>>>>> USING AUTO REPLY...
finance_bot (to chat_manager):

Let's process the transactions one by one:

1. **Transaction: $500 to Staples. Memo: Quarterly supplies.**
   - **Duplicate Check Result:** This transaction is a duplicate of a recent payment to Staples for $500. Therefore, I will reject this transaction.

2. **Transaction: $4000 to Unicorn LLC. Memo: Reimbursement.**
   - **Duplicate Check Result:** No recent duplicates found.
   - **Evaluation:** The amount is below $10,000, and the vendor is not unusual. Therefore, I will approve this transaction automatically.

3. **Transaction: $12000 to Globex. Memo: Confidential.**
   - **Duplicate Check Result:** This transaction is a duplicate of a recent payment to Globex for $12,000. Therefore, I will reject this transaction.

4. **Transaction: $9999 to Initech. Memo: Urgent request.**
   - **Duplicate Check Result:** No recent duplicates found.
   - **Evaluation:** The amount is below $10,000, but the memo is vague. I will ask for human agent approval for this transaction.

### Summary of Results:
- **Rejected Transactions:**
  - $500 to Staples (duplicate)
  - $12,000 to Globex (duplicate)

- **Approved Transactions:**
  - $4,000 to Unicorn LLC (approved automatically)

- **Pending Approval:**
  - $9,999 to Initech (requires human agent approval due to vague memo)

You can type exit to finish.

--------------------------------------------------------------------------------

Next speaker: human

Replying as human. Provide feedback to chat_manager. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: approved
human (to chat_manager):

approved

--------------------------------------------------------------------------------

Next speaker: summary_bot

>>>>>>>> USING AUTO REPLY...
summary_bot (to chat_manager):

{"total_transactions":4,"approved_count":2,"rejected_count":2,"transactions":[{"vendor":"Staples","amount":500,"memo":"Quarterly supplies","status":"Rejected","reason":"Duplicate payment to Staples for $500.0 on 2025-04-15"},{"vendor":"Unicorn LLC","amount":4000,"memo":"Reimbursement","status":"Approved","reason":"No recent duplicates found"},{"vendor":"Globex","amount":12000,"memo":"Confidential","status":"Rejected","reason":"Duplicate payment to Globex for $12000.0 on 2025-04-17"},{"vendor":"Initech","amount":9999,"memo":"Urgent request","status":"Approved","reason":"Approved by human agent"}],"summary_generated_time":"2023-10-05T12:00:00Z"}

--------------------------------------------------------------------------------

>>>>>>>> TERMINATING RUN (cd2eca7c-d739-4373-b1be-8a60a2f0159b): Termination message condition on the GroupChatManager 'chat_manager' met
{
    "total_transactions": 4,
    "approved_count": 2,
    "rejected_count": 2,
    "transactions": [
        {
            "vendor": "Staples",
            "amount": 500,
            "memo": "Quarterly supplies",
            "status": "Rejected",
            "reason": "Duplicate payment to Staples for $500.0 on 2025-04-15"
        },
        {
            "vendor": "Unicorn LLC",
            "amount": 4000,
            "memo": "Reimbursement",
            "status": "Approved",
            "reason": "No recent duplicates found"
        },
        {
            "vendor": "Globex",
            "amount": 12000,
            "memo": "Confidential",
            "status": "Rejected",
            "reason": "Duplicate payment to Globex for $12000.0 on 2025-04-17"
        },
        {
            "vendor": "Initech",
            "amount": 9999,
            "memo": "Urgent request",
            "status": "Approved",
            "reason": "Approved by human agent"
        }
    ],
    "summary_generated_time": "2023-10-05T12:00:00Z"
}

Key Benefits of Structured Outputs#

Structured outputs offer several compelling benefits:

  • Consistency: Every response has the same structure, making integration reliable
  • Validation: Built-in type checking ensures data integrity
  • Simplicity: Reduced need for complex formatting instructions in system messages
  • Integration: Easy to parse and use in downstream systems
  • Maintainability: Schema changes can be managed centrally

Next Steps#

You've now learned how to create a complete financial compliance system incorporating the basic concepts of AG2:

  • LLM Configuration: Powering our agents with the right language models
  • ConversableAgent: Creating smart, interactive agents for different tasks
  • Human in the Loop: Adding human oversight for suspicious transactions
  • Agent Orchestration: Coordinating multiple specialized agents with the swarm pattern
  • Tools: Extending agent capabilities with duplicate payment detection
  • Structured Outputs: Ensuring consistent, validated summary reports

In the Advanced Concepts section, you'll learn how to further enhance your AG2 applications with more sophisticated patterns, agent capabilities, and integration options. You'll also find more in-depth explanations of the concepts we've covered here, allowing you to build even more powerful multi-agent systems.