Image Generation#

Some providers can produce images as part of an agent's reply. Generated images are always returned the same way — as a list of BinaryResult objects on reply.files — regardless of which provider produced them.

AG2 exposes two different mechanisms, because the providers expose two different APIs:

Reading generated images#

Every generated image is a BinaryResult on reply.files. It carries the raw bytes and a metadata dict; the image's media type is stored under the media_type key.

1
2
3
4
5
6
7

reply = await agent.ask("Generate an image of a red bicycle on a beach.")

for index, image in enumerate(reply.files):
    media_type = image.metadata.get("media_type", "image/png")
    extension = media_type.split("/")[-1]
    with open(f"image_{index}.{extension}", "wb") as file:
        file.write(image.data)

reply.files is empty when the model returns only text, so it is safe to iterate even when no image was produced.

OpenAI#

Add ImageGenerationTool to an agent configured with the Responses API (OpenAIResponsesConfig). The model decides when to call the tool and the generated image is appended to reply.files.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

from ag2 import Agent
from ag2.config import OpenAIResponsesConfig
from ag2.tools import ImageGenerationTool

agent = Agent(
    "designer",
    config=OpenAIResponsesConfig(model="gpt-4.1"),
    tools=[
        ImageGenerationTool(
            quality="high",
            size="1024x1024",
            output_format="png",
            background="transparent",
        ),
    ],
)

reply = await agent.ask("Generate a logo for a coffee shop.")
image = reply.files[0]

Mistral#

Add ImageGenerationTool to an agent configured with MistralConfig. Unlike OpenAI, this works on the ordinary chat-completions endpoint — no separate API.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

from ag2 import Agent
from ag2.config import MistralConfig
from ag2.tools import ImageGenerationTool

agent = Agent(
    "designer",
    config=MistralConfig(model="mistral-medium-latest"),
    tools=[ImageGenerationTool()],
)

reply = await agent.ask("Generate a logo for a coffee shop.")
image = reply.files[0]

Mistral's tool takes no configuration, so quality, size, background, and output_format are accepted on the AG2 tool but ignored. The generated image is returned to AG2 as a short-lived signed URL rather than inline bytes; AG2 downloads it so reply.files stays consistent with the other providers. The originating URL is kept on the image's metadata under the url key.

Mistral's other server-side tools (web_search, code_interpreter, document_library) belong to its Agents API and raise UnsupportedToolError on chat completions.

Gemini#

Gemini does not use a tool for image generation. Instead, you select an image-capable model and request the IMAGE response modality via response_modalities. The model returns the image inline and AG2 surfaces it on reply.files.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

from ag2 import Agent
from ag2.config import GeminiConfig

config = GeminiConfig(
    model="gemini-3.1-flash-image",
    response_modalities=["TEXT", "IMAGE"],
)

agent = Agent("designer", config=config)

reply = await agent.ask("Generate an image of a friendly robot waving hello.")
image = reply.files[0]

response_modalities is also available on VertexAIConfig for Gemini models served through Vertex AI.

Controlling size and aspect ratio#

Gemini does not take a pixel size string like OpenAI. Instead, pass a types.ImageConfig through image_config to set the aspect ratio and a resolution tier:

1
2
3
4
5
6
7
8
9

from google.genai import types

from ag2.config import GeminiConfig

config = GeminiConfig(
    model="gemini-3.1-flash-image",
    response_modalities=["TEXT", "IMAGE"],
    image_config=types.ImageConfig(aspect_ratio="16:9", image_size="2K"),
)

image_config is a full passthrough of the SDK's types.ImageConfig, so any other field it supports (such as person_generation) is available too. It is also accepted on VertexAIConfig.

Editing an existing image#

To edit an image instead of generating one from scratch, pass it in as an ImageInput alongside your instruction. The edited image is returned on reply.files, exactly like a freshly generated one — so the same image can be sent back in for further rounds of editing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

from ag2 import Agent
from ag2.config import OpenAIResponsesConfig
from ag2.events import ImageInput
from ag2.tools import ImageGenerationTool

agent = Agent(
    "editor",
    config=OpenAIResponsesConfig(model="gpt-4.1"),
    tools=[ImageGenerationTool(size="1024x1024", output_format="png")],
)

reply = await agent.ask(
    "Put a party hat on the robot. Keep everything else the same.",
    ImageInput(path="robot.png"),
)
edited = reply.files[0]

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

from ag2 import Agent
from ag2.config import GeminiConfig
from ag2.events import ImageInput

config = GeminiConfig(model="gemini-3.1-flash-image", response_modalities=["TEXT", "IMAGE"])
agent = Agent("editor", config=config)

reply = await agent.ask(
    "Put a party hat on the robot. Keep everything else the same.",
    ImageInput(path="robot.png"),
)
edited = reply.files[0]

ImageInput also accepts raw bytes — ImageInput(data=image.data, media_type="image/png") — which lets you feed a generated image straight back in for another edit without writing it to disk.