Notebooks
RAG OpenAI Assistants in AutoGen
Use Cases
- Use cases
- Reference Agents
- Notebooks
- All Notebooks
- Solving Multiple Tasks in a Sequence of Chats
- Solving Multiple Tasks in a Sequence of Async Chats
- Using RetrieveChat for Retrieve Augmented Code Generation and Question Answering
- Interactive LLM Agent Dealing with Data Stream
- Agent Chat with Async Human Inputs
- Using RetrieveChat Powered by MongoDB Atlas for Retrieve Augmented Code Generation and Question Answering
- Auto Generated Agent Chat: Task Solving with Langchain Provided Tools as Functions
- Task Solving with Code Generation, Execution and Debugging
- Runtime Logging with AutoGen
- Agent Chat with Multimodal Models: DALLE and GPT-4V
- Using RetrieveChat with Qdrant for Retrieve Augmented Code Generation and Question Answering
- Enhanced Swarm Orchestration with AG2
- Translating Video audio using Whisper and GPT-3.5-turbo
- Agent Tracking with AgentOps
- Group Chat with Coder and Visualization Critic
- ReasoningAgent - Advanced LLM Reasoning with Multiple Search Strategies
- DeepResearchAgent
- Tools with Dependency Injection
- A Uniform interface to call different LLMs
- SQL Agent for Spider text-to-SQL benchmark
- DeepSeek: Adding Browsing Capabilities to AG2
- CaptainAgent
- Adding Browsing Capabilities to AG2
- Auto Generated Agent Chat: Teaching AI New Skills via Natural Language Interaction
- Writing a software application using function calls
- Auto Generated Agent Chat: Function Inception
- Agentic RAG workflow on tabular data from a PDF file
- RealtimeAgent in a Swarm Orchestration
- Using FalkorGraphRagCapability with agents for GraphRAG Question & Answering
- Making OpenAI Assistants Teachable
- Cross-Framework LLM Tool Integration with AG2
- RealtimeAgent with local websocket connection
- Using RetrieveChat Powered by PGVector for Retrieve Augmented Code Generation and Question Answering
- OpenAI Assistants in AutoGen
- Solving Complex Tasks with Nested Chats
- Auto Generated Agent Chat: GPTAssistant with Code Interpreter
- Group Chat with Customized Speaker Selection Method
- Structured output
- Agent Chat with custom model loading
- AutoBuild
- FSM - User can input speaker transition constraints
- Using Neo4j's graph database with AG2 agents for Question & Answering
- Use AutoGen in Databricks with DBRX
- Language Agent Tree Search
- Using a local Telemetry server to monitor a GraphRAG agent
- SocietyOfMindAgent
- Perform Research with Multi-Agent Group Chat
- Group Chat
- StateFlow: Build Workflows through State-Oriented Actions
- Agent Chat with Multimodal Models: LLaVA
- Trip planning with a FalkorDB GraphRAG agent using a Swarm
- Auto Generated Agent Chat: Collaborative Task Solving with Multiple Agents and Human Users
- Agent with memory using Mem0
- Preprocessing Chat History with `TransformMessages`
- AgentOptimizer: An Agentic Way to Train Your LLM Agent
- Using RetrieveChat Powered by Couchbase Capella for Retrieve Augmented Code Generation and Question Answering
- Agent Observability with OpenLIT
- Chatting with a teachable agent
- RealtimeAgent in a Swarm Orchestration
- Conversational Chess using non-OpenAI clients
- Solving Complex Tasks with A Sequence of Nested Chats
- Config loader utility functions
- Currency Calculator: Task Solving with Provided Tools as Functions
- Chat with OpenAI Assistant using function call in AutoGen: OSS Insights for Advanced GitHub Data Analysis
- Web Scraping using Apify Tools
- Agent with memory using Mem0
- Auto Generated Agent Chat: Collaborative Task Solving with Coding and Planning Agent
- Demonstrating the `AgentEval` framework using the task of solving math problems as an example
- RealtimeAgent with WebRTC connection
- Automatically Build Multi-agent System from Agent Library
- Groupchat with Llamaindex agents
- Using Guidance with AutoGen
- RealtimeAgent in a Swarm Orchestration using WebRTC
- Cross-Framework LLM Tool for CaptainAgent
- RealtimeAgent in a Swarm Orchestration
- Websockets: Streaming input and output using websockets
- Auto Generated Agent Chat: Solving Tasks Requiring Web Info
- Auto Generated Agent Chat: Task Solving with Code Generation, Execution, Debugging & Human Feedback
- Assistants with Azure Cognitive Search and Azure Identity
- (Legacy) Implement Swarm-style orchestration with GroupChat
- Auto Generated Agent Chat: Task Solving with Provided Tools as Functions
- Agentchat MathChat
- Small, Local Model (IBM Granite) Multi-Agent RAG
- Group Chat with Retrieval Augmented Generation
- Run a standalone AssistantAgent
- Nested Chats for Tool Use in Conversational Chess
- Swarm Orchestration with AG2
- OptiGuide with Nested Chats in AutoGen
- Usage tracking with AutoGen
- From Dad Jokes To Sad Jokes: Function Calling with GPTAssistantAgent
- Discord, Slack, and Telegram messaging tools
- RAG OpenAI Assistants in AutoGen
- Auto Generated Agent Chat: Group Chat with GPTAssistantAgent
- Task Solving with Provided Tools as Functions (Asynchronous Function Calls)
- Generate Dalle Images With Conversable Agents
- Chat Context Dependency Injection
- Solving Multiple Tasks in a Sequence of Chats with Different Conversable Agent Pairs
- Engaging with Multimodal Models: GPT-4V in AutoGen
- Mitigating Prompt hacking with JSON Mode in Autogen
- Using Neo4j's native GraphRAG SDK with AG2 agents for Question & Answering
- Supercharging Web Crawling with Crawl4AI
- WebSurferAgent
- RealtimeAgent with gemini client
- Community Gallery
Notebooks
RAG OpenAI Assistants in AutoGen
OpenAI Assistant with retrieval augmentation.
This notebook shows an example of the
GPTAssistantAgent
with retrieval augmented generation. GPTAssistantAgent
is an
experimental AutoGen agent class that leverages the OpenAI Assistant
API for
conversational capabilities, working with UserProxyAgent
in AutoGen.
import logging
import os
from autogen import UserProxyAgent, config_list_from_json
from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
logger = logging.getLogger(__name__)
logger.setLevel(logging.WARNING)
assistant_id = os.environ.get("ASSISTANT_ID", None)
config_list = config_list_from_json("OAI_CONFIG_LIST")
llm_config = {
"config_list": config_list,
}
assistant_config = {
"assistant_id": assistant_id,
"tools": [{"type": "retrieval"}],
"file_ids": ["file-AcnBk5PCwAjJMCVO0zLSbzKP"],
# add id of an existing file in your openai account
# in this case I added the implementation of conversable_agent.py
}
gpt_assistant = GPTAssistantAgent(
name="assistant",
instructions="You are adapt at question answering",
llm_config=llm_config,
assistant_config=assistant_config,
)
user_proxy = UserProxyAgent(
name="user_proxy",
code_execution_config=False,
is_termination_msg=lambda msg: "TERMINATE" in msg["content"],
human_input_mode="ALWAYS",
)
user_proxy.initiate_chat(gpt_assistant, message="Please explain the code in this file!")
gpt_assistant.delete_assistant()
OpenAI client config of GPTAssistantAgent(assistant) - model: gpt-4-turbo-preview
GPT Assistant only supports one OpenAI client. Using the first client in the list.
Matching assistant found, using the first matching assistant: {'id': 'asst_sKUCUXkaXyTidtlyovbqppH3', 'created_at': 1710320924, 'description': None, 'file_ids': ['file-AcnBk5PCwAjJMCVO0zLSbzKP'], 'instructions': 'You are adapt at question answering', 'metadata': {}, 'model': 'gpt-4-turbo-preview', 'name': 'assistant', 'object': 'assistant', 'tools': [ToolRetrieval(type='retrieval')]}
user_proxy (to assistant):
Please explain the code in this file!
--------------------------------------------------------------------------------
assistant (to user_proxy):
The code in the file appears to define tests for various functionalities related to a GPT-based assistant agent. Here is a summary of the main components and functionalities described in the visible portion of the code:
1. **Imports and Setup**: The script imports necessary libraries and modules, such as `pytest` for testing, `uuid` for generating unique identifiers, and `openai` along with custom modules like `autogen` and `OpenAIWrapper`. It sets up the system path to include specific directories for importing test configurations and dependencies.
2. **Conditional Test Skipping**: The script includes logic to conditionally skip tests based on the execution environment or missing dependencies. This is done through the `@pytest.mark.skipif` decorator, which conditionally skips test functions based on the `skip` flag, determined by whether certain imports are successful or other conditions are met.
3. **Test Configurations**: The code loads configurations for interacting with the OpenAI API and a hypothetical Azure API (as indicated by the placeholder `azure`), filtering these configurations by certain criteria such as API type and version.
4. **Test Cases**:
- **Configuration List Test (`test_config_list`)**: Ensures that the configurations for both OpenAI and the hypothetical Azure API are loaded correctly by asserting the presence of at least one configuration for each.
- **GPT Assistant Chat Test (`test_gpt_assistant_chat`)**: Tests the functionality of a GPT Assistant Agent by simulating a chat interaction. It uses a mock function to simulate an external API call and checks if the GPT Assistant properly processes the chat input, invokes the external function, and provides an expected response.
- **Assistant Instructions Test (`test_get_assistant_instructions` and related functions)**: These tests verify that instructions can be set and retrieved correctly for a GPTAssistantAgent. It covers creating an agent, setting instructions, and ensuring the set instructions can be retrieved as expected.
- **Instructions Overwrite Test (`test_gpt_assistant_instructions_overwrite`)**: Examines whether the instructions for a GPTAssistantAgent can be successfully overwritten by creating a new agent with the same ID but different instructions, with an explicit indication to overwrite the previous instructions.
Each test case aims to cover different aspects of the GPTAssistantAgent's functionality, such as configuration loading, interactive chat behavior, function registration and invocation, and instruction management. The mocks and assertions within the tests are designed to ensure that each component of the GPTAssistantAgent behaves as expected under controlled conditions.
--------------------------------------------------------------------------------
user_proxy (to assistant):
TERMINATE
--------------------------------------------------------------------------------
Permanently deleting assistant...