Use Cases
- Use cases
- Reference Agents
- Notebooks
- All Notebooks
- Group Chat with Customized Speaker Selection Method
- RAG OpenAI Assistants in AutoGen
- Using RetrieveChat with Qdrant for Retrieve Augmented Code Generation and Question Answering
- Auto Generated Agent Chat: Function Inception
- Task Solving with Provided Tools as Functions (Asynchronous Function Calls)
- Using Guidance with AutoGen
- Solving Complex Tasks with A Sequence of Nested Chats
- Group Chat
- Solving Multiple Tasks in a Sequence of Async Chats
- Auto Generated Agent Chat: Task Solving with Provided Tools as Functions
- Conversational Chess using non-OpenAI clients
- RealtimeAgent with local websocket connection
- Web Scraping using Apify Tools
- DeepSeek: Adding Browsing Capabilities to AG2
- Interactive LLM Agent Dealing with Data Stream
- Generate Dalle Images With Conversable Agents
- Supercharging Web Crawling with Crawl4AI
- RealtimeAgent in a Swarm Orchestration
- Perform Research with Multi-Agent Group Chat
- Agent Tracking with AgentOps
- Translating Video audio using Whisper and GPT-3.5-turbo
- Automatically Build Multi-agent System from Agent Library
- Auto Generated Agent Chat: Collaborative Task Solving with Multiple Agents and Human Users
- Structured output
- CaptainAgent
- Group Chat with Coder and Visualization Critic
- Cross-Framework LLM Tool Integration with AG2
- Using FalkorGraphRagCapability with agents for GraphRAG Question & Answering
- Demonstrating the `AgentEval` framework using the task of solving math problems as an example
- RealtimeAgent in a Swarm Orchestration using WebRTC
- A Uniform interface to call different LLMs
- From Dad Jokes To Sad Jokes: Function Calling with GPTAssistantAgent
- Solving Complex Tasks with Nested Chats
- Usage tracking with AutoGen
- Agent with memory using Mem0
- Using RetrieveChat Powered by PGVector for Retrieve Augmented Code Generation and Question Answering
- Tools with Dependency Injection
- Solving Multiple Tasks in a Sequence of Chats with Different Conversable Agent Pairs
- WebSurferAgent
- Using RetrieveChat Powered by MongoDB Atlas for Retrieve Augmented Code Generation and Question Answering
- Assistants with Azure Cognitive Search and Azure Identity
- ReasoningAgent - Advanced LLM Reasoning with Multiple Search Strategies
- Agentic RAG workflow on tabular data from a PDF file
- Making OpenAI Assistants Teachable
- Run a standalone AssistantAgent
- AutoBuild
- Solving Multiple Tasks in a Sequence of Chats
- Currency Calculator: Task Solving with Provided Tools as Functions
- Swarm Orchestration with AG2
- Use AutoGen in Databricks with DBRX
- Using a local Telemetry server to monitor a GraphRAG agent
- Auto Generated Agent Chat: Solving Tasks Requiring Web Info
- StateFlow: Build Workflows through State-Oriented Actions
- Groupchat with Llamaindex agents
- Using Neo4j's native GraphRAG SDK with AG2 agents for Question & Answering
- Agent Chat with Multimodal Models: LLaVA
- Group Chat with Retrieval Augmented Generation
- Runtime Logging with AutoGen
- SocietyOfMindAgent
- Agent Chat with Multimodal Models: DALLE and GPT-4V
- Agent Observability with OpenLIT
- Mitigating Prompt hacking with JSON Mode in Autogen
- Trip planning with a FalkorDB GraphRAG agent using a Swarm
- Language Agent Tree Search
- Auto Generated Agent Chat: Collaborative Task Solving with Coding and Planning Agent
- OptiGuide with Nested Chats in AutoGen
- Auto Generated Agent Chat: Task Solving with Langchain Provided Tools as Functions
- Writing a software application using function calls
- Auto Generated Agent Chat: GPTAssistant with Code Interpreter
- Adding Browsing Capabilities to AG2
- Agentchat MathChat
- Chatting with a teachable agent
- RealtimeAgent with gemini client
- Preprocessing Chat History with `TransformMessages`
- Chat with OpenAI Assistant using function call in AutoGen: OSS Insights for Advanced GitHub Data Analysis
- Websockets: Streaming input and output using websockets
- Task Solving with Code Generation, Execution and Debugging
- Agent Chat with Async Human Inputs
- Agent Chat with custom model loading
- Chat Context Dependency Injection
- Nested Chats for Tool Use in Conversational Chess
- Auto Generated Agent Chat: Group Chat with GPTAssistantAgent
- Cross-Framework LLM Tool for CaptainAgent
- Auto Generated Agent Chat: Teaching AI New Skills via Natural Language Interaction
- SQL Agent for Spider text-to-SQL benchmark
- Auto Generated Agent Chat: Task Solving with Code Generation, Execution, Debugging & Human Feedback
- OpenAI Assistants in AutoGen
- (Legacy) Implement Swarm-style orchestration with GroupChat
- Enhanced Swarm Orchestration with AG2
- Using RetrieveChat for Retrieve Augmented Code Generation and Question Answering
- Using Neo4j's graph database with AG2 agents for Question & Answering
- AgentOptimizer: An Agentic Way to Train Your LLM Agent
- Engaging with Multimodal Models: GPT-4V in AutoGen
- RealtimeAgent with WebRTC connection
- FSM - User can input speaker transition constraints
- Config loader utility functions
- Community Gallery
WebSurferAgent
Browse the web with agents.
AutoGen provides a proof-of-concept WebSurferAgent that can command a simple text-based browser (similar to Lynx) to search the web, visit pages, navigate within pages, download files, etc. The browsing is stateful, meaning that browsing history, viewport state, and other details are maintained throughout the conversation.
This work was largely inspired by OpenAI’s WebGPT project from December 2021.
Requirements
AutoGen requires Python>=3.9
. To run this notebook example, please
install AutoGen with the optional websurfer
dependencies:
pip install "autogen[websurfer]"
%pip install --quiet "autogen[websurfer]"
Set your API Endpoint
The
config_list_from_json
function loads a list of configurations from an environment variable or
a json file.
It first looks for environment variable “OAI_CONFIG_LIST” which needs to be a valid json string. If that variable is not found, it then looks for a json file named “OAI_CONFIG_LIST”. It filters the configs by models (you can filter by other keys as well).
The WebSurferAgent uses a combination of models. GPT-4 and GPT-3.5-turbo-16k are recommended.
Your json config should look something like the following:
[
{
"model": "gpt-4",
"api_key": "<your OpenAI API key here>"
},
{
"model": "gpt-3.5-turbo-16k",
"api_key": "<your OpenAI API key here>"
}
]
If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose “upload file” icon.
import autogen
llm_config = {
"timeout": 600,
"cache_seed": 44, # change the seed for different trials
"config_list": autogen.config_list_from_json(
"OAI_CONFIG_LIST",
filter_dict={"model": ["gpt-4", "gpt-4-0613", "gpt-4-32k", "gpt-4-32k-0613", "gpt-4-1106-preview"]},
),
"temperature": 0,
}
summarizer_llm_config = {
"timeout": 600,
"cache_seed": 44, # change the seed for different trials
"config_list": autogen.config_list_from_json(
"OAI_CONFIG_LIST",
filter_dict={"model": ["gpt-3.5-turbo-1106", "gpt-3.5-turbo-16k-0613", "gpt-3.5-turbo-16k"]},
),
"temperature": 0,
}
Configure Bing
For WebSurferAgent to be reasonably useful, it needs to be able to search the web – and that means it needs a Bing API key. You can read more about how to get an API on the Bing Web Search API page.
Once you have your key, either set it as the BING_API_KEY
system
environment variable, or simply input your key below.
import os
bing_api_key = os.environ["BING_API_KEY"]
Construct Agents
We now create out WebSurferAgent, and a UserProxyAgent to surf the web.
from autogen.agentchat.contrib.web_surfer import WebSurferAgent
web_surfer = WebSurferAgent(
"web_surfer",
llm_config=llm_config,
summarizer_llm_config=summarizer_llm_config,
browser_config={"viewport_size": 4096, "bing_api_key": bing_api_key},
)
user_proxy = autogen.UserProxyAgent(
"user_proxy",
human_input_mode="NEVER",
code_execution_config=False,
default_auto_reply="",
is_termination_msg=lambda x: True,
)
Navigational search, scroll, answer questions - Search for Microsoft’s wikipedia page, then naviagate to it - Scroll down - Answer questions about the content
task1 = """Find Microsoft's Wikipedia page."""
user_proxy.initiate_chat(web_surfer, message=task1, clear_history=False)
user_proxy (to web_surfer):
Find Microsoft's Wikipedia page.
--------------------------------------------------------------------------------
>>>>>>>> USING AUTO REPLY...
>>>>>>>> EXECUTING FUNCTION navigational_web_search...
web_surfer (to user_proxy):
Address: https://en.wikipedia.org/wiki/Microsoft
Title: Microsoft - Wikipedia
Viewport position: Showing page 1 of 64.
=======================
# Microsoft
American multinational technology corporation
Microsoft Corporation| [A square divided into four sub-squares, colored red-orange, green, yellow and blue (clockwise), with the company name appearing to its right](/wiki/File:Microsoft_logo_(2012).svg) |
| Building 92 on the [Microsoft Redmond campus](/wiki/Microsoft_Redmond_campus "Microsoft Redmond campus") |
| Type | [Public](/wiki/Public_company "Public company") |
| [Traded as](/wiki/Ticker_symbol "Ticker symbol") | * [Nasdaq](/wiki/Nasdaq "Nasdaq"): [MSFT](https://www.nasdaq.com/market-activity/stocks/msft)
* [Nasdaq-100](/wiki/Nasdaq-100 "Nasdaq-100") component
* [DJIA](/wiki/Dow_Jones_Industrial_Average "Dow Jones Industrial Average") component
* [S&P 100](/wiki/S%26P_100 "S&P 100") component
* [S&P 500](/wiki/S%26P_500 "S&P 500") component
|
| [ISIN](/wiki/International_Securities_Identification_Number "International Securities Identification Number") | [US5949181045](https://isin.toolforge.org/?language=en&isin=US5949181045) |
| Industry | [Information technology](/wiki/Information_technology "Information technology") |
| Founded | April 4, 1975; 48 years ago (1975-04-04) in [Albuquerque, New Mexico](/wiki/Albuquerque,_New_Mexico "Albuquerque, New Mexico"), U.S. |
| Founders | * [Bill Gates](/wiki/Bill_Gates "Bill Gates")
* [Paul Allen](/wiki/Paul_Allen "Paul Allen")
|
| Headquarters | [One Microsoft Way](/wiki/Microsoft_campus "Microsoft campus")[Redmond, Washington](/wiki/Redmond,_Washington "Redmond, Washington"), U.S. |
| Area served | Worldwide |
| Key people | * [Satya Nadella](/wiki/Satya_Nadella "Satya Nadella")([Chairman](/wiki/Chairman "Chairman") & [CEO](/wiki/Chief_executive_officer "Chief executive officer"))
* [Brad Smith](/wiki/Brad_Smith_(American_lawyer) "Brad Smith (American lawyer)")([Vice Chairman](/wiki/Vice-Chairman "Vice-Chairman") & [President](/wiki/President_(corporate_title) "President (corporate title)"))
* [Bill Gates](/wiki/Bill_Gates "Bill Gates")([technical adviser](/wiki/Adviser "Adviser"))
|
| Products | * [Software development](/wiki/Software_development "Software development")
* [Computer hardware](/wiki/Computer_hardware "Computer hardware")
* [Consumer electronics](/wiki/Consumer_electronics "Consumer electronics")
* [Social networking service](/wiki/Social_networking_service "Social networking service")
* [Cloud computing](/wiki/Cloud_computing "Cloud computing")
* [Video games](/wiki/Video_game_industry "Video game industry")
* [Internet](/wiki/Internet "Internet")
* [Corporate venture capital](/wiki/Corporate_venture_capital "Corporate venture capital")
|
| Brands |
* [Windows](/wiki/Microsoft_Windows "Microsoft Windows")
* [Microsoft 365](/wiki/Microsoft_365 "Microsoft 365")
* [Skype](/wiki/Skype "Skype")
* [Visual Studio](/wiki/Visual_Studio "Visual Studio")
* [Xbox](/wiki/Xbox "Xbox")
* [Dynamics](/wiki/Microsoft_Dynamics_365 "Microsoft Dynamics 365")
* [Surface](/wiki/Microsoft_Surface "Microsoft Surface")
|
| Services |
* [Edge](/wiki/Microsoft_Edge "Microsoft Edge")
* [Azure](/wiki/Microsoft_Azure "Microsoft Azure")
* [Bing](/wiki/Microsoft_Bing "Microsoft Bing")
* [LinkedIn](/wiki/LinkedIn "LinkedIn")
* [Yammer](/wiki/Yammer "Yammer")
* [Microsoft 365](/wiki/Microsoft_365 "Microsoft 365")
* [OneDrive](/wiki/OneDrive "OneDrive")
* [Outlook](/wiki/Microsoft_Outlook "Microsoft Outlook")
* [GitHub](/wiki/GitHub "GitHub")
* [Microsoft Store](/wiki/Microsoft_Store_(digital) "Microsoft Store (digital)")
* [Windows Update](/wiki/Windows_Update "Windows Update")
* [Xbox Game Pass](/wiki/Xbox_Game_Pass "Xbox Game Pass")
* [Xbox network](/wiki/Xbox_network "Xbox network")
|
| Revenue | Increase [US$](/wiki/United_States_dollar "United States dollar")211.9 billion (2023) |
| [Operating income](/wiki/Earnings_before_interest_and_taxes "Earnings before interest and taxes") | Increase US$88.5 billion (2023) |
| [Net income](/wiki/Net_income "Net income") | Increase US$73.4 billion (2023) |
| [Total assets](/wiki/Asset "Asset") | Increase US$411.9 billion (2023) |
| [Total equity](/wiki/Equity_(finance) "Equity
--------------------------------------------------------------------------------
task2 = """Scroll down."""
user_proxy.initiate_chat(web_surfer, message=task2, clear_history=False)
user_proxy (to web_surfer):
Scroll down.
--------------------------------------------------------------------------------
>>>>>>>> USING AUTO REPLY...
>>>>>>>> EXECUTING FUNCTION page_down...
web_surfer (to user_proxy):
Address: https://en.wikipedia.org/wiki/Microsoft
Title: Microsoft - Wikipedia
Viewport position: Showing page 2 of 64.
=======================
(finance)") | Increase US$206.2 billion (2023) |
| Number of employees | 238,000 (2023) |
| [Divisions](/wiki/Division_(business) "Division (business)") |
* [Microsoft Engineering Groups](/wiki/Microsoft_engineering_groups "Microsoft engineering groups")
* [Microsoft Digital Crimes Unit](/wiki/Microsoft_Digital_Crimes_Unit "Microsoft Digital Crimes Unit")
* [Microsoft Press](/wiki/Microsoft_Press "Microsoft Press")
* [Microsoft Japan](/wiki/Microsoft_Japan "Microsoft Japan")
* [Microsoft Gaming](/wiki/Microsoft_Gaming "Microsoft Gaming")
|
| [Subsidiaries](/wiki/Subsidiary "Subsidiary") |
* [GitHub](/wiki/GitHub "GitHub")
* [LinkedIn](/wiki/LinkedIn "LinkedIn")
* [Metaswitch](/wiki/Metaswitch "Metaswitch")
* [Nuance Communications](/wiki/Nuance_Communications "Nuance Communications")
* [RiskIQ](/wiki/RiskIQ "RiskIQ")
* [Skype Technologies](/wiki/Skype_Technologies "Skype Technologies")
* [OpenAI](/wiki/OpenAI "OpenAI") (49%)[[1]](#cite_note-1)
* [Xamarin](/wiki/Xamarin "Xamarin")
* [Xandr](/wiki/Xandr "Xandr")
|
| |
| [ASN](/wiki/Autonomous_System_Number "Autonomous System Number") | * [8075](https://bgp.tools/as/8075)
|
| |
| Website | [microsoft.com](https://www.microsoft.com/) |
| **Footnotes / references**Financials as of June 30, 2023[[update]](https://en.wikipedia.org/w/index.php?title=Microsoft&action=edit)[[2]](#cite_note-2) |
| | | |
| --- | --- | --- |
|
| | |
| --- | --- |
| [Bill Gates in 2023](/wiki/File:Bill_Gates_2017_(cropped).jpg) | This article is part of a series about
[Bill Gates](/wiki/Bill_Gates "Bill Gates") |
|
| * [Awards and honors](/wiki/Bill_Gates#Recognition "Bill Gates")
* [Philanthropy](/wiki/Bill_Gates#Philanthropy "Bill Gates")
* [Political positions](/wiki/Bill_Gates#Political_positions "Bill Gates")
* [Public image](/wiki/Bill_Gates#Public_image "Bill Gates")
* [Residence](/wiki/Bill_Gates%27s_house "Bill Gates's house")
---
Companies* [Traf-O-Data](/wiki/Traf-O-Data "Traf-O-Data")
* Microsoft ([criticism](/wiki/Criticism_of_Microsoft "Criticism of Microsoft"))
* [BEN](/wiki/Branded_Entertainment_Network "Branded Entertainment Network")
* [Cascade Investment](/wiki/Cascade_Investment "Cascade Investment")
* [TerraPower](/wiki/TerraPower "TerraPower")
* [Gates Ventures](/wiki/Gates_Ventures "Gates Ventures")
---
Charitable organizations* [Bill & Melinda Gates Foundation](/wiki/Bill_%26_Melinda_Gates_Foundation "Bill & Melinda Gates Foundation")
* [Match for Africa](/wiki/Match_for_Africa "Match for Africa")
* [The Giving Pledge](/wiki/The_Giving_Pledge "The Giving Pledge")
* [OER Project](/wiki/OER_Project "OER Project")
* [Breakthrough Energy](/wiki/Breakthrough_Energy "Breakthrough Energy")
* [Mission Innovation](/wiki/Mission_Innovation "Mission Innovation")
---
Writings* "[An Open Letter to Hobbyists](/wiki/An_Open_Letter_to_Hobbyists "An Open Letter to Hobbyists")"
* *[The Road Ahead](/wiki/The_Road_Ahead_(Gates_book) "The Road Ahead (Gates book)")*
* *[Business @ the Speed of Thought](/wiki/Business_@_the_Speed_of_Thought "Business @ the Speed of Thought")*
* *[How to Avoid a Climate Disaster](/wiki/How_to_Avoid_a_Climate_Disaster "How to Avoid a Climate Disaster")*
* *[How to Prevent the Next Pandemic](/wiki/How_to_Prevent_the_Next_Pandemic "How to Prevent the Next Pandemic")*
---
Related* [Bill Gates' flower fly](/wiki/Bill_Gates%27_flower_fly "Bill Gates' flower fly")
* [Codex Leicester](/wiki/Codex_Leicester "Codex Leicester")
* *[Lost on the Grand Banks](/wiki/Lost_on_the_Grand_Banks "Lost on the Grand Banks")*
* [History of Microsoft](/wiki/History_of_Microsoft "History of Microsoft")
* [Timeline of Microsoft](/wiki/Timeline_of_Microsoft "Timeline of Microsoft")
* [Paul Allen](/wiki/Paul_Allen "Paul Allen")
---
|
| * [v](/wiki/Template:Bill_Gates_series "Template:Bill Gates series")
* [t](/wiki/Template_talk:Bill_Gates_series "Template talk:Bill Gates series")
* [e](/wiki/Special:EditPage/Template:Bill_Gates_series "Special:EditPage/Template:Bill Gates series")
|
**Microsoft Corporation** is an American multinational [technology corporation](/wiki/Technology_company
--------------------------------------------------------------------------------
task3 = """Where was the first office location, and when did they move to Redmond?"""
user_proxy.initiate_chat(web_surfer, message=task3, clear_history=False)
user_proxy (to web_surfer):
Where was the first office location, and when did they move to Redmond?
--------------------------------------------------------------------------------
>>>>>>>> USING AUTO REPLY...
>>>>>>>> EXECUTING FUNCTION answer_from_page...
web_surfer (to user_proxy):
Microsoft's first office location was in Albuquerque, New Mexico, where it was founded on April 4, 1975. However, Microsoft later moved its headquarters to Redmond, Washington in January 1979. Since then, Redmond has been the main office location for Microsoft.
--------------------------------------------------------------------------------