WebSurferAgent

AutoGen provides a proof-of-concept WebSurferAgent that can command a simple text-based browser (similar to Lynx) to search the web, visit pages, navigate within pages, download files, etc. The browsing is stateful, meaning that browsing history, viewport state, and other details are maintained throughout the conversation.

This work was largely inspired by OpenAI’s WebGPT project from December 2021.

Requirements

AutoGen requires Python>=3.9. To run this notebook example, please install AutoGen with the optional websurfer dependencies:

pip install "autogen[websurfer]"

%pip install --quiet "autogen[websurfer]"

Set your API Endpoint

The config_list_from_json function loads a list of configurations from an environment variable or a json file.

It first looks for environment variable “OAI_CONFIG_LIST” which needs to be a valid json string. If that variable is not found, it then looks for a json file named “OAI_CONFIG_LIST”. It filters the configs by models (you can filter by other keys as well).

The WebSurferAgent uses a combination of models. GPT-4 and GPT-3.5-turbo-16k are recommended.

Your json config should look something like the following:

[
    {
        "model": "gpt-4",
        "api_key": "<your OpenAI API key here>"
    },
    {
        "model": "gpt-3.5-turbo-16k",
        "api_key": "<your OpenAI API key here>"
    }
]

If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose “upload file” icon.

import autogen  # noqa: E402

llm_config = {
    "timeout": 600,
    "cache_seed": 44,  # change the seed for different trials
    "config_list": autogen.config_list_from_json(
        "OAI_CONFIG_LIST",
        filter_dict={"model": ["gpt-4", "gpt-4-0613", "gpt-4-32k", "gpt-4-32k-0613", "gpt-4-1106-preview"]},
    ),
    "temperature": 0,
}

summarizer_llm_config = {
    "timeout": 600,
    "cache_seed": 44,  # change the seed for different trials
    "config_list": autogen.config_list_from_json(
        "OAI_CONFIG_LIST",
        filter_dict={"model": ["gpt-3.5-turbo-1106", "gpt-3.5-turbo-16k-0613", "gpt-3.5-turbo-16k"]},
    ),
    "temperature": 0,
}

Configure Bing

For WebSurferAgent to be reasonably useful, it needs to be able to search the web – and that means it needs a Bing API key. You can read more about how to get an API on the Bing Web Search API page.

Once you have your key, either set it as the BING_API_KEY system environment variable, or simply input your key below.

import os  # noqa: E402

bing_api_key = os.environ["BING_API_KEY"]

Construct Agents

We now create out WebSurferAgent, and a UserProxyAgent to surf the web.

from autogen.agentchat.contrib.web_surfer import WebSurferAgent  # noqa: E402

web_surfer = WebSurferAgent(
    "web_surfer",
    llm_config=llm_config,
    summarizer_llm_config=summarizer_llm_config,
    browser_config={"viewport_size": 4096, "bing_api_key": bing_api_key},
)

user_proxy = autogen.UserProxyAgent(
    "user_proxy",
    human_input_mode="NEVER",
    code_execution_config=False,
    default_auto_reply="",
    is_termination_msg=lambda x: True,
)

Navigational search, scroll, answer questions - Search for Microsoft’s wikipedia page, then naviagate to it - Scroll down - Answer questions about the content

task1 = """Find Microsoft's Wikipedia page."""
user_proxy.initiate_chat(web_surfer, message=task1, clear_history=False)

user_proxy (to web_surfer):

Find Microsoft's Wikipedia page.

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

>>>>>>>> EXECUTING FUNCTION navigational_web_search...
web_surfer (to user_proxy):

Address: https://en.wikipedia.org/wiki/Microsoft
Title: Microsoft - Wikipedia
Viewport position: Showing page 1 of 64.
=======================
# Microsoft

American multinational technology corporation

Microsoft Corporation| [A square divided into four sub-squares, colored red-orange, green, yellow and blue (clockwise), with the company name appearing to its right](/wiki/File:Microsoft_logo_(2012).svg) |
| Building 92 on the [Microsoft Redmond campus](/wiki/Microsoft_Redmond_campus "Microsoft Redmond campus") |
| Type | [Public](/wiki/Public_company "Public company") |
| [Traded as](/wiki/Ticker_symbol "Ticker symbol") | * [Nasdaq](/wiki/Nasdaq "Nasdaq"): [MSFT](https://www.nasdaq.com/market-activity/stocks/msft)
* [Nasdaq-100](/wiki/Nasdaq-100 "Nasdaq-100") component
* [DJIA](/wiki/Dow_Jones_Industrial_Average "Dow Jones Industrial Average") component
* [S&P 100](/wiki/S%26P_100 "S&P 100") component
* [S&P 500](/wiki/S%26P_500 "S&P 500") component
 |
| [ISIN](/wiki/International_Securities_Identification_Number "International Securities Identification Number") | [US5949181045](https://isin.toolforge.org/?language=en&isin=US5949181045) |
| Industry | [Information technology](/wiki/Information_technology "Information technology") |
| Founded | April 4, 1975; 48 years ago (1975-04-04) in [Albuquerque, New Mexico](/wiki/Albuquerque,_New_Mexico "Albuquerque, New Mexico"), U.S. |
| Founders | * [Bill Gates](/wiki/Bill_Gates "Bill Gates")
* [Paul Allen](/wiki/Paul_Allen "Paul Allen")
 |
| Headquarters | [One Microsoft Way](/wiki/Microsoft_campus "Microsoft campus")[Redmond, Washington](/wiki/Redmond,_Washington "Redmond, Washington"), U.S. |
| Area served | Worldwide |
| Key people | * [Satya Nadella](/wiki/Satya_Nadella "Satya Nadella")([Chairman](/wiki/Chairman "Chairman") & [CEO](/wiki/Chief_executive_officer "Chief executive officer"))
* [Brad Smith](/wiki/Brad_Smith_(American_lawyer) "Brad Smith (American lawyer)")([Vice Chairman](/wiki/Vice-Chairman "Vice-Chairman") & [President](/wiki/President_(corporate_title) "President (corporate title)"))
* [Bill Gates](/wiki/Bill_Gates "Bill Gates")([technical adviser](/wiki/Adviser "Adviser"))
 |
| Products | * [Software development](/wiki/Software_development "Software development")
* [Computer hardware](/wiki/Computer_hardware "Computer hardware")
* [Consumer electronics](/wiki/Consumer_electronics "Consumer electronics")
* [Social networking service](/wiki/Social_networking_service "Social networking service")
* [Cloud computing](/wiki/Cloud_computing "Cloud computing")
* [Video games](/wiki/Video_game_industry "Video game industry")
* [Internet](/wiki/Internet "Internet")
* [Corporate venture capital](/wiki/Corporate_venture_capital "Corporate venture capital")
 |
| Brands | 
* [Windows](/wiki/Microsoft_Windows "Microsoft Windows")
* [Microsoft 365](/wiki/Microsoft_365 "Microsoft 365")
* [Skype](/wiki/Skype "Skype")
* [Visual Studio](/wiki/Visual_Studio "Visual Studio")
* [Xbox](/wiki/Xbox "Xbox")
* [Dynamics](/wiki/Microsoft_Dynamics_365 "Microsoft Dynamics 365")
* [Surface](/wiki/Microsoft_Surface "Microsoft Surface")

 |
| Services | 
* [Edge](/wiki/Microsoft_Edge "Microsoft Edge")
* [Azure](/wiki/Microsoft_Azure "Microsoft Azure")
* [Bing](/wiki/Microsoft_Bing "Microsoft Bing")
* [LinkedIn](/wiki/LinkedIn "LinkedIn")
* [Yammer](/wiki/Yammer "Yammer")
* [Microsoft 365](/wiki/Microsoft_365 "Microsoft 365")
* [OneDrive](/wiki/OneDrive "OneDrive")
* [Outlook](/wiki/Microsoft_Outlook "Microsoft Outlook")
* [GitHub](/wiki/GitHub "GitHub")
* [Microsoft Store](/wiki/Microsoft_Store_(digital) "Microsoft Store (digital)")
* [Windows Update](/wiki/Windows_Update "Windows Update")
* [Xbox Game Pass](/wiki/Xbox_Game_Pass "Xbox Game Pass")
* [Xbox network](/wiki/Xbox_network "Xbox network")

 |
| Revenue | Increase [US$](/wiki/United_States_dollar "United States dollar")211.9 billion (2023) |
| [Operating income](/wiki/Earnings_before_interest_and_taxes "Earnings before interest and taxes") | Increase US$88.5 billion (2023) |
| [Net income](/wiki/Net_income "Net income") | Increase US$73.4 billion (2023) |
| [Total assets](/wiki/Asset "Asset") | Increase US$411.9 billion (2023) |
| [Total equity](/wiki/Equity_(finance) "Equity 

--------------------------------------------------------------------------------

task2 = """Scroll down."""
user_proxy.initiate_chat(web_surfer, message=task2, clear_history=False)

user_proxy (to web_surfer):

Scroll down.

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

>>>>>>>> EXECUTING FUNCTION page_down...
web_surfer (to user_proxy):

Address: https://en.wikipedia.org/wiki/Microsoft
Title: Microsoft - Wikipedia
Viewport position: Showing page 2 of 64.
=======================
(finance)") | Increase US$206.2 billion (2023) |
| Number of employees | 238,000 (2023) |
| [Divisions](/wiki/Division_(business) "Division (business)") | 
* [Microsoft Engineering Groups](/wiki/Microsoft_engineering_groups "Microsoft engineering groups")
* [Microsoft Digital Crimes Unit](/wiki/Microsoft_Digital_Crimes_Unit "Microsoft Digital Crimes Unit")
* [Microsoft Press](/wiki/Microsoft_Press "Microsoft Press")
* [Microsoft Japan](/wiki/Microsoft_Japan "Microsoft Japan")
* [Microsoft Gaming](/wiki/Microsoft_Gaming "Microsoft Gaming")

 |
| [Subsidiaries](/wiki/Subsidiary "Subsidiary") | 
* [GitHub](/wiki/GitHub "GitHub")
* [LinkedIn](/wiki/LinkedIn "LinkedIn")
* [Metaswitch](/wiki/Metaswitch "Metaswitch")
* [Nuance Communications](/wiki/Nuance_Communications "Nuance Communications")
* [RiskIQ](/wiki/RiskIQ "RiskIQ")
* [Skype Technologies](/wiki/Skype_Technologies "Skype Technologies")
* [OpenAI](/wiki/OpenAI "OpenAI") (49%)[[1]](#cite_note-1)
* [Xamarin](/wiki/Xamarin "Xamarin")
* [Xandr](/wiki/Xandr "Xandr")

 |
|  |
| [ASN](/wiki/Autonomous_System_Number "Autonomous System Number") | * [8075](https://bgp.tools/as/8075)
 |
|  |
| Website | [microsoft.com](https://www.microsoft.com/) |
| **Footnotes / references**Financials as of June 30, 2023[[update]](https://en.wikipedia.org/w/index.php?title=Microsoft&action=edit)[[2]](#cite_note-2) |

|  |  |  |
| --- | --- | --- |
| 

|  |  |
| --- | --- |
| [Bill Gates in 2023](/wiki/File:Bill_Gates_2017_(cropped).jpg) | This article is part of a series about
[Bill Gates](/wiki/Bill_Gates "Bill Gates") |

 |
| * [Awards and honors](/wiki/Bill_Gates#Recognition "Bill Gates")
* [Philanthropy](/wiki/Bill_Gates#Philanthropy "Bill Gates")
* [Political positions](/wiki/Bill_Gates#Political_positions "Bill Gates")
* [Public image](/wiki/Bill_Gates#Public_image "Bill Gates")
* [Residence](/wiki/Bill_Gates%27s_house "Bill Gates's house")

---

Companies* [Traf-O-Data](/wiki/Traf-O-Data "Traf-O-Data")
* Microsoft ([criticism](/wiki/Criticism_of_Microsoft "Criticism of Microsoft"))
* [BEN](/wiki/Branded_Entertainment_Network "Branded Entertainment Network")
* [Cascade Investment](/wiki/Cascade_Investment "Cascade Investment")
* [TerraPower](/wiki/TerraPower "TerraPower")
* [Gates Ventures](/wiki/Gates_Ventures "Gates Ventures")

---

Charitable organizations* [Bill & Melinda Gates Foundation](/wiki/Bill_%26_Melinda_Gates_Foundation "Bill & Melinda Gates Foundation")
* [Match for Africa](/wiki/Match_for_Africa "Match for Africa")
* [The Giving Pledge](/wiki/The_Giving_Pledge "The Giving Pledge")
* [OER Project](/wiki/OER_Project "OER Project")
* [Breakthrough Energy](/wiki/Breakthrough_Energy "Breakthrough Energy")
* [Mission Innovation](/wiki/Mission_Innovation "Mission Innovation")

---

Writings* "[An Open Letter to Hobbyists](/wiki/An_Open_Letter_to_Hobbyists "An Open Letter to Hobbyists")"
* *[The Road Ahead](/wiki/The_Road_Ahead_(Gates_book) "The Road Ahead (Gates book)")*
* *[Business @ the Speed of Thought](/wiki/Business_@_the_Speed_of_Thought "Business @ the Speed of Thought")*
* *[How to Avoid a Climate Disaster](/wiki/How_to_Avoid_a_Climate_Disaster "How to Avoid a Climate Disaster")*
* *[How to Prevent the Next Pandemic](/wiki/How_to_Prevent_the_Next_Pandemic "How to Prevent the Next Pandemic")*

---

Related* [Bill Gates' flower fly](/wiki/Bill_Gates%27_flower_fly "Bill Gates' flower fly")
* [Codex Leicester](/wiki/Codex_Leicester "Codex Leicester")
* *[Lost on the Grand Banks](/wiki/Lost_on_the_Grand_Banks "Lost on the Grand Banks")*
* [History of Microsoft](/wiki/History_of_Microsoft "History of Microsoft")
* [Timeline of Microsoft](/wiki/Timeline_of_Microsoft "Timeline of Microsoft")
* [Paul Allen](/wiki/Paul_Allen "Paul Allen")

---

 |
| * [v](/wiki/Template:Bill_Gates_series "Template:Bill Gates series")
* [t](/wiki/Template_talk:Bill_Gates_series "Template talk:Bill Gates series")
* [e](/wiki/Special:EditPage/Template:Bill_Gates_series "Special:EditPage/Template:Bill Gates series")
 |

**Microsoft Corporation** is an American multinational [technology corporation](/wiki/Technology_company 

--------------------------------------------------------------------------------

task3 = """Where was the first office location, and when did they move to Redmond?"""
user_proxy.initiate_chat(web_surfer, message=task3, clear_history=False)

user_proxy (to web_surfer):

Where was the first office location, and when did they move to Redmond?

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

>>>>>>>> EXECUTING FUNCTION answer_from_page...
web_surfer (to user_proxy):

Microsoft's first office location was in Albuquerque, New Mexico, where it was founded on April 4, 1975. However, Microsoft later moved its headquarters to Redmond, Washington in January 1979. Since then, Redmond has been the main office location for Microsoft.

--------------------------------------------------------------------------------

Structured output Swarm Orchestration with AG2

On this page

Requirements
Set your API Endpoint
Configure Bing
Construct Agents

Examples

​Requirements

​Set your API Endpoint

​Configure Bing

​Construct Agents

Requirements

Set your API Endpoint

Configure Bing

Construct Agents