In CaptainAgent, tools are in the form of python functions. The agents can write code to import functions and call them according to their needs. This can significantly enhance the functionality and capabilities of the agents.

We provide a list of tools that comes with the release of CaptainAgent. Its full content can be found here

​

Using the Built-in Tool Library

​

Install dependencies

First install the requirements for running the tools via pip.

pip install -r https://raw.githubusercontent.com/ag2ai/ag2/refs/heads/main/autogen/agentchat/contrib/captainagent/tools/requirements.txt

​

Subscribe to Certain APIs

To use the provided built-in tools, it is required to obtain a Bing Search API key and RapidAPI key. For Bing API, you can read more about how to get an API on the Bing Web Search API page. For RapidAPI, you can sign up and subscribe to these two links(link1, link2). These two apis have free billing options and there is no need to worry about extra costs for small scale runs.

Whenever you run the tool-related code, remember to export the api keys to system variables.

export BING_API_KEY=""
export RAPID_API_KEY=""

or

import os
os.environ["BING_API_KEY"] = ""
os.environ["RAPID_API_KEY"] = ""

Then you are good to go. Feel free to try out the examples provided in the CaptainAgent notebook.

​

What is in the Tool Library

The tool library consists of three types of tools: math, data_analysis and information_retrieval. Its folder layout is as follows. Each ‘py’ file implements a tool.

tools
├── README.md
├── data_analysis
│   ├── calculate_correlation.py
│   └── ...
├── information_retrieval
│   ├── arxiv_download.py
│   ├── arxiv_search.py
│   └── ...
├── math
│   ├── calculate_circle_area_from_diameter.py
│   └── ...
└── tool_description.tsv

Tools can be imported from tools/{category}/{tool_name}.py with exactly the same tool name. tool_description.tsv contains descriptions of tools for retrieval.

​

How Tool Library works

When an agent’s description is provided, a retriever will retrieve top_k tool candidates from the library based on the semantic similarity between agent description and tool description. The tool description is same as that in tool_description.tsv.

After candidates are retrieved, the agent’s system message will be updated with the tool candidates’ information and how to call the tools. A user proxy with the ability to execute the code will be added to the nested chat. Under the hood, this is achieved by leveraging the User Defined Functions feature. A LocalCommandLineCodeExecutor equipped with all the functions serves as code executor for the user proxy.

​

Building your own Tool Library

Building your own tool library is simple, follow the same directory layout as the one we provided. The python files should be follow the layout tools/{category}/{tool_name}.py. The tool you’d like to add should be imported in the following fashion:

from tools.category.tool_name import tool_name

The tool_description.tsv file should be a tab-separated file with two columns docid and document_content. The document_content should always follow the format "category tool_name tool_description". The category and tool_name should always be one word with no space in between. The document_content is used to calculate semantic similarity for retrieval.

Once your library is ready, specify the path in nested_config of CaptainAgent.

nested_config = {
    ...
    "autobuild_tool_config": {
        "tool_root": "Your tool root here",
        "retriever": "all-mpnet-base-v2", # This is the default embedding model, you can reove this line if you are not intending to change it
    },
    ...
}

By following these steps, you can easily customize the tools library of CaptainAgent and empower your agents with new tools and capabilities.

​

Note on Adding Customized Tools

Due to the implementation of User Defined Functions, when writing your own tool, you need to write your import statement in the function definition. For example, adding an audio transcription tool:

def audio_transcription(audio_file):
    import whisper
    model = whisper.load_model("base")
    result = model.transcribe(audio_file)
    return result["text"]

There is also decorator with_requirements that may become handy for adding dependencies.