diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index c9a56e88..4f93980a 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -8,6 +8,9 @@ on: branches: - main +env: + UV_FROZEN: "1" + jobs: lint: runs-on: ubuntu-latest diff --git a/Makefile b/Makefile index f6b779e3..accf2d91 100644 --- a/Makefile +++ b/Makefile @@ -37,7 +37,6 @@ snapshots-create: .PHONY: old_version_tests old_version_tests: UV_PROJECT_ENVIRONMENT=.venv_39 uv run --python 3.9 -m pytest - UV_PROJECT_ENVIRONMENT=.venv_39 uv run --python 3.9 -m mypy . .PHONY: build-docs build-docs: diff --git a/docs/agents.md b/docs/agents.md index 5d726561..49c19132 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -142,4 +142,6 @@ Supplying a list of tools doesn't always mean the LLM will use a tool. You can f !!! note - If requiring tool use, you should consider setting [`Agent.tool_use_behavior`] to stop the Agent from running when a tool output is produced. Otherwise, the Agent might run in an infinite loop, where the LLM produces a tool call , and the tool result is sent to the LLM, and this infinite loops because the LLM is always forced to use a tool. + To prevent infinite loops, the framework automatically resets `tool_choice` to "auto" after a tool call. This behavior is configurable via [`agent.reset_tool_choice`][agents.agent.Agent.reset_tool_choice]. The infinite loop is because tool results are sent to the LLM, which then generates another tool call because of `tool_choice`, ad infinitum. + + If you want the Agent to completely stop after a tool call (rather than continuing with auto mode), you can set [`Agent.tool_use_behavior="stop_on_first_tool"`] which will directly use the tool output as the final response without further LLM processing. diff --git a/docs/assets/images/graph.png b/docs/assets/images/graph.png new file mode 100644 index 00000000..13e2d6eb Binary files /dev/null and b/docs/assets/images/graph.png differ diff --git a/docs/assets/images/mcp-tracing.jpg b/docs/assets/images/mcp-tracing.jpg new file mode 100644 index 00000000..cefeb66b Binary files /dev/null and b/docs/assets/images/mcp-tracing.jpg differ diff --git a/docs/mcp.md b/docs/mcp.md new file mode 100644 index 00000000..e279a25e --- /dev/null +++ b/docs/mcp.md @@ -0,0 +1,60 @@ +# Model context protocol (MCP) + +The [Model context protocol](https://modelcontextprotocol.io/introduction) (aka MCP) is a way to provide tools and context to the LLM. From the MCP docs: + +> MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools. + +The Agents SDK has support for MCP. This enables you to use a wide range of MCP servers to provide tools to your Agents. + +## MCP servers + +Currently, the MCP spec defines two kinds of servers, based on the transport mechanism they use: + +1. **stdio** servers run as a subprocess of your application. You can think of them as running "locally". +2. **HTTP over SSE** servers run remotely. You connect to them via a URL. + +You can use the [`MCPServerStdio`][agents.mcp.server.MCPServerStdio] and [`MCPServerSse`][agents.mcp.server.MCPServerSse] classes to connect to these servers. + +For example, this is how you'd use the [official MCP filesystem server](https://www.npmjs.com/package/@modelcontextprotocol/server-filesystem). + +```python +async with MCPServerStdio( + params={ + "command": "npx", + "args": ["-y", "@modelcontextprotocol/server-filesystem", samples_dir], + } +) as server: + tools = await server.list_tools() +``` + +## Using MCP servers + +MCP servers can be added to Agents. The Agents SDK will call `list_tools()` on the MCP servers each time the Agent is run. This makes the LLM aware of the MCP server's tools. When the LLM calls a tool from an MCP server, the SDK calls `call_tool()` on that server. + +```python + +agent=Agent( + name="Assistant", + instructions="Use the tools to achieve the task", + mcp_servers=[mcp_server_1, mcp_server_2] +) +``` + +## Caching + +Every time an Agent runs, it calls `list_tools()` on the MCP server. This can be a latency hit, especially if the server is a remote server. To automatically cache the list of tools, you can pass `cache_tools_list=True` to both [`MCPServerStdio`][agents.mcp.server.MCPServerStdio] and [`MCPServerSse`][agents.mcp.server.MCPServerSse]. You should only do this if you're certain the tool list will not change. + +If you want to invalidate the cache, you can call `invalidate_tools_cache()` on the servers. + +## End-to-end examples + +View complete working examples at [examples/mcp](https://github.com/openai/openai-agents-python/tree/main/examples/mcp). + +## Tracing + +[Tracing](./tracing.md) automatically captures MCP operations, including: + +1. Calls to the MCP server to list tools +2. MCP-related info on function calls + +![MCP Tracing Screenshot](./assets/images/mcp-tracing.jpg) diff --git a/docs/ref/mcp/server.md b/docs/ref/mcp/server.md new file mode 100644 index 00000000..e58efab2 --- /dev/null +++ b/docs/ref/mcp/server.md @@ -0,0 +1,3 @@ +# `MCP Servers` + +::: agents.mcp.server diff --git a/docs/ref/mcp/util.md b/docs/ref/mcp/util.md new file mode 100644 index 00000000..b3f7db25 --- /dev/null +++ b/docs/ref/mcp/util.md @@ -0,0 +1,3 @@ +# `MCP Util` + +::: agents.mcp.util diff --git a/docs/tracing.md b/docs/tracing.md index 1c8c58b6..2cefd03c 100644 --- a/docs/tracing.md +++ b/docs/tracing.md @@ -101,7 +101,8 @@ To customize this default setup, to send traces to alternative or additional bac - [Weights & Biases](https://weave-docs.wandb.ai/guides/integrations/openai_agents) - [Arize-Phoenix](https://docs.arize.com/phoenix/tracing/integrations-tracing/openai-agents-sdk) -- [MLflow](https://mlflow.org/docs/latest/tracing/integrations/openai-agent) +- [MLflow (self-hosted/OSS](https://mlflow.org/docs/latest/tracing/integrations/openai-agent) +- [MLflow (Databricks hosted](https://docs.databricks.com/aws/en/mlflow/mlflow-tracing#-automatic-tracing) - [Braintrust](https://braintrust.dev/docs/guides/traces/integrations#openai-agents-sdk) - [Pydantic Logfire](https://logfire.pydantic.dev/docs/integrations/llms/openai/#openai-agents) - [AgentOps](https://docs.agentops.ai/v1/integrations/agentssdk) @@ -111,3 +112,4 @@ To customize this default setup, to send traces to alternative or additional bac - [Maxim AI](https://www.getmaxim.ai/docs/observe/integrations/openai-agents-sdk) - [Comet Opik](https://www.comet.com/docs/opik/tracing/integrations/openai_agents) - [Langfuse](https://langfuse.com/docs/integrations/openaiagentssdk/openai-agents) +- [Langtrace](https://docs.langtrace.ai/supported-integrations/llm-frameworks/openai-agents-sdk) diff --git a/docs/visualization.md b/docs/visualization.md new file mode 100644 index 00000000..00f3126d --- /dev/null +++ b/docs/visualization.md @@ -0,0 +1,86 @@ +# Agent Visualization + +Agent visualization allows you to generate a structured graphical representation of agents and their relationships using **Graphviz**. This is useful for understanding how agents, tools, and handoffs interact within an application. + +## Installation + +Install the optional `viz` dependency group: + +```bash +pip install "openai-agents[viz]" +``` + +## Generating a Graph + +You can generate an agent visualization using the `draw_graph` function. This function creates a directed graph where: + +- **Agents** are represented as yellow boxes. +- **Tools** are represented as green ellipses. +- **Handoffs** are directed edges from one agent to another. + +### Example Usage + +```python +from agents import Agent, function_tool +from agents.extensions.visualization import draw_graph + +@function_tool +def get_weather(city: str) -> str: + return f"The weather in {city} is sunny." + +spanish_agent = Agent( + name="Spanish agent", + instructions="You only speak Spanish.", +) + +english_agent = Agent( + name="English agent", + instructions="You only speak English", +) + +triage_agent = Agent( + name="Triage agent", + instructions="Handoff to the appropriate agent based on the language of the request.", + handoffs=[spanish_agent, english_agent], + tools=[get_weather], +) + +draw_graph(triage_agent) +``` + +![Agent Graph](./assets/images/graph.png) + +This generates a graph that visually represents the structure of the **triage agent** and its connections to sub-agents and tools. + + +## Understanding the Visualization + +The generated graph includes: + +- A **start node** (`__start__`) indicating the entry point. +- Agents represented as **rectangles** with yellow fill. +- Tools represented as **ellipses** with green fill. +- Directed edges indicating interactions: + - **Solid arrows** for agent-to-agent handoffs. + - **Dotted arrows** for tool invocations. +- An **end node** (`__end__`) indicating where execution terminates. + +## Customizing the Graph + +### Showing the Graph +By default, `draw_graph` displays the graph inline. To show the graph in a separate window, write the following: + +```python +draw_graph(triage_agent).view() +``` + +### Saving the Graph +By default, `draw_graph` displays the graph inline. To save it as a file, specify a filename: + +```python +draw_graph(triage_agent, filename="agent_graph.png") +``` + +This will generate `agent_graph.png` in the working directory. + + diff --git a/examples/mcp/filesystem_example/README.md b/examples/mcp/filesystem_example/README.md new file mode 100644 index 00000000..4ed6ac46 --- /dev/null +++ b/examples/mcp/filesystem_example/README.md @@ -0,0 +1,26 @@ +# MCP Filesystem Example + +This example uses the [filesystem MCP server](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem), running locally via `npx`. + +Run it via: + +``` +uv run python examples/mcp/filesystem_example/main.py +``` + +## Details + +The example uses the `MCPServerStdio` class from `agents.mcp`, with the command: + +```bash +npx -y "@modelcontextprotocol/server-filesystem" +``` + +It's only given access to the `sample_files` directory adjacent to the example, which contains some sample data. + +Under the hood: + +1. The server is spun up in a subprocess, and exposes a bunch of tools like `list_directory()`, `read_file()`, etc. +2. We add the server instance to the Agent via `mcp_agents`. +3. Each time the agent runs, we call out to the MCP server to fetch the list of tools via `server.list_tools()`. +4. If the LLM chooses to use an MCP tool, we call the MCP server to run the tool via `server.run_tool()`. diff --git a/examples/mcp/filesystem_example/main.py b/examples/mcp/filesystem_example/main.py new file mode 100644 index 00000000..ae6fadd2 --- /dev/null +++ b/examples/mcp/filesystem_example/main.py @@ -0,0 +1,57 @@ +import asyncio +import os +import shutil + +from agents import Agent, Runner, gen_trace_id, trace +from agents.mcp import MCPServer, MCPServerStdio + + +async def run(mcp_server: MCPServer): + agent = Agent( + name="Assistant", + instructions="Use the tools to read the filesystem and answer questions based on those files.", + mcp_servers=[mcp_server], + ) + + # List the files it can read + message = "Read the files and list them." + print(f"Running: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + # Ask about books + message = "What is my #1 favorite book?" + print(f"\n\nRunning: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + # Ask a question that reads then reasons. + message = "Look at my favorite songs. Suggest one new song that I might like." + print(f"\n\nRunning: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + +async def main(): + current_dir = os.path.dirname(os.path.abspath(__file__)) + samples_dir = os.path.join(current_dir, "sample_files") + + async with MCPServerStdio( + name="Filesystem Server, via npx", + params={ + "command": "npx", + "args": ["-y", "@modelcontextprotocol/server-filesystem", samples_dir], + }, + ) as server: + trace_id = gen_trace_id() + with trace(workflow_name="MCP Filesystem Example", trace_id=trace_id): + print(f"View trace: https://platform.openai.com/traces/{trace_id}\n") + await run(server) + + +if __name__ == "__main__": + # Let's make sure the user has npx installed + if not shutil.which("npx"): + raise RuntimeError("npx is not installed. Please install it with `npm install -g npx`.") + + asyncio.run(main()) diff --git a/examples/mcp/filesystem_example/sample_files/favorite_books.txt b/examples/mcp/filesystem_example/sample_files/favorite_books.txt new file mode 100644 index 00000000..c55f457e --- /dev/null +++ b/examples/mcp/filesystem_example/sample_files/favorite_books.txt @@ -0,0 +1,20 @@ +1. To Kill a Mockingbird – Harper Lee +2. Pride and Prejudice – Jane Austen +3. 1984 – George Orwell +4. The Hobbit – J.R.R. Tolkien +5. Harry Potter and the Sorcerer’s Stone – J.K. Rowling +6. The Great Gatsby – F. Scott Fitzgerald +7. Charlotte’s Web – E.B. White +8. Anne of Green Gables – Lucy Maud Montgomery +9. The Alchemist – Paulo Coelho +10. Little Women – Louisa May Alcott +11. The Catcher in the Rye – J.D. Salinger +12. Animal Farm – George Orwell +13. The Chronicles of Narnia: The Lion, the Witch, and the Wardrobe – C.S. Lewis +14. The Book Thief – Markus Zusak +15. A Wrinkle in Time – Madeleine L’Engle +16. The Secret Garden – Frances Hodgson Burnett +17. Moby-Dick – Herman Melville +18. Fahrenheit 451 – Ray Bradbury +19. Jane Eyre – Charlotte Brontë +20. The Little Prince – Antoine de Saint-Exupéry \ No newline at end of file diff --git a/examples/mcp/filesystem_example/sample_files/favorite_cities.txt b/examples/mcp/filesystem_example/sample_files/favorite_cities.txt new file mode 100644 index 00000000..1d3354f2 --- /dev/null +++ b/examples/mcp/filesystem_example/sample_files/favorite_cities.txt @@ -0,0 +1,4 @@ +- In the summer, I love visiting London. +- In the winter, Tokyo is great. +- In the spring, San Francisco. +- In the fall, New York is the best. \ No newline at end of file diff --git a/examples/mcp/filesystem_example/sample_files/favorite_songs.txt b/examples/mcp/filesystem_example/sample_files/favorite_songs.txt new file mode 100644 index 00000000..d659bb58 --- /dev/null +++ b/examples/mcp/filesystem_example/sample_files/favorite_songs.txt @@ -0,0 +1,10 @@ +1. "Here Comes the Sun" – The Beatles +2. "Imagine" – John Lennon +3. "Bohemian Rhapsody" – Queen +4. "Shake It Off" – Taylor Swift +5. "Billie Jean" – Michael Jackson +6. "Uptown Funk" – Mark Ronson ft. Bruno Mars +7. "Don’t Stop Believin’" – Journey +8. "Dancing Queen" – ABBA +9. "Happy" – Pharrell Williams +10. "Wonderwall" – Oasis diff --git a/examples/mcp/git_example/README.md b/examples/mcp/git_example/README.md new file mode 100644 index 00000000..6a809afa --- /dev/null +++ b/examples/mcp/git_example/README.md @@ -0,0 +1,26 @@ +# MCP Git Example + +This example uses the [git MCP server](https://github.com/modelcontextprotocol/servers/tree/main/src/git), running locally via `uvx`. + +Run it via: + +``` +uv run python examples/mcp/git_example/main.py +``` + +## Details + +The example uses the `MCPServerStdio` class from `agents.mcp`, with the command: + +```bash +uvx mcp-server-git +``` + +Prior to running the agent, the user is prompted to provide a local directory path to their git repo. Using that, the Agent can invoke Git MCP tools like `git_log` to inspect the git commit log. + +Under the hood: + +1. The server is spun up in a subprocess, and exposes a bunch of tools like `git_log()` +2. We add the server instance to the Agent via `mcp_agents`. +3. Each time the agent runs, we call out to the MCP server to fetch the list of tools via `server.list_tools()`. The result is cached. +4. If the LLM chooses to use an MCP tool, we call the MCP server to run the tool via `server.run_tool()`. diff --git a/examples/mcp/git_example/main.py b/examples/mcp/git_example/main.py new file mode 100644 index 00000000..ab229e85 --- /dev/null +++ b/examples/mcp/git_example/main.py @@ -0,0 +1,44 @@ +import asyncio +import shutil + +from agents import Agent, Runner, trace +from agents.mcp import MCPServer, MCPServerStdio + + +async def run(mcp_server: MCPServer, directory_path: str): + agent = Agent( + name="Assistant", + instructions=f"Answer questions about the git repository at {directory_path}, use that for repo_path", + mcp_servers=[mcp_server], + ) + + message = "Who's the most frequent contributor?" + print("\n" + "-" * 40) + print(f"Running: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + message = "Summarize the last change in the repository." + print("\n" + "-" * 40) + print(f"Running: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + +async def main(): + # Ask the user for the directory path + directory_path = input("Please enter the path to the git repository: ") + + async with MCPServerStdio( + cache_tools_list=True, # Cache the tools list, for demonstration + params={"command": "uvx", "args": ["mcp-server-git"]}, + ) as server: + with trace(workflow_name="MCP Git Example"): + await run(server, directory_path) + + +if __name__ == "__main__": + if not shutil.which("uvx"): + raise RuntimeError("uvx is not installed. Please install it with `pip install uvx`.") + + asyncio.run(main()) diff --git a/examples/mcp/sse_example/README.md b/examples/mcp/sse_example/README.md new file mode 100644 index 00000000..605c2cc9 --- /dev/null +++ b/examples/mcp/sse_example/README.md @@ -0,0 +1,13 @@ +# MCP SSE Example + +This example uses a local SSE server in [server.py](server.py). + +Run the example via: + +``` +uv run python python examples/mcp/sse_example/main.py +``` + +## Details + +The example uses the `MCPServerSse` class from `agents.mcp`. The server runs in a sub-process at `https://localhost:8000/sse`. diff --git a/examples/mcp/sse_example/main.py b/examples/mcp/sse_example/main.py new file mode 100644 index 00000000..024b087e --- /dev/null +++ b/examples/mcp/sse_example/main.py @@ -0,0 +1,83 @@ +import asyncio +import os +import shutil +import subprocess +import time +from typing import Any + +from agents import Agent, Runner, gen_trace_id, trace +from agents.mcp import MCPServer, MCPServerSse +from agents.model_settings import ModelSettings + + +async def run(mcp_server: MCPServer): + agent = Agent( + name="Assistant", + instructions="Use the tools to answer the questions.", + mcp_servers=[mcp_server], + model_settings=ModelSettings(tool_choice="required"), + ) + + # Use the `add` tool to add two numbers + message = "Add these numbers: 7 and 22." + print(f"Running: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + # Run the `get_weather` tool + message = "What's the weather in Tokyo?" + print(f"\n\nRunning: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + # Run the `get_secret_word` tool + message = "What's the secret word?" + print(f"\n\nRunning: {message}") + result = await Runner.run(starting_agent=agent, input=message) + print(result.final_output) + + +async def main(): + async with MCPServerSse( + name="SSE Python Server", + params={ + "url": "http://localhost:8000/sse", + }, + ) as server: + trace_id = gen_trace_id() + with trace(workflow_name="SSE Example", trace_id=trace_id): + print(f"View trace: https://platform.openai.com/traces/{trace_id}\n") + await run(server) + + +if __name__ == "__main__": + # Let's make sure the user has uv installed + if not shutil.which("uv"): + raise RuntimeError( + "uv is not installed. Please install it: https://docs.astral.sh/uv/getting-started/installation/" + ) + + # We'll run the SSE server in a subprocess. Usually this would be a remote server, but for this + # demo, we'll run it locally at http://localhost:8000/sse + process: subprocess.Popen[Any] | None = None + try: + this_dir = os.path.dirname(os.path.abspath(__file__)) + server_file = os.path.join(this_dir, "server.py") + + print("Starting SSE server at http://localhost:8000/sse ...") + + # Run `uv run server.py` to start the SSE server + process = subprocess.Popen(["uv", "run", server_file]) + # Give it 3 seconds to start + time.sleep(3) + + print("SSE server started. Running example...\n\n") + except Exception as e: + print(f"Error starting SSE server: {e}") + exit(1) + + try: + asyncio.run(main()) + finally: + if process: + process.terminate() diff --git a/examples/mcp/sse_example/server.py b/examples/mcp/sse_example/server.py new file mode 100644 index 00000000..df364aa3 --- /dev/null +++ b/examples/mcp/sse_example/server.py @@ -0,0 +1,33 @@ +import random + +import requests +from mcp.server.fastmcp import FastMCP + +# Create server +mcp = FastMCP("Echo Server") + + +@mcp.tool() +def add(a: int, b: int) -> int: + """Add two numbers""" + print(f"[debug-server] add({a}, {b})") + return a + b + + +@mcp.tool() +def get_secret_word() -> str: + print("[debug-server] get_secret_word()") + return random.choice(["apple", "banana", "cherry"]) + + +@mcp.tool() +def get_current_weather(city: str) -> str: + print(f"[debug-server] get_current_weather({city})") + + endpoint = "https://wttr.in" + response = requests.get(f"{endpoint}/{city}") + return response.text + + +if __name__ == "__main__": + mcp.run(transport="sse") diff --git a/examples/voice/static/main.py b/examples/voice/static/main.py index 4e3840fc..1b9e2024 100644 --- a/examples/voice/static/main.py +++ b/examples/voice/static/main.py @@ -1,6 +1,8 @@ import asyncio import random +import numpy as np + from agents import Agent, function_tool from agents.extensions.handoff_prompt import prompt_with_handoff_instructions from agents.voice import ( @@ -78,6 +80,9 @@ async def main(): elif event.type == "voice_stream_event_lifecycle": print(f"Received lifecycle event: {event.event}") + # Add 1 second of silence to the end of the stream to avoid cutting off the last audio. + player.add_audio(np.zeros(24000 * 1, dtype=np.int16)) + if __name__ == "__main__": asyncio.run(main()) diff --git a/examples/voice/static/util.py b/examples/voice/static/util.py index 455bafee..a5806f41 100644 --- a/examples/voice/static/util.py +++ b/examples/voice/static/util.py @@ -62,6 +62,7 @@ class AudioPlayer: return self def __exit__(self, exc_type, exc_value, traceback): + self.stream.stop() # wait for the stream to finish self.stream.close() def add_audio(self, audio_data: npt.NDArray[np.int16]): diff --git a/mkdocs.yml b/mkdocs.yml index f1b6e865..934eb798 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -30,6 +30,7 @@ nav: - results.md - streaming.md - tools.md + - mcp.md - handoffs.md - tracing.md - context.md @@ -37,6 +38,7 @@ nav: - multi_agent.md - models.md - config.md + - visualization.md - Voice agents: - voice/quickstart.md - voice/pipeline.md @@ -62,6 +64,8 @@ nav: - ref/models/interface.md - ref/models/openai_chatcompletions.md - ref/models/openai_responses.md + - ref/mcp/server.md + - ref/mcp/util.md - Tracing: - ref/tracing/index.md - ref/tracing/create.md @@ -110,6 +114,8 @@ plugins: show_signature_annotations: true # Makes the font sizes nicer heading_level: 3 + # Show inherited members + inherited_members: true extra: # Remove material generation message in footer diff --git a/pyproject.toml b/pyproject.toml index ef7c36a8..c49933a1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -19,7 +19,8 @@ dependencies = [ "prompt_toolkit>=3.0.39", "dotenv>=0.9.9", "litellm>=1.63.7", - "mako>=1.3.9" + "mako>=1.3.9", + "mcp; python_version >= '3.10'", ] classifiers = [ "Typing :: Typed", @@ -41,6 +42,7 @@ Repository = "https://github.com/openai/openai-agents-python" [project.optional-dependencies] voice = ["numpy>=2.2.0, <3; python_version>='3.10'", "websockets>=15.0, <16"] +viz = ["graphviz>=0.17"] [dependency-groups] dev = [ @@ -59,12 +61,13 @@ dev = [ "pynput", "types-pynput", "sounddevice", - "pynput", "textual", "websockets", "litellm", - "prisma" + "prisma", + "graphviz", ] + [tool.uv.workspace] members = ["agents"] diff --git a/src/cai/sdk/agents/__init__.py b/src/cai/sdk/agents/__init__.py index 47bb2649..242f5649 100644 --- a/src/cai/sdk/agents/__init__.py +++ b/src/cai/sdk/agents/__init__.py @@ -70,6 +70,7 @@ from .tracing import ( GenerationSpanData, GuardrailSpanData, HandoffSpanData, + MCPListToolsSpanData, Span, SpanData, SpanError, @@ -89,6 +90,7 @@ from .tracing import ( get_current_trace, guardrail_span, handoff_span, + mcp_tools_span, set_trace_processors, set_tracing_disabled, set_tracing_export_api_key, @@ -220,6 +222,7 @@ __all__ = [ "speech_group_span", "transcription_span", "speech_span", + "mcp_tools_span", "trace", "Trace", "TracingProcessor", @@ -234,6 +237,7 @@ __all__ = [ "HandoffSpanData", "SpeechGroupSpanData", "SpeechSpanData", + "MCPListToolsSpanData", "TranscriptionSpanData", "set_default_openai_key", "set_default_openai_client", diff --git a/src/cai/sdk/agents/_run_impl.py b/src/cai/sdk/agents/_run_impl.py index 2849538d..94c181b7 100644 --- a/src/cai/sdk/agents/_run_impl.py +++ b/src/cai/sdk/agents/_run_impl.py @@ -1,9 +1,10 @@ from __future__ import annotations import asyncio +import dataclasses import inspect from collections.abc import Awaitable -from dataclasses import dataclass +from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, cast from openai.types.responses import ( @@ -47,10 +48,11 @@ from .items import ( ) from .lifecycle import RunHooks from .logger import logger +from .model_settings import ModelSettings from .models.interface import ModelTracing from .run_context import RunContextWrapper, TContext from .stream_events import RunItemStreamEvent, StreamEvent -from .tool import ComputerTool, FunctionTool, FunctionToolResult +from .tool import ComputerTool, FunctionTool, FunctionToolResult, Tool from .tracing import ( SpanError, Trace, @@ -75,6 +77,23 @@ QUEUE_COMPLETE_SENTINEL = QueueCompleteSentinel() _NOT_FINAL_OUTPUT = ToolsToFinalOutputResult(is_final_output=False, final_output=None) +@dataclass +class AgentToolUseTracker: + agent_to_tools: list[tuple[Agent, list[str]]] = field(default_factory=list) + """Tuple of (agent, list of tools used). Can't use a dict because agents aren't hashable.""" + + def add_tool_use(self, agent: Agent[Any], tool_names: list[str]) -> None: + existing_data = next((item for item in self.agent_to_tools if item[0] == agent), None) + if existing_data: + existing_data[1].extend(tool_names) + else: + self.agent_to_tools.append((agent, tool_names)) + + def has_used_tools(self, agent: Agent[Any]) -> bool: + existing_data = next((item for item in self.agent_to_tools if item[0] == agent), None) + return existing_data is not None and len(existing_data[1]) > 0 + + @dataclass class ToolRunHandoff: handoff: Handoff @@ -99,6 +118,7 @@ class ProcessedResponse: handoffs: list[ToolRunHandoff] functions: list[ToolRunFunction] computer_actions: list[ToolRunComputerAction] + tools_used: list[str] # Names of all tools used, including hosted tools def has_tools_to_run(self) -> bool: # Handoffs, functions and computer actions need local processing @@ -296,11 +316,24 @@ class RunImpl: next_step=NextStepRunAgain(), ) + @classmethod + def maybe_reset_tool_choice( + cls, agent: Agent[Any], tool_use_tracker: AgentToolUseTracker, model_settings: ModelSettings + ) -> ModelSettings: + """Resets tool choice to None if the agent has used tools and the agent's reset_tool_choice + flag is True.""" + + if agent.reset_tool_choice is True and tool_use_tracker.has_used_tools(agent): + return dataclasses.replace(model_settings, tool_choice=None) + + return model_settings + @classmethod def process_model_response( cls, *, agent: Agent[Any], + all_tools: list[Tool], response: ModelResponse, output_schema: AgentOutputSchema | None, handoffs: list[Handoff], @@ -310,22 +343,25 @@ class RunImpl: run_handoffs = [] functions = [] computer_actions = [] - + tools_used: list[str] = [] handoff_map = {handoff.tool_name: handoff for handoff in handoffs} - function_map = {tool.name: tool for tool in agent.tools if isinstance(tool, FunctionTool)} - computer_tool = next((tool for tool in agent.tools if isinstance(tool, ComputerTool)), None) + function_map = {tool.name: tool for tool in all_tools if isinstance(tool, FunctionTool)} + computer_tool = next((tool for tool in all_tools if isinstance(tool, ComputerTool)), None) for output in response.output: if isinstance(output, ResponseOutputMessage): items.append(MessageOutputItem(raw_item=output, agent=agent)) elif isinstance(output, ResponseFileSearchToolCall): items.append(ToolCallItem(raw_item=output, agent=agent)) + tools_used.append("file_search") elif isinstance(output, ResponseFunctionWebSearch): items.append(ToolCallItem(raw_item=output, agent=agent)) + tools_used.append("web_search") elif isinstance(output, ResponseReasoningItem): items.append(ReasoningItem(raw_item=output, agent=agent)) elif isinstance(output, ResponseComputerToolCall): items.append(ToolCallItem(raw_item=output, agent=agent)) + tools_used.append("computer_use") if not computer_tool: _error_tracing.attach_error_to_current_span( SpanError( @@ -347,6 +383,8 @@ class RunImpl: if not isinstance(output, ResponseFunctionToolCall): continue + tools_used.append(output.name) + # Handoffs if output.name in handoff_map: items.append(HandoffCallItem(raw_item=output, agent=agent)) @@ -378,6 +416,7 @@ class RunImpl: handoffs=run_handoffs, functions=functions, computer_actions=computer_actions, + tools_used=tools_used, ) @classmethod @@ -490,7 +529,8 @@ class RunImpl: run_config: RunConfig, ) -> SingleStepResult: # If there is more than one handoff, add tool responses that reject those handoffs - if len(run_handoffs) > 1: + multiple_handoffs = len(run_handoffs) > 1 + if multiple_handoffs: output_message = "Multiple handoffs detected, ignoring this one." new_step_items.extend( [ @@ -512,6 +552,16 @@ class RunImpl: context_wrapper, actual_handoff.tool_call.arguments ) span_handoff.span_data.to_agent = new_agent.name + if multiple_handoffs: + requested_agents = [handoff.handoff.agent_name for handoff in run_handoffs] + span_handoff.set_error( + SpanError( + message="Multiple handoffs requested", + data={ + "requested_agents": requested_agents, + }, + ) + ) # Append a tool output item for the handoff new_step_items.append( diff --git a/src/cai/sdk/agents/agent.py b/src/cai/sdk/agents/agent.py index 38a0c551..b96b1e90 100644 --- a/src/cai/sdk/agents/agent.py +++ b/src/cai/sdk/agents/agent.py @@ -12,6 +12,7 @@ from .guardrail import InputGuardrail, OutputGuardrail from .handoffs import Handoff from .items import ItemHelpers from .logger import logger +from .mcp import MCPUtil from .model_settings import ModelSettings from .models.interface import Model from .run_context import RunContextWrapper, TContext @@ -21,6 +22,7 @@ from .util._types import MaybeAwaitable if TYPE_CHECKING: from .lifecycle import AgentHooks + from .mcp import MCPServer from .result import RunResult @@ -110,6 +112,16 @@ class Agent(Generic[TContext]): tools: list[Tool] = field(default_factory=list) """A list of tools that the agent can use.""" + mcp_servers: list[MCPServer] = field(default_factory=list) + """A list of [Model Context Protocol](https://modelcontextprotocol.io/) servers that + the agent can use. Every time the agent runs, it will include tools from these servers in the + list of available tools. + + NOTE: You are expected to manage the lifecycle of these servers. Specifically, you must call + `server.connect()` before passing it to the agent, and `server.cleanup()` when the server is no + longer needed. + """ + input_guardrails: list[InputGuardrail[TContext]] = field(default_factory=list) """A list of checks that run in parallel to the agent's execution, before generating a response. Runs only if the agent is the first agent in the chain. @@ -146,6 +158,10 @@ class Agent(Generic[TContext]): web search, etc are always processed by the LLM. """ + reset_tool_choice: bool = True + """Whether to reset the tool choice to the default value after a tool has been called. Defaults + to True. This ensures that the agent doesn't enter an infinite loop of tool usage.""" + def clone(self, **kwargs: Any) -> Agent[TContext]: """Make a copy of the agent, with the given arguments changed. For example, you could do: ``` @@ -208,3 +224,12 @@ class Agent(Generic[TContext]): logger.error(f"Instructions must be a string or a function, got {self.instructions}") return None + + async def get_mcp_tools(self) -> list[Tool]: + """Fetches the available tools from the MCP servers.""" + return await MCPUtil.get_all_function_tools(self.mcp_servers) + + async def get_all_tools(self) -> list[Tool]: + """All agent tools, including MCP tools and function tools.""" + mcp_tools = await self.get_mcp_tools() + return mcp_tools + self.tools diff --git a/src/cai/sdk/agents/extensions/visualization.py b/src/cai/sdk/agents/extensions/visualization.py new file mode 100644 index 00000000..5fb35062 --- /dev/null +++ b/src/cai/sdk/agents/extensions/visualization.py @@ -0,0 +1,137 @@ +from typing import Optional + +import graphviz # type: ignore + +from agents import Agent +from agents.handoffs import Handoff +from agents.tool import Tool + + +def get_main_graph(agent: Agent) -> str: + """ + Generates the main graph structure in DOT format for the given agent. + + Args: + agent (Agent): The agent for which the graph is to be generated. + + Returns: + str: The DOT format string representing the graph. + """ + parts = [ + """ + digraph G { + graph [splines=true]; + node [fontname="Arial"]; + edge [penwidth=1.5]; + """ + ] + parts.append(get_all_nodes(agent)) + parts.append(get_all_edges(agent)) + parts.append("}") + return "".join(parts) + + +def get_all_nodes(agent: Agent, parent: Optional[Agent] = None) -> str: + """ + Recursively generates the nodes for the given agent and its handoffs in DOT format. + + Args: + agent (Agent): The agent for which the nodes are to be generated. + + Returns: + str: The DOT format string representing the nodes. + """ + parts = [] + + # Start and end the graph + parts.append( + '"__start__" [label="__start__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" + '"__end__" [label="__end__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" + ) + # Ensure parent agent node is colored + if not parent: + parts.append( + f'"{agent.name}" [label="{agent.name}", shape=box, style=filled, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" + ) + + for tool in agent.tools: + parts.append( + f'"{tool.name}" [label="{tool.name}", shape=ellipse, style=filled, ' + f"fillcolor=lightgreen, width=0.5, height=0.3];" + ) + + for handoff in agent.handoffs: + if isinstance(handoff, Handoff): + parts.append( + f'"{handoff.agent_name}" [label="{handoff.agent_name}", ' + f"shape=box, style=filled, style=rounded, " + f"fillcolor=lightyellow, width=1.5, height=0.8];" + ) + if isinstance(handoff, Agent): + parts.append( + f'"{handoff.name}" [label="{handoff.name}", ' + f"shape=box, style=filled, style=rounded, " + f"fillcolor=lightyellow, width=1.5, height=0.8];" + ) + parts.append(get_all_nodes(handoff)) + + return "".join(parts) + + +def get_all_edges(agent: Agent, parent: Optional[Agent] = None) -> str: + """ + Recursively generates the edges for the given agent and its handoffs in DOT format. + + Args: + agent (Agent): The agent for which the edges are to be generated. + parent (Agent, optional): The parent agent. Defaults to None. + + Returns: + str: The DOT format string representing the edges. + """ + parts = [] + + if not parent: + parts.append(f'"__start__" -> "{agent.name}";') + + for tool in agent.tools: + parts.append(f""" + "{agent.name}" -> "{tool.name}" [style=dotted, penwidth=1.5]; + "{tool.name}" -> "{agent.name}" [style=dotted, penwidth=1.5];""") + + for handoff in agent.handoffs: + if isinstance(handoff, Handoff): + parts.append(f""" + "{agent.name}" -> "{handoff.agent_name}";""") + if isinstance(handoff, Agent): + parts.append(f""" + "{agent.name}" -> "{handoff.name}";""") + parts.append(get_all_edges(handoff, agent)) + + if not agent.handoffs and not isinstance(agent, Tool): # type: ignore + parts.append(f'"{agent.name}" -> "__end__";') + + return "".join(parts) + + +def draw_graph(agent: Agent, filename: Optional[str] = None) -> graphviz.Source: + """ + Draws the graph for the given agent and optionally saves it as a PNG file. + + Args: + agent (Agent): The agent for which the graph is to be drawn. + filename (str): The name of the file to save the graph as a PNG. + + Returns: + graphviz.Source: The graphviz Source object representing the graph. + """ + dot_code = get_main_graph(agent) + graph = graphviz.Source(dot_code) + + if filename: + graph.render(filename, format="png") + + return graph diff --git a/src/cai/sdk/agents/mcp/__init__.py b/src/cai/sdk/agents/mcp/__init__.py new file mode 100644 index 00000000..1a72a89f --- /dev/null +++ b/src/cai/sdk/agents/mcp/__init__.py @@ -0,0 +1,21 @@ +try: + from .server import ( + MCPServer, + MCPServerSse, + MCPServerSseParams, + MCPServerStdio, + MCPServerStdioParams, + ) +except ImportError: + pass + +from .util import MCPUtil + +__all__ = [ + "MCPServer", + "MCPServerSse", + "MCPServerSseParams", + "MCPServerStdio", + "MCPServerStdioParams", + "MCPUtil", +] diff --git a/src/cai/sdk/agents/mcp/server.py b/src/cai/sdk/agents/mcp/server.py new file mode 100644 index 00000000..e70d7ce6 --- /dev/null +++ b/src/cai/sdk/agents/mcp/server.py @@ -0,0 +1,301 @@ +from __future__ import annotations + +import abc +import asyncio +from contextlib import AbstractAsyncContextManager, AsyncExitStack +from pathlib import Path +from typing import Any, Literal + +from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream +from mcp import ClientSession, StdioServerParameters, Tool as MCPTool, stdio_client +from mcp.client.sse import sse_client +from mcp.types import CallToolResult, JSONRPCMessage +from typing_extensions import NotRequired, TypedDict + +from ..exceptions import UserError +from ..logger import logger + + +class MCPServer(abc.ABC): + """Base class for Model Context Protocol servers.""" + + @abc.abstractmethod + async def connect(self): + """Connect to the server. For example, this might mean spawning a subprocess or + opening a network connection. The server is expected to remain connected until + `cleanup()` is called. + """ + pass + + @property + @abc.abstractmethod + def name(self) -> str: + """A readable name for the server.""" + pass + + @abc.abstractmethod + async def cleanup(self): + """Cleanup the server. For example, this might mean closing a subprocess or + closing a network connection. + """ + pass + + @abc.abstractmethod + async def list_tools(self) -> list[MCPTool]: + """List the tools available on the server.""" + pass + + @abc.abstractmethod + async def call_tool(self, tool_name: str, arguments: dict[str, Any] | None) -> CallToolResult: + """Invoke a tool on the server.""" + pass + + +class _MCPServerWithClientSession(MCPServer, abc.ABC): + """Base class for MCP servers that use a `ClientSession` to communicate with the server.""" + + def __init__(self, cache_tools_list: bool): + """ + Args: + cache_tools_list: Whether to cache the tools list. If `True`, the tools list will be + cached and only fetched from the server once. If `False`, the tools list will be + fetched from the server on each call to `list_tools()`. The cache can be invalidated + by calling `invalidate_tools_cache()`. You should set this to `True` if you know the + server will not change its tools list, because it can drastically improve latency + (by avoiding a round-trip to the server every time). + """ + self.session: ClientSession | None = None + self.exit_stack: AsyncExitStack = AsyncExitStack() + self._cleanup_lock: asyncio.Lock = asyncio.Lock() + self.cache_tools_list = cache_tools_list + + # The cache is always dirty at startup, so that we fetch tools at least once + self._cache_dirty = True + self._tools_list: list[MCPTool] | None = None + + @abc.abstractmethod + def create_streams( + self, + ) -> AbstractAsyncContextManager[ + tuple[ + MemoryObjectReceiveStream[JSONRPCMessage | Exception], + MemoryObjectSendStream[JSONRPCMessage], + ] + ]: + """Create the streams for the server.""" + pass + + async def __aenter__(self): + await self.connect() + return self + + async def __aexit__(self, exc_type, exc_value, traceback): + await self.cleanup() + + def invalidate_tools_cache(self): + """Invalidate the tools cache.""" + self._cache_dirty = True + + async def connect(self): + """Connect to the server.""" + try: + transport = await self.exit_stack.enter_async_context(self.create_streams()) + read, write = transport + session = await self.exit_stack.enter_async_context(ClientSession(read, write)) + await session.initialize() + self.session = session + except Exception as e: + logger.error(f"Error initializing MCP server: {e}") + await self.cleanup() + raise + + async def list_tools(self) -> list[MCPTool]: + """List the tools available on the server.""" + if not self.session: + raise UserError("Server not initialized. Make sure you call `connect()` first.") + + # Return from cache if caching is enabled, we have tools, and the cache is not dirty + if self.cache_tools_list and not self._cache_dirty and self._tools_list: + return self._tools_list + + # Reset the cache dirty to False + self._cache_dirty = False + + # Fetch the tools from the server + self._tools_list = (await self.session.list_tools()).tools + return self._tools_list + + async def call_tool(self, tool_name: str, arguments: dict[str, Any] | None) -> CallToolResult: + """Invoke a tool on the server.""" + if not self.session: + raise UserError("Server not initialized. Make sure you call `connect()` first.") + + return await self.session.call_tool(tool_name, arguments) + + async def cleanup(self): + """Cleanup the server.""" + async with self._cleanup_lock: + try: + await self.exit_stack.aclose() + self.session = None + except Exception as e: + logger.error(f"Error cleaning up server: {e}") + + +class MCPServerStdioParams(TypedDict): + """Mirrors `mcp.client.stdio.StdioServerParameters`, but lets you pass params without another + import. + """ + + command: str + """The executable to run to start the server. For example, `python` or `node`.""" + + args: NotRequired[list[str]] + """Command line args to pass to the `command` executable. For example, `['foo.py']` or + `['server.js', '--port', '8080']`.""" + + env: NotRequired[dict[str, str]] + """The environment variables to set for the server. .""" + + cwd: NotRequired[str | Path] + """The working directory to use when spawning the process.""" + + encoding: NotRequired[str] + """The text encoding used when sending/receiving messages to the server. Defaults to `utf-8`.""" + + encoding_error_handler: NotRequired[Literal["strict", "ignore", "replace"]] + """The text encoding error handler. Defaults to `strict`. + + See https://docs.python.org/3/library/codecs.html#codec-base-classes for + explanations of possible values. + """ + + +class MCPServerStdio(_MCPServerWithClientSession): + """MCP server implementation that uses the stdio transport. See the [spec] + (https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#stdio) for + details. + """ + + def __init__( + self, + params: MCPServerStdioParams, + cache_tools_list: bool = False, + name: str | None = None, + ): + """Create a new MCP server based on the stdio transport. + + Args: + params: The params that configure the server. This includes the command to run to + start the server, the args to pass to the command, the environment variables to + set for the server, the working directory to use when spawning the process, and + the text encoding used when sending/receiving messages to the server. + cache_tools_list: Whether to cache the tools list. If `True`, the tools list will be + cached and only fetched from the server once. If `False`, the tools list will be + fetched from the server on each call to `list_tools()`. The cache can be + invalidated by calling `invalidate_tools_cache()`. You should set this to `True` + if you know the server will not change its tools list, because it can drastically + improve latency (by avoiding a round-trip to the server every time). + name: A readable name for the server. If not provided, we'll create one from the + command. + """ + super().__init__(cache_tools_list) + + self.params = StdioServerParameters( + command=params["command"], + args=params.get("args", []), + env=params.get("env"), + cwd=params.get("cwd"), + encoding=params.get("encoding", "utf-8"), + encoding_error_handler=params.get("encoding_error_handler", "strict"), + ) + + self._name = name or f"stdio: {self.params.command}" + + def create_streams( + self, + ) -> AbstractAsyncContextManager[ + tuple[ + MemoryObjectReceiveStream[JSONRPCMessage | Exception], + MemoryObjectSendStream[JSONRPCMessage], + ] + ]: + """Create the streams for the server.""" + return stdio_client(self.params) + + @property + def name(self) -> str: + """A readable name for the server.""" + return self._name + + +class MCPServerSseParams(TypedDict): + """Mirrors the params in`mcp.client.sse.sse_client`.""" + + url: str + """The URL of the server.""" + + headers: NotRequired[dict[str, str]] + """The headers to send to the server.""" + + timeout: NotRequired[float] + """The timeout for the HTTP request. Defaults to 5 seconds.""" + + sse_read_timeout: NotRequired[float] + """The timeout for the SSE connection, in seconds. Defaults to 5 minutes.""" + + +class MCPServerSse(_MCPServerWithClientSession): + """MCP server implementation that uses the HTTP with SSE transport. See the [spec] + (https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#http-with-sse) + for details. + """ + + def __init__( + self, + params: MCPServerSseParams, + cache_tools_list: bool = False, + name: str | None = None, + ): + """Create a new MCP server based on the HTTP with SSE transport. + + Args: + params: The params that configure the server. This includes the URL of the server, + the headers to send to the server, the timeout for the HTTP request, and the + timeout for the SSE connection. + + cache_tools_list: Whether to cache the tools list. If `True`, the tools list will be + cached and only fetched from the server once. If `False`, the tools list will be + fetched from the server on each call to `list_tools()`. The cache can be + invalidated by calling `invalidate_tools_cache()`. You should set this to `True` + if you know the server will not change its tools list, because it can drastically + improve latency (by avoiding a round-trip to the server every time). + + name: A readable name for the server. If not provided, we'll create one from the + URL. + """ + super().__init__(cache_tools_list) + + self.params = params + self._name = name or f"sse: {self.params['url']}" + + def create_streams( + self, + ) -> AbstractAsyncContextManager[ + tuple[ + MemoryObjectReceiveStream[JSONRPCMessage | Exception], + MemoryObjectSendStream[JSONRPCMessage], + ] + ]: + """Create the streams for the server.""" + return sse_client( + url=self.params["url"], + headers=self.params.get("headers", None), + timeout=self.params.get("timeout", 5), + sse_read_timeout=self.params.get("sse_read_timeout", 60 * 5), + ) + + @property + def name(self) -> str: + """A readable name for the server.""" + return self._name diff --git a/src/cai/sdk/agents/mcp/util.py b/src/cai/sdk/agents/mcp/util.py new file mode 100644 index 00000000..36c18bea --- /dev/null +++ b/src/cai/sdk/agents/mcp/util.py @@ -0,0 +1,115 @@ +import functools +import json +from typing import TYPE_CHECKING, Any + +from .. import _debug +from ..exceptions import AgentsException, ModelBehaviorError, UserError +from ..logger import logger +from ..run_context import RunContextWrapper +from ..tool import FunctionTool, Tool +from ..tracing import FunctionSpanData, get_current_span, mcp_tools_span + +if TYPE_CHECKING: + from mcp.types import Tool as MCPTool + + from .server import MCPServer + + +class MCPUtil: + """Set of utilities for interop between MCP and Agents SDK tools.""" + + @classmethod + async def get_all_function_tools(cls, servers: list["MCPServer"]) -> list[Tool]: + """Get all function tools from a list of MCP servers.""" + tools = [] + tool_names: set[str] = set() + for server in servers: + server_tools = await cls.get_function_tools(server) + server_tool_names = {tool.name for tool in server_tools} + if len(server_tool_names & tool_names) > 0: + raise UserError( + f"Duplicate tool names found across MCP servers: " + f"{server_tool_names & tool_names}" + ) + tool_names.update(server_tool_names) + tools.extend(server_tools) + + return tools + + @classmethod + async def get_function_tools(cls, server: "MCPServer") -> list[Tool]: + """Get all function tools from a single MCP server.""" + + with mcp_tools_span(server=server.name) as span: + tools = await server.list_tools() + span.span_data.result = [tool.name for tool in tools] + + return [cls.to_function_tool(tool, server) for tool in tools] + + @classmethod + def to_function_tool(cls, tool: "MCPTool", server: "MCPServer") -> FunctionTool: + """Convert an MCP tool to an Agents SDK function tool.""" + invoke_func = functools.partial(cls.invoke_mcp_tool, server, tool) + return FunctionTool( + name=tool.name, + description=tool.description or "", + params_json_schema=tool.inputSchema, + on_invoke_tool=invoke_func, + strict_json_schema=False, + ) + + @classmethod + async def invoke_mcp_tool( + cls, server: "MCPServer", tool: "MCPTool", context: RunContextWrapper[Any], input_json: str + ) -> str: + """Invoke an MCP tool and return the result as a string.""" + try: + json_data: dict[str, Any] = json.loads(input_json) if input_json else {} + except Exception as e: + if _debug.DONT_LOG_TOOL_DATA: + logger.debug(f"Invalid JSON input for tool {tool.name}") + else: + logger.debug(f"Invalid JSON input for tool {tool.name}: {input_json}") + raise ModelBehaviorError( + f"Invalid JSON input for tool {tool.name}: {input_json}" + ) from e + + if _debug.DONT_LOG_TOOL_DATA: + logger.debug(f"Invoking MCP tool {tool.name}") + else: + logger.debug(f"Invoking MCP tool {tool.name} with input {input_json}") + + try: + result = await server.call_tool(tool.name, json_data) + except Exception as e: + logger.error(f"Error invoking MCP tool {tool.name}: {e}") + raise AgentsException(f"Error invoking MCP tool {tool.name}: {e}") from e + + if _debug.DONT_LOG_TOOL_DATA: + logger.debug(f"MCP tool {tool.name} completed.") + else: + logger.debug(f"MCP tool {tool.name} returned {result}") + + # The MCP tool result is a list of content items, whereas OpenAI tool outputs are a single + # string. We'll try to convert. + if len(result.content) == 1: + tool_output = result.content[0].model_dump_json() + elif len(result.content) > 1: + tool_output = json.dumps([item.model_dump() for item in result.content]) + else: + logger.error(f"Errored MCP tool result: {result}") + tool_output = "Error running tool." + + current_span = get_current_span() + if current_span: + if isinstance(current_span.span_data, FunctionSpanData): + current_span.span_data.output = tool_output + current_span.span_data.mcp_data = { + "server": server.name, + } + else: + logger.warning( + f"Current span is not a FunctionSpanData, skipping tool output: {current_span}" + ) + + return tool_output diff --git a/src/cai/sdk/agents/model_settings.py b/src/cai/sdk/agents/model_settings.py index cc4b6cb6..2b0885ab 100644 --- a/src/cai/sdk/agents/model_settings.py +++ b/src/cai/sdk/agents/model_settings.py @@ -1,6 +1,6 @@ from __future__ import annotations -from dataclasses import dataclass +from dataclasses import dataclass, fields, replace from typing import Literal @@ -30,8 +30,9 @@ class ModelSettings: tool_choice: Literal["auto", "required", "none"] | str | None = None """The tool choice to use when calling the model.""" - parallel_tool_calls: bool | None = False - """Whether to use parallel tool calls when calling the model.""" + parallel_tool_calls: bool | None = None + """Whether to use parallel tool calls when calling the model. + Defaults to False if not provided.""" truncation: Literal["auto", "disabled"] | None = None """The truncation strategy to use when calling the model.""" @@ -39,18 +40,19 @@ class ModelSettings: max_tokens: int | None = None """The maximum number of output tokens to generate.""" + store: bool | None = None + """Whether to store the generated model response for later retrieval. + Defaults to True if not provided.""" + def resolve(self, override: ModelSettings | None) -> ModelSettings: """Produce a new ModelSettings by overlaying any non-None values from the override on top of this instance.""" if override is None: return self - return ModelSettings( - temperature=override.temperature or self.temperature, - top_p=override.top_p or self.top_p, - frequency_penalty=override.frequency_penalty or self.frequency_penalty, - presence_penalty=override.presence_penalty or self.presence_penalty, - tool_choice=override.tool_choice or self.tool_choice, - parallel_tool_calls=override.parallel_tool_calls or self.parallel_tool_calls, - truncation=override.truncation or self.truncation, - max_tokens=override.max_tokens or self.max_tokens, - ) + + changes = { + field.name: getattr(override, field.name) + for field in fields(self) + if getattr(override, field.name) is not None + } + return replace(self, **changes) diff --git a/src/cai/sdk/agents/models/openai_chatcompletions.py b/src/cai/sdk/agents/models/openai_chatcompletions.py index 7bcdd17f..c0c1ba18 100644 --- a/src/cai/sdk/agents/models/openai_chatcompletions.py +++ b/src/cai/sdk/agents/models/openai_chatcompletions.py @@ -617,6 +617,9 @@ class OpenAIChatCompletionsModel(Model): f"Using OLLAMA: {self.is_ollama}\n" ) + # Match the behavior of Responses where store is True when not given + store = model_settings.store if model_settings.store is not None else True + # Prepare kwargs for the API call kwargs = { "model": self.model, @@ -632,6 +635,7 @@ class OpenAIChatCompletionsModel(Model): "parallel_tool_calls": parallel_tool_calls, "stream": stream, "stream_options": {"include_usage": True} if stream else NOT_GIVEN, + "store": store, "extra_headers": _HEADERS, } diff --git a/src/cai/sdk/agents/models/openai_responses.py b/src/cai/sdk/agents/models/openai_responses.py index 3eea39ce..17803fda 100644 --- a/src/cai/sdk/agents/models/openai_responses.py +++ b/src/cai/sdk/agents/models/openai_responses.py @@ -208,7 +208,11 @@ class OpenAIResponsesModel(Model): list_input = ItemHelpers.input_to_new_input_list(input) parallel_tool_calls = ( - True if model_settings.parallel_tool_calls and tools and len(tools) > 0 else NOT_GIVEN + True + if model_settings.parallel_tool_calls and tools and len(tools) > 0 + else False + if model_settings.parallel_tool_calls is False + else NOT_GIVEN ) tool_choice = Converter.convert_tool_choice(model_settings.tool_choice) @@ -242,6 +246,7 @@ class OpenAIResponsesModel(Model): stream=stream, extra_headers=_HEADERS, text=response_format, + store=self._non_null_or_not_given(model_settings.store), ) def _get_client(self) -> AsyncOpenAI: diff --git a/src/cai/sdk/agents/run.py b/src/cai/sdk/agents/run.py index 53791027..0159822a 100644 --- a/src/cai/sdk/agents/run.py +++ b/src/cai/sdk/agents/run.py @@ -8,6 +8,7 @@ from typing import Any, cast from openai.types.responses import ResponseCompletedEvent from ._run_impl import ( + AgentToolUseTracker, NextStepFinalOutput, NextStepHandoff, NextStepRunAgain, @@ -37,6 +38,7 @@ from .models.openai_provider import OpenAIProvider from .result import RunResult, RunResultStreaming from .run_context import RunContextWrapper, TContext from .stream_events import AgentUpdatedStreamEvent, RawResponsesStreamEvent +from .tool import Tool from .tracing import Span, SpanError, agent_span, get_current_trace, trace from .tracing.span_data import AgentSpanData from .usage import Usage @@ -149,6 +151,8 @@ class Runner: if run_config is None: run_config = RunConfig() + tool_use_tracker = AgentToolUseTracker() + with TraceCtxManager( workflow_name=run_config.workflow_name, trace_id=run_config.trace_id, @@ -177,7 +181,6 @@ class Runner: # agent changes, or if the agent loop ends. if current_span is None: handoff_names = [h.agent_name for h in cls._get_handoffs(current_agent)] - tool_names = [t.name for t in current_agent.tools] if output_schema := cls._get_output_schema(current_agent): output_type_name = output_schema.output_type_name() else: @@ -186,11 +189,13 @@ class Runner: current_span = agent_span( name=current_agent.name, handoffs=handoff_names, - tools=tool_names, output_type=output_type_name, ) current_span.start(mark_as_current=True) + all_tools = await cls._get_all_tools(current_agent) + current_span.span_data.tools = [t.name for t in all_tools] + current_turn += 1 if current_turn > max_turns: _error_tracing.attach_error_to_span( @@ -217,23 +222,27 @@ class Runner: ), cls._run_single_turn( agent=current_agent, + all_tools=all_tools, original_input=original_input, generated_items=generated_items, hooks=hooks, context_wrapper=context_wrapper, run_config=run_config, should_run_agent_start_hooks=should_run_agent_start_hooks, + tool_use_tracker=tool_use_tracker, ), ) else: turn_result = await cls._run_single_turn( agent=current_agent, + all_tools=all_tools, original_input=original_input, generated_items=generated_items, hooks=hooks, context_wrapper=context_wrapper, run_config=run_config, should_run_agent_start_hooks=should_run_agent_start_hooks, + tool_use_tracker=tool_use_tracker, ) should_run_agent_start_hooks = False @@ -481,6 +490,7 @@ class Runner: current_agent = starting_agent current_turn = 0 should_run_agent_start_hooks = True + tool_use_tracker = AgentToolUseTracker() streamed_result._event_queue.put_nowait(AgentUpdatedStreamEvent(new_agent=current_agent)) @@ -493,9 +503,6 @@ class Runner: # agent changes, or if the agent loop ends. if current_span is None: handoff_names = [h.agent_name for h in cls._get_handoffs(current_agent)] - # print("current_agent: ", current_agent) - # print("current_agent.tools: ", current_agent.tools) - tool_names = [t.name for t in current_agent.tools] if output_schema := cls._get_output_schema(current_agent): output_type_name = output_schema.output_type_name() else: @@ -504,11 +511,13 @@ class Runner: current_span = agent_span( name=current_agent.name, handoffs=handoff_names, - tools=tool_names, output_type=output_type_name, ) current_span.start(mark_as_current=True) + all_tools = await cls._get_all_tools(current_agent) + tool_names = [t.name for t in all_tools] + current_span.span_data.tools = tool_names current_turn += 1 streamed_result.current_turn = current_turn @@ -543,6 +552,8 @@ class Runner: context_wrapper, run_config, should_run_agent_start_hooks, + tool_use_tracker, + all_tools, ) should_run_agent_start_hooks = False @@ -610,6 +621,8 @@ class Runner: context_wrapper: RunContextWrapper[TContext], run_config: RunConfig, should_run_agent_start_hooks: bool, + tool_use_tracker: AgentToolUseTracker, + all_tools: list[Tool], ) -> SingleStepResult: if should_run_agent_start_hooks: await asyncio.gather( @@ -629,9 +642,10 @@ class Runner: system_prompt = await agent.get_system_prompt(context_wrapper) handoffs = cls._get_handoffs(agent) - model = cls._get_model(agent, run_config) model_settings = agent.model_settings.resolve(run_config.model_settings) + model_settings = RunImpl.maybe_reset_tool_choice(agent, tool_use_tracker, model_settings) + final_response: ModelResponse | None = None input = ItemHelpers.input_to_new_input_list(streamed_result.input) @@ -642,7 +656,7 @@ class Runner: system_prompt, input, model_settings, - agent.tools, + all_tools, output_schema, handoffs, get_model_tracing_impl( @@ -679,10 +693,12 @@ class Runner: pre_step_items=streamed_result.new_items, new_response=final_response, output_schema=output_schema, + all_tools=all_tools, handoffs=handoffs, hooks=hooks, context_wrapper=context_wrapper, run_config=run_config, + tool_use_tracker=tool_use_tracker, ) RunImpl.stream_step_result_to_queue(single_step_result, streamed_result._event_queue) @@ -693,12 +709,14 @@ class Runner: cls, *, agent: Agent[TContext], + all_tools: list[Tool], original_input: str | list[TResponseInputItem], generated_items: list[RunItem], hooks: RunHooks[TContext], context_wrapper: RunContextWrapper[TContext], run_config: RunConfig, should_run_agent_start_hooks: bool, + tool_use_tracker: AgentToolUseTracker, ) -> SingleStepResult: # Ensure we run the hooks before anything else if should_run_agent_start_hooks: @@ -723,9 +741,11 @@ class Runner: system_prompt, input, output_schema, + all_tools, handoffs, context_wrapper, run_config, + tool_use_tracker, ) return await cls._get_single_step_result_from_response( @@ -734,10 +754,12 @@ class Runner: pre_step_items=generated_items, new_response=new_response, output_schema=output_schema, + all_tools=all_tools, handoffs=handoffs, hooks=hooks, context_wrapper=context_wrapper, run_config=run_config, + tool_use_tracker=tool_use_tracker, ) @classmethod @@ -745,6 +767,7 @@ class Runner: cls, *, agent: Agent[TContext], + all_tools: list[Tool], original_input: str | list[TResponseInputItem], pre_step_items: list[RunItem], new_response: ModelResponse, @@ -753,13 +776,18 @@ class Runner: hooks: RunHooks[TContext], context_wrapper: RunContextWrapper[TContext], run_config: RunConfig, + tool_use_tracker: AgentToolUseTracker, ) -> SingleStepResult: processed_response = RunImpl.process_model_response( agent=agent, + all_tools=all_tools, response=new_response, output_schema=output_schema, handoffs=handoffs, ) + + tool_use_tracker.add_tool_use(agent, processed_response.tools_used) + return await RunImpl.execute_tools_and_side_effects( agent=agent, original_input=original_input, @@ -855,17 +883,21 @@ class Runner: system_prompt: str | None, input: list[TResponseInputItem], output_schema: AgentOutputSchema | None, + all_tools: list[Tool], handoffs: list[Handoff], context_wrapper: RunContextWrapper[TContext], run_config: RunConfig, + tool_use_tracker: AgentToolUseTracker, ) -> ModelResponse: model = cls._get_model(agent, run_config) model_settings = agent.model_settings.resolve(run_config.model_settings) + model_settings = RunImpl.maybe_reset_tool_choice(agent, tool_use_tracker, model_settings) + new_response = await model.get_response( system_instructions=system_prompt, input=input, model_settings=model_settings, - tools=agent.tools, + tools=all_tools, output_schema=output_schema, handoffs=handoffs, tracing=get_model_tracing_impl( @@ -894,6 +926,10 @@ class Runner: handoffs.append(handoff(handoff_item)) return handoffs + @classmethod + async def _get_all_tools(cls, agent: Agent[Any]) -> list[Tool]: + return await agent.get_all_tools() + @classmethod def _get_model(cls, agent: Agent[Any], run_config: RunConfig) -> Model: if isinstance(run_config.model, Model): diff --git a/src/cai/sdk/agents/strict_schema.py b/src/cai/sdk/agents/strict_schema.py index 910ad85f..3f37660a 100644 --- a/src/cai/sdk/agents/strict_schema.py +++ b/src/cai/sdk/agents/strict_schema.py @@ -54,7 +54,7 @@ def _ensure_strict_json_schema( elif ( typ == "object" and "additionalProperties" in json_schema - and json_schema["additionalProperties"] is True + and json_schema["additionalProperties"] ): raise UserError( "additionalProperties should not be set for object types. This could be because " diff --git a/src/cai/sdk/agents/tracing/__init__.py b/src/cai/sdk/agents/tracing/__init__.py index dc7c7cfd..9df94426 100644 --- a/src/cai/sdk/agents/tracing/__init__.py +++ b/src/cai/sdk/agents/tracing/__init__.py @@ -9,6 +9,7 @@ from .create import ( get_current_trace, guardrail_span, handoff_span, + mcp_tools_span, response_span, speech_group_span, speech_span, @@ -25,6 +26,7 @@ from .span_data import ( GenerationSpanData, GuardrailSpanData, HandoffSpanData, + MCPListToolsSpanData, ResponseSpanData, SpanData, SpeechGroupSpanData, @@ -59,6 +61,7 @@ __all__ = [ "GenerationSpanData", "GuardrailSpanData", "HandoffSpanData", + "MCPListToolsSpanData", "ResponseSpanData", "SpeechGroupSpanData", "SpeechSpanData", @@ -69,6 +72,7 @@ __all__ = [ "speech_group_span", "speech_span", "transcription_span", + "mcp_tools_span", ] diff --git a/src/cai/sdk/agents/tracing/create.py b/src/cai/sdk/agents/tracing/create.py index af2f156f..b6fe4610 100644 --- a/src/cai/sdk/agents/tracing/create.py +++ b/src/cai/sdk/agents/tracing/create.py @@ -12,6 +12,7 @@ from .span_data import ( GenerationSpanData, GuardrailSpanData, HandoffSpanData, + MCPListToolsSpanData, ResponseSpanData, SpeechGroupSpanData, SpeechSpanData, @@ -424,3 +425,31 @@ def speech_group_span( parent=parent, disabled=disabled, ) + + +def mcp_tools_span( + server: str | None = None, + result: list[str] | None = None, + span_id: str | None = None, + parent: Trace | Span[Any] | None = None, + disabled: bool = False, +) -> Span[MCPListToolsSpanData]: + """Create a new MCP list tools span. The span will not be started automatically, you should + either do `with mcp_tools_span() ...` or call `span.start()` + `span.finish()` manually. + + Args: + server: The name of the MCP server. + result: The result of the MCP list tools call. + span_id: The ID of the span. Optional. If not provided, we will generate an ID. We + recommend using `util.gen_span_id()` to generate a span ID, to guarantee that IDs are + correctly formatted. + parent: The parent span or trace. If not provided, we will automatically use the current + trace/span as the parent. + disabled: If True, we will return a Span but the Span will not be recorded. + """ + return GLOBAL_TRACE_PROVIDER.create_span( + span_data=MCPListToolsSpanData(server=server, result=result), + span_id=span_id, + parent=parent, + disabled=disabled, + ) diff --git a/src/cai/sdk/agents/tracing/span_data.py b/src/cai/sdk/agents/tracing/span_data.py index 95e7fe0f..ed2a3f2d 100644 --- a/src/cai/sdk/agents/tracing/span_data.py +++ b/src/cai/sdk/agents/tracing/span_data.py @@ -49,12 +49,19 @@ class AgentSpanData(SpanData): class FunctionSpanData(SpanData): - __slots__ = ("name", "input", "output") + __slots__ = ("name", "input", "output", "mcp_data") - def __init__(self, name: str, input: str | None, output: Any | None): + def __init__( + self, + name: str, + input: str | None, + output: Any | None, + mcp_data: dict[str, Any] | None = None, + ): self.name = name self.input = input self.output = output + self.mcp_data = mcp_data @property def type(self) -> str: @@ -66,6 +73,7 @@ class FunctionSpanData(SpanData): "name": self.name, "input": self.input, "output": str(self.output) if self.output else None, + "mcp_data": self.mcp_data, } @@ -282,3 +290,25 @@ class SpeechGroupSpanData(SpanData): "type": self.type, "input": self.input, } + + +class MCPListToolsSpanData(SpanData): + __slots__ = ( + "server", + "result", + ) + + def __init__(self, server: str | None = None, result: list[str] | None = None): + self.server = server + self.result = result + + @property + def type(self) -> str: + return "mcp_tools" + + def export(self) -> dict[str, Any]: + return { + "type": self.type, + "server": self.server, + "result": self.result, + } diff --git a/src/cai/sdk/agents/voice/models/openai_stt.py b/src/cai/sdk/agents/voice/models/openai_stt.py index a5cf8acd..1ae4ea14 100644 --- a/src/cai/sdk/agents/voice/models/openai_stt.py +++ b/src/cai/sdk/agents/voice/models/openai_stt.py @@ -10,9 +10,8 @@ from typing import Any, cast from openai import AsyncOpenAI -from agents.exceptions import AgentsException - from ... import _debug +from ...exceptions import AgentsException from ...logger import logger from ...tracing import Span, SpanError, TranscriptionSpanData, transcription_span from ..exceptions import STTWebsocketConnectionError diff --git a/tests/fake_model.py b/tests/fake_model.py index f2ba6229..ecbb7583 100644 --- a/tests/fake_model.py +++ b/tests/fake_model.py @@ -1,6 +1,7 @@ from __future__ import annotations from collections.abc import AsyncIterator +from typing import Any from openai.types.responses import Response, ResponseCompletedEvent @@ -31,6 +32,7 @@ class FakeModel(Model): [initial_output] if initial_output else [] ) self.tracing_enabled = tracing_enabled + self.last_turn_args: dict[str, Any] = {} def set_next_output(self, output: list[TResponseOutputItem] | Exception): self.turn_outputs.append(output) @@ -53,6 +55,14 @@ class FakeModel(Model): handoffs: list[Handoff], tracing: ModelTracing, ) -> ModelResponse: + self.last_turn_args = { + "system_instructions": system_instructions, + "input": input, + "model_settings": model_settings, + "tools": tools, + "output_schema": output_schema, + } + with generation_span(disabled=not self.tracing_enabled) as span: output = self.get_next_output() diff --git a/tests/mcp/__init__.py b/tests/mcp/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/mcp/conftest.py b/tests/mcp/conftest.py new file mode 100644 index 00000000..80fd15ec --- /dev/null +++ b/tests/mcp/conftest.py @@ -0,0 +1,11 @@ +import os +import sys + + +# Skip MCP tests on Python 3.9 +def pytest_ignore_collect(collection_path, config): + if sys.version_info[:2] == (3, 9): + this_dir = os.path.dirname(__file__) + + if str(collection_path).startswith(this_dir): + return True diff --git a/tests/mcp/helpers.py b/tests/mcp/helpers.py new file mode 100644 index 00000000..8ff153c1 --- /dev/null +++ b/tests/mcp/helpers.py @@ -0,0 +1,58 @@ +import json +import shutil +from typing import Any + +from mcp import Tool as MCPTool +from mcp.types import CallToolResult, TextContent + +from agents.mcp import MCPServer + +tee = shutil.which("tee") or "" +assert tee, "tee not found" + + +# Added dummy stream classes for patching stdio_client to avoid real I/O during tests +class DummyStream: + async def send(self, msg): + pass + + async def receive(self): + raise Exception("Dummy receive not implemented") + + +class DummyStreamsContextManager: + async def __aenter__(self): + return (DummyStream(), DummyStream()) + + async def __aexit__(self, exc_type, exc_val, exc_tb): + pass + + +class FakeMCPServer(MCPServer): + def __init__(self, tools: list[MCPTool] | None = None): + self.tools: list[MCPTool] = tools or [] + self.tool_calls: list[str] = [] + self.tool_results: list[str] = [] + + def add_tool(self, name: str, input_schema: dict[str, Any]): + self.tools.append(MCPTool(name=name, inputSchema=input_schema)) + + async def connect(self): + pass + + async def cleanup(self): + pass + + async def list_tools(self): + return self.tools + + async def call_tool(self, tool_name: str, arguments: dict[str, Any] | None) -> CallToolResult: + self.tool_calls.append(tool_name) + self.tool_results.append(f"result_{tool_name}_{json.dumps(arguments)}") + return CallToolResult( + content=[TextContent(text=self.tool_results[-1], type="text")], + ) + + @property + def name(self) -> str: + return "fake_mcp_server" diff --git a/tests/mcp/test_caching.py b/tests/mcp/test_caching.py new file mode 100644 index 00000000..cac409e6 --- /dev/null +++ b/tests/mcp/test_caching.py @@ -0,0 +1,57 @@ +from unittest.mock import AsyncMock, patch + +import pytest +from mcp.types import ListToolsResult, Tool as MCPTool + +from agents.mcp import MCPServerStdio + +from .helpers import DummyStreamsContextManager, tee + + +@pytest.mark.asyncio +@patch("mcp.client.stdio.stdio_client", return_value=DummyStreamsContextManager()) +@patch("mcp.client.session.ClientSession.initialize", new_callable=AsyncMock, return_value=None) +@patch("mcp.client.session.ClientSession.list_tools") +async def test_server_caching_works( + mock_list_tools: AsyncMock, mock_initialize: AsyncMock, mock_stdio_client +): + """Test that if we turn caching on, the list of tools is cached and not fetched from the server + on each call to `list_tools()`. + """ + server = MCPServerStdio( + params={ + "command": tee, + }, + cache_tools_list=True, + ) + + tools = [ + MCPTool(name="tool1", inputSchema={}), + MCPTool(name="tool2", inputSchema={}), + ] + + mock_list_tools.return_value = ListToolsResult(tools=tools) + + async with server: + # Call list_tools() multiple times + tools = await server.list_tools() + assert tools == tools + + assert mock_list_tools.call_count == 1, "list_tools() should have been called once" + + # Call list_tools() again, should return the cached value + tools = await server.list_tools() + assert tools == tools + + assert mock_list_tools.call_count == 1, "list_tools() should not have been called again" + + # Invalidate the cache and call list_tools() again + server.invalidate_tools_cache() + tools = await server.list_tools() + assert tools == tools + + assert mock_list_tools.call_count == 2, "list_tools() should be called again" + + # Without invalidating the cache, calling list_tools() again should return the cached value + tools = await server.list_tools() + assert tools == tools diff --git a/tests/mcp/test_connect_disconnect.py b/tests/mcp/test_connect_disconnect.py new file mode 100644 index 00000000..b0013039 --- /dev/null +++ b/tests/mcp/test_connect_disconnect.py @@ -0,0 +1,69 @@ +from unittest.mock import AsyncMock, patch + +import pytest +from mcp.types import ListToolsResult, Tool as MCPTool + +from agents.mcp import MCPServerStdio + +from .helpers import DummyStreamsContextManager, tee + + +@pytest.mark.asyncio +@patch("mcp.client.stdio.stdio_client", return_value=DummyStreamsContextManager()) +@patch("mcp.client.session.ClientSession.initialize", new_callable=AsyncMock, return_value=None) +@patch("mcp.client.session.ClientSession.list_tools") +async def test_async_ctx_manager_works( + mock_list_tools: AsyncMock, mock_initialize: AsyncMock, mock_stdio_client +): + """Test that the async context manager works.""" + server = MCPServerStdio( + params={ + "command": tee, + }, + cache_tools_list=True, + ) + + tools = [ + MCPTool(name="tool1", inputSchema={}), + MCPTool(name="tool2", inputSchema={}), + ] + + mock_list_tools.return_value = ListToolsResult(tools=tools) + + assert server.session is None, "Server should not be connected" + + async with server: + assert server.session is not None, "Server should be connected" + + assert server.session is None, "Server should be disconnected" + + +@pytest.mark.asyncio +@patch("mcp.client.stdio.stdio_client", return_value=DummyStreamsContextManager()) +@patch("mcp.client.session.ClientSession.initialize", new_callable=AsyncMock, return_value=None) +@patch("mcp.client.session.ClientSession.list_tools") +async def test_manual_connect_disconnect_works( + mock_list_tools: AsyncMock, mock_initialize: AsyncMock, mock_stdio_client +): + """Test that the async context manager works.""" + server = MCPServerStdio( + params={ + "command": tee, + }, + cache_tools_list=True, + ) + + tools = [ + MCPTool(name="tool1", inputSchema={}), + MCPTool(name="tool2", inputSchema={}), + ] + + mock_list_tools.return_value = ListToolsResult(tools=tools) + + assert server.session is None, "Server should not be connected" + + await server.connect() + assert server.session is not None, "Server should be connected" + + await server.cleanup() + assert server.session is None, "Server should be disconnected" diff --git a/tests/mcp/test_mcp_tracing.py b/tests/mcp/test_mcp_tracing.py new file mode 100644 index 00000000..b71954b5 --- /dev/null +++ b/tests/mcp/test_mcp_tracing.py @@ -0,0 +1,198 @@ +import pytest +from inline_snapshot import snapshot + +from agents import Agent, Runner + +from ..fake_model import FakeModel +from ..test_responses import get_function_tool, get_function_tool_call, get_text_message +from ..testing_processor import SPAN_PROCESSOR_TESTING, fetch_normalized_spans +from .helpers import FakeMCPServer + + +@pytest.mark.asyncio +async def test_mcp_tracing(): + model = FakeModel() + server = FakeMCPServer() + server.add_tool("test_tool_1", {}) + agent = Agent( + name="test", + model=model, + mcp_servers=[server], + tools=[get_function_tool("non_mcp_tool", "tool_result")], + ) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_1", "")], + # Second turn: text message + [get_text_message("done")], + ] + ) + + # First run: should list MCP tools before first and second steps + x = Runner.run_streamed(agent, input="first_test") + async for _ in x.stream_events(): + pass + + assert x.final_output == "done" + spans = fetch_normalized_spans() + + # Should have a single tool listing, and the function span should have MCP data + assert spans == snapshot( + [ + { + "workflow_name": "Agent workflow", + "children": [ + { + "type": "agent", + "data": { + "name": "test", + "handoffs": [], + "tools": ["test_tool_1", "non_mcp_tool"], + "output_type": "str", + }, + "children": [ + { + "type": "mcp_tools", + "data": {"server": "fake_mcp_server", "result": ["test_tool_1"]}, + }, + { + "type": "function", + "data": { + "name": "test_tool_1", + "input": "", + "output": '{"type":"text","text":"result_test_tool_1_{}","annotations":null}', # noqa: E501 + "mcp_data": {"server": "fake_mcp_server"}, + }, + }, + ], + } + ], + } + ] + ) + + server.add_tool("test_tool_2", {}) + + SPAN_PROCESSOR_TESTING.clear() + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [ + get_text_message("a_message"), + get_function_tool_call("non_mcp_tool", ""), + get_function_tool_call("test_tool_2", ""), + ], + # Second turn: text message + [get_text_message("done")], + ] + ) + + await Runner.run(agent, input="second_test") + spans = fetch_normalized_spans() + + # Should have a single tool listing, and the function span should have MCP data, and the non-mcp + # tool function span should not have MCP data + assert spans == snapshot( + [ + { + "workflow_name": "Agent workflow", + "children": [ + { + "type": "agent", + "data": { + "name": "test", + "handoffs": [], + "tools": ["test_tool_1", "test_tool_2", "non_mcp_tool"], + "output_type": "str", + }, + "children": [ + { + "type": "mcp_tools", + "data": { + "server": "fake_mcp_server", + "result": ["test_tool_1", "test_tool_2"], + }, + }, + { + "type": "function", + "data": { + "name": "non_mcp_tool", + "input": "", + "output": "tool_result", + }, + }, + { + "type": "function", + "data": { + "name": "test_tool_2", + "input": "", + "output": '{"type":"text","text":"result_test_tool_2_{}","annotations":null}', # noqa: E501 + "mcp_data": {"server": "fake_mcp_server"}, + }, + }, + ], + } + ], + } + ] + ) + + SPAN_PROCESSOR_TESTING.clear() + + # Add more tools to the server + server.add_tool("test_tool_3", {}) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_3", "")], + # Second turn: text message + [get_text_message("done")], + ] + ) + + await Runner.run(agent, input="third_test") + + spans = fetch_normalized_spans() + + # Should have a single tool listing, and the function span should have MCP data, and the non-mcp + # tool function span should not have MCP data + assert spans == snapshot( + [ + { + "workflow_name": "Agent workflow", + "children": [ + { + "type": "agent", + "data": { + "name": "test", + "handoffs": [], + "tools": ["test_tool_1", "test_tool_2", "test_tool_3", "non_mcp_tool"], + "output_type": "str", + }, + "children": [ + { + "type": "mcp_tools", + "data": { + "server": "fake_mcp_server", + "result": ["test_tool_1", "test_tool_2", "test_tool_3"], + }, + }, + { + "type": "function", + "data": { + "name": "test_tool_3", + "input": "", + "output": '{"type":"text","text":"result_test_tool_3_{}","annotations":null}', # noqa: E501 + "mcp_data": {"server": "fake_mcp_server"}, + }, + }, + ], + } + ], + } + ] + ) diff --git a/tests/mcp/test_mcp_util.py b/tests/mcp/test_mcp_util.py new file mode 100644 index 00000000..345df996 --- /dev/null +++ b/tests/mcp/test_mcp_util.py @@ -0,0 +1,109 @@ +import logging +from typing import Any + +import pytest +from mcp.types import Tool as MCPTool +from pydantic import BaseModel + +from agents import FunctionTool, RunContextWrapper +from agents.exceptions import AgentsException, ModelBehaviorError +from agents.mcp import MCPServer, MCPUtil + +from .helpers import FakeMCPServer + + +class Foo(BaseModel): + bar: str + baz: int + + +class Bar(BaseModel): + qux: str + + +@pytest.mark.asyncio +async def test_get_all_function_tools(): + """Test that the get_all_function_tools function returns all function tools from a list of MCP + servers. + """ + names = ["test_tool_1", "test_tool_2", "test_tool_3", "test_tool_4", "test_tool_5"] + schemas = [ + {}, + {}, + {}, + Foo.model_json_schema(), + Bar.model_json_schema(), + ] + + server1 = FakeMCPServer() + server1.add_tool(names[0], schemas[0]) + server1.add_tool(names[1], schemas[1]) + + server2 = FakeMCPServer() + server2.add_tool(names[2], schemas[2]) + server2.add_tool(names[3], schemas[3]) + + server3 = FakeMCPServer() + server3.add_tool(names[4], schemas[4]) + + servers: list[MCPServer] = [server1, server2, server3] + tools = await MCPUtil.get_all_function_tools(servers) + assert len(tools) == 5 + assert all(tool.name in names for tool in tools) + + for idx, tool in enumerate(tools): + assert isinstance(tool, FunctionTool) + assert tool.params_json_schema == schemas[idx] + assert tool.name == names[idx] + + +@pytest.mark.asyncio +async def test_invoke_mcp_tool(): + """Test that the invoke_mcp_tool function invokes an MCP tool and returns the result.""" + server = FakeMCPServer() + server.add_tool("test_tool_1", {}) + + ctx = RunContextWrapper(context=None) + tool = MCPTool(name="test_tool_1", inputSchema={}) + + await MCPUtil.invoke_mcp_tool(server, tool, ctx, "") + # Just making sure it doesn't crash + + +@pytest.mark.asyncio +async def test_mcp_invoke_bad_json_errors(caplog: pytest.LogCaptureFixture): + caplog.set_level(logging.DEBUG) + + """Test that bad JSON input errors are logged and re-raised.""" + server = FakeMCPServer() + server.add_tool("test_tool_1", {}) + + ctx = RunContextWrapper(context=None) + tool = MCPTool(name="test_tool_1", inputSchema={}) + + with pytest.raises(ModelBehaviorError): + await MCPUtil.invoke_mcp_tool(server, tool, ctx, "not_json") + + assert "Invalid JSON input for tool test_tool_1" in caplog.text + + +class CrashingFakeMCPServer(FakeMCPServer): + async def call_tool(self, tool_name: str, arguments: dict[str, Any] | None): + raise Exception("Crash!") + + +@pytest.mark.asyncio +async def test_mcp_invocation_crash_causes_error(caplog: pytest.LogCaptureFixture): + caplog.set_level(logging.DEBUG) + + """Test that bad JSON input errors are logged and re-raised.""" + server = CrashingFakeMCPServer() + server.add_tool("test_tool_1", {}) + + ctx = RunContextWrapper(context=None) + tool = MCPTool(name="test_tool_1", inputSchema={}) + + with pytest.raises(AgentsException): + await MCPUtil.invoke_mcp_tool(server, tool, ctx, "") + + assert "Error invoking MCP tool test_tool_1" in caplog.text diff --git a/tests/mcp/test_runner_calls_mcp.py b/tests/mcp/test_runner_calls_mcp.py new file mode 100644 index 00000000..3319c097 --- /dev/null +++ b/tests/mcp/test_runner_calls_mcp.py @@ -0,0 +1,197 @@ +import json + +import pytest +from pydantic import BaseModel + +from agents import Agent, ModelBehaviorError, Runner, UserError + +from ..fake_model import FakeModel +from ..test_responses import get_function_tool_call, get_text_message +from .helpers import FakeMCPServer + + +@pytest.mark.asyncio +@pytest.mark.parametrize("streaming", [False, True]) +async def test_runner_calls_mcp_tool(streaming: bool): + """Test that the runner calls an MCP tool when the model produces a tool call.""" + server = FakeMCPServer() + server.add_tool("test_tool_1", {}) + server.add_tool("test_tool_2", {}) + server.add_tool("test_tool_3", {}) + model = FakeModel() + agent = Agent( + name="test", + model=model, + mcp_servers=[server], + ) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_2", "")], + # Second turn: text message + [get_text_message("done")], + ] + ) + + if streaming: + result = Runner.run_streamed(agent, input="user_message") + async for _ in result.stream_events(): + pass + else: + await Runner.run(agent, input="user_message") + + assert server.tool_calls == ["test_tool_2"] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("streaming", [False, True]) +async def test_runner_asserts_when_mcp_tool_not_found(streaming: bool): + """Test that the runner asserts when an MCP tool is not found.""" + server = FakeMCPServer() + server.add_tool("test_tool_1", {}) + server.add_tool("test_tool_2", {}) + server.add_tool("test_tool_3", {}) + model = FakeModel() + agent = Agent( + name="test", + model=model, + mcp_servers=[server], + ) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_doesnt_exist", "")], + # Second turn: text message + [get_text_message("done")], + ] + ) + + with pytest.raises(ModelBehaviorError): + if streaming: + result = Runner.run_streamed(agent, input="user_message") + async for _ in result.stream_events(): + pass + else: + await Runner.run(agent, input="user_message") + + +@pytest.mark.asyncio +@pytest.mark.parametrize("streaming", [False, True]) +async def test_runner_works_with_multiple_mcp_servers(streaming: bool): + """Test that the runner works with multiple MCP servers.""" + server1 = FakeMCPServer() + server1.add_tool("test_tool_1", {}) + + server2 = FakeMCPServer() + server2.add_tool("test_tool_2", {}) + server2.add_tool("test_tool_3", {}) + + model = FakeModel() + agent = Agent( + name="test", + model=model, + mcp_servers=[server1, server2], + ) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_2", "")], + # Second turn: text message + [get_text_message("done")], + ] + ) + + if streaming: + result = Runner.run_streamed(agent, input="user_message") + async for _ in result.stream_events(): + pass + else: + await Runner.run(agent, input="user_message") + + assert server1.tool_calls == [] + assert server2.tool_calls == ["test_tool_2"] + + +@pytest.mark.asyncio +@pytest.mark.parametrize("streaming", [False, True]) +async def test_runner_errors_when_mcp_tools_clash(streaming: bool): + """Test that the runner errors when multiple servers have the same tool name.""" + server1 = FakeMCPServer() + server1.add_tool("test_tool_1", {}) + server1.add_tool("test_tool_2", {}) + + server2 = FakeMCPServer() + server2.add_tool("test_tool_2", {}) + server2.add_tool("test_tool_3", {}) + + model = FakeModel() + agent = Agent( + name="test", + model=model, + mcp_servers=[server1, server2], + ) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_3", "")], + # Second turn: text message + [get_text_message("done")], + ] + ) + + with pytest.raises(UserError): + if streaming: + result = Runner.run_streamed(agent, input="user_message") + async for _ in result.stream_events(): + pass + else: + await Runner.run(agent, input="user_message") + + +class Foo(BaseModel): + bar: str + baz: int + + +@pytest.mark.asyncio +@pytest.mark.parametrize("streaming", [False, True]) +async def test_runner_calls_mcp_tool_with_args(streaming: bool): + """Test that the runner calls an MCP tool when the model produces a tool call.""" + server = FakeMCPServer() + await server.connect() + server.add_tool("test_tool_1", {}) + server.add_tool("test_tool_2", Foo.model_json_schema()) + server.add_tool("test_tool_3", {}) + model = FakeModel() + agent = Agent( + name="test", + model=model, + mcp_servers=[server], + ) + + json_args = json.dumps(Foo(bar="baz", baz=1).model_dump()) + + model.add_multiple_turn_outputs( + [ + # First turn: a message and tool call + [get_text_message("a_message"), get_function_tool_call("test_tool_2", json_args)], + # Second turn: text message + [get_text_message("done")], + ] + ) + + if streaming: + result = Runner.run_streamed(agent, input="user_message") + async for _ in result.stream_events(): + pass + else: + await Runner.run(agent, input="user_message") + + assert server.tool_calls == ["test_tool_2"] + assert server.tool_results == [f"result_test_tool_2_{json_args}"] + + await server.cleanup() diff --git a/tests/mcp/test_server_errors.py b/tests/mcp/test_server_errors.py new file mode 100644 index 00000000..bdca7ce6 --- /dev/null +++ b/tests/mcp/test_server_errors.py @@ -0,0 +1,42 @@ +import pytest + +from agents.exceptions import UserError +from agents.mcp.server import _MCPServerWithClientSession + + +class CrashingClientSessionServer(_MCPServerWithClientSession): + def __init__(self): + super().__init__(cache_tools_list=False) + self.cleanup_called = False + + def create_streams(self): + raise ValueError("Crash!") + + async def cleanup(self): + self.cleanup_called = True + await super().cleanup() + + @property + def name(self) -> str: + return "crashing_client_session_server" + + +@pytest.mark.asyncio +async def test_server_errors_cause_error_and_cleanup_called(): + server = CrashingClientSessionServer() + + with pytest.raises(ValueError): + await server.connect() + + assert server.cleanup_called + + +@pytest.mark.asyncio +async def test_not_calling_connect_causes_error(): + server = CrashingClientSessionServer() + + with pytest.raises(UserError): + await server.list_tools() + + with pytest.raises(UserError): + await server.call_tool("foo", {}) diff --git a/tests/test_agent_runner.py b/tests/test_agent_runner.py index e8e060fd..ce0c5804 100644 --- a/tests/test_agent_runner.py +++ b/tests/test_agent_runner.py @@ -14,8 +14,10 @@ from agents import ( InputGuardrail, InputGuardrailTripwireTriggered, ModelBehaviorError, + ModelSettings, OutputGuardrail, OutputGuardrailTripwireTriggered, + RunConfig, RunContextWrapper, Runner, UserError, @@ -634,3 +636,31 @@ async def test_tool_use_behavior_custom_function(): assert len(result.raw_responses) == 2, "should have two model responses" assert result.final_output == "the_final_output", "should have used the custom function" + + +@pytest.mark.asyncio +async def test_model_settings_override(): + model = FakeModel() + agent = Agent( + name="test", + model=model, + model_settings=ModelSettings(temperature=1.0, max_tokens=1000) + ) + + model.add_multiple_turn_outputs( + [ + [ + get_text_message("a_message"), + ], + ] + ) + + await Runner.run( + agent, + input="user_message", + run_config=RunConfig(model_settings=ModelSettings(0.5)), + ) + + # temperature is overridden by Runner.run, but max_tokens is not + assert model.last_turn_args["model_settings"].temperature == 0.5 + assert model.last_turn_args["model_settings"].max_tokens == 1000 diff --git a/tests/test_function_schema.py b/tests/test_function_schema.py index 2407ab03..ef1e9c22 100644 --- a/tests/test_function_schema.py +++ b/tests/test_function_schema.py @@ -1,3 +1,4 @@ +from collections.abc import Mapping from enum import Enum from typing import Any, Literal @@ -421,10 +422,20 @@ def test_var_keyword_dict_annotation(): def func(**kwargs: dict[str, int]): return kwargs - fs = function_schema(func, use_docstring_info=False) + fs = function_schema(func, use_docstring_info=False, strict_json_schema=False) properties = fs.params_json_schema.get("properties", {}) # The name of the field is "kwargs", and it's a JSON object i.e. a dict. assert properties.get("kwargs").get("type") == "object" # The values in the dict are integers. assert properties.get("kwargs").get("additionalProperties").get("type") == "integer" + + +def test_schema_with_mapping_raises_strict_mode_error(): + """A mapping type is not allowed in strict mode. Same for dicts. Ensure we raise a UserError.""" + + def func_with_mapping(test_one: Mapping[str, int]) -> str: + return "foo" + + with pytest.raises(UserError): + function_schema(func_with_mapping) diff --git a/tests/test_openai_chatcompletions.py b/tests/test_openai_chatcompletions.py index 95216476..9a53e2b7 100644 --- a/tests/test_openai_chatcompletions.py +++ b/tests/test_openai_chatcompletions.py @@ -226,6 +226,7 @@ async def test_fetch_response_non_stream(monkeypatch) -> None: # Ensure expected args were passed through to OpenAI client. kwargs = completions.kwargs assert kwargs["stream"] is False + assert kwargs["store"] is True assert kwargs["model"] == "gpt-4" assert kwargs["messages"][0]["role"] == "system" assert kwargs["messages"][0]["content"] == "sys" @@ -279,6 +280,7 @@ async def test_fetch_response_stream(monkeypatch) -> None: ) # Check OpenAI client was called for streaming assert completions.kwargs["stream"] is True + assert completions.kwargs["store"] is True assert completions.kwargs["stream_options"] == {"include_usage": True} # Response is a proper openai Response assert isinstance(response, Response) diff --git a/tests/test_run_step_execution.py b/tests/test_run_step_execution.py index 2d581bf6..16c62c84 100644 --- a/tests/test_run_step_execution.py +++ b/tests/test_run_step_execution.py @@ -290,6 +290,7 @@ async def get_execute_result( processed_response = RunImpl.process_model_response( agent=agent, + all_tools=await agent.get_all_tools(), response=response, output_schema=output_schema, handoffs=handoffs, diff --git a/tests/test_run_step_processing.py b/tests/test_run_step_processing.py index 24f9e8e3..2a6634ac 100644 --- a/tests/test_run_step_processing.py +++ b/tests/test_run_step_processing.py @@ -43,7 +43,11 @@ def test_empty_response(): ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=[], ) assert not result.handoffs assert not result.functions @@ -57,13 +61,14 @@ def test_no_tool_calls(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, response=response, output_schema=None, handoffs=[], all_tools=[] ) assert not result.handoffs assert not result.functions -def test_single_tool_call(): +@pytest.mark.asyncio +async def test_single_tool_call(): agent = Agent(name="test", tools=[get_function_tool(name="test")]) response = ModelResponse( output=[ @@ -74,7 +79,11 @@ def test_single_tool_call(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent.get_all_tools(), ) assert not result.handoffs assert result.functions and len(result.functions) == 1 @@ -84,7 +93,8 @@ def test_single_tool_call(): assert func.tool_call.arguments == "" -def test_missing_tool_call_raises_error(): +@pytest.mark.asyncio +async def test_missing_tool_call_raises_error(): agent = Agent(name="test", tools=[get_function_tool(name="test")]) response = ModelResponse( output=[ @@ -97,11 +107,16 @@ def test_missing_tool_call_raises_error(): with pytest.raises(ModelBehaviorError): RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent.get_all_tools(), ) -def test_multiple_tool_calls(): +@pytest.mark.asyncio +async def test_multiple_tool_calls(): agent = Agent( name="test", tools=[ @@ -121,7 +136,11 @@ def test_multiple_tool_calls(): ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent.get_all_tools(), ) assert not result.handoffs assert result.functions and len(result.functions) == 2 @@ -146,7 +165,11 @@ async def test_handoffs_parsed_correctly(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=agent_3, response=response, output_schema=None, handoffs=[] + agent=agent_3, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent_3.get_all_tools(), ) assert not result.handoffs, "Shouldn't have a handoff here" @@ -160,6 +183,7 @@ async def test_handoffs_parsed_correctly(): response=response, output_schema=None, handoffs=Runner._get_handoffs(agent_3), + all_tools=await agent_3.get_all_tools(), ) assert len(result.handoffs) == 1, "Should have a handoff here" handoff = result.handoffs[0] @@ -189,10 +213,12 @@ async def test_missing_handoff_fails(): response=response, output_schema=None, handoffs=Runner._get_handoffs(agent_3), + all_tools=await agent_3.get_all_tools(), ) -def test_multiple_handoffs_doesnt_error(): +@pytest.mark.asyncio +async def test_multiple_handoffs_doesnt_error(): agent_1 = Agent(name="test_1") agent_2 = Agent(name="test_2") agent_3 = Agent(name="test_3", handoffs=[agent_1, agent_2]) @@ -210,6 +236,7 @@ def test_multiple_handoffs_doesnt_error(): response=response, output_schema=None, handoffs=Runner._get_handoffs(agent_3), + all_tools=await agent_3.get_all_tools(), ) assert len(result.handoffs) == 2, "Should have multiple handoffs here" @@ -218,7 +245,8 @@ class Foo(BaseModel): bar: str -def test_final_output_parsed_correctly(): +@pytest.mark.asyncio +async def test_final_output_parsed_correctly(): agent = Agent(name="test", output_type=Foo) response = ModelResponse( output=[ @@ -234,10 +262,12 @@ def test_final_output_parsed_correctly(): response=response, output_schema=Runner._get_output_schema(agent), handoffs=[], + all_tools=await agent.get_all_tools(), ) -def test_file_search_tool_call_parsed_correctly(): +@pytest.mark.asyncio +async def test_file_search_tool_call_parsed_correctly(): # Ensure that a ResponseFileSearchToolCall output is parsed into a ToolCallItem and that no tool # runs are scheduled. @@ -254,7 +284,11 @@ def test_file_search_tool_call_parsed_correctly(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent.get_all_tools(), ) # The final item should be a ToolCallItem for the file search call assert any( @@ -265,7 +299,8 @@ def test_file_search_tool_call_parsed_correctly(): assert not result.handoffs -def test_function_web_search_tool_call_parsed_correctly(): +@pytest.mark.asyncio +async def test_function_web_search_tool_call_parsed_correctly(): agent = Agent(name="test") web_search_call = ResponseFunctionWebSearch(id="w1", status="completed", type="web_search_call") response = ModelResponse( @@ -274,7 +309,11 @@ def test_function_web_search_tool_call_parsed_correctly(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent.get_all_tools(), ) assert any( isinstance(item, ToolCallItem) and item.raw_item is web_search_call @@ -284,7 +323,8 @@ def test_function_web_search_tool_call_parsed_correctly(): assert not result.handoffs -def test_reasoning_item_parsed_correctly(): +@pytest.mark.asyncio +async def test_reasoning_item_parsed_correctly(): # Verify that a Reasoning output item is converted into a ReasoningItem. reasoning = ResponseReasoningItem( @@ -296,7 +336,11 @@ def test_reasoning_item_parsed_correctly(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=Agent(name="test"), response=response, output_schema=None, handoffs=[] + agent=Agent(name="test"), + response=response, + output_schema=None, + handoffs=[], + all_tools=await Agent(name="test").get_all_tools(), ) assert any( isinstance(item, ReasoningItem) and item.raw_item is reasoning for item in result.new_items @@ -342,7 +386,8 @@ class DummyComputer(Computer): return None # pragma: no cover -def test_computer_tool_call_without_computer_tool_raises_error(): +@pytest.mark.asyncio +async def test_computer_tool_call_without_computer_tool_raises_error(): # If the agent has no ComputerTool in its tools, process_model_response should raise a # ModelBehaviorError when encountering a ResponseComputerToolCall. computer_call = ResponseComputerToolCall( @@ -360,11 +405,16 @@ def test_computer_tool_call_without_computer_tool_raises_error(): ) with pytest.raises(ModelBehaviorError): RunImpl.process_model_response( - agent=Agent(name="test"), response=response, output_schema=None, handoffs=[] + agent=Agent(name="test"), + response=response, + output_schema=None, + handoffs=[], + all_tools=await Agent(name="test").get_all_tools(), ) -def test_computer_tool_call_with_computer_tool_parsed_correctly(): +@pytest.mark.asyncio +async def test_computer_tool_call_with_computer_tool_parsed_correctly(): # If the agent contains a ComputerTool, ensure that a ResponseComputerToolCall is parsed into a # ToolCallItem and scheduled to run in computer_actions. dummy_computer = DummyComputer() @@ -383,7 +433,11 @@ def test_computer_tool_call_with_computer_tool_parsed_correctly(): referenceable_id=None, ) result = RunImpl.process_model_response( - agent=agent, response=response, output_schema=None, handoffs=[] + agent=agent, + response=response, + output_schema=None, + handoffs=[], + all_tools=await agent.get_all_tools(), ) assert any( isinstance(item, ToolCallItem) and item.raw_item is computer_call @@ -392,7 +446,8 @@ def test_computer_tool_call_with_computer_tool_parsed_correctly(): assert result.computer_actions and result.computer_actions[0].tool_call == computer_call -def test_tool_and_handoff_parsed_correctly(): +@pytest.mark.asyncio +async def test_tool_and_handoff_parsed_correctly(): agent_1 = Agent(name="test_1") agent_2 = Agent(name="test_2") agent_3 = Agent( @@ -413,6 +468,7 @@ def test_tool_and_handoff_parsed_correctly(): response=response, output_schema=None, handoffs=Runner._get_handoffs(agent_3), + all_tools=await agent_3.get_all_tools(), ) assert result.functions and len(result.functions) == 1 assert len(result.handoffs) == 1, "Should have a handoff here" diff --git a/tests/test_tool_choice_reset.py b/tests/test_tool_choice_reset.py new file mode 100644 index 00000000..f95117fd --- /dev/null +++ b/tests/test_tool_choice_reset.py @@ -0,0 +1,210 @@ +import pytest + +from agents import Agent, ModelSettings, Runner +from agents._run_impl import AgentToolUseTracker, RunImpl + +from .fake_model import FakeModel +from .test_responses import get_function_tool, get_function_tool_call, get_text_message + + +class TestToolChoiceReset: + def test_should_reset_tool_choice_direct(self): + """ + Test the _should_reset_tool_choice method directly with various inputs + to ensure it correctly identifies cases where reset is needed. + """ + agent = Agent(name="test_agent") + + # Case 1: Empty tool use tracker should not change the "None" tool choice + model_settings = ModelSettings(tool_choice=None) + tracker = AgentToolUseTracker() + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert new_settings.tool_choice == model_settings.tool_choice + + # Case 2: Empty tool use tracker should not change the "auto" tool choice + model_settings = ModelSettings(tool_choice="auto") + tracker = AgentToolUseTracker() + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert model_settings.tool_choice == new_settings.tool_choice + + # Case 3: Empty tool use tracker should not change the "required" tool choice + model_settings = ModelSettings(tool_choice="required") + tracker = AgentToolUseTracker() + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert model_settings.tool_choice == new_settings.tool_choice + + # Case 4: tool_choice = "required" with one tool should reset + model_settings = ModelSettings(tool_choice="required") + tracker = AgentToolUseTracker() + tracker.add_tool_use(agent, ["tool1"]) + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert new_settings.tool_choice is None + + # Case 5: tool_choice = "required" with multiple tools should reset + model_settings = ModelSettings(tool_choice="required") + tracker = AgentToolUseTracker() + tracker.add_tool_use(agent, ["tool1", "tool2"]) + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert new_settings.tool_choice is None + + # Case 6: Tool usage on a different agent should not affect the tool choice + model_settings = ModelSettings(tool_choice="foo_bar") + tracker = AgentToolUseTracker() + tracker.add_tool_use(Agent(name="other_agent"), ["foo_bar", "baz"]) + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert new_settings.tool_choice == model_settings.tool_choice + + # Case 7: tool_choice = "foo_bar" with multiple tools should reset + model_settings = ModelSettings(tool_choice="foo_bar") + tracker = AgentToolUseTracker() + tracker.add_tool_use(agent, ["foo_bar", "baz"]) + new_settings = RunImpl.maybe_reset_tool_choice(agent, tracker, model_settings) + assert new_settings.tool_choice is None + + @pytest.mark.asyncio + async def test_required_tool_choice_with_multiple_runs(self): + """ + Test scenario 1: When multiple runs are executed with tool_choice="required", ensure each + run works correctly and doesn't get stuck in an infinite loop. Also verify that tool_choice + remains "required" between runs. + """ + # Set up our fake model with responses for two runs + fake_model = FakeModel() + fake_model.add_multiple_turn_outputs( + [[get_text_message("First run response")], [get_text_message("Second run response")]] + ) + + # Create agent with a custom tool and tool_choice="required" + custom_tool = get_function_tool("custom_tool") + agent = Agent( + name="test_agent", + model=fake_model, + tools=[custom_tool], + model_settings=ModelSettings(tool_choice="required"), + ) + + # First run should work correctly and preserve tool_choice + result1 = await Runner.run(agent, "first run") + assert result1.final_output == "First run response" + assert fake_model.last_turn_args["model_settings"].tool_choice == "required", ( + "tool_choice should stay required" + ) + + # Second run should also work correctly with tool_choice still required + result2 = await Runner.run(agent, "second run") + assert result2.final_output == "Second run response" + assert fake_model.last_turn_args["model_settings"].tool_choice == "required", ( + "tool_choice should stay required" + ) + + @pytest.mark.asyncio + async def test_required_with_stop_at_tool_name(self): + """ + Test scenario 2: When using required tool_choice with stop_at_tool_names behavior, ensure + it correctly stops at the specified tool + """ + # Set up fake model to return a tool call for second_tool + fake_model = FakeModel() + fake_model.set_next_output([get_function_tool_call("second_tool", "{}")]) + + # Create agent with two tools and tool_choice="required" and stop_at_tool behavior + first_tool = get_function_tool("first_tool", return_value="first tool result") + second_tool = get_function_tool("second_tool", return_value="second tool result") + + agent = Agent( + name="test_agent", + model=fake_model, + tools=[first_tool, second_tool], + model_settings=ModelSettings(tool_choice="required"), + tool_use_behavior={"stop_at_tool_names": ["second_tool"]}, + ) + + # Run should stop after using second_tool + result = await Runner.run(agent, "run test") + assert result.final_output == "second tool result" + + @pytest.mark.asyncio + async def test_specific_tool_choice(self): + """ + Test scenario 3: When using a specific tool choice name, ensure it doesn't cause infinite + loops. + """ + # Set up fake model to return a text message + fake_model = FakeModel() + fake_model.set_next_output([get_text_message("Test message")]) + + # Create agent with specific tool_choice + tool1 = get_function_tool("tool1") + tool2 = get_function_tool("tool2") + tool3 = get_function_tool("tool3") + + agent = Agent( + name="test_agent", + model=fake_model, + tools=[tool1, tool2, tool3], + model_settings=ModelSettings(tool_choice="tool1"), # Specific tool + ) + + # Run should complete without infinite loops + result = await Runner.run(agent, "first run") + assert result.final_output == "Test message" + + @pytest.mark.asyncio + async def test_required_with_single_tool(self): + """ + Test scenario 4: When using required tool_choice with only one tool, ensure it doesn't cause + infinite loops. + """ + # Set up fake model to return a tool call followed by a text message + fake_model = FakeModel() + fake_model.add_multiple_turn_outputs( + [ + # First call returns a tool call + [get_function_tool_call("custom_tool", "{}")], + # Second call returns a text message + [get_text_message("Final response")], + ] + ) + + # Create agent with a single tool and tool_choice="required" + custom_tool = get_function_tool("custom_tool", return_value="tool result") + agent = Agent( + name="test_agent", + model=fake_model, + tools=[custom_tool], + model_settings=ModelSettings(tool_choice="required"), + ) + + # Run should complete without infinite loops + result = await Runner.run(agent, "first run") + assert result.final_output == "Final response" + + @pytest.mark.asyncio + async def test_dont_reset_tool_choice_if_not_required(self): + """ + Test scenario 5: When agent.reset_tool_choice is False, ensure tool_choice is not reset. + """ + # Set up fake model to return a tool call followed by a text message + fake_model = FakeModel() + fake_model.add_multiple_turn_outputs( + [ + # First call returns a tool call + [get_function_tool_call("custom_tool", "{}")], + # Second call returns a text message + [get_text_message("Final response")], + ] + ) + + # Create agent with a single tool and tool_choice="required" and reset_tool_choice=False + custom_tool = get_function_tool("custom_tool", return_value="tool result") + agent = Agent( + name="test_agent", + model=fake_model, + tools=[custom_tool], + model_settings=ModelSettings(tool_choice="required"), + reset_tool_choice=False, + ) + + await Runner.run(agent, "test") + + assert fake_model.last_turn_args["model_settings"].tool_choice == "required" diff --git a/tests/test_tracing_errors.py b/tests/test_tracing_errors.py index baa77681..6d698bcc 100644 --- a/tests/test_tracing_errors.py +++ b/tests/test_tracing_errors.py @@ -244,7 +244,18 @@ async def test_multiple_handoff_doesnt_error(): }, }, {"type": "generation"}, - {"type": "handoff", "data": {"from_agent": "test", "to_agent": "test"}}, + {"type": "handoff", + "data": {"from_agent": "test", "to_agent": "test"}, + "error": { + "data": { + "requested_agents": [ + "test", + "test", + ], + }, + "message": "Multiple handoffs requested", + }, + }, ], }, { @@ -372,7 +383,19 @@ async def test_handoffs_lead_to_correct_agent_spans(): {"type": "generation"}, { "type": "handoff", - "data": {"from_agent": "test_agent_3", "to_agent": "test_agent_1"}, + "data": { + "from_agent": "test_agent_3", + "to_agent": "test_agent_1" + }, + "error": { + "data": { + "requested_agents": [ + "test_agent_1", + "test_agent_2", + ], + }, + "message": "Multiple handoffs requested", + }, }, ], }, diff --git a/tests/test_tracing_errors_streamed.py b/tests/test_tracing_errors_streamed.py index 7e65ff12..416793e7 100644 --- a/tests/test_tracing_errors_streamed.py +++ b/tests/test_tracing_errors_streamed.py @@ -262,7 +262,14 @@ async def test_multiple_handoff_doesnt_error(): }, }, {"type": "generation"}, - {"type": "handoff", "data": {"from_agent": "test", "to_agent": "test"}}, + { + "type": "handoff", + "data": {"from_agent": "test", "to_agent": "test"}, + "error": { + "data": {"requested_agents": ["test", "test"]}, + "message": "Multiple handoffs requested", + }, + }, ], }, { @@ -396,6 +403,10 @@ async def test_handoffs_lead_to_correct_agent_spans(): {"type": "generation"}, { "type": "handoff", + "error": { + "message": "Multiple handoffs requested", + "data": {"requested_agents": ["test_agent_1", "test_agent_2"]}, + }, "data": {"from_agent": "test_agent_3", "to_agent": "test_agent_1"}, }, ], diff --git a/tests/test_visualization.py b/tests/test_visualization.py new file mode 100644 index 00000000..6aa86774 --- /dev/null +++ b/tests/test_visualization.py @@ -0,0 +1,136 @@ +from unittest.mock import Mock + +import graphviz # type: ignore +import pytest + +from agents import Agent +from agents.extensions.visualization import ( + draw_graph, + get_all_edges, + get_all_nodes, + get_main_graph, +) +from agents.handoffs import Handoff + + +@pytest.fixture +def mock_agent(): + tool1 = Mock() + tool1.name = "Tool1" + tool2 = Mock() + tool2.name = "Tool2" + + handoff1 = Mock(spec=Handoff) + handoff1.agent_name = "Handoff1" + + agent = Mock(spec=Agent) + agent.name = "Agent1" + agent.tools = [tool1, tool2] + agent.handoffs = [handoff1] + + return agent + + +def test_get_main_graph(mock_agent): + result = get_main_graph(mock_agent) + print(result) + assert "digraph G" in result + assert "graph [splines=true];" in result + assert 'node [fontname="Arial"];' in result + assert "edge [penwidth=1.5];" in result + assert ( + '"__start__" [label="__start__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" in result + ) + assert ( + '"__end__" [label="__end__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" in result + ) + assert ( + '"Agent1" [label="Agent1", shape=box, style=filled, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" in result + ) + assert ( + '"Tool1" [label="Tool1", shape=ellipse, style=filled, ' + "fillcolor=lightgreen, width=0.5, height=0.3];" in result + ) + assert ( + '"Tool2" [label="Tool2", shape=ellipse, style=filled, ' + "fillcolor=lightgreen, width=0.5, height=0.3];" in result + ) + assert ( + '"Handoff1" [label="Handoff1", shape=box, style=filled, style=rounded, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" in result + ) + + +def test_get_all_nodes(mock_agent): + result = get_all_nodes(mock_agent) + assert ( + '"__start__" [label="__start__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" in result + ) + assert ( + '"__end__" [label="__end__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" in result + ) + assert ( + '"Agent1" [label="Agent1", shape=box, style=filled, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" in result + ) + assert ( + '"Tool1" [label="Tool1", shape=ellipse, style=filled, ' + "fillcolor=lightgreen, width=0.5, height=0.3];" in result + ) + assert ( + '"Tool2" [label="Tool2", shape=ellipse, style=filled, ' + "fillcolor=lightgreen, width=0.5, height=0.3];" in result + ) + assert ( + '"Handoff1" [label="Handoff1", shape=box, style=filled, style=rounded, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" in result + ) + + +def test_get_all_edges(mock_agent): + result = get_all_edges(mock_agent) + assert '"__start__" -> "Agent1";' in result + assert '"Agent1" -> "__end__";' + assert '"Agent1" -> "Tool1" [style=dotted, penwidth=1.5];' in result + assert '"Tool1" -> "Agent1" [style=dotted, penwidth=1.5];' in result + assert '"Agent1" -> "Tool2" [style=dotted, penwidth=1.5];' in result + assert '"Tool2" -> "Agent1" [style=dotted, penwidth=1.5];' in result + assert '"Agent1" -> "Handoff1";' in result + + +def test_draw_graph(mock_agent): + graph = draw_graph(mock_agent) + assert isinstance(graph, graphviz.Source) + assert "digraph G" in graph.source + assert "graph [splines=true];" in graph.source + assert 'node [fontname="Arial"];' in graph.source + assert "edge [penwidth=1.5];" in graph.source + assert ( + '"__start__" [label="__start__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" in graph.source + ) + assert ( + '"__end__" [label="__end__", shape=ellipse, style=filled, ' + "fillcolor=lightblue, width=0.5, height=0.3];" in graph.source + ) + assert ( + '"Agent1" [label="Agent1", shape=box, style=filled, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" in graph.source + ) + assert ( + '"Tool1" [label="Tool1", shape=ellipse, style=filled, ' + "fillcolor=lightgreen, width=0.5, height=0.3];" in graph.source + ) + assert ( + '"Tool2" [label="Tool2", shape=ellipse, style=filled, ' + "fillcolor=lightgreen, width=0.5, height=0.3];" in graph.source + ) + assert ( + '"Handoff1" [label="Handoff1", shape=box, style=filled, style=rounded, ' + "fillcolor=lightyellow, width=1.5, height=0.8];" in graph.source + ) diff --git a/tests/voice/test_openai_stt.py b/tests/voice/test_openai_stt.py index 75559232..89b5cca7 100644 --- a/tests/voice/test_openai_stt.py +++ b/tests/voice/test_openai_stt.py @@ -269,7 +269,7 @@ async def test_timeout_waiting_for_created_event(monkeypatch): async for _ in turns: pass - assert "Timeout waiting for transcription_session.created event" in str(exc_info.value) + assert "Timeout waiting for transcription_session.created event" in str(exc_info.value) await session.close() @@ -302,13 +302,11 @@ async def test_session_error_event(): trace_include_sensitive_audio_data=False, ) - with pytest.raises(STTWebsocketConnectionError) as exc_info: + with pytest.raises(STTWebsocketConnectionError): turns = session.transcribe_turns() async for _ in turns: pass - assert "Simulated server error!" in str(exc_info.value) - await session.close() @@ -362,8 +360,8 @@ async def test_inactivity_timeout(): async for turn in session.transcribe_turns(): collected_turns.append(turn) - assert "Timeout waiting for transcription_session" in str(exc_info.value) + assert "Timeout waiting for transcription_session" in str(exc_info.value) - assert len(collected_turns) == 0, "No transcripts expected, but we got something?" + assert len(collected_turns) == 0, "No transcripts expected, but we got something?" await session.close() diff --git a/uv.lock b/uv.lock index c0cf8098..50b8b90b 100644 --- a/uv.lock +++ b/uv.lock @@ -1,5 +1,4 @@ version = 1 -revision = 1 requires-python = ">=3.9" resolution-markers = [ "python_full_version >= '3.10'", @@ -703,6 +702,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034 }, ] +[[package]] +name = "graphviz" +version = "0.20.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/fa/83/5a40d19b8347f017e417710907f824915fba411a9befd092e52746b63e9f/graphviz-0.20.3.zip", hash = "sha256:09d6bc81e6a9fa392e7ba52135a9d49f1ed62526f96499325930e87ca1b5925d", size = 256455 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/00/be/d59db2d1d52697c6adc9eacaf50e8965b6345cc143f671e1ed068818d5cf/graphviz-0.20.3-py3-none-any.whl", hash = "sha256:81f848f2904515d8cd359cc611faba817598d2feaac4027b266aa3eda7b3dde5", size = 47126 }, +] + [[package]] name = "greenlet" version = "3.1.1" @@ -814,21 +822,12 @@ wheels = [ ] [[package]] -name = "huggingface-hub" -version = "0.29.3" +name = "httpx-sse" +version = "0.4.0" source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "filelock" }, - { name = "fsspec" }, - { name = "packaging" }, - { name = "pyyaml" }, - { name = "requests" }, - { name = "tqdm" }, - { name = "typing-extensions" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/e5/f9/851f34b02970e8143d41d4001b2d49e54ef113f273902103823b8bc95ada/huggingface_hub-0.29.3.tar.gz", hash = "sha256:64519a25716e0ba382ba2d3fb3ca082e7c7eb4a2fc634d200e8380006e0760e5", size = 390123 } +sdist = { url = "https://files.pythonhosted.org/packages/4c/60/8f4281fa9bbf3c8034fd54c0e7412e66edbab6bc74c4996bd616f8d0406e/httpx-sse-0.4.0.tar.gz", hash = "sha256:1e81a3a3070ce322add1d3529ed42eb5f70817f45ed6ec915ab753f961139721", size = 12624 } wheels = [ - { url = "https://files.pythonhosted.org/packages/40/0c/37d380846a2e5c9a3c6a73d26ffbcfdcad5fc3eacf42fdf7cff56f2af634/huggingface_hub-0.29.3-py3-none-any.whl", hash = "sha256:0b25710932ac649c08cdbefa6c6ccb8e88eef82927cacdb048efb726429453aa", size = 468997 }, + { url = "https://files.pythonhosted.org/packages/e1/9b/a181f281f65d776426002f330c31849b86b31fc9d848db62e16f03ff739f/httpx_sse-0.4.0-py3-none-any.whl", hash = "sha256:f329af6eae57eaa2bdfd962b42524764af68075ea87370a2de920af5341e318f", size = 7819 }, ] [[package]] @@ -1132,6 +1131,25 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b3/73/085399401383ce949f727afec55ec3abd76648d04b9f22e1c0e99cb4bec3/MarkupSafe-3.0.2-cp39-cp39-win_amd64.whl", hash = "sha256:6e296a513ca3d94054c2c881cc913116e90fd030ad1c656b3869762b754f5f8a", size = 15506 }, ] +[[package]] +name = "mcp" +version = "1.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "python_full_version >= '3.10'" }, + { name = "httpx", marker = "python_full_version >= '3.10'" }, + { name = "httpx-sse", marker = "python_full_version >= '3.10'" }, + { name = "pydantic", marker = "python_full_version >= '3.10'" }, + { name = "pydantic-settings", marker = "python_full_version >= '3.10'" }, + { name = "sse-starlette", marker = "python_full_version >= '3.10'" }, + { name = "starlette", marker = "python_full_version >= '3.10'" }, + { name = "uvicorn", marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/6d/c9/c55764824e893fdebe777ac7223200986a275c3191dba9169f8eb6d7c978/mcp-1.5.0.tar.gz", hash = "sha256:5b2766c05e68e01a2034875e250139839498c61792163a7b221fc170c12f5aa9", size = 159128 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/d1/3ff566ecf322077d861f1a68a1ff025cad337417bd66ad22a7c6f7dfcfaf/mcp-1.5.0-py3-none-any.whl", hash = "sha256:51c3f35ce93cb702f7513c12406bbea9665ef75a08db909200b07da9db641527", size = 73734 }, +] + [[package]] name = "mdit-py-plugins" version = "0.4.2" @@ -1532,99 +1550,84 @@ wheels = [ ] [[package]] -name = "openinference-instrumentation" -version = "0.1.25" -source = { registry = "https://pypi.org/simple" } +name = "openai-agents" +version = "0.0.7" +source = { editable = "." } dependencies = [ - { name = "openinference-semantic-conventions" }, - { name = "opentelemetry-api" }, - { name = "opentelemetry-sdk" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/8d/b4/132bcd9d108419d02bd1f86a7584587fa014725abef7746d4f96dc5748f0/openinference_instrumentation-0.1.25.tar.gz", hash = "sha256:5208d7a0abb23f34d218ac431aedafb928d9cf02200570fece08a4815e6fd31f", size = 17494 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/fe/6b/fc017d752eb7d1460151911b906d4cb9015fcaa4182ba7df390c87a162fc/openinference_instrumentation-0.1.25-py3-none-any.whl", hash = "sha256:89e60310e568c14b8fb9a36ab6ac7b7dfac6c7ee9bf8cddce2d3b833f402f3f8", size = 22854 }, -] - -[[package]] -name = "openinference-instrumentation-openai" -version = "0.1.22" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "openinference-instrumentation" }, - { name = "openinference-semantic-conventions" }, - { name = "opentelemetry-api" }, - { name = "opentelemetry-instrumentation" }, - { name = "opentelemetry-semantic-conventions" }, - { name = "typing-extensions" }, - { name = "wrapt" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/c1/d6/4bd2c552d8bc5c9330f81acdd30bb1935dcabd08ee8f24b4d2c28bd7d054/openinference_instrumentation_openai-0.1.22.tar.gz", hash = "sha256:df174fd7ba4b77c9d0e1da60bf4d9c4cfe1eec2083aeed505c64daca374e035b", size = 17993 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/bc/cd/a9b67d971fc823f77f091a179602bcda967da73aa51b5c94d5cff542f047/openinference_instrumentation_openai-0.1.22-py3-none-any.whl", hash = "sha256:02e69abc2e07990df2fd4b6e4281ab2aae4d1e1833922a583863c5ca7786f266", size = 24310 }, -] - -[[package]] -name = "openinference-semantic-conventions" -version = "0.1.15" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/85/66/8552919df880ddf9236066b9adac3bcd16191380c188dbec3cbce096cc47/openinference_semantic_conventions-0.1.15.tar.gz", hash = "sha256:f2e3af34e997600dd1e2ba5ddd885e272099f9dc1170bbef279f1ae73e953d3b", size = 9431 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b1/87/9a60d06249b36fa8cfded1e7e46efaaaf3a7742cc01113c772058139140b/openinference_semantic_conventions-0.1.15-py3-none-any.whl", hash = "sha256:487041cd430abcf5909d0d9614902a604abedc93f4be7d3cd3f92d68c3bc7423", size = 9227 }, -] - -[[package]] -name = "opentelemetry-api" -version = "1.31.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "deprecated" }, - { name = "importlib-metadata" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/8a/cf/db26ab9d748bf50d6edf524fb863aa4da616ba1ce46c57a7dff1112b73fb/opentelemetry_api-1.31.1.tar.gz", hash = "sha256:137ad4b64215f02b3000a0292e077641c8611aab636414632a9b9068593b7e91", size = 64059 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/6c/c8/86557ff0da32f3817bc4face57ea35cfdc2f9d3bcefd42311ef860dcefb7/opentelemetry_api-1.31.1-py3-none-any.whl", hash = "sha256:1511a3f470c9c8a32eeea68d4ea37835880c0eed09dd1a0187acc8b1301da0a1", size = 65197 }, -] - -[[package]] -name = "opentelemetry-instrumentation" -version = "0.52b1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-api" }, - { name = "opentelemetry-semantic-conventions" }, - { name = "packaging" }, - { name = "wrapt" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/49/c9/c52d444576b0776dbee71d2a4485be276cf46bec0123a5ba2f43f0cf7cde/opentelemetry_instrumentation-0.52b1.tar.gz", hash = "sha256:739f3bfadbbeec04dd59297479e15660a53df93c131d907bb61052e3d3c1406f", size = 28406 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/61/dd/a2b35078170941990e7a5194b9600fa75868958a9a2196a752da0e7b97a0/opentelemetry_instrumentation-0.52b1-py3-none-any.whl", hash = "sha256:8c0059c4379d77bbd8015c8d8476020efe873c123047ec069bb335e4b8717477", size = 31036 }, -] - -[[package]] -name = "opentelemetry-sdk" -version = "1.31.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "opentelemetry-api" }, - { name = "opentelemetry-semantic-conventions" }, + { name = "griffe" }, + { name = "mcp", marker = "python_full_version >= '3.10'" }, + { name = "openai" }, + { name = "pydantic" }, + { name = "requests" }, + { name = "types-requests" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/63/d9/4fe159908a63661e9e635e66edc0d0d816ed20cebcce886132b19ae87761/opentelemetry_sdk-1.31.1.tar.gz", hash = "sha256:c95f61e74b60769f8ff01ec6ffd3d29684743404603df34b20aa16a49dc8d903", size = 159523 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/bc/36/758e5d3746bc86a2af20aa5e2236a7c5aa4264b501dc0e9f40efd9078ef0/opentelemetry_sdk-1.31.1-py3-none-any.whl", hash = "sha256:882d021321f223e37afaca7b4e06c1d8bbc013f9e17ff48a7aa017460a8e7dae", size = 118866 }, + +[package.optional-dependencies] +viz = [ + { name = "graphviz" }, +] +voice = [ + { name = "numpy", version = "2.2.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "websockets" }, ] -[[package]] -name = "opentelemetry-semantic-conventions" -version = "0.52b1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "deprecated" }, - { name = "opentelemetry-api" }, +[package.dev-dependencies] +dev = [ + { name = "coverage" }, + { name = "graphviz" }, + { name = "inline-snapshot" }, + { name = "mkdocs" }, + { name = "mkdocs-material" }, + { name = "mkdocstrings", extra = ["python"] }, + { name = "mypy" }, + { name = "playwright" }, + { name = "pynput" }, + { name = "pytest" }, + { name = "pytest-asyncio" }, + { name = "pytest-mock" }, + { name = "rich" }, + { name = "ruff" }, + { name = "sounddevice" }, + { name = "textual" }, + { name = "types-pynput" }, + { name = "websockets" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/06/8c/599f9f27cff097ec4d76fbe9fe6d1a74577ceec52efe1a999511e3c42ef5/opentelemetry_semantic_conventions-0.52b1.tar.gz", hash = "sha256:7b3d226ecf7523c27499758a58b542b48a0ac8d12be03c0488ff8ec60c5bae5d", size = 111275 } -wheels = [ - { url = "https://files.pythonhosted.org/packages/98/be/d4ba300cfc1d4980886efbc9b48ee75242b9fcf940d9c4ccdc9ef413a7cf/opentelemetry_semantic_conventions-0.52b1-py3-none-any.whl", hash = "sha256:72b42db327e29ca8bb1b91e8082514ddf3bbf33f32ec088feb09526ade4bc77e", size = 183409 }, + +[package.metadata] +requires-dist = [ + { name = "graphviz", marker = "extra == 'viz'", specifier = ">=0.17" }, + { name = "griffe", specifier = ">=1.5.6,<2" }, + { name = "mcp", marker = "python_full_version >= '3.10'" }, + { name = "numpy", marker = "python_full_version >= '3.10' and extra == 'voice'", specifier = ">=2.2.0,<3" }, + { name = "openai", specifier = ">=1.66.5" }, + { name = "pydantic", specifier = ">=2.10,<3" }, + { name = "requests", specifier = ">=2.0,<3" }, + { name = "types-requests", specifier = ">=2.0,<3" }, + { name = "typing-extensions", specifier = ">=4.12.2,<5" }, + { name = "websockets", marker = "extra == 'voice'", specifier = ">=15.0,<16" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "coverage", specifier = ">=7.6.12" }, + { name = "graphviz" }, + { name = "inline-snapshot", specifier = ">=0.20.7" }, + { name = "mkdocs", specifier = ">=1.6.0" }, + { name = "mkdocs-material", specifier = ">=9.6.0" }, + { name = "mkdocstrings", extras = ["python"], specifier = ">=0.28.0" }, + { name = "mypy" }, + { name = "playwright", specifier = "==1.50.0" }, + { name = "pynput" }, + { name = "pytest" }, + { name = "pytest-asyncio" }, + { name = "pytest-mock", specifier = ">=3.14.0" }, + { name = "rich" }, + { name = "ruff", specifier = "==0.9.2" }, + { name = "sounddevice" }, + { name = "textual" }, + { name = "types-pynput" }, + { name = "websockets" }, ] [[package]] @@ -1947,6 +1950,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a1/0c/c5c5cd3689c32ed1fe8c5d234b079c12c281c051759770c05b8bed6412b5/pydantic_core-2.27.2-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:7d0c8399fcc1848491f00e0314bd59fb34a9c008761bcb422a057670c3f65e35", size = 2004961 }, ] +[[package]] +name = "pydantic-settings" +version = "2.8.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pydantic", marker = "python_full_version >= '3.10'" }, + { name = "python-dotenv", marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/88/82/c79424d7d8c29b994fb01d277da57b0a9b09cc03c3ff875f9bd8a86b2145/pydantic_settings-2.8.1.tar.gz", hash = "sha256:d5c663dfbe9db9d5e1c646b2e161da12f0d734d422ee56f567d0ea2cee4e8585", size = 83550 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/53/a64f03044927dc47aafe029c42a5b7aabc38dfb813475e0e1bf71c4a59d0/pydantic_settings-2.8.1-py3-none-any.whl", hash = "sha256:81942d5ac3d905f7f3ee1a70df5dfb62d5569c12f51a5a647defc1c3d9ee2e9c", size = 30839 }, +] + [[package]] name = "pyee" version = "12.1.1" @@ -2140,11 +2156,11 @@ wheels = [ [[package]] name = "python-dotenv" -version = "1.1.0" +version = "1.0.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/88/2c/7bb1416c5620485aa793f2de31d3df393d3686aa8a8506d11e10e13c5baf/python_dotenv-1.1.0.tar.gz", hash = "sha256:41f90bc6f5f177fb41f53e87666db362025010eb28f60a01c9143bfa33a2b2d5", size = 39920 } +sdist = { url = "https://files.pythonhosted.org/packages/bc/57/e84d88dfe0aec03b7a2d4327012c1627ab5f03652216c63d49846d7a6c58/python-dotenv-1.0.1.tar.gz", hash = "sha256:e324ee90a023d808f1959c46bcbc04446a10ced277783dc6ee09987c37ec10ca", size = 39115 } wheels = [ - { url = "https://files.pythonhosted.org/packages/1e/18/98a99ad95133c6a6e2005fe89faedf294a748bd5dc803008059409ac9b1e/python_dotenv-1.1.0-py3-none-any.whl", hash = "sha256:d7c01d9e2293916c18baf562d95698754b0dbbb5e74d457c45d4f6561fb9d55d", size = 20256 }, + { url = "https://files.pythonhosted.org/packages/6a/3e/b68c118422ec867fa7ab88444e1274aa40681c606d59ac27de5a5588f082/python_dotenv-1.0.1-py3-none-any.whl", hash = "sha256:f7b63ef50f1b690dddf550d03497b66d609393b40b564ed0d674909a68ebf16a", size = 19863 }, ] [[package]] @@ -2521,12 +2537,28 @@ wheels = [ ] [[package]] -name = "strenum" -version = "0.4.15" +name = "sse-starlette" +version = "2.2.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/85/ad/430fb60d90e1d112a62ff57bdd1f286ec73a2a0331272febfddd21f330e1/StrEnum-0.4.15.tar.gz", hash = "sha256:878fb5ab705442070e4dd1929bb5e2249511c0bcf2b0eeacf3bcd80875c82eff", size = 23384 } +dependencies = [ + { name = "anyio", marker = "python_full_version >= '3.10'" }, + { name = "starlette", marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/71/a4/80d2a11af59fe75b48230846989e93979c892d3a20016b42bb44edb9e398/sse_starlette-2.2.1.tar.gz", hash = "sha256:54470d5f19274aeed6b2d473430b08b4b379ea851d953b11d7f1c4a2c118b419", size = 17376 } wheels = [ - { url = "https://files.pythonhosted.org/packages/81/69/297302c5f5f59c862faa31e6cb9a4cd74721cd1e052b38e464c5b402df8b/StrEnum-0.4.15-py3-none-any.whl", hash = "sha256:a30cda4af7cc6b5bf52c8055bc4bf4b2b6b14a93b574626da33df53cf7740659", size = 8851 }, + { url = "https://files.pythonhosted.org/packages/d9/e0/5b8bd393f27f4a62461c5cf2479c75a2cc2ffa330976f9f00f5f6e4f50eb/sse_starlette-2.2.1-py3-none-any.whl", hash = "sha256:6410a3d3ba0c89e7675d4c273a301d64649c03a5ef1ca101f10b47f895fd0e99", size = 10120 }, +] + +[[package]] +name = "starlette" +version = "0.46.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio", marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/04/1b/52b27f2e13ceedc79a908e29eac426a63465a1a01248e5f24aa36a62aeb3/starlette-0.46.1.tar.gz", hash = "sha256:3c88d58ee4bd1bb807c0d1acb381838afc7752f9ddaec81bbe4383611d833230", size = 2580102 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/4b/528ccf7a982216885a1ff4908e886b8fb5f19862d1962f56a3fce2435a70/starlette-0.46.1-py3-none-any.whl", hash = "sha256:77c74ed9d2720138b25875133f3a2dae6d854af2ec37dceb56aef370c1d8a227", size = 71995 }, ] [[package]] @@ -2720,15 +2752,17 @@ wheels = [ ] [[package]] -name = "wasabi" -version = "1.1.3" +name = "uvicorn" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "click", marker = "python_full_version >= '3.10'" }, + { name = "h11", marker = "python_full_version >= '3.10'" }, + { name = "typing-extensions", marker = "python_full_version == '3.10.*'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/ac/f9/054e6e2f1071e963b5e746b48d1e3727470b2a490834d18ad92364929db3/wasabi-1.1.3.tar.gz", hash = "sha256:4bb3008f003809db0c3e28b4daf20906ea871a2bb43f9914197d540f4f2e0878", size = 30391 } +sdist = { url = "https://files.pythonhosted.org/packages/4b/4d/938bd85e5bf2edeec766267a5015ad969730bb91e31b44021dfe8b22df6c/uvicorn-0.34.0.tar.gz", hash = "sha256:404051050cd7e905de2c9a7e61790943440b3416f49cb409f965d9dcd0fa73e9", size = 76568 } wheels = [ - { url = "https://files.pythonhosted.org/packages/06/7c/34330a89da55610daa5f245ddce5aab81244321101614751e7537f125133/wasabi-1.1.3-py3-none-any.whl", hash = "sha256:f76e16e8f7e79f8c4c8be49b4024ac725713ab10cd7f19350ad18a8e3f71728c", size = 27880 }, + { url = "https://files.pythonhosted.org/packages/61/14/33a3a1352cfa71812a3a21e8c9bfb83f60b0011f5e36f2b1399d51928209/uvicorn-0.34.0-py3-none-any.whl", hash = "sha256:023dc038422502fa28a09c7a30bf2b6991512da7dcdb8fd35fe57cfc154126f4", size = 62315 }, ] [[package]]