diff --git a/README.md b/README.md index d35bf720..490e82f6 100644 --- a/README.md +++ b/README.md @@ -125,7 +125,7 @@ Cybersecurity AI (CAI) is a lightweight, open-source framework that empowers sec ### ๐Ÿ“Š Research Impact - Pioneered LLM-powered AI Security with PentestGPT, establishing the foundation for the `Cybersecurity AI` research domain [![arXiv](https://img.shields.io/badge/arXiv-2308.06782-b31b1b.svg)](https://arxiv.org/pdf/2308.06782) -- Established the `Cybersecurity AI` research line with **4 peer-reviewed papers and technical reports** and active research collaborations [![arXiv](https://img.shields.io/badge/arXiv-2504.06017-b31b1b.svg)](https://arxiv.org/pdf/2504.06017) [![arXiv](https://img.shields.io/badge/arXiv-2506.23592-b31b1b.svg)](https://arxiv.org/abs/2506.23592) [![arXiv](https://img.shields.io/badge/arXiv-2508.13588-b31b1b.svg)](https://arxiv.org/abs/2508.13588) [![arXiv](https://img.shields.io/badge/arXiv-2508.21669-b31b1b.svg)](https://arxiv.org/abs/2508.21669) +- Established the `Cybersecurity AI` research line with **4 papers and technical reports** and active research collaborations [![arXiv](https://img.shields.io/badge/arXiv-2504.06017-b31b1b.svg)](https://arxiv.org/pdf/2504.06017) [![arXiv](https://img.shields.io/badge/arXiv-2506.23592-b31b1b.svg)](https://arxiv.org/abs/2506.23592) [![arXiv](https://img.shields.io/badge/arXiv-2508.13588-b31b1b.svg)](https://arxiv.org/abs/2508.13588) [![arXiv](https://img.shields.io/badge/arXiv-2508.21669-b31b1b.svg)](https://arxiv.org/abs/2508.21669) - Demonstrated **3,600ร— performance improvement** over human penetration testers in standardized CTF benchmark evaluations [![arXiv](https://img.shields.io/badge/arXiv-2504.06017-b31b1b.svg)](https://arxiv.org/pdf/2504.06017) - Identified **CVSS 4.3-7.5 severity vulnerabilities** in production systems through automated security assessment [![arXiv](https://img.shields.io/badge/arXiv-2504.06017-b31b1b.svg)](https://arxiv.org/pdf/2504.06017) - **Democratization of AI-empowered vulnerability research**: CAI enables both non-security domain experts and experienced researchers to conduct more efficient vulnerability discovery, expanding the security research community while empowering small and medium enterprises to conduct autonomous security assessments [![arXiv](https://img.shields.io/badge/arXiv-2504.06017-b31b1b.svg)](https://arxiv.org/pdf/2504.06017) @@ -199,6 +199,7 @@ Cybersecurity AI is a critical field, yet many groups are misguidedly pursuing i - [NDAY Security](https://ndaysecurity.com/) - [Runsybil](https://www.runsybil.com) - [Selfhack](https://www.selfhack.fi) +- [Sola Security](https://sola.security/) - [SQUR](https://squr.ai/) - [Staris](https://staris.tech/) - [Sxipher](https://www.sxipher.com/) (seems discontinued) @@ -515,7 +516,7 @@ result = await Runner.run(agent, message) ``` -You may find different [tools](cai/tools). They are grouped in 6 major categories inspired by the security kill chain [^2]: +You may find different [tools](tools). They are grouped in 6 major categories inspired by the security kill chain [^2]: 1. Reconnaissance and weaponization - *reconnaissance* (crypto, listing, etc) 2. Exploitation - *exploitation* @@ -748,6 +749,22 @@ OPENROUTER_API_KEY= # note, add yours OPENROUTER_API_BASE=https://openrouter.ai/api/v1 ``` +### Azure OpenAI + +The Cybersecurity AI (CAI) platform integrates seamlessly with Azure OpenAI, enabling organizations to run CAI against enterprise-hosted models (e.g., gpt-4o). This pathway is ideal for teams that must operate within Azure governance while leveraging advanced model capabilities. +To enable Azure OpenAI support in CAI, configure your environment by adding the following entries to your .env. This ensures CAI can reach your Azure deployment endpoint and authenticate correctly. + +```bash +CAI_AGENT_TYPE=redteam_agent +CAI_MODEL=azure/ +# Required: keep non-empty even when using Azure +OPENAI_API_KEY=dummy +# Azure credentials and endpoint +AZURE_API_KEY= +AZURE_API_BASE=https://.openai.azure.com/openai/deployments//chat/completions?api-version=2025-01-01-preview +``` + + ### MCP CAI supports the Model Context Protocol (MCP) for integrating external tools and services with AI agents. MCP is supported via two transport mechanisms: @@ -1004,22 +1021,23 @@ When CAI is prompted by the first time, the user is provided with two paths, the
Can I expand CAI capabilities using previous run logs? -Absolutely! The **memory extension** allows you to use a previously sucessful runs ( the log object is stored as a **.jsonl file in the [log](cai/logs) folder** ) in a new run against the same target. -The user is also given the path highlighted in orange as shown below. +Yes. Today CAI performs best by relying on Inโ€‘Context Learning (ICL). Rather than building longโ€‘term stores, the recommended workflow is to load relevant prior logs directly into the current session so the model can reason with them in context. + +Use the `/load` command to bring JSONL logs into CAIโ€™s context (this replaces the legacy memory-loading tool): + +```bash +CAI>/load logs/cai_20250408_111856.jsonl # Load into current agent +CAI>/load agent # Load into a specific agent +CAI>/load all # Distribute across all agents +CAI>/load parallel # Match to configured parallel agents +# Tip: if you omit , /load uses `logs/last`. Alias: /l +``` + +CAI prints the path to the current runโ€™s JSONL log at startup (highlighted in orange), which you can pass to `/load`: ![cai-009-logs](imgs/readme_imgs/cai-009-logs.png) -How to make use of this functionality? - -1. Run CAI against the target. Let's assume the target name is: `target001`. -2. Get the log file path, something like: ```logs/cai_20250408_111856.jsonl``` -3. Generate the memory using any model of your preference: -```shell JSONL_FILE_PATH="logs/cai_20250408_111856.jsonl" CTF_INSIDE="false" CAI_MEMORY_COLLECTION="target001" CAI_MEMORY="episodic" CAI_MODEL="claude-3-5-sonnet-20241022" python3 tools/2_jsonl_to_memory.py ``` - -The script [`tools/2_jsonl_to_memory.py`](cai/tools/2_jsonl_to_memory.py) will generate a memory collection file with the most relevant steps. The quality of the memory collection will depend on the model you use. - -4. Use the generated memory collection and execute a new run: -```shell CAI_MEMORY="episodic" CAI_MODEL="gpt-4o" CAI_MEMORY_COLLECTION="target001" CAI_TRACING=false python3 cai/cli.py``` +Legacy notes: earlier โ€œmemory extensionโ€ mechanisms (episodic/semantic stores and offline ingestion) are retained for reference only. See [src/cai/agents/memory.py](src/cai/agents/memory.py) for background and legacy details. Our current direction prioritizes ICL over persistent memory.
@@ -1042,7 +1060,12 @@ CAI itself is not a profit-seeking initiative. Our goal is to build a sustainabl +
I get a `Unable to locate package python3.12-venv` when installing the prerequisites on my debian based system! +The easiest way to get around this is to simply install [`python3.12`](https://www.python.org/downloads/release/python-3120/) from source. + + +
## Citation If you want to cite our work, please use the following: diff --git a/docs/cai_list_of_models.md b/docs/cai_list_of_models.md index d431fb7b..b1baf48b 100644 --- a/docs/cai_list_of_models.md +++ b/docs/cai_list_of_models.md @@ -7,25 +7,4 @@ CAI supports **over 300 models**, thanks to its integration with [LiteLLM](https - **DeepSeek**: DeepSeek V3, DeepSeek R1 - **Ollama**: Qwen2.5 72B, Qwen2.5 14B, and more -CAI is also compatibile with other platforms like OpenRouter and Ollama. Below youโ€™ll find some configurations to help you get started. - -#### [OpenRouter Integration](https://openrouter.ai/) - -To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Hereโ€™s how you can configure it: - -```bash -CAI_MODEL=openrouter/meta-llama/llama-4-maverick -OPENROUTER_API_KEY= # note, add yours -OPENROUTER_API_BASE=https://openrouter.ai/api/v1 -``` - - -#### [Ollama Integration](https://ollama.com/) -For local models using Ollama, add the following to your .env: - -```bash -CAI_MODEL=qwen2.5:72b -OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint -``` - -Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance. +CAI is also compatibile with other platforms like OpenRouter, Ollama and Azure OpenAI. At providers section youโ€™ll find some configurations to help you get started. diff --git a/docs/providers/azure.md b/docs/providers/azure.md new file mode 100644 index 00000000..5f373ec5 --- /dev/null +++ b/docs/providers/azure.md @@ -0,0 +1,84 @@ +# Azure OpenAI configuration + +> This guide shows how to run CAI against Azure-hosted OpenAI's models + +## Prerequisites +- Azure subscription with **Azure OpenAI** access. +- A **deployed model** in Azure AI Portal (e.g., a deployment named `gpt-4o`). + See Microsoft docs on creating the resource & deploying models. + - [Create resource & deploy](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/create-resource) + - [Working with models](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/working-with-models) + +#### 1. Deploy the base model +In Azure AI Portal, go to **Deployments** and deploy the requested base model (e.g., gpt-4o). + +#### 2. Get the deployment URL +From **Deployments**, select your deployment and copy the endpoint in this form: + +`https://.openai.azure.com/openai/deployments//chat/completions?api-version=2025-01-01-preview` + +Set this value as `AZURE_API_BASE` in your `.env`. +**Note:** CAI uses the OpenAI SDK style `base_url + /chat/completions`. For Azure, providing the full endpoint above (including `chat/completions?api-version=...`) ensures correct routing. + +#### 3. Get your API key +From your Azure OpenAI resource home page (it is displayed on the resource home page, along with the subscription ID, resource name, etc.). Put it in `.env` as `AZURE_API_KEY`. + +#### 4. Complete your `.env` +`OPENAI_API_KEY` must NOT be empty (use any placeholder like `"dummy"`). + +Example of good configured `.env`: + +```bash +OPENAI_API_KEY="dummy" +AZURE_API_KEY="your_subscription_api_key" +AZURE_API_BASE="https://.openai.azure.com/openai/deployments//chat/completions?api-version=2025-01-01-preview" +# Optional (if your setup expects it): +# AZURE_API_VERSION="2025-01-01-preview" + +ANTHROPIC_API_KEY="" +OLLAMA="" +PROMPT_TOOLKIT_NO_CPR=1 +``` + +#### 5. Start CAI and select the model +Launch CAI and select the Azure model: + +```vbnet +CAI> /model azure/ +โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Model Changed โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ +โ”‚ Model changed to: azure/ โ”‚ +โ”‚ Note: This will take effect on the next agent interaction โ”‚ +โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ +``` + +From this point you are interacting with your Azure-hosted OpenAI model. + +> โš ๏ธ Remember: you must select the model each time you start CAI. + + +> EXTRA configuration: +You can set the variable `CAI_MODEL` to avoid the need for repeated model setup during initialization. + +```bash +CAI_MODEL=azure/ +``` + +## Troubleshooting + +- 404 or โ€œdeployment not foundโ€: Ensure you have correctly copied the URL of the deployed model. + +Error example: +```sh +ERROR:cai.cli:Error in main loop: litellm.APIError: AzureException APIError - Resource not found +openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} +``` + +- 401: verify `AZURE_API_KEY` and that your region has access to the chosen model. + +Error example: +```sh +ERROR:cai.cli:Error in main loop: litellm.AuthenticationError: AzureException AuthenticationError - Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource. +openai.AuthenticationError: Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}} +``` + +- Time-outs / rate limits: check Azure usage and quota. \ No newline at end of file diff --git a/docs/providers/ollama.md b/docs/providers/ollama.md new file mode 100644 index 00000000..1ff39758 --- /dev/null +++ b/docs/providers/ollama.md @@ -0,0 +1,11 @@ +# Ollama Configuration + +#### [Ollama Integration](https://ollama.com/) +For local models using Ollama, add the following to your .env: + +```bash +CAI_MODEL=qwen2.5:72b +OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint +``` + +Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance. diff --git a/docs/providers/openrouter.md b/docs/providers/openrouter.md new file mode 100644 index 00000000..cf96b854 --- /dev/null +++ b/docs/providers/openrouter.md @@ -0,0 +1,11 @@ +# OpenRouter Configuration + +#### [OpenRouter Integration](https://openrouter.ai/) + +To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Hereโ€™s how you can configure it: + +```bash +CAI_MODEL=openrouter/meta-llama/llama-4-maverick +OPENROUTER_API_KEY= # note, add yours +OPENROUTER_API_BASE=https://openrouter.ai/api/v1 +``` diff --git a/mkdocs.yml b/mkdocs.yml index c8ca446b..30dcb8d5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -21,8 +21,13 @@ theme: nav: - Intro: index.md - Installation: cai_installation.md - - Quickstart: cai_quickstart.md - - List of Models: cai_list_of_models.md + - Quickstart: cai_quickstart.md + - List of Models: + - Available Models: cai_list_of_models.md + - Providers: + - OpenRouter: providers/openrouter.md + - Ollama: providers/ollama.md + - Azure OpenAI: providers/azure.md - Architecture: cai_architecture.md - Development: cai_development.md - Start Building: diff --git a/src/cai/sdk/agents/models/openai_chatcompletions.py b/src/cai/sdk/agents/models/openai_chatcompletions.py index c1f892f6..b7e424f9 100644 --- a/src/cai/sdk/agents/models/openai_chatcompletions.py +++ b/src/cai/sdk/agents/models/openai_chatcompletions.py @@ -12,6 +12,7 @@ from collections.abc import AsyncIterator, Iterable from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, Literal, cast, overload +import uuid import litellm import tiktoken from openai import NOT_GIVEN, AsyncOpenAI, AsyncStream, NotGiven @@ -969,6 +970,15 @@ class OpenAIChatCompletionsModel(Model): if not hasattr(self, "_pending_tool_calls"): self._pending_tool_calls = {} + + # Fix Google Gemini OpenAI compatibility issues. + # When using the OpenAI-compatible API to call tools with Google Gemini + # tool_call.id is returned as an empty string. + if "openai/gemini" in os.getenv("CAI_MODEL"): + for tool_call in assistant_msg.tool_calls: + if tool_call.id is None or tool_call.id == "": + tool_call.id = uuid.uuid4().hex[:16] + for tool_call in assistant_msg.tool_calls: # Handle empty arguments before storing tool_args = tool_call.function.arguments