mirror of https://github.com/aliasrobotics/cai.git
Merge branch 'aliasrobotics:main' into main
This commit is contained in:
Cristobal Veas
GitHub
commit
e466d64ccd
53
README.md
53
README.md
|
|
@ -125,7 +125,7 @@ Cybersecurity AI (CAI) is a lightweight, open-source framework that empowers sec
|
|||
|
||||
### 📊 Research Impact
|
||||
- Pioneered LLM-powered AI Security with PentestGPT, establishing the foundation for the `Cybersecurity AI` research domain [](https://arxiv.org/pdf/2308.06782)
|
||||
- Established the `Cybersecurity AI` research line with **4 peer-reviewed papers and technical reports** and active research collaborations [](https://arxiv.org/pdf/2504.06017) [](https://arxiv.org/abs/2506.23592) [](https://arxiv.org/abs/2508.13588) [](https://arxiv.org/abs/2508.21669)
|
||||
- Established the `Cybersecurity AI` research line with **4 papers and technical reports** and active research collaborations [](https://arxiv.org/pdf/2504.06017) [](https://arxiv.org/abs/2506.23592) [](https://arxiv.org/abs/2508.13588) [](https://arxiv.org/abs/2508.21669)
|
||||
- Demonstrated **3,600× performance improvement** over human penetration testers in standardized CTF benchmark evaluations [](https://arxiv.org/pdf/2504.06017)
|
||||
- Identified **CVSS 4.3-7.5 severity vulnerabilities** in production systems through automated security assessment [](https://arxiv.org/pdf/2504.06017)
|
||||
- **Democratization of AI-empowered vulnerability research**: CAI enables both non-security domain experts and experienced researchers to conduct more efficient vulnerability discovery, expanding the security research community while empowering small and medium enterprises to conduct autonomous security assessments [](https://arxiv.org/pdf/2504.06017)
|
||||
|
|
@ -199,6 +199,7 @@ Cybersecurity AI is a critical field, yet many groups are misguidedly pursuing i
|
|||
- [NDAY Security](https://ndaysecurity.com/)
|
||||
- [Runsybil](https://www.runsybil.com)
|
||||
- [Selfhack](https://www.selfhack.fi)
|
||||
- [Sola Security](https://sola.security/)
|
||||
- [SQUR](https://squr.ai/)
|
||||
- [Staris](https://staris.tech/)
|
||||
- [Sxipher](https://www.sxipher.com/) (seems discontinued)
|
||||
|
|
@ -515,7 +516,7 @@ result = await Runner.run(agent, message)
|
|||
```
|
||||
|
||||
|
||||
You may find different [tools](cai/tools). They are grouped in 6 major categories inspired by the security kill chain [^2]:
|
||||
You may find different [tools](tools). They are grouped in 6 major categories inspired by the security kill chain [^2]:
|
||||
|
||||
1. Reconnaissance and weaponization - *reconnaissance* (crypto, listing, etc)
|
||||
2. Exploitation - *exploitation*
|
||||
|
|
@ -748,6 +749,22 @@ OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
|||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||
```
|
||||
|
||||
### Azure OpenAI
|
||||
|
||||
The Cybersecurity AI (CAI) platform integrates seamlessly with Azure OpenAI, enabling organizations to run CAI against enterprise-hosted models (e.g., gpt-4o). This pathway is ideal for teams that must operate within Azure governance while leveraging advanced model capabilities.
|
||||
To enable Azure OpenAI support in CAI, configure your environment by adding the following entries to your .env. This ensures CAI can reach your Azure deployment endpoint and authenticate correctly.
|
||||
|
||||
```bash
|
||||
CAI_AGENT_TYPE=redteam_agent
|
||||
CAI_MODEL=azure/<model-name-deployed>
|
||||
# Required: keep non-empty even when using Azure
|
||||
OPENAI_API_KEY=dummy
|
||||
# Azure credentials and endpoint
|
||||
AZURE_API_KEY=<your-azure-openai-key>
|
||||
AZURE_API_BASE=https://<resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview
|
||||
```
|
||||
|
||||
|
||||
### MCP
|
||||
|
||||
CAI supports the Model Context Protocol (MCP) for integrating external tools and services with AI agents. MCP is supported via two transport mechanisms:
|
||||
|
|
@ -1004,22 +1021,23 @@ When CAI is prompted by the first time, the user is provided with two paths, the
|
|||
<details>
|
||||
<summary>Can I expand CAI capabilities using previous run logs?</summary>
|
||||
|
||||
Absolutely! The **memory extension** allows you to use a previously sucessful runs ( the log object is stored as a **.jsonl file in the [log](cai/logs) folder** ) in a new run against the same target.
|
||||
The user is also given the path highlighted in orange as shown below.
|
||||
Yes. Today CAI performs best by relying on In‑Context Learning (ICL). Rather than building long‑term stores, the recommended workflow is to load relevant prior logs directly into the current session so the model can reason with them in context.
|
||||
|
||||
Use the `/load` command to bring JSONL logs into CAI’s context (this replaces the legacy memory-loading tool):
|
||||
|
||||
```bash
|
||||
CAI>/load logs/cai_20250408_111856.jsonl # Load into current agent
|
||||
CAI>/load <file> agent <name> # Load into a specific agent
|
||||
CAI>/load <file> all # Distribute across all agents
|
||||
CAI>/load <file> parallel # Match to configured parallel agents
|
||||
# Tip: if you omit <file>, /load uses `logs/last`. Alias: /l
|
||||
```
|
||||
|
||||
CAI prints the path to the current run’s JSONL log at startup (highlighted in orange), which you can pass to `/load`:
|
||||
|
||||
|
||||
|
||||
How to make use of this functionality?
|
||||
|
||||
1. Run CAI against the target. Let's assume the target name is: `target001`.
|
||||
2. Get the log file path, something like: ```logs/cai_20250408_111856.jsonl```
|
||||
3. Generate the memory using any model of your preference:
|
||||
```shell JSONL_FILE_PATH="logs/cai_20250408_111856.jsonl" CTF_INSIDE="false" CAI_MEMORY_COLLECTION="target001" CAI_MEMORY="episodic" CAI_MODEL="claude-3-5-sonnet-20241022" python3 tools/2_jsonl_to_memory.py ```
|
||||
|
||||
The script [`tools/2_jsonl_to_memory.py`](cai/tools/2_jsonl_to_memory.py) will generate a memory collection file with the most relevant steps. The quality of the memory collection will depend on the model you use.
|
||||
|
||||
4. Use the generated memory collection and execute a new run:
|
||||
```shell CAI_MEMORY="episodic" CAI_MODEL="gpt-4o" CAI_MEMORY_COLLECTION="target001" CAI_TRACING=false python3 cai/cli.py```
|
||||
Legacy notes: earlier “memory extension” mechanisms (episodic/semantic stores and offline ingestion) are retained for reference only. See [src/cai/agents/memory.py](src/cai/agents/memory.py) for background and legacy details. Our current direction prioritizes ICL over persistent memory.
|
||||
|
||||
</details>
|
||||
|
||||
|
|
@ -1042,7 +1060,12 @@ CAI itself is not a profit-seeking initiative. Our goal is to build a sustainabl
|
|||
|
||||
</details>
|
||||
|
||||
<details><summary>I get a `Unable to locate package python3.12-venv` when installing the prerequisites on my debian based system!</summary>
|
||||
|
||||
The easiest way to get around this is to simply install [`python3.12`](https://www.python.org/downloads/release/python-3120/) from source.
|
||||
|
||||
|
||||
</details>
|
||||
|
||||
## Citation
|
||||
If you want to cite our work, please use the following:
|
||||
|
|
|
|||
|
|
@ -7,25 +7,4 @@ CAI supports **over 300 models**, thanks to its integration with [LiteLLM](https
|
|||
- **DeepSeek**: DeepSeek V3, DeepSeek R1
|
||||
- **Ollama**: Qwen2.5 72B, Qwen2.5 14B, and more
|
||||
|
||||
CAI is also compatibile with other platforms like OpenRouter and Ollama. Below you’ll find some configurations to help you get started.
|
||||
|
||||
#### [OpenRouter Integration](https://openrouter.ai/)
|
||||
|
||||
To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=openrouter/meta-llama/llama-4-maverick
|
||||
OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
||||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||
```
|
||||
|
||||
|
||||
#### [Ollama Integration](https://ollama.com/)
|
||||
For local models using Ollama, add the following to your .env:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=qwen2.5:72b
|
||||
OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint
|
||||
```
|
||||
|
||||
Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance.
|
||||
CAI is also compatibile with other platforms like OpenRouter, Ollama and Azure OpenAI. At providers section you’ll find some configurations to help you get started.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,84 @@
|
|||
# Azure OpenAI configuration
|
||||
|
||||
> This guide shows how to run CAI against Azure-hosted OpenAI's models
|
||||
|
||||
## Prerequisites
|
||||
- Azure subscription with **Azure OpenAI** access.
|
||||
- A **deployed model** in Azure AI Portal (e.g., a deployment named `gpt-4o`).
|
||||
See Microsoft docs on creating the resource & deploying models.
|
||||
- [Create resource & deploy](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/create-resource)
|
||||
- [Working with models](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/working-with-models)
|
||||
|
||||
#### 1. Deploy the base model
|
||||
In Azure AI Portal, go to **Deployments** and deploy the requested base model (e.g., gpt-4o).
|
||||
|
||||
#### 2. Get the deployment URL
|
||||
From **Deployments**, select your deployment and copy the endpoint in this form:
|
||||
|
||||
`https://<your-resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview`
|
||||
|
||||
Set this value as `AZURE_API_BASE` in your `.env`.
|
||||
**Note:** CAI uses the OpenAI SDK style `base_url + /chat/completions`. For Azure, providing the full endpoint above (including `chat/completions?api-version=...`) ensures correct routing.
|
||||
|
||||
#### 3. Get your API key
|
||||
From your Azure OpenAI resource home page (it is displayed on the resource home page, along with the subscription ID, resource name, etc.). Put it in `.env` as `AZURE_API_KEY`.
|
||||
|
||||
#### 4. Complete your `.env`
|
||||
`OPENAI_API_KEY` must NOT be empty (use any placeholder like `"dummy"`).
|
||||
|
||||
Example of good configured `.env`:
|
||||
|
||||
```bash
|
||||
OPENAI_API_KEY="dummy"
|
||||
AZURE_API_KEY="your_subscription_api_key"
|
||||
AZURE_API_BASE="https://<your-resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview"
|
||||
# Optional (if your setup expects it):
|
||||
# AZURE_API_VERSION="2025-01-01-preview"
|
||||
|
||||
ANTHROPIC_API_KEY=""
|
||||
OLLAMA=""
|
||||
PROMPT_TOOLKIT_NO_CPR=1
|
||||
```
|
||||
|
||||
#### 5. Start CAI and select the model
|
||||
Launch CAI and select the Azure model:
|
||||
|
||||
```vbnet
|
||||
CAI> /model azure/<model-name>
|
||||
╭─────────────────────────────────────────────────── Model Changed ────────────────────────────────────────────────────╮
|
||||
│ Model changed to: azure/<model-name> │
|
||||
│ Note: This will take effect on the next agent interaction │
|
||||
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
||||
```
|
||||
|
||||
From this point you are interacting with your Azure-hosted OpenAI model.
|
||||
|
||||
> ⚠️ Remember: you must select the model each time you start CAI.
|
||||
|
||||
|
||||
> EXTRA configuration:
|
||||
You can set the variable `CAI_MODEL` to avoid the need for repeated model setup during initialization.
|
||||
|
||||
```bash
|
||||
CAI_MODEL=azure/<model-name-deployed>
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- 404 or “deployment not found”: Ensure you have correctly copied the URL of the deployed model.
|
||||
|
||||
Error example:
|
||||
```sh
|
||||
ERROR:cai.cli:Error in main loop: litellm.APIError: AzureException APIError - Resource not found
|
||||
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
|
||||
```
|
||||
|
||||
- 401: verify `AZURE_API_KEY` and that your region has access to the chosen model.
|
||||
|
||||
Error example:
|
||||
```sh
|
||||
ERROR:cai.cli:Error in main loop: litellm.AuthenticationError: AzureException AuthenticationError - Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.
|
||||
openai.AuthenticationError: Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}}
|
||||
```
|
||||
|
||||
- Time-outs / rate limits: check Azure usage and quota.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# Ollama Configuration
|
||||
|
||||
#### [Ollama Integration](https://ollama.com/)
|
||||
For local models using Ollama, add the following to your .env:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=qwen2.5:72b
|
||||
OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint
|
||||
```
|
||||
|
||||
Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# OpenRouter Configuration
|
||||
|
||||
#### [OpenRouter Integration](https://openrouter.ai/)
|
||||
|
||||
To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=openrouter/meta-llama/llama-4-maverick
|
||||
OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
||||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||
```
|
||||
|
|
@ -21,8 +21,13 @@ theme:
|
|||
nav:
|
||||
- Intro: index.md
|
||||
- Installation: cai_installation.md
|
||||
- Quickstart: cai_quickstart.md
|
||||
- List of Models: cai_list_of_models.md
|
||||
- Quickstart: cai_quickstart.md
|
||||
- List of Models:
|
||||
- Available Models: cai_list_of_models.md
|
||||
- Providers:
|
||||
- OpenRouter: providers/openrouter.md
|
||||
- Ollama: providers/ollama.md
|
||||
- Azure OpenAI: providers/azure.md
|
||||
- Architecture: cai_architecture.md
|
||||
- Development: cai_development.md
|
||||
- Start Building:
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from collections.abc import AsyncIterator, Iterable
|
|||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, Literal, cast, overload
|
||||
|
||||
import uuid
|
||||
import litellm
|
||||
import tiktoken
|
||||
from openai import NOT_GIVEN, AsyncOpenAI, AsyncStream, NotGiven
|
||||
|
|
@ -969,6 +970,15 @@ class OpenAIChatCompletionsModel(Model):
|
|||
if not hasattr(self, "_pending_tool_calls"):
|
||||
self._pending_tool_calls = {}
|
||||
|
||||
|
||||
# Fix Google Gemini OpenAI compatibility issues.
|
||||
# When using the OpenAI-compatible API to call tools with Google Gemini
|
||||
# tool_call.id is returned as an empty string.
|
||||
if "openai/gemini" in os.getenv("CAI_MODEL"):
|
||||
for tool_call in assistant_msg.tool_calls:
|
||||
if tool_call.id is None or tool_call.id == "":
|
||||
tool_call.id = uuid.uuid4().hex[:16]
|
||||
|
||||
for tool_call in assistant_msg.tool_calls:
|
||||
# Handle empty arguments before storing
|
||||
tool_args = tool_call.function.arguments
|
||||
|
|
|
|||
Loading…
Reference in New Issue