diff --git a/README.md b/README.md index 47e7839e..490e82f6 100644 --- a/README.md +++ b/README.md @@ -749,6 +749,22 @@ OPENROUTER_API_KEY= # note, add yours OPENROUTER_API_BASE=https://openrouter.ai/api/v1 ``` +### Azure OpenAI + +The Cybersecurity AI (CAI) platform integrates seamlessly with Azure OpenAI, enabling organizations to run CAI against enterprise-hosted models (e.g., gpt-4o). This pathway is ideal for teams that must operate within Azure governance while leveraging advanced model capabilities. +To enable Azure OpenAI support in CAI, configure your environment by adding the following entries to your .env. This ensures CAI can reach your Azure deployment endpoint and authenticate correctly. + +```bash +CAI_AGENT_TYPE=redteam_agent +CAI_MODEL=azure/ +# Required: keep non-empty even when using Azure +OPENAI_API_KEY=dummy +# Azure credentials and endpoint +AZURE_API_KEY= +AZURE_API_BASE=https://.openai.azure.com/openai/deployments//chat/completions?api-version=2025-01-01-preview +``` + + ### MCP CAI supports the Model Context Protocol (MCP) for integrating external tools and services with AI agents. MCP is supported via two transport mechanisms: diff --git a/docs/cai_list_of_models.md b/docs/cai_list_of_models.md index d431fb7b..b1baf48b 100644 --- a/docs/cai_list_of_models.md +++ b/docs/cai_list_of_models.md @@ -7,25 +7,4 @@ CAI supports **over 300 models**, thanks to its integration with [LiteLLM](https - **DeepSeek**: DeepSeek V3, DeepSeek R1 - **Ollama**: Qwen2.5 72B, Qwen2.5 14B, and more -CAI is also compatibile with other platforms like OpenRouter and Ollama. Below you’ll find some configurations to help you get started. - -#### [OpenRouter Integration](https://openrouter.ai/) - -To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it: - -```bash -CAI_MODEL=openrouter/meta-llama/llama-4-maverick -OPENROUTER_API_KEY= # note, add yours -OPENROUTER_API_BASE=https://openrouter.ai/api/v1 -``` - - -#### [Ollama Integration](https://ollama.com/) -For local models using Ollama, add the following to your .env: - -```bash -CAI_MODEL=qwen2.5:72b -OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint -``` - -Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance. +CAI is also compatibile with other platforms like OpenRouter, Ollama and Azure OpenAI. At providers section you’ll find some configurations to help you get started. diff --git a/docs/providers/azure.md b/docs/providers/azure.md new file mode 100644 index 00000000..5f373ec5 --- /dev/null +++ b/docs/providers/azure.md @@ -0,0 +1,84 @@ +# Azure OpenAI configuration + +> This guide shows how to run CAI against Azure-hosted OpenAI's models + +## Prerequisites +- Azure subscription with **Azure OpenAI** access. +- A **deployed model** in Azure AI Portal (e.g., a deployment named `gpt-4o`). + See Microsoft docs on creating the resource & deploying models. + - [Create resource & deploy](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/create-resource) + - [Working with models](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/working-with-models) + +#### 1. Deploy the base model +In Azure AI Portal, go to **Deployments** and deploy the requested base model (e.g., gpt-4o). + +#### 2. Get the deployment URL +From **Deployments**, select your deployment and copy the endpoint in this form: + +`https://.openai.azure.com/openai/deployments//chat/completions?api-version=2025-01-01-preview` + +Set this value as `AZURE_API_BASE` in your `.env`. +**Note:** CAI uses the OpenAI SDK style `base_url + /chat/completions`. For Azure, providing the full endpoint above (including `chat/completions?api-version=...`) ensures correct routing. + +#### 3. Get your API key +From your Azure OpenAI resource home page (it is displayed on the resource home page, along with the subscription ID, resource name, etc.). Put it in `.env` as `AZURE_API_KEY`. + +#### 4. Complete your `.env` +`OPENAI_API_KEY` must NOT be empty (use any placeholder like `"dummy"`). + +Example of good configured `.env`: + +```bash +OPENAI_API_KEY="dummy" +AZURE_API_KEY="your_subscription_api_key" +AZURE_API_BASE="https://.openai.azure.com/openai/deployments//chat/completions?api-version=2025-01-01-preview" +# Optional (if your setup expects it): +# AZURE_API_VERSION="2025-01-01-preview" + +ANTHROPIC_API_KEY="" +OLLAMA="" +PROMPT_TOOLKIT_NO_CPR=1 +``` + +#### 5. Start CAI and select the model +Launch CAI and select the Azure model: + +```vbnet +CAI> /model azure/ +╭─────────────────────────────────────────────────── Model Changed ────────────────────────────────────────────────────╮ +│ Model changed to: azure/ │ +│ Note: This will take effect on the next agent interaction │ +╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ +``` + +From this point you are interacting with your Azure-hosted OpenAI model. + +> ⚠️ Remember: you must select the model each time you start CAI. + + +> EXTRA configuration: +You can set the variable `CAI_MODEL` to avoid the need for repeated model setup during initialization. + +```bash +CAI_MODEL=azure/ +``` + +## Troubleshooting + +- 404 or “deployment not found”: Ensure you have correctly copied the URL of the deployed model. + +Error example: +```sh +ERROR:cai.cli:Error in main loop: litellm.APIError: AzureException APIError - Resource not found +openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}} +``` + +- 401: verify `AZURE_API_KEY` and that your region has access to the chosen model. + +Error example: +```sh +ERROR:cai.cli:Error in main loop: litellm.AuthenticationError: AzureException AuthenticationError - Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource. +openai.AuthenticationError: Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}} +``` + +- Time-outs / rate limits: check Azure usage and quota. \ No newline at end of file diff --git a/docs/providers/ollama.md b/docs/providers/ollama.md new file mode 100644 index 00000000..1ff39758 --- /dev/null +++ b/docs/providers/ollama.md @@ -0,0 +1,11 @@ +# Ollama Configuration + +#### [Ollama Integration](https://ollama.com/) +For local models using Ollama, add the following to your .env: + +```bash +CAI_MODEL=qwen2.5:72b +OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint +``` + +Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance. diff --git a/docs/providers/openrouter.md b/docs/providers/openrouter.md new file mode 100644 index 00000000..cf96b854 --- /dev/null +++ b/docs/providers/openrouter.md @@ -0,0 +1,11 @@ +# OpenRouter Configuration + +#### [OpenRouter Integration](https://openrouter.ai/) + +To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it: + +```bash +CAI_MODEL=openrouter/meta-llama/llama-4-maverick +OPENROUTER_API_KEY= # note, add yours +OPENROUTER_API_BASE=https://openrouter.ai/api/v1 +``` diff --git a/mkdocs.yml b/mkdocs.yml index c8ca446b..30dcb8d5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -21,8 +21,13 @@ theme: nav: - Intro: index.md - Installation: cai_installation.md - - Quickstart: cai_quickstart.md - - List of Models: cai_list_of_models.md + - Quickstart: cai_quickstart.md + - List of Models: + - Available Models: cai_list_of_models.md + - Providers: + - OpenRouter: providers/openrouter.md + - Ollama: providers/ollama.md + - Azure OpenAI: providers/azure.md - Architecture: cai_architecture.md - Development: cai_development.md - Start Building: