mirror of https://github.com/aliasrobotics/cai.git
docs: Azure OpenAI update; create 'Providers' section in mkdocs.yml and specify configuration for CAI to work correctly (#264) (#273)
docs: Azure OpenAI update; create 'Providers' section in mkdocs.yml and specify configuration for CAI to work correctly (aliasrobotics/cai#264)
This commit is contained in:
Javier Garcia
GitHub
parent
46c25e7f17
commit
c494f95664
16
README.md
16
README.md
|
|
@ -749,6 +749,22 @@ OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
|||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||
```
|
||||
|
||||
### Azure OpenAI
|
||||
|
||||
The Cybersecurity AI (CAI) platform integrates seamlessly with Azure OpenAI, enabling organizations to run CAI against enterprise-hosted models (e.g., gpt-4o). This pathway is ideal for teams that must operate within Azure governance while leveraging advanced model capabilities.
|
||||
To enable Azure OpenAI support in CAI, configure your environment by adding the following entries to your .env. This ensures CAI can reach your Azure deployment endpoint and authenticate correctly.
|
||||
|
||||
```bash
|
||||
CAI_AGENT_TYPE=redteam_agent
|
||||
CAI_MODEL=azure/<model-name-deployed>
|
||||
# Required: keep non-empty even when using Azure
|
||||
OPENAI_API_KEY=dummy
|
||||
# Azure credentials and endpoint
|
||||
AZURE_API_KEY=<your-azure-openai-key>
|
||||
AZURE_API_BASE=https://<resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview
|
||||
```
|
||||
|
||||
|
||||
### MCP
|
||||
|
||||
CAI supports the Model Context Protocol (MCP) for integrating external tools and services with AI agents. MCP is supported via two transport mechanisms:
|
||||
|
|
|
|||
|
|
@ -7,25 +7,4 @@ CAI supports **over 300 models**, thanks to its integration with [LiteLLM](https
|
|||
- **DeepSeek**: DeepSeek V3, DeepSeek R1
|
||||
- **Ollama**: Qwen2.5 72B, Qwen2.5 14B, and more
|
||||
|
||||
CAI is also compatibile with other platforms like OpenRouter and Ollama. Below you’ll find some configurations to help you get started.
|
||||
|
||||
#### [OpenRouter Integration](https://openrouter.ai/)
|
||||
|
||||
To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=openrouter/meta-llama/llama-4-maverick
|
||||
OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
||||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||
```
|
||||
|
||||
|
||||
#### [Ollama Integration](https://ollama.com/)
|
||||
For local models using Ollama, add the following to your .env:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=qwen2.5:72b
|
||||
OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint
|
||||
```
|
||||
|
||||
Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance.
|
||||
CAI is also compatibile with other platforms like OpenRouter, Ollama and Azure OpenAI. At providers section you’ll find some configurations to help you get started.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,84 @@
|
|||
# Azure OpenAI configuration
|
||||
|
||||
> This guide shows how to run CAI against Azure-hosted OpenAI's models
|
||||
|
||||
## Prerequisites
|
||||
- Azure subscription with **Azure OpenAI** access.
|
||||
- A **deployed model** in Azure AI Portal (e.g., a deployment named `gpt-4o`).
|
||||
See Microsoft docs on creating the resource & deploying models.
|
||||
- [Create resource & deploy](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/create-resource)
|
||||
- [Working with models](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/working-with-models)
|
||||
|
||||
#### 1. Deploy the base model
|
||||
In Azure AI Portal, go to **Deployments** and deploy the requested base model (e.g., gpt-4o).
|
||||
|
||||
#### 2. Get the deployment URL
|
||||
From **Deployments**, select your deployment and copy the endpoint in this form:
|
||||
|
||||
`https://<your-resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview`
|
||||
|
||||
Set this value as `AZURE_API_BASE` in your `.env`.
|
||||
**Note:** CAI uses the OpenAI SDK style `base_url + /chat/completions`. For Azure, providing the full endpoint above (including `chat/completions?api-version=...`) ensures correct routing.
|
||||
|
||||
#### 3. Get your API key
|
||||
From your Azure OpenAI resource home page (it is displayed on the resource home page, along with the subscription ID, resource name, etc.). Put it in `.env` as `AZURE_API_KEY`.
|
||||
|
||||
#### 4. Complete your `.env`
|
||||
`OPENAI_API_KEY` must NOT be empty (use any placeholder like `"dummy"`).
|
||||
|
||||
Example of good configured `.env`:
|
||||
|
||||
```bash
|
||||
OPENAI_API_KEY="dummy"
|
||||
AZURE_API_KEY="your_subscription_api_key"
|
||||
AZURE_API_BASE="https://<your-resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview"
|
||||
# Optional (if your setup expects it):
|
||||
# AZURE_API_VERSION="2025-01-01-preview"
|
||||
|
||||
ANTHROPIC_API_KEY=""
|
||||
OLLAMA=""
|
||||
PROMPT_TOOLKIT_NO_CPR=1
|
||||
```
|
||||
|
||||
#### 5. Start CAI and select the model
|
||||
Launch CAI and select the Azure model:
|
||||
|
||||
```vbnet
|
||||
CAI> /model azure/<model-name>
|
||||
╭─────────────────────────────────────────────────── Model Changed ────────────────────────────────────────────────────╮
|
||||
│ Model changed to: azure/<model-name> │
|
||||
│ Note: This will take effect on the next agent interaction │
|
||||
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
||||
```
|
||||
|
||||
From this point you are interacting with your Azure-hosted OpenAI model.
|
||||
|
||||
> ⚠️ Remember: you must select the model each time you start CAI.
|
||||
|
||||
|
||||
> EXTRA configuration:
|
||||
You can set the variable `CAI_MODEL` to avoid the need for repeated model setup during initialization.
|
||||
|
||||
```bash
|
||||
CAI_MODEL=azure/<model-name-deployed>
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- 404 or “deployment not found”: Ensure you have correctly copied the URL of the deployed model.
|
||||
|
||||
Error example:
|
||||
```sh
|
||||
ERROR:cai.cli:Error in main loop: litellm.APIError: AzureException APIError - Resource not found
|
||||
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
|
||||
```
|
||||
|
||||
- 401: verify `AZURE_API_KEY` and that your region has access to the chosen model.
|
||||
|
||||
Error example:
|
||||
```sh
|
||||
ERROR:cai.cli:Error in main loop: litellm.AuthenticationError: AzureException AuthenticationError - Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.
|
||||
openai.AuthenticationError: Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}}
|
||||
```
|
||||
|
||||
- Time-outs / rate limits: check Azure usage and quota.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# Ollama Configuration
|
||||
|
||||
#### [Ollama Integration](https://ollama.com/)
|
||||
For local models using Ollama, add the following to your .env:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=qwen2.5:72b
|
||||
OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint
|
||||
```
|
||||
|
||||
Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# OpenRouter Configuration
|
||||
|
||||
#### [OpenRouter Integration](https://openrouter.ai/)
|
||||
|
||||
To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it:
|
||||
|
||||
```bash
|
||||
CAI_MODEL=openrouter/meta-llama/llama-4-maverick
|
||||
OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
||||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||
```
|
||||
|
|
@ -21,8 +21,13 @@ theme:
|
|||
nav:
|
||||
- Intro: index.md
|
||||
- Installation: cai_installation.md
|
||||
- Quickstart: cai_quickstart.md
|
||||
- List of Models: cai_list_of_models.md
|
||||
- Quickstart: cai_quickstart.md
|
||||
- List of Models:
|
||||
- Available Models: cai_list_of_models.md
|
||||
- Providers:
|
||||
- OpenRouter: providers/openrouter.md
|
||||
- Ollama: providers/ollama.md
|
||||
- Azure OpenAI: providers/azure.md
|
||||
- Architecture: cai_architecture.md
|
||||
- Development: cai_development.md
|
||||
- Start Building:
|
||||
|
|
|
|||
Loading…
Reference in New Issue