mirror of https://github.com/aliasrobotics/cai.git
docs: Azure OpenAI update; create 'Providers' section in mkdocs.yml and specify configuration for CAI to work correctly (#264) (#273)
docs: Azure OpenAI update; create 'Providers' section in mkdocs.yml and specify configuration for CAI to work correctly (aliasrobotics/cai#264)
This commit is contained in:
parent
46c25e7f17
commit
c494f95664
16
README.md
16
README.md
|
|
@ -749,6 +749,22 @@ OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
||||||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Azure OpenAI
|
||||||
|
|
||||||
|
The Cybersecurity AI (CAI) platform integrates seamlessly with Azure OpenAI, enabling organizations to run CAI against enterprise-hosted models (e.g., gpt-4o). This pathway is ideal for teams that must operate within Azure governance while leveraging advanced model capabilities.
|
||||||
|
To enable Azure OpenAI support in CAI, configure your environment by adding the following entries to your .env. This ensures CAI can reach your Azure deployment endpoint and authenticate correctly.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
CAI_AGENT_TYPE=redteam_agent
|
||||||
|
CAI_MODEL=azure/<model-name-deployed>
|
||||||
|
# Required: keep non-empty even when using Azure
|
||||||
|
OPENAI_API_KEY=dummy
|
||||||
|
# Azure credentials and endpoint
|
||||||
|
AZURE_API_KEY=<your-azure-openai-key>
|
||||||
|
AZURE_API_BASE=https://<resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
### MCP
|
### MCP
|
||||||
|
|
||||||
CAI supports the Model Context Protocol (MCP) for integrating external tools and services with AI agents. MCP is supported via two transport mechanisms:
|
CAI supports the Model Context Protocol (MCP) for integrating external tools and services with AI agents. MCP is supported via two transport mechanisms:
|
||||||
|
|
|
||||||
|
|
@ -7,25 +7,4 @@ CAI supports **over 300 models**, thanks to its integration with [LiteLLM](https
|
||||||
- **DeepSeek**: DeepSeek V3, DeepSeek R1
|
- **DeepSeek**: DeepSeek V3, DeepSeek R1
|
||||||
- **Ollama**: Qwen2.5 72B, Qwen2.5 14B, and more
|
- **Ollama**: Qwen2.5 72B, Qwen2.5 14B, and more
|
||||||
|
|
||||||
CAI is also compatibile with other platforms like OpenRouter and Ollama. Below you’ll find some configurations to help you get started.
|
CAI is also compatibile with other platforms like OpenRouter, Ollama and Azure OpenAI. At providers section you’ll find some configurations to help you get started.
|
||||||
|
|
||||||
#### [OpenRouter Integration](https://openrouter.ai/)
|
|
||||||
|
|
||||||
To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
CAI_MODEL=openrouter/meta-llama/llama-4-maverick
|
|
||||||
OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
|
||||||
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
#### [Ollama Integration](https://ollama.com/)
|
|
||||||
For local models using Ollama, add the following to your .env:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
CAI_MODEL=qwen2.5:72b
|
|
||||||
OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint
|
|
||||||
```
|
|
||||||
|
|
||||||
Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance.
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,84 @@
|
||||||
|
# Azure OpenAI configuration
|
||||||
|
|
||||||
|
> This guide shows how to run CAI against Azure-hosted OpenAI's models
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
- Azure subscription with **Azure OpenAI** access.
|
||||||
|
- A **deployed model** in Azure AI Portal (e.g., a deployment named `gpt-4o`).
|
||||||
|
See Microsoft docs on creating the resource & deploying models.
|
||||||
|
- [Create resource & deploy](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/create-resource)
|
||||||
|
- [Working with models](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/working-with-models)
|
||||||
|
|
||||||
|
#### 1. Deploy the base model
|
||||||
|
In Azure AI Portal, go to **Deployments** and deploy the requested base model (e.g., gpt-4o).
|
||||||
|
|
||||||
|
#### 2. Get the deployment URL
|
||||||
|
From **Deployments**, select your deployment and copy the endpoint in this form:
|
||||||
|
|
||||||
|
`https://<your-resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview`
|
||||||
|
|
||||||
|
Set this value as `AZURE_API_BASE` in your `.env`.
|
||||||
|
**Note:** CAI uses the OpenAI SDK style `base_url + /chat/completions`. For Azure, providing the full endpoint above (including `chat/completions?api-version=...`) ensures correct routing.
|
||||||
|
|
||||||
|
#### 3. Get your API key
|
||||||
|
From your Azure OpenAI resource home page (it is displayed on the resource home page, along with the subscription ID, resource name, etc.). Put it in `.env` as `AZURE_API_KEY`.
|
||||||
|
|
||||||
|
#### 4. Complete your `.env`
|
||||||
|
`OPENAI_API_KEY` must NOT be empty (use any placeholder like `"dummy"`).
|
||||||
|
|
||||||
|
Example of good configured `.env`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
OPENAI_API_KEY="dummy"
|
||||||
|
AZURE_API_KEY="your_subscription_api_key"
|
||||||
|
AZURE_API_BASE="https://<your-resource>.openai.azure.com/openai/deployments/<deployment-name>/chat/completions?api-version=2025-01-01-preview"
|
||||||
|
# Optional (if your setup expects it):
|
||||||
|
# AZURE_API_VERSION="2025-01-01-preview"
|
||||||
|
|
||||||
|
ANTHROPIC_API_KEY=""
|
||||||
|
OLLAMA=""
|
||||||
|
PROMPT_TOOLKIT_NO_CPR=1
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 5. Start CAI and select the model
|
||||||
|
Launch CAI and select the Azure model:
|
||||||
|
|
||||||
|
```vbnet
|
||||||
|
CAI> /model azure/<model-name>
|
||||||
|
╭─────────────────────────────────────────────────── Model Changed ────────────────────────────────────────────────────╮
|
||||||
|
│ Model changed to: azure/<model-name> │
|
||||||
|
│ Note: This will take effect on the next agent interaction │
|
||||||
|
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
|
||||||
|
```
|
||||||
|
|
||||||
|
From this point you are interacting with your Azure-hosted OpenAI model.
|
||||||
|
|
||||||
|
> ⚠️ Remember: you must select the model each time you start CAI.
|
||||||
|
|
||||||
|
|
||||||
|
> EXTRA configuration:
|
||||||
|
You can set the variable `CAI_MODEL` to avoid the need for repeated model setup during initialization.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
CAI_MODEL=azure/<model-name-deployed>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
- 404 or “deployment not found”: Ensure you have correctly copied the URL of the deployed model.
|
||||||
|
|
||||||
|
Error example:
|
||||||
|
```sh
|
||||||
|
ERROR:cai.cli:Error in main loop: litellm.APIError: AzureException APIError - Resource not found
|
||||||
|
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
|
||||||
|
```
|
||||||
|
|
||||||
|
- 401: verify `AZURE_API_KEY` and that your region has access to the chosen model.
|
||||||
|
|
||||||
|
Error example:
|
||||||
|
```sh
|
||||||
|
ERROR:cai.cli:Error in main loop: litellm.AuthenticationError: AzureException AuthenticationError - Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.
|
||||||
|
openai.AuthenticationError: Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}}
|
||||||
|
```
|
||||||
|
|
||||||
|
- Time-outs / rate limits: check Azure usage and quota.
|
||||||
|
|
@ -0,0 +1,11 @@
|
||||||
|
# Ollama Configuration
|
||||||
|
|
||||||
|
#### [Ollama Integration](https://ollama.com/)
|
||||||
|
For local models using Ollama, add the following to your .env:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
CAI_MODEL=qwen2.5:72b
|
||||||
|
OLLAMA_API_BASE=http://localhost:8000/v1 # note, maybe you have a different endpoint
|
||||||
|
```
|
||||||
|
|
||||||
|
Make sure that the Ollama server is running and accessible at the specified base URL. You can swap the model with any other supported by your local Ollama instance.
|
||||||
|
|
@ -0,0 +1,11 @@
|
||||||
|
# OpenRouter Configuration
|
||||||
|
|
||||||
|
#### [OpenRouter Integration](https://openrouter.ai/)
|
||||||
|
|
||||||
|
To enable OpenRouter support in CAI, you need to configure your environment by adding specific entries to your `.env` file. This setup ensures that CAI can interact with the OpenRouter API, facilitating the use of sophisticated models like Meta-LLaMA. Here’s how you can configure it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
CAI_MODEL=openrouter/meta-llama/llama-4-maverick
|
||||||
|
OPENROUTER_API_KEY=<sk-your-key> # note, add yours
|
||||||
|
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
|
||||||
|
```
|
||||||
|
|
@ -21,8 +21,13 @@ theme:
|
||||||
nav:
|
nav:
|
||||||
- Intro: index.md
|
- Intro: index.md
|
||||||
- Installation: cai_installation.md
|
- Installation: cai_installation.md
|
||||||
- Quickstart: cai_quickstart.md
|
- Quickstart: cai_quickstart.md
|
||||||
- List of Models: cai_list_of_models.md
|
- List of Models:
|
||||||
|
- Available Models: cai_list_of_models.md
|
||||||
|
- Providers:
|
||||||
|
- OpenRouter: providers/openrouter.md
|
||||||
|
- Ollama: providers/ollama.md
|
||||||
|
- Azure OpenAI: providers/azure.md
|
||||||
- Architecture: cai_architecture.md
|
- Architecture: cai_architecture.md
|
||||||
- Development: cai_development.md
|
- Development: cai_development.md
|
||||||
- Start Building:
|
- Start Building:
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue