mirror of https://github.com/aliasrobotics/cai.git
4546033b8e
Adds cai.util.nim_rotation which cycles NVIDIA_NIM_API_KEY_1..N per request, keeping under NIM per-key rate limits (~40 req/min). Rotation applies in the OpenAI-compatible LLM path and the direct httpx completion path when the API base is a NIM endpoint (api.nvidia.com). Also defaults cache_write_tokens to 0 in the streamed usage object: litellm CustomResponseUsage now requires the field while NIM usage payloads omit it, which previously aborted streaming at end of response. |
||
|---|---|---|
| .. | ||
| agents | ||
| api | ||
| cli | ||
| commands | ||
| continuous_ops | ||
| core | ||
| ctfs | ||
| fixtures | ||
| integration | ||
| mcp | ||
| others | ||
| prompts | ||
| refusals | ||
| repl | ||
| sdk | ||
| tools | ||
| tracing | ||
| util | ||
| voice | ||
| README.md | ||
| __init__.py | ||
| conftest.py | ||
| fake_model.py | ||
| helpers.py | ||
| test_cli_print_deduplication.py | ||
| test_compact_command.py | ||
| test_config_loader_agents.py | ||
| test_unified_pattern.py | ||
| testing_processor.py | ||
README.md
Tests
Before running any tests, make sure you have uv installed (and ideally run make sync after).
Running tests
make tests
Snapshots
We use inline-snapshots for some tests. If your code adds new snapshot tests or breaks existing ones, you can fix/create them. After fixing/creating snapshots, run make tests again to verify the tests pass.
Fixing snapshots
make snapshots-fix
Creating snapshots
make snapshots-update