diff --git a/skills/vastai-sdk/SKILL.md b/skills/vastai-sdk/SKILL.md new file mode 100644 index 00000000..26e9e07d --- /dev/null +++ b/skills/vastai-sdk/SKILL.md @@ -0,0 +1,237 @@ +--- +name: vastai-sdk +description: Vast.ai Python SDK for renting on-demand GPU compute — train models, run inference, and manage cloud instances programmatically. Use when writing Python code that provisions or interacts with Vast.ai GPU instances, volumes, or serverless endpoints. +allowed-tools: Python(vastai:*) +compatibility: Python 3.9+ +metadata: + author: vast-ai +--- + +# Vast.ai Python SDK (`vastai` / `vastai_sdk`) + +The `vastai` package provides a Python SDK for managing GPU instances, volumes, serverless endpoints, and billing on Vast.ai. The `vastai_sdk` package is a backward-compatibility shim that re-exports `vastai`. + +## Installation + +```bash +pip install vastai +``` + +For serverless and async support: +```bash +pip install "vastai[serverless]" +``` + +## Authentication + +The SDK reads the API key from `~/.vast_api_key` by default. You can also pass it explicitly: + +```python +from vastai import VastAI +vast = VastAI() # reads ~/.vast_api_key +vast = VastAI(api_key="YOUR_API_KEY") # explicit key +``` + +Get your API key from https://console.vast.ai/manage-keys/ + +## Backward Compatibility + +The old `vastai_sdk` import still works: + +```python +from vastai_sdk import VastAI # equivalent to: from vastai import VastAI +``` + +## VastAI Class (High-Level SDK) + +```python +from vastai import VastAI +vast = VastAI(api_key=None, server_url=None, retry=3, raw=False, quiet=False) +``` + +### Instance Management + +```python +# List all your instances +instances = vast.show_instances() + +# Get a single instance +instance = vast.show_instance(id=12345) + +# Search GPU offers +offers = vast.search_offers(query='gpu_name=RTX_4090 num_gpus>=4 reliability>0.99') + +# Create an instance from an offer +result = vast.create_instance(id=, image="pytorch/pytorch:latest", disk=50) + +# Lifecycle +vast.start_instance(id=12345) +vast.stop_instance(id=12345) +vast.reboot_instance(id=12345) +vast.destroy_instance(id=12345) + +# Label an instance +vast.label_instance(id=12345, label="my-training-run") + +# Get SSH connection string +ssh_url = vast.ssh_url(id=12345) # returns "ssh -p PORT user@host" +scp_url = vast.scp_url(id=12345) # returns scp-compatible URL +``` + +### Search + +```python +# Search GPU offers (use help(vast.search_offers) for full query syntax) +offers = vast.search_offers(query='gpu_name=RTX_3090 num_gpus>=2') + +# Search volume offers +volumes = vast.search_volumes(query='...') + +# Search network volumes +net_vols = vast.search_network_volumes() + +# Search templates +templates = vast.search_templates() + +# Search invoices +invoices = vast.search_invoices() +``` + +### Serverless Deployments + +```python +# List all deployments +deployments = vast.show_deployments() + +# Get a deployment +deployment = vast.show_deployment(id=42) + +# Delete a deployment +vast.delete_deployment(id=42) +``` + +### Machine Management (Hosting) + +```python +machines = vast.show_machines() +machine = vast.show_machine(id=10) +vast.list_machine(id=10, price_gpu=0.30) +vast.unlist_machine(id=10) +``` + +### SSH Keys + +```python +keys = vast.show_ssh_keys() +vast.create_ssh_key(ssh_key="ssh-rsa AAAA...") +vast.delete_ssh_key(id=5) +``` + +### Team Management + +```python +members = vast.show_members() +vast.invite_member(email="user@example.com", role="developer") +vast.remove_member(id=7) +``` + +## SyncClient (Low-Level Sync) + +`SyncClient` provides typed, synchronous access to GPU offers and instances. + +```python +from vastai import SyncClient + +client = SyncClient(api_key="YOUR_API_KEY") # or reads ~/.vast_api_key + +# Search offers with structured filters +offers = client.search( + num_gpus=2, + gpu_name="RTX_4090", + min_reliability=0.99, + max_dph_total=2.0, +) + +# Create an instance +instance = client.create_instance( + offer_id=, + image="pytorch/pytorch:latest", + disk_gb=50, +) + +# List your instances +instances = client.show_instances() # returns list[SyncInstance] + +# Destroy an instance +client.destroy_instance(instance_or_id=12345) +``` + +## AsyncClient (Low-Level Async) + +`AsyncClient` provides async access to GPU offers and instances. Use as an async context manager. + +```python +import asyncio +from vastai import AsyncClient + +async def main(): + async with AsyncClient(api_key="YOUR_API_KEY") as client: + # Search offers + offers = await client.search(num_gpus=1, gpu_name="A100") + + # Create instance + instance = await client.create_instance(offer_id=, image="ubuntu:22.04") + + # List instances + instances = await client.show_instances() # returns list[AsyncInstance] + + # Destroy instance + await client.destroy_instance(instance_or_id=instance.id) + +asyncio.run(main()) +``` + +## Serverless Client + +For inference endpoints (requires `pip install "vastai[serverless]"`): + +```python +import asyncio +from vastai import Serverless + +async def main(): + serverless = Serverless() # reads ~/.vast_api_key + + # Get an endpoint + endpoint = await serverless.get_endpoint("my-endpoint") + + # Make a request + response = await serverless.request("/v1/completions", { + "model": "Qwen/Qwen3-8B", + "prompt": "Who are you?", + "max_tokens": 100, + "temperature": 0.7, + }) + + text = response["response"]["choices"][0]["text"] + print(text) + +asyncio.run(main()) +``` + +## Common Patterns + +```python +# Find cheapest 4x RTX 4090 and launch a job +from vastai import VastAI +vast = VastAI() + +offers = vast.search_offers(query='gpu_name=RTX_4090 num_gpus=4 reliability>0.99') +cheapest = min(offers, key=lambda o: o['dph_total']) +result = vast.create_instance(id=cheapest['id'], image="pytorch/pytorch:latest", disk=100) +print(f"Launched instance: {result['new_contract']}") + +# Use help() to explore method signatures +help(vast.search_offers) +help(vast.create_instance) +``` diff --git a/skills/vastai/SKILL.md b/skills/vastai/SKILL.md new file mode 100644 index 00000000..f0ed3349 --- /dev/null +++ b/skills/vastai/SKILL.md @@ -0,0 +1,366 @@ +--- +name: vastai +description: Vast.ai CLI to manage GPU instances, volumes, serverless endpoints, and billing. +allowed-tools: Bash(vastai:*) +compatibility: Linux, macOS +metadata: + author: vast-ai +--- + +# vastai + +Manage GPU instances, templates, volumes, serverless endpoints, SSH keys, and billing on Vast.ai. + +> Command is `vastai` (lowercase). Always use `--raw` for machine-readable JSON output. + +## Install + +```bash +# PyPI (recommended) +pip install vastai + +# Latest from GitHub +wget https://raw.githubusercontent.com/vast-ai/vast-python/master/vast.py -O vast +chmod +x vast +``` + +## Quick start + +```bash +vastai set api-key # Authenticate (one-time); Create API Key in account at https://console.vast.ai/cli +vastai show user # Verify auth + check balance +vastai create ssh-key ~/.ssh/id_ed25519.pub # Register SSH key (do BEFORE create) +vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-' +# Note the offer ID from the output +vastai create instance --image pytorch/pytorch:@vastai-automatic-tag --disk 20 --ssh --direct +# Automatically grab appropriate image tag; Response: {"success": true, "new_contract": } +vastai show instance # Poll until actual_status == "running" (see Instance status values below) +vastai ssh-url # Get SSH connection string +vastai copy local:./data/ :/workspace/ # Upload files +vastai destroy instance -y # Clean up (stops all billing; -y skips confirmation) +``` + +API key: https://console.vast.ai/cli + +## Global flags + +Available on every command: + +``` +--api-key KEY Override stored API key +--raw Output machine-readable JSON (agents should always use this) +--full Print full results (don't page with less) +--explain Show underlying API calls (useful for debugging) +--curl Show equivalent curl command +--no-color Disable colored output +--url URL Override server REST API URL +--retry RETRY Set retry limit for API calls +--version Show CLI version +``` + +## Query syntax + +Search commands accept filter expressions. Operators: `=`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `notin`. + +```bash +# Examples +'gpu_name=RTX_4090 num_gpus=1' # Exact match + numeric +'gpu_ram>=48 reliability>0.95' # Greater-than filters +'geolocation=EU dph_total<=2.0' # Region + price cap +``` + +Common filter fields: `num_gpus`, `gpu_name`, `gpu_ram`, `cpu_ram`, `disk_space`, `reliability`, `compute_cap`, `inet_up`, `inet_down`, `dph_total`, `geolocation`, `direct_port_count`, `verified`, `rentable` + +Common sort fields: `score` (default — overall value), `dlperf_usd` (DL perf per dollar), `dph_total` (price), `num_gpus`, `reliability` + +## Commands + +### Instances + +```bash +vastai show instances # List all your instances +vastai show instances-v1 # Paginated instances with full filter/sort/cols support +vastai show instances-v1 --status running loading # Filter by status +vastai show instances-v1 --gpu-name 'RTX 4090' # Filter by GPU +vastai show instances-v1 --label training # Filter by label +vastai show instances-v1 --order-by start_date desc # Sort by column +vastai show instances-v1 --cols id,status,gpu,dph # Custom columns +vastai show instance # Poll single instance (use for status checks) +vastai create instance --image pytorch/pytorch:2.4.0-cuda12.4-cudnn9-runtime --disk 20 --ssh --direct +# Response includes "new_contract": — that is your instance ID +vastai launch instance --gpu-name RTX_4090 --num-gpus 1 --image pytorch/pytorch +vastai start instance # Start stopped instance +vastai stop instance # Stop (preserves disk, no GPU charges) +vastai reboot instance # Stop + start +vastai destroy instance -y # Delete permanently (irreversible; -y required for non-interactive use) +vastai destroy instances -y # Batch delete (-y skips confirmation prompt) +vastai label instance --label "training-run-1" # Tag instance +vastai update instance # Recreate from updated template +vastai prepay instance # Deposit credits into reserved instance +vastai recycle instance # Destroy + recreate +``` + +**Recommended Vast.ai images** (use `@vastai-automatic-tag` to get the right tag for the machine automatically; browse pre-configured models at https://vast.ai/model-library): + + + +```bash +vastai/base-image:@vastai-automatic-tag # Minimal Ubuntu base +vastai/pytorch:@vastai-automatic-tag # PyTorch + CUDA +vastai/linux-desktop:@vastai-automatic-tag # Linux desktop (VNC/RDP) + +# vLLM — set model via env vars with huggingface model example +vastai create instance --image vastai/vllm:@vastai-automatic-tag --disk 40 --ssh --direct \ + --env '-e MODEL_NAME=Qwen/Qwen2.5-3B-Instruct -e HF_TOKEN=hf_xxx' + +# ComfyUI — set model checkpoint via env vars with huggingface model example +vastai create instance --image vastai/comfy:@vastai-automatic-tag --disk 40 --ssh --direct \ + --env '-e CHECKPOINT_MODEL=black-forest-labs/FLUX.1-schnell -e HF_TOKEN=hf_xxx' +``` + +**create instance flags:** +- `--image IMAGE` — Docker image +- `--disk DISK` — Local disk in GB +- `--ssh` / `--jupyter` — Connection type +- `--direct` — Faster direct connections +- `--label LABEL` — Instance label +- `--env ENV` — Env vars and port mappings, e.g. `'-e TZ=UTC -p 8080:8080'` +- `--onstart FILE` — Path to a startup script file +- `--onstart-cmd CMD` — Startup script as inline string (for longer scripts use `--onstart` or gzip+base64 encode) +- `--bid_price PRICE` — Interruptible (spot) pricing in $/hr +- `--template_hash HASH` — Create from template +- `--create-volume ID` — Attach new volume +- `--link-volume ID` — Attach existing volume +- `--cancel-unavail` — Fail if no machine available (vs. create stopped) + +**Instance status values:** + +| `actual_status` | Meaning | +|-----------------|---------| +| `null` | Provisioning | +| `created` | Instance created, not yet provisioned | +| `loading` | Image downloading / container starting | +| `running` | Active — GPU charges apply | +| `stopped` | Halted — disk charges only | +| `frozen` | Paused with memory — GPU charges apply | +| `exited` | Container process exited unexpectedly | +| `rebooting` | Restarting (transient) | +| `unknown` | No recent heartbeat from host | +| `offline` | Host disconnected from Vast servers | + +> **Poll loop warning:** If `actual_status` becomes `exited`, `unknown`, or `offline` it will never reach `running`. Always add a timeout and error branch — otherwise your script loops forever while disk charges accrue. Destroy and retry with a different offer. + +> **Charges:** Storage charges begin at creation. GPU charges begin when status reaches `running`. + +### Search + +```bash +vastai search offers # Default: verified, on-demand, sorted by score +vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1' -o 'dlperf_usd-' +vastai search offers 'num_gpus>=4 reliability>0.99' -o 'num_gpus-' +vastai search offers --type bid # Interruptible (spot) pricing +vastai search offers --type reserved # Reserved pricing +vastai search offers -n 'gpu_name=H100_SXM' # No default filters +vastai search volumes # Search volume offers +vastai search templates "pytorch" # Search templates +vastai search benchmarks # Search benchmarks +vastai search invoices # Search invoice history +``` + +**search offers flags:** `--type on-demand|reserved|bid`, `--order/-o FIELD[-]`, `--limit`, `--storage GB`, `--no-default/-n` + +### SSH & File Transfer + +```bash +vastai ssh-url # Get ssh:// connection URL +vastai scp-url # Get scp:// URL +vastai attach ssh "ssh-ed25519 AAAA..." # Attach SSH key to instance +vastai detach ssh # Remove SSH key (ssh_key_id from show ssh-keys) +vastai show ssh-keys # List account SSH keys +vastai create ssh-key ~/.ssh/id_ed25519.pub # Add SSH key from file (do BEFORE create instance) +vastai create ssh-key # Generate new key if you don't have one +vastai create ssh-key "ssh-ed25519 AAAA..." # Add SSH key inline +vastai delete ssh-key # Remove SSH key from account +vastai update ssh-key "ssh-ed25519 AAAA..." # Update SSH key value +``` + +**Note:** `ssh-url` returns a connection string — it does not open an interactive session. Use `$(vastai ssh-url )` to extract the URL, or parse `--raw` JSON output. + +### File Copy + +```bash +vastai copy # Copy between instance and local +vastai copy local:./data/ :/workspace/data/ # Local → instance (preferred format) +vastai copy :/workspace/results/ local:./results/ # Instance → local +vastai copy :/workspace/ :/workspace/ # Instance → instance +# Legacy format also works: vastai copy 12345:./data ./local-data +vastai cloud copy --src ./data --dst s3://bucket/path \ + --instance 12345 --connection \ + --transfer "Instance To Cloud" # To cloud storage (add connection in UI on settings page) +vastai cancel copy # Cancel in-progress copy +``` + +**cloud copy flags:** `--src`, `--dst`, `--instance`, `--connection`, `--transfer` + +### Logs & Exec + +```bash +vastai logs # Container logs (last 1000 lines) +vastai logs --tail 100 # Last 100 lines +vastai logs --filter "error" # Grep filter +vastai execute "nvidia-smi" # Run command on instance +vastai execute "ls /workspace" --schedule DAILY # Scheduled execution +``` + +### Volumes + +```bash +vastai search volumes # Search available volume offers +vastai show volumes # List your volumes +vastai create volume [-s SIZE] [-n NAME] # Create volume (offer_id from search volumes) +vastai clone volume [-s SIZE] # Clone volume (dest_id from search volumes) +vastai delete volume # Delete volume +vastai create network-volume ... # Create network volume +vastai list network-volume # List network volumes +vastai take snapshot --repo REPO --docker_login_user USER --docker_login_pass PASS # Snapshot instance to volume +``` + +### Serverless & Deployments + +```bash +vastai show endpoints # List endpoint groups +vastai create endpoint --name "my-ep" ... # Create endpoint +vastai update endpoint ... # Update endpoint +vastai delete endpoint # Delete endpoint +vastai get endpt-logs # Endpoint logs + +vastai show workergroups # List worker groups +vastai create workergroup --name "wg" ... # Create worker group +vastai update workergroup ... # Update worker group +vastai update workers # Rolling update of workers +vastai delete workergroup # Delete worker group +vastai get wrkgrp-logs # Worker group logs + +vastai show deployments # List deployments +vastai show deployment # Deployment details +vastai show deployment-versions # Version history +vastai delete deployment # Delete deployment + +vastai show scheduled-jobs # List scheduled jobs +vastai delete scheduled-job # Delete scheduled job +``` + +### Templates + +```bash +vastai search templates "pytorch" # Search templates +vastai create template --name "x" --image "img" # Create template +vastai update template ... # Update template +vastai delete template # Delete template +``` + +### Account & API Keys + +```bash +vastai set api-key # Save API key locally +vastai show api-key # Show a specific key +vastai show api-keys # List all your API keys +vastai create api-key --name "ci" --permissions '{...}' # Create restricted key +vastai delete api-key # Delete key +vastai reset api-key # Reset main key (get new from console) +vastai show user # Account info, credit balance +vastai show audit-logs # Account action history +vastai show connections # Cloud storage connections +vastai show ipaddrs # IP address history +``` + +### Billing + +```bash +vastai show invoices-v1 # Charges/invoices (paginated) +vastai show invoices-v1 --charges # Charges only +vastai show invoices-v1 --invoices # Invoices only +vastai show invoices-v1 --start-date 2026-01-01 --end-date 2026-02-01 +vastai show invoices-v1 --limit 50 --latest-first +vastai show deposit # Reserved instance deposit info +``` + +### Teams + +```bash +vastai create team --name "myteam" # Create team +vastai show members # List team members +vastai invite member --email user@example.com # Invite member +vastai remove member # Remove member +vastai create team-role --name "viewer" ... # Create role +vastai show team-role # Role details +vastai show team-roles # List roles +vastai update team-role ... # Update role +vastai remove team-role # Remove role +vastai destroy team # Delete team +``` + +### Environment Variables + +```bash +vastai show env-vars # List user env vars +vastai create env-var KEY val # Create env var +vastai update env-var KEY newval # Update env var +vastai delete env-var KEY # Delete env var +``` + +### Machine Management (hosts) + +```bash +vastai show machines # List your machines +vastai list machine # List/unlist machine on marketplace +vastai show machine # Machine details +vastai cleanup machine # Clean up machine state +vastai schedule maint ... # Schedule maintenance window +vastai cancel maint # Cancel scheduled maintenance +vastai show maints # List maintenance windows +vastai unlist machine # Remove from marketplace +``` + +## Common errors + +| Error | Cause | Fix | +|-------|-------|-----| +| `401 Unauthorized` | Invalid or expired API key | `vastai set api-key ` | +| `Insufficient credits` | Account balance too low | Add credits at https://cloud.vast.ai/billing/ | +| `No offers found` | Filters too restrictive | Relax filters, try `--no-default/-n` | +| `Permission denied` | SSH key not attached | `vastai create ssh-key` before `create instance` | +| `Connection refused` | Instance not yet running | Poll `show instance ` until `actual_status == "running"` | +| Hangs on `destroy instance` | Confirmation prompt waiting for input | Add `-y` flag: `vastai destroy instance -y` | + +## URLs + +### Console + +``` +https://console.vast.ai/instances/ # Your instances +https://console.vast.ai/create/ # Search GPU offers +https://console.vast.ai/cli # Create and manage API keys +https://cloud.vast.ai/billing/ # Billing +``` + +### API + +``` +https://console.vast.ai/api/v0/instances/ # Instances endpoint +https://console.vast.ai/api/v0/asks/ # Offers search +``` + +### Instance Ports + +Access a port exposed on your instance: + +``` +ssh://root@: # SSH (from vastai ssh-url) +``` + +Direct connections (when `--direct` used at creation): `` field in instance JSON. + +## Documentation + +Fetch the complete documentation index at: https://docs.vast.ai/llms.txt