RomanNum3ral
/
soup
mirror of https://github.com/razor-ai/soup.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
151 Commits 1 Branch 0 Tags 10 MiB
Commit Graph

14 Commits

Author SHA1 Message Date
Alpamys 1b1d679141 feat: v0.21.0 — migrate, recipes, NEFTune, rsLoRA 
- `soup migrate` — import configs from LLaMA-Factory, Axolotl, Unsloth
  notebooks (AST-only .ipynb parsing, path traversal protection)
- `soup recipes` — 30 ready-made configs for popular models
  (list/show/use/search with path traversal protection)
- NEFTune (`neftune_alpha`) — noisy embeddings for SFT/DPO/KTO/ORPO/SimPO/IPO
- rsLoRA (`use_rslora`) — rank-stabilized LoRA scaling in all 11 trainers
- Fix: `soup doctor` torchvision circular import crash
- Fix: `load_eval_tasks()` now accepts str in addition to Path
- Security: Rich markup injection prevention in migration warnings
- Security: 10 MB file size limit on migration input files
- 1789 tests, 62 test files, 64% coverage
 2026-04-02 14:08:36 +05:00
Alpamys c46265fd18 feat: add eval platform with custom evals, LLM judge, human eval, leaderboard (v0.19.0) 
Full-featured evaluation system with 7 subcommands:
- soup eval benchmark: standard benchmarks via lm-evaluation-harness
- soup eval custom: custom JSONL eval tasks with 4 scoring modes
- soup eval judge: LLM-as-a-judge (OpenAI/Ollama/server backends)
- soup eval auto: automatic post-training evaluation from config
- soup eval compare: side-by-side eval comparison with regression detection
- soup eval leaderboard: local model leaderboard with JSON/CSV export
- soup eval human: terminal A/B comparison with Elo ratings

New modules: soup_cli/eval/ (custom.py, judge.py, human.py, leaderboard.py)
Config: EvalConfig added to schema.py (auto_eval, benchmarks, custom_tasks, judge)
Callback: SoupTrainerCallback.on_train_end triggers auto-eval when configured

Security: SSRF protection on judge API, ReDoS guard on regex scoring,
API key isolation per provider, 10k task/prompt caps, read-only SQL queries

1585 tests, 58 test files, ruff clean
 2026-04-01 14:47:08 +05:00
Alpamys f98519ef87 feat: add Ollama integration — deploy GGUF models in one command (v0.18.0) 
New commands:
- `soup deploy ollama` — deploy GGUF to local Ollama with auto-template detection
- `soup deploy ollama --list` / `--remove` — manage Soup-deployed models
- `soup export --deploy ollama` — export + auto-deploy in one step

New files:
- soup_cli/utils/ollama.py — detect, deploy, list, remove, Modelfile generation
- soup_cli/commands/deploy.py — Typer command group with Rich panels
- tests/test_deploy_ollama.py — 78 tests covering all paths

Security hardening:
- GGUF path traversal protection + .gguf extension validation
- Model name validation (no path separators, null bytes)
- Modelfile parameter key allowlist prevents directive injection
- Parameter value newline/null sanitization
- Subprocess calls use list args (no shell injection)
- Warning panel before overwriting existing Ollama models

1449 tests, 57 test files, all passing.
 2026-04-01 13:47:40 +05:00
Alpamys 986f8cb26c feat: add GitHub repo link to CLI output, bump version to v0.17.3 
Show GitHub URL in `soup version`, `soup version --full`, `soup doctor`,
and `soup --help` so users can find and star the repo.
Extract URL to GITHUB_URL constant in utils/constants.py.
 2026-03-26 15:48:04 +05:00
Alpamys edaa208d73 v0.13.0: batch inference + TensorBoard logging + supported models 
- Add `soup infer` command for batch inference on JSONL prompts
  (--model, --input, --output, --max-tokens, --temperature, --device)
- Add `--tensorboard` flag to `soup train` (report_to="tensorboard")
- Validate --wandb and --tensorboard mutual exclusivity
- Add supported models table to README (Llama 4, Gemma 3, Qwen 2.5/3,
  Phi-4, DeepSeek R1/V3, Mistral, CodeLlama)
- 906 tests (29 new), 44 test files, 56.32% coverage
 2026-03-25 18:43:43 +05:00
Alpamys 14f619cc00 Add vLLM backend for soup serve (Phase 9) — v0.8.0 
- Add --backend vllm flag to soup serve for 2-4x better inference throughput
- Add --tensor-parallel and --gpu-memory flags for vLLM tuning
- Auto-detect vLLM and show hint when installed but not enabled
- New utils/vllm.py with engine creation, app factory, LoRA support
- Native token-by-token streaming via vLLM AsyncLLMEngine
- Add serve-fast extra: pip install 'soup-cli[serve-fast]'
- Add vllm detection to version --full
- 30 new tests (560 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 21:30:08 +05:00
Alpamys 823e36eea8 Add Web UI for experiment management (Phase 8) — v0.7.0 
- `soup ui` command launches local web interface at http://127.0.0.1:7860
- FastAPI backend with REST API: runs, metrics, config validation, training
  control, data inspection, templates, system info
- Self-contained SPA frontend (Dashboard, New Training, Data Explorer,
  Model Chat) with Chart.js loss/LR charts
- Auto-opens browser on launch (--no-browser to disable)
- Config validation via new load_config_from_string() in config/loader.py
- 40 new tests (530 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 20:55:22 +05:00
Alpamys 83e44a5dd0 Add soup version --full, bump to v0.3.2 
- `soup version --full` shows version, Python, GPU backend, installed extras
- Dynamic test count badge via Gist endpoint in CI
- README: Optional Extras table, --verbose note, CSV/Parquet, Changelog link
- 323 tests passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-05 19:46:16 +05:00
Alpamys df21329a52 Add Phase 3.1: friendly errors, soup doctor, soup quickstart, UX polish (v0.3.1) 
- Friendly error messages: wrap all commands in try/except, map known errors
  (CUDA OOM, missing deps, connection errors) to 2-3 line messages with fix hints
- Global --verbose flag for full tracebacks
- soup doctor: check system info, GPU, all dependency versions with fix suggestions
- soup quickstart: one-command demo (creates data + config + trains TinyLlama)
- Confirmation prompts before train/sweep (skip with --yes)
- 40 new tests (321 total), all passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-05 19:10:36 +05:00
Alpamys 87fd760847 Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 
- soup serve: FastAPI inference server with OpenAI-compatible API, SSE streaming
- soup data generate: synthetic data generation via OpenAI API or local models
- soup sweep: grid/random hyperparameter search with experiment tracker integration
- soup diff: side-by-side model comparison with metrics
- Multi-GPU/DeepSpeed: ZeRO Stage 2/3 configs, --deepspeed flag in train command
- 95 new tests (281 total), all passing
- Removed TESTING_GUIDE.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-05 17:14:08 +05:00
Alpamys f03b578428 Phase 2.5: add export GGUF, merge LoRA, resume training, W&B integration (v0.2.0) 
New commands:
- soup export --model ./output --format gguf --quant q4_k_m
- soup merge --adapter ./output

New train flags:
- soup train --resume auto (or --resume ./checkpoint-500)
- soup train --wandb

184 tests passing (was 147), all lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-03 22:29:44 +05:00
Alpamys 2aaa87fb4e Phase 2: experiment tracking, data tools, model evaluation 
- Add SQLite experiment tracker (~/.soup/experiments.db) with auto-logging
  of config, per-step metrics, hardware info, and eval results
- Add soup runs commands: list, show (with plotext loss curves), compare, delete
- Integrate tracker into soup train (auto start_run/finish_run/fail_run)
- Add soup data convert (alpaca/sharegpt/chatml bidirectional conversion)
- Add soup data merge (concatenate datasets with optional shuffle)
- Add soup data dedup (MinHash near-duplicate removal via datasketch)
- Add soup data stats (length percentiles, token counts, language detection)
- Add soup eval (lm-evaluation-harness wrapper with tracker integration)
- Add reverse format conversion: messages_to_format() in data/formats.py
- Add extended_stats() to data/validator.py
- Update monitoring callback to log metrics to tracker
- Add plotext to deps, datasketch as optional [data] dep
- Update README and CLAUDE.md with Phase 2 docs
- 70 tests passing, ruff clean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 23:34:28 +05:00
Alpamys a2a0f2cab3 Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 
- soup chat --model ./path: interactive terminal chat with LoRA adapters
  (auto-detects base model, supports /quit /clear /system commands)
- soup push --model ./path --repo user/model: upload to HuggingFace Hub
  (auto model card generation, token from env/cache/flag)
- DPO trainer: full DPOTrainerWrapper with LoRA + quantization support
  (configurable dpo_beta, preference data format {prompt, chosen, rejected})
- Smoke tests: real SFT + DPO training with tiny-gpt2 (pytest -m smoke)
- SFT trainer: fallback for models without chat_template
- Updated README, schema, formats, pyproject.toml, .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 21:18:19 +05:00
Alpamys d6e932a1d3 Initial project setup: CLI skeleton + config + trainer + data pipeline 
- Typer CLI: soup init, soup train, soup data inspect/validate
- Pydantic config schema with YAML loader and validation
- Data pipeline: JSONL/JSON/CSV/Parquet + HuggingFace datasets
- Format detection: Alpaca, ShareGPT, ChatML (auto-detect)
- SFT trainer wrapper over transformers + peft + trl
- QLoRA/LoRA support with auto batch size estimation
- GPU detection (CUDA/MPS/CPU) and memory calculation
- Rich live terminal dashboard for training monitoring
- Config templates: chat, code, medical
- Tests (pytest) + GitHub Actions CI
- MIT license

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 16:14:56 +05:00