From f5ad0f5a45c0aeb9e4b8c23410b0f9415583d998 Mon Sep 17 00:00:00 2001 From: Alpamys Date: Wed, 25 Mar 2026 23:33:46 +0500 Subject: [PATCH] docs: update SECURITY, CONTRIBUTING, examples README to v0.14.3 - SECURITY.md: update supported versions to v0.14.x, add security hardening history - CONTRIBUTING.md: update test counts (47 files, 1022 tests), add all trainers, fix project structure - examples/README.md: add KTO/ORPO/SimPO/IPO, pre-training, MoE, batch inference sections - CLAUDE.md: add SECURITY/CONTRIBUTING/examples to release checklist --- .claude/CLAUDE.md | 13 ++++++---- CONTRIBUTING.md | 29 +++++++++++---------- SECURITY.md | 16 +++++++++--- examples/README.md | 65 ++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 99 insertions(+), 24 deletions(-) diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index a42fb97..455875d 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -220,11 +220,14 @@ soup version # Show version (--full for details) 6. **Version**: bump version in `pyproject.toml` + `soup_cli/__init__.py` 7. **CLAUDE.md**: update architecture, test table, and any new sections 8. **README.md**: add docs for new feature, update Features / All Commands / Data Formats -9. **plan.md**: mark the phase as complete, update version/test counters -10. **Commit**: one commit per phase with a descriptive message -11. **Push**: `git push origin main` -12. **Tag**: `git tag v0.X.Y && git push origin v0.X.Y` -13. **Release**: `gh release create v0.X.Y` with changelog (What's New, Install/Upgrade) +9. **SECURITY.md**: update if new security fixes or supported versions changed +10. **CONTRIBUTING.md**: update if dev workflow, deps, or project structure changed +11. **examples/README.md**: update if new example configs or datasets added +12. **plan.md**: mark the phase as complete, update version/test counters +13. **Commit**: one commit per phase with a descriptive message +14. **Push**: `git push origin main` +15. **Tag**: `git tag v0.X.Y && git push origin v0.X.Y` +16. **Release**: `gh release create v0.X.Y` with changelog (What's New, Install/Upgrade) ## Tests (47 test files, 1022 tests) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 21a447c..0d443e5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -92,14 +92,13 @@ soup_cli/ commands/ - Command implementations (train, chat, eval, etc.) config/ - Config schema (schema.py) and loader (loader.py) data/ - Data loading and format conversion - trainer/ - Training wrappers (SFT, DPO, GRPO, PPO, reward_model) + trainer/ - Training wrappers (SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO, Pretrain, Reward Model) monitoring/ - Callbacks and live dashboard experiment/ - SQLite experiment tracking - utils/ - GPU detection, batch size estimation, error handling - ui/ - Web UI (FastAPI + HTML/JS) + utils/ - GPU detection, batch size estimation, error handling, MoE, GaLore, QAT, Unsloth, vLLM + ui/ - Web UI (FastAPI + HTML/JS SPA) -templates/ - YAML config templates (chat, code, medical, vision, rlhf, reasoning) -tests/ - Test suite (40+ files, 600+ tests) +tests/ - Test suite (47 files, 1022 tests) examples/ - Real-world config examples and datasets ``` @@ -133,11 +132,15 @@ pytest tests/ --cov=soup_cli --cov-report=html - `test_config.py` — Config loading and validation - `test_data.py` — Data format detection and conversion -- `test_trainer_*.py` — Individual trainer tests -- `test_smoke_train.py` — Full pipeline tests (GPU required) - `test_cli.py` — Command-line interface tests - `test_errors.py` — Error message handling -- `test_*_command.py` — Specific command tests (chat, push, eval, etc.) +- `test_smoke_train.py` — Full pipeline tests (GPU required) +- `test_grpo.py`, `test_ppo.py`, `test_kto.py`, `test_orpo.py`, `test_simpo.py`, `test_ipo.py` — Trainer-specific tests +- `test_pretrain.py`, `test_moe.py` — Pre-training and MoE tests +- `test_serve.py`, `test_vllm_serve.py` — Serve command and vLLM backend +- `test_ui.py` — Web UI endpoints, auth, static files +- `test_infer.py` — Batch inference command +- `test_bugfixes.py` — Regression fixes (v0.10.1–v0.14.3) ## Making Changes @@ -258,12 +261,12 @@ Multiple formats (Alpaca, ShareGPT, ChatML, LLaVA, ShareGPT4V) are normalized to If adding a new training algorithm (e.g., DPO, GRPO): -1. Create `trainer/your_trainer.py` with a class inheriting from `BaseTrainer` -2. Add Pydantic config class to `config/schema.py` -3. Add template to `templates/your.yaml` and `config/schema.py` +1. Create `trainer/your_trainer.py` wrapping the appropriate TRL trainer +2. Add config fields to `config/schema.py` (Pydantic v2) +3. Add template to `config/schema.py` (see existing 12 templates) 4. Update `commands/train.py` to route to your trainer 5. Add 30+ tests in `tests/test_your_trainer.py` -6. Update `CLAUDE.md` and `README.md` +6. Update `CLAUDE.md`, `README.md`, and `CONTRIBUTING.md` ### 2. New Data Format @@ -295,7 +298,7 @@ The project follows semantic versioning: `MAJOR.MINOR.PATCH` 1. Update version in `pyproject.toml` and `soup_cli/__init__.py` 2. Run full test suite and linting -3. Update `CLAUDE.md`, `README.md`, `plan.md` +3. Update `CLAUDE.md`, `README.md`, `SECURITY.md` (if security-related), `CONTRIBUTING.md` (if workflow changed) 4. Commit with message: `Release v0.X.0` 5. Tag: `git tag v0.X.0 && git push --tags` 6. GitHub Actions auto-publishes to PyPI diff --git a/SECURITY.md b/SECURITY.md index 9b45bb8..80d68cd 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -4,14 +4,14 @@ We provide security updates for the following versions: -- **Latest minor version:** Active support (e.g., v0.9.x) +- **Latest minor version:** Active support (e.g., v0.14.x) - **Previous minor versions:** Bug-fix support only - **Versions older than 3 minor versions:** No support Example: -- v0.9.0-0.9.x → Full support (latest) -- v0.8.0-0.8.x → Bug-fix support only -- v0.7.x and below → No support +- v0.14.0-0.14.x → Full support (latest) +- v0.13.0-0.13.x → Bug-fix support only +- v0.12.x and below → No support ## Reporting a Vulnerability @@ -110,6 +110,14 @@ We maintain a log of known security issues and their fixes. This will be updated No known critical vulnerabilities in current releases. +### Security Hardening History + +- **v0.10.10**: Bearer token auth on Web UI, CORS restrictions, path traversal protection, SSRF prevention, max_tokens limits, supply-chain pinning (llama.cpp b5270), deprecated CLI secret flags +- **v0.12.0**: experiment_name path traversal validation, GaLore parameter type enforcement +- **v0.13.0**: Batch inference max_tokens capped at 16384, trust_remote_code warning +- **v0.14.0**: Plaintext loader UTF-8 encoding, MoE config validation (moe_aux_loss_coeff ge=0, moe_lora boolean only) +- **v0.14.3**: Data validate auto-detects format, Web UI `--show-token` flag + auth token documented + ## Security Scanning - All code is scanned with `ruff` for style and common issues diff --git a/examples/README.md b/examples/README.md index b35788a..38172c2 100644 --- a/examples/README.md +++ b/examples/README.md @@ -59,7 +59,65 @@ soup train --config examples/configs/vision_llama.yaml - Uses LLaVA format for images + text - Outputs to `./output_vision/` -### 5. Full RLHF Pipeline +### 5. Alignment Methods (KTO / ORPO / SimPO / IPO) + +Train with alternative preference optimization: + +```bash +# KTO — unpaired preference (only needs thumbs up/down labels) +soup init --template kto +soup train + +# ORPO — reference-free alignment (no reference model needed) +soup init --template orpo +soup train + +# SimPO — length-normalized preference optimization +soup init --template simpo +soup train + +# IPO — regularized preference (squared hinge loss) +soup init --template ipo +soup train +``` + +### 6. Continued Pre-training + +Continue training on raw text corpora: + +```bash +soup init --template pretrain +soup train +``` + +**What it does:** +- Trains on plain text (`.txt` files or JSONL with `text` field) +- No instruction format needed — just raw text +- Useful for domain adaptation (legal, medical, code) + +### 7. MoE Models + +Fine-tune Mixture-of-Experts models (Qwen3, Mixtral, DeepSeek V3): + +```bash +soup init --template moe +soup train +``` + +**What it does:** +- Auto-detects MoE architecture (ScatterMoE / SwitchTransformers) +- `moe_lora: true` targets expert-specific LoRA modules +- Optional `moe_aux_loss_coeff` for load balancing + +### 8. Batch Inference + +Run inference on a batch of prompts: + +```bash +soup infer --model ./output_sft_basic/ --input prompts.jsonl --output results.jsonl +``` + +### 9. Full RLHF Pipeline Complete reinforcement learning from human feedback: @@ -81,7 +139,10 @@ Datasets are included in JSONL format. Soup auto-detects and normalizes: - **Alpaca**: `instruction`, `input`, `output` fields - **ShareGPT**: `conversations` with `from`/`value` fields - **ChatML**: OpenAI-style `messages` with `role`/`content` -- **LLaVA**: Vision format with `image` + `conversations` +- **DPO/ORPO/SimPO/IPO**: `prompt` + `chosen` + `rejected` fields +- **KTO**: `prompt` + `completion` + `label` fields +- **LLaVA / ShareGPT4V**: Vision format with `image` + `conversations` +- **Plaintext**: Raw `.txt` files or JSONL with `text` field (for pre-training) ### Example: Inspect a Dataset