RomanNum3ral
/
soup
mirror of https://github.com/razor-ai/soup.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
96 Commits 1 Branch 0 Tags 10 MiB
Commit Graph

12 Commits

Author SHA1 Message Date
Alpamys 0be8a03a8a v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 
v0.11.0 — Alignment methods:
- ORPO (task: orpo) — wraps trl.ORPOTrainer, no reference model needed
- SimPO (task: simpo) — wraps trl.CPOTrainer with loss_type='simpo'
- IPO (task: ipo) — wraps trl.DPOTrainer with loss_type='ipo'
- Templates: soup init --template orpo/simpo/ipo
- Init wizard, train routing, sweep shortcuts for all three

v0.12.0 — Advanced PEFT:
- DoRA (use_dora: true) — weight-decomposed LoRA in all 9 trainers
- LoRA+ (loraplus_lr_ratio) — different lr for A and B matrices
- GaLore (use_galore: true) — memory-efficient full-param training
- GaLore validation: incompatible with quantization and unsloth

Security:
- experiment_name path traversal validation (no / \ : null bytes)
- GaLore optim_args type enforcement before string interpolation

Tests: 877 passed (was 746), 42 test files, 56.98% coverage
 2026-03-25 18:12:36 +05:00
Alpamys 428c0f09a4 v0.10.1: Fix 6 bugs from manual testing report 
- BUG-001: Replace Unicode arrows/dashes with ASCII in all console output
  to fix UnicodeEncodeError on Windows cp1252 (~10 commands affected)
- BUG-002: PPO trainer uses inspect.signature to detect trl parameter names
  (ppo_epochs vs num_ppo_epochs) for trl 0.28.0 compatibility
- BUG-003: Add get_compute_dtype() - uses float32 on CPU, bfloat16/float16
  on CUDA. Fixes dtype mismatch in reward model and all trainers
- BUG-004: Add warning when using quantization on CPU
- BUG-005: Fix dtype -> torch_dtype in diff.py model loading
- BUG-006: Pin wandb<0.18.0 to avoid trl import conflict, add runtime guard

13 new tests (624 total), ruff clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 12:31:37 +05:00
Alpamys 83da7f071d Add PPO / Full RLHF pipeline (Phase 10) — v0.9.0 
Three-stage RLHF training: SFT → Reward Model → PPO.

- task: ppo — PPO trainer with manual training loop (generate → score → optimize)
- task: reward_model — RewardTrainer for preference data (prompt/chosen/rejected)
- PPO config: ppo_epochs, ppo_clip_ratio, ppo_kl_penalty, reward_model
- Two reward sources: reward_model (pre-trained RM) and/or reward_fn (callable)
- soup init --template rlhf — generates PPO config with reward model
- Sweep shortcuts for all PPO parameters
- 51 new tests (611 total), ruff clean, all passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 22:17:55 +05:00
Alpamys 823e36eea8 Add Web UI for experiment management (Phase 8) — v0.7.0 
- `soup ui` command launches local web interface at http://127.0.0.1:7860
- FastAPI backend with REST API: runs, metrics, config validation, training
  control, data inspection, templates, system info
- Self-contained SPA frontend (Dashboard, New Training, Data Explorer,
  Model Chat) with Chart.js loss/LR charts
- Auto-opens browser on launch (--no-browser to disable)
- Config validation via new load_config_from_string() in config/loader.py
- 40 new tests (530 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 20:55:22 +05:00
Alpamys 5f77bc57da Add Quantization-Aware Training support (Phase 7) — v0.6.0 
- Add `quantization_aware: true` config option (TrainingConfig)
- New `utils/qat.py` module: torchao Int8WeightOnlyConfig integration
  with `prepare_model_for_qat()`, `validate_qat_config()`, `is_qat_available()`
- QAT support in all trainers: SFT, DPO, GRPO (+ vision modality)
- Train command shows `+ QAT` in setup panel, validates QAT config
  (rejects unsloth backend, warns on 8bit)
- `pip install 'soup-cli[qat]'` optional dependency (torchao>=0.4.0)
- `torchao` added to `soup doctor` dependency checks
- 35 new tests (test_qat.py): config, validation, trainer integration,
  export compatibility, sweep, doctor
- Total: 490 tests across 37 files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 20:35:35 +05:00
Alpamys 5b7ad3c358 Add multimodal vision fine-tuning support (Phase 6) — v0.5.0 
- Add `modality: vision` config option for vision-language model training
- Add LLaVA and ShareGPT4V data format detection and conversion
- Add `image_dir` field in DataConfig for resolving image paths
- Add vision model loading via AutoModelForVision2Seq + AutoProcessor in SFT trainer
- Add `soup init --template vision` with LLaMA-3.2-Vision config
- Add image statistics display in `soup data inspect` for vision datasets
- Add Pillow as optional `vision` extra dependency
- Add Pillow to `soup doctor` dependency checks
- 51 new tests (455 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 20:16:24 +05:00
Alpamys 6de66fb28c Add Unsloth backend for 2-5x faster training (Phase 5) — v0.4.3 
- Add `backend: unsloth` config option (default: transformers)
- Create utils/unsloth.py with FastLanguageModel integration
- Update SFT/DPO/GRPO trainers with _setup_unsloth() path
- Auto-detect unsloth and suggest enabling in `soup train`
- Add `fast` extra: pip install 'soup-cli[fast]'
- Add unsloth to `soup doctor` dependency check
- Add `backend` sweep shortcut
- 33 new tests (404 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 16:55:44 +05:00
Alpamys 5296b0019b Add GRPO reasoning training (Phase 4) — v0.4.2 
- New `task: grpo` for Group Relative Policy Optimization (DeepSeek-R1 style)
- Reward function API: built-in (accuracy, format) + custom Python files
- Config fields: grpo_beta, num_generations, reward_fn
- `soup init --template reasoning` template
- GRPOTrainerWrapper with auto batch size scaling for multi-generation
- Sweep shortcuts for grpo_beta, num_generations, reward_fn
- 42 new tests (371 total), lint clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 16:29:25 +05:00
Alpamys a2a0f2cab3 Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 
- soup chat --model ./path: interactive terminal chat with LoRA adapters
  (auto-detects base model, supports /quit /clear /system commands)
- soup push --model ./path --repo user/model: upload to HuggingFace Hub
  (auto model card generation, token from env/cache/flag)
- DPO trainer: full DPOTrainerWrapper with LoRA + quantization support
  (configurable dpo_beta, preference data format {prompt, chosen, rejected})
- Smoke tests: real SFT + DPO training with tiny-gpt2 (pytest -m smoke)
- SFT trainer: fallback for models without chat_template
- Updated README, schema, formats, pyproject.toml, .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 21:18:19 +05:00
Alpamys d167cd4ddd Fix Python 3.9 compatibility + add .claude project settings 
- Replace `str | list[str]` with `Union[str, List[str]]` (3.9 compat)
- Replace `str | None` with `Optional[str]` in validator.py
- Replace `Live | None` with `Optional[Live]` in display.py
- Add .claude/settings.json: auto-allow git, ruff, pytest, pip, soup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 16:36:18 +05:00
Alpamys 7433029d19 Fix all ruff lint errors and failing test 
- Fix 23 ruff errors: line too long, unused imports, ambiguous vars
- Fix validator: empty string is valid data, only count None as empty
- Remove unused imports in display.py and validator.py
- Rename ambiguous `l` vars to `part`, `entry`, `length`
- Break long lines in callback.py, display.py, sft.py, constants.py

All 20 tests passing, ruff clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 16:25:46 +05:00
Alpamys d6e932a1d3 Initial project setup: CLI skeleton + config + trainer + data pipeline 
- Typer CLI: soup init, soup train, soup data inspect/validate
- Pydantic config schema with YAML loader and validation
- Data pipeline: JSONL/JSON/CSV/Parquet + HuggingFace datasets
- Format detection: Alpaca, ShareGPT, ChatML (auto-detect)
- SFT trainer wrapper over transformers + peft + trl
- QLoRA/LoRA support with auto batch size estimation
- GPU detection (CUDA/MPS/CPU) and memory calculation
- Rich live terminal dashboard for training monitoring
- Config templates: chat, code, medical
- Tests (pytest) + GitHub Actions CI
- MIT license

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 16:14:56 +05:00