Commit Graph

442 Commits

Author SHA1 Message Date
Alpamys 823e36eea8 Add Web UI for experiment management (Phase 8) — v0.7.0
- `soup ui` command launches local web interface at http://127.0.0.1:7860
- FastAPI backend with REST API: runs, metrics, config validation, training
  control, data inspection, templates, system info
- Self-contained SPA frontend (Dashboard, New Training, Data Explorer,
  Model Chat) with Chart.js loss/LR charts
- Auto-opens browser on launch (--no-browser to disable)
- Config validation via new load_config_from_string() in config/loader.py
- 40 new tests (530 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 20:55:22 +05:00
Alpamys 5f77bc57da Add Quantization-Aware Training support (Phase 7) — v0.6.0
- Add `quantization_aware: true` config option (TrainingConfig)
- New `utils/qat.py` module: torchao Int8WeightOnlyConfig integration
  with `prepare_model_for_qat()`, `validate_qat_config()`, `is_qat_available()`
- QAT support in all trainers: SFT, DPO, GRPO (+ vision modality)
- Train command shows `+ QAT` in setup panel, validates QAT config
  (rejects unsloth backend, warns on 8bit)
- `pip install 'soup-cli[qat]'` optional dependency (torchao>=0.4.0)
- `torchao` added to `soup doctor` dependency checks
- 35 new tests (test_qat.py): config, validation, trainer integration,
  export compatibility, sweep, doctor
- Total: 490 tests across 37 files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 20:35:35 +05:00
Alpamys 5b7ad3c358 Add multimodal vision fine-tuning support (Phase 6) — v0.5.0
- Add `modality: vision` config option for vision-language model training
- Add LLaVA and ShareGPT4V data format detection and conversion
- Add `image_dir` field in DataConfig for resolving image paths
- Add vision model loading via AutoModelForVision2Seq + AutoProcessor in SFT trainer
- Add `soup init --template vision` with LLaMA-3.2-Vision config
- Add image statistics display in `soup data inspect` for vision datasets
- Add Pillow as optional `vision` extra dependency
- Add Pillow to `soup doctor` dependency checks
- 51 new tests (455 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 20:16:24 +05:00
Alpamys 3b46e96797 Add report.xml to .gitignore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 20:01:27 +05:00
Alpamys dc518ba1fc Add GitHub templates, downloads badge, and repo topics
- Issue templates (bug report with soup doctor output, feature request)
- PR template with checklist
- Downloads badge via pepy.tech
- GitHub topics: llm, fine-tuning, lora, qlora, pytorch, etc.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 20:00:32 +05:00
Alpamys 6de66fb28c Add Unsloth backend for 2-5x faster training (Phase 5) — v0.4.3
- Add `backend: unsloth` config option (default: transformers)
- Create utils/unsloth.py with FastLanguageModel integration
- Update SFT/DPO/GRPO trainers with _setup_unsloth() path
- Auto-detect unsloth and suggest enabling in `soup train`
- Add `fast` extra: pip install 'soup-cli[fast]'
- Add unsloth to `soup doctor` dependency check
- Add `backend` sweep shortcut
- 33 new tests (404 total), ruff clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:55:44 +05:00
Alpamys dd73d91356 Remove Features table from README — redundant with docs sections
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:35:25 +05:00
Alpamys 26cccaf730 Fix Python 3.9 compat: add __future__ annotations to rewards.py
`str | None` union syntax requires Python 3.10+. CI runs on 3.9.
Added `from __future__ import annotations` to fix.

Also added Release Checklist section to CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:34:11 +05:00
Alpamys 5296b0019b Add GRPO reasoning training (Phase 4) — v0.4.2
- New `task: grpo` for Group Relative Policy Optimization (DeepSeek-R1 style)
- Reward function API: built-in (accuracy, format) + custom Python files
- Config fields: grpo_beta, num_generations, reward_fn
- `soup init --template reasoning` template
- GRPOTrainerWrapper with auto batch size scaling for multi-generation
- Sweep shortcuts for grpo_beta, num_generations, reward_fn
- 42 new tests (371 total), lint clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:29:25 +05:00
Alpamys 0ad2b182b9 Bump to v0.4.1: fix typer pin for Python 3.9 (click 8.2+ needs 3.10+)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:08:06 +05:00
Alpamys 6a5ee1ae7a Fix CI: pin typer<0.21.0 for Python 3.9 compat (click 8.2+ needs 3.10+)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:04:13 +05:00
Alpamys df8c341b73 Add sweep early stopping, Rich download progress bars, fix click compat — v0.4.0
- Add --early-stop flag to soup sweep: skip remaining runs when loss exceeds best by threshold
- Add Rich progress bars for HuggingFace Hub model downloads in SFT/DPO trainers
- Pin click>=8.2.0 to fix typer compatibility (11 tests were failing)
- Bump version 0.3.2 → 0.4.0
- 329 tests passing, lint clean

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 15:59:27 +05:00
Alpamys 5de18dc325 Fix CI badge: read test count from testsuite element, not root
pytest junitxml wraps results in <testsuites><testsuite tests="N">,
the count is on the child element, not the root.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:01:44 +05:00
Alpamys f467dc8014 Trigger CI to test dynamic badge update
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:56:45 +05:00
Alpamys 83e44a5dd0 Add soup version --full, bump to v0.3.2
- `soup version --full` shows version, Python, GPU backend, installed extras
- Dynamic test count badge via Gist endpoint in CI
- README: Optional Extras table, --verbose note, CSV/Parquet, Changelog link
- 323 tests passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:46:16 +05:00
Alpamys ee9aefcc55 Add dynamic test count badge via Gist endpoint
CI now parses pytest junitxml output and updates a public Gist with the
exact test count. Shields.io reads the Gist as an endpoint badge.
Requires GIST_TOKEN secret (PAT with gist scope).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:36:08 +05:00
Alpamys c0ed2f376b Polish README: add extras table, fix badge, clean up commands list
- Replace hardcoded "321 passed" badge with "passing"
- Add Optional Extras table (serve, data, eval, deepspeed, dev)
- Add note that --verbose must go before command name
- Mention CSV/Parquet support in Data Formats
- Add Changelog section linking to GitHub Releases
- Remove duplicate commands from All Commands (resume, wandb, deepspeed variants)
- Remove runs delete from commands list (rarely used)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:30:12 +05:00
Alpamys 588eae4b9f Fix ANSI escape code issue in verbose help output test
Rich markup splits --verbose with escape codes on CI, check for
"verbose" instead of "--verbose".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:22:53 +05:00
Alpamys 120d5a98d2 Fix --verbose example in README: global flag goes before command
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:19:07 +05:00
Alpamys df21329a52 Add Phase 3.1: friendly errors, soup doctor, soup quickstart, UX polish (v0.3.1)
- Friendly error messages: wrap all commands in try/except, map known errors
  (CUDA OOM, missing deps, connection errors) to 2-3 line messages with fix hints
- Global --verbose flag for full tracebacks
- soup doctor: check system info, GPU, all dependency versions with fix suggestions
- soup quickstart: one-command demo (creates data + config + trains TinyLlama)
- Confirmation prompts before train/sweep (skip with --yes)
- 40 new tests (321 total), all passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:10:36 +05:00
Alpamys 8cd19164a0 Fix ANSI escape code issue in deepspeed help output test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:22:21 +05:00
Alpamys 87fd760847 Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0)
- soup serve: FastAPI inference server with OpenAI-compatible API, SSE streaming
- soup data generate: synthetic data generation via OpenAI API or local models
- soup sweep: grid/random hyperparameter search with experiment tracker integration
- soup diff: side-by-side model comparison with metrics
- Multi-GPU/DeepSpeed: ZeRO Stage 2/3 configs, --deepspeed flag in train command
- 95 new tests (281 total), all passing
- Removed TESTING_GUIDE.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 17:14:08 +05:00
Alpamys 8b1844d316 Fix deprecated torch_dtype and warmup_ratio warnings, add testing guide (v0.2.2)
- Replace torch_dtype with dtype in chat, merge, export commands (transformers deprecation)
- Replace warmup_ratio with warmup_steps in SFT/DPO trainers (transformers v5.2 deprecation)
- Add TESTING_GUIDE.md for local Windows testing workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 13:29:07 +05:00
Alpamys 7a7e289dcb Fix test_version to use dynamic version instead of hardcoded string
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 00:40:14 +05:00
Alpamys 4010798e2b Fix PyTorch 2.7 compatibility: total_mem → total_memory (v0.2.1)
PyTorch 2.7+ renamed `get_device_properties().total_mem` to
`total_memory`. Fixed in gpu.py and callback.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 00:37:07 +05:00
Alpamys cab3c0b0fb Update test count badge: 184 → 186
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 23:58:29 +05:00
Alpamys e147ec21c4 Fix ANSI escape code issue in help output tests
Rich markup splits --resume/--wandb with escape codes in CI (Python 3.9).
Use case-insensitive substring match instead of exact flag match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 22:34:09 +05:00
Alpamys f03b578428 Phase 2.5: add export GGUF, merge LoRA, resume training, W&B integration (v0.2.0)
New commands:
- soup export --model ./output --format gguf --quant q4_k_m
- soup merge --adapter ./output

New train flags:
- soup train --resume auto (or --resume ./checkpoint-500)
- soup train --wandb

184 tests passing (was 147), all lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 22:29:44 +05:00
Alpamys 2f3cf90d7b Add Publishing section to CLAUDE.md
- PyPI link and install command
- Auto-publish workflow description
- Release instructions (tag-based)
- Trusted Publisher auth note

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:12:33 +05:00
Alpamys fe17addf5c Add GitHub Actions workflow for auto-publish to PyPI on tag
Triggers on `git tag v0.2.0 && git push --tags`.
Uses PyPI trusted publisher (OIDC) — no token needed in secrets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:07:55 +05:00
Alpamys 097624e44d Update README: soup-cli now on PyPI, add PyPI badge
- Change primary install to `pip install soup-cli`
- Move GitHub install to secondary option
- Add PyPI version badge

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:06:38 +05:00
Alpamys 37538d633f Update README: add badges, command reference, test count
- Add Python/License/Tests/CI badges
- Add "All Commands" section with full CLI reference
- Add lint command to Development section
- Update test count to 147

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:54:04 +05:00
Alpamys 5f728c04cc Expand test suite from ~70 to 147 tests, fix flaky ordering bug
Add 8 new test files covering previously untested modules:
- test_chat.py: _detect_base_model, adapter validation
- test_push.py: _format_size, _generate_model_card, token checks
- test_init.py: all templates, overwrite confirm/deny, YAML validation
- test_callback.py: SoupTrainerCallback with mocks
- test_display.py: TrainingDisplay rendering + edge cases
- test_loader.py: JSON/CSV/JSONL loading, empty lines, bad JSON
- test_validator.py: validate_and_stats, extended_stats, _percentile
- test_formats.py: reverse conversion, round-trips, edge cases

Fix flaky test_list_runs_ordering by adding rowid DESC as tiebreaker
in list_runs SQL query (runs created in same second had nondeterministic order).

Update CLAUDE.md with test file inventory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:52:33 +05:00
Alpamys 54282a2f66 Add logo to README and improve header layout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 12:41:32 +05:00
Alpamys 2dba07a7b3 Fix run_id collision in CI: increase suffix from 4 to 8 hex chars
The 4 hex char suffix (65536 possibilities) caused a collision when
generating 100 IDs within the same second on fast CI runners.
Increased to 8 hex chars (4 billion possibilities).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 23:48:12 +05:00
Alpamys 2aaa87fb4e Phase 2: experiment tracking, data tools, model evaluation
- Add SQLite experiment tracker (~/.soup/experiments.db) with auto-logging
  of config, per-step metrics, hardware info, and eval results
- Add soup runs commands: list, show (with plotext loss curves), compare, delete
- Integrate tracker into soup train (auto start_run/finish_run/fail_run)
- Add soup data convert (alpaca/sharegpt/chatml bidirectional conversion)
- Add soup data merge (concatenate datasets with optional shuffle)
- Add soup data dedup (MinHash near-duplicate removal via datasketch)
- Add soup data stats (length percentiles, token counts, language detection)
- Add soup eval (lm-evaluation-harness wrapper with tracker integration)
- Add reverse format conversion: messages_to_format() in data/formats.py
- Add extended_stats() to data/validator.py
- Update monitoring callback to log metrics to tracker
- Add plotext to deps, datasketch as optional [data] dep
- Update README and CLAUDE.md with Phase 2 docs
- 70 tests passing, ruff clean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 23:34:28 +05:00
Alpamys a2a0f2cab3 Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests
- soup chat --model ./path: interactive terminal chat with LoRA adapters
  (auto-detects base model, supports /quit /clear /system commands)
- soup push --model ./path --repo user/model: upload to HuggingFace Hub
  (auto model card generation, token from env/cache/flag)
- DPO trainer: full DPOTrainerWrapper with LoRA + quantization support
  (configurable dpo_beta, preference data format {prompt, chosen, rejected})
- Smoke tests: real SFT + DPO training with tiny-gpt2 (pytest -m smoke)
- SFT trainer: fallback for models without chat_template
- Updated README, schema, formats, pyproject.toml, .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 21:18:19 +05:00
Alpamys d167cd4ddd Fix Python 3.9 compatibility + add .claude project settings
- Replace `str | list[str]` with `Union[str, List[str]]` (3.9 compat)
- Replace `str | None` with `Optional[str]` in validator.py
- Replace `Live | None` with `Optional[Live]` in display.py
- Add .claude/settings.json: auto-allow git, ruff, pytest, pip, soup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:36:18 +05:00
Alpamys 7665e7c3aa Add CLAUDE.md for Claude Code project context
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:31:33 +05:00
Alpamys d30ff9ae18 Update install instructions: GitHub install as primary, PyPI coming soon
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:29:10 +05:00
Alpamys 7433029d19 Fix all ruff lint errors and failing test
- Fix 23 ruff errors: line too long, unused imports, ambiguous vars
- Fix validator: empty string is valid data, only count None as empty
- Remove unused imports in display.py and validator.py
- Rename ambiguous `l` vars to `part`, `entry`, `length`
- Break long lines in callback.py, display.py, sft.py, constants.py

All 20 tests passing, ruff clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:25:46 +05:00
Alpamys d6e932a1d3 Initial project setup: CLI skeleton + config + trainer + data pipeline
- Typer CLI: soup init, soup train, soup data inspect/validate
- Pydantic config schema with YAML loader and validation
- Data pipeline: JSONL/JSON/CSV/Parquet + HuggingFace datasets
- Format detection: Alpaca, ShareGPT, ChatML (auto-detect)
- SFT trainer wrapper over transformers + peft + trl
- QLoRA/LoRA support with auto batch size estimation
- GPU detection (CUDA/MPS/CPU) and memory calculation
- Rich live terminal dashboard for training monitoring
- Config templates: chat, code, medical
- Tests (pytest) + GitHub Actions CI
- MIT license

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:14:56 +05:00