RomanNum3ral
/
soup
mirror of https://github.com/razor-ai/soup.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
soup/tests
History
Alpamys 1b6b428aaa feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 
Part A: HuggingFace Dataset browser
- soup data search: search HF Hub for datasets (sort by downloads/likes)
- soup data preview: preview remote dataset metadata, splits, features
- soup data download: stream HF dataset to local JSONL (with format conversion)
- Security: trust_remote_code=False, path traversal protection, samples cap at 1M

Part B: Freeze training (like LLaMA-Factory finetuning_type: freeze)
- freeze_layers / freeze_ratio config fields
- soup_cli/utils/freeze.py: detect layers, freeze bottom N
- Wired into SFT trainer before LoRA application
- Supports LLaMA (layers.N) and GPT-2 (h.N) naming

Part C: Loss watchdog (like Axolotl loss_watchdog_threshold)
- loss_watchdog, loss_watchdog_threshold, loss_watchdog_patience config
- Implemented in SoupTrainerCallback with patience counter
- Rich warning panel (stops Live display first), fires only once
- Wired into all 11 trainers via callback kwargs

Part D: Dataset info registry
- soup data register/unregister/registry commands
- ~/.soup/datasets.json local name→path+format mapping
- Name validation, path traversal protection, Rich markup escaping

82 new tests (2061 total), 74 test files.
 		2026-04-03 16:35:23 +05:00
..
__init__.py Initial project setup: CLI skeleton + config + trainer + data pipeline 2026-02-20 16:14:56 +05:00
conftest.py Fix all ruff lint errors and failing test 2026-02-20 16:25:46 +05:00
test_adapters.py feat: v0.22.0 — Training Profiler, Multi-Adapter Serving, Data Sampling, Adapter Management 2026-04-03 12:54:24 +05:00
test_advanced_peft.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_audio.py v0.17.0: data quality filters, audio modality, SGLang backend, server provider 2026-03-26 13:46:17 +05:00
test_awq_gptq_export.py fix: CLI help tests use inspect.signature instead of Rich-rendered output 2026-04-03 14:08:36 +05:00
test_bugfixes.py v0.14.3: fix plotext histogram on Windows (cp1251/cp1252 encoding) 2026-03-25 23:23:38 +05:00
test_callback.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_chat.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_cli.py Add soup version --full, bump to v0.3.2 2026-03-05 19:46:16 +05:00
test_cli_subprocess.py feat: add eval platform with custom evals, LLM judge, human eval, leaderboard (v0.19.0) 2026-04-01 14:47:08 +05:00
test_config.py Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 2026-02-23 21:18:19 +05:00
test_curriculum.py fix: v0.23.1 — CI fix, security warnings, expanded test coverage 2026-04-03 14:20:21 +05:00
test_data.py Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 2026-02-23 21:18:19 +05:00
test_data_sample.py feat: v0.22.0 — Training Profiler, Multi-Adapter Serving, Data Sampling, Adapter Management 2026-04-03 12:54:24 +05:00
test_data_split.py feat: v0.23.0 — AWQ/GPTQ Export, Sample Packing, Data Split, Curriculum Learning 2026-04-03 13:55:01 +05:00
test_data_tools.py Phase 2: experiment tracking, data tools, model evaluation 2026-02-23 23:34:28 +05:00
test_dataset_hub.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_dataset_registry.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_deepspeed.py Fix ANSI escape code issue in deepspeed help output test 2026-03-05 17:22:21 +05:00
test_deploy_ollama.py fix: use ANSI-safe assertions in deploy help tests (macOS CI fix) 2026-04-01 14:01:39 +05:00
test_diff.py Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 2026-03-05 17:14:08 +05:00
test_display.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_doctor.py Add Phase 3.1: friendly errors, soup doctor, soup quickstart, UX polish (v0.3.1) 2026-03-05 19:10:36 +05:00
test_embedding.py v0.16.0: embedding models, ONNX/TensorRT export, speculative decoding 2026-03-26 12:41:39 +05:00
test_errors.py Fix ANSI escape code issue in verbose help output test 2026-03-05 19:22:53 +05:00
test_eval.py feat: add eval platform with custom evals, LLM judge, human eval, leaderboard (v0.19.0) 2026-04-01 14:47:08 +05:00
test_eval_platform.py fix: use ANSI-safe assertions in eval human help test (macOS CI fix) 2026-04-01 14:51:38 +05:00
test_export.py v0.16.0: embedding models, ONNX/TensorRT export, speculative decoding 2026-03-26 12:41:39 +05:00
test_formats.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_freeze_training.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_generate.py Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 2026-03-05 17:14:08 +05:00
test_gpu.py Initial project setup: CLI skeleton + config + trainer + data pipeline 2026-02-20 16:14:56 +05:00
test_grpo.py Add GRPO reasoning training (Phase 4) — v0.4.2 2026-03-23 16:29:25 +05:00
test_infer.py v0.13.2: add missing test coverage for infer + tensorboard 2026-03-25 18:57:15 +05:00
test_init.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_ipo.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_kto.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_loader.py feat: v0.14.0 — pre-training + MoE support 2026-03-25 22:26:01 +05:00
test_loss_watchdog.py feat: v0.24.0 — Dataset Hub, Freeze Training, Loss Watchdog, Dataset Registry 2026-04-03 16:35:23 +05:00
test_merge.py Phase 2.5: add export GGUF, merge LoRA, resume training, W&B integration (v0.2.0) 2026-03-03 22:29:44 +05:00
test_migrate.py feat: v0.21.0 — migrate, recipes, NEFTune, rsLoRA 2026-04-02 14:08:36 +05:00
test_moe.py test: add MoE integration, DeepSeek naming, and model type coverage 2026-03-25 22:34:55 +05:00
test_multi_adapter.py feat: v0.22.0 — Training Profiler, Multi-Adapter Serving, Data Sampling, Adapter Management 2026-04-03 12:54:24 +05:00
test_neftune_rslora.py feat: v0.21.0 — migrate, recipes, NEFTune, rsLoRA 2026-04-02 14:08:36 +05:00
test_onnx_tensorrt_export.py feat: v0.23.0 — AWQ/GPTQ Export, Sample Packing, Data Split, Curriculum Learning 2026-04-03 13:55:01 +05:00
test_orpo.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_packing.py fix: v0.23.1 — CI fix, security warnings, expanded test coverage 2026-04-03 14:20:21 +05:00
test_performance.py fix: rename APIs to match test plan, fix RoPE factor detection 2026-03-26 15:14:24 +05:00
test_ppo.py Add PPO / Full RLHF pipeline (Phase 10) — v0.9.0 2026-03-23 22:17:55 +05:00
test_pretrain.py test: add MoE integration, DeepSeek naming, and model type coverage 2026-03-25 22:34:55 +05:00
test_profile.py feat: v0.22.0 — Training Profiler, Multi-Adapter Serving, Data Sampling, Adapter Management 2026-04-03 12:54:24 +05:00
test_progress.py Add sweep early stopping, Rich download progress bars, fix click compat — v0.4.0 2026-03-23 15:59:27 +05:00
test_push.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_qat.py Add Quantization-Aware Training support (Phase 7) — v0.6.0 2026-03-23 20:35:35 +05:00
test_quality_filter.py fix: rename APIs to match test plan, fix RoPE factor detection 2026-03-26 15:14:24 +05:00
test_quickstart.py Add Phase 3.1: friendly errors, soup doctor, soup quickstart, UX polish (v0.3.1) 2026-03-05 19:10:36 +05:00
test_recipes.py fix: allow exit code 2 for `soup recipes` no-args help (Typer compat) 2026-04-02 14:12:32 +05:00
test_resume.py Fix ANSI escape code issue in help output tests 2026-03-03 22:34:09 +05:00
test_runs.py Phase 2: experiment tracking, data tools, model evaluation 2026-02-23 23:34:28 +05:00
test_serve.py Add Phase 3: serve, data generate, sweep, diff, DeepSpeed (v0.3.0) 2026-03-05 17:14:08 +05:00
test_server_generate.py test: add missing coverage for _parse_json_array, _validate_example, SSRF guards 2026-03-26 13:57:03 +05:00
test_sglang_serve.py fix: rename APIs to match test plan, fix RoPE factor detection 2026-03-26 15:14:24 +05:00
test_simpo.py v0.12.0: ORPO/SimPO/IPO trainers + DoRA/LoRA+/GaLore 2026-03-25 18:12:36 +05:00
test_smoke_train.py Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests 2026-02-23 21:18:19 +05:00
test_speculative_decoding.py fix: strip ANSI escape codes in CLI help flag tests 2026-03-26 12:54:38 +05:00
test_sweep.py Add sweep early stopping, Rich download progress bars, fix click compat — v0.4.0 2026-03-23 15:59:27 +05:00
test_synth_data_pro.py fix: use ANSI-safe assertions in synth data pro help tests (macOS CI fix) 2026-04-01 18:16:09 +05:00
test_tensorboard.py v0.13.2: add missing test coverage for infer + tensorboard 2026-03-25 18:57:15 +05:00
test_tracker.py Fix run_id collision in CI: increase suffix from 4 to 8 hex chars 2026-02-23 23:48:12 +05:00
test_trainer_init.py chore: add trainer init tests, fix coverage threshold for CI 2026-03-25 12:36:05 +05:00
test_ui.py v0.10.10: Security hardening — Web UI auth, CORS, SSRF, path traversal protection 2026-03-25 12:14:10 +05:00
test_unsloth.py Add Unsloth backend for 2-5x faster training (Phase 5) — v0.4.3 2026-03-23 16:55:44 +05:00
test_validator.py Expand test suite from ~70 to 147 tests, fix flaky ordering bug 2026-03-02 20:52:33 +05:00
test_vision.py v0.17.0: data quality filters, audio modality, SGLang backend, server provider 2026-03-26 13:46:17 +05:00
test_vllm_serve.py v0.16.0: embedding models, ONNX/TensorRT export, speculative decoding 2026-03-26 12:41:39 +05:00