mirror of https://github.com/razor-ai/soup.git
1b6b428aaa
Part A: HuggingFace Dataset browser - soup data search: search HF Hub for datasets (sort by downloads/likes) - soup data preview: preview remote dataset metadata, splits, features - soup data download: stream HF dataset to local JSONL (with format conversion) - Security: trust_remote_code=False, path traversal protection, samples cap at 1M Part B: Freeze training (like LLaMA-Factory finetuning_type: freeze) - freeze_layers / freeze_ratio config fields - soup_cli/utils/freeze.py: detect layers, freeze bottom N - Wired into SFT trainer before LoRA application - Supports LLaMA (layers.N) and GPT-2 (h.N) naming Part C: Loss watchdog (like Axolotl loss_watchdog_threshold) - loss_watchdog, loss_watchdog_threshold, loss_watchdog_patience config - Implemented in SoupTrainerCallback with patience counter - Rich warning panel (stops Live display first), fires only once - Wired into all 11 trainers via callback kwargs Part D: Dataset info registry - soup data register/unregister/registry commands - ~/.soup/datasets.json local name→path+format mapping - Name validation, path traversal protection, Rich markup escaping 82 new tests (2061 total), 74 test files. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| loader.py | ||
| schema.py | ||