Alpamys
|
5f728c04cc
|
Expand test suite from ~70 to 147 tests, fix flaky ordering bug
Add 8 new test files covering previously untested modules:
- test_chat.py: _detect_base_model, adapter validation
- test_push.py: _format_size, _generate_model_card, token checks
- test_init.py: all templates, overwrite confirm/deny, YAML validation
- test_callback.py: SoupTrainerCallback with mocks
- test_display.py: TrainingDisplay rendering + edge cases
- test_loader.py: JSON/CSV/JSONL loading, empty lines, bad JSON
- test_validator.py: validate_and_stats, extended_stats, _percentile
- test_formats.py: reverse conversion, round-trips, edge cases
Fix flaky test_list_runs_ordering by adding rowid DESC as tiebreaker
in list_runs SQL query (runs created in same second had nondeterministic order).
Update CLAUDE.md with test file inventory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-03-02 20:52:33 +05:00 |
|
Alpamys
|
2aaa87fb4e
|
Phase 2: experiment tracking, data tools, model evaluation
- Add SQLite experiment tracker (~/.soup/experiments.db) with auto-logging
of config, per-step metrics, hardware info, and eval results
- Add soup runs commands: list, show (with plotext loss curves), compare, delete
- Integrate tracker into soup train (auto start_run/finish_run/fail_run)
- Add soup data convert (alpaca/sharegpt/chatml bidirectional conversion)
- Add soup data merge (concatenate datasets with optional shuffle)
- Add soup data dedup (MinHash near-duplicate removal via datasketch)
- Add soup data stats (length percentiles, token counts, language detection)
- Add soup eval (lm-evaluation-harness wrapper with tracker integration)
- Add reverse format conversion: messages_to_format() in data/formats.py
- Add extended_stats() to data/validator.py
- Update monitoring callback to log metrics to tracker
- Add plotext to deps, datasketch as optional [data] dep
- Update README and CLAUDE.md with Phase 2 docs
- 70 tests passing, ruff clean
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-02-23 23:34:28 +05:00 |