42b56f1570
fix: rename APIs to match test plan, fix RoPE factor detection
...
- Rename is_liger_available -> check_liger_available
- Rename detect_flash_attention -> check_flash_attn_available
- Rename is_ring_attention_available -> check_ring_attention_available
- Rename is_sglang_available -> check_sglang_available
- Rename compute_coherence_scores -> compute_coherence_score
- Rename FSDP keys: fsdp_full_shard -> full_shard, etc.
- Fix get_rope_scaling_config to accept factor-style args (e.g., 4.0)
- Update all callers, tests, and README
- 1371 tests pass, ruff clean, 58.81% coverage
2026-03-26 15:14:24 +05:00
bc743e2814
fix: pad token perf in perplexity scoring, SGLang SSRF validation
...
- Use -100 ignore_index for pad tokens in perplexity computation (avoids
redundant softmax on padding positions)
- Block URL-based model paths in SGLang create_sglang_runtime (SSRF protection)
2026-03-26 13:51:57 +05:00
0b7759898c
fix: address review findings — immutable rows, response guard, GPU cleanup
...
- Stop mutating dataset rows in-place in _validate_audio_files (use shallow copy)
- Guard _generate_server response parsing against unexpected JSON shape
- Add empty dataset guard in _prepare_audio_dataset
- Free GPU memory after perplexity scoring in compute_perplexity_scores
2026-03-26 13:49:40 +05:00
3d66b41d00
v0.17.0: data quality filters, audio modality, SGLang backend, server provider
...
New features:
- soup data filter: quality filters with perplexity and coherence scoring
- modality: audio — Qwen2-Audio, Whisper fine-tuning with audio data format
- --backend sglang for soup serve (SGLang high-throughput inference)
- --provider server for soup data generate (local OpenAI-compatible servers)
- Audio template: soup init --template audio
Security hardening:
- Server provider SSRF validation (scheme whitelist, localhost-only HTTP)
- Audio file path traversal protection (resolved paths confined to audio_dir)
- trust_remote_code warning panels for audio models and SGLang runtime
1348 tests, 56 test files, 58.8% coverage, ruff clean.
2026-03-26 13:46:17 +05:00
Alpamys