RomanNum3ral/soup - soup - RomanNum3ral's Gitea

Commit Graph

Author	SHA1	Message	Date
Alpamys	3ac9e305ce	test(ci): harden flaky MiniLLM anchor test + restore coverage network-free CI failed on the HF-rate-limited runners: test_anchor_term_with_file did a live from_pretrained that 429'd, so it failed AND its unique MiniLLM-anchor lines went uncovered, tipping the 77% gate to 76.77% on exactly those jobs (macos + 3.11 stayed green where the cache warmed). - Skip test_anchor_term_with_file on OSError (offline / rate-limited) instead of failing. - Add test_anchor_term_with_fake_model: a fake tokenizer + tiny nn.Module exercise the identical _load_anchor + anchor_term lines with no network, so coverage no longer depends on HF availability. - Add TestReachableInternals cushion (prompt_compile._resolve_metric, prompt_distill._build_provider_fn + default-provider wiring) so the gate sits comfortably above 77% (the DSPy/TextGrad/GEPA optimiser bodies are uncoverable without the [compile] extra). Tests 13424 -> 13430.	2026-06-04 22:41:22 +05:00
Alpamys	f316d334bc	feat(rl): live GRPO/RL callbacks — reward-hack, echo-trap, RL ckpt, ULD, MiniLLM, iterative-DPO (v0.71.11) Lifts the v0.70.0 schema-only build_*_callback / build_uld_projection / run_iterative_dpo stubs. Validated end-to-end on SmolLM2-135M. Closes #235, #236, #237, #238, #239, #240, #159, #160 - #235 RewardHackCallback: info_rm cluster-sep / rm_ensemble divergence, OK/WARN/HACK, halt on HACK. Shared thread-safe RLSignalBuffer captures per-step rewards by wrapping the reward fns (no TRL monkeypatching). - #236 ULD: Wasserstein-1 / top-k aligned distill loss in DistillTrainer. - #237 MiniLLM: teacher-mixed length-normalised reverse-KL + pretrain anchor. - #238 RLCheckpointCallback: adapter + optimizer.pt + manifest + keep_last prune. - #239 run_iterative_dpo: sample -> RM-score -> build-pairs -> DPO-train per round. - #240 EchoTrapCallback: n-gram repetition OK/WARN/TRAP, halt on TRAP. - #159 one-shot WARNING when a GRPO variant compute_loss falls back to super(). - #160 in-place ref-model EMA (no state_dict round-trip) + 0-overlap warning. Tests 13142 -> 13203 (+62 in tests/test_v07111.py).	2026-06-04 17:14:08 +05:00

Author

SHA1

Message

Date

Alpamys

3ac9e305ce

test(ci): harden flaky MiniLLM anchor test + restore coverage network-free

CI failed on the HF-rate-limited runners: test_anchor_term_with_file did a
live from_pretrained that 429'd, so it failed AND its unique MiniLLM-anchor
lines went uncovered, tipping the 77% gate to 76.77% on exactly those jobs
(macos + 3.11 stayed green where the cache warmed).

- Skip test_anchor_term_with_file on OSError (offline / rate-limited) instead
  of failing.
- Add test_anchor_term_with_fake_model: a fake tokenizer + tiny nn.Module
  exercise the identical _load_anchor + anchor_term lines with no network, so
  coverage no longer depends on HF availability.
- Add TestReachableInternals cushion (prompt_compile._resolve_metric,
  prompt_distill._build_provider_fn + default-provider wiring) so the gate
  sits comfortably above 77% (the DSPy/TextGrad/GEPA optimiser bodies are
  uncoverable without the [compile] extra).

Tests 13424 -> 13430.

2026-06-04 22:41:22 +05:00

Alpamys

f316d334bc

feat(rl): live GRPO/RL callbacks — reward-hack, echo-trap, RL ckpt, ULD, MiniLLM, iterative-DPO (v0.71.11)

Lifts the v0.70.0 schema-only build_*_callback / build_uld_projection /
run_iterative_dpo stubs. Validated end-to-end on SmolLM2-135M.

Closes #235, #236, #237, #238, #239, #240, #159, #160

- #235 RewardHackCallback: info_rm cluster-sep / rm_ensemble divergence,
  OK/WARN/HACK, halt on HACK. Shared thread-safe RLSignalBuffer captures
  per-step rewards by wrapping the reward fns (no TRL monkeypatching).
- #236 ULD: Wasserstein-1 / top-k aligned distill loss in DistillTrainer.
- #237 MiniLLM: teacher-mixed length-normalised reverse-KL + pretrain anchor.
- #238 RLCheckpointCallback: adapter + optimizer.pt + manifest + keep_last prune.
- #239 run_iterative_dpo: sample -> RM-score -> build-pairs -> DPO-train per round.
- #240 EchoTrapCallback: n-gram repetition OK/WARN/TRAP, halt on TRAP.
- #159 one-shot WARNING when a GRPO variant compute_loss falls back to super().
- #160 in-place ref-model EMA (no state_dict round-trip) + 0-overlap warning.

Tests 13142 -> 13203 (+62 in tests/test_v07111.py).

2026-06-04 17:14:08 +05:00

2 Commits