mirror of https://github.com/razor-ai/soup.git
1f050fc235
Three v0.X.0 deferred-stub features become live runtime — closes #33 (harvester judge filter + serve trace log) and #64 (live CUDA OOM probe). #65 (multipack live wiring in HF Trainer) remains deferred to v0.40.4 after the adversarial 5th-review pass surfaced a Sampler[int] vs list[list[int]] shape mismatch with HF Trainer's DataLoader; helpers (`make_multipack_trainer_class`, `attach_multipack_state`, `lengths_from_dataset`, `detect_arch_name`) ship as a stub used by unit tests, but the SFT/Pretrain wrappers print a yellow advisory and fall back to the standard sampler when `multipack: true`. Live CUDA OOM probe (#64): `make_cuda_probe_fn` builds a closure that runs ONE forward+backward+step on a synthetic batch per candidate. `model.zero_grad(set_to_none=True)` runs BEFORE forward; intermediate ids/attn/labels/outputs are del-ed before `loss.backward()` so peak VRAM reflects a realistic training step (matches v0.35.0 #45 policy). `pad_id` is bounded by `len(tokenizer)` (not `vocab_size`) so extended vocabs (Llama-3 + `<|pad|>`) don't fold pad to a random byte token. SFT-only this release. Trace-to-Preference judge filter (#33 (a)): `judge_filter_pairs` reuses v0.19.0 JudgeEvaluator backends (openai/server/ollama). Threshold rejects bool/NaN/out-of-[0,1]; `_MAX_BATCH=100_000` cap applied via lazy `itertools.islice`; per-pair backend exceptions caught and DEBUG- logged (matches v0.33.0 #47 policy); `judge_provider` validated against the allowlist at the CLI boundary BEFORE constructor with a Rich-escape error message; yellow projected-call-count warning before the loop (2× per pair). Inference Server trace log (#33 (b)): `TraceLogWriter` is thread-safe (single-process lock — multi-worker documented as known limitation); path containment via shared `is_under_cwd`; null-byte/empty/non-string path rejected; cap_mb bounds [1, 10000] with explicit bool rejection. Rotation (one backup retained) refuses symlink at the backup path via `os.lstat + stat.S_ISLNK` (matches v0.33.0 #22 TOCTOU policy). Secret redaction (`hf_*` ≥8, `sk-*` ≥16, `Bearer …` ≥8 with `.` excluded so end-of-sentence period survives) applied to prompt + response and recursively to caller-supplied `extra` dict values. Streaming SSE path also records (was a coverage gap caught in adversarial review). Behaviour change: v0.40.2 users with `auto_batch_size_strategy: probe` were silently getting the static fallback. v0.40.3 actually runs a CUDA probe on first run (~5–30s, cached per (model, max_length, quant, lora_r, gpu) tuple). Reviews: 5 agents (python, code, security, tdd, verification-loop). Verification-loop run twice — once shallow smoke (PASS), once adversarial bug-hunt which found C1/C2 (multipack live wiring crash — demoted to v0.40.4), H1 (streaming SSE missing trace log — fixed), H4 (vocab_size vs len(tokenizer) on extended vocabs — fixed), H3 (Bearer regex consumed trailing period — fixed), H2 (judge cost shock — warning added), M2 (empty lengths accepted — rejected), L1 (extra dict bypassed redaction — recursive walk added). Tests: 4756 → 4855 (+99 net new) across test_v0403_part_a/b/c.py. Lint clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| activation_offload.py | ||
| auto_quant.py | ||
| batch_probe.py | ||
| constants.py | ||
| convergence.py | ||
| crash.py | ||
| cross_doc_attn.py | ||
| curriculum.py | ||
| cut_ce.py | ||
| deepspeed.py | ||
| dpo_variants.py | ||
| encoding.py | ||
| errors.py | ||
| flash_attn.py | ||
| fp8.py | ||
| freeze.py | ||
| fsdp.py | ||
| galore.py | ||
| gpu.py | ||
| grad_accum.py | ||
| gradient_ckpt.py | ||
| hf.py | ||
| hf_space.py | ||
| jinja_analyzer.py | ||
| kernel_picker.py | ||
| launcher.py | ||
| liger.py | ||
| log_level.py | ||
| long_context.py | ||
| lr_finder.py | ||
| metrics.py | ||
| mii.py | ||
| mixed_precision.py | ||
| mlx.py | ||
| moe.py | ||
| multipack.py | ||
| multipack_sampler.py | ||
| multipack_trainer.py | ||
| neat_packing.py | ||
| ollama.py | ||
| paths.py | ||
| peft_builder.py | ||
| peft_patches.py | ||
| pipeline.py | ||
| preference_combine.py | ||
| profiler.py | ||
| profiling.py | ||
| qat.py | ||
| quality.py | ||
| quant_menu.py | ||
| registry.py | ||
| relora.py | ||
| replay.py | ||
| ring_attention.py | ||
| run_cost.py | ||
| sglang.py | ||
| spec_pairing.py | ||
| spike_recovery.py | ||
| structured_output.py | ||
| topology.py | ||
| tracing.py | ||
| trust_remote.py | ||
| unsloth.py | ||
| v028_features.py | ||
| vllm.py | ||
| warmup.py | ||
| why.py | ||