Alpamys
|
ed5fc3a8b3
|
feat(precision,rollout): live fp8/nvfp4 + vLLM sleep + openenv rollout + apple-adapter + delinearize-llama4 (v0.71.21)
Closes #141, #124, #125, #228, #97.
- #141: apply_fp8_attention (torchao float8 on attention projections, Hopper
gate) + apply_nvfp4 (NVFP4Config, Blackwell gate); partial-conversion honesty;
wired into the v0.28 speed/memory pipeline with yellow-advisory degrade.
- #124: vllm_sleep_mode live - create_vllm_engine(sleep_mode=True) +
vllm_sleep_cycle ctx (wake in finally) + TRL GRPOConfig hook probe.
- #125: openenv rollout fully live via training.rollout_func module:fn
resolver; rows replace the prompt dataset; art/ruler/nemo_gym honest dep
gates + _EXTERNAL_ROLLOUT_RUNNERS seam. Real GRPO train on SmolLM2-135M.
- #228: convert_apple_adapter live - PEFT LoRA <-> mlx-lm (both matrices
transpose, bf16 upcast, adapters.safetensors + num_layers, npz legacy read,
np.ascontiguousarray fix for safetensors non-contiguous mangling);
*-to-apple upstream-gated exit 3.
- #97: delinearize-llama4 live - [E*din,dout] -> [E,din,dout] per shard,
config.json expert-count probe + --num-experts, sidecar copy, atomic writes.
Review waves: 3 HIGH + ~8 MEDIUM + ~12 LOW fixed.
Tests: 13874 -> 14084 (+210 in tests/test_v07121.py).
Full suite: 13967 passed, 117 skipped. ruff clean.
|
2026-06-10 16:29:52 +05:00 |
|
Alpamys
|
6ec36b5ff6
|
feat: live FSDP-shard consolidation + serve KV-cache type + ONNX QA (v0.71.14)
Close the doable tail of the export-QA + deferred-stub family.
- #96 consolidate_shards: lazy-torch safetensors merge, TOCTOU-hardened
(enforce_under_cwd_and_no_symlink, weights_only=True, atomic_write_bytes),
16 GiB/shard cap, dup-key shape-conflict reject. --plan-only flag.
- #140 apply_kv_cache_type -> KvCacheRuntime; soup serve --kv-cache-type
(bf16/f16 dtype, q8_0 quantized cache + hqq probe, fp8 Hopper-gated).
- #71 ONNX export QA: tiny-GPT2 PASS, TinyLlama host-RAM-bound (qa doc).
- #70/#72/#144/#74/#79 deferred to INFRA-BLOCKED tail (kept open).
Tests 13430 -> 13476 (+46). ruff clean. Suite green, 78.52% cov.
|
2026-06-05 13:39:36 +05:00 |
|
Alpamys
|
c4ac3da695
|
feat(v0.44.0): Live Dashboard & UX - 21 features, +192 tests
Part A - Live monitoring: soup monitor (nvidia-smi panel), EMA + p95/p99
tail-latency stats, SSE training-stream schema, phone-visible URL +
ASCII-QR helper, llama-server timings parser + KV-cache bar, thread-safe
ToolOutputsBuffer + ToolCallTimer.
Part B - UX fixes: GracefulSaveHandler (first SIGINT saves, second stops),
.checkpoint_now trigger file (cwd-contained, symlink-rejected), desktop /
.command / .cmd shortcut builders, onboarding-wizard YAML renderer.
Part C - UI tabs: drop-in soup_cli/ui/plugins/*.py registry with kebab-case
name allowlist + 32-tab cap, API_HOST / API_PORT / API_KEY +
GRADIO_HOST / GRADIO_PORT env knobs.
Part D - Standalone CLIs: soup fetch (bundled examples + configs +
deepspeed_configs catalog), soup quantize (ergonomic alias), soup
merge-sharded-fsdp-weights, soup delinearize-llama4 (planners; live
runtime in v0.44.1), soup llama <sub> (closed-allowlist proxy with
filtered child env that drops HF_TOKEN / OPENAI_API_KEY /
ANTHROPIC_API_KEY), soup_cli.utils.sweep_config (separate sweep.yaml
loader), reasoning_parser allowlist for soup serve.
Security review fixes: fetch symlink-at-target rejection +
bundled-source commonpath check, write_trigger symlink rejection
(TOCTOU), llama child-env secret allowlist, onboarding output
cwd-containment, qr token moved from URL fragment to query string (so
server actually sees it), sweep-config scalar allowlist +
MappingProxyType[Tuple] immutability.
Code/Python review fixes: detect_apple_silicon clean rewrite (was buggy
parser-priority ternary), all frozen-dataclass List fields -> Tuple,
ToolOutputsBuffer -> collections.deque(maxlen=1000), os.path.realpath
over abspath, IPv6 host auto-bracketing per RFC 3986, frozenset over
mutable set, type hints on __exit__/_make_proxy.
Test count: 5628 -> 5820 (+192). Lint clean. Help output ASCII-safe
(em-dash check enforced by test_cli_subprocess).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-10 19:20:18 +05:00 |