Six closes lifting the v0.49.0 LongLoRA hardening + v0.41.0 LLaMA Pro
deferred stubs, plus a UX upgrade to the CUDA-OOM friendly message:
- #11 utils/errors.py: OOM hint now names --batch-size / --grad-accum
- #122 flash_attn.is_flash_attn_v3_available() + LongLoRA+FA3 schema reject
- #120 LongLoRA arch allowlist expansion (Mistral / Qwen / Phi); Mixtral
intentionally excluded (regex matches the bare 'mistral' token only)
- #121 apply_long_context_config auto-detects 'llama3' when caller passes
rope_scaling_type=None and the model config carries a Llama 3.1
rope_scaling block
- #83 block_expansion.expand_model_blocks LIVE (deepcopy last-N blocks,
zero-init residual projections, append, bump num_hidden_layers) +
apply_llama_pro_freeze + shared apply_block_expansion_if_configured
helper wired into SFT + Pretrain (mirrors v0.40.6 peft_wiring
centralisation policy so SFT and Pretrain stay in lock-step)
- #74 HF push surface QA — test plan recorded in tests/qa/v053_qa.md;
live execution against a private HF repo deferred to a credentialed
contributor
Review pipeline (python / code / security / tdd agents) ran; every
CRITICAL -> LOW finding addressed:
- bool-first guards in _check_model_name (defends against int subclass)
- is_supported_longlora_arch defensive non-string surface (returns False,
never raises) matching v0.53.3 is_known_vlm_base policy
- _truncate_for_message(value, limit=64) bounds the base echo in
LongLoRA error messages (security MEDIUM, mirrors v0.34.0 crash.py)
- null-byte + non-string TypeError guards on validate_longlora_compat
task / backend params (matches v0.50.0 validate_long_context_grpo_compat)
- _get_layers_module uses explicit `is None` not falsy shortcut (defends
against nn.Module.__bool__ overrides on subclasses)
- _zero_init_block_residual returns bool + warnings.warn when neither
standard projection matches the cloned block (non-Llama-shaped arches
still train but lose the LLaMA Pro identity-init guarantee)
Test count: 7879 -> 7935 (+56 net; +49 in new tests/test_v0534.py).
Lint clean. CPU smoke verified on a real transformers.LlamaForCausalLM:
4 -> 6 layers, down_proj + o_proj actually zeroed on PyTorch tensors,
old blocks frozen + new blocks trainable, forward pass finite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- PPO: detect trl API via inspect — args= (>=0.28) vs config= (<0.28)
- PPO: split train into _train_builtin (trl >=0.28) and _train_manual
- GRPO: update error message to mention GRPO/PPO CPU limitation
- 2 new tests for PPO API detection (639 total, all passing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PPO: add use_cpu=True to PPOConfig when running on CPU
- GRPO: add CPU warning + use_cpu flag via inspect (trl bug workaround)
- Add use_cpu error pattern to friendly error map
- 7 new tests for CPU fixes (637 total, all passing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ASCII progress bar in display.py (replace Unicode blocks with #/-)
- Catch UnicodeEncodeError from plotext in data stats (Windows fallback)
- Auto-disable 4bit/8bit quantization on CPU (bitsandbytes doesn't support it)
- Friendly error messages for CPU tensor/dtype/bf16/torchvision issues
- Fix diff.py: torch_dtype= -> dtype= (deprecation warning)
- Add torchvision version compatibility check to soup doctor
7 new tests (631 total), ruff clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Friendly error messages: wrap all commands in try/except, map known errors
(CUDA OOM, missing deps, connection errors) to 2-3 line messages with fix hints
- Global --verbose flag for full tracebacks
- soup doctor: check system info, GPU, all dependency versions with fix suggestions
- soup quickstart: one-command demo (creates data + config + trains TinyLlama)
- Confirmation prompts before train/sweep (skip with --yes)
- 40 new tests (321 total), all passing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Alpamys