RomanNum3ral
/
soup
mirror of https://github.com/razor-ai/soup.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
442 Commits 1 Branch 0 Tags 10 MiB
Commit Graph

4 Commits

Author SHA1 Message Date
Alpamys 18d8b36114 feat(v0.53.7): Data Forge + Pipeline live (wave 1) — 11 items 
Closes v0.47.0 deferrals (#111, #112), community QA (#75), v0.42.1 wave 1
(#87, #86, #88), and 5 v0.53.6 stub-to-live carry-overs (#102 vLLM parity
+ SSE streaming, #103 tool HTTP endpoints, #105 instantiate_trainer_plugins,
#106 run_recipe DAG runner).

- #88 markdown ingest heading split (split_markdown_by_headings)
- #112 soup data decontaminate --benchmark-file (cwd-contained operator
  corpus + symlink rejection)
- #87 prompt_strategy live resolver (resolve_prompt_strategy + lru_cache,
  importlib-based; per-row hook in sft_format.py)
- #86 soup data preprocess AOT tokenize (atomic Arrow shard write + cache
  metadata sidecar; SFT + Pretrain wrappers short-circuit on
  format='pre_tokenized' + tokenized_path with cache-hash gate)
- #111 forge --judge-provider {ollama,anthropic,vllm} live (lazy v0.20.0
  providers; SSRF parity)
- #75 QA log entry for synth-data provider manual smoke
- #106 run_recipe LIVE for 6 NODE_KINDS (seed / llm_text / code / judge /
  validator / sampler); atomic checkpoint via tempfile.mkstemp; resume
  rehydrates predecessor outputs from per-node sidecar JSONL; lstat-on-
  raw-path symlink rejection (v0.33.0 #22 TOCTOU parity); failed_reason
  path-redacted
- #105 instantiate_trainer_plugins LIVE for cce_plugin / grokfast /
  spectrum / llmcompressor / sonicmoe / math_verify (lazy imports,
  friendly pip-install advisory on missing dep)
- #103 POST /v1/tools/python + /v1/tools/web_search LIVE (Bearer auth gate,
  deny-by-default domain allowlist, 5s timeout, 5-result cap). bash
  reverted to HTTP 501 — security review caught /bin/sh -c child escapes
  RLVR sandbox's OS-level isolation; deferred to v0.53.8.
- #102 vLLM /v1/messages parity LIVE on both backends; CORS loopback-only
- #102 Anthropic-shape SSE streaming on /v1/messages LIVE; Cache-Control:
  no-store; CRLF/NUL/oversize strip on model+msg_id (header injection)

Review fixes (1 CRITICAL + 11 HIGH + 17 MEDIUM + 10 LOW from python-reviewer
+ code-reviewer + security-reviewer + tdd-guide) all addressed in this commit.

Test count: 8051 → 8162 (+111 in tests/test_v0537.py).
Test files: 189 → 190.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 19:28:11 +05:00
Alpamys 89b06efb8c feat(longctx): v0.53.4 — Long Context + Architecture 
Six closes lifting the v0.49.0 LongLoRA hardening + v0.41.0 LLaMA Pro
deferred stubs, plus a UX upgrade to the CUDA-OOM friendly message:

- #11   utils/errors.py: OOM hint now names --batch-size / --grad-accum
- #122  flash_attn.is_flash_attn_v3_available() + LongLoRA+FA3 schema reject
- #120  LongLoRA arch allowlist expansion (Mistral / Qwen / Phi); Mixtral
        intentionally excluded (regex matches the bare 'mistral' token only)
- #121  apply_long_context_config auto-detects 'llama3' when caller passes
        rope_scaling_type=None and the model config carries a Llama 3.1
        rope_scaling block
- #83   block_expansion.expand_model_blocks LIVE (deepcopy last-N blocks,
        zero-init residual projections, append, bump num_hidden_layers) +
        apply_llama_pro_freeze + shared apply_block_expansion_if_configured
        helper wired into SFT + Pretrain (mirrors v0.40.6 peft_wiring
        centralisation policy so SFT and Pretrain stay in lock-step)
- #74   HF push surface QA — test plan recorded in tests/qa/v053_qa.md;
        live execution against a private HF repo deferred to a credentialed
        contributor

Review pipeline (python / code / security / tdd agents) ran; every
CRITICAL -> LOW finding addressed:
- bool-first guards in _check_model_name (defends against int subclass)
- is_supported_longlora_arch defensive non-string surface (returns False,
  never raises) matching v0.53.3 is_known_vlm_base policy
- _truncate_for_message(value, limit=64) bounds the base echo in
  LongLoRA error messages (security MEDIUM, mirrors v0.34.0 crash.py)
- null-byte + non-string TypeError guards on validate_longlora_compat
  task / backend params (matches v0.50.0 validate_long_context_grpo_compat)
- _get_layers_module uses explicit `is None` not falsy shortcut (defends
  against nn.Module.__bool__ overrides on subclasses)
- _zero_init_block_residual returns bool + warnings.warn when neither
  standard projection matches the cloned block (non-Llama-shaped arches
  still train but lose the LLaMA Pro identity-init guarantee)

Test count: 7879 -> 7935 (+56 net; +49 in new tests/test_v0534.py).
Lint clean. CPU smoke verified on a real transformers.LlamaForCausalLM:
4 -> 6 layers, down_proj + o_proj actually zeroed on PyTorch tensors,
old blocks frozen + new blocks trainable, forward pass finite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 14:52:10 +05:00
Alpamys 2292e81c3f feat(modality): v0.53.2 — lift Modality II stubs (distill + classifier + EBFT/GDPO + reasoning_effort) 
Closes #132, #133, #135, #137. Records #71 ONNX QA (partial — tiny-gpt2
PASS, TinyLlama-1.1B blocked by host RAM during onnx.load post-process).

New trainer wrappers:
- DistillTrainerWrapper (soup_cli/trainer/distill.py) — student + frozen
  teacher, KL/JS divergence kernels scaled by T**2, device-bridge for HF
  Trainer auto-CUDA promotion, DataCollatorForSeq2Seq for variable-length
  loss-masked rows, separate trust_remote_code resolution per model.
- ClassifierTrainerWrapper (soup_cli/trainer/classifier.py) — single/multi
  label sequence classification, 1024-entry multi-label cap, label_names
  string-to-int resolution. Routes classifier / reranker / cross_encoder.

Live loss kernels:
- apply_ebft_loss (structured / strided) + attach_ebft_compute_loss (SFT)
- apply_gdpo_loss (standard / length_normalized / margin) +
  attach_gdpo_compute_loss (DPO). Both attach hooks idempotent.

Prompt-format wiring:
- apply_reasoning_effort_prefix injects gpt-oss
  <|reasoning_effort|>{low,medium,high}<|/reasoning_effort|> header.
- build_assistant_only_labels(train_on_eot=True) keeps EOT/EOS unmasked.

Bugs surfaced + fixed during Wave 3 CPU smoke (regression guards in tests):
- Distill collator did not pad pre-tokenised labels (variable-length crash)
- Distill compute_loss device-mismatch when HF Trainer auto-promoted
  student to CUDA while teacher stayed on CPU.

Tests: 7722 -> 7842 (+120 in test_v0532.py). 5 review agents run; every
CRITICAL/HIGH/MEDIUM/LOW finding fixed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 12:56:50 +05:00
Alpamys 725696b1da feat(v0.53.1): Quant Menu II + Export pipeline live 
Lift six v0.53.0 deferred stubs from NotImplementedError to live wiring:

- #82 autopilot pre-quantized base detection
  utils name regex over gptq/awq/aqlm/eetq/fp8/mxfp4 with word-boundary
  anchoring + HQQ Nbit extraction + config.json quantization_config probe
  (cwd-contained + symlink-rejected). decide_quantization() short-circuits
  the VRAM heuristic when prequantized is set. autopilot pipeline auto-
  applies so TheBloke/Llama-2-7B-Chat-GPTQ is recommended gptq instead of
  4bit-on-top-of-quantized.

- #142 merge_4bit + export_torchao live writers
  soup merge --save-format {fp16|4bit|4bit_forced}: single BNB-4bit
  merged checkpoint without the dequant->merge->requant cycle (fixes
  wrong-name llm_int8_skip_modules to bnb_4bit_skip_modules per code-
  review). soup export --format torchao --quant-config <yaml>: torchao
  .quantize_ + save_pretrained with per-scheme closed kwarg allowlist
  (Int4WeightOnly accepts {group_size, inner_k_tiles}, NVFP4 accepts
  nothing extra; dunder + unknown keys rejected per security-review H1).
  load_quant_config enforces yaml.safe_load + 256 KB cap + extension
  allowlist + cwd containment + S_ISLNK rejection.

- #139 export_advanced_gguf via llama.cpp imatrix
  3-stage pipeline: convert_hf_to_gguf.py -> optional imatrix ->
  quantize. argv-list subprocess (no shell), 30-min timeout, realpath-
  verified convert script stays inside llama_cpp_dir (security-review
  M5). _prepare_calibration_text accepts JSONL with text/prompt/content
  field aliases + raw text fallback; strips null bytes, collapses
  newlines, 8 KB per-line + 50 MB total cap (security-review M1); POSIX
  O_NOFOLLOW closes the TOCTOU window between dispatch-time check and
  open() (security-review M3). UD- prefix stripped before passing to
  llama-quantize. _safe_stderr Rich-escapes subprocess stderr before
  embedding in RuntimeError (security-review L4).

- #109 soup deploy autopilot --measure
  Live Quant-Lobotomy scorecard: classifies each candidate quant OK /
  MINOR / MAJOR (thresholds 2% / 5% mirror v0.26.0 Part D). Results
  cached at ~/.soup/deploy_autopilot_cache.json (atomic write, 0o600
  perms on POSIX, S_ISLNK rejection on BOTH load and save). pick_best
  soft-fallback now picks max-by-delta (was max-by-after) matching the
  v0.33.0 #54 design intent. _DEPLOY_MEASURE_BEFORE_GEN / _AFTER_FACTORY
  module-level hooks act as the stop-gap escape hatch until v0.46.1
  ships first-party transformers / vLLM generator factories.

- #70/#72 manual QA log scripted at tests/qa/v053_qa.md with exact
  reproduction recipes + acceptance criteria for the CUDA + llama.cpp
  smokes that can't run on the CI runners.

Shared cleanup:
- soup_cli/utils/paths.enforce_under_cwd_and_no_symlink consolidates the
  v0.33.0 #22 TOCTOU pattern previously copy-pasted in save_formats.py
  and gguf_quant.py (code-review HIGH fix).

Reviews ran: python / code / security / tdd. Every CRITICAL / HIGH /
MEDIUM / LOW finding fixed or documented.

Test count: 7610 -> 7722 (+112 across 4 new files).

Known limitations: live GPU + bitsandbytes / torchao smokes for the
new merge / export paths remain pending (recipes in QA log); injected-
generator escape hatch is non-public until v0.46.1; cache key truncates
base_sha to 16 hex (1-in-2^32 collision floor).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 00:16:03 +05:00