Commit Graph

4 Commits

Author SHA1 Message Date
Alpamys 06d8ea7cc5 chore: migrate to src-layout
Move soup_cli/ -> src/soup_cli/ (history preserved via git mv). src-layout
forces the test suite to import the installed package instead of the
repo-root source tree, surfacing packaging bugs that flat-layout masks —
e.g. the v0.53.8 double-shipped-fixtures regression, invisible because
`pytest tests/` imports ./soup_cli directly and never from the wheel.

- pyproject: packages = ["src/soup_cli"]; artifacts globs -> src/soup_cli/...
  The import name is unchanged, so the `soup` entry point, --cov=soup_cli,
  and report_to/module-path strings stay `soup_cli`.
- CI / ownership: ruff lint path (ci.yml), recipe-validation `paths:` filters,
  CODEOWNERS patterns, and the PR-template checklist all repointed to
  src/soup_cli/.
- tests: source-grep regression tests that read package files by repo-relative
  path repointed to src/soup_cli/ (64 files; 170 path literals). Lines pushed
  over 100 chars by the prefix were wrapped to keep ruff E501 clean. Module
  references (`import soup_cli`, `-m soup_cli`, mock.patch("soup_cli.x")) and
  the `--cov=soup_cli` coverage target are deliberately unchanged.
- docs: AGENTS.md + CONTRIBUTING.md structure tree and lint commands.

Verified locally: ruff clean (src/soup_cli + tests); `import soup_cli`
resolves to src/soup_cli/__init__.py; built wheel ships
soup_cli/data/_fixtures/*.jsonl (10 files, no duplicates, no src/ prefix);
3174 tests across every touched test file pass. Packaging-only — no version bump.
2026-05-31 12:40:06 +05:00
Alpamys 18d8b36114 feat(v0.53.7): Data Forge + Pipeline live (wave 1) — 11 items
Closes v0.47.0 deferrals (#111, #112), community QA (#75), v0.42.1 wave 1
(#87, #86, #88), and 5 v0.53.6 stub-to-live carry-overs (#102 vLLM parity
+ SSE streaming, #103 tool HTTP endpoints, #105 instantiate_trainer_plugins,
#106 run_recipe DAG runner).

- #88 markdown ingest heading split (split_markdown_by_headings)
- #112 soup data decontaminate --benchmark-file (cwd-contained operator
  corpus + symlink rejection)
- #87 prompt_strategy live resolver (resolve_prompt_strategy + lru_cache,
  importlib-based; per-row hook in sft_format.py)
- #86 soup data preprocess AOT tokenize (atomic Arrow shard write + cache
  metadata sidecar; SFT + Pretrain wrappers short-circuit on
  format='pre_tokenized' + tokenized_path with cache-hash gate)
- #111 forge --judge-provider {ollama,anthropic,vllm} live (lazy v0.20.0
  providers; SSRF parity)
- #75 QA log entry for synth-data provider manual smoke
- #106 run_recipe LIVE for 6 NODE_KINDS (seed / llm_text / code / judge /
  validator / sampler); atomic checkpoint via tempfile.mkstemp; resume
  rehydrates predecessor outputs from per-node sidecar JSONL; lstat-on-
  raw-path symlink rejection (v0.33.0 #22 TOCTOU parity); failed_reason
  path-redacted
- #105 instantiate_trainer_plugins LIVE for cce_plugin / grokfast /
  spectrum / llmcompressor / sonicmoe / math_verify (lazy imports,
  friendly pip-install advisory on missing dep)
- #103 POST /v1/tools/python + /v1/tools/web_search LIVE (Bearer auth gate,
  deny-by-default domain allowlist, 5s timeout, 5-result cap). bash
  reverted to HTTP 501 — security review caught /bin/sh -c child escapes
  RLVR sandbox's OS-level isolation; deferred to v0.53.8.
- #102 vLLM /v1/messages parity LIVE on both backends; CORS loopback-only
- #102 Anthropic-shape SSE streaming on /v1/messages LIVE; Cache-Control:
  no-store; CRLF/NUL/oversize strip on model+msg_id (header injection)

Review fixes (1 CRITICAL + 11 HIGH + 17 MEDIUM + 10 LOW from python-reviewer
+ code-reviewer + security-reviewer + tdd-guide) all addressed in this commit.

Test count: 8051 → 8162 (+111 in tests/test_v0537.py).
Test files: 189 → 190.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 19:28:11 +05:00
Alpamys c9e32f4e19 fix(test): v0.53.6 CI hardening — importorskip(fastapi) + strip ANSI
CI failures on `tests/test_v0536.py`:

1. `ModuleNotFoundError: No module named 'fastapi'` on Ubuntu/macOS runners
   — fastapi is in the [serve] extra, not [dev]. Added
   `pytest.importorskip("fastapi")` to all 7 tests that use TestClient
   (matches the pattern in `_build_app`).

2. `--execute` / `--output` / `v0.53.7` substring checks failed because
   CI terminals render Typer/Rich help text with style spans (`-` and
   `-execute` end up in separate `\x1b[...]m` runs). Added `_strip_ansi`
   helper + wrapped 5 substring checks. Same fix pattern as the v0.53.5
   `--live` CI fix on test_v0535.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 17:22:16 +05:00
Alpamys 70326f1aa7 feat(v0.53.6): plugin callback + Anthropic /v1/messages + n-gram spec + 3 stubs
Ships v0.53.6 "Plugin + Agent + Anthropic API" — 6 features, 3 live and
3 stub-then-live with v0.53.7 markers.

LIVE
- #101 SoupPluginCallback fans HF Trainer events to enabled Soup plugin
  hooks (pre_train/post_train/pre_step/post_step). Hook exceptions
  swallowed at WARNING. Hook snapshot collected once and passed to ctor
  (race-free per code-review fix). Wired into all 13 transformer-backend
  trainers via utils/peft_wiring.attach_plugin_callback.
- #102 POST /v1/messages on transformers backend reuses the v0.45.0
  anthropic_messages converter + existing chat handler. Streaming -> 501.
  Validation errors -> generic 'Invalid request' body, details at DEBUG
  (security-review redaction fix).
- #104 n-gram speculative decoding: NgramSpecConfig.num_draft_tokens
  threaded through model.generate(prompt_lookup_num_tokens=N).
  Mutually exclusive with assistant_model.

STUB-THEN-LIVE (v0.53.7)
- #103 /v1/tools/{python,bash,web_search} return HTTP 501.
- #105 instantiate_trainer_plugins validates then NotImplementedError.
- #106 run_recipe + 'soup data recipe --execute --output <dir>' with
       CLI-side is_under_cwd containment before the live runner.

Tests: 7998 -> 8051 (+53 in tests/test_v0536.py). Lint clean.
Three review agents (python / code / security); every HIGH/MEDIUM fixed.
TDD coverage gaps closed (ngram-None regression, max_tokens cap,
run_recipe boundaries, console-print failure swallow).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 17:10:16 +05:00