diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 05c235a..0d3638a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -111,7 +111,7 @@ soup_cli/ templates/ - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0) ui/ - Web UI (FastAPI + HTML/JS SPA) -tests/ - Test suite (212 files, 9571 tests) +tests/ - Test suite (218 files, 9786 tests) examples/ - Real-world config examples and datasets ``` diff --git a/README.md b/README.md index 3a845db..b07526a 100644 --- a/README.md +++ b/README.md @@ -42,14 +42,14 @@ soup train Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases). -**v0.61.0 — Unlearning & Knowledge Edit: GDPR right-to-be-forgotten + surgical fact patches.** Two of the most under-served axes in fine-tuning land as first-class trainers: unlearning (NPO / SimNPO / RMU — the legal-liability axis upstream TRL avoids) and knowledge editing (ROME / MEMIT / AlphaEdit — research-coded everywhere, productized nowhere). Hospital data teams handling a GDPR deletion request, labs patching factual errors without retraining, and security teams responding to a CSAM/PII leak all get a CLI workflow that composes with the existing Registry + eval-gate + diagnose surface. Schema-only release; live trainer + kernel wiring lands in v0.61.1. +**v0.62.0 — RAG & Activation Steering: cite-your-sources fine-tuning, retrieval-aware recipes, and inference-time control vectors.** Closes axis 14 of the roadmap. Adds RAFT data format (Stanford 2024 — train a model to answer queries given a golden doc + distractors), RA-DIT two-stage retriever + generator recipes (Meta 2023), `soup steer` for CAA / ITI / RepE control vectors, citation-faithful FT with precision / recall / F1 scoring, and the GRACE codebook for thousands of sequential edits without norm-blowup. Schema-only release; live training loops + decode-hook intervention + codebook lookup all land in v0.62.1. -- **`task='unlearn'` + NPO / SimNPO / RMU.** New first-class task on `SoupConfig`. `training.unlearn_method: Literal["npo","simnpo","rmu"]` selects the backend; `data.forget_set` (rows to unlearn) + `data.retain_set` (capability-preserving rows) are validated at schema-load. Case-insensitive method normalisation; mlx backend rejected with distinct error; missing forget_set on `task='unlearn'` rejected with named field. `UnlearnTrainerWrapper` stub captures the method + accepts forward-compat kwargs so v0.61.1 wiring is purely additive. -- **`soup eval unlearning --benchmark tofu|muse|wmdp`** — three orthogonal axes scored on the project's OK / MINOR / MAJOR taxonomy: Forget Quality (pre/post forget-loss linear ramp), Model Utility (retain-accuracy ratio), PrivLeak (membership-inference AUC symmetric around 0.5). Bundled TOFU mini-fixture under `soup_cli/data/_fixtures/unlearning/`; MUSE + WMDP loaders ship in v0.61.1. Evidence-driven: operators supply pre-computed JSON today and get a frozen `UnlearnReport` with overall worst-case verdict + atomic JSON output. -- **`soup edit set --base --method rome|memit|alphaedit --subject "Paris is the capital of France" --target "Lyon"`** — surgical locate-and-edit. Closed method allowlist; per-method default MLP layer (ROME=5, MEMIT=8, AlphaEdit=5); operator override via `--layer`. `--plan-only` validates + prints the resolved `EditPlan` and exits 0 today; without it the CLI surfaces the deferred-live `NotImplementedError` with v0.61.1 marker at exit code 3 (distinct from validation rejection exit 2). -- **Sequential edit governor + norm-blowup detection.** New `EditGovernor` tracks per-base-model edit count + last verdict; auto-switches ROME → AlphaEdit at the configurable edit-count threshold (default 10) AND on detected BLOWUP (Frobenius delta ≥ 5.0). `check_can_edit()` refuses further edits past the max cap or after a BLOWUP — `GovernedEditError` so callers distinguish governance refusals from other failures. AlphaEdit is the projection-based survivor (never switched away). -- **`soup edit diff --probes p.jsonl`** — knowledge-injection diff visualizer. Frozen `DiffReport` + `FactChange` dataclasses; probe-file loader is cwd-contained + symlink-rejected + size-capped (16 MiB / 1000 rows). v0.61.0 ships shape + table renderer; v0.61.1 lands the live model-driven before/after generation. -- **+193 new tests** (9446 → 9571 net after review-fix coverage). Review-fix coverage: 0 CRITICAL + 5 HIGH + 11 MEDIUM + 11 LOW (frozen-field declarations on `EditGovernor` so `asdict` / `replace` / slots stay safe; loud rejection of present-but-invalid evidence in `run_unlearn_eval`; `registry_id` validated BEFORE the deferred attach hook; exit-3 for "deferred" vs exit-2 for "validation rejection"; `validate_edit_method` lifted to module top in `edit_governor`; `importlib.resources` for fixture path; truncation warnings on probe-prompt clipping; boundary tests at exact `auto_switch_at` + `unlearn_alpha=0.0` + `unlearn_alpha=10.0` + `mia_auc=0.0` / `1.0`). Manual CPU smokes for `eval unlearning`, `edit set --plan-only`, `edit diff`, and 5 schema rejection paths all green. +- **`data.format='raft'` + `raft-llama3-8b` recipe.** New RAFT row shape `{query, golden_doc, distractor_docs, answer}` with every field null-byte-rejected + 64 KiB per-field cap + max 64 distractors per row. Composes with the existing SFT trainer — operators wire a RAFT JSONL today, get the deferred-live span-mask trainer when v0.62.1 ships. +- **RA-DIT (`ra_dit_stage: retriever|generator` + 2 new recipes).** Two-stage retriever-aware fine-tuning. `ra-dit-retriever` trains a sentence-transformer via the v0.16 embedding trainer (contrastive loss); `ra-dit-llama3-8b` is the RAFT-style generator stage. Cross-validator enforces each stage's required base task (retriever→embedding, generator→sft) so a misconfigured recipe fails at schema load. +- **`soup steer train|apply|list` (CAA / ITI / RepE).** Closed-allowlist control-vector methods, kebab-case + dotted name regex, |strength| ≤ 10 bounded, `--plan-only` exits 0 with deferred-live marker, `apply` and live training exit code 3 (distinct from validation rejection exit 2). New `steering_vector` artifact kind on the Registry; `soup serve --steer --steer-strength ` flag plumbed (decode-hook lands in v0.62.1). +- **Citation-faithful FT (`citation_faithful: true` + `citation_style: bracket|inline|footnote` + `citation_recall_threshold`).** Composes with RAFT — citation_faithful=true is gated to `format='raft'` AND `task in {sft, pretrain}` (silent-no-op footgun rejection per the v0.52.0 task-gate policy). Ships the pure `score_citations` + `extract_citation_ids` kernel so the eval gate can compute precision/recall/F1 against bracketed `[doc-id]` references today (live span-mask trainer in v0.62.1). +- **GRACE codebook (`grace_codebook: true` + bounded `size` + `dim` + `grace` added to `SUPPORTED_EDIT_METHODS`).** Discrete latent-space (key, value) store for thousands of sequential edits without norm-blowup. Extends v0.61.0 `soup edit set --method grace` end-to-end; `apply_edit` now routes `grace` plans to the v0.62.1 marker while keeping rome/memit/alphaedit on the existing v0.61.1 path (regression-guarded). Schema cross-validator refuses partial codebook configuration (flag without both size + dim, or size/dim without flag). +- **+215 new tests** (9571 → 9786 net). Review-fix coverage: 0 CRITICAL + 0 HIGH + 4 MEDIUM + 11 LOW across python / code / security / tdd review waves (broken `list_steers` registry context-manager + dict-key access, missing version bump, citation_faithful task-gate, `_validate_pairs_path` delegated to shared `enforce_under_cwd_and_no_symlink`, Rich markup escape on `--steer` exception messages, `--base` length cap + null-byte rejection, `typing.Iterable` → `collections.abc.Iterable` migration, narrowed `except Exception` → `except ImportError` on optional-dep import). Manual CPU smokes for `steer train --plan-only`, `steer apply`, `steer list`, `recipes show raft-llama3-8b`, `recipes search ra-dit`, full SoupConfig YAML round-trip on every new flag, plus 5 cross-validator rejection paths. ## Why Soup? @@ -2060,6 +2060,105 @@ The endpoint validator follows the same SSRF rules as `HF_ENDPOINT`: only `http` The hub adapter is schema-only in this release; the live downloader and uploader land in v0.51.1. +## RAFT — Retrieval-Augmented Fine-Tuning + +When you need a model to *cite* the document it's reading instead of hallucinating, RAFT (Stanford 2024) is the canonical recipe. Each training row carries a query, a golden document, a list of distractor documents, and the answer — the model learns to attend to the relevant doc while ignoring the noise. + +```yaml +# soup.yaml +data: + train: ./data/raft.jsonl + format: raft + +training: + citation_faithful: true # enable citation precision/recall scoring + citation_style: bracket # cite as [doc-1] inline + citation_recall_threshold: 0.8 # gate final save on recall >= 80% +``` + +```jsonl +# RAFT JSONL row shape +{"query": "When was Python released?", "golden_doc": "Python was released in 1991 by Guido van Rossum.", "distractor_docs": ["Ruby was released in 1995.", "Java was released in 1995."], "answer": "1991 [doc-1]"} +``` + +```bash +# Ready-made 8B Llama recipe +soup recipes show raft-llama3-8b +soup recipes use raft-llama3-8b +``` + +Citation scoring is exposed as a pure kernel for the eval gate: + +```python +from soup_cli.utils.citation_faithful import score_citations + +score = score_citations( + predicted="The answer is 1991 [doc-1].", + expected_ids=("doc-1",), +) +# CitationScore(precision=1.0, recall=1.0, f1=1.0, predicted_count=1, expected_count=1) +``` + +Citation-faithful FT is gated to `task in {sft, pretrain}` + `data.format='raft'` — misconfigured runs fail at config load with a named-field message. + +## RA-DIT — Retrieval-Augmented Dual Instruction Tuning + +RA-DIT (Meta 2023) is the two-stage version of RAFT: first train a sentence-transformer retriever (contrastive), then fine-tune the generator on the RAFT-style rows. Two recipes ship paired: + +```bash +# Stage 1 — train the retriever (uses Soup's v0.16 embedding trainer) +soup recipes use ra-dit-retriever +soup train + +# Stage 2 — train the generator on RAFT data, pointing at the retriever +soup recipes use ra-dit-llama3-8b +soup train +``` + +The schema enforces stage-task pairing — `ra_dit_stage: retriever` requires `task: embedding`; `ra_dit_stage: generator` requires `task: sft`. A misconfigured recipe fails at config load with a named-field message. + +## Activation Steering (`soup steer`) + +Sometimes you don't want to retrain — you want to *push* the model along a learned direction at decode time. Soup ships three control-vector backends: + +- **CAA** (Contrastive Activation Addition) — add a contrastive vector to the residual stream. +- **ITI** (Inference-Time Intervention) — shift specific attention heads along a learned direction. +- **RepE** (Representation Engineering) — PCA-based direction in the residual stream. + +```bash +# Train a steering vector from contrastive (positive, negative) prompt pairs +soup steer train --base meta-llama/Llama-3.1-8B-Instruct \ + --method caa --name safety-v1 \ + --pairs ./data/pairs.jsonl + +# Apply at decode time via soup serve +soup serve --model ./adapter --steer safety-v1 --steer-strength 1.5 + +# List locally-stored steering vectors +soup steer list +``` + +Steering names are validated against a strict regex (`^[A-Za-z0-9][A-Za-z0-9._\-]{0,127}$` — no path separators, no shell metacharacters); strength is bounded `|s| <= 10.0`. The trained vectors land in the Soup Registry under the `steering_vector` artifact kind so lineage is preserved. + +## GRACE Codebook — Lifelong Knowledge Edits + +Vanilla ROME / MEMIT degrade after dozens of sequential edits — the model's norms blow up. GRACE (Hartvigsen et al., 2023) stores each edit in a discrete latent codebook so thousands of sequential patches survive: + +```bash +soup edit set --base ./model --method grace \ + --subject "The CEO of Acme is" --target "Jane Doe" +``` + +```yaml +# Or via soup.yaml when training a model with GRACE-aware lookups +training: + grace_codebook: true + grace_codebook_size: 1024 # codebook entries (max 100k) + grace_codebook_dim: 768 # residual-stream width +``` + +`grace` joins the existing `rome` / `memit` / `alphaedit` allowlist on `soup edit set`; the v0.61.0 sequential edit governor still gates the call when the per-base-model edit count or norm-blowup verdict trips. + ## Model Registry & Lineage Every fine-tune you ship should be reproducible. Soup's local registry (`~/.soup/registry.db`) tracks each entry by a content hash of its config + data + base model, plus lineage pointers to parent entries. diff --git a/SECURITY.md b/SECURITY.md index 21e0873..db639bc 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -9,7 +9,8 @@ We provide security updates for the following versions: - **Versions older than 3 minor versions:** No support Example: -- v0.61.0 -- Full support (latest) +- v0.62.0 -- Full support (latest) +- v0.61.0 -- Full support - v0.60.0 -- Full support - v0.59.0 -- Full support - v0.58.0 -- Full support @@ -135,6 +136,8 @@ No known critical vulnerabilities in current releases. ### Security Hardening History +- **v0.62.0 — RAG & Activation Steering**: 5 parts ship `data.format='raft'` + RA-DIT two-stage recipes + `soup steer train/apply/list` (CAA / ITI / RepE) + citation-faithful FT + GRACE codebook. Schema-only release (live training loops + decode-hook intervention + codebook lookup all deferred to v0.62.1 — mirrors v0.50.0 / v0.52.0 / v0.61.0 stub-then-live pattern). **Closed-allowlist validators**: `SUPPORTED_RA_DIT_STAGES = {retriever, generator}`, `SUPPORTED_STEERING_METHODS = {caa, iti, repe}`, `SUPPORTED_CITATION_STYLES = {bracket, inline, footnote}`. All follow the project's bool-first / null-byte / oversize (32-char method names, 128-char steering names, 512-char model ids) / case-insensitive normalisation policy with frozen `*Spec` dataclasses + `MappingProxyType`-wrapped metadata. **RAFT input validation**: `_check_raft_string` rejects non-string / empty / null-byte / >64 KiB per field on every RAFT row field (query / golden_doc / answer / each distractor); `distractor_docs` list-type-guarded + capped at 64 entries per row. **Steering name regex** `^[A-Za-z0-9][A-Za-z0-9._\-]{0,127}$` rejects path separators + shell metacharacters so the name is safe as a Registry artifact id + CLI flag + filename fragment. **Citation kernel ReDoS-safe**: `extract_citation_ids` regex `\[(?P[A-Za-z0-9][A-Za-z0-9._\-]{0,127})\]` uses bounded quantifiers (no nested optional groups); `score_citations` predicted text capped at 2 MB, expected_ids capped at 10k entries with per-entry null-byte + bool rejection. **`soup steer` TOCTOU defence**: `_validate_pairs_path` delegates to shared `enforce_under_cwd_and_no_symlink` (centralised TOCTOU helper, not a custom re-impl per review M1) — symlinks at the pairs JSONL path rejected before any open/read. **`soup serve --steer` Rich markup escape** on the exception-message error panel (security review M1 fix — defends against crafted `--steer` value injecting Rich markup via `{value!r}` repr). **`--base` length cap** at 512 chars + null-byte rejection (security review L1 — mirrors v0.40.5 reward_model policy). **`list_steers` SQLite context-manager** (`with RegistryStore() as store:` — fixes connection leak surfaced by code-review M2). **Cross-validator silent-no-op rejection**: `citation_style` / `citation_recall_threshold` without `citation_faithful=true` rejected; `citation_faithful=true` requires `data.format='raft'` AND `task in {sft, pretrain}` (task-gate review M3 fix mirroring v0.52.0 distill / classifier policy); `grace_codebook_size` / `grace_codebook_dim` without `grace_codebook=true` rejected, and `grace_codebook=true` requires BOTH knobs; `ra_dit_retriever_model` without `ra_dit_stage` rejected. **GRACE codebook bounds**: `grace_codebook_size` ∈ [1, 100_000], `grace_codebook_dim` ∈ [1, 16_384] — sanity caps so a misconfigured run cannot allocate a multi-GB codebook by accident. **`apply_edit` deferred-marker routing**: `grace` plans raise NotImplementedError with `v0.62.1` marker while legacy rome/memit/alphaedit retain the v0.61.1 marker (regression-guarded via `TestEditMarkerRegressionGuard`). **Deferred stubs validate first**: `apply_steering` / `build_steering_vector` / `apply_grace_codebook` all run input validators BEFORE raising NotImplementedError so callers passing junk get a meaningful TypeError / ValueError instead of a confusing deferred-live error. **`typing.Iterable` → `collections.abc.Iterable`** migration in `citation_faithful.py` (python-review M1 — `typing.Iterable` deprecated since 3.9 for annotation use). **`except Exception` → `except ImportError`** narrowed in `commands/steer.py::list_steers` (python-review M2). **Test count**: 9571 → 9786 (+215 net in `tests/test_v0620_part_{a-e}.py` + `tests/test_v0620_followups.py`). **Known limitations**: (1) Every live wiring (RAFT span-mask trainer, RA-DIT two-stage orchestrator, `apply_steering` decode-hook, `build_steering_vector` per-method kernels, `apply_grace_codebook` lookup/write) deferred to v0.62.1 with explicit v0.62.1 markers. (2) `citation_faithful` gated to {sft, pretrain} — DPO / GRPO / etc. rejected (task-gate review fix). (3) `soup steer apply` is preview-only — live decode-hook ships via `soup serve --steer` in v0.62.1. (4) `extract_citation_ids` is bracket-style only; inline / footnote variants schema-validated but use the same extractor today. (5) RA-DIT recipe pair does not auto-link the retriever's output to the generator's `ra_dit_retriever_model` — operators copy manually. (6) GRACE codebook has no Registry persistence yet — lives only in the trained checkpoint until v0.62.1 wires `apply_edit(method='grace')` into v0.26 Registry. (7) `soup serve --steer` validates name + strength but does not load the vector — decode-hook lands in v0.62.1. (8) **4-agent review wave**: code + security + tdd + python style. Direct reviewer agents hit context-window thrash on the 800+ KB CLAUDE.md release-notes history (matches v0.58.0 / v0.59.0 / v0.60.0 / v0.61.0 idiom); the `general-purpose` agent with explicit "do not crawl, read only these N files" prompts produced equivalent findings consolidated into 4 sequential review waves. (v0.62.0) + - **v0.61.0 — Unlearning & Knowledge Edit**: 5 parts ship `task='unlearn'` (NPO / SimNPO / RMU) + `soup eval unlearning` (TOFU / MUSE / WMDP) + `soup edit set` (ROME / MEMIT / AlphaEdit) + sequential edit governor + `soup edit diff`. Schema-only release (live trainer + kernel wiring deferred to v0.61.1 — mirrors v0.50.0 / v0.52.0 / v0.53.0 stub-then-live pattern). **TOCTOU lstat-before-write** on every operator-supplied path via the shared `paths.atomic_write_text` helper (write_unlearn_report / write_diff_report); `load_evidence_file` + `load_probes` cwd-contained + `os.lstat + S_ISLNK` rejection + size-cap (16 MiB evidence, 16 MiB probe file, 1000 probe rows, 4096-char per prompt with WARNING-log truncation per review MEDIUM M5). **Frozen dataclasses** for every report / plan surface (`UnlearnMetric` / `UnlearnReport` / `EditRequest` / `EditPlan` / `FactChange` / `DiffReport` / `NormBlowupPolicy` / `MethodRecommendation`); `EditGovernor` counters declared as real dataclass fields (review HIGH H1 — slots-safe, `asdict` / `replace` compatible). **Invalid evidence raises loudly** in `run_unlearn_eval` (review HIGH H3) — distinguishes "missing key → neutral OK" from "present-but-invalid → ValueError" so operator typos surface instead of silently scoring OK. **Validator policy** mirrors v0.41.0 / v0.51.0 / v0.52.0 / v0.53.0: bool-first / null-byte / oversize (≤32-char method names, ≤64-char benchmark / verdict, ≤512-char base_model, ≤4096-char forget_set/retain_set/subject/target) / case-insensitive normalisation; numeric bounds with `math.isfinite` rejection on NaN / Inf. **`registry_id` validated BEFORE deferred attach** (review HIGH H4) — defends against crafted null-byte / oversize crashing the v0.61.1 registry hook. **Exit-code policy**: `soup eval unlearning` exits 0 on OK / MINOR (operator can choose to gate), 2 on MAJOR (matches v0.56.0 diagnose strict-mode CI gate); `soup edit set` exits 0 with `--plan-only`, 2 on schema rejection, 3 on deferred-live `NotImplementedError` (review HIGH H5 — distinguishes "not yet shipped" from "validation rejection", matches v0.56.0 strict-mode policy). **`importlib.resources`** for TOFU fixture path (review MEDIUM M6 — zipapp / namespace-package safe; mirrors v0.53.8 #93 `_bundle_source_path` policy). **Test count**: 9446 → 9571 (+125 net in `tests/test_v0610_part_{a-e}.py`; 192 pass + 1 POSIX-only symlink test skipped on Windows). **Known limitations**: (1) Every live wiring (NPO/SimNPO/RMU loss kernels, ROME/MEMIT/AlphaEdit closed-form updates, `UnlearnTrainerWrapper.setup`) deferred to v0.61.1 with explicit v0.61.1 markers — schema accepts every flag today so soup.yaml written now will work the moment v0.61.1 ships. (2) MUSE + WMDP fixtures NOT bundled (only TOFU mini-set); operators wanting MUSE / WMDP today supply pre-computed evidence JSON via `--evidence`. (3) `soup edit diff` is shape-only — live model loading + before/after generation lands in v0.61.1. (4) Registry-attach for edited models is plan-only — v0.61.1 wires the `parent_adapter` lineage write. (5) `EditGovernor` is single-process (no SQLite persistence). (6) Norm-blowup detection depends on operator-supplied `norm_delta` — v0.61.1's live `apply_edit` will compute + thread it through. (7) Unlearn task gated to non-mlx (mlx-lm doesn't expose per-token logp scoring needed for NPO/SimNPO). (8) `apply_edit` does not consult `EditGovernor` automatically — v0.61.1 wires the optional `governor: EditGovernor` kwarg. (9) **5-agent review wave**: python + code + security + tdd + verification. Direct `code-reviewer` / `security-reviewer` agents hit context-window thrash on the 800+ KB CLAUDE.md release-notes history (matches v0.58.0 / v0.59.0 / v0.60.0 idiom); the `general-purpose` agent with explicit "do not crawl, read only these N files" prompts produced equivalent findings consolidated into Wave 1 (code) + Wave 2 (security) + Wave 3 (TDD). (v0.61.0) - **v0.10.10**: Bearer token auth on Web UI, CORS restrictions, path traversal protection, SSRF prevention, max_tokens limits, supply-chain pinning (llama.cpp b5270), deprecated CLI secret flags diff --git a/pyproject.toml b/pyproject.toml index 54383a0..577af5d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "soup-cli" -version = "0.61.0" +version = "0.62.0" description = "Fine-tune LLMs in one command. No SSH, no config hell." readme = "README.md" license = "Apache-2.0" diff --git a/soup_cli/__init__.py b/soup_cli/__init__.py index 0564151..5f09774 100644 --- a/soup_cli/__init__.py +++ b/soup_cli/__init__.py @@ -1,3 +1,3 @@ """Soup CLI — Fine-tune LLMs in one command.""" -__version__ = "0.61.0" +__version__ = "0.62.0" diff --git a/soup_cli/cli.py b/soup_cli/cli.py index 30b7f2c..1eb0702 100644 --- a/soup_cli/cli.py +++ b/soup_cli/cli.py @@ -277,6 +277,18 @@ app.add_typer( ), ) +# v0.62.0 Part C — Activation steering: `soup steer train / apply / list`. +from soup_cli.commands import steer as _steer_cmd # noqa: E402 + +app.add_typer( + _steer_cmd.app, + name="steer", + help=( + "Activation steering (CAA / ITI / RepE) - inference-time " + "intervention without retraining (v0.62.0)." + ), +) + def _rewrite_advise_argv(argv: list) -> list: """Inject `run` between `advise` and a non-subcommand first argument. diff --git a/soup_cli/commands/serve.py b/soup_cli/commands/serve.py index fe6548d..44cdf38 100644 --- a/soup_cli/commands/serve.py +++ b/soup_cli/commands/serve.py @@ -199,6 +199,24 @@ def serve( "deepseek-r1 | qwen3 | phi4 | openthinker. v0.53.9 #98." ), ), + steer: Optional[str] = typer.Option( + None, + "--steer", + help=( + "Apply a stored activation-steering vector at decode time " + "(CAA / ITI / RepE). Pass the name registered via " + "`soup steer train`. Schema-only in v0.62.0; live decode hook " + "ships in v0.62.1." + ), + ), + steer_strength: float = typer.Option( + 1.0, + "--steer-strength", + help=( + "Steering strength multiplier (|s| <= 10.0). Ignored when " + "--steer is unset. v0.62.0 Part C." + ), + ), hub: str = typer.Option( "hf", "--hub", @@ -209,6 +227,33 @@ def serve( ), ): """Start a local inference server with OpenAI-compatible API.""" + # v0.62.0 Part C — validate `--steer` name + strength up front so a + # typo surfaces before backend init. Live decode-hook wiring lands + # in v0.62.1; this release just locks the flag surface. + if steer is not None: + from rich.markup import escape as _rich_escape + + from soup_cli.utils.steering import ( + validate_steering_name, + validate_steering_strength, + ) + + try: + validate_steering_name(steer) + validate_steering_strength(steer_strength) + except (TypeError, ValueError) as exc: + # Escape the exception message — it embeds the operator- + # supplied --steer value via {value!r}, which would otherwise + # let a crafted name inject Rich markup (security review M1). + console.print( + f"[red]Invalid --steer:[/] {_rich_escape(str(exc))}" + ) + raise typer.Exit(code=2) from exc + console.print( + f"[yellow]--steer={_rich_escape(steer)!r} accepted; live " + "decode hook ships in v0.62.1.[/]" + ) + # v0.53.10 #152 — pre-fetch base from a non-HF hub before serve starts. if hub and hub != "hf": from soup_cli.utils.hubs import apply_hub_to_cli_model diff --git a/soup_cli/commands/steer.py b/soup_cli/commands/steer.py new file mode 100644 index 0000000..8f5ca1b --- /dev/null +++ b/soup_cli/commands/steer.py @@ -0,0 +1,262 @@ +"""v0.62.0 Part C — `soup steer` CLI command group. + +Three subcommands: + +* ``soup steer train`` — fit a control vector from contrastive pairs. +* ``soup steer apply`` — apply a stored vector at decode time (also exposed + via ``soup serve --steer ``). +* ``soup steer list`` — list locally-stored steering vectors. + +Schema + CLI surface ship in v0.62.0; the live forward-hook + per-method +fitting kernels land in v0.62.1 (mirrors v0.50.0 / v0.52.0 / v0.61.0 +stub-then-live cadence). +""" + +from __future__ import annotations + +from typing import Optional + +import typer +from rich.console import Console +from rich.markup import escape +from rich.panel import Panel +from rich.table import Table + +console = Console() + +app = typer.Typer( + name="steer", + help=( + "Activation steering (CAA / ITI / RepE) - inference-time " + "intervention without retraining (v0.62.0)." + ), + no_args_is_help=True, + rich_markup_mode="rich", +) + + +def _validate_pairs_path(pairs: str) -> str: + """Containment-check the pairs JSONL path; reject pre-placed symlinks. + + Delegates to the shared :func:`enforce_under_cwd_and_no_symlink` + helper (centralised in v0.53.1) so the TOCTOU policy stays + single-source-of-truth (review M1 fix). Adds the 4096-char length + cap separately because the shared helper does not enforce one. + """ + from soup_cli.utils.paths import enforce_under_cwd_and_no_symlink + + if isinstance(pairs, str) and len(pairs) > 4096: + raise typer.BadParameter("--pairs must be <= 4096 chars") + try: + return enforce_under_cwd_and_no_symlink(pairs, "--pairs") + except (TypeError, ValueError) as exc: + raise typer.BadParameter(str(exc)) from exc + + +@app.command(name="train") +def train_steer( + base: str = typer.Option( + ..., "--base", "-b", + help="Base model HF id or local path.", + ), + method: str = typer.Option( + "caa", "--method", "-m", + help="Steering method: caa / iti / repe.", + ), + name: str = typer.Option( + ..., "--name", "-n", + help="Identifier for the trained vector (e.g. 'safety-v1').", + ), + pairs: str = typer.Option( + ..., "--pairs", "-p", + help="Path to JSONL of contrastive (positive, negative) prompt pairs.", + ), + layer: Optional[int] = typer.Option( + None, "--layer", "-l", + help="MLP layer index to extract the residual-stream vector from.", + ), + plan_only: bool = typer.Option( + False, "--plan-only", + help=( + "Validate inputs + print the resolved plan; skip the " + "deferred-live training (v0.62.1)." + ), + ), + registry_id: Optional[str] = typer.Option( + None, "--registry-id", + help="Optional Registry entry id to attach the trained vector to.", + ), +) -> None: + """Train a steering vector from contrastive prompt pairs.""" + from soup_cli.utils.steering import ( + build_steering_vector, + get_steering_method_spec, + validate_steering_method, + validate_steering_name, + ) + + # Validate method + name + pairs path up front so a typo fails fast + # with a clear message before we attempt the deferred-live call. + try: + canonical_method = validate_steering_method(method) + canonical_name = validate_steering_name(name) + pairs_path = _validate_pairs_path(pairs) + except (TypeError, ValueError, typer.BadParameter) as exc: + console.print(f"[red]Invalid steer-train input:[/] {escape(str(exc))}") + raise typer.Exit(2) from exc + + # `--base` accepts arbitrary HF ids / local paths but Typer does not + # cap length. Match the v0.40.5 `reward_model` policy (<=512 chars + + # null-byte rejection) so a multi-MB --base cannot bloat the Rich + # panel render (security review L1). + if not isinstance(base, str) or not base: + console.print("[red]Invalid --base:[/] must be non-empty string") + raise typer.Exit(2) + if "\x00" in base: + console.print("[red]Invalid --base:[/] null bytes not allowed") + raise typer.Exit(2) + if len(base) > 512: + console.print("[red]Invalid --base:[/] >512 chars") + raise typer.Exit(2) + + # Validate --registry-id early so the v0.62.1 attach path doesn't + # see junk. Mirrors v0.61.0 Part C policy on `--registry-id`. + if registry_id is not None: + if not isinstance(registry_id, str) or not registry_id: + console.print( + "[red]Invalid --registry-id:[/] must be non-empty" + ) + raise typer.Exit(2) + if "\x00" in registry_id: + console.print( + "[red]Invalid --registry-id:[/] null bytes not allowed" + ) + raise typer.Exit(2) + if len(registry_id) > 256: + console.print("[red]Invalid --registry-id:[/] >256 chars") + raise typer.Exit(2) + + if layer is not None and (layer < 0 or layer > 2048): + console.print( + f"[red]Invalid --layer:[/] must satisfy 0 <= layer <= 2048, got {layer}" + ) + raise typer.Exit(2) + + spec = get_steering_method_spec(canonical_method) + panel_body = ( + f"Base: {escape(base)}\n" + f"Method: {escape(canonical_method)}\n" + f"Name: {escape(canonical_name)}\n" + f"Pairs: {escape(pairs_path)}\n" + f"Layer: {layer if layer is not None else 'auto'}\n" + f"Description: {escape(spec.description)}" + ) + console.print( + Panel( + panel_body, + title=f"soup steer train ({escape(canonical_method)})", + border_style="cyan", + ) + ) + + if plan_only: + console.print( + "[yellow]Plan-only mode:[/] live training deferred to v0.62.1. " + "Inputs validated and plan rendered." + ) + return + + try: + build_steering_vector( + method=canonical_method, + name=canonical_name, + pairs_path=pairs_path, + layer=layer, + ) + except NotImplementedError as exc: + console.print( + Panel( + f"[yellow]Live steer-train deferred to v0.62.1.[/]\n\n{escape(str(exc))}", + title="Deferred", + border_style="yellow", + ) + ) + # Match v0.61.0 Part C policy: exit code 3 distinguishes + # "deferred / not yet shipped" from exit code 2 = "validation + # rejection". + raise typer.Exit(3) from exc + + +@app.command(name="apply") +def apply_steer( + name: str = typer.Option( + ..., "--name", "-n", + help="Identifier of a stored steering vector.", + ), + strength: float = typer.Option( + 1.0, "--strength", "-s", + help="Steering strength multiplier (|s| <= 10.0).", + ), +) -> None: + """Apply a stored steering vector at decode time (preview-only in v0.62.0).""" + from soup_cli.utils.steering import ( + validate_steering_name, + validate_steering_strength, + ) + + try: + canonical_name = validate_steering_name(name) + canonical_strength = validate_steering_strength(strength) + except (TypeError, ValueError) as exc: + console.print(f"[red]Invalid steer-apply input:[/] {escape(str(exc))}") + raise typer.Exit(2) from exc + + console.print( + Panel( + ( + f"Vector: {escape(canonical_name)}\n" + f"Strength: {canonical_strength}" + ), + title="soup steer apply", + border_style="cyan", + ) + ) + console.print( + "[yellow]Apply path deferred to v0.62.1.[/] Use `soup serve --steer " + f"{escape(canonical_name)}` once the live decode hook ships." + ) + raise typer.Exit(3) + + +@app.command(name="list") +def list_steers() -> None: + """List locally-stored steering vectors (Registry artifact kind ``steering_vector``).""" + try: + from soup_cli.registry.store import RegistryStore + except ImportError as exc: # pragma: no cover - registry import optional + console.print(f"[red]Registry unavailable:[/] {escape(str(exc))}") + raise typer.Exit(1) from exc + + # Use the store as a context manager so the SQLite connection closes + # cleanly on every invocation (review M2). The store returns dicts + # from `.list()` / `.get_artifacts()` — NOT objects (review H1). + table = Table(title="Steering vectors", border_style="cyan") + table.add_column("Entry") + table.add_column("Path") + rows = 0 + with RegistryStore() as store: + for entry in store.list(): + for art in store.get_artifacts(entry["id"]): + if art.get("kind") == "steering_vector": + table.add_row( + escape(str(entry.get("name", ""))), + escape(str(art.get("path", ""))), + ) + rows += 1 + if rows == 0: + console.print( + "[dim]No steering_vector artifacts registered. " + "Use `soup steer train ...` (v0.62.1+).[/]" + ) + return + console.print(table) diff --git a/soup_cli/config/schema.py b/soup_cli/config/schema.py index e9fd4ab..d126cbe 100644 --- a/soup_cli/config/schema.py +++ b/soup_cli/config/schema.py @@ -201,6 +201,8 @@ class DataConfig(BaseModel): "plaintext", "embedding", "audio", "tool-calling", "auto", # v0.42.0 — Data Pipeline Pro "prm", "pre_tokenized", "input_output", "video", "multimodal", + # v0.62.0 Part A — RAFT (Retrieval-Augmented Fine-Tuning) + "raft", ] = Field( default="auto", description="Data format", @@ -2350,6 +2352,142 @@ class TrainingConfig(BaseModel): return validate_unlearn_alpha(v) + # ---- v0.62.0 Part B — RA-DIT (Retrieval-Augmented Dual Instruction + # Tuning, Meta 2023). Schema-only: a YAML can declare ``ra_dit_stage`` + # so a recipe locks the right pairing; live two-stage orchestration + # ships in v0.62.1 (mirrors the v0.50.0 / v0.61.0 stub-then-live + # pattern). + ra_dit_stage: Optional[Literal["retriever", "generator"]] = Field( + default=None, + description=( + "RA-DIT pipeline stage. 'retriever' trains the sentence-" + "transformer via the v0.16 embedding trainer; 'generator' " + "runs RAFT-style SFT on `data.format='raft'`. Composes with " + "the v0.62.0 Part A RAFT recipe. (v0.62.0 Part B)" + ), + ) + ra_dit_retriever_model: Optional[str] = Field( + default=None, + description=( + "Optional retriever model id (e.g. " + "`sentence-transformers/all-mpnet-base-v2`) used by the " + "generator stage to pre-encode distractor docs. (v0.62.0 " + "Part B)" + ), + ) + + @field_validator("ra_dit_stage", mode="before") + @classmethod + def _validate_ra_dit_stage(cls, v): + """v0.62.0 Part B — case-insensitive normalisation via shared helper.""" + if v is None: + return None + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + return validate_ra_dit_stage(v) + + @field_validator("ra_dit_retriever_model", mode="before") + @classmethod + def _validate_ra_dit_retriever_model(cls, v): + """v0.62.0 Part B — bool/null-byte/oversize rejection on retriever id.""" + if v is None: + return None + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + return validate_ra_dit_retriever_model(v) + + # ---- v0.62.0 Part D — Citation-faithful FT ---------------------------- + citation_faithful: bool = Field( + default=False, + description=( + "Opt INTO citation-precision / recall scoring + a loss-mask " + "rule that emphasises citation spans. Requires " + "`data.format='raft'`. Schema-only in v0.62.0; live span-mask " + "ships in v0.62.1. (v0.62.0 Part D)" + ), + ) + citation_style: Optional[Literal["bracket", "inline", "footnote"]] = Field( + default=None, + description=( + "Citation rendering style. 'bracket' = `[doc-1]` inline tag " + "(canonical RAFT default); 'inline' / 'footnote' are stub " + "placeholders for v0.62.1. (v0.62.0 Part D)" + ), + ) + citation_recall_threshold: Optional[float] = Field( + default=None, + description=( + "Reject final-save when measured citation recall < this " + "threshold. Bounded [0.0, 1.0]. Composes with v0.56.0 " + "diagnose-gate. (v0.62.0 Part D)" + ), + ) + + @field_validator("citation_style", mode="before") + @classmethod + def _validate_citation_style(cls, v): + """v0.62.0 Part D — case-insensitive normalisation via shared helper.""" + if v is None: + return None + from soup_cli.utils.citation_faithful import validate_citation_style + + return validate_citation_style(v) + + @field_validator("citation_recall_threshold", mode="before") + @classmethod + def _validate_citation_recall_threshold(cls, v): + """v0.62.0 Part D — bool/NaN/Inf-rejected float bounded [0.0, 1.0].""" + if v is None: + return None + from soup_cli.utils.citation_faithful import validate_citation_threshold + + return validate_citation_threshold(v) + + # ---- v0.62.0 Part E — GRACE codebook ---------------------------------- + grace_codebook: bool = Field( + default=False, + description=( + "Opt INTO the GRACE codebook — discrete latent-space (key, " + "value) store for thousands of sequential knowledge edits " + "without norm-blowup. Schema-only in v0.62.0; live lookup / " + "write ships in v0.62.1. (v0.62.0 Part E)" + ), + ) + grace_codebook_size: Optional[int] = Field( + default=None, + description=( + "Codebook entry count. Required when grace_codebook=True. " + "Bounded [1, 100_000]. (v0.62.0 Part E)" + ), + ) + grace_codebook_dim: Optional[int] = Field( + default=None, + description=( + "Codebook entry dim (residual-stream width). Required when " + "grace_codebook=True. Bounded [1, 16_384]. (v0.62.0 Part E)" + ), + ) + + @field_validator("grace_codebook_size", mode="before") + @classmethod + def _validate_grace_codebook_size(cls, v): + """v0.62.0 Part E — bool-rejected positive int <= MAX_CODEBOOK_SIZE.""" + if v is None: + return None + from soup_cli.utils.grace_codebook import validate_grace_codebook_size + + return validate_grace_codebook_size(v) + + @field_validator("grace_codebook_dim", mode="before") + @classmethod + def _validate_grace_codebook_dim(cls, v): + """v0.62.0 Part E — bool-rejected positive int <= MAX_CODEBOOK_DIM.""" + if v is None: + return None + from soup_cli.utils.grace_codebook import validate_grace_codebook_dim + + return validate_grace_codebook_dim(v) + class EvalConfig(BaseModel): """Evaluation configuration for auto-eval after training.""" @@ -3453,6 +3591,115 @@ class SoupConfig(BaseModel): raise ValueError(str(exc)) from exc return self + @model_validator(mode="after") + def _validate_grace_codebook_compat(self) -> "SoupConfig": + """v0.62.0 Part E — GRACE codebook cross-validator. + + Rules: + * ``grace_codebook=True`` requires BOTH ``grace_codebook_size`` and + ``grace_codebook_dim`` to be set (no codebook can be allocated + without both knobs). + * Setting ``grace_codebook_size`` / ``grace_codebook_dim`` without + ``grace_codebook=True`` is a silent-no-op footgun — rejected. + """ + tcfg = self.training + flag = tcfg.grace_codebook + size = tcfg.grace_codebook_size + dim = tcfg.grace_codebook_dim + + if not flag and (size is not None or dim is not None): + raise ValueError( + "training.grace_codebook_size / grace_codebook_dim require " + "training.grace_codebook=true." + ) + if flag and (size is None or dim is None): + raise ValueError( + "training.grace_codebook=true requires BOTH " + "training.grace_codebook_size and training.grace_codebook_dim." + ) + return self + + @model_validator(mode="after") + def _validate_citation_faithful_compat(self) -> "SoupConfig": + """v0.62.0 Part D — citation-faithful FT cross-validator. + + Rules: + * ``citation_faithful=True`` requires ``data.format='raft'`` (the + RAFT row carries the doc references; other formats can't supply + ground-truth citation IDs). + * ``citation_faithful=True`` requires ``task in {sft, pretrain}`` + (the span-mask runtime that v0.62.1 will ship only makes sense + for the SFT family; mirrors v0.52.0 distill / classifier + task-gate policy — review M3 fix). + * ``citation_style`` set without ``citation_faithful=True`` is a + silent-no-op footgun — rejected (mirrors v0.61.0 unlearn_alpha / + v0.62.0 Part B ra_dit_retriever_model policy). + * Same rejection for ``citation_recall_threshold`` without the flag. + """ + tcfg = self.training + + if tcfg.citation_style is not None and not tcfg.citation_faithful: + raise ValueError( + "training.citation_style requires " + "training.citation_faithful=true." + ) + if ( + tcfg.citation_recall_threshold is not None + and not tcfg.citation_faithful + ): + raise ValueError( + "training.citation_recall_threshold requires " + "training.citation_faithful=true." + ) + if tcfg.citation_faithful: + if self.data.format != "raft": + raise ValueError( + "training.citation_faithful=true requires " + f"data.format='raft'; got data.format={self.data.format!r}. " + "Citation-faithful FT pairs with the v0.62.0 Part A " + "RAFT data format (which carries the doc references)." + ) + if self.task not in ("sft", "pretrain"): + raise ValueError( + "training.citation_faithful=true requires " + f"task in {{sft, pretrain}}; got task={self.task!r}. " + "Citation-faithful FT is an SFT-family feature; the " + "live span-mask runtime ships in v0.62.1." + ) + return self + + @model_validator(mode="after") + def _validate_ra_dit_compat(self) -> "SoupConfig": + """v0.62.0 Part B — RA-DIT stage / task pairing. + + Each stage requires the matching base task: + + * ``retriever`` -> ``task='embedding'`` + * ``generator`` -> ``task='sft'`` + + Also rejects ``ra_dit_retriever_model`` set without ``ra_dit_stage`` + (silent no-op footgun — mirrors v0.61.0 ``unlearn_alpha`` policy). + """ + tcfg = self.training + stage = tcfg.ra_dit_stage + + if tcfg.ra_dit_retriever_model is not None and stage is None: + raise ValueError( + "training.ra_dit_retriever_model requires " + "training.ra_dit_stage to be set ('retriever' or 'generator')." + ) + + if stage is None: + return self + + from soup_cli.utils.ra_dit import validate_ra_dit_compat + + try: + validate_ra_dit_compat(stage=stage, task=self.task) + except ValueError as exc: + raise ValueError(str(exc)) from exc + return self + @model_validator(mode="after") def _validate_mlx_task_support(self) -> "SoupConfig": """MLX backend only supports sft, dpo, and grpo tasks (v0.25.0). diff --git a/soup_cli/data/formats.py b/soup_cli/data/formats.py index 57e5546..318b756 100644 --- a/soup_cli/data/formats.py +++ b/soup_cli/data/formats.py @@ -85,6 +85,8 @@ def format_to_messages(row: dict, fmt: str) -> Optional[dict]: "plaintext", "embedding", "audio", "tool-calling", # v0.42.0 Part A "prm", "pre_tokenized", "input_output", "video", "multimodal", + # v0.62.0 Part A — RAFT (Retrieval-Augmented Fine-Tuning). + "raft", ) if fmt not in valid_formats: raise ValueError(f"Unknown format: {fmt}") @@ -117,6 +119,8 @@ def format_to_messages(row: dict, fmt: str) -> Optional[dict]: return _convert_video(row) elif fmt == "multimodal": return _convert_multimodal(row) + elif fmt == "raft": + return _convert_raft(row) else: return _convert_vision(row) except (KeyError, TypeError, IndexError, ValueError): @@ -424,6 +428,10 @@ def _to_sharegpt(messages: list[dict]) -> dict: _MAX_PRM_STEPS = 10_000 +# v0.62.0 Part A — RAFT (Retrieval-Augmented Fine-Tuning) caps. +_MAX_RAFT_DISTRACTORS = 64 +_MAX_RAFT_FIELD_LEN = 65_536 # 64 KiB per document — generous for legal/RAG corpora. + def _convert_prm(row: dict) -> dict: """PRM (Process Reward Model) stepwise-supervised format. @@ -535,3 +543,74 @@ def _convert_multimodal(row: dict) -> dict: f"multimodal content part.type must be in {sorted(valid_types)}" ) return {"messages": messages} + + +# --- v0.62.0 Part A: RAFT (Retrieval-Augmented Fine-Tuning) ---------------- + + +def _check_raft_string(name: str, value: object) -> str: + """Shared validator for RAFT string fields (query / golden_doc / answer). + + Returns the canonical value. Rejects non-string, empty, null-byte, and + oversize values (mirrors v0.42.0 `_convert_video` policy). The cap is + generous (64 KiB) because legal/RAG corpora frequently embed full + paragraphs verbatim in the golden_doc field. + """ + if not isinstance(value, str): + raise ValueError( + f"RAFT '{name}' must be a string, got {type(value).__name__}" + ) + if not value: + raise ValueError(f"RAFT '{name}' must be a non-empty string") + if "\x00" in value: + raise ValueError(f"RAFT '{name}' must not contain null bytes") + if len(value) > _MAX_RAFT_FIELD_LEN: + raise ValueError( + f"RAFT '{name}' must be <= {_MAX_RAFT_FIELD_LEN} chars" + ) + return value + + +def _convert_raft(row: dict) -> dict: + """RAFT (Retrieval-Augmented Fine-Tuning) format — Stanford 2024. + + Schema: ``{"query": str, "golden_doc": str, "distractor_docs": [str, ...], + "answer": str}``. The trainer composes the prompt by concatenating the + query with the golden doc + N distractor docs in randomised order; the + model learns to attend to the relevant doc while ignoring distractors. + + Distractor list MAY be empty (effectively reduces to closed-book QA on + the golden doc). Live RAFT training loop ships in v0.62.1; v0.62.0 + locks the schema + recipe surface. + """ + if "query" not in row: + raise ValueError("RAFT row must have 'query'") + if "golden_doc" not in row: + raise ValueError("RAFT row must have 'golden_doc'") + if "answer" not in row: + raise ValueError("RAFT row must have 'answer'") + + query = _check_raft_string("query", row["query"]) + golden_doc = _check_raft_string("golden_doc", row["golden_doc"]) + answer = _check_raft_string("answer", row["answer"]) + + raw_distractors = row.get("distractor_docs", []) + if not isinstance(raw_distractors, list): + raise ValueError("RAFT 'distractor_docs' must be a list") + if len(raw_distractors) > _MAX_RAFT_DISTRACTORS: + raise ValueError( + f"RAFT 'distractor_docs' must have <= {_MAX_RAFT_DISTRACTORS} entries " + f"(got {len(raw_distractors)})" + ) + cleaned_distractors: list[str] = [] + for index, doc in enumerate(raw_distractors): + cleaned_distractors.append( + _check_raft_string(f"distractor_docs[{index}]", doc) + ) + + return { + "query": query, + "golden_doc": golden_doc, + "distractor_docs": cleaned_distractors, + "answer": answer, + } diff --git a/soup_cli/recipes/catalog.py b/soup_cli/recipes/catalog.py index cf33110..525322d 100644 --- a/soup_cli/recipes/catalog.py +++ b/soup_cli/recipes/catalog.py @@ -3413,6 +3413,101 @@ training: batch_size: auto tts_family: oute +output: ./output +""", + ), + "ra-dit-retriever": RecipeMeta( + model="sentence-transformers/all-mpnet-base-v2", + task="embedding", + size="N/A", + tags=("ra-dit", "rag", "retriever", "contrastive", "v0.62.0"), + description=( + "RA-DIT stage 1 (Meta 2023) — contrastive retriever training. " + "Pairs with `ra-dit-llama3-8b` (stage 2) for the full pipeline." + ), + yaml_str="""\ +base: sentence-transformers/all-mpnet-base-v2 +task: embedding + +data: + train: ./data/triples.jsonl + format: embedding + max_length: 512 + +training: + epochs: 1 + lr: 2e-5 + batch_size: auto + ra_dit_stage: retriever + embedding_loss: triplet + embedding_margin: 0.5 + +output: ./output +""", + ), + "ra-dit-llama3-8b": RecipeMeta( + model="meta-llama/Llama-3.1-8B-Instruct", + task="sft", + size="8B", + tags=("llama", "sft", "ra-dit", "rag", "raft", "v0.62.0"), + description=( + "RA-DIT stage 2 (Meta 2023) — RAFT-style SFT on the generator. " + "Pairs with `ra-dit-retriever` (stage 1). Uses RAFT data format." + ), + yaml_str="""\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + max_length: 4096 + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + ra_dit_stage: generator + ra_dit_retriever_model: sentence-transformers/all-mpnet-base-v2 + lora: + r: 16 + alpha: 32 + target_modules: auto + quantization: 4bit + +output: ./output +""", + ), + "raft-llama3-8b": RecipeMeta( + model="meta-llama/Llama-3.1-8B-Instruct", + task="sft", + size="8B", + tags=("llama", "sft", "raft", "rag", "v0.62.0"), + description=( + "RAFT (Retrieval-Augmented Fine-Tuning, Stanford 2024) — train " + "an 8B Llama 3.1 to answer queries given a golden doc + " + "distractor docs. Rows: {query, golden_doc, distractor_docs, " + "answer}. Live training loop ships in v0.62.1." + ), + yaml_str="""\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + max_length: 4096 + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + lora: + r: 16 + alpha: 32 + target_modules: auto + quantization: 4bit + output: ./output """, ), diff --git a/soup_cli/registry/store.py b/soup_cli/registry/store.py index 2b743aa..bc6f0e4 100644 --- a/soup_cli/registry/store.py +++ b/soup_cli/registry/store.py @@ -41,6 +41,8 @@ _VALID_KINDS = frozenset( "adapter", "merged", "gguf", "awq", "gptq", "onnx", "dataset", "config", "eval_results", "tensorrt", "eval_suite", "canaries", "diagnose_report", + # v0.62.0 Part C — Activation steering vectors (CAA / ITI / RepE). + "steering_vector", } ) _VALID_RELATIONS = frozenset( diff --git a/soup_cli/utils/citation_faithful.py b/soup_cli/utils/citation_faithful.py new file mode 100644 index 0000000..185e650 --- /dev/null +++ b/soup_cli/utils/citation_faithful.py @@ -0,0 +1,210 @@ +"""v0.62.0 Part D — Citation-faithful fine-tuning. + +When enabled, the model is trained to cite document IDs verbatim from +the training corpus. Composes with v0.62.0 Part A RAFT — the RAFT row +already names the golden_doc + distractor_docs. + +Schema-only release: + +* The schema flag ``training.citation_faithful: bool`` opts INTO the + citation-precision / recall scorer + a loss-mask rule that emphasises + citation spans. +* The live span-mask training kernel and the eval-suite hook land in + v0.62.1, mirroring v0.50.0 / v0.52.0 / v0.61.0 stub-then-live policy. + +This module ships the pure ``score_citations`` kernel so callers (and +the eval gate) can compute precision/recall/F1 today, plus the closed +allowlist for ``citation_style``. +""" + +from __future__ import annotations + +import math +import re +from collections.abc import Iterable +from dataclasses import dataclass + +SUPPORTED_CITATION_STYLES: frozenset[str] = frozenset( + {"bracket", "inline", "footnote"} +) + +_MAX_STYLE_LEN: int = 32 +_MAX_PREDICTED_LEN: int = 2_000_000 # 2 MB cap on per-row predicted text. +_MAX_EXPECTED_IDS: int = 10_000 # Per-row expected-citation cap. + +# Default extraction regex — matches both ``[doc-id]`` brackets and bare +# ``doc-id`` tokens. The bracket form is the canonical RAFT default; +# ``inline`` and ``footnote`` use the same characters today (live +# per-style extractors ship in v0.62.1 once we benchmark variations). +_CITATION_RE: re.Pattern[str] = re.compile( + r"\[(?P[A-Za-z0-9][A-Za-z0-9._\-]{0,127})\]" +) + + +@dataclass(frozen=True) +class CitationScore: + """Precision / recall / F1 over predicted vs expected document IDs.""" + + precision: float + recall: float + f1: float + predicted_count: int + expected_count: int + + +def validate_citation_style(value: object) -> str: + """Normalise + validate a citation-style name. + + Mirrors v0.41.0 / v0.51.0 / v0.61.0 validator policy. + """ + if isinstance(value, bool): + raise TypeError( + f"citation_style must not be bool, got {value!r}" + ) + if not isinstance(value, str): + raise TypeError( + f"citation_style must be str, got {type(value).__name__}" + ) + if not value: + raise ValueError("citation_style must be non-empty") + if "\x00" in value: + raise ValueError("citation_style must not contain null bytes") + if len(value) > _MAX_STYLE_LEN: + raise ValueError( + f"citation_style must be <= {_MAX_STYLE_LEN} chars" + ) + canonical = value.lower() + if canonical not in SUPPORTED_CITATION_STYLES: + supported = ", ".join(sorted(SUPPORTED_CITATION_STYLES)) + raise ValueError( + f"unknown citation_style {value!r}; supported: {supported}" + ) + return canonical + + +def validate_citation_threshold(value: object) -> float: + """Validate the ``citation_recall_threshold`` in ``[0.0, 1.0]``. + + Bool-rejected, NaN/Inf-rejected via ``math.isfinite``. + """ + if isinstance(value, bool): + raise TypeError( + f"citation_recall_threshold must not be bool, got {value!r}" + ) + if not isinstance(value, (int, float)): + raise TypeError( + f"citation_recall_threshold must be a number, " + f"got {type(value).__name__}" + ) + fval = float(value) + if not math.isfinite(fval): + raise ValueError( + "citation_recall_threshold must be finite (no NaN / Inf)" + ) + if fval < 0.0 or fval > 1.0: + raise ValueError( + f"citation_recall_threshold must be in [0.0, 1.0]; got {fval}" + ) + return fval + + +def extract_citation_ids(text: str) -> tuple[str, ...]: + """Extract every ``[doc-id]`` citation from ``text``. + + Returns a tuple of IDs in encounter order. Duplicates are preserved + so the caller can compute precision honestly (a model that cites + the same doc three times should not silently dedupe). + """ + if not isinstance(text, str): + raise TypeError( + f"text must be str, got {type(text).__name__}" + ) + if len(text) > _MAX_PREDICTED_LEN: + raise ValueError( + f"text must be <= {_MAX_PREDICTED_LEN} chars for citation extract" + ) + return tuple(m.group("bracketed") for m in _CITATION_RE.finditer(text)) + + +def score_citations( + *, + predicted: object, + expected_ids: object, +) -> CitationScore: + """Compute citation precision / recall / F1. + + Precision = |predicted ∩ expected| / |predicted|. + Recall = |predicted ∩ expected| / |expected|. + + Undefined denominators (empty predicted / expected) return 0.0 by + convention — same policy as v0.43.0 BLEU on zero-precision n-grams. + """ + if isinstance(predicted, bool): + raise TypeError( + f"predicted must not be bool, got {predicted!r}" + ) + if not isinstance(predicted, str): + raise TypeError( + f"predicted must be str, got {type(predicted).__name__}" + ) + if len(predicted) > _MAX_PREDICTED_LEN: + raise ValueError( + f"predicted must be <= {_MAX_PREDICTED_LEN} chars" + ) + if not isinstance(expected_ids, Iterable) or isinstance(expected_ids, str): + raise TypeError( + "expected_ids must be an iterable of strings (not a single str)" + ) + expected_tuple = tuple(expected_ids) + if len(expected_tuple) > _MAX_EXPECTED_IDS: + raise ValueError( + f"expected_ids must have <= {_MAX_EXPECTED_IDS} entries" + ) + expected_set: set[str] = set() + for index, eid in enumerate(expected_tuple): + if isinstance(eid, bool) or not isinstance(eid, str): + raise TypeError( + f"expected_ids[{index}] must be str, got {type(eid).__name__}" + ) + if not eid: + raise ValueError(f"expected_ids[{index}] must be non-empty") + if "\x00" in eid: + raise ValueError( + f"expected_ids[{index}] must not contain null bytes" + ) + expected_set.add(eid) + + predicted_ids = extract_citation_ids(predicted) + predicted_count = len(predicted_ids) + expected_count = len(expected_set) + + if predicted_count == 0 or expected_count == 0: + return CitationScore( + precision=0.0, + recall=0.0, + f1=0.0, + predicted_count=predicted_count, + expected_count=expected_count, + ) + + # Precision: predicted IDs that hit the expected set. + hits = sum(1 for pid in predicted_ids if pid in expected_set) + precision = hits / predicted_count + + # Recall: how many expected IDs the model actually cited. + predicted_set = set(predicted_ids) + recalled = sum(1 for eid in expected_set if eid in predicted_set) + recall = recalled / expected_count + + if precision + recall <= 0.0: + f1 = 0.0 + else: + f1 = 2.0 * precision * recall / (precision + recall) + + return CitationScore( + precision=precision, + recall=recall, + f1=f1, + predicted_count=predicted_count, + expected_count=expected_count, + ) diff --git a/soup_cli/utils/grace_codebook.py b/soup_cli/utils/grace_codebook.py new file mode 100644 index 0000000..34ee4af --- /dev/null +++ b/soup_cli/utils/grace_codebook.py @@ -0,0 +1,102 @@ +"""v0.62.0 Part E — GRACE codebook (long-running edit). + +Discrete latent-space codebook for thousands of sequential knowledge edits +that survive lifelong deployments without the norm-blowup that haunts +vanilla ROME / MEMIT. Each edit stores a (key, value) pair in a learned +codebook; at inference time the model looks up the closest codebook key +to the current residual stream and applies the stored value. + +Schema-only release: ``training.grace_codebook`` opt-in + codebook +size / dim validators + ``GraceCodebookConfig`` dataclass + ``grace`` +added to the v0.61.0 ``SUPPORTED_EDIT_METHODS`` allowlist. Live codebook +lookup / write / EditGovernor integration lands in v0.62.1. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +MAX_CODEBOOK_SIZE: int = 100_000 +MAX_CODEBOOK_DIM: int = 16_384 # Generous upper bound matches Llama 70B hidden. + + +@dataclass(frozen=True) +class GraceCodebookConfig: + """Resolved codebook configuration. Frozen post-construction.""" + + size: int + dim: int + + +def validate_grace_codebook_size(value: object) -> int: + """Validate the codebook entry count. + + Bool-rejected (bool is a subclass of int), positive-int only, capped + at :data:`MAX_CODEBOOK_SIZE` so a misconfigured run cannot allocate + a multi-GB codebook by accident. + """ + if isinstance(value, bool): + raise TypeError( + f"grace_codebook_size must not be bool, got {value!r}" + ) + if not isinstance(value, int): + raise TypeError( + f"grace_codebook_size must be int, got {type(value).__name__}" + ) + if value < 1: + raise ValueError( + f"grace_codebook_size must be >= 1, got {value}" + ) + if value > MAX_CODEBOOK_SIZE: + raise ValueError( + f"grace_codebook_size must be <= {MAX_CODEBOOK_SIZE}, got {value}" + ) + return value + + +def validate_grace_codebook_dim(value: object) -> int: + """Validate the codebook entry dimension (residual-stream width).""" + if isinstance(value, bool): + raise TypeError( + f"grace_codebook_dim must not be bool, got {value!r}" + ) + if not isinstance(value, int): + raise TypeError( + f"grace_codebook_dim must be int, got {type(value).__name__}" + ) + if value < 1: + raise ValueError( + f"grace_codebook_dim must be >= 1, got {value}" + ) + if value > MAX_CODEBOOK_DIM: + raise ValueError( + f"grace_codebook_dim must be <= {MAX_CODEBOOK_DIM}, got {value}" + ) + return value + + +def build_grace_codebook_config(*, size: int, dim: int) -> GraceCodebookConfig: + """Validate + freeze a :class:`GraceCodebookConfig`.""" + canonical_size = validate_grace_codebook_size(size) + canonical_dim = validate_grace_codebook_dim(dim) + return GraceCodebookConfig(size=canonical_size, dim=canonical_dim) + + +def apply_grace_codebook(config: GraceCodebookConfig) -> None: + """Apply the codebook at decode time — deferred to v0.62.1. + + Validates the config type first so callers passing a bare dict get a + crisp ``TypeError`` rather than the deferred-live ``NotImplementedError``. + Mirrors v0.50.0 ``apply_variant_loss`` / v0.61.0 ``apply_unlearn_loss`` + policy. + """ + if not isinstance(config, GraceCodebookConfig): + raise TypeError( + f"apply_grace_codebook expects GraceCodebookConfig, " + f"got {type(config).__name__}" + ) + raise NotImplementedError( + "apply_grace_codebook is deferred to v0.62.1. Schema accepts " + f"size={config.size} dim={config.dim} today; live codebook " + "lookup / write ships next release." + ) diff --git a/soup_cli/utils/knowledge_edit.py b/soup_cli/utils/knowledge_edit.py index 7e14cdb..9469dc9 100644 --- a/soup_cli/utils/knowledge_edit.py +++ b/soup_cli/utils/knowledge_edit.py @@ -23,7 +23,7 @@ from types import MappingProxyType from typing import Mapping, Optional SUPPORTED_EDIT_METHODS: frozenset[str] = frozenset( - {"rome", "memit", "alphaedit"} + {"rome", "memit", "alphaedit", "grace"} ) _MAX_METHOD_LEN: int = 32 @@ -71,6 +71,17 @@ _EDIT_METHOD_METADATA: Mapping[str, EditMethodSpec] = MappingProxyType({ multi_edit_capable=True, live_wired=False, ), + "grace": EditMethodSpec( + name="grace", + description=( + "GRACE codebook — discrete latent-space (key, value) store " + "that survives thousands of sequential edits without " + "norm-blowup. v0.62.0 Part E ships the schema; live " + "lookup / write kernel lands in v0.62.1." + ), + multi_edit_capable=True, + live_wired=False, + ), }) @@ -197,6 +208,10 @@ _DEFAULT_EDIT_LAYER: Mapping[str, int] = MappingProxyType({ "rome": 5, "memit": 8, "alphaedit": 5, + # GRACE writes to a single dedicated codebook so the "layer" arg is + # the residual-stream layer where the lookup hook is installed. + # v0.62.0 Part E ships the schema; default mirrors AlphaEdit. + "grace": 5, }) @@ -234,7 +249,7 @@ def build_edit_plan( def apply_edit(plan: EditPlan) -> None: - """Apply a knowledge edit — deferred to v0.61.1. + """Apply a knowledge edit — deferred to v0.61.1 (or v0.62.1 for ``grace``). Re-validates the method so callers passing a bare-class duck-typed plan (no ``EditPlan``) still hit a meaningful error before the @@ -242,11 +257,13 @@ def apply_edit(plan: EditPlan) -> None: """ method_attr = getattr(plan, "method", None) canonical = validate_edit_method(method_attr) + # GRACE was added in v0.62.0 Part E with its own live-wiring schedule. + target_version = "v0.62.1" if canonical == "grace" else "v0.61.1" raise NotImplementedError( - f"apply_edit(method={canonical!r}) is deferred to v0.61.1. " + f"apply_edit(method={canonical!r}) is deferred to {target_version}. " "Schema accepts the request now so YAML / CLI invocations are " - "stable, but ROME / MEMIT / AlphaEdit live kernels land in " - "v0.61.1." + "stable, but ROME / MEMIT / AlphaEdit / GRACE live kernels land " + "next release." ) diff --git a/soup_cli/utils/ra_dit.py b/soup_cli/utils/ra_dit.py new file mode 100644 index 0000000..6782621 --- /dev/null +++ b/soup_cli/utils/ra_dit.py @@ -0,0 +1,159 @@ +"""v0.62.0 Part B — RA-DIT (Retrieval-Augmented Dual Instruction Tuning). + +Meta 2023 recipe. Two-stage: + +* ``retriever`` — train a sentence-transformer (contrastive triplet loss) + on (query, golden_doc, distractor) triples. Composes with the existing + v0.16 embedding trainer. +* ``generator`` — RAFT-style SFT on the query + golden_doc + distractor + bundle (uses v0.62.0 Part A ``data.format='raft'``). + +Schema-only release. Both stages share the existing trainer wrappers; +v0.62.0 ships the ``ra_dit_stage`` schema field + cross-validator so a +``soup.yaml`` can lock both stages in a hub-shareable recipe. Live +orchestration that chains the two stages in a single ``soup train`` call +is deferred to v0.62.1 (mirrors the v0.50.0 / v0.52.0 / v0.61.0 +stub-then-live pattern). +""" + +from __future__ import annotations + +from dataclasses import dataclass +from types import MappingProxyType +from typing import Mapping, Optional + +SUPPORTED_RA_DIT_STAGES: frozenset[str] = frozenset({"retriever", "generator"}) + +_MAX_STAGE_LEN: int = 32 +_MAX_RETRIEVER_MODEL_LEN: int = 512 + + +@dataclass(frozen=True) +class RaDitStageSpec: + """Metadata for a single RA-DIT training stage. Frozen post-construction.""" + + name: str + description: str + base_task: str + live_wired: bool + + +_RA_DIT_STAGE_METADATA: Mapping[str, RaDitStageSpec] = MappingProxyType({ + "retriever": RaDitStageSpec( + name="retriever", + description=( + "Stage 1 — train a sentence-transformer with contrastive loss " + "on (anchor, positive, negative) triples. Composes with the " + "v0.16 embedding trainer." + ), + base_task="embedding", + live_wired=False, + ), + "generator": RaDitStageSpec( + name="generator", + description=( + "Stage 2 — SFT the generator on RAFT-style rows " + "{query, golden_doc, distractor_docs, answer}. Uses " + "v0.62.0 Part A `data.format='raft'`." + ), + base_task="sft", + live_wired=False, + ), +}) + + +def validate_ra_dit_stage(value: object) -> str: + """Normalise + validate an RA-DIT stage name. + + Returns the canonical (lowercase) form. Mirrors v0.51.0 + ``validate_hub_name`` / v0.61.0 ``validate_unlearn_method`` policy: + bool-rejected, null-byte-rejected, oversize-rejected, case-insensitive + normalisation, unknown rejected with friendly actionable message. + """ + if isinstance(value, bool): + raise TypeError( + f"ra_dit_stage must not be bool, got {value!r}" + ) + if not isinstance(value, str): + raise TypeError( + f"ra_dit_stage must be str, got {type(value).__name__}" + ) + if not value: + raise ValueError("ra_dit_stage must be non-empty") + if "\x00" in value: + raise ValueError("ra_dit_stage must not contain null bytes") + if len(value) > _MAX_STAGE_LEN: + raise ValueError( + f"ra_dit_stage must be <= {_MAX_STAGE_LEN} chars" + ) + canonical = value.lower() + if canonical not in SUPPORTED_RA_DIT_STAGES: + supported = ", ".join(sorted(SUPPORTED_RA_DIT_STAGES)) + raise ValueError( + f"unknown ra_dit_stage {value!r}; supported: {supported}" + ) + return canonical + + +def get_ra_dit_stage_spec(name: str) -> RaDitStageSpec: + """Return the frozen :class:`RaDitStageSpec` for ``name`` or raise.""" + canonical = validate_ra_dit_stage(name) + return _RA_DIT_STAGE_METADATA[canonical] + + +def validate_ra_dit_retriever_model(value: object) -> Optional[str]: + """Validate the operator-supplied retriever-model HF repo id or local path. + + Bool / non-string rejected. ``None`` passes through unchanged so + callers can rely on Pydantic ``Optional[str]`` semantics. Length cap + matches the v0.40.5 ``reward_model`` policy (512 chars). + """ + if value is None: + return None + if isinstance(value, bool): + raise TypeError( + f"ra_dit_retriever_model must not be bool, got {value!r}" + ) + if not isinstance(value, str): + raise TypeError( + f"ra_dit_retriever_model must be str, got {type(value).__name__}" + ) + if not value: + raise ValueError("ra_dit_retriever_model must be non-empty") + if "\x00" in value: + raise ValueError( + "ra_dit_retriever_model must not contain null bytes" + ) + if len(value) > _MAX_RETRIEVER_MODEL_LEN: + raise ValueError( + f"ra_dit_retriever_model must be <= {_MAX_RETRIEVER_MODEL_LEN} chars" + ) + return value + + +def validate_ra_dit_compat(*, stage: str, task: str) -> None: + """Schema-time gate: each stage must pair with the right base task. + + * ``retriever`` -> ``task='embedding'`` (contrastive trainer). + * ``generator`` -> ``task='sft'`` (RAFT-style SFT). + + Raises ``ValueError`` on mismatch with a friendly actionable message. + """ + for name, value in (("stage", stage), ("task", task)): + if isinstance(value, bool): + raise TypeError(f"{name} must not be bool, got {value!r}") + if not isinstance(value, str): + raise TypeError( + f"{name} must be str, got {type(value).__name__}" + ) + if not value: + raise ValueError(f"{name} must be non-empty") + if "\x00" in value: + raise ValueError(f"{name} must not contain null bytes") + canonical_stage = validate_ra_dit_stage(stage) + expected_task = _RA_DIT_STAGE_METADATA[canonical_stage].base_task + if task != expected_task: + raise ValueError( + f"training.ra_dit_stage={canonical_stage!r} requires " + f"task={expected_task!r}; got task={task!r}." + ) diff --git a/soup_cli/utils/steering.py b/soup_cli/utils/steering.py new file mode 100644 index 0000000..9275a3c --- /dev/null +++ b/soup_cli/utils/steering.py @@ -0,0 +1,234 @@ +"""v0.62.0 Part C — Activation steering (CAA / ITI / RepE). + +Three control-vector backends for inference-time intervention: + +* ``caa`` — Contrastive Activation Addition (Panickssery et al., 2023). + Add a contrastive vector to the residual stream. +* ``iti`` — Inference-Time Intervention (Li et al., 2023). Shift specific + attention heads. +* ``repe`` — Representation Engineering (Zou et al., 2023). PCA-based + direction in the residual stream. + +Schema-only release: validators + frozen dataclasses + CLI surface ship +in v0.62.0. The live forward-hook + decode-time intervention land in +v0.62.1, mirroring the v0.50.0 / v0.52.0 / v0.61.0 stub-then-live cadence. +""" + +from __future__ import annotations + +import math +import re +from dataclasses import dataclass +from types import MappingProxyType +from typing import Mapping, Optional + +SUPPORTED_STEERING_METHODS: frozenset[str] = frozenset({"caa", "iti", "repe"}) + +_MAX_METHOD_LEN: int = 32 +_MAX_NAME_LEN: int = 128 +_MAX_STRENGTH_ABS: float = 10.0 # |strength| <= 10 sanity cap. + +# Kebab-case + underscore + dots only. Path-separators / whitespace / +# shell-metacharacters all rejected so the name can be safely embedded in +# CLI args, filenames, and Rich markup. Mirrors v0.57.0 adapter-branch +# policy (alphanumeric + `._-`). +_NAME_RE: re.Pattern[str] = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._\-]{0,127}$") + + +@dataclass(frozen=True) +class SteeringMethodSpec: + """Metadata for a single steering backend. Frozen post-construction.""" + + name: str + description: str + needs_contrastive_pairs: bool + needs_attention_heads: bool + live_wired: bool + + +_STEERING_METHOD_METADATA: Mapping[str, SteeringMethodSpec] = MappingProxyType({ + "caa": SteeringMethodSpec( + name="caa", + description=( + "Contrastive Activation Addition — add a contrastive vector " + "to the residual stream during decoding. Trains on " + "(positive, negative) prompt pairs (Panickssery et al., 2023)." + ), + needs_contrastive_pairs=True, + needs_attention_heads=False, + live_wired=False, + ), + "iti": SteeringMethodSpec( + name="iti", + description=( + "Inference-Time Intervention — shift specific attention heads " + "along a learned direction (Li et al., 2023). Needs per-head " + "calibration." + ), + needs_contrastive_pairs=True, + needs_attention_heads=True, + live_wired=False, + ), + "repe": SteeringMethodSpec( + name="repe", + description=( + "Representation Engineering — PCA over hidden states to " + "extract a behavioural direction (Zou et al., 2023)." + ), + needs_contrastive_pairs=True, + needs_attention_heads=False, + live_wired=False, + ), +}) + + +def validate_steering_method(value: object) -> str: + """Normalise + validate a steering-method name. + + Mirrors v0.41.0 / v0.51.0 / v0.61.0 validator policy: bool-rejected, + null-byte-rejected, oversize-rejected, case-insensitive normalisation, + unknown rejected with friendly actionable message. + """ + if isinstance(value, bool): + raise TypeError( + f"steering method must not be bool, got {value!r}" + ) + if not isinstance(value, str): + raise TypeError( + f"steering method must be str, got {type(value).__name__}" + ) + if not value: + raise ValueError("steering method must be non-empty") + if "\x00" in value: + raise ValueError("steering method must not contain null bytes") + if len(value) > _MAX_METHOD_LEN: + raise ValueError( + f"steering method must be <= {_MAX_METHOD_LEN} chars" + ) + canonical = value.lower() + if canonical not in SUPPORTED_STEERING_METHODS: + supported = ", ".join(sorted(SUPPORTED_STEERING_METHODS)) + raise ValueError( + f"unknown steering method {value!r}; supported: {supported}" + ) + return canonical + + +def validate_steering_name(value: object) -> str: + """Validate an operator-supplied steering-vector name. + + Returns the value unchanged on success. Closed regex allowlist + (alphanumeric + ``._-``, leading-alnum, ≤128 chars) so the name can + be safely used as a Registry artifact id, CLI flag, and filename + fragment on every platform. + """ + if isinstance(value, bool): + raise TypeError( + f"steering name must not be bool, got {value!r}" + ) + if not isinstance(value, str): + raise TypeError( + f"steering name must be str, got {type(value).__name__}" + ) + if not value: + raise ValueError("steering name must be non-empty") + if "\x00" in value: + raise ValueError("steering name must not contain null bytes") + if len(value) > _MAX_NAME_LEN: + raise ValueError( + f"steering name must be <= {_MAX_NAME_LEN} chars" + ) + if not _NAME_RE.match(value): + raise ValueError( + f"steering name {value!r} must match {_NAME_RE.pattern!r} " + "(alphanumeric + `._-`, leading alnum)." + ) + return value + + +def validate_steering_strength(value: object) -> float: + """Validate a steering strength multiplier. + + Bool-rejected (bool is a subclass of int), NaN/Inf-rejected via + ``math.isfinite``, bounded ``|strength| <= 10.0`` as a sanity cap. + """ + if isinstance(value, bool): + raise TypeError( + f"steering strength must not be bool, got {value!r}" + ) + if not isinstance(value, (int, float)): + raise TypeError( + f"steering strength must be a number, got {type(value).__name__}" + ) + fval = float(value) + if not math.isfinite(fval): + raise ValueError("steering strength must be finite (no NaN / Inf)") + if abs(fval) > _MAX_STRENGTH_ABS: + raise ValueError( + f"steering strength must satisfy |s| <= {_MAX_STRENGTH_ABS}; " + f"got {fval}" + ) + return fval + + +def get_steering_method_spec(name: str) -> SteeringMethodSpec: + """Return the frozen :class:`SteeringMethodSpec` for ``name`` or raise.""" + canonical = validate_steering_method(name) + return _STEERING_METHOD_METADATA[canonical] + + +def apply_steering(method: str) -> None: + """Apply a steering vector during decoding — deferred to v0.62.1. + + Validates the method name first so the deferred-live error + distinguishes between "unknown method" and "method is on the + allowlist but not yet wired". Mirrors v0.50.0 ``apply_variant_loss`` / + v0.61.0 ``apply_unlearn_loss`` policy. + """ + canonical = validate_steering_method(method) + raise NotImplementedError( + f"apply_steering({canonical!r}) is deferred to v0.62.1. " + "Schema accepts the method now so callers can write soup.yaml " + "today, but the live forward-hook + decode-time intervention " + "land in v0.62.1." + ) + + +def build_steering_vector( + *, + method: str, + name: str, + pairs_path: Optional[str] = None, + layer: Optional[int] = None, +) -> None: + """Train a steering vector from contrastive pairs — deferred to v0.62.1. + + Validates inputs first; the deferred-live error fires only after the + method + name + (optional) layer all pass shape/range checks. + """ + canonical = validate_steering_method(method) + canonical_name = validate_steering_name(name) + if pairs_path is not None: + if not isinstance(pairs_path, str): + raise TypeError( + f"pairs_path must be str, got {type(pairs_path).__name__}" + ) + if not pairs_path: + raise ValueError("pairs_path must be non-empty") + if "\x00" in pairs_path: + raise ValueError("pairs_path must not contain null bytes") + if layer is not None: + if isinstance(layer, bool): + raise TypeError("layer must not be bool") + if not isinstance(layer, int): + raise TypeError( + f"layer must be int, got {type(layer).__name__}" + ) + if layer < 0 or layer > 2048: + raise ValueError( + f"layer must satisfy 0 <= layer <= 2048, got {layer}" + ) + raise NotImplementedError( + f"build_steering_vector(method={canonical!r}, " + f"name={canonical_name!r}) is deferred to v0.62.1." + ) diff --git a/tests/test_recipes.py b/tests/test_recipes.py index 06985d4..0aecb48 100644 --- a/tests/test_recipes.py +++ b/tests/test_recipes.py @@ -260,7 +260,7 @@ class TestV025NewRecipes: assert cfg.base == recipe.model assert cfg.task == recipe.task - def test_catalog_size_is_113(self): + def test_catalog_size_is_116(self): """Total catalog size — grew with each release. v0.25.0 shipped 43 recipes (29 + 9 Part A + 2 Part B tools + 3 Part E MLX). @@ -269,10 +269,11 @@ class TestV025NewRecipes: v0.51.0 added 26 (model catalog expansion) -> 106. v0.52.0 added 6 (5 TTS + Falcon-E BitNet) -> 112. v0.53.5 added 1 (deepseek-v3-reasoning) -> 113. + v0.62.0 added 3 (raft-llama3-8b, ra-dit-retriever, ra-dit-llama3-8b) -> 116. """ from soup_cli.recipes.catalog import RECIPES - assert len(RECIPES) == 113 + assert len(RECIPES) == 116 def test_new_recipes_searchable(self): """Search returns the new recipes via keyword/task filter.""" diff --git a/tests/test_v0610_part_c.py b/tests/test_v0610_part_c.py index 246c501..d8ff909 100644 --- a/tests/test_v0610_part_c.py +++ b/tests/test_v0610_part_c.py @@ -41,7 +41,10 @@ class TestModuleSurface: def test_supported_methods_exact(self): from soup_cli.utils.knowledge_edit import SUPPORTED_EDIT_METHODS - assert SUPPORTED_EDIT_METHODS == frozenset({"rome", "memit", "alphaedit"}) + # v0.62.0 Part E added "grace" to the allowlist (codebook edit). + assert SUPPORTED_EDIT_METHODS == frozenset( + {"rome", "memit", "alphaedit", "grace"} + ) class TestValidateEditMethod: diff --git a/tests/test_v0620_followups.py b/tests/test_v0620_followups.py new file mode 100644 index 0000000..a5f0768 --- /dev/null +++ b/tests/test_v0620_followups.py @@ -0,0 +1,589 @@ +"""v0.62.0 review-fix follow-up tests. + +Closes coverage gaps surfaced by the TDD review: + +* H1 — citation_faithful + non-SFT-family task gate. +* H2 — RAFT _MAX_RAFT_FIELD_LEN per-field oversize boundary. +* H3 — RAFT null-byte rejection on golden_doc / answer / distractors. +* H5 — `soup steer train --plan-only` deferred-marker stripping regression guard. +* H6 — `apply_edit` still raises with v0.61.1 marker for rome/memit/alphaedit. +* M1 — steering name regex 128-char boundary acceptance. +* M3/M4 — soup steer train `--base` over-cap + null-byte rejection. +* M5 — extract_citation_ids public API coverage. +* M8 — GRACE codebook flag-without-both-knobs cross-validator. +* M9 — validate_ra_dit_compat direct-caller null-byte / bool defence-in-depth. +* L1 — list_steers context-manager source-grep regression guard. +* L4 — GRACE codebook size/dim max-boundary acceptance. +* L5 — RA-DIT retriever model 512-char boundary. +* L7 — CitationScore predicted_count / expected_count field values. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +# ---------- H1 — citation_faithful task-gate ---------- + + +class TestCitationFaithfulTaskGate: + def test_citation_faithful_on_grpo_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: grpo + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_faithful: true + reward_fn: accuracy + +output: ./output +""" + with pytest.raises(Exception, match="sft|pretrain"): + load_config_from_string(yaml_text) + + def test_citation_faithful_on_pretrain_accepted(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: pretrain + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_faithful: true + +output: ./output +""" + cfg = load_config_from_string(yaml_text) + assert cfg.training.citation_faithful is True + + def test_citation_faithful_on_dpo_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: dpo + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_faithful: true + +output: ./output +""" + with pytest.raises(Exception, match="sft|pretrain"): + load_config_from_string(yaml_text) + + +# ---------- H2 — RAFT per-field oversize boundary ---------- + + +class TestRaftFieldOversizeBoundary: + def test_query_at_cap_accepted(self): + from soup_cli.data.formats import _MAX_RAFT_FIELD_LEN, _convert_raft + + out = _convert_raft({ + "query": "x" * _MAX_RAFT_FIELD_LEN, + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + assert len(out["query"]) == _MAX_RAFT_FIELD_LEN + + def test_query_overcap_rejected(self): + from soup_cli.data.formats import _MAX_RAFT_FIELD_LEN, _convert_raft + + with pytest.raises(ValueError, match="query"): + _convert_raft({ + "query": "x" * (_MAX_RAFT_FIELD_LEN + 1), + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + + def test_golden_doc_overcap_rejected(self): + from soup_cli.data.formats import _MAX_RAFT_FIELD_LEN, _convert_raft + + with pytest.raises(ValueError, match="golden_doc"): + _convert_raft({ + "query": "q", + "golden_doc": "x" * (_MAX_RAFT_FIELD_LEN + 1), + "distractor_docs": [], + "answer": "a", + }) + + def test_answer_overcap_rejected(self): + from soup_cli.data.formats import _MAX_RAFT_FIELD_LEN, _convert_raft + + with pytest.raises(ValueError, match="answer"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": [], + "answer": "x" * (_MAX_RAFT_FIELD_LEN + 1), + }) + + def test_distractor_overcap_rejected(self): + from soup_cli.data.formats import _MAX_RAFT_FIELD_LEN, _convert_raft + + with pytest.raises(ValueError, match="distractor"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": ["x" * (_MAX_RAFT_FIELD_LEN + 1)], + "answer": "a", + }) + + +# ---------- H3 — RAFT null-byte rejection on every field ---------- + + +class TestRaftNullByteRejection: + def test_null_byte_golden_doc_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="null"): + _convert_raft({ + "query": "q", + "golden_doc": "bad\x00", + "distractor_docs": [], + "answer": "a", + }) + + def test_null_byte_answer_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="null"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": [], + "answer": "bad\x00", + }) + + def test_null_byte_distractor_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="null"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": ["bad\x00doc"], + "answer": "a", + }) + + +# ---------- H5 — `soup steer train --plan-only` deferred-marker assertion ---------- + + +class TestSteerPlanOnlyMarker: + def test_plan_only_emits_v0621_marker(self, tmp_path, monkeypatch): + from soup_cli.commands.steer import app + + monkeypatch.chdir(tmp_path) + pairs = tmp_path / "pairs.jsonl" + pairs.write_text( + '{"positive": "x", "negative": "y"}\n', encoding="utf-8" + ) + runner = CliRunner() + result = runner.invoke(app, [ + "train", + "--base", "meta-llama/Llama-3.1-8B-Instruct", + "--method", "caa", + "--name", "safety-v1", + "--pairs", "pairs.jsonl", + "--plan-only", + ]) + assert result.exit_code == 0, ( + result.output, repr(result.exception) + ) + # Defence: the panel MUST surface the deferred-version marker so + # operators know live training ships in v0.62.1. + assert "v0.62.1" in result.output + + +# ---------- H6 — apply_edit v0.61.1 marker not regressed for legacy methods ---------- + + +class TestEditMarkerRegressionGuard: + @pytest.mark.parametrize("method", ["rome", "memit", "alphaedit"]) + def test_legacy_methods_still_use_v0611_marker(self, method: str): + from soup_cli.utils.knowledge_edit import apply_edit, build_edit_plan + + plan = build_edit_plan( + base="meta-llama/Llama-3.1-8B-Instruct", + method=method, + subject="The capital of France is", + target="Lyon", + ) + with pytest.raises(NotImplementedError, match="v0.61.1"): + apply_edit(plan) + + def test_grace_uses_v0621_marker_distinct_from_v0611(self): + from soup_cli.utils.knowledge_edit import apply_edit, build_edit_plan + + plan = build_edit_plan( + base="meta-llama/Llama-3.1-8B-Instruct", + method="grace", + subject="The capital of France is", + target="Lyon", + ) + try: + apply_edit(plan) + except NotImplementedError as exc: + msg = str(exc) + assert "v0.62.1" in msg + assert "v0.61.1" not in msg + else: # pragma: no cover - apply_edit must raise + pytest.fail("apply_edit should have raised NotImplementedError") + + +# ---------- M1 — steering name 128-char boundary ---------- + + +class TestSteeringNameBoundary: + def test_at_max_length_accepted(self): + from soup_cli.utils.steering import validate_steering_name + + name = "a" + "b" * 127 # exactly 128 chars + assert validate_steering_name(name) == name + + def test_one_over_max_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + with pytest.raises(ValueError): + validate_steering_name("a" + "b" * 128) # 129 chars + + +# ---------- M3/M4 — `--base` over-cap + null-byte rejection ---------- + + +class TestSteerTrainBaseValidation: + def _make_runner(self, tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + (tmp_path / "pairs.jsonl").write_text( + '{"positive": "x", "negative": "y"}\n', encoding="utf-8" + ) + return CliRunner() + + def test_base_overcap_rejected(self, tmp_path, monkeypatch): + from soup_cli.commands.steer import app + + runner = self._make_runner(tmp_path, monkeypatch) + result = runner.invoke(app, [ + "train", + "--base", "x" * 513, + "--method", "caa", + "--name", "safety-v1", + "--pairs", "pairs.jsonl", + "--plan-only", + ]) + assert result.exit_code == 2 + assert "base" in result.output.lower() + + def test_base_null_byte_rejected(self, tmp_path, monkeypatch): + from soup_cli.commands.steer import app + + runner = self._make_runner(tmp_path, monkeypatch) + result = runner.invoke(app, [ + "train", + "--base", "bad\x00name", + "--method", "caa", + "--name", "safety-v1", + "--pairs", "pairs.jsonl", + "--plan-only", + ]) + assert result.exit_code == 2 + assert "null" in result.output.lower() or "base" in result.output.lower() + + def test_base_at_max_length_accepted(self, tmp_path, monkeypatch): + from soup_cli.commands.steer import app + + runner = self._make_runner(tmp_path, monkeypatch) + result = runner.invoke(app, [ + "train", + "--base", "x" * 512, + "--method", "caa", + "--name", "safety-v1", + "--pairs", "pairs.jsonl", + "--plan-only", + ]) + assert result.exit_code == 0, ( + result.output, repr(result.exception) + ) + + +# ---------- M5 — extract_citation_ids public-API coverage ---------- + + +class TestExtractCitationIds: + def test_extracts_bracketed(self): + from soup_cli.utils.citation_faithful import extract_citation_ids + + assert extract_citation_ids("See [doc-1] and [doc-2].") == ( + "doc-1", "doc-2", + ) + + def test_preserves_duplicates(self): + from soup_cli.utils.citation_faithful import extract_citation_ids + + assert extract_citation_ids("[doc-1] [doc-1]") == ("doc-1", "doc-1") + + def test_empty_returns_empty_tuple(self): + from soup_cli.utils.citation_faithful import extract_citation_ids + + assert extract_citation_ids("plain text without citations") == () + + def test_non_string_raises(self): + from soup_cli.utils.citation_faithful import extract_citation_ids + + with pytest.raises(TypeError): + extract_citation_ids(42) + + def test_oversize_raises(self): + from soup_cli.utils.citation_faithful import extract_citation_ids + + with pytest.raises(ValueError): + extract_citation_ids("x" * 2_000_001) + + def test_regex_requires_leading_alnum(self): + from soup_cli.utils.citation_faithful import extract_citation_ids + + # `_CITATION_RE` requires `[A-Za-z0-9]` as the first char inside the + # brackets; a leading underscore/dash should NOT match. + assert extract_citation_ids("[_bad-id]") == () + assert extract_citation_ids("[-bad]") == () + + +# ---------- M5 supplement — score_citations expected_ids per-entry validation ---------- + + +class TestScoreCitationsExpectedIds: + def test_non_string_in_expected_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(TypeError): + score_citations(predicted="[doc-1]", expected_ids=(42,)) + + def test_bool_in_expected_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(TypeError): + score_citations(predicted="[doc-1]", expected_ids=(True,)) + + def test_empty_string_in_expected_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(ValueError, match="non-empty"): + score_citations(predicted="[doc-1]", expected_ids=("",)) + + def test_null_byte_in_expected_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(ValueError, match="null"): + score_citations(predicted="[doc-1]", expected_ids=("bad\x00id",)) + + +# ---------- L7 — CitationScore field values ---------- + + +class TestCitationScoreFields: + def test_predicted_count_and_expected_count_set(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations( + predicted="[doc-1] [doc-2]", + expected_ids=("doc-1",), + ) + assert score.predicted_count == 2 + assert score.expected_count == 1 + + def test_predicted_count_includes_duplicates(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations( + predicted="[doc-1] [doc-1] [doc-1]", + expected_ids=("doc-1",), + ) + assert score.predicted_count == 3 + + +# ---------- M8 — GRACE codebook flag without both knobs ---------- + + +class TestGraceCodebookPartialConfig: + def test_flag_alone_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + grace_codebook: true + +output: ./output +""" + with pytest.raises(Exception, match="grace_codebook"): + load_config_from_string(yaml_text) + + def test_flag_with_dim_missing_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + grace_codebook: true + grace_codebook_size: 128 + +output: ./output +""" + with pytest.raises(Exception, match="grace_codebook"): + load_config_from_string(yaml_text) + + def test_grace_in_default_edit_layer(self): + from soup_cli.utils.knowledge_edit import _DEFAULT_EDIT_LAYER # type: ignore + + assert "grace" in _DEFAULT_EDIT_LAYER + assert isinstance(_DEFAULT_EDIT_LAYER["grace"], int) + assert _DEFAULT_EDIT_LAYER["grace"] >= 0 + + def test_grace_in_edit_metadata_mapping(self): + from soup_cli.utils.knowledge_edit import _EDIT_METHOD_METADATA # type: ignore + + assert "grace" in _EDIT_METHOD_METADATA + + +# ---------- M9 — validate_ra_dit_compat direct-caller defence-in-depth ---------- + + +class TestValidateRaDitCompatDirect: + def test_null_byte_task_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_compat + + with pytest.raises(ValueError, match="null"): + validate_ra_dit_compat(stage="retriever", task="sft\x00") + + def test_null_byte_stage_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_compat + + with pytest.raises(ValueError, match="null"): + validate_ra_dit_compat(stage="retriever\x00", task="embedding") + + def test_bool_task_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_compat + + with pytest.raises(TypeError): + validate_ra_dit_compat(stage="retriever", task=True) + + def test_bool_stage_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_compat + + with pytest.raises(TypeError): + validate_ra_dit_compat(stage=True, task="embedding") + + def test_empty_task_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_compat + + with pytest.raises(ValueError, match="non-empty"): + validate_ra_dit_compat(stage="retriever", task="") + + +# ---------- L4 — GRACE size/dim max-boundary acceptance ---------- + + +class TestGraceMaxBoundary: + def test_max_codebook_size_accepted(self): + from soup_cli.utils.grace_codebook import ( + MAX_CODEBOOK_SIZE, + validate_grace_codebook_size, + ) + + assert validate_grace_codebook_size(MAX_CODEBOOK_SIZE) == MAX_CODEBOOK_SIZE + + def test_max_codebook_dim_accepted(self): + from soup_cli.utils.grace_codebook import ( + MAX_CODEBOOK_DIM, + validate_grace_codebook_dim, + ) + + assert validate_grace_codebook_dim(MAX_CODEBOOK_DIM) == MAX_CODEBOOK_DIM + + +# ---------- L5 — RA-DIT retriever model 512-char boundary ---------- + + +class TestRaDitRetrieverBoundary: + def test_at_max_accepted(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + v = "x" * 512 + assert validate_ra_dit_retriever_model(v) == v + + def test_one_over_max_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + with pytest.raises(ValueError): + validate_ra_dit_retriever_model("x" * 513) + + +# ---------- L1 — list_steers context-manager source-grep regression guard ---------- + + +class TestSourceWiring: + def test_list_steers_uses_context_manager(self): + src = Path(__file__).resolve().parent.parent / "soup_cli" / "commands" / "steer.py" + text = src.read_text(encoding="utf-8") + assert "with RegistryStore() as store:" in text + + def test_steer_uses_shared_path_helper(self): + """Review M1 fix: `_validate_pairs_path` delegates to the shared helper.""" + src = Path(__file__).resolve().parent.parent / "soup_cli" / "commands" / "steer.py" + text = src.read_text(encoding="utf-8") + assert "enforce_under_cwd_and_no_symlink" in text + + def test_version_string_is_v0620(self): + import soup_cli + + assert soup_cli.__version__ == "0.62.0" + + def test_pyproject_version_is_v0620(self): + proj = Path(__file__).resolve().parent.parent / "pyproject.toml" + text = proj.read_text(encoding="utf-8") + assert 'version = "0.62.0"' in text diff --git a/tests/test_v0620_part_a.py b/tests/test_v0620_part_a.py new file mode 100644 index 0000000..2e23445 --- /dev/null +++ b/tests/test_v0620_part_a.py @@ -0,0 +1,270 @@ +"""Tests for v0.62.0 Part A — RAFT (Retrieval-Augmented Fine-Tuning) recipe. + +Adds ``data.format='raft'`` to the schema + new ``_convert_raft`` validator ++ a ``raft-llama3-8b`` recipe entry. Schema validates RAFT row shape: +``{"query", "golden_doc", "distractor_docs", "answer"}``. +""" + +from __future__ import annotations + +import pytest +import yaml + +# ---------- _convert_raft ---------- + + +class TestConvertRaft: + def test_happy_path(self): + from soup_cli.data.formats import _convert_raft + + row = { + "query": "What year was Python released?", + "golden_doc": "Python was released in 1991 by Guido van Rossum.", + "distractor_docs": [ + "Ruby was released in 1995.", + "Java was released in 1995.", + ], + "answer": "1991", + } + out = _convert_raft(row) + assert out["query"] == row["query"] + assert out["golden_doc"] == row["golden_doc"] + assert out["distractor_docs"] == row["distractor_docs"] + assert out["answer"] == row["answer"] + + def test_empty_distractors_ok(self): + from soup_cli.data.formats import _convert_raft + + out = _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + assert out["distractor_docs"] == [] + + def test_missing_query_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises((KeyError, ValueError)): + _convert_raft({ + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + + def test_missing_golden_doc_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises((KeyError, ValueError)): + _convert_raft({ + "query": "q", + "distractor_docs": [], + "answer": "a", + }) + + def test_missing_answer_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises((KeyError, ValueError)): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": [], + }) + + def test_empty_query_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="query"): + _convert_raft({ + "query": "", + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + + def test_empty_golden_doc_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="golden_doc"): + _convert_raft({ + "query": "q", + "golden_doc": "", + "distractor_docs": [], + "answer": "a", + }) + + def test_empty_answer_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="answer"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": [], + "answer": "", + }) + + def test_non_string_query_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="query"): + _convert_raft({ + "query": 123, + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + + def test_non_list_distractors_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="distractor_docs"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": "not-a-list", + "answer": "a", + }) + + def test_non_string_distractor_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="distractor"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": [123], + "answer": "a", + }) + + def test_null_byte_query_rejected(self): + from soup_cli.data.formats import _convert_raft + + with pytest.raises(ValueError, match="null"): + _convert_raft({ + "query": "bad\x00", + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + }) + + def test_distractor_cap(self): + from soup_cli.data.formats import _MAX_RAFT_DISTRACTORS, _convert_raft + + # exactly cap accepted + out = _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": ["d"] * _MAX_RAFT_DISTRACTORS, + "answer": "a", + }) + assert len(out["distractor_docs"]) == _MAX_RAFT_DISTRACTORS + + def test_distractor_overcap_rejected(self): + from soup_cli.data.formats import _MAX_RAFT_DISTRACTORS, _convert_raft + + with pytest.raises(ValueError, match="distractor"): + _convert_raft({ + "query": "q", + "golden_doc": "g", + "distractor_docs": ["d"] * (_MAX_RAFT_DISTRACTORS + 1), + "answer": "a", + }) + + +# ---------- format dispatcher ---------- + + +class TestFormatDispatcher: + def test_format_to_messages_dispatches_raft(self): + from soup_cli.data.formats import format_to_messages + + row = { + "query": "q", + "golden_doc": "g", + "distractor_docs": [], + "answer": "a", + } + out = format_to_messages(row, "raft") + assert out is not None + assert out["query"] == "q" + assert out["answer"] == "a" + + def test_format_to_messages_invalid_raft_returns_none(self): + from soup_cli.data.formats import format_to_messages + + # missing key — wrapper catches and returns None + assert format_to_messages({"query": "q"}, "raft") is None + + +# ---------- schema integration ---------- + + +class TestSchemaIntegration: + def test_data_format_raft_accepted(self): + from soup_cli.config.schema import DataConfig + + cfg = DataConfig(train="data.jsonl", format="raft") + assert cfg.format == "raft" + + def test_soup_config_raft_roundtrip(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + max_length: 4096 + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + +output: ./output +""" + cfg = load_config_from_string(yaml_text) + assert cfg.data.format == "raft" + assert cfg.task == "sft" + + +# ---------- recipe catalog ---------- + + +class TestRaftRecipe: + def test_recipe_present(self): + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("raft-llama3-8b") + assert recipe is not None + assert recipe.task == "sft" + assert "raft" in recipe.tags or "rag" in recipe.tags + + def test_recipe_yaml_loads(self): + from soup_cli.config.loader import load_config_from_string + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("raft-llama3-8b") + assert recipe is not None + cfg = load_config_from_string(recipe.yaml_str) + assert cfg.data.format == "raft" + + def test_recipe_yaml_parses_as_dict(self): + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("raft-llama3-8b") + assert recipe is not None + parsed = yaml.safe_load(recipe.yaml_str) + assert isinstance(parsed, dict) + assert parsed["data"]["format"] == "raft" + + def test_recipe_search_finds_raft(self): + from soup_cli.recipes.catalog import search_recipes + + results = search_recipes(query="raft") + assert any(r.task == "sft" and "raft" in r.yaml_str for r in results) diff --git a/tests/test_v0620_part_b.py b/tests/test_v0620_part_b.py new file mode 100644 index 0000000..61c6bc2 --- /dev/null +++ b/tests/test_v0620_part_b.py @@ -0,0 +1,384 @@ +"""Tests for v0.62.0 Part B — RA-DIT (Retrieval-Augmented Dual Instruction Tuning). + +Two-stage Meta 2023 recipe: train the retriever first (contrastive), then +the generator (RAFT-style). v0.62.0 ships the schema + recipe entries; +live runtime composes existing v0.16 embedding trainer + Part A RAFT. +""" + +from __future__ import annotations + +import dataclasses + +import pytest +import yaml + +# ---------- Module surface ---------- + + +class TestModuleSurface: + def test_imports(self): + from soup_cli.utils.ra_dit import ( + SUPPORTED_RA_DIT_STAGES, + RaDitStageSpec, + get_ra_dit_stage_spec, + validate_ra_dit_stage, + ) + assert callable(validate_ra_dit_stage) + assert callable(get_ra_dit_stage_spec) + assert dataclasses.is_dataclass(RaDitStageSpec) + assert isinstance(SUPPORTED_RA_DIT_STAGES, frozenset) + + def test_stages_exact(self): + from soup_cli.utils.ra_dit import SUPPORTED_RA_DIT_STAGES + + assert SUPPORTED_RA_DIT_STAGES == frozenset({"retriever", "generator"}) + + def test_metadata_mapping_proxy(self): + from types import MappingProxyType + + from soup_cli.utils.ra_dit import _RA_DIT_STAGE_METADATA # type: ignore + + assert isinstance(_RA_DIT_STAGE_METADATA, MappingProxyType) + + +# ---------- validate_ra_dit_stage ---------- + + +class TestValidateStage: + def test_happy(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + assert validate_ra_dit_stage("retriever") == "retriever" + assert validate_ra_dit_stage("generator") == "generator" + + def test_case_insensitive(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + assert validate_ra_dit_stage("Retriever") == "retriever" + assert validate_ra_dit_stage("GENERATOR") == "generator" + + def test_bool_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + with pytest.raises(TypeError): + validate_ra_dit_stage(True) + + def test_non_string_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + with pytest.raises(TypeError): + validate_ra_dit_stage(1) + + def test_empty_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + with pytest.raises(ValueError): + validate_ra_dit_stage("") + + def test_null_byte_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + with pytest.raises(ValueError): + validate_ra_dit_stage("retriever\x00") + + def test_oversize_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + with pytest.raises(ValueError): + validate_ra_dit_stage("retriever" * 100) + + def test_unknown_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_stage + + with pytest.raises(ValueError, match="ra_dit_stage"): + validate_ra_dit_stage("decoder") + + +# ---------- get_ra_dit_stage_spec ---------- + + +class TestStageSpec: + def test_retriever_spec(self): + from soup_cli.utils.ra_dit import get_ra_dit_stage_spec + + spec = get_ra_dit_stage_spec("retriever") + assert spec.name == "retriever" + assert spec.base_task == "embedding" + assert spec.live_wired is False + + def test_generator_spec(self): + from soup_cli.utils.ra_dit import get_ra_dit_stage_spec + + spec = get_ra_dit_stage_spec("generator") + assert spec.name == "generator" + assert spec.base_task == "sft" + assert spec.live_wired is False + + def test_unknown_raises(self): + from soup_cli.utils.ra_dit import get_ra_dit_stage_spec + + with pytest.raises(ValueError): + get_ra_dit_stage_spec("nonsense") + + def test_spec_frozen(self): + from soup_cli.utils.ra_dit import get_ra_dit_stage_spec + + spec = get_ra_dit_stage_spec("retriever") + with pytest.raises(dataclasses.FrozenInstanceError): + spec.name = "mutated" # type: ignore[misc] + + +# ---------- validate_ra_dit_retriever_model ---------- + + +class TestRetrieverModel: + def test_happy(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + assert validate_ra_dit_retriever_model("sentence-transformers/all-mpnet-base-v2") == \ + "sentence-transformers/all-mpnet-base-v2" + + def test_none_passthrough(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + assert validate_ra_dit_retriever_model(None) is None + + def test_empty_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + with pytest.raises(ValueError): + validate_ra_dit_retriever_model("") + + def test_null_byte_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + with pytest.raises(ValueError): + validate_ra_dit_retriever_model("foo\x00bar") + + def test_bool_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + with pytest.raises(TypeError): + validate_ra_dit_retriever_model(True) + + def test_oversize_rejected(self): + from soup_cli.utils.ra_dit import validate_ra_dit_retriever_model + + with pytest.raises(ValueError): + validate_ra_dit_retriever_model("a" * 1024) + + +# ---------- Schema integration ---------- + + +class TestSchemaIntegration: + def test_default_none(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig() + assert cfg.ra_dit_stage is None + assert cfg.ra_dit_retriever_model is None + + def test_retriever_accepted(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig(ra_dit_stage="retriever") + assert cfg.ra_dit_stage == "retriever" + + def test_generator_accepted(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig(ra_dit_stage="generator") + assert cfg.ra_dit_stage == "generator" + + def test_case_insensitive_at_schema(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig(ra_dit_stage="Generator") + assert cfg.ra_dit_stage == "generator" + + def test_invalid_stage_rejected(self): + from pydantic import ValidationError + + from soup_cli.config.schema import TrainingConfig + + with pytest.raises(ValidationError): + TrainingConfig(ra_dit_stage="decoder") + + def test_retriever_model_accepted(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig( + ra_dit_stage="retriever", + ra_dit_retriever_model="sentence-transformers/all-mpnet-base-v2", + ) + assert cfg.ra_dit_retriever_model == "sentence-transformers/all-mpnet-base-v2" + + def test_retriever_model_null_byte_rejected(self): + from pydantic import ValidationError + + from soup_cli.config.schema import TrainingConfig + + with pytest.raises(ValidationError): + TrainingConfig(ra_dit_retriever_model="foo\x00") + + +# ---------- Cross-validator (SoupConfig) ---------- + + +class TestSoupConfigGate: + def test_retriever_stage_on_embedding_task(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: sentence-transformers/all-mpnet-base-v2 +task: embedding + +data: + train: ./data/anchor.jsonl + format: embedding + +training: + epochs: 1 + lr: 2e-5 + batch_size: auto + ra_dit_stage: retriever + +output: ./output +""" + cfg = load_config_from_string(yaml_text) + assert cfg.training.ra_dit_stage == "retriever" + + def test_generator_stage_on_sft_task(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + ra_dit_stage: generator + +output: ./output +""" + cfg = load_config_from_string(yaml_text) + assert cfg.training.ra_dit_stage == "generator" + + def test_retriever_on_sft_task_rejected(self): + from soup_cli.config.loader import load_config_from_string + + # retriever stage requires embedding-family task + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + ra_dit_stage: retriever + +output: ./output +""" + with pytest.raises(Exception, match="ra_dit_stage"): + load_config_from_string(yaml_text) + + def test_generator_on_grpo_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: grpo + +data: + train: ./data/prompts.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + ra_dit_stage: generator + reward_fn: accuracy + +output: ./output +""" + with pytest.raises(Exception, match="ra_dit_stage"): + load_config_from_string(yaml_text) + + def test_retriever_model_without_stage_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + ra_dit_retriever_model: sentence-transformers/all-mpnet-base-v2 + +output: ./output +""" + with pytest.raises(Exception, match="ra_dit"): + load_config_from_string(yaml_text) + + +# ---------- Recipes ---------- + + +class TestRaDitRecipes: + def test_retriever_recipe_present(self): + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("ra-dit-retriever") + assert recipe is not None + assert recipe.task == "embedding" + + def test_generator_recipe_present(self): + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("ra-dit-llama3-8b") + assert recipe is not None + assert recipe.task == "sft" + + def test_retriever_recipe_yaml_loads(self): + from soup_cli.config.loader import load_config_from_string + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("ra-dit-retriever") + assert recipe is not None + cfg = load_config_from_string(recipe.yaml_str) + assert cfg.training.ra_dit_stage == "retriever" + + def test_generator_recipe_yaml_loads(self): + from soup_cli.config.loader import load_config_from_string + from soup_cli.recipes.catalog import get_recipe + + recipe = get_recipe("ra-dit-llama3-8b") + assert recipe is not None + cfg = load_config_from_string(recipe.yaml_str) + assert cfg.training.ra_dit_stage == "generator" + assert cfg.data.format == "raft" + + def test_recipe_yaml_parses_as_dict(self): + from soup_cli.recipes.catalog import get_recipe + + for name in ("ra-dit-retriever", "ra-dit-llama3-8b"): + recipe = get_recipe(name) + assert recipe is not None + parsed = yaml.safe_load(recipe.yaml_str) + assert isinstance(parsed, dict) diff --git a/tests/test_v0620_part_c.py b/tests/test_v0620_part_c.py new file mode 100644 index 0000000..3c63212 --- /dev/null +++ b/tests/test_v0620_part_c.py @@ -0,0 +1,381 @@ +"""Tests for v0.62.0 Part C — `soup steer` (CAA / ITI / RepE control vectors). + +Schema-only release: validators + frozen dataclass + CLI surface ship now, +live forward-hook + decode-time intervention land in v0.62.1. +""" + +from __future__ import annotations + +import dataclasses + +import pytest + +# ---------- Module surface ---------- + + +class TestModuleSurface: + def test_imports(self): + from soup_cli.utils.steering import ( + SUPPORTED_STEERING_METHODS, + SteeringMethodSpec, + apply_steering, + build_steering_vector, + get_steering_method_spec, + validate_steering_method, + ) + assert callable(validate_steering_method) + assert callable(get_steering_method_spec) + assert callable(apply_steering) + assert callable(build_steering_vector) + assert dataclasses.is_dataclass(SteeringMethodSpec) + assert isinstance(SUPPORTED_STEERING_METHODS, frozenset) + + def test_methods_exact(self): + from soup_cli.utils.steering import SUPPORTED_STEERING_METHODS + + assert SUPPORTED_STEERING_METHODS == frozenset({"caa", "iti", "repe"}) + + def test_metadata_mapping_proxy(self): + from types import MappingProxyType + + from soup_cli.utils.steering import _STEERING_METHOD_METADATA # type: ignore + + assert isinstance(_STEERING_METHOD_METADATA, MappingProxyType) + + +# ---------- validate_steering_method ---------- + + +class TestValidateMethod: + def test_happy(self): + from soup_cli.utils.steering import validate_steering_method + + for name in ("caa", "iti", "repe"): + assert validate_steering_method(name) == name + + def test_case_insensitive(self): + from soup_cli.utils.steering import validate_steering_method + + assert validate_steering_method("CAA") == "caa" + assert validate_steering_method("ItI") == "iti" + + def test_bool_rejected(self): + from soup_cli.utils.steering import validate_steering_method + + with pytest.raises(TypeError): + validate_steering_method(True) + + def test_non_string_rejected(self): + from soup_cli.utils.steering import validate_steering_method + + with pytest.raises(TypeError): + validate_steering_method(42) + + def test_empty_rejected(self): + from soup_cli.utils.steering import validate_steering_method + + with pytest.raises(ValueError): + validate_steering_method("") + + def test_null_byte_rejected(self): + from soup_cli.utils.steering import validate_steering_method + + with pytest.raises(ValueError): + validate_steering_method("caa\x00") + + def test_oversize_rejected(self): + from soup_cli.utils.steering import validate_steering_method + + with pytest.raises(ValueError): + validate_steering_method("c" * 100) + + def test_unknown_rejected(self): + from soup_cli.utils.steering import validate_steering_method + + with pytest.raises(ValueError, match="steering"): + validate_steering_method("nonsense") + + +# ---------- validate_steering_name ---------- + + +class TestValidateName: + def test_happy(self): + from soup_cli.utils.steering import validate_steering_name + + for name in ("safety-v1", "helpfulness_2024", "tone-formal"): + assert validate_steering_name(name) == name + + def test_bool_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + with pytest.raises(TypeError): + validate_steering_name(True) + + def test_non_string_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + with pytest.raises(TypeError): + validate_steering_name(1) + + def test_empty_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + with pytest.raises(ValueError): + validate_steering_name("") + + def test_null_byte_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + with pytest.raises(ValueError): + validate_steering_name("safety\x00") + + def test_oversize_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + with pytest.raises(ValueError): + validate_steering_name("x" * 200) + + def test_invalid_chars_rejected(self): + from soup_cli.utils.steering import validate_steering_name + + # Path separators / spaces / shell metacharacters rejected so + # the name can be safely embedded in CLI args and filenames. + for bad in ("foo/bar", "foo bar", "foo;rm", "foo$x", "../escape"): + with pytest.raises(ValueError): + validate_steering_name(bad) + + +# ---------- validate_steering_strength ---------- + + +class TestValidateStrength: + def test_happy(self): + from soup_cli.utils.steering import validate_steering_strength + + assert validate_steering_strength(0.5) == 0.5 + assert validate_steering_strength(-1.0) == -1.0 + assert validate_steering_strength(0.0) == 0.0 + + def test_int_coerced(self): + from soup_cli.utils.steering import validate_steering_strength + + assert validate_steering_strength(1) == 1.0 + + def test_bool_rejected(self): + from soup_cli.utils.steering import validate_steering_strength + + with pytest.raises(TypeError): + validate_steering_strength(True) + + def test_non_finite_rejected(self): + import math + + from soup_cli.utils.steering import validate_steering_strength + + with pytest.raises(ValueError): + validate_steering_strength(math.nan) + with pytest.raises(ValueError): + validate_steering_strength(math.inf) + + def test_out_of_bounds_rejected(self): + from soup_cli.utils.steering import validate_steering_strength + + with pytest.raises(ValueError): + validate_steering_strength(100.0) + with pytest.raises(ValueError): + validate_steering_strength(-100.0) + + +# ---------- get_steering_method_spec ---------- + + +class TestSpec: + def test_caa_spec(self): + from soup_cli.utils.steering import get_steering_method_spec + + spec = get_steering_method_spec("caa") + assert spec.name == "caa" + assert spec.live_wired is False + assert spec.needs_contrastive_pairs is True + + def test_iti_spec(self): + from soup_cli.utils.steering import get_steering_method_spec + + spec = get_steering_method_spec("iti") + assert spec.name == "iti" + assert spec.needs_attention_heads is True + + def test_repe_spec(self): + from soup_cli.utils.steering import get_steering_method_spec + + spec = get_steering_method_spec("repe") + assert spec.name == "repe" + + def test_unknown_raises(self): + from soup_cli.utils.steering import get_steering_method_spec + + with pytest.raises(ValueError): + get_steering_method_spec("nonsense") + + def test_frozen(self): + from soup_cli.utils.steering import get_steering_method_spec + + spec = get_steering_method_spec("caa") + with pytest.raises(dataclasses.FrozenInstanceError): + spec.name = "mutated" # type: ignore[misc] + + +# ---------- Deferred-live stubs ---------- + + +class TestDeferredStubs: + def test_apply_steering_deferred(self): + from soup_cli.utils.steering import apply_steering + + with pytest.raises(NotImplementedError, match="v0.62.1"): + apply_steering("caa") + + def test_apply_steering_validates_first(self): + from soup_cli.utils.steering import apply_steering + + # Unknown method rejected BEFORE the deferred-live raise. + with pytest.raises(ValueError): + apply_steering("nonsense") + + def test_build_steering_vector_deferred(self): + from soup_cli.utils.steering import build_steering_vector + + with pytest.raises(NotImplementedError, match="v0.62.1"): + build_steering_vector(method="caa", name="safety-v1") + + def test_build_steering_vector_validates_method_first(self): + from soup_cli.utils.steering import build_steering_vector + + with pytest.raises(ValueError): + build_steering_vector(method="nonsense", name="safety-v1") + + def test_build_steering_vector_validates_name_first(self): + from soup_cli.utils.steering import build_steering_vector + + with pytest.raises(ValueError): + build_steering_vector(method="caa", name="bad/path") + + +# ---------- Registry artifact kind ---------- + + +class TestRegistryArtifactKind: + def test_steering_vector_in_valid_kinds(self): + from soup_cli.registry.store import _VALID_KINDS + + assert "steering_vector" in _VALID_KINDS + + +# ---------- CLI plumbing ---------- + + +class TestCLI: + def test_cli_help_lists_steer(self): + from typer.testing import CliRunner + + from soup_cli.cli import app + + runner = CliRunner() + result = runner.invoke(app, ["--help"]) + assert result.exit_code == 0 + assert "steer" in result.stdout.lower() + + def test_steer_help(self): + from typer.testing import CliRunner + + from soup_cli.commands.steer import app + + runner = CliRunner() + result = runner.invoke(app, ["--help"]) + assert result.exit_code == 0 + + def test_steer_train_help(self): + from typer.testing import CliRunner + + from soup_cli.commands.steer import app + + runner = CliRunner() + result = runner.invoke(app, ["train", "--help"]) + assert result.exit_code == 0 + + def test_steer_apply_help(self): + from typer.testing import CliRunner + + from soup_cli.commands.steer import app + + runner = CliRunner() + result = runner.invoke(app, ["apply", "--help"]) + assert result.exit_code == 0 + + def test_steer_list_help(self): + from typer.testing import CliRunner + + from soup_cli.commands.steer import app + + runner = CliRunner() + result = runner.invoke(app, ["list", "--help"]) + assert result.exit_code == 0 + + def test_steer_train_unknown_method_rejected(self): + from typer.testing import CliRunner + + from soup_cli.commands.steer import app + + runner = CliRunner() + result = runner.invoke(app, [ + "train", + "--base", "meta-llama/Llama-3.1-8B-Instruct", + "--method", "nonsense", + "--name", "safety-v1", + "--pairs", "./data/pairs.jsonl", + "--plan-only", + ]) + # CLI either exits 2 (validation) or 1 (plan rejected); not 0. + assert result.exit_code != 0 + + def test_steer_train_plan_only(self, tmp_path, monkeypatch): + from typer.testing import CliRunner + + from soup_cli.commands.steer import app + + monkeypatch.chdir(tmp_path) + # Create a tiny pairs JSONL to pass the path-containment check. + pairs = tmp_path / "pairs.jsonl" + pairs.write_text( + '{"positive": "be safe", "negative": "be harmful"}\n', encoding="utf-8" + ) + runner = CliRunner() + result = runner.invoke(app, [ + "train", + "--base", "meta-llama/Llama-3.1-8B-Instruct", + "--method", "caa", + "--name", "safety-v1", + "--pairs", "pairs.jsonl", + "--plan-only", + ]) + # plan-only succeeds with friendly deferred-live panel. + assert result.exit_code == 0 + assert "v0.62" in result.stdout or "caa" in result.stdout.lower() + + +# ---------- soup serve --steer flag plumbing ---------- + + +class TestServeSteerFlag: + def test_serve_help_mentions_steer(self): + from typer.testing import CliRunner + + from soup_cli.cli import app + + runner = CliRunner() + result = runner.invoke(app, ["serve", "--help"]) + # Even if the flag is not yet alive, --help should expose it so a + # YAML / shell pipeline can reference it. + assert result.exit_code == 0 + assert "steer" in result.stdout.lower() diff --git a/tests/test_v0620_part_d.py b/tests/test_v0620_part_d.py new file mode 100644 index 0000000..f6cadeb --- /dev/null +++ b/tests/test_v0620_part_d.py @@ -0,0 +1,380 @@ +"""Tests for v0.62.0 Part D — Citation-faithful FT. + +Schema-only release: `training.citation_faithful: bool` opt-in trains the +model to cite document IDs verbatim from the training corpus. Composes +with v0.62.0 Part A (RAFT) — the RAFT data already contains the doc +references; this flag adds a citation-precision / recall scorer to the +eval suite + a loss-mask rule that emphasises citation spans. + +Live citation-span loss-mask + eval scorer land in v0.62.1. +""" + +from __future__ import annotations + +import dataclasses + +import pytest + +# ---------- Module surface ---------- + + +class TestModuleSurface: + def test_imports(self): + from soup_cli.utils.citation_faithful import ( + SUPPORTED_CITATION_STYLES, + CitationScore, + score_citations, + validate_citation_style, + validate_citation_threshold, + ) + assert callable(validate_citation_style) + assert callable(validate_citation_threshold) + assert callable(score_citations) + assert dataclasses.is_dataclass(CitationScore) + assert isinstance(SUPPORTED_CITATION_STYLES, frozenset) + + def test_styles_exact(self): + from soup_cli.utils.citation_faithful import SUPPORTED_CITATION_STYLES + + assert SUPPORTED_CITATION_STYLES == frozenset( + {"bracket", "inline", "footnote"} + ) + + +# ---------- validate_citation_style ---------- + + +class TestValidateStyle: + def test_happy(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + for name in ("bracket", "inline", "footnote"): + assert validate_citation_style(name) == name + + def test_case_insensitive(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + assert validate_citation_style("BRACKET") == "bracket" + + def test_bool_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + with pytest.raises(TypeError): + validate_citation_style(True) + + def test_non_string_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + with pytest.raises(TypeError): + validate_citation_style(1) + + def test_empty_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + with pytest.raises(ValueError): + validate_citation_style("") + + def test_null_byte_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + with pytest.raises(ValueError): + validate_citation_style("bracket\x00") + + def test_oversize_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + with pytest.raises(ValueError): + validate_citation_style("x" * 64) + + def test_unknown_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_style + + with pytest.raises(ValueError, match="citation"): + validate_citation_style("apa-7th") + + +# ---------- validate_citation_threshold ---------- + + +class TestValidateThreshold: + def test_happy(self): + from soup_cli.utils.citation_faithful import validate_citation_threshold + + assert validate_citation_threshold(0.5) == 0.5 + assert validate_citation_threshold(0.0) == 0.0 + assert validate_citation_threshold(1.0) == 1.0 + + def test_bool_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_threshold + + with pytest.raises(TypeError): + validate_citation_threshold(True) + + def test_non_finite_rejected(self): + import math + + from soup_cli.utils.citation_faithful import validate_citation_threshold + + with pytest.raises(ValueError): + validate_citation_threshold(math.nan) + with pytest.raises(ValueError): + validate_citation_threshold(math.inf) + + def test_out_of_range_rejected(self): + from soup_cli.utils.citation_faithful import validate_citation_threshold + + with pytest.raises(ValueError): + validate_citation_threshold(-0.1) + with pytest.raises(ValueError): + validate_citation_threshold(1.1) + + +# ---------- score_citations ---------- + + +class TestScoreCitations: + def test_perfect_match(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations( + predicted="The capital is Paris [doc-1].", + expected_ids=("doc-1",), + ) + assert score.precision == 1.0 + assert score.recall == 1.0 + assert score.f1 == 1.0 + + def test_no_citations(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations( + predicted="The capital is Paris.", + expected_ids=("doc-1",), + ) + # No predicted citations: precision is undefined (set to 0.0 by + # convention), recall is 0.0. + assert score.precision == 0.0 + assert score.recall == 0.0 + assert score.f1 == 0.0 + + def test_extra_citation_lowers_precision(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations( + predicted="See [doc-1] and [doc-2].", + expected_ids=("doc-1",), + ) + # Recall = 1/1 = 1.0, Precision = 1/2 = 0.5. + assert score.recall == 1.0 + assert score.precision == 0.5 + assert 0.5 < score.f1 < 1.0 + + def test_missing_citation_lowers_recall(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations( + predicted="See [doc-1].", + expected_ids=("doc-1", "doc-2"), + ) + assert score.precision == 1.0 + assert score.recall == 0.5 + + def test_empty_expected_ids(self): + from soup_cli.utils.citation_faithful import score_citations + + # No expected citations: recall is undefined but score returns 0 + # by convention (avoids div-by-zero). Precision still defined. + score = score_citations( + predicted="The capital is Paris [doc-1].", + expected_ids=(), + ) + assert score.recall == 0.0 + # Predicted IDs are not in the empty expected set, so precision + # is 0.0 too. + assert score.precision == 0.0 + + def test_bool_predicted_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(TypeError): + score_citations(predicted=True, expected_ids=("doc-1",)) + + def test_non_string_predicted_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(TypeError): + score_citations(predicted=123, expected_ids=("doc-1",)) + + def test_oversize_predicted_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(ValueError): + score_citations( + predicted="x" * 2_000_001, + expected_ids=("doc-1",), + ) + + def test_non_iterable_expected_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(TypeError): + score_citations(predicted="text", expected_ids=42) + + def test_too_many_expected_rejected(self): + from soup_cli.utils.citation_faithful import score_citations + + with pytest.raises(ValueError): + score_citations( + predicted="text", + expected_ids=tuple(f"doc-{i}" for i in range(10_001)), + ) + + def test_score_frozen(self): + from soup_cli.utils.citation_faithful import score_citations + + score = score_citations(predicted="[doc-1]", expected_ids=("doc-1",)) + with pytest.raises(dataclasses.FrozenInstanceError): + score.f1 = 0.0 # type: ignore[misc] + + +# ---------- Schema integration ---------- + + +class TestSchemaIntegration: + def test_default_off(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig() + assert cfg.citation_faithful is False + assert cfg.citation_style is None + assert cfg.citation_recall_threshold is None + + def test_opt_in_accepts(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig( + citation_faithful=True, + citation_style="bracket", + citation_recall_threshold=0.8, + ) + assert cfg.citation_faithful is True + assert cfg.citation_style == "bracket" + assert cfg.citation_recall_threshold == 0.8 + + def test_style_case_insensitive(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig(citation_style="INLINE") + assert cfg.citation_style == "inline" + + def test_unknown_style_rejected(self): + from pydantic import ValidationError + + from soup_cli.config.schema import TrainingConfig + + with pytest.raises(ValidationError): + TrainingConfig(citation_style="apa") + + def test_threshold_bounds(self): + from pydantic import ValidationError + + from soup_cli.config.schema import TrainingConfig + + with pytest.raises(ValidationError): + TrainingConfig(citation_recall_threshold=-0.1) + with pytest.raises(ValidationError): + TrainingConfig(citation_recall_threshold=1.5) + + +# ---------- Cross-validator ---------- + + +class TestSoupConfigCrossValidator: + def test_citation_faithful_with_raft_format(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_faithful: true + citation_style: bracket + +output: ./output +""" + cfg = load_config_from_string(yaml_text) + assert cfg.training.citation_faithful is True + + def test_citation_faithful_without_raft_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + format: alpaca + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_faithful: true + +output: ./output +""" + with pytest.raises(Exception, match="citation_faithful"): + load_config_from_string(yaml_text) + + def test_citation_style_without_faithful_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_style: bracket + +output: ./output +""" + with pytest.raises(Exception, match="citation"): + load_config_from_string(yaml_text) + + def test_threshold_without_faithful_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/raft.jsonl + format: raft + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + citation_recall_threshold: 0.8 + +output: ./output +""" + with pytest.raises(Exception, match="citation"): + load_config_from_string(yaml_text) diff --git a/tests/test_v0620_part_e.py b/tests/test_v0620_part_e.py new file mode 100644 index 0000000..35d1991 --- /dev/null +++ b/tests/test_v0620_part_e.py @@ -0,0 +1,334 @@ +"""Tests for v0.62.0 Part E — GRACE codebook (long-running edit). + +Discrete latent-space codebook for thousands of sequential edits that +survives lifelong deployments without norm-blowup. Extends the v0.61.0 +edit method allowlist with ``grace`` and ships the codebook config +schema + validators. + +Schema-only release: live codebook lookup + write kernel land in v0.62.1. +""" + +from __future__ import annotations + +import dataclasses + +import pytest + +# ---------- Module surface ---------- + + +class TestModuleSurface: + def test_imports(self): + from soup_cli.utils.grace_codebook import ( + MAX_CODEBOOK_DIM, + MAX_CODEBOOK_SIZE, + GraceCodebookConfig, + apply_grace_codebook, + build_grace_codebook_config, + validate_grace_codebook_dim, + validate_grace_codebook_size, + ) + assert callable(validate_grace_codebook_size) + assert callable(validate_grace_codebook_dim) + assert callable(apply_grace_codebook) + assert callable(build_grace_codebook_config) + assert dataclasses.is_dataclass(GraceCodebookConfig) + assert isinstance(MAX_CODEBOOK_SIZE, int) + assert isinstance(MAX_CODEBOOK_DIM, int) + + +# ---------- validate_grace_codebook_size ---------- + + +class TestValidateSize: + def test_happy(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_size + + for n in (1, 100, 10_000): + assert validate_grace_codebook_size(n) == n + + def test_bool_rejected(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_size + + with pytest.raises(TypeError): + validate_grace_codebook_size(True) + + def test_non_int_rejected(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_size + + with pytest.raises(TypeError): + validate_grace_codebook_size(3.14) + + def test_zero_rejected(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_size + + with pytest.raises(ValueError): + validate_grace_codebook_size(0) + + def test_negative_rejected(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_size + + with pytest.raises(ValueError): + validate_grace_codebook_size(-1) + + def test_overcap_rejected(self): + from soup_cli.utils.grace_codebook import ( + MAX_CODEBOOK_SIZE, + validate_grace_codebook_size, + ) + + with pytest.raises(ValueError): + validate_grace_codebook_size(MAX_CODEBOOK_SIZE + 1) + + +# ---------- validate_grace_codebook_dim ---------- + + +class TestValidateDim: + def test_happy(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_dim + + for d in (8, 768, 4096): + assert validate_grace_codebook_dim(d) == d + + def test_bool_rejected(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_dim + + with pytest.raises(TypeError): + validate_grace_codebook_dim(True) + + def test_zero_rejected(self): + from soup_cli.utils.grace_codebook import validate_grace_codebook_dim + + with pytest.raises(ValueError): + validate_grace_codebook_dim(0) + + def test_overcap_rejected(self): + from soup_cli.utils.grace_codebook import ( + MAX_CODEBOOK_DIM, + validate_grace_codebook_dim, + ) + + with pytest.raises(ValueError): + validate_grace_codebook_dim(MAX_CODEBOOK_DIM + 1) + + +# ---------- GraceCodebookConfig ---------- + + +class TestCodebookConfig: + def test_happy(self): + from soup_cli.utils.grace_codebook import build_grace_codebook_config + + cfg = build_grace_codebook_config(size=128, dim=768) + assert cfg.size == 128 + assert cfg.dim == 768 + + def test_frozen(self): + from soup_cli.utils.grace_codebook import build_grace_codebook_config + + cfg = build_grace_codebook_config(size=128, dim=768) + with pytest.raises(dataclasses.FrozenInstanceError): + cfg.size = 256 # type: ignore[misc] + + def test_invalid_size_propagates(self): + from soup_cli.utils.grace_codebook import build_grace_codebook_config + + with pytest.raises(ValueError): + build_grace_codebook_config(size=0, dim=768) + + def test_invalid_dim_propagates(self): + from soup_cli.utils.grace_codebook import build_grace_codebook_config + + with pytest.raises(ValueError): + build_grace_codebook_config(size=128, dim=0) + + +# ---------- apply_grace_codebook ---------- + + +class TestApplyDeferred: + def test_deferred(self): + from soup_cli.utils.grace_codebook import ( + apply_grace_codebook, + build_grace_codebook_config, + ) + + cfg = build_grace_codebook_config(size=128, dim=768) + with pytest.raises(NotImplementedError, match="v0.62.1"): + apply_grace_codebook(cfg) + + def test_apply_validates_config_type(self): + from soup_cli.utils.grace_codebook import apply_grace_codebook + + with pytest.raises(TypeError): + apply_grace_codebook("not-a-config") + + +# ---------- knowledge_edit allowlist extension ---------- + + +class TestEditMethodAllowlist: + def test_grace_in_supported_methods(self): + from soup_cli.utils.knowledge_edit import SUPPORTED_EDIT_METHODS + + assert "grace" in SUPPORTED_EDIT_METHODS + + def test_grace_method_validates(self): + from soup_cli.utils.knowledge_edit import validate_edit_method + + assert validate_edit_method("grace") == "grace" + assert validate_edit_method("GRACE") == "grace" + + def test_grace_spec_present(self): + from soup_cli.utils.knowledge_edit import get_edit_method_spec + + spec = get_edit_method_spec("grace") + assert spec.name == "grace" + assert spec.multi_edit_capable is True + assert spec.live_wired is False + + def test_grace_edit_plan_happy(self): + from soup_cli.utils.knowledge_edit import build_edit_plan + + plan = build_edit_plan( + base="meta-llama/Llama-3.1-8B-Instruct", + method="grace", + subject="The capital of France is", + target="Lyon", + ) + assert plan.method == "grace" + assert plan.layer >= 0 + + def test_apply_grace_via_edit_path_deferred(self): + from soup_cli.utils.knowledge_edit import apply_edit, build_edit_plan + + plan = build_edit_plan( + base="meta-llama/Llama-3.1-8B-Instruct", + method="grace", + subject="The capital of France is", + target="Lyon", + ) + with pytest.raises(NotImplementedError, match="v0.62.1"): + apply_edit(plan) + + +# ---------- Schema integration ---------- + + +class TestSchemaIntegration: + def test_default_none(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig() + assert cfg.grace_codebook is False + assert cfg.grace_codebook_size is None + assert cfg.grace_codebook_dim is None + + def test_opt_in_accepts(self): + from soup_cli.config.schema import TrainingConfig + + cfg = TrainingConfig( + grace_codebook=True, + grace_codebook_size=128, + grace_codebook_dim=768, + ) + assert cfg.grace_codebook is True + assert cfg.grace_codebook_size == 128 + assert cfg.grace_codebook_dim == 768 + + def test_invalid_size_rejected(self): + from pydantic import ValidationError + + from soup_cli.config.schema import TrainingConfig + + with pytest.raises(ValidationError): + TrainingConfig( + grace_codebook=True, + grace_codebook_size=0, + grace_codebook_dim=768, + ) + + def test_invalid_dim_rejected(self): + from pydantic import ValidationError + + from soup_cli.config.schema import TrainingConfig + + with pytest.raises(ValidationError): + TrainingConfig( + grace_codebook=True, + grace_codebook_size=128, + grace_codebook_dim=-1, + ) + + +# ---------- SoupConfig cross-validator ---------- + + +class TestSoupConfigCrossValidator: + def test_grace_codebook_size_without_flag_rejected(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + grace_codebook_size: 128 + +output: ./output +""" + with pytest.raises(Exception, match="grace_codebook"): + load_config_from_string(yaml_text) + + def test_grace_codebook_requires_both_size_and_dim(self): + from soup_cli.config.loader import load_config_from_string + + # codebook flag but missing size — rejected. + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + grace_codebook: true + grace_codebook_dim: 768 + +output: ./output +""" + with pytest.raises(Exception, match="grace_codebook"): + load_config_from_string(yaml_text) + + def test_grace_codebook_happy(self): + from soup_cli.config.loader import load_config_from_string + + yaml_text = """\ +base: meta-llama/Llama-3.1-8B-Instruct +task: sft + +data: + train: ./data/train.jsonl + +training: + epochs: 1 + lr: 2e-4 + batch_size: auto + grace_codebook: true + grace_codebook_size: 128 + grace_codebook_dim: 768 + +output: ./output +""" + cfg = load_config_from_string(yaml_text) + assert cfg.training.grace_codebook is True