Alpamys
|
96c339f184
|
feat(edit): live ROME/MEMIT/AlphaEdit + GRACE + NPO/SimNPO/RMU unlearn (v0.71.9)
Closes #193, #194, #196, #197, #203.
- #194 utils/edit_kernels.py: covariance-free rank-1 ROME/MEMIT/AlphaEdit;
apply_edit live (load -> optimise residual -> rank-1 update -> save);
edit diff live before/after generation.
- #196 EditGovernorStore SQLite persistence + cross-process lock.
- #197 apply_edit consults the governor (check_can_edit before, record after).
- #203 GraceCodebook + apply_grace_edit + install_grace_hook + Registry kinds.
- #193 utils/unlearn_kernels.py (NPO/SimNPO/RMU) + live UnlearnTrainerWrapper
+ soup train --task unlearn.
Validated on SmolLM2-135M: ROME 0.0016->0.96, NPO/SimNPO forget loss down.
+81 tests (tests/test_v0719.py). 2 review waves, all findings fixed.
|
2026-06-03 17:04:03 +05:00 |
|
Alpamys
|
0d6f95181a
|
feat(v0.62.0): RAG & Activation Steering — RAFT + RA-DIT + soup steer + citation-faithful + GRACE codebook
5 Parts shipping wedge 14 of the roadmap (RAG-aware fine-tuning +
activation steering + lifelong edit codebook). Schema-only release;
live training loops + decode-hook intervention + codebook lookup all
land in v0.62.1 (mirrors v0.50.0 / v0.52.0 / v0.61.0 stub-then-live).
Part A — RAFT data format: new data.format='raft' schema +
_convert_raft validator (64 KiB per-field cap, 64-distractor cap,
null-byte rejection on every field) + raft-llama3-8b recipe.
Part B — RA-DIT two-stage: TrainingConfig.ra_dit_stage Literal
{retriever, generator} + ra_dit_retriever_model field + closed
allowlist + cross-validator enforcing stage to base-task pairing
(retriever to embedding, generator to sft) + 2 recipes.
Part C — soup steer (CAA / ITI / RepE): closed-allowlist control-vector
methods + validate_steering_method/name/strength + Typer subcommands
train/apply/list + soup serve --steer/--steer-strength flags +
steering_vector Registry artifact kind. apply_steering +
build_steering_vector deferred-live stubs raise NotImplementedError
with v0.62.1 marker after validating inputs.
Part D — Citation-faithful FT: score_citations precision/recall/F1
kernel + extract_citation_ids public API + citation_faithful /
citation_style / citation_recall_threshold schema. Cross-validator:
citation_faithful=true requires data.format='raft' AND task in
{sft, pretrain} (silent-no-op footgun rejection mirroring v0.52.0
distill / classifier task-gate policy).
Part E — GRACE codebook: GraceCodebookConfig + bounded size [1, 100k]
+ bounded dim [1, 16384]. Extends v0.61.0 SUPPORTED_EDIT_METHODS
allowlist with 'grace'; apply_edit routes grace plans to v0.62.1
marker while legacy rome/memit/alphaedit retain v0.61.1 marker
(regression-guarded via TestEditMarkerRegressionGuard).
Test count: 9571 -> 9786 (+215 net). 4 review-agent waves resolved
0 CRITICAL + 0 HIGH + 4 MEDIUM + 11 LOW (broken list_steers registry
context-manager + dict-key access; missing version bump;
citation_faithful task-gate; shared TOCTOU helper delegation; Rich
markup escape on --steer exception messages; --base length cap +
null-byte rejection; typing.Iterable -> collections.abc.Iterable
migration; except Exception -> except ImportError narrowing).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-20 20:20:30 +05:00 |
|
Alpamys
|
740832e1b4
|
feat(unlearn/edit): v0.61.0 — Unlearning & Knowledge Edit (NPO/SimNPO/RMU + ROME/MEMIT/AlphaEdit)
5 Parts shipping schema + CLI surface for two of the most under-served axes
in fine-tuning: GDPR right-to-be-forgotten unlearning (the legal-liability
axis upstream TRL avoids) and surgical knowledge editing (research-coded
everywhere, productized nowhere). Schema-only release; live trainer +
kernel wiring deferred to v0.61.1 (matches established v0.50.0 / v0.52.0
/ v0.53.0 stub-then-live cadence).
Part A — task='unlearn' + NPO/SimNPO/RMU allowlist + UnlearnTrainerWrapper
+ data.forget_set / data.retain_set + training.unlearn_method/_alpha
Part B — soup eval unlearning (TOFU/MUSE/WMDP) with Forget Quality + Model
Utility + PrivLeak kernels + OK/MINOR/MAJOR taxonomy; bundled TOFU
mini-fixture under soup_cli/data/_fixtures/unlearning/
Part C — soup edit set (ROME/MEMIT/AlphaEdit) + EditPlan + per-method
default layer; --plan-only ships live, apply_edit kernel deferred
Part D — Sequential edit governor: norm-blowup detection (OK/WARN/BLOWUP),
auto-switch ROME→AlphaEdit at edit#10 or BLOWUP, refuses past cap
Part E — soup edit diff: cwd-contained probe loader, atomic JSONL out,
shape + table renderer (live before/after generation v0.61.1)
Net: +125 tests (9446 → 9571), +5 utility modules + 1 trainer wrapper +
2 commands. Review-fix coverage: 0 CRITICAL + 5 HIGH + 11 MEDIUM + 11 LOW.
All ruff + pytest green; Step 6 smokes (CLI plumbing + happy paths + 5
schema rejection paths) confirmed end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-19 15:16:15 +05:00 |