diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b1a5d3..e0c4106 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,18 @@ reproducing 70+ versions of notes. ## [Unreleased] +### Fixed + +- **Eval-gate `type: benchmark` tasks now actually run.** `eval/gate.py` probed + for a `forgetting.run_mini_benchmark` helper that never existed, so every + `type: benchmark` task in a gate suite failed 100% of the time — while + advising an `[eval]` extras install that could not fix it. The gate now calls + `ForgettingDetector` directly (the same way `soup ship` already did), and an + unknown benchmark name fails with the list of valid names. Thanks + [@Sanjays2402](https://github.com/Sanjays2402)! + ([#315](https://github.com/MakazhanAlpamys/Soup/pull/315), closes + [#310](https://github.com/MakazhanAlpamys/Soup/issues/310)) + ## [0.71.36] - 2026-07-16 **Data Moat II** — a semantic layer over your training data, plus two tools for diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 9b9482e..40a089b 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -52,6 +52,8 @@ Listed by first contribution. PR numbers link the work. - Honor configured vocab expansion in the ORPO / SimPO / GRPO trainers ([#295](https://github.com/MakazhanAlpamys/Soup/pull/295)) - **Ekaanksh Patil** ([@Ekaanksh-dev](https://github.com/Ekaanksh-dev)) - Batch the PRM reward forward pass in `PRMScorer.__call__` (single `[B, T]` forward) ([#301](https://github.com/MakazhanAlpamys/Soup/pull/301)) +- **Sanjay Santhanam** ([@Sanjays2402](https://github.com/Sanjays2402)) + - Run built-in benchmark gate tasks through `ForgettingDetector` — every `type: benchmark` eval-gate task had always failed ([#315](https://github.com/MakazhanAlpamys/Soup/pull/315)) ---