test: record the MINI_MMLU coupling behind the 0.4 gate expectation (#315)

The benchmark gate-task test pins score == 0.4, which is 2/5 -- exactly
two MINI_MMLU answers are "B" and the fake generate_fn returns "B".
Editing the fixture in forgetting.py moves the number, so say so.

Comment only, no logic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alpamys 2026-07-17 16:31:43 +05:00
parent 85450309f4
commit 780758a6ef
1 changed files with 2 additions and 0 deletions

View File

@ -131,6 +131,8 @@ class TestRunGateErrorPropagation:
result = run_gate(suite, generate_fn=lambda _p: "B")
row = result.task_results[0]
# 0.4 == 2/5: exactly two MINI_MMLU answers are "B". Editing that
# fixture in forgetting.py moves this number.
assert row.score == 0.4
assert row.error is None
assert row.passed is True