Score v2 Phase 1. Safe under v1 — no scoring, weight, reference, or
submission-payload changes; only the benchmark's failure behavior and
forensic metadata.
- Fail-on-parse-miss (W3): the four SCORED sysbench metrics (CPU
events/sec, memory ops/sec, disk read/write MiB/s) now THROW when the
regex misses or the value is <= 0, instead of silently returning 0.
A parse failure (e.g. an upstream image output-format change) now
fails the run with a clear error via the existing _runBenchmark
try/catch, rather than submitting a phantom zero sub-score. Secondary/
informational fields keep their existing defaults.
- Pin sysbench by digest (W3): severalnines/sysbench@sha256:64cd003b...
(was :latest), so a latest-tag format change can't break the parsers
fleet-wide. Digest validated on the NOMAD6 reference build.
- Record provenance (W7): sysbench_digest + ollama_version (from Ollama
/api/version, null-tolerant) stored on each result. New nullable
columns + additive migration.
Verified: pinned digest pulls/runs/parses on NOMAD3; migration applies
(columns present); typecheck clean.
Part of the NOMAD Score v2 effort.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Chris Sherwood