project-nomad/admin/inertia/pages/settings
Chris Sherwood 398ca018cd feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI
Phase 4 of the NOMAD Score v2 effort (stacked on #1089). Captures the raw
channel values the leaderboard scores from, computes the uncapped v2 score
in-app (byte-matching the leaderboard's server-side recompute), submits the
v2 payload, and surfaces it in the benchmark UI.

- Capture raws: per-channel sysbench values + O_DIRECT disk (W4), single/multi
  thread CPU + memory, W6 consistency companions, and run-environment metadata
  (run_environment, storage_path_type, gpu_compute_detected, #1016).
- AI hardening: reference model llama3.1:8b, num_predict=256, VRAM eviction
  before the run + unload own model after, pre-flight disk check (~6.5GB, only
  when the model is uncached), median-of-3 run (W7).
- Score: _calculateNomadScoreV2 (uncapped, reference->1000) byte-matches the
  leaderboard recompute. nomad_score_v2 stays null unless a full run with AI.
- Payload: submitToRepository sends the v2 raws + score alongside legacy v1.
- UI: v2 headline score, legacy v1 as secondary, raws/env in details.
- Migration: 13 nullable columns; double (not knex float(8,2)) so millions-scale
  raws keep full precision and don't break the leaderboard byte-match.

Reference constant REFERENCE_SCORES_V2.ai_tokens_per_second is still the stale
40.5 placeholder (TODO in source); locking it to the measured 13.2 is a
follow-up gated on the AMD iGPU fix (#1074). Live prod v2 submit is separately
blocked by the leaderboard TTFT floor (tracked in the leaderboard follow-up).

Verified on NOMAD3 (RTX 5060) dev env: migration applies, raws recorded,
v2 score computed, backend + benchmark.tsx typecheck clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 09:36:14 -07:00
..
zim feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-07-20 16:21:18 -07:00
advanced.tsx chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
apps.tsx chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
benchmark.tsx feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI 2026-07-21 09:36:14 -07:00
creator-packs.tsx feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-07-20 16:21:18 -07:00
legal.tsx feat(brand): add ™ to Project NOMAD wordmark on prominent surfaces 2026-07-18 09:56:54 -07:00
maps.tsx fix(maps): warn when world basemap missing instead of silent grey map (#1104) 2026-07-20 18:25:39 -07:00
models.tsx feat(AI): per-model thinking toggle with global default (off) (#1079) 2026-07-20 16:33:09 -07:00
support.tsx chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00
system.tsx feat(AI): improved AMD GPU acceleration for Ollama via ROCm + HSA override (#804) 2026-05-20 10:16:00 -07:00
update.tsx chore: standardize brand name to Project NOMAD, retire backronym 2026-07-18 09:56:54 -07:00