project-nomad/admin/app/services
Chris Sherwood 398ca018cd feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI
Phase 4 of the NOMAD Score v2 effort (stacked on #1089). Captures the raw
channel values the leaderboard scores from, computes the uncapped v2 score
in-app (byte-matching the leaderboard's server-side recompute), submits the
v2 payload, and surfaces it in the benchmark UI.

- Capture raws: per-channel sysbench values + O_DIRECT disk (W4), single/multi
  thread CPU + memory, W6 consistency companions, and run-environment metadata
  (run_environment, storage_path_type, gpu_compute_detected, #1016).
- AI hardening: reference model llama3.1:8b, num_predict=256, VRAM eviction
  before the run + unload own model after, pre-flight disk check (~6.5GB, only
  when the model is uncached), median-of-3 run (W7).
- Score: _calculateNomadScoreV2 (uncapped, reference->1000) byte-matches the
  leaderboard recompute. nomad_score_v2 stays null unless a full run with AI.
- Payload: submitToRepository sends the v2 raws + score alongside legacy v1.
- UI: v2 headline score, legacy v1 as secondary, raws/env in details.
- Migration: 13 nullable columns; double (not knex float(8,2)) so millions-scale
  raws keep full precision and don't break the leaderboard byte-match.

Reference constant REFERENCE_SCORES_V2.ai_tokens_per_second is still the stale
40.5 placeholder (TODO in source); locking it to the measured 13.2 is a
follow-up gated on the AMD iGPU fix (#1074). Live prod v2 submit is separately
blocked by the leaderboard TTFT floor (tracked in the leaderboard follow-up).

Verified on NOMAD3 (RTX 5060) dev env: migration applies, raws recorded,
v2 score computed, backend + benchmark.tsx typecheck clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 09:36:14 -07:00
..
app_auto_update_service.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
auto_update_service.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
benchmark_service.ts feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI 2026-07-21 09:36:14 -07:00
benchmark_telemetry.ts feat(benchmark): end-of-run score reveal + NVIDIA GPU-util overlay (#1087) 2026-07-20 12:07:45 -07:00
chat_service.ts fix(chat): prefer selected model for suggestions, fall back to smallest 2026-06-23 04:47:00 +00:00
collection_manifest_service.ts feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-07-20 16:21:18 -07:00
collection_update_service.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
container_registry_service.ts fix(updates): resolve relative registry pagination URL so tag listing doesn't crash (#945) 2026-06-23 04:46:59 +00:00
content_auto_update_service.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
countries_service.ts feat(Maps): regional map downloads via go-pmtiles extract (#780) 2026-05-20 10:16:00 -07:00
creator_pack_service.ts feat(creator-packs): gated per-creator video packs, offline via Kiwix (#1106) 2026-07-20 16:21:18 -07:00
custom_app_guard.ts feat: supply depot 2026-06-23 04:46:45 +00:00
docker_service.ts feat(debug-info): add storage, docker, GPU health, and auto-update diagnostics (#1102) 2026-07-19 22:25:53 -07:00
docs_service.ts docs: update in-app docs for v1.33 Supply Depot + auto-updates 2026-06-23 04:47:04 +00:00
download_service.ts fix(downloads): add retry button and resource download link for failed downloads (#1059) 2026-07-18 09:25:42 -07:00
kiwix_catalog_service.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
kiwix_library_service.ts fix(kiwix): self-heal a missing or corrupt library XML on startup 2026-06-23 04:46:56 +00:00
map_service.ts fix(maps): warn when world basemap missing instead of silent grey map (#1104) 2026-07-20 18:25:39 -07:00
ollama_service.ts feat(AI): per-model thinking toggle with global default (off) (#1079) 2026-07-20 16:33:09 -07:00
queue_service.ts fix(queue): share one ioredis connection across BullMQ queues and workers (#1009) 2026-06-23 04:47:08 +00:00
rag_service.ts fix(rag): add proper .docx text extraction via mammoth (#1100) 2026-07-20 15:17:50 -07:00
system_service.ts feat(debug-info): add storage, docker, GPU health, and auto-update diagnostics (#1102) 2026-07-19 22:25:53 -07:00
system_update_service.ts feat(system): add opt-in automatic updates for the core NOMAD app 2026-06-23 04:46:51 +00:00
zim_extraction_service.ts chore(KB): filter non-content sections + render tables in ZIM extraction (#1044) 2026-07-18 09:03:53 -07:00
zim_service.ts fix(content): resolve current ZIM URL before download (#1091) 2026-07-20 19:36:53 -07:00