project-nomad/admin/app/services
Chris Sherwood 651df80dc8 feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI
Phase 4 of the NOMAD Score v2 effort (stacked on #1089). Captures the raw
channel values the leaderboard scores from, computes the uncapped v2 score
in-app (byte-matching the leaderboard's server-side recompute), submits the
v2 payload, and surfaces it in the benchmark UI.

- Capture raws: per-channel sysbench values + O_DIRECT disk (W4), single/multi
  thread CPU + memory, W6 consistency companions, and run-environment metadata
  (run_environment, storage_path_type, gpu_compute_detected, #1016).
- AI hardening: reference model llama3.1:8b, num_predict=256, VRAM eviction
  before the run + unload own model after, pre-flight disk check (~6.5GB, only
  when the model is uncached), median-of-3 run (W7).
- Score: _calculateNomadScoreV2 (uncapped, reference->1000) byte-matches the
  leaderboard recompute. nomad_score_v2 stays null unless a full run with AI.
- Payload: submitToRepository sends the v2 raws + score alongside legacy v1.
- UI: v2 headline score, legacy v1 as secondary, raws/env in details.
- Migration: 13 nullable columns; double (not knex float(8,2)) so millions-scale
  raws keep full precision and don't break the leaderboard byte-match.

Reference constant REFERENCE_SCORES_V2.ai_tokens_per_second is still the stale
40.5 placeholder (TODO in source); locking it to the measured 13.2 is a
follow-up gated on the AMD iGPU fix (#1074). Live prod v2 submit is separately
blocked by the leaderboard TTFT floor (tracked in the leaderboard follow-up).

Verified on NOMAD3 (RTX 5060) dev env: migration applies, raws recorded,
v2 score computed, backend + benchmark.tsx typecheck clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 10:40:24 -07:00
..
app_auto_update_service.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
auto_update_service.ts feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
benchmark_service.ts feat(benchmark): Score v2 app client — raws, uncapped score, v2 payload + UI 2026-07-13 10:40:24 -07:00
chat_service.ts fix(chat): prefer selected model for suggestions, fall back to smallest 2026-06-23 04:47:00 +00:00
collection_manifest_service.ts fix(content): narrow Wikipedia reconcile-skip to the managed selection file 2026-06-23 04:46:57 +00:00
collection_update_service.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
container_registry_service.ts fix(updates): resolve relative registry pagination URL so tag listing doesn't crash (#945) 2026-06-23 04:46:59 +00:00
content_auto_update_service.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
countries_service.ts feat(Maps): regional map downloads via go-pmtiles extract (#780) 2026-05-20 10:16:00 -07:00
custom_app_guard.ts feat: supply depot 2026-06-23 04:46:45 +00:00
docker_service.ts feat(supply-depot): add MeshCore Web with self-signed HTTPS 2026-06-23 04:47:06 +00:00
docs_service.ts docs: update in-app docs for v1.33 Supply Depot + auto-updates 2026-06-23 04:47:04 +00:00
download_service.ts feat(Maps): regional map downloads via go-pmtiles extract (#780) 2026-05-20 10:16:00 -07:00
kiwix_catalog_service.ts feat(content): opt-in automatic updates for installed ZIM & map content 2026-06-23 04:47:01 +00:00
kiwix_library_service.ts fix(kiwix): self-heal a missing or corrupt library XML on startup 2026-06-23 04:46:56 +00:00
map_service.ts fix(content): remove superseded curated map/ZIM files when a new version installs 2026-06-23 04:47:01 +00:00
ollama_service.ts fix(AI): truncate-and-retry oversized embed chunks; stop 30x retry storm (#881) 2026-06-23 04:46:57 +00:00
queue_service.ts fix(queue): share one ioredis connection across BullMQ queues and workers (#1009) 2026-06-23 04:47:08 +00:00
rag_service.ts feat(KnowledgeBase): add document viewer, download, metadata, and sorting (#721) 2026-06-23 04:47:09 +00:00
system_service.ts feat: configurable internet test url override in new Advanced Settings page 2026-06-23 04:47:10 +00:00
system_update_service.ts feat(system): add opt-in automatic updates for the core NOMAD app 2026-06-23 04:46:51 +00:00
zim_extraction_service.ts fix(RAG): report ZIM ingestion progress in overall-file frame 2026-05-20 10:16:00 -07:00
zim_service.ts feat(zim): add zim uploader in content manager 2026-06-23 04:47:08 +00:00