Three changes that together let ARM hardware appear on the leaderboard honestly.
Shipping them separately would leave ARM half-supported either way: without the
image a Pi cannot submit at all, and without the architecture field it submits
but is indistinguishable from x86.
1. PIN THE OFFICIAL MULTI-ARCH SYSBENCH IMAGE
severalnines/sysbench publishes amd64 only, so ARM hosts could not run the
System Benchmark at all — not a graceful failure, the container simply cannot
execute. Apple Silicon could only run it under Rosetta emulation, which distorts
the measurement it is taking, and that is what drove a community macOS fork to
substitute a different benchmark and submit incomparable numbers.
Swaps to ghcr.io/crosstalk-solutions/nomad-sysbench (Debian 12 + sysbench
1.0.20+ds-5, built for linux/amd64 + linux/arm64). One digest covers both
architectures; verified that pulling the pinned manifest-list digest resolves to
arm64 on a Raspberry Pi 5 and amd64 on x86, and that RepoDigests reports the
same manifest-list digest on both — so a single allowlist entry serves both.
No rescoring: 1.0.17 -> 1.0.20 measured 1.25% apart on identical hardware with
identical flags (7170.18 vs 7259.56 events/sec), inside run-to-run noise and
~0.3% on a composite. Both digests are allowlisted server-side, so the fleet can
cross over gradually.
2. REPORT THE DIGEST ACTUALLY RESOLVED
The submission previously sent SYSBENCH_DIGEST, the constant the client was
compiled with. The leaderboard validates that field, but a constant attests to
how a client was BUILT rather than what it RAN, so any build inherits a valid
value simply by carrying the same source.
Now reads it back from the image. Uses RepoDigests (the manifest digest we
pulled by), never Id — Id is the config digest, differs per architecture, and
would never match the allowlist. Falls back to the constant if inspection yields
nothing usable, so a benchmark never fails over provenance metadata.
Still forgeable, and always will be with an open-source client. It moves the bar
from "no effort" to "deliberate", which is the distinction that matters when
judging whether a submission is a mistake or a choice.
3. RECORD CPU ARCHITECTURE AND OS
The leaderboard is a single board across instruction sets by design, with
disclosure as the fairness mechanism. Without an architecture field an ARM result
sits unlabelled beside x86 — exactly what the disclosure exists to prevent.
All three fields come from the Docker daemon, reusing the docker.info() call
_detectRunEnvironment already makes. That is deliberate: inside the admin
container os.arch() and si.osInfo() describe the CONTAINER, not the host being
benchmarked.
cpu_architecture Architecture x86_64 -> amd64, aarch64 -> arm64
os_version OSVersion '24.04' (already structured, no parsing)
os_name OperatingSystem 'Ubuntu 24.04.4 LTS' minus the version
run_environment is kept rather than replaced: "which distro" and "is this
virtualised" are different questions, and WSL2 is a real performance factor.
String handling lives in app/utils/platform_metadata.ts with unit tests, matching
the amd_hsa_override convention, so it is testable without a Docker daemon.
Unknown architectures pass through verbatim rather than being guessed at, and
os_name falls back to the full description whenever the version is missing or
absent from it — an over-long name is harmless, a wrong one is not.
Columns are nullable and the submission fields optional, so results recorded
before this shipped remain submittable.
Closes#1156
Refs #1151
DockerService.getServiceURL() resolves ai.remoteOllamaUrl ahead of the local
container, so when a remote AI host is configured the AI channel measures THAT
machine while every other channel measures this one. The submission then reports
someone else's tok/s under this hardware's CPU, RAM and disk.
Under v2 this matters more than it did under v1: ai_tokens_per_second carries
0.30 of the weight and the score is uncapped, so a remote GPU's throughput is no
longer limited by a clamp.
Adds a guard alongside the existing submit-time checks. Uses the same truthiness
predicate as getServiceURL, so the guard fires exactly when the remote routing
it is guarding against would occur. KVStore.clearValue() nulls the value and
getValue() returns null for that, so a cleared key correctly does not trip it.
Blocks submission only. Running the benchmark locally is still useful to the
operator — it just isn't a result about this box, so it shouldn't go on a board
that ranks hardware.
Known limitation: the check reads the KV at submit time, not at measurement
time, so benchmarking with a remote host and then clearing the setting before
submitting would still get through. That is a deliberate workaround rather than
an accident, and closing it properly needs a column on benchmark_results to
record how inference was reached. Worth doing if it ever shows up in practice;
not worth a migration on the evidence available.
Refs #1151
The AI benchmark evicted resident models (forcing the benchmark model cold)
and then went straight into the timed median-of-N loop with no warm-up. On a
cold box the model-load + GPU spin-up cost landed inside the timed runs
(observed: 173s TTFT / 5.83 tok/s vs a ~80 tok/s warm steady-state), and
consecutive runs weren't isolated (a prior run left the model warm). Because
the AI channel is uncapped and ~30% of the composite, the same machine could
post a ~2x-different NOMAD Score depending on warm/cold state (888 vs 1958
observed back-to-back).
Add one discarded warm-up inference after eviction and before the timed loop
so every timed run measures warm, steady-state throughput. Cold and warm
invocations now converge on the same score. Best-effort: a warm-up hiccup
never fails the run.
Closes#1139
The submit flow discarded the real failure reason and always returned the
generic "Failed to submit benchmark results." to the UI. The most common
cause is the leaderboard's one-per-hour rate limit (HTTP 429), which left
users with no idea why their submission failed or that retrying shortly
would also fail.
- benchmark_controller: return a clear, actionable message. Name the rate
limit explicitly on 429; otherwise pass through the underlying detail
(repository error or a service validation message like "already
submitted"), falling back to the generic only when we have nothing.
- benchmark_service: attach the raw upstream `detail` to the thrown error so
the controller can surface it.
The frontend already renders the server `error` string, so no client change
is needed.
Replaces the stale 40.5 placeholder (a llama3.2:1b CPU-era number) with 13.2,
measured on the Reference Build (NOMAD6, 780M) under the v2 AI harness with
llama3.1:8b and iGPU acceleration (OLLAMA_IGPU_ENABLE — the provisioning fix in
PR #1074). This is the AI channel's reference for the uncapped v2 score.
Must ship together with #1074 in v1.34.0: the reference assumes iGPU-accelerated
AMD boxes, so shipping it without the iGPU fix would score AMD installs on CPU
numbers against a GPU bar. The exact value gets a final confirm against the
shipping Ollama config before GA.
Must stay byte-identical to the leaderboard's score_service.ts REFERENCE_SCORES_V2
(the server recomputes the score on submit); the matching leaderboard change is a
sibling PR. Changing the leaderboard reference has no effect on existing rows —
the v1->v2 backfill neutralizes the AI channel (ratio 1 regardless of the
reference), and there are no real v2 submissions yet.
Sibling to the Score v2 app client (#1094); stacked on it (same file/constant).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Shows a dismissible dashboard banner to users who submitted a benchmark under
v1 scoring but don't yet have a Score v2 result, nudging them to re-run so the
community leaderboard gets their v2 number.
- Show logic (benchmark_service.shouldShowRerunBanner): has a result with
submitted_to_repository=true AND no result with nomad_score_v2 AND the
dismiss KV isn't set. Self-clears two ways — dismiss sets the KV, and any v2
run gives a result a nomad_score_v2 so the condition flips off on its own.
- New GET /api/benchmark/rerun-banner + useBenchmarkRerunBanner hook (mirrors
useUpdateAvailable), rendered as a dismissible Alert on the dashboard with a
"Re-run benchmark" CTA to /settings/benchmark.
- New KV key benchmark.rerunBannerDismissed (boolean) in both KV_STORE_SCHEMA
and the SETTINGS_KEYS whitelist (the PATCH /system/settings validator enum);
dismiss writes it via the existing updateSetting endpoint + invalidates the
query.
Sibling to the Score v2 app client (#1094); stacked on it for the
nomad_score_v2 column. Browser-verified on the NOMAD3 dev env: show/dismiss/
reload-persist/self-clear-on-v2 all correct.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 4 of the NOMAD Score v2 effort (stacked on #1089). Captures the raw
channel values the leaderboard scores from, computes the uncapped v2 score
in-app (byte-matching the leaderboard's server-side recompute), submits the
v2 payload, and surfaces it in the benchmark UI.
- Capture raws: per-channel sysbench values + O_DIRECT disk (W4), single/multi
thread CPU + memory, W6 consistency companions, and run-environment metadata
(run_environment, storage_path_type, gpu_compute_detected, #1016).
- AI hardening: reference model llama3.1:8b, num_predict=256, VRAM eviction
before the run + unload own model after, pre-flight disk check (~6.5GB, only
when the model is uncached), median-of-3 run (W7).
- Score: _calculateNomadScoreV2 (uncapped, reference->1000) byte-matches the
leaderboard recompute. nomad_score_v2 stays null unless a full run with AI.
- Payload: submitToRepository sends the v2 raws + score alongside legacy v1.
- UI: v2 headline score, legacy v1 as secondary, raws/env in details.
- Migration: 13 nullable columns; double (not knex float(8,2)) so millions-scale
raws keep full precision and don't break the leaderboard byte-match.
Reference constant REFERENCE_SCORES_V2.ai_tokens_per_second is still the stale
40.5 placeholder (TODO in source); locking it to the measured 13.2 is a
follow-up gated on the AMD iGPU fix (#1074). Live prod v2 submit is separately
blocked by the leaderboard TTFT floor (tracked in the leaderboard follow-up).
Verified on NOMAD3 (RTX 5060) dev env: migration applies, raws recorded,
v2 score computed, backend + benchmark.tsx typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Score v2 Phase 1. Safe under v1 — no scoring, weight, reference, or
submission-payload changes; only the benchmark's failure behavior and
forensic metadata.
- Fail-on-parse-miss (W3): the four SCORED sysbench metrics (CPU
events/sec, memory ops/sec, disk read/write MiB/s) now THROW when the
regex misses or the value is <= 0, instead of silently returning 0.
A parse failure (e.g. an upstream image output-format change) now
fails the run with a clear error via the existing _runBenchmark
try/catch, rather than submitting a phantom zero sub-score. Secondary/
informational fields keep their existing defaults.
- Pin sysbench by digest (W3): severalnines/sysbench@sha256:64cd003b...
(was :latest), so a latest-tag format change can't break the parsers
fleet-wide. Digest validated on the NOMAD6 reference build.
- Record provenance (W7): sysbench_digest + ollama_version (from Ollama
/api/version, null-tolerant) stored on each result. New nullable
columns + additive migration.
Verified: pinned digest pulls/runs/parses on NOMAD3; migration applies
(columns present); typecheck clean.
Part of the NOMAD Score v2 effort.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A System-Only or AI-Only run is a partial result, not the NOMAD Score
(which is the full-benchmark composite). Two problems addressed:
1. Scoring bug: AI-only runs were NOT renormalized. _calculateNomadScore
always added the system weights (0.60) to the denominator even for an
AI-only run (default-zero system scores), so an excellent AI-only run
scored ~39.8 -- scaled against the full NOMAD 100 where AI is only 40%
-- while system-only already renormalized correctly. Fix: pass
systemScores only when the system benchmarks actually ran, so AI-only
renormalizes to its own 0-100 (39.8 -> ~99.7). Full and System-only
scores are unchanged.
2. Presentation: partial runs were shown with the full "NOMAD Score"
label + big gauge, outweighing the small "Partial" notice. Now partial
runs are relabelled "System Score" / "AI Score" with a PARTIAL badge,
a muted (neutral) gauge + number, and a "run a Full Benchmark for your
NOMAD Score" CTA -- applied to both the persistent score section and
the Phase 3 ScoreReveal via a shared getScoreDisplay() helper. Adds a
`muted` prop to CircularGauge.
Browser-validated on NOMAD3: AI-only now shows "AI Score" + PARTIAL,
muted gauge, 99.7 (was 39.8); Full still shows "NOMAD Score" green.
Implements the display-layer fix from the Score v2 red-team's W2; the
AI-ceiling saturation (W1) remains v2 work.
Part of #1082.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 3 of the live benchmark run experience.
- ScoreReveal: replaces the abrupt run-view unmount with a deliberate
end-of-run "REPORT" card -- animated NOMAD score gauge + odometer
count-up number, sub-score gauges cascading in, Continue button +
5s auto-dismiss. Takes score scale as a prop so it survives Score v2.
- GPU-util overlay (NVIDIA): during the AI stage, a ~1Hz nvidia-smi poll
inside the Ollama container feeds live GPU utilization + VRAM into the
telemetry frames; shown in the AI hero, hidden when absent (AMD/none).
Poller is side-effect-only and cleared in a finally -- scored numbers
unchanged.
- Disk polish: reset in-test buffers on stage transition so the write
stage no longer briefly shows the carried disk-read value.
Browser-validated on NOMAD3 (RTX 5060): GPU overlay live (1% util,
0.2/8.0 GB VRAM during model load); reveal cascade + count-up; AI-only
score 39.8 (scored path unchanged).
Part of #1082 (tracker stays open).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2 of the live benchmark run experience. Streams sysbench
--report-interval=1 interim lines over a Docker attach so the run view
shows authoritative in-test CPU events/sec and disk read/write MiB/s
(overlaying the Phase 1 host-proxy disk numbers), and fills a
"results so far" strip as each stage completes.
- _runSysbenchCommandStreaming: attaches to the container output for
live onLine callbacks, but returns the authoritative output via
container.logs() after exit -- byte-identical to _runSysbenchCommand,
so the SCORED numbers are unchanged (attach 'data' can flush after
container.wait() resolves, which would truncate the final report).
- CPU + disk stages stream interim eps: / reads:/writes: MiB/s into
setStageMetric; memory stays non-streaming (too fast to sample).
- _emitPartialResult broadcasts each finished stage's raw result on the
progress channel; useBenchmarkRun accumulates them; ResultsSoFar
renders the chip strip.
- Frontend: live CPU ev/s readout+sparkline, disk hero switches to
"Benchmark throughput" when in-test numbers arrive.
Browser-validated on NOMAD3 (System-Only): CPU 6536 ev/s live, disk
10399 MB/s benchmark throughput, results strip, final score 68.3.
Part of #1082 (tracker stays open for Phase 3).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 of the live benchmark-run experience. Replaces the opaque (and
in sync mode, simulated) progress bar with a real-time run view driven by
actual host telemetry.
- Async run path: the UI now dispatches to the queue worker and keys off
SSE instead of faking stage progress with client-side timers.
- New BenchmarkTelemetrySampler broadcasts per-core CPU load, CPU temp
(best-effort, hidden when unavailable), and disk MB/s at 1 Hz over a new
benchmark-telemetry SSE channel. Runs in the orchestration process, never
the sysbench container, so it cannot affect scores.
- BenchmarkProgress carries the ordered stage plan + index so the frontend
renders a live stage rail.
- AI benchmark streams /api/generate for live tokens/sec and true TTFT; the
scored numbers still come from Ollama's authoritative final eval fields.
- Frontend: useBenchmarkRun hook owns both subscriptions; self-contained SVG
components (StageRail, CoreGrid, Sparkline, LiveReadout) + BenchmarkRunView,
styled in the desert palette. No chart library added.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every Docker pull went through `followProgress(pullStream, resolve)`, passing
the Promise's resolve as dockerode's onFinished(err, output) callback — so the
error argument was ignored and a failed pull (dropped/metered connection, bad
manifest, registry error, disk full mid-pull) resolved as if it had succeeded.
The code then tried to create/start a container from a missing or partial
image, surfacing a confusing downstream error rather than the real cause. (#790)
Add a DockerService.pullImage() helper that rejects when followProgress reports
an error, and route all five pull sites through it:
- service install
- AMD ROCm image pull
- service update
- force-reinstall / recreate (forcePull)
- sysbench benchmark image pull
Closes#790
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#810.
## Bug A: HSA_OVERRIDE_GFX_VERSION=11.0.0 was unconditional
PR #804 set HSA_OVERRIDE_GFX_VERSION=11.0.0 for any AMD GPU. The inline comment
claimed this was harmless on supported discrete cards (gfx1030 RX 6800, etc.) — empirically false. With the override, Ollama crashes during GPU discovery on
gfx1030 and falls back to CPU silently. Affects every NOMAD user with an
RX 6800 or other RDNA 2 discrete card.
The correct value depends on the gfx version:
- gfx1030, gfx1100, gfx1101, gfx1102: officially supported by ROCm — no override
- gfx1031..gfx1036 (RDNA 2 variants + iGPUs like Rembrandt 680M): 10.3.0
- gfx1103, gfx1150, gfx1151 (Phoenix 780M, Strix 890M, Strix Halo): 11.0.0
### Resolution chain in `_resolveAmdHsaOverride()`
1. KV `ai.amdHsaOverride` — manual override; accepts 'none' to disable, or a
semver-style value to force.
2. Marker file `/app/storage/.nomad-amd-gfx` — written by install_nomad.sh
based on lspci codename. Mapped to override via `_mapGfxToHsaOverride()`.
3. Default: `11.0.0` — preserves prior behavior so existing iGPU users
(780M / 890M, the dominant AMD population today) don't regress on upgrade.
Discrete RDNA 2 users on existing installs can opt out via
`ai.amdHsaOverride='none'` and force-reinstall AI Assistant, OR re-run
install_nomad.sh to refresh the marker file.
The helper is used in both `createContainer` (initial install) and
`updateContainer` (image update) paths, replacing the unconditional push.
## Bug B: BenchmarkService had no AMD discrete detection path
`BenchmarkService.getHardwareInfo()` had three GPU detection fallbacks:
1. `si.graphics()` — empty inside Docker for AMD
2. nvidia-smi — NVIDIA only
3. AMD APU regex from CPU model — integrated only
Result: AMD discrete cards (RX 6800, RX 7900 XTX, etc.) showed up as
"GPU: Not detected" on the leaderboard despite ROCm working. Corrupts
leaderboard data quality for that population.
Fix: after the existing fallbacks, call `SystemService.getSystemInfo()` and
read `graphics.controllers[0].model`. That path already handles AMD via the
marker file + Ollama log probe added in PR #804, so we're reusing existing
plumbing rather than duplicating detection logic.
## install_nomad.sh changes
The existing AMD detection block already runs lspci. Added a codename parse
step that maps Navi 21/22/23/24, Rembrandt, Phoenix1/Phoenix2, Strix/Strix
Point/Strix Halo, and Navi 31/32/33 to gfx versions, then writes
`/opt/project-nomad/storage/.nomad-amd-gfx`. Unknown codenames write nothing
(admin handles missing-marker case via the backward-compat default).
## Validation
Both bugs were originally surfaced and validated empirically on RX 6800 /
gfx1030 / Ubuntu 24.04 + kernel 6.17 + ollama/ollama:rocm during the #810
filing. Validation grid from that report:
| Run | NOMAD Score | tok/s | GPU detected |
|-----------------------------------------------|-------------|-------|-------------------------|
| Pre-fix (Bug A active) | n/a | 0 | yes, but library=cpu |
| HSA_OVERRIDE removed, Bug B unfixed | 73.8 | 221.6 | "Not detected" |
| Both fixes hot-patched (this PR's behavior) | 73.7 | 216.0 | AMD Radeon RX 6800 |
Local checks: `npm run typecheck` clean, `npm run build` clean.
The log2 normalization formula `50 * (1 + log2(ratio))` produces negative
values (clamped to 0) whenever the measured value is less than half the
reference. For example, a CPU scoring 1993 events/sec against a 5000
reference gives ratio=0.4, log2(0.4)=-1.32, score=-16 -> 0%.
Fix by dividing log2 by 3 to widen the usable range. This preserves the
50% score at the reference value while allowing below-average hardware
to receive proportional non-zero scores (e.g., 28% for the CPU above).
Also adds debug logging for CPU sysbench output parsing to aid future
diagnosis of parsing issues.
Fixes#415
Rotate the HMAC secret used for signing benchmark submissions to the
community leaderboard. The previous secret was compromised (hardcoded
in open-source code and used to submit a fake leaderboard entry).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous lspci-based GPU detection fails inside Docker containers
because lspci isn't available, causing Ollama to always run CPU-only
even when a GPU + NVIDIA Container Toolkit are present on the host.
Replace with Docker API runtime check (docker.info() -> Runtimes) as
primary detection method. This works from inside any container via the
mounted Docker socket and confirms both GPU presence and toolkit
installation. Keep lspci as fallback for host-based installs and AMD.
Also add Docker-based GPU detection to benchmark hardware info — exec
nvidia-smi inside the Ollama container to get the actual GPU model name
instead of showing "Not detected".
Tested on nomad3 (Intel Core Ultra 9 285HX + RTX 5060): AI performance
went from 12.7 tok/s (CPU) to 281.4 tok/s (GPU) — a 22x improvement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When running in Docker, the systeminformation library cannot see the
host's GPU hardware. This adds a fallback to detect Intel integrated
graphics from the CPU model name, similar to how we handle AMD APUs
with Radeon graphics.
Intel Core Ultra processors (Meteor Lake, Arrow Lake) include Intel
Arc Graphics integrated. This change detects "Core Ultra" in the CPU
brand and reports "Intel Arc Graphics (Integrated)" as the GPU model.
Note: This is for display purposes only - Ollama does not support
Intel integrated graphics for AI acceleration.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(benchmark): Require full benchmark with AI for community sharing
Only allow users to share benchmark results with the community leaderboard
when they have completed a full benchmark that includes AI performance data.
Frontend changes:
- Add AI Assistant installation check via service API query
- Show pre-flight warning when clicking Full Benchmark without AI installed
- Disable AI Only button when AI Assistant not installed
- Show "Partial Benchmark" info alert for non-shareable results
- Only display "Share with Community" for full benchmarks with AI data
- Add note about AI installation requirement with link to Apps page
Backend changes:
- Validate benchmark_type is 'full' before allowing submission
- Require ai_tokens_per_second > 0 for community submission
- Return clear error messages explaining requirements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(benchmark): UI improvements and GPU detection fix
- Fix GPU detection to properly identify AMD discrete GPUs
- Fix gauge colors (high scores now green, low scores red)
- Fix gauge centering (SVG size matches container)
- Add info tooltips for Tokens/sec and Time to First Token
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(benchmark): Extract iGPU from AMD APU CPU name as fallback
When systeminformation doesn't detect graphics controllers (common on
headless Linux), extract the integrated GPU name from AMD APU CPU model
strings like "AMD Ryzen AI 9 HX 370 w/ Radeon 890M".
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(benchmark): Add Builder Tag system for community leaderboard
- Add builder_tag column to benchmark_results table
- Create BuilderTagSelector component with word dropdowns + randomize
- Add 50 adjectives and 50 nouns for NOMAD-themed tags (e.g., Tactical-Llama-1234)
- Add anonymous sharing option checkbox
- Add builder tag display in Benchmark Details section
- Add Benchmark History section showing all past benchmarks
- Update submission API to accept anonymous flag
- Add /api/benchmark/builder-tag endpoint to update tags
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(benchmark): Add HMAC signing for leaderboard submissions
Sign benchmark submissions with HMAC-SHA256 to prevent casual API abuse.
Includes X-NOMAD-Timestamp and X-NOMAD-Signature headers.
Note: Since NOMAD is open source, a determined attacker could extract
the secret. This provides protection against casual abuse only.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Add OLLAMA_API_URL environment variable for Docker networking
- Use host.docker.internal to reach Ollama from NOMAD container
- Add extra_hosts config in compose for Linux compatibility
- Add downloading_ai_model status with clear progress indicator
- Show model download progress on first AI benchmark run
- Fail AI-only benchmarks with clear error if AI unavailable
- Display benchmark errors to users via Alert component
- Improve error messages with error codes for debugging
Fixes issue where AI benchmark silently failed due to NOMAD container
being unable to reach Ollama at localhost:11434.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Benchmark to Settings sidebar navigation
- Fix Luxon DateTime bug when saving submission timestamp
- Add privacy explanation text before Share button
- Add error handling and display for failed submissions
- Show "Submitting..." state and success confirmation
- Add link to view leaderboard after successful submission
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add comprehensive benchmarking capability to measure server performance:
Backend:
- BenchmarkService with CPU, memory, disk, and AI benchmarks using sysbench
- Database migrations for benchmark_results and benchmark_settings tables
- REST API endpoints for running benchmarks and retrieving results
- CLI commands: benchmark:run, benchmark:results, benchmark:submit
- BullMQ job for async benchmark execution with SSE progress updates
- Synchronous mode option (?sync=true) for simpler local dev setup
Frontend:
- Benchmark settings page with circular gauges for scores
- NOMAD Score display with weighted composite calculation
- System Performance section (CPU, Memory, Disk Read/Write)
- AI Performance section (tokens/sec, time to first token)
- Hardware Information display
- Expandable Benchmark Details section
- Progress simulation during sync benchmark execution
Easy Setup Integration:
- Added System Benchmark to Additional Tools section
- Built-in capability pattern for non-Docker features
- Click-to-navigate behavior for built-in tools
Fixes:
- Docker log multiplexing issue (Tty: true) for proper output parsing
- Consolidated disk benchmarks into single container execution
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
chriscrosstalk
jakeaturner
Chris Sherwood
Bortlesboat