Each in-flight (or stuck) embedding job gets a colored health pill,
relative-activity timestamp, and chunk counter so users can tell at a
glance whether ingestion is making progress.
## Health states
- **🟢 Active** — last batch < 2 min ago
- **🟡 Slow** — last batch 2-5 min ago (CPU-paced multi-batch ingestion
lives here naturally; not always a problem)
- **🔴 Stalled** — last batch > 5 min ago (likely real problem)
- **⚪ Waiting** — queued, no batch started yet
- **🔴 Failed** — job recorded failed status
## What lands
- New backend util `kb_job_health.ts` with pure `computeJobHealth(input)`
decision function. Time-based thresholds (2 min / 5 min) inlined as
constants. 9 unit tests pin the boundaries.
- `EmbedJobWithProgress` gains `lastBatchAt`, `startedAt`, `chunks` —
already set by `EmbedFileJob.handle` on every batch transition, just
not previously surfaced through `listActiveJobs`.
- Frontend `kb_job_health_display.ts` maps each status to a Tailwind
dot color, label, and aria-label so backend and UI stay in sync.
- `ActiveEmbedJobs.tsx` renders the pill, "last activity Xs ago", and
chunk counter above each progress bar. Adds a manual Refresh button
and "Last updated Xs ago" line — the existing 2s/30s auto-poll
cadence in `useEmbedJobs` is left intact.
- Live tick at 5s keeps the relative timestamps current without
re-fetching from the API.
## Not in scope
- Per-card Cancel / Retry / Un-index — separate Phase 2 PR
- Conditional warnings A/B/C — separate Phase 2 PR
- Computing throughput rate (chunks/min) — needs ratio registry consumer
(Phase 2 follow-up); for now the pill answers the "is it stuck?"
question directly without a rate estimate.
Project NOMAD's bundled docs (`/app/docs/*.md` and `README.md`) each
embed as their own KB source — currently rendering as 12+ individual
rows that swamp user-uploaded content in the Stored Files table.
Collapse them into one informational row:
> Project NOMAD documentation · 12 files · Managed by NOMAD
The admin-docs row hides the Delete button (those files would be
re-embedded on the next sync anyway, so deleting is a footgun). User
uploads and ZIMs keep their existing per-row Delete UX.
Also adds deterministic sort: ZIMs → user uploads → admin docs → other,
alphabetical within each bucket. Pure frontend change — `/api/rag/files`
response shape unchanged.
Decision logic extracted to `kb_file_grouping.ts` with 9 unit tests
covering bucket classification, sort order, count noun pluralization,
and empty-input handling.
Closes#796.
The maps API has accepted and persisted `notes` on map markers since
PR #770, but the marker popup component still rendered name only and
ignored the field. Now the popup shows a notes block beneath the name
when it's populated, with whitespace preserved and long text wrapped.
Threaded `notes` through the read path:
- `api.listMapMarkers` / `api.createMapMarker` response types
- `MapMarker` interface in `useMapMarkers` and the data.map projection
- `MapComponent`'s selectedMarker popup
The create/update UI is unchanged — users still set notes via the API
or DB directly, matching the issue's stated scope. A marker entry with
empty/whitespace-only notes renders the same as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(content): add custom ZIM library sources with pre-seeded mirrors
Users reported slow download speeds from the default Kiwix CDN. This adds
the ability to browse and download ZIM files from alternative Kiwix mirrors
or self-hosted repositories, all through the GUI.
- Add "Custom Libraries" button next to "Browse the Kiwix Library"
- Source dropdown to switch between Default (Kiwix) and custom libraries
- Browsable directory structure with breadcrumb navigation
- 5 pre-seeded official Kiwix mirrors (US, DE, DK, UK, Global CDN)
- Built-in mirrors protected from deletion
- Downloads use existing pipeline (progress, cancel, Kiwix restart)
- Source selection persists across page loads via localStorage
- Scrollable directory browser (600px max) with sticky header
- SSRF protection on all custom library URLs
Closes#576
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(content): recognize Wikipedia downloads from mirror sources
When Wikipedia is downloaded via a custom mirror instead of the default
Kiwix server, the completion callback now matches by filename instead
of exact URL. This ensures the Wikipedia selector correctly shows
"Installed" status and triggers old-version cleanup regardless of
which mirror was used.
Also handles the case where no Wikipedia selection exists yet (file
downloaded before visiting the selector), creating the record
automatically.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ZIM): use cheerio for custom mirror directory parsing
* fix(ZIM): use URL constructor for more robust joining
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jake Turner <jturner@cosmistack.com>
* feat(maps): add regional map downloads via go-pmtiles extract
* address Copilot review feedback on PR #780
- auto-refresh preflight on selection/maxzoom change with 400ms debounce and
requestId stale-safety so the confirm button no longer requires a two-step
"Estimate Size" -> "Start Download" dance
- safeUpdateProgress helper replaces fire-and-forget updateProgress().catch()
pattern so cancelled-job errors (code -1) can't surface as unhandled rejections
- gate world basemap source on worldBasemapReady - when ensureWorldBasemap()
fails we already delete world.pmtiles, so emitting the source was producing
404s on every tile request
- verify go-pmtiles binary SHA256 at image build time; upstream doesn't ship a
checksums file so per-arch hashes are pinned as build args with a regenerate
note when bumping PMTILES_VERSION
Adds a check to RAG health to make sure nomad_qdrant is online, if not
then the user will be blocked from clicking any buttons in the KB modal
until they click the start qdrant button and let the container start
There is a new file qdrant_restart_policy_provider.ts, which tries to
ensure that the restart policy always exists for the nomad_qdrant
container even though the policy should have been there when the
container is created.
Add distance scale bar and user-placed location pins to the offline maps viewer.
- Scale bar (bottom-left) shows distance reference that updates with zoom level
- Click anywhere on map to place a named pin with color selection (6 colors)
- Collapsible "Saved Locations" panel lists all pins with fly-to navigation
- Full dark mode support for popups and panel via CSS overrides
- New `map_markers` table with future-proofed columns for routing (marker_type,
route_id, route_order, notes) to avoid a migration when routes are added later
- CRUD endpoints: GET/POST /api/maps/markers, PATCH/DELETE /api/maps/markers/:id
- VineJS validation on create/update
- MapMarker Lucid model
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(downloads): rich progress, friendly names, cancel, and live status
Redesign the Active Downloads UI with four improvements:
- Rich progress: BullMQ jobs now report downloadedBytes/totalBytes instead
of just a percentage, showing "2.3 GB / 5.1 GB" instead of "78% / 100%"
- Friendly names: dispatch title metadata from curated categories, Content
Explorer library, Wikipedia selector, and map collections
- Cancel button: Redis-based cross-process abort signal lets users cancel
active downloads with file cleanup. Confirmation step prevents accidents.
- Live status indicator: green pulsing dot with transfer speed for active
downloads, orange stall warning after 60s of no data, gray dot for queued
Backward compatible with in-flight jobs that have integer-only progress.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(downloads): fix cancel, dismiss, speed, and retry bugs
- Speed indicator: only set prevBytesRef on first observation to prevent
intermediate re-renders from inflating the calculated speed
- Cancel: throw UnrecoverableError on abort to prevent BullMQ retries
- Dismiss: remove stale BullMQ lock before job.remove() so cancelled
jobs can actually be dismissed
- Retry: add getActiveByUrl() helper that checks job state before
blocking re-download, auto-cleans terminal jobs
- Wikipedia: reset selection status to failed on cancel so the
"downloading" state doesn't persist
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(downloads): improve cancellation logic and surface true BullMQ job states
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jake Turner <jturner@cosmistack.com>
Backend returned { error: message } on 400 but frontend expected { message }.
catchInternal swallowed Axios errors and returned undefined, causing a
generic 'An internal error occurred' message instead of the real reason
(already installed, already in progress, not found).
- Fix 400 response shape to { success: false, message } in controller
- Replace catchInternal with direct error handling in installService,
affectService, and forceReinstallService API methods
- Extract error.response.data.message from Axios errors so callers
see the actual server message
O Content Manager chamava api.deleteZimFile() para deletar arquivos
ZIM, mas esse método nunca foi implementado na classe API, causando
"TypeError: deleteZimFile is not a function".
O backend (DELETE /api/zim/:filename → ZimController.delete) já
existia e funcionava corretamente — só faltava o método no client
frontend que faz a ponte.
Closes#372
Failed download jobs persist in BullMQ forever with no way to clear
them, leaving stale error notifications in Content Explorer and Easy
Setup. Adds a dismiss button (X) on failed download cards that removes
the job from the queue via a new DELETE endpoint.
- Backend: DELETE /api/downloads/jobs/:jobId endpoint
- Frontend: X button on failed download cards with immediate refresh
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a "Debug Info" link to the footer and settings sidebar that opens a
modal with non-sensitive system information (version, OS, hardware, GPU,
installed services, internet status, update availability). Users can copy
the formatted text and paste it into GitHub issues.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes the InstalledTier model and instead checks presence of files on-the-fly. Avoid broken state by handling on the server-side vs. marking as installed by client-side API call
Content Manager now shows Title and Summary columns from Kiwix metadata
instead of just raw filenames. Metadata is captured when files are
downloaded from Content Explorer and stored in a new zim_file_metadata
table. Existing files without metadata gracefully fall back to showing
the filename.
Changes:
- Add zim_file_metadata table and model for storing title, summary, author
- Update download flow to capture and store metadata from Kiwix library
- Update Content Manager UI to display Title and Summary columns
- Clean up metadata when ZIM files are deleted
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds a standalone Wikipedia selection section that appears prominently in both
the Easy Setup Wizard and Content Explorer. Features include:
- Six Wikipedia package options ranging from Quick Reference (313MB) to Complete
Wikipedia with Full Media (99.6GB)
- Card-based radio selection UI with clear size indicators
- Smart replacement: downloads new package before deleting old one
- Status tracking: shows Installed, Selected, or Downloading badges
- "No Wikipedia" option for users who want to skip or remove Wikipedia
Technical changes:
- New wikipedia_selections database table and model
- New /api/zim/wikipedia and /api/zim/wikipedia/select endpoints
- WikipediaSelector component with consistent styling
- Integration with existing download queue system
- Callback updates status to 'installed' on successful download
- Wikipedia removed from tiered category system to avoid duplication
UI improvements:
- Added section dividers and icons (AI Models, Wikipedia, Additional Content)
- Consistent spacing between major sections in Easy Setup Wizard
- Content Explorer gets matching Wikipedia section with submit button
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(benchmark): Require full benchmark with AI for community sharing
Only allow users to share benchmark results with the community leaderboard
when they have completed a full benchmark that includes AI performance data.
Frontend changes:
- Add AI Assistant installation check via service API query
- Show pre-flight warning when clicking Full Benchmark without AI installed
- Disable AI Only button when AI Assistant not installed
- Show "Partial Benchmark" info alert for non-shareable results
- Only display "Share with Community" for full benchmarks with AI data
- Add note about AI installation requirement with link to Apps page
Backend changes:
- Validate benchmark_type is 'full' before allowing submission
- Require ai_tokens_per_second > 0 for community submission
- Return clear error messages explaining requirements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(benchmark): UI improvements and GPU detection fix
- Fix GPU detection to properly identify AMD discrete GPUs
- Fix gauge colors (high scores now green, low scores red)
- Fix gauge centering (SVG size matches container)
- Add info tooltips for Tokens/sec and Time to First Token
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(benchmark): Extract iGPU from AMD APU CPU name as fallback
When systeminformation doesn't detect graphics controllers (common on
headless Linux), extract the integrated GPU name from AMD APU CPU model
strings like "AMD Ryzen AI 9 HX 370 w/ Radeon 890M".
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(benchmark): Add Builder Tag system for community leaderboard
- Add builder_tag column to benchmark_results table
- Create BuilderTagSelector component with word dropdowns + randomize
- Add 50 adjectives and 50 nouns for NOMAD-themed tags (e.g., Tactical-Llama-1234)
- Add anonymous sharing option checkbox
- Add builder tag display in Benchmark Details section
- Add Benchmark History section showing all past benchmarks
- Update submission API to accept anonymous flag
- Add /api/benchmark/builder-tag endpoint to update tags
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(benchmark): Add HMAC signing for leaderboard submissions
Sign benchmark submissions with HMAC-SHA256 to prevent casual API abuse.
Includes X-NOMAD-Timestamp and X-NOMAD-Signature headers.
Note: Since NOMAD is open source, a determined attacker could extract
the secret. This provides protection against casual abuse only.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
- Add installed_tiers table to persist user's tier selection per category
- Change tier selection behavior: clicking a tier now highlights it locally,
user must click "Submit" to confirm (previously clicked = immediate download)
- Remove "Recommended" badge and asterisk (*) from tier displays
- Highlight installed tier instead of recommended tier in CategoryCard
- Add "Click to choose" hint when no tier is installed
- Save installed tier when downloading from Content Explorer or Easy Setup
- Pass installed tier to modal as default selection
Database:
- New migration: create installed_tiers table (category_slug unique, tier_slug)
- New model: InstalledTier
Backend:
- ZimService.listCuratedCategories() now includes installedTierSlug
- New ZimService.saveInstalledTier() method
- New POST /api/zim/save-installed-tier endpoint
Frontend:
- TierSelectionModal: local selection state, "Close" → "Submit" button
- CategoryCard: highlight based on installedTierSlug, add "Click to choose"
- Content Explorer: save tier after download, refresh categories
- Easy Setup: save tiers on wizard completion
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Chris Sherwood
chriscrosstalk
0xGlitch
cuyua9
Ryanba
Henry Estela
Jake Turner
Tom Boucher
LuisMIguelFurlanettoSousa