The 401/403 branch added in #1172 threw a plain Error, and
RunDownloadJob is registered attempts: 10 with exponential backoff from
30s. A permanent "this build has no entitlement key" rejection therefore
retried nine more times over roughly 4h15m, during which the job reads
`delayed` rather than `failed` - so the user sees a stuck download
instead of the clear message #1172 was written to give them. Each retry
also re-hits our own rate-limited Worker.
Throw a named GatedContentAuthError from the download util and translate
it to UnrecoverableError at the queue boundary in RunDownloadJob,
alongside the existing cancellation case. Declaring the class in
downloads.ts rather than throwing UnrecoverableError directly keeps
BullMQ out of a module that docker_service and map_service also use.
Harmless today because no catalog entry uses `auth`, but that ends with
the first gated card - see #1204.
Fixes#1195
doResumableDownload already implements resume - stats the .tmp, sends a
Range header, handles a server that ignores it. The code was unreachable
for ZIMs because every dispatch site passed forceNew: true, which skips
the partial-file check and opens the stream with 'w' instead of 'a', so
an interrupted 12.5 GB Wikipedia download truncated and restarted at
byte 0. Maps already do this correctly (map_service.ts:681, "so retries
resume partial downloads"); ZIMs never got the same treatment.
Worse in combination with attempts: 10 - every retry also restarted from
zero, so a flaky connection re-downloaded the whole file up to ten times.
Drop forceNew from the content-download dispatch sites (it already
defaults to false) and add the guard that enabling resume requires: a
.tmp larger than the file now on the server cannot be a prefix of it,
because openZIM re-publishes builds under the same name. Resuming would
request a range past the end, 416 on every attempt, and never delete the
.tmp - so the download could never recover on its own.
Verified on a test appliance: a 470 MB partial survived a container
restart and continued rather than truncating, and a planted oversized
.tmp was discarded with the file then downloading to its exact size.
Refs #1201
Assigning a collection before indexing lost it silently. The per-row
value still showed, but Manage Collections and the Search in dropdown
stayed empty, because getKnowledgeCollections() facets on the Qdrant
payload while only MySQL had been written.
Five gaps on one path:
updateFileCollection() sets the payload filtered on `source`, which
matches nothing before the file is indexed. It also only persisted to
kb_ingest_state `if (row)`, so a file with no row stored the value
nowhere at all and still returned "Moved to ...".
Six of the seven EmbedFileJob.dispatch sites never pass `collection`,
and none read the existing row, so Index dispatched a job with no
knowledge of the assignment. The ZIM branch of processAndEmbedFile then
dropped `collection` even when the job had one, so ZIM content could
never be tagged at embed time by any path. Batch continuations dropped
it too, which would have tagged only batch 1.
Resolve the effective collection once inside EmbedFileJob.handle rather
than at seven call sites, thread it through the ZIM path into the point
payload, carry it across batch continuations, and make the pre-index
assignment durable with getOrCreate.
Verified end to end on a test appliance: assigned a collection to an
unindexed ZIM, indexed it across multiple batch continuations, and all
6106 chunks carry the tag.
Two layout defects in the Knowledge Base modal, both found during
v1.34.0-rc.4 QA.
The collection combobox list was clipped to a single row's height on
every row of the table. The cause is not stacking order - StyledTable
gives each cell `truncate` (overflow:hidden) plus `relative`, so an
absolutely-positioned child cannot escape the cell box. No z-index can
win against a clip. The modal body is also an `overflow-y-auto`
scroller, so simply opting the cell out of `truncate` still left rows
lower down clipped by the scroller instead.
Render the list in a portal with fixed positioning instead, measured
from the input's rect and flipped above when it would run off the
bottom of the viewport. Reposition on scroll and resize (capture
listener, so any ancestor scroller counts). The click-outside handler
now also checks the portaled list, otherwise mousedown on an option
would close it before the click landed.
Separately, the modal was capped at max-w-4xl (896px) while its table
needs 906px, so the Delete button lost its right edge for everyone
regardless of monitor size. max-w-5xl gives the table room with none to
spare wasted.
Verified against a real build on a test appliance.
The banner shipped with two highlights while the 1.34 line delivered 21
features, so it undersold the release. Adds the three with the widest
user-facing surface: Score v2, NOMAD.md and Knowledge Base collections.
Also converts the two existing bullets from em dashes to spaced hyphens
to match project copy convention.
Kept to five bullets - this is a dashboard alert, not release notes.
A job id can outlive its payload hash. BullMQ still returns an entry for
it with empty `data`, so `normalize(job.data.filepath)` threw and took the
whole response with it.
`fetchJobsWithStates()` includes failed jobs, and failed jobs are retained
deliberately, so one orphan made GET /api/downloads/jobs throw on every
call, permanently. Content Explorer polls that endpoint every ~3s, so the
symptom was a 500 loop and a page that never loaded. It survived restarts
and only cleared by editing Redis by hand.
Drop entries with no usable payload at the source, and guard both
normalize() calls (the pmtiles-extract map had the same pattern). With no
payload there is nothing to render anyway.
Seen on NOMAD3 running rc.3: 93 occurrences in 15 minutes.
Closes#1190
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ZimService.registerLocalUpload() dispatched EmbedFileJob unconditionally,
gated only on whether Ollama was reachable. It never read
rag.defaultIngestPolicy, so a user who deliberately chose Manual still got
sideloaded ZIMs embedded into the knowledge base behind their back.
PR #919 fixed exactly this for the post-download dispatch path. The
local-upload path was missed.
Rather than inline a third copy of the Always/Manual conditional, this
reuses decideScanAction, the same helper the scanner uses. That also means
an existing browse_only or pending_decision row is now honored instead of
being overridden by the act of re-uploading the file, which the inline
version in run_download_job.ts does not do.
Unset policy is still treated as Always, so existing installs keep their
current behavior.
Reported by @just-jbc on #1119, which also proposed disabling ZIM
auto-discovery entirely. That larger change is not included here: turning
discovery off by default would mean a user who downloads Wikipedia through
the curated flow gets no AI answers from it and no explanation why.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The chat modal's pop-out button called window.open('/chat', '_blank').
/chat is served by the admin app itself, so this spawned a second browser
window for a same-origin internal route.
That breaks anyone running NOMAD as an installed web app or in kiosk mode:
clicking it leaves a stray window they then have to get back out of, which
is exactly the complaint in #1123.
Navigate with router.visit instead, and relabel the button from "Open in
New Tab" to "Open Full Chat" so it describes what now happens. IconMessage
isn't in the DynamicIcon registry (deliberately curated for tree-shaking),
so use the already-registered IconArrowRight.
Refs #1123
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds an optional `auth: 'nomad_app_key'` discriminator to curated manifest
resources so a curated collection tier can carry content we host ourselves,
gated to official release builds. Without this, only Creator Packs could use
the entitlement Worker; curated tier installs always downloaded
unauthenticated.
No behaviour change for any existing manifest entry: absent `auth` means
unauthenticated, exactly as today.
- `auth` declared on both the type and the VineJS validator. It has to be on
the validator or VineJS strips it silently on fetch, and the gated download
would then go out with no header and 401 for everyone. A dedicated spec
guards that regression.
- Gated resources are pinned to their manifest URL (resolveZimDownload skips
the catalog comparison) and excluded from catalog update checks, so a
resource-id collision cannot let a third-party mirror overwrite our content.
Consequence, commented rather than implied: gated content does not
auto-update; new versions ship via the manifest.
- 401/403 on a download now reports that an official build is required instead
of a raw axios status, which is what a fork build will hit.
- The pure `isGatedResource` predicate is deliberately split from the
env-reading header builder: importing `#start/env` into
zim_download_resolution triggers env validation at import time and breaks its
unit tests.
Reuses CREATOR_PACKS_APP_KEY rather than minting a second secret — the question
it answers ("is this an official build?") is identical for both content types.
Verified end to end on a test server: `auth` survives validation into the
cached spec, the Bearer header attaches to only the gated resource, the file
lands byte-exact with an installed_resources row and a Kiwix library entry, and
an entry with a gated URL but no `auth` field fails with the intended message.
No catalog entry is included here. Manifests are fetched live from `main`, so a
gated entry must not merge until this ships and is adopted — pre-`auth` builds
strip the field and 401.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The #1157 gate blocks leaderboard submission whenever ai.remoteOllamaUrl is
set, using bare truthiness with no exemption for addresses that point back at
this same box. Someone running Ollama natively on the host while NOMAD runs in
Docker is measuring THIS hardware, yet is permanently blocked from submitting
with no in-app indication of why.
That configuration is not exotic: it is how the AI assistant is expected to
work on macOS, so under the current gate a Mac could never submit at all.
Exempts host.docker.internal (and gateway.docker.internal) plus the loopback
forms. host.docker.internal is the meaningful one — from inside the admin
container it resolves to the host, and it is what a native host install uses.
A LAN address is deliberately NOT exempt. 192.168.1.50 is indistinguishable
from another machine on the same network, and wrongly exempting it would let a
genuinely remote GPU's throughput be attributed to this hardware. A false block
is recoverable by clearing the setting; a false pass silently corrupts the
board.
Unparseable values fall through to blocked rather than allowed, so a malformed
setting cannot be used to slip past the gate.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three changes that together let ARM hardware appear on the leaderboard honestly.
Shipping them separately would leave ARM half-supported either way: without the
image a Pi cannot submit at all, and without the architecture field it submits
but is indistinguishable from x86.
1. PIN THE OFFICIAL MULTI-ARCH SYSBENCH IMAGE
severalnines/sysbench publishes amd64 only, so ARM hosts could not run the
System Benchmark at all — not a graceful failure, the container simply cannot
execute. Apple Silicon could only run it under Rosetta emulation, which distorts
the measurement it is taking, and that is what drove a community macOS fork to
substitute a different benchmark and submit incomparable numbers.
Swaps to ghcr.io/crosstalk-solutions/nomad-sysbench (Debian 12 + sysbench
1.0.20+ds-5, built for linux/amd64 + linux/arm64). One digest covers both
architectures; verified that pulling the pinned manifest-list digest resolves to
arm64 on a Raspberry Pi 5 and amd64 on x86, and that RepoDigests reports the
same manifest-list digest on both — so a single allowlist entry serves both.
No rescoring: 1.0.17 -> 1.0.20 measured 1.25% apart on identical hardware with
identical flags (7170.18 vs 7259.56 events/sec), inside run-to-run noise and
~0.3% on a composite. Both digests are allowlisted server-side, so the fleet can
cross over gradually.
2. REPORT THE DIGEST ACTUALLY RESOLVED
The submission previously sent SYSBENCH_DIGEST, the constant the client was
compiled with. The leaderboard validates that field, but a constant attests to
how a client was BUILT rather than what it RAN, so any build inherits a valid
value simply by carrying the same source.
Now reads it back from the image. Uses RepoDigests (the manifest digest we
pulled by), never Id — Id is the config digest, differs per architecture, and
would never match the allowlist. Falls back to the constant if inspection yields
nothing usable, so a benchmark never fails over provenance metadata.
Still forgeable, and always will be with an open-source client. It moves the bar
from "no effort" to "deliberate", which is the distinction that matters when
judging whether a submission is a mistake or a choice.
3. RECORD CPU ARCHITECTURE AND OS
The leaderboard is a single board across instruction sets by design, with
disclosure as the fairness mechanism. Without an architecture field an ARM result
sits unlabelled beside x86 — exactly what the disclosure exists to prevent.
All three fields come from the Docker daemon, reusing the docker.info() call
_detectRunEnvironment already makes. That is deliberate: inside the admin
container os.arch() and si.osInfo() describe the CONTAINER, not the host being
benchmarked.
cpu_architecture Architecture x86_64 -> amd64, aarch64 -> arm64
os_version OSVersion '24.04' (already structured, no parsing)
os_name OperatingSystem 'Ubuntu 24.04.4 LTS' minus the version
run_environment is kept rather than replaced: "which distro" and "is this
virtualised" are different questions, and WSL2 is a real performance factor.
String handling lives in app/utils/platform_metadata.ts with unit tests, matching
the amd_hsa_override convention, so it is testable without a Docker daemon.
Unknown architectures pass through verbatim rather than being guessed at, and
os_name falls back to the full description whenever the version is missing or
absent from it — an over-long name is harmless, a wrong one is not.
Columns are nullable and the submission fields optional, so results recorded
before this shipped remain submittable.
Closes#1156
Refs #1151
DockerService.getServiceURL() resolves ai.remoteOllamaUrl ahead of the local
container, so when a remote AI host is configured the AI channel measures THAT
machine while every other channel measures this one. The submission then reports
someone else's tok/s under this hardware's CPU, RAM and disk.
Under v2 this matters more than it did under v1: ai_tokens_per_second carries
0.30 of the weight and the score is uncapped, so a remote GPU's throughput is no
longer limited by a clamp.
Adds a guard alongside the existing submit-time checks. Uses the same truthiness
predicate as getServiceURL, so the guard fires exactly when the remote routing
it is guarding against would occur. KVStore.clearValue() nulls the value and
getValue() returns null for that, so a cleared key correctly does not trip it.
Blocks submission only. Running the benchmark locally is still useful to the
operator — it just isn't a result about this box, so it shouldn't go on a board
that ranks hardware.
Known limitation: the check reads the KV at submit time, not at measurement
time, so benchmarking with a remote host and then clearing the setting before
submitting would still get through. That is a deliberate workaround rather than
an accident, and closing it properly needs a column on benchmark_results to
record how inference was reached. Worth doing if it ever shows up in practice;
not worth a migration on the evidence available.
Refs #1151
The modern-rogue pack entry landed in collections/creator-packs.json
(#1145/#1146) but its banner image was never committed, so the app fell
back to /creator-packs/modern-rogue.webp which 404s and the card rendered
without a header.
Adds the 1060x175 banner (RGB webp), matching the existing
project-nomad.webp and crosstalk-solutions.webp, generated from the
pack's source banner.png built on the pack-build workspace.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MeshCore Web app doc listed meshcore.co.uk as the official site. The
canonical MeshCore project (github.com/meshcore-dev/MeshCore, the firmware +
protocol) declares its homepage as meshcore.io, so update the two references
in the MeshCore Web section to point there.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validated Project NOMAD on Ubuntu 26.04 LTS across two builds (an AMD/CPU
box and an NVIDIA RTX 5060 box), so recommend 26.04 as the default going
forward. 24.04 LTS and Debian 12 remain supported.
- bug_report.yml: add Ubuntu 26.04 (Resolute Raccoon) to the OS dropdown
- README / CONTRIBUTING / FAQ: name Ubuntu 26.04 LTS as the recommended version
- getting-started.md: correct the GPU note. NOMAD's installer sets up the
NVIDIA Container Toolkit and Docker runtime automatically; the user only
needs the NVIDIA driver (via "Install third-party drivers" at OS setup)
The AI benchmark evicted resident models (forcing the benchmark model cold)
and then went straight into the timed median-of-N loop with no warm-up. On a
cold box the model-load + GPU spin-up cost landed inside the timed runs
(observed: 173s TTFT / 5.83 tok/s vs a ~80 tok/s warm steady-state), and
consecutive runs weren't isolated (a prior run left the model warm). Because
the AI channel is uncapped and ~30% of the composite, the same machine could
post a ~2x-different NOMAD Score depending on warm/cold state (888 vs 1958
observed back-to-back).
Add one discarded warm-up inference after eviction and before the timed loop
so every timed run measures warm, steady-state throughput. Cold and warm
invocations now converge on the same score. Best-effort: a warm-up hiccup
never fails the run.
Closes#1139
The submit flow discarded the real failure reason and always returned the
generic "Failed to submit benchmark results." to the UI. The most common
cause is the leaderboard's one-per-hour rate limit (HTTP 429), which left
users with no idea why their submission failed or that retrying shortly
would also fail.
- benchmark_controller: return a clear, actionable message. Name the rate
limit explicitly on 429; otherwise pass through the underlying detail
(repository error or a service validation message like "already
submitted"), falling back to the generic only when we have nothing.
- benchmark_service: attach the raw upstream `detail` to the thrown error so
the controller can surface it.
The frontend already renders the server `error` string, so no client change
is needed.
* feat(drug-reference): compact header + single dashboard tile
Phase 1 of the drug-reference redesign:
- AppLayout gains an opt-in `compact` prop (small inline logo+title) so tool
pages reclaim the ~230px the full branding block costs; drug-reference/index
opts in.
- Consolidate the two dashboard tiles (Drug Reference + When to use what) into a
single Drug Reference tile with a broadened description (/conditions already
redirects to /drug-reference).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(drug-reference): tabbed redesign — search-by-drug + by-situation + FDA data
- Split /drug-reference into three tabs (Headless UI TabGroup): 'Search by
drug', 'By situation', 'FDA data'. Each tab runs only its own direction,
which removes the two-overlapping-sections confusion.
- Search by drug: results grouped by active ingredient (IngredientGroup,
single-ingredient groups first, combos after), drug-first result rows, and a
collapsible de-jargoned filter drawer (Over-the-counter / Prescription, Form,
Sort) that auto-collapses once results land.
- By situation: multi-select symptom chips → an 'Treats all N selected'
intersection section pinned on top (computed client-side from each
situation's result set) + one union group per situation.
- FDA data: the download/ingest control + status moved behind its own tab
(the pre-ingest empty state stays the prominent download prompt).
- DrugResultRow now leads with the active ingredient (drug-first) by default,
or the brand when rendered inside an ingredient group.
- Rename 'Compare interactions' → 'Compare label warnings' to match what it does.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(drug-reference): rank situation matches single-ingredient-first
The raw FDA indication match floods a situation (e.g. Headache) with
many-ingredient homeopathic products, burying real OTC drugs and leaving the
cross-situation intersection empty. Pull a wider result set (200) and sort by
active-ingredient count ascending — not a medical judgement, the same
'single-ingredient first' principle as the drug-search grouping. Now real
drugs (acetaminophen, ibuprofen) surface on top and the 'Treats all N selected'
intersection actually finds the shared OTC options. Per-situation cards cap the
display to the top 25 (ranked), intersection uses the full set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(drug-reference): first-open disclaimer gate + first-group-expanded
- Add a required disclaimer modal on first open of the Drug Reference (Jake's
mechanism): comprehensive not-medical-advice notice the user must acknowledge;
acceptance is saved to the browser's localStorage (versioned key) so it isn't
shown again on that browser, while new browsers/devices see it on first open.
Non-dismissible (no backdrop/Escape) — only the acknowledge button closes it.
- Search-by-drug: expand the first ingredient group by default, collapse all
subsequent groups (IngredientGroup gains an explicit defaultOpen prop,
replacing the size heuristic).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(drug-reference): intersection-first multi-select
Multiple situations now lead with the 'Treats all N selected' intersection and
only break out per-situation sections when the intersection is empty (nothing
treats all) — with a 'No single option treats all N of these' explainer. Keeps
the view combined when there's a shared answer, and only fragments as a fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(drug-reference): compact header on detail/interactions/conditions pages
Apply the compact AppLayout header to the drug detail, interactions, and
condition pages so they match the redesigned index instead of the full-height
branding block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(drug-reference): rename interactions heading to 'Compare label warnings'
Match the page heading + title to the button label, so the name reflects what
the view does (each drug's own FDA-labeled warnings side by side, not a
cross-drug interaction checker).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_ensureCollection() runs once per document on the embed path, but only
createCollection sat behind the collectionExists guard — the
getCollections probe and the three createPayloadIndex calls fired
unconditionally every time. On large ZIM ingestions those redundant
requests consumed roughly 45% of per-document Qdrant time, making jobs
look stalled while they were slowly progressing.
Memoize ensured collections in a per-instance Set, recorded only after
every step succeeds so partial failures retry. The cache is cleared
when the Qdrant health check resets the client (server may have been
recreated), and the entry is dropped before resetAndRebuild()
recreates the collection it just deleted.
Memoizing instead of moving the index calls inside the guard keeps
missing indexes healing on collections that predate the current
payload schema.
Closes#1129
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
PR #1076 stopped forcing HSA_OVERRIDE_GFX_VERSION=11.0.0 on
"natively-supported" AMD iGPUs. That was correct for gfx1150/gfx1151
(Strix 890M / Strix Halo, which are in the bundled rocblas allowlist)
but wrong for gfx1103 (Phoenix/Hawk Point 780M/760M), which is NOT in
that list. Without the override, ollama drops the 780M with "no rocblas
support for gfx target" and falls back to CPU on a fresh AI provision.
Extract the gfx→HSA mapping into a pure, unit-tested util and map
gfx1103 → 11.0.0 (gfx1100 kernels), the value that worked on v1.33.0 and
that restores full GPU offload in the field. gfx1150/1151 stay native.
Also harden the installer's 780M detection (Hawk Point / "Radeon 780M/
760M" strings) so the gfx marker isn't silently deleted, and upgrade the
no-marker fallback log from info to warn since it can mask CPU fallback.
Generate app/data/{conditions,natural_remedies,home_remedies}.ts from the
repo-root collections/*.json via `npm run gen:curated-data`, so the JSON is the
only file edited by hand. The generated modules keep the data compiled into the
image (no runtime file read, no path fragility, which is why the data was a TS
constant), and curated_data_sync.standalone.ts fails CI if a generated module
ever drifts from its JSON.
This removes the burden of hand-keeping the .json mirror and the .ts constant in
sync. Follow-up to the review discussion on #1040.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): offline FDA drug labels, conditions, and remedies
Adds an offline medical-reference feature with three coupled layers:
- Drug Reference: full-text search over openFDA drug-label indications,
a detail page per label, and a side-by-side single-drug comparison view.
A two-phase background pipeline downloads the openFDA label parts to the
storage volume (resumable) and ingests them into the search table.
- Conditions ("When to use what"): a curated spine of first-aid situations
that maps each situation to matching OTC drugs, each linking back to its
Drug Reference detail page.
- Curated remedies: hand-authored natural and home-remedy entries drawn from
US-government public-domain sources (NCCIH, CDC, MedlinePlus, FDA), shown
with their source links and the same safety disclaimers as the rest of the
feature.
The drug-reference and conditions layers are intentionally coupled: the drug
detail page shows the situations a drug treats, and the conditions controller
reads the same drug_labels table.
Safety surfaces ship as written. The amber SafetyBanner ("informational only,
not medical advice, not an FDA endorsement, not a drug-interaction checker, in
an emergency call emergency services") renders on the condition pages and the
search page; the detail and comparison pages carry their own "not a cross-drug
interaction checker" callout; and every page carries the openFDA CC0 source
citation and no-FDA-affiliation footer.
Wiring on this branch:
- start/routes.ts: the /drug-reference and /conditions page GETs plus their
/api/* groups.
- commands/queue/work.ts: the drug-download and drug-ingest queues, both at
concurrency 1. The two drug queues get a per-queue stall override
(lockDuration 1_800_000, maxStalledCount 3) because each part is one long
stream; every other queue keeps the existing 300000 default.
- inertia/pages/home.tsx: Drug Reference and "When to use what" tiles. The
icon and display_order are a starting point, open to change.
- types/kv_store.ts: the two drugReference.* keys the pipeline reads and writes.
- package.json: yauzl and stream-json (plus their @types), used by the ingest
job to stream the label JSON out of the downloaded zips.
The app reads the conditions and remedy data from the compiled TS constants in
app/data/; the repo-root collections/*.json files are the browseable mirrors.
The natural-remedies standalone test reads collections/natural_remedies.json to
assert the two stay in sync, so that file is also a test fixture.
The four standalone tests pass (drug_interactions, drug_ingest_status,
conditions, natural_remedies). tsc reports no errors in the feature code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(drug-reference): drop a fork-internal comment reference
* fix(drug-reference): address review — defer FULLTEXT, fallback remedies prop, strip fork refs
- migration: wrap the FULLTEXT ALTER in this.defer so it runs after the
deferred createTable (was silently swallowed, index never created)
- controller: add remedies:[] to the index() error fallback (required prop)
- strip fork-internal issue/spec references from ported comments
- tsconfig: exclude tests/standalone (node --experimental-strip-types only)
- correct the varchar(768) byte-math comment; extend remedy-spine test
* fix(drug-reference): make the interaction comparison readable at five drugs
The comparison view laid its columns out on an equal-fraction CSS grid
(repeat(N, minmax(0, 1fr))), so each added drug shrank every column; at the
five-drug maximum the FDA interaction text was squeezed into unreadable slivers.
Lay the columns out with flex instead: full-width and stacked on phones, then
fixed-width columns that scroll sideways from the sm: breakpoint up, so they
never shrink below a readable width. Theme the columns with the same palette as
the rest of the page (they were on stock gray), and give the headers a fixed
min-height so columns line up when drug names wrap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(collections): route a 'dataset' tier resource to the drug pipeline
Add an optional `type` discriminator to SpecResource ('zim' | 'dataset',
absent == 'zim'), so the tier installer can carry a DB-ingested resource
alongside ZIM files. ZimService.downloadCategoryTier branches on it: a
'dataset' resource dispatches the existing FDA download+ingest pipeline
instead of RunDownloadJob, guarded against duplicate dispatch by the drug
ingest status. Every existing manifest entry has no `type` and keeps the
exact ZIM path.
Widen InstalledResource.resource_type to include 'dataset' and exclude
dataset rows from the ZIM/map catalog-update scan (datasets aren't
filename-versioned; their freshness path is separate). No dataset rows are
written yet: the InstalledResource 'dataset' row on ingest-ready, the
manifest entry, install-gating, and the downloads-aggregator integration
are follow-up commits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): rework into an opt-in medicine-standard tier dataset
Reshapes the offline FDA drug reference from a built-in feature into curated
content installed by selecting the Medicine / Standard tier, per maintainer
direction.
- install-state: the ingest writes an installed_resources 'dataset' row on
ready (version = the openFDA export_date), threaded installer to download to
ingest; the tier-status math and the home-tile gate read it. Manual ingests
write no row, so install-state stays tied to the curated path.
- manifest: declare the dataset in the medicine-standard tier (runtime fetches
the remote manifest, so this also needs to land upstream).
- install-gating: the drug-reference home tiles render only when installed.
- uninstall: DrugReferenceService.uninstall() stops the two drug queues, deletes
the on-disk parts, truncates drug_labels (schema kept), clears the KV markers,
and drops the install row. Best-effort, logged, scoped to drug data only.
- downloads: the download phase reports the canonical {percent, downloadedBytes,
totalBytes} shape as one drug-data card in the Active Downloads aggregator with
cancel/remove; the heavy ingest stays in the IngestStatus surface with an
Indexing handoff on the card.
- auto-update: a daily DrugAutoUpdateJob compares the manifest export_date and
re-downloads when newer, gated on installed + no active job.
typecheck clean; the drug standalone suites pass. Three points are flagged in
code for the maintainer: the InstalledResource 'dataset' approach, the tier
home, and the export_date string format.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(downloads): keep the drug download card live across all parts
The Active Downloads card filtered to the deterministic jobId, but the
download's continuations run under auto-generated jobIds (only part 0 uses the
deterministic one). So the card tracked part 0 and then vanished while parts
2..N kept downloading. The queue is concurrency 1, so collapse to whichever
single part is in flight and report the deterministic jobId: one card tracks
aggregate progress through the whole download and cancel/remove still routes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): co-locate a persistent safety note with affirmative remedy guidance
Add RemedySafetyNote at the head of every natural-remedy section — the two on
Drug Reference and the one on "When to use what" — so the "informational only,
not medical advice, seek real medical care in an emergency" framing appears with
the guidance itself, not only in the page-top banner. Replaces the terse
per-section caveat with the same amber alert language as SafetyBanner.
Addresses the upstream #1040 review request that the disclaimer be unmistakable
and present wherever affirmative self-care guidance appears, not a one-time banner.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(drug-reference): fold dataset freshness into the content-auto-update path
The drug dataset auto-updated on its own daily cron that ignored the
contentAutoUpdate.* master switch, so it would refresh even with content
auto-update turned off, and it didn't ride the content-update path the way
ZIMs and maps do.
Move the export_date freshness/apply orchestration onto
DrugReferenceService.attemptAutoUpdate(), add
ContentAutoUpdateService.attemptDrugDataset() gated on the same enabled +
window config, and have the hourly ContentAutoUpdateJob drive both. Retire
the standalone DrugAutoUpdateJob. The ZIM/map attempt() path is unchanged.
Addresses the upstream #1040 request to wire the openFDA export_date check
into the content updater so the dataset updates alongside ZIMs and maps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): strengthen the remedy note to consult a clinician before combining with meds
Widen the affirmative-remedy safety note from "talk to a clinician before use"
to explicitly cover using a remedy AND combining one with a medication the user
already takes — the interaction case is the higher-risk path for an off-grid
user self-treating.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(drug-reference): gate affirmative remedy content behind an off-by-default flag
Add drugReference.remediesEnabled (default off), independent of the tier
install. When off, the server emits no remedy data at any boundary — the
drug-reference page prop, conditions show, and the /api/conditions/drugs
situation search — and the "Natural" filter is hidden, so installing the
medicine-standard tier lights up the verbatim FDA label search and the
condition-to-OTC matching but not the hand-authored self-care and herbal
sections. No user-facing toggle: it is flipped on after a clinician content-pass.
Implements the upstream #1040 split-by-risk request: the regulated label content
ships with the tier; the authored remedy guidance stays gated until sign-off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Jake Turner <52841588+jakeaturner@users.noreply.github.com>
Replaces the stale 40.5 placeholder (a llama3.2:1b CPU-era number) with 13.2,
measured on the Reference Build (NOMAD6, 780M) under the v2 AI harness with
llama3.1:8b and iGPU acceleration (OLLAMA_IGPU_ENABLE — the provisioning fix in
PR #1074). This is the AI channel's reference for the uncapped v2 score.
Must ship together with #1074 in v1.34.0: the reference assumes iGPU-accelerated
AMD boxes, so shipping it without the iGPU fix would score AMD installs on CPU
numbers against a GPU bar. The exact value gets a final confirm against the
shipping Ollama config before GA.
Must stay byte-identical to the leaderboard's score_service.ts REFERENCE_SCORES_V2
(the server recomputes the score on submit); the matching leaderboard change is a
sibling PR. Changing the leaderboard reference has no effect on existing rows —
the v1->v2 backfill neutralizes the AI channel (ratio 1 regardless of the
reference), and there are no real v2 submissions yet.
Sibling to the Score v2 app client (#1094); stacked on it (same file/constant).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Shows a dismissible dashboard banner to users who submitted a benchmark under
v1 scoring but don't yet have a Score v2 result, nudging them to re-run so the
community leaderboard gets their v2 number.
- Show logic (benchmark_service.shouldShowRerunBanner): has a result with
submitted_to_repository=true AND no result with nomad_score_v2 AND the
dismiss KV isn't set. Self-clears two ways — dismiss sets the KV, and any v2
run gives a result a nomad_score_v2 so the condition flips off on its own.
- New GET /api/benchmark/rerun-banner + useBenchmarkRerunBanner hook (mirrors
useUpdateAvailable), rendered as a dismissible Alert on the dashboard with a
"Re-run benchmark" CTA to /settings/benchmark.
- New KV key benchmark.rerunBannerDismissed (boolean) in both KV_STORE_SCHEMA
and the SETTINGS_KEYS whitelist (the PATCH /system/settings validator enum);
dismiss writes it via the existing updateSetting endpoint + invalidates the
query.
Sibling to the Score v2 app client (#1094); stacked on it for the
nomad_score_v2 column. Browser-verified on the NOMAD3 dev env: show/dismiss/
reload-persist/self-clear-on-v2 all correct.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 4 of the NOMAD Score v2 effort (stacked on #1089). Captures the raw
channel values the leaderboard scores from, computes the uncapped v2 score
in-app (byte-matching the leaderboard's server-side recompute), submits the
v2 payload, and surfaces it in the benchmark UI.
- Capture raws: per-channel sysbench values + O_DIRECT disk (W4), single/multi
thread CPU + memory, W6 consistency companions, and run-environment metadata
(run_environment, storage_path_type, gpu_compute_detected, #1016).
- AI hardening: reference model llama3.1:8b, num_predict=256, VRAM eviction
before the run + unload own model after, pre-flight disk check (~6.5GB, only
when the model is uncached), median-of-3 run (W7).
- Score: _calculateNomadScoreV2 (uncapped, reference->1000) byte-matches the
leaderboard recompute. nomad_score_v2 stays null unless a full run with AI.
- Payload: submitToRepository sends the v2 raws + score alongside legacy v1.
- UI: v2 headline score, legacy v1 as secondary, raws/env in details.
- Migration: 13 nullable columns; double (not knex float(8,2)) so millions-scale
raws keep full precision and don't break the leaderboard byte-match.
Reference constant REFERENCE_SCORES_V2.ai_tokens_per_second is still the stale
40.5 placeholder (TODO in source); locking it to the measured 13.2 is a
follow-up gated on the AMD iGPU fix (#1074). Live prod v2 submit is separately
blocked by the leaderboard TTFT floor (tracked in the leaderboard follow-up).
Verified on NOMAD3 (RTX 5060) dev env: migration applies, raws recorded,
v2 score computed, backend + benchmark.tsx typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An install that never went online while visiting /maps never provisions
world.pmtiles (it is lazily extracted from the internet), leaving the map
blank/grey offline outside any downloaded regional extracts.
Detect the world basemap's presence on disk and surface it to the UI so:
- /maps shows a clear notice explaining the base map isn't downloaded and
how to fix it, instead of a silent grey screen.
- /settings/maps offers an explicit "Download Base Map" action (~15 MB) to
provision it deliberately while online.
Adds MapService.checkWorldBasemapExists() / provisionWorldBasemap(), a
POST /maps/setup-world-basemap endpoint, and threads worldBasemapExists
through the maps page props.
Refs #1030
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(AI): per-model thinking toggle with global default (off)
Stop forcing thinking on for every capable model. Adds a global default
(ai.autoThinking, ships OFF) and a per-model override in the chat window,
shown only for thinking-capable models and remembered client-side.
The /v1 (OpenAI-compat) endpoint ignores `think`; reasoning_effort is the
real lever. The controller resolves per-request preference -> global default
-> OFF (gated on capability), and the service maps that to reasoning_effort
('none' to suppress on a capable model, 'medium' for gpt-oss, unset to let a
capable model default thinking on). A new thinkingCapable flag keeps
non-Ollama backends from ever receiving reasoning_effort.
installed-models is enriched with a `thinking` flag (checkModelHasThinking,
now memoized) so the picker knows which models get the toggle.
Stacks on #1078 (reasoning-field read + client-disconnect abort).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(AI): thinking toggle as NOMAD Switch + info tooltip
Address review feedback on the per-model thinking control:
- Use the shared Switch component (matches AI Assistant settings) instead
of a raw checkbox.
- Label "Thinking:" with a colon to match the adjacent "Model:" label.
- Add an InfoTooltip explaining what thinking does and where the default
lives. Extend InfoTooltip with optional `position` ('top'|'bottom') and
`align` ('center'|'right') so it opens downward and expands leftward from
the right-edge header slot instead of being clipped/crushed against the
viewport edge. Defaults preserve existing (benchmark page) behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): backend rail for gated creator content packs
Phase 1 of Creator Packs: the app-side download rail for per-creator
video ZIMs served from the private R2 bucket behind the entitlement
Worker. No UI yet (Phase 2).
- Thread optional requestHeaders through the download util + job so the
ZIM fetch can carry `Authorization: Bearer <APP_KEY>` on both the HEAD
probe and the GET stream (Worker streams the ZIM directly, gated).
- Add resourceMetadata.skip_embedding and gate the RAG/EmbedFileJob
branch on it so video ZIMs are never sent to the knowledge base.
- New creator_packs ManifestType + creatorPacksSpecSchema (display
metadata only, no url) + CollectionManifestService wiring and
getCreatorPacksWithStatus() status join.
- collections/creator-packs.json catalog with the two seed packs.
- CreatorPackService.installPack: ensure Kiwix (auto-install if absent),
resolve the stable Worker URL, dispatch the gated download with
skip_embedding. Typed result codes.
- CreatorPacksController + GET/POST /api/creator-packs routes.
- CREATOR_PACKS_APP_KEY / CREATOR_PACKS_WORKER_BASE env (release-injected
key; app degrades to a clear not_configured error when unset).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): Phase 2 UI — Content Explorer, settings page, wizard step
Adds the three user-facing surfaces for Creator Packs on top of the Phase 1
backend rail. All surfaces HIDE entirely on builds without the release-injected
key (configured=false) so a fork never shows a broken install button.
- Backend: CreatorPackService.isConfigured() + `configured` flag on the
GET /api/creator-packs response; /settings/creator-packs route + controller
action.
- api client + useCreatorPacks hook (shared cached source of configured/packs/
downloads for every surface) + CreatorPackCard (status badges, art fallback,
update pill) + CreatorPacksSection (install-on-click grid + confirm modal with
the license note), reused by the Content Explorer block and settings page.
- Content Explorer: Creator Packs section before Additional Content.
- Settings: new /settings/creator-packs manage page + conditional nav item.
- Easy Setup wizard: new Creator Packs step after Content. Replaces the fragile
hardcoded AI-skip math with a computed activeSteps list so the two optional
steps (Creator Packs + AI) navigate correctly in every on/off combination;
selection state, storage projection, review summary, and finish-install wired.
Browser-tested on NOMAD3 across all surfaces (configured + fork paths), incl.
live install→download→cancel and AI-on/off wizard renumbering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): use branded pack banners on the cards
Replace the generic colored-block cards with the branded 1060x175 banners we
build into each pack ZIM, shown on both the settings page and the Easy Setup
wizard step (and the Content Explorer block, via the shared card).
- Bundle the seed-pack banners as local webp under admin/public/creator-packs/
so they render offline (no external image dependency, matching the existing
/rogue-support-banner.webp convention).
- CreatorPackCard is now banner-forward: the banner is the hero, with a compact
status/metadata footer (videos · size, Installed/Downloading/Selected/Install,
update pill). Falls back to a simple header if an image is ever missing.
- Prefer an optional catalog `banner_url` (future remote creators) over the
bundled-by-id path; add banner_url to the CreatorPack type + validator.
- Widen the card grids to 1–2 columns since banners are wide.
Browser-tested on NOMAD3: banners render on the settings page and wizard step;
selection/installed/downloading states and storage projection still correct.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): Phase 3 — uninstall, license, release key-injection
Completes the Creator Packs feature.
- Uninstall: CreatorPackService.uninstallPack reuses ZimService.delete (removes
the ZIM, drops it from the Kiwix library, clears the InstalledResource) behind
DELETE /api/creator-packs/:id. A trash control on installed cards appears only
on the settings "manage" surface (allowUninstall) — never on the Content
Explorer block or the wizard — with a danger confirm modal.
- License: draft "Project NOMAD Creator Pack License v1.0" at
collections/creator-pack-license.md (marked DRAFT — pending legal review;
personal-use, no-redistribution, official-channel-only, creators retain
copyright). The install modal links to it ("View license").
- Release key-injection: Dockerfile ARG/ENV CREATOR_PACKS_APP_KEY in the runtime
stage (empty default → source/CI builds ship unconfigured and hide the UI) +
build-primary-image.yml passes it from the CREATOR_PACKS_APP_KEY CI secret.
NOTE FOR JAKE: add that repo secret = the entitlement Worker's key.
- Update flow was already functional (installed-with-update cards → "Update
pack"); verified.
Browser-tested on NOMAD3: uninstall round-trip (file + DB row + kiwix library
cleaned, then reinstalled), settings-only uninstall control, install-modal
license link, and the "Update available" pill via a temporary catalog bump.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(downloads): clean up header merge logic
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: jakeaturner <jturner@cosmistack.com>
* fix(rag): add proper .docx text extraction via mammoth
.docx files were classified as plain text and routed through raw-text
extraction (extractTXTText). Since .docx is a ZIP archive containing XML,
this produced garbage content in the Knowledge Base — XML tags and binary
noise instead of the actual document text.
Adds a dedicated 'docx' file type (split out of the generic 'text' bucket
in determineFileType) and a mammoth-based extractor that parses the
document XML properly.
* fix: clean up package-lock.json diff
* chore(deps): pin mammoth version
---------
Co-authored-by: John Cortright <jcortright@zscaler.com>
Co-authored-by: jakeaturner <jturner@cosmistack.com>
* feat(rag): add subject/collection organization to knowledge base
- Add nullable collection field to KbIngestState, propagated through the
embed job, RAG service, and Qdrant point payloads (indexed for filtering)
- Add upload-time category selection and per-file collection reassignment
in the Knowledge Base modal, with a filterable Stored Files table
- Add a 'Search in' collection filter to the chat interface, threaded
through to searchSimilarDocuments as an optional Qdrant filter
- Fix .docx extraction: previously routed through raw-text extraction
(garbage output for a ZIP-based XML format); adds a proper mammoth-based
extractor and a dedicated 'docx' file-type case
* feat(rag): support dynamic KB collection creation, rename, and removal
Extends collection organization with a Manage Collections UI: collections
are created on the fly when a file is assigned to a new name, can be
renamed (bulk-updates every tagged file and Qdrant point), and can be
removed (reassigns tagged files back to Uncategorized rather than
deleting anything).
* fix(rag): use dynamic collections query in chat search filter
chat/index.tsx still imported the static KB_COLLECTIONS constant for its
'Search in' dropdown, inconsistent with KnowledgeBaseModal.tsx which already
uses the live getKnowledgeCollections() query. Renamed/added collections
via the new Manage Collections UI weren't reflected in the chat filter.
* feat(rag): broaden preset tags and add creatable collection combobox
Replaces the survival-specific preset list with general-purpose starter
tags (recipes, diy, health, technology, finance, travel, hobbies,
reference, survival, energy) so the Knowledge Base reads well for
home-lab/reference use, not just prepping.
Adds sanitizeCollectionName() (trim, lowercase, length cap) applied on
every write path server-side, and a dependency-free CollectionCombobox
component replacing the plain <select> + window.prompt pattern for
tagging — autocompletes against presets + tags already in use, with a
'+ Create' option for anything new.
* chore(rag): remove .docx fix from this branch, split into #1100
Per review feedback, the .docx extraction fix is unrelated to the
collections feature and can merge independently. Moved to a standalone
PR (Crosstalk-Solutions/project-nomad#1100) off dev.
* chore: remove unrelated diff noise (lockfile, comments, indentation)
---------
Co-authored-by: John Cortright <jcortright@zscaler.com>
cosmistack-bot
jakeaturner
chriscrosstalk
NgoQuocViet2001
Experimentos em Série
Jake Turner
caweis
Chris Sherwood
Roberto
Andrew Barnes
just-jbc