Commit Graph

658 Commits

Author SHA1 Message Date
Chris Sherwood 36068c645e
fix(KB): stop ZIM ingestion progress freezing at 99% on multi-page archives
On ZIMs that pack one logical article as several sub-pages (e.g. iFixit),
iterByPath yields more entries passing our isArticleEntry() filter than
archive.articleCount reports. The inter-batch progress used nextOffset /
articleCount, so the numerator outran the denominator, the ratio overflowed past
100%, and the UI (which clamps at 99%) pinned the file at 99% for the entire
remaining tail, making it look hung.

Grow the denominator to max(articleCount, nextOffset + ZIM_BATCH_SIZE) once we
pass the reported article count, so progress keeps creeping forward monotonically
instead of freezing, and clamp to 99% so only the genuinely-final batch reports
100%.

This is a graceful heuristic, not exact progress (true accuracy would require a
pre-scan to count isArticleEntry matches up front); it removes the user-visible
"stuck at 99%" symptom with no change to batch semantics.

Closes #903

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:54 +00:00
Chris Sherwood 3977c723c2
fix(KB): stop partial_stall warning firing on atypical ZIMs (link-out/PDF-heavy)
The Stored Files "partial stall" warning compares chunks in Qdrant against an
expected count from the ratio registry. The registry has an empty-pattern
catch-all (100 chunks/MB) that matches any filename, so a ZIM that matches no
specific pattern still gets a size-based estimate. For archives that are mostly
PDFs, images, or link-out stubs (e.g. irp.fas.org military-medicine), byte size
wildly over-predicts embeddable text: a 75 MB ZIM estimates ~7,236 chunks but
produces ~1, tripping a false "ingestion may have stalled" warning that re-embed
can't clear.

The catch-all is fine for rough aggregate disk-cost estimates, but it should not
drive a per-file stall signal. Add an `ignoreCatchAll` option to the ratio
lookup that excludes the empty-pattern row (returning null when only the
fallback would match), and use it in the warnings path so partial_stall only
fires when the registry has a *specific* expectation for the file. Files that
match a real pattern (wikipedia_, devdocs_, ifixit_, ...) are unaffected;
disk-cost/batch estimates keep using the fallback.

Closes #913

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:54 +00:00
Chris Sherwood 25000b9869
fix(system): roll back service update when the new container fails to start
The service update path stopped and renamed the old container aside before
the new one was confirmed running, but only wired up rollback if
createContainer threw or the 5s health check failed. A throw from
newContainer.start() itself (bad device/GPU config, host port already bound,
image incompatibility) bubbled straight to the outer catch, which returned a
generic 400 and never restored the old container, leaving the service down.

Retrying then wedged: the failed new container still held the service name, so
the next attempt's rename to `<name>_old` collided with the leftover from the
first attempt and threw the same error every time.

- Wrap newContainer.start() so a start failure removes the half-created
  container and rolls back to the previous one.
- Clear any stale `<name>_old` before renaming so retries can't collide.
- Dedupe the three rollback sites into a single rollbackToOld() helper (also
  removes a non-null assertion in the create-failure path that could itself
  throw when no `_old` existed).

Refs #949

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:53 +00:00
jakeaturner dfc284c34d
docs: fix out of place JSDoc comment 2026-06-23 04:46:53 +00:00
Chris Sherwood 3f574e4003
fix(system): show a clear message when a service port is already in use
When a service fails to install because something on the host already binds its
port, the user saw the raw dockerode error ("Bind for 0.0.0.0:11434 failed:
port is already allocated"), which is meaningless to a non-technical user. The
most common case is a native Ollama install holding 11434.

Add _humanizeDockerError() to map host port-conflict errors to an actionable
message that names the port and, for Ollama/11434, points at the likely cause
(a host Ollama service) with the commands to stop it. Unrecognized errors pass
through unchanged. Wired into the install failure broadcast and the thrown
error.

Closes #934

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:52 +00:00
jakeaturner cf8db6218d
feat(supply-depot): opt-in automatic updates for installed apps 2026-06-23 04:46:52 +00:00
jakeaturner 02d985db5e
feat(system): add opt-in automatic updates for the core NOMAD app
Adds a self-update path for the NOMAD admin/core image that runs without a
human in the loop, gated by layered safety checks. Recreation remains the
sidecar's job; this only decides *whether* to update now and requests it.

An hourly job (AutoUpdateJob) evaluates a side-effect-free decision pipeline
(AutoUpdateService.evaluate):
 - opt-in: disabled by default
 - in a user-configured local-time window (handles midnight wrap)
 - an eligible release: same major (major bumps stay manual), strictly newer,
   past a configurable cool-off, GA only (no drafts/prereleases), strict semver
 - pre-flight: sidecar present, no in-flight system update / downloads / app
   installs, and sufficient host disk (estimated from the registry manifest)
When all pass, it drives SystemUpdateService.requestUpdate() with a vetted tag.

Failure backoff auto-disables after 3 genuine update-request failures; transient
release-lookup errors are skips (offline-first appliances are routinely without
connectivity). Re-enabling clears the backoff state.

Settings UI exposes the toggle, window, and cool-off, with live status (eligible
target, in/out of window, last result/error). A `node ace auto-update:dry-run`
command exercises the full pipeline — and a deterministic --scenarios suite the
pure decision logic — without ever triggering an update.

New KVStore keys under autoUpdate.* hold config + last-run state.
2026-06-23 04:46:51 +00:00
Metbcy d175259219
fix(KB): persist accumulated chunk count across batched ZIM dispatches
The continuation dispatch in EmbedFileJob did not pass the running chunk
count forward, so each batch started with job.data.chunks undefined.
On the final batch, totalChunks collapsed to just that batch's result
and KbIngestState.markIndexed stored a value far below what Qdrant
actually held.

Add chunksSoFar to EmbedFileJobParams and thread it through the
continuation chain so the final markIndexed call reflects the true
total across all batches.

Closes #933
2026-06-23 04:46:51 +00:00
John Onysko f553a0d57d
feat(config): respect REDIS_DB env var for queue and transmit
Allow operators to select a Redis logical database index via the
REDIS_DB environment variable. Without this, the BullMQ queue and the
@adonisjs/transmit Redis transport both implicitly used db 0, causing
key collisions when sharing a Redis instance across multiple services
or environments.

REDIS_DB is added to the env schema as an optional number; both
config/queue.ts and config/transmit.ts fall back to db 0 when unset,
preserving existing behavior.
2026-06-23 04:46:51 +00:00
gujishh e1c7435fc4
fix(install): harden toolkit and helper script downloads 2026-06-23 04:46:50 +00:00
Chris Sherwood ccc221f88d
chore: align package license with Apache-2.0 and fix README docs links/typos
Correct package metadata and documentation accuracy:

- Set license to Apache-2.0 in package.json, admin/package.json and both
  lockfiles (project has been Apache-2.0 since #197; metadata still said ISC)
- Replace the placeholder root package.json description with a real one
- Remove the dead README "Troubleshooting Guide" link (TROUBLESHOOTING.md
  does not exist; FAQ.md already has its own entry)
- Fix README typos: "harware", "LLM's", "of of", "uses cases"

Carries forward the worthwhile fixes from #849 by @aqilaziz, minus the
version bump that conflicted with the current release line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:50 +00:00
jakeaturner 057fb693f6
fix(storage): match default prefix too when relocating child-app binds
_applyHostStorageRoot only rewrote binds carrying the current
NOMAD_STORAGE_PATH and short-circuited when env == resolved root. That
left user-modified/custom apps — whose binds freeze the default prefix at
edit time — mounting an empty dir on the documented "set NOMAD_STORAGE_PATH
+ relocate the volume" path (#938).

- Match either the env value or the hardcoded default prefix; drop the
  root == seededRoot no-op, excluding root from the candidates instead so
  rewrites stay idempotent.
- Add ADMIN_STORAGE_DEST / DEFAULT_HOST_STORAGE_ROOT constants; replace the
  cwd-derived storage dest so admin-mount lookup can't silently break.
- Comment why a transient inspect failure is deliberately not cached.
2026-06-23 04:46:49 +00:00
Chris Sherwood fab15c68a9
fix(storage): derive child-app bind paths from the admin's actual storage mount
Child services (Kiwix, Ollama, Qdrant, Flatnotes, Kolibri) are created via the
Docker socket, so their bind mounts use a HOST path. That path was baked into
the services table at seed time from NOMAD_STORAGE_PATH (default
/opt/project-nomad/storage) — and NOMAD_STORAGE_PATH wasn't even present in
management_compose.yaml, so it always fell back to the default.

Result: relocating the admin storage volume in compose (e.g.
/mnt/big/storage:/app/storage) moved the admin's own data but left child apps
mounting the old, now-empty /opt/project-nomad/storage. Kiwix would come up
with no content. (#938)

- Add _resolveHostStorageRoot(): inspect the admin's own container, find the
  bind backing /app/storage, and use its host Source as the single source of
  truth (cached). Falls back to NOMAD_STORAGE_PATH/default if it can't be
  inspected.
- Add _applyHostStorageRoot(): rewrite the host-side prefix of each storage bind
  to that root. No-op when it already matches the seeded prefix, so default
  installs are unaffected.
- Apply it in _createContainer (covers installs + dependency recursion) and in
  the Kiwix library-mode recreate path.
- management_compose.yaml: add an explicit NOMAD_STORAGE_PATH knob and rewrite
  the comments so the admin volume, env var, and disk-collector volume that must
  agree are spelled out.

Closes #938

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:49 +00:00
jakeaturner b2bea191a8
fix: minor type guarding fixes after axios bump 2026-06-23 04:46:48 +00:00
jakeaturner 555ffa8790
fix(supply-depot): reintroduce app update UI 2026-06-23 04:46:48 +00:00
Chris Sherwood f488f08c96
feat(supply-depot): per-app onboarding docs, install fixes, and in-app Docs links
Add NOMAD-specific getting-started docs for all 9 curated Supply Depot apps,
the catalog/install fixes each one surfaced, and a way to reach the docs from
each app card.

Docs:
- New in-app Markdoc page admin/docs/supply-depot-apps.md covering all 9 apps
  (Stirling PDF, File Browser, Calibre-Web, IT Tools, Excalidraw, Homebox,
  Vaultwarden, Jellyfin, Meshtastic Web): first run/login, where data lives,
  and offline behaviour. Registered in docs_service DOC_ORDER.
- Manage > Docs dropdown item linking each app to its section
  (/docs/supply-depot-apps#<anchor>): anchor map in constants/supply_depot_docs.ts,
  heading anchors via Markdoc {% #id %}, and hash-scroll on the docs page.

Install / catalog fixes:
- Stirling PDF: open straight to the tools (SECURITY_ENABLELOGIN=false; the old
  v1 DOCKER_ENABLE_SECURITY flag was dead).
- File Browser: seed a known admin/nomad login (bcrypt) instead of a random
  log-only password; scope visibility to content folders via mount selection and
  move the DB out of the browsable root.
- Calibre-Web: bundle an empty Calibre library and seed it on install so setup
  doesn't dead-end at db config (_runPreinstallActions__CalibreWeb).
- Homebox: swap the archived hay-kot image for the maintained sysadminsmedia fork.
- Vaultwarden: generate a self-signed cert on install and serve HTTPS by default
  (_runPreinstallActions__Vaultwarden + ROCKET_TLS + ui_location https:8480), so
  the web vault has the secure context it requires.
- Jellyfin: pre-create storage/media/{Movies,TV Shows,Music,Photos} so each
  library points at its own subfolder, avoiding the overlapping-path issue that
  silently hides content (_runPreinstallActions__Jellyfin).
- Seeder run() now also syncs ui_location for non-modified curated services, so a
  catalog link/scheme/port change reaches existing installs on update.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:47 +00:00
Chris Sherwood b8674193da
fix(supply-depot): show clean port + lock on card pill for https:port ui_location 2026-06-23 04:46:47 +00:00
Chris Sherwood 3501144caa
feat(supply-depot): scheme-aware service links (https:port) for TLS-serving apps 2026-06-23 04:46:47 +00:00
Chris Sherwood 7b1b480c5e
fix(supply-depot): correct Meshtastic Web internal port (80->8080); add catalog port audit script 2026-06-23 04:46:46 +00:00
Chris Sherwood 02c33b277e
feat: edit curated apps + fix dropdown clip + stale _old rollback 2026-06-23 04:46:46 +00:00
Jake Turner be434d755a
feat: supply depot 2026-06-23 04:46:45 +00:00
Chris Sherwood e38dbf8a65
feat(zim): add "Rescan Library" button for sideloaded ZIM files
Adds a user-facing trigger to rebuild the Kiwix library index from the
ZIM files currently on disk. Covers the sideload case: a user copies a
.zim onto the box (USB, SSH, network share) outside NOMAD's download
flow, and Kiwix has no way to discover it without regenerating the
library index.

Reuses the existing native KiwixLibraryService.rebuildFromDisk() (which
also extracts embedded favicons natively), so no kiwix-tools container
and no icon-patch step are needed. In library mode (--monitorLibrary)
kiwix-serve hot-reloads the XML automatically; only legacy glob-mode
containers are restarted.

- KiwixLibraryService.rebuildFromDisk now returns the book count; adds
  getBookCount() for the before/after delta
- ZimService.rescanLibrary() orchestrates rebuild + legacy restart
- POST /api/zim/rescan-library + ZimController.rescanLibrary
- "Rescan Library" button on the Content Manager page (shown when Kiwix
  is installed) with a success toast reporting books found

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 04:46:45 +00:00
jakeaturner 7943ae5d1f
chore(deps): bump axios to 1.17.0 in admin 2026-06-23 04:46:44 +00:00
teccin 820c5c415e
Fix a few typos and punctuation in README.md 2026-06-23 04:46:44 +00:00
jakeaturner b8961d27dd
fix(rag): improve context-reliance hedging and use heading metadata at query time
Users often saw the assistant disclaim ("Sorry, I couldn't find specific
context regarding X, but here's a general answer...") even when material
that directly answered the query was embedded. Two compounding causes,
both on the read/generation side:

1. The rag_context system prompt explicitly authorized the hedge: it told
   the model to fall back to general knowledge and "acknowledge the
   limitations," and mandated "According to the information available..."
   citations that pushed small models into meta-commentary preambles.
   Rewrote it to treat the retrieved context as the primary, authoritative
   source, lead with the answer, fall back to general knowledge *silently*,
   and never emit "couldn't find specific context" phrasing, while still allowing
   the model itself to make some "sanity-check" decisions if the retrieved context
   seems wildly incorrect or unrelated to the user's query.

2. The injected context labeled each block with a raw relevance score
   (e.g. "Relevance: 42.3%"). nomic-embed cosine scores for genuinely
   relevant passages sit ~0.4-0.6, so the number primed the model to
   distrust correct context. Dropped the model-visible score (it stays in
   the logs) and replaced it with a neutral source-title label.

Also added a conservative, score-scaled heading boost in rerankResults:
when query keywords match a chunk's section/article title (ZIM metadata we
already fetch), nudge its rank. Same diminishing-returns shape as the
existing boosts and gated behind the existing semantic-quality threshold,
so it can't promote a weak match.

Scope note: this removes the visible hedge and improves ranking of
already-retrieved chunks. It does NOT fix retrieval recall — diluted
1500-token chunks that never reach dense search's top-k are unaffected.
That's a follow-up (smaller chunks + BM25/RRF hybrid).
2026-06-23 04:46:43 +00:00
Jake Turner 17630c048f
docs: update release notes 2026-06-23 04:46:43 +00:00
cosmistack-bot d5f7c3f615 docs(release): finalize v1.32.1 release notes [skip ci] 2026-05-27 22:36:20 +00:00
cosmistack-bot fa144c4777 chore(release): 1.32.1 [skip ci] 2026-05-27 22:36:12 +00:00
jakeaturner e26ce4f1f6 docs: update release notes 2026-05-27 15:32:28 -07:00
jakeaturner 8d2bf785e1 chore(deps): bump various deps 2026-05-27 15:32:28 -07:00
chriscrosstalk 97c65cca67 perf(KB): swap Qdrant full-scroll for facet on source enumeration (#928)
The Stored Files modal and per-file warnings panel both enumerate
distinct 'source' values in the qdrant content collection. The
existing implementation scrolls every point in the collection, 100 per
page, just to learn the unique sources. On a fully-ingested NOMAD this
is brutally slow: NOMAD3's 3.1M-point collection takes 50+ seconds to
return its ~40 distinct sources, and the same scan runs twice per modal
open (getStoredFiles + computeFileWarnings each pay it independently).

Qdrant 1.10+ supports a facet aggregation that returns the distinct
values of a payload field with their counts in a single call.
@qdrant/js-client-rest@1.16.2 (already pinned) exposes this as
`client.facet()`. Swap the three scroll loops:

- RagService.getStoredFiles
- RagService.computeFileWarnings
- RagService.\_scanAndSync (the source-set used for skip-already-embedded
  decisions)

As a bonus, computeFileWarnings's per-source chunk count comes back
inline from the facet, so the secondary scroll that was incrementing a
counter is also gone.

`exact: true` everywhere because the counts feed Warning A's
zero_chunks decision and could mis-fire near thresholds otherwise. New
RagService.FACET_SOURCE_LIMIT = 10000 caps the facet response; real
NOMADs run ~100 sources max, this is comfortable headroom.

Expected impact on NOMAD3 (3M points, 40 sources): ~50s -> ~100ms per
endpoint. On an empty/fresh KB: no measurable change either way.
2026-05-27 15:32:28 -07:00
chriscrosstalk 5bf5bc33b7 fix(KB): cursor on Always/Manual ingest policy buttons (#927)
Tailwind v4 dropped the default `button { cursor: pointer }` preflight
rule. The Always/Manual toggle in the KB modal uses inline <button>
elements (not the StyledButton wrapper that sets cursor-pointer
explicitly), so hovering them shows the default arrow cursor instead of
a pointer like every other clickable in the modal. Reported on NOMAD1
during v1.32.0 dogfooding.

Add cursor-pointer when not pending; keep cursor-not-allowed when the
mutation is in flight.
2026-05-27 15:32:28 -07:00
chriscrosstalk 737e6784f0 fix(logging): also write production logs to stdout for docker visibility (#870)
In production, the logger was configured with a single file target writing to
`/app/storage/logs/admin.log`. Because the production pino transport had no
stdout target, `docker logs nomad_admin` only saw startup banner lines from
non-pino sources — every `logger.info`/`logger.debug` call from controllers,
services, and providers was effectively invisible from outside the container.

That's been silently masking diagnostics for anyone trying to debug a running
NOMAD without exec-ing into the admin container and tailing the log file.
RAG retrieval scores, query rewrites, container preflight decisions, version
check results — all of it was there in `admin.log` but absent from any
external observation point (docker logs, container log aggregators, etc.).

This adds a second production target writing JSON to stdout (fd 1) via the
same pino/file transport. Effect: `docker logs nomad_admin` now shows the
full runtime telemetry, the persisted log file is unchanged (so Debug Info
bundle export keeps working), and external log aggregators that scrape
container stdout now have something to scrape.

Verified on NOMAD8 (v1.32.0-rc.3): post-patch `docker logs --tail 15
nomad_admin` shows the structured `{"level":30,...,"msg":"[VersionCheckProvider]
Checking for stale updateAvailable..."}` lines that previously only existed
in admin.log. File destination still receives writes (size delta confirmed
after an `/api/system/info` hit).

This is the unblock for the AI Quality eval work — without log visibility,
every diagnostic conversation about RAG retrieval and query rewriting is
guesswork.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 15:32:28 -07:00
cosmistack-bot a96e0d682f chore(release): 1.32.0 [skip ci] 2026-05-20 17:20:32 +00:00
jakeaturner d2dd8b4f9e docs: update release notes 2026-05-20 10:16:00 -07:00
jakeaturner 95bd05ed78 chore(deps): pin ipaddr.js version 2026-05-20 10:16:00 -07:00
jakeaturner 736c9bd672 fix(security): canonicalize hostnames to block IPv4-mapped IPv6 IMDS bypass
Replace literal string matching with ipaddr.js parsing
so equivalent encodings of 169.254.169.254
(::ffff:169.254.169.254, ::ffff:a9fe:a9fe,fully-expanded forms)
and fd00:ec2::254 are all rejected.
2026-05-20 10:16:00 -07:00
jakeaturner b3dac9b324 fix(security): match IPv6 SSRF patterns against unbracketed hostnames 2026-05-20 10:16:00 -07:00
jakeaturner 989a401f28 fix(AI): improve remote Ollama url validation to prevent SSRF vulnerability 2026-05-20 10:16:00 -07:00
Jake Turner 82f67debc1 fix(models): correct inverted belongsTo keys on ChatMessage.session (#921)
foreignKey/localKey were swapped on the ChatMessage → ChatSession
relation. Per Lucid's belongsTo contract, foreignKey is the column
on the child model and localKey is on the parent — so this must be
{ foreignKey: 'session_id', localKey: 'id' }, mirroring the inverse
hasMany on ChatSession.

The relation is not currently preloaded anywhere in the codebase, so
no runtime behavior changes today; this closes a latent bug that
would have broken any future preload('session') call.
2026-05-20 10:16:00 -07:00
Chris Sherwood a5fe52f66f fix(KB): respect Manual ingest policy on post-download dispatch
RunDownloadJob's onComplete handler was unconditionally firing
EmbedFileJob.dispatch after every ZIM download, gated only by "is
Ollama installed?". The rag.defaultIngestPolicy KV setting was never
consulted, so users who explicitly set Auto-index to Manual still got
every newly-downloaded ZIM auto-embedded.

RagService.scanAndSync already handles Manual correctly by recording
pending_decision rows instead of dispatching (rag_service.ts:1587-1638
via decideScanAction). The post-download path skipped that gate.

Mirror the same check at the dispatch site: read the policy KV; if
Manual, firstOrCreate a pending_decision row in kb_ingest_state so the
per-file Index affordance from PR #909 surfaces the file the same way
scan-time-discovered Manual files do. firstOrCreate (not create) so a
re-download doesn't demote an existing indexed/failed row — the user
can explicitly re-index from the KB panel if they want fresh content.

Verified on NOMAD3: with rag.defaultIngestPolicy='Manual', every ZIM
downloaded today via Content Explorer (agriculture-essential +
computing-essential, ~62 MB across 7 files) wrote kb_ingest_state
rows with state='indexed' instead of pending_decision. Real bug,
not a hot-patch artifact.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:16:00 -07:00
Chris Sherwood 059cf2afbe fix(content): show selected tier on cards while downloads are in flight
Since PR #36b6d8e moved tier-installation tracking from a client-side
persistence model to a server-side derive-from-disk model, the card
display only ever updates once every file in a tier is fully on disk.
A user who picks Standard sees a blank card for the duration of the
download (often hours for large tiers like Wikibooks). Worse, if some
files finish before others, the card briefly shows a lower tier (e.g.
Essential) before promoting to the selected tier on completion, which
reads as "the system didn't accept my pick."

Backend: compute a sibling `downloadingTierSlug` by unioning installed
resource IDs with the IDs from active RunDownloadJob queue entries
(waiting + active + delayed, failed deliberately excluded), then
resolving the highest tier whose every resource is in that union. Set
only when it differs from `installedTierSlug` — no point reporting
"downloading Standard" when Standard is already fully installed.

Frontend: unify the prominent corner badge logic in CategoryCard to a
single `badgeTier` derived from selectedTier > downloadingTier >
installedTier. Spinner + "(downloading)" suffix when in flight,
checkmark for installed/selected. The pill row and lime border follow
the same source.

Verified on NOMAD3: backend correctly resolves the downloading tier
from in-flight BullMQ jobs; CategoryCard shows the spinner badge
immediately on Submit and switches to the checkmark variant when
downloads complete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:16:00 -07:00
Chris Sherwood 6e5284e563 fix(KB): TierSelectionModal hook order + register IconLibrary
Two related fixes surfaced by armandoescalante in #915 when clicking a
Content Explorer category card (e.g. Medicine) on v1.32.0-rc.6:

1. TierSelectionModal placed a useMemo for freeBytes *after* the
   `if (!category) return null` early return (introduced in PR #901's
   guardrail integration). When `category` transitioned from null to
   non-null on first open, React saw a different hook count between
   renders and crashed the entire component tree with "Rendered more
   hooks than during the previous render", blanking the modal. Moved
   the freeBytes useMemo above the early return so hook order is
   constant.

2. `IconLibrary` was used as the icon prop on the Manage Custom
   Libraries button in remote-explorer.tsx but never registered in
   the DynamicIcon allowlist at admin/inertia/lib/icons.ts. Added it
   to both the import block and the icons map so the warning stops
   firing and the icon renders.

Closes #915.
2026-05-20 10:16:00 -07:00
jakeaturner a9c48fc098 refactor(AI): single source of truth for embedding model name
Lift the hardcoded 'nomic-embed-text:v1.5' string out of both
RagService and OllamaService into a shared EMBEDDING_MODEL_NAME
constant in constants/ollama.ts. The duplicate in OllamaService
existed only to dodge a circular import with RagService; the
constants module has no service imports, so a shared constant
eliminates both the duplication and the drift risk called out
in the inline "keep in sync" comment.
2026-05-20 10:16:00 -07:00
Chris Sherwood ffa70a54bc feat(chat): confirm-on-switch + one-chat-model-at-a-time enforcement
Surfaces NOMAD's previously-silent model-stacking behavior and enforces a
"one chat model in VRAM at a time" invariant (the embedding model is
always exempt). Addresses Chris's NOMAD3 testing observation that
switching the dropdown in the chat header was invisibly slow on low-VRAM
hardware because the prior model was never unloaded — Ollama would
either evict it under memory pressure or load the new one on CPU after
the runner choked.

Three integration points all funnel through one new helper:

- **User changes the model dropdown** in an active chat session →
  confirm modal "Switch to {newModel}? Switching to {newModel} will
  start a new chat. Your current conversation stays available in the
  sidebar." On confirm, fire `keep_alive: 0` against the previous chat
  model, clear active session, set the new selection. Cancel snaps the
  visible dropdown back to the previous value (no popup state leaks
  into `selectedModel`).

- **User clicks a session in the sidebar** → no popup (system-initiated).
  Restore the session's stored model into the dropdown and fire
  `unloadChatModels(targetModel)` so anything that isn't the target
  gets the unload hint.

- **Chat page first mount** → page-load normalization. Anything stacked
  from a prior session gets the unload hint with the current selected
  model as the target-to-preserve. Guarded by a ref so it only fires
  once per page lifetime; gated on `selectedModel` being populated.

Backend surface is a single new helper and a single new route:

  `OllamaService.unloadAllChatModelsExcept(targetModel: string | null)`
  → queries `/api/ps`, filters out (a) the embedding model name
  (hardcoded `nomic-embed-text:v1.5` to avoid the RagService circular
  import) and (b) `targetModel`, fires `POST /api/generate` with empty
  prompt + `keep_alive: 0` in parallel against everything else.
  Returns the names that were hinted. Best-effort: network or Ollama
  errors are logged and swallowed so callers don't fail on housekeeping.

  `POST /api/ollama/unload-chat-models` → thin wrapper validating
  `{ targetModel?: string | null }`.

Why `keep_alive: 0` is safe against in-flight inference: per Ollama's
scheduler semantics, the hint sets the post-completion eviction timer
to zero — the runner is not terminated. If Session A is mid-response
on gemma when Session B fires the unload, gemma stays resident until
A's request completes, then evicts. The user-visible worst case is the
race where A's longer-running request re-extends the timer back to the
default and the unload is no-op'd; the next transition (or page reload)
gets another chance, and Ollama's own LRU catches up under memory
pressure regardless. Robust in-flight tracking deferred to a follow-up
if we see stale-state in the wild.

Base `rc`: v1.40.0 will inherit everything from rc.6 via the backmerge.
Frontend tests deferred to a follow-up PR; existing inertia tsconfig
errors are pre-existing and unrelated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:16:00 -07:00
jakeaturner 21ab37cee4 chore(release): 1.32.0-rc.6 [skip ci] 2026-05-20 10:16:00 -07:00
jakeaturner 9356443d73 refactor(KB): typed failure codes for embedSingleFile + accurate HTTP status
Return a discriminated `EmbedSingleFileResult` from `RagService.embedSingleFile`
with `code: 'not_found' | 'inflight' | 'delete_failed' | 'dispatch_failed'` on
failure. `RagController.embedFile` now maps those codes to the correct status
instead of collapsing every failure to 409:

- not_found       → 404
- inflight        → 409
- delete_failed   → 500
- dispatch_failed → 500

The `code` is also included in the JSON body so clients can branch without
string-matching `error`.
2026-05-20 10:16:00 -07:00
Chris Sherwood d850cb9588 feat(KB): per-file ingest action + state indicator on Stored Files (RFC #883 §5)
Closes the Manual-mode UX dead-end: after toggling 'Auto-index new content
for AI?' to Manual, a freshly-downloaded ZIM (or any pending_decision file)
had no UI path to opt in for embedding short of the global Sync Storage /
Re-embed All bulk actions. Per RFC #883 §5, each Stored Files row now
carries a state pill and an adaptive single-button action.

State pill (left of any existing warning chips):
  - 'Indexed'    — green; row had chunks in Qdrant or state row is 'indexed'
  - 'Not Indexed' — neutral; state is pending_decision or browse_only
  - 'Failed'     — red
  - 'Stalled'    — amber
  - admin_docs collapsed row has no pill ('Managed by NOMAD' carries it)

Adaptive action button (paired with the existing Delete button per row):
  - pending_decision         → 'Index' (force=false)
  - browse_only              → 'Index' (force=true)
  - failed / stalled         → 'Retry' (force=true)
  - indexed + warning chip   → 'Re-embed' (force=true; confirm modal first)
  - indexed healthy / null   → no action button (bulk Re-embed All covers it)

Backend: GET /api/rag/files now returns
  { files: Array<{ source, state, chunksEmbedded }> }
instead of a flat string[]. State + chunk-count come from a single
KbIngestState query unioned into the existing Qdrant-derived source list
(no new round trips). New POST /api/rag/files/embed validates the source is
known, refuses if any inflight job already targets the same filePath
(prevents double-click duplicate-chunk hazard), pre-deletes Qdrant points
when force=true, then dispatches via the existing _dispatchEmbedJobsFor
helper used by reembedAll.

Per-file Re-embed (force=true on an already-indexed file) routes through a
StyledModal confirmation since it deletes existing vectors before queueing
a fresh job — same destructive-action weight as Delete's inline confirm but
heavier since it affects search until the rebuild finishes.

Folds in PR #907's blank-screen fix because my new render needs the same
generic restored: `<StyledTable<KbFileGroup>>` and `record.displayName`
(instead of the unresolved `sourceToDisplayName(record.source)` that ships
in rc.5 and ReferenceErrors on modal open). PR #907 also adds title
tooltips on the three bulk-action buttons; those tooltips are NOT included
here — let PR #907 land first or independently for that part.

Multi-select bulk-opt-in deferred per discussion: most Manual-mode users
ingest 1-2 files at a time, the existing global toggle covers the bulk
case, and checkboxes would expand scope past what rc.6 should hold. Will
file a follow-up issue for an 'Index N pending files' single-click button
once this lands.

Tests-in-PR scope was limited to keeping `kb_file_grouping.spec.ts` green
after the StoredFileInfo[] signature change (added asInfos() wrapper).
Dedicated unit tests for embedSingleFile (unknown source / inflight refused
/ force=true delete-then-dispatch) and the new state-pill rendering will
land in a follow-up PR alongside Playwright coverage of the row actions.

Verification path: NOMAD3 currently runs project-nomad-admin:integration-
rc6-preview (PRs #907 + #908 atop rc.5). After this branch is built into a
new integration tag, I'll re-run targeted Playwright UAT on the KB modal
covering: state pill rendering per state, Index click on pending_decision
opts in cleanly, Retry on failed re-dispatches successfully, Re-embed
confirmation modal copy + delete-then-dispatch on the military-medicine
partial-stall row, and Delete flow untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:16:00 -07:00
Chris Sherwood 0617d54762 feat(easy-setup): split AI into its own conditional step (issue #905)
Easy Setup wizard previously bundled AI model selection + the new
ingest-policy radio into Step 3 alongside Wikipedia/ZIM tiers and curated
content. Three problems with that:

1. Predicate divergence: "is AI selected?" was answered three different
   ways across Step 3 radio, Step 4 review card, and handleFinish
   persistence. Surfaced in @jakeaturner's review of PR #900. The three
   predicates disagree in real cases (e.g. Ollama already installed but
   user didn't re-select any models -- handleFinish writes the ingest KV
   while the review hides the AI summary).

2. Step 3 was overloaded -- ZIM tiers + curated content + AI models +
   ingest policy in one screen.

3. No way to opt out of seeing the AI policy radio when AI isn't part of
   the user's setup.

This restructure makes step 4 a dedicated, conditional AI step:

  Step 1 (Apps)     -- unchanged (services + remote Ollama toggle/URL)
  Step 2 (Maps)     -- unchanged
  Step 3 (Content)  -- Wikipedia + curated tiers only
  Step 4 (AI)       -- NEW, conditional: model picker (or remote notice)
                        + auto-index policy radio. Skipped entirely when
                        AI isn't in the setup.
  Step 5 (Review)   -- summary, reads back step 4's output via the same
                        canonical predicate

Decisions per issue #905 discussion:

- Canonical predicate `isAiInSetup` as a useMemo. Single source consumed
  by step indicator, nav skip logic, review summary, and handleFinish.
  Both prior divergence cases collapse.

- Step indicator renders dynamically: 4 dots when AI is off (positional
  display numbers 1..4), 5 dots when AI is on. WizardStep semantic values
  (1=Apps, 2=Maps, 3=Content, 4=AI, 5=Review) stay stable so nav handlers
  don't have to translate; the dot's `displayNumber` is decoupled from
  its `step` so users see sequential 1..N with no gap.

- handleNext / handleBack are symmetric: 3 -> 5 forward, 5 -> 3 back,
  when !isAiInSetup. Same predicate gate.

- Toggling AI capability off in Step 1 after AI step selections were
  made fires a confirm dialog ("Turning off AI will discard your AI
  model picks, indexing policy, and remote Ollama configuration") and
  clears selectedAiModels / ingestPolicy / remoteOllamaEnabled on
  confirm. Silent clear when nothing was set.

- Remote Ollama toggle stays in Step 1 alongside the capability card.
  Don't fragment "am I using remote AI?" across two steps.

The bundled review summary (renderStep5, was renderStep4) now uses
`isAiInSetup` for the auto-index card visibility instead of the
divergent `(selectedAiModels.length > 0 || remoteOllamaEnabled)`.

Inertia tsconfig clean for this file (the only outstanding errors are
the 3 KnowledgeBaseModal ones from issue tracked in PR #907 and the
~64 pre-existing errors elsewhere).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:16:00 -07:00
Chris Sherwood 633a3c3500 fix(KB): blank-screen on panel open + tooltips on bulk-action buttons
The Stored Knowledge Base Files render crashed on first open in v1.32.0-rc.5
with `ReferenceError: sourceToDisplayName is not defined`. The table column's
render() called `sourceToDisplayName(record.source)` but the function was
extracted to `lib/kb_file_grouping.ts` in PR #892 and never imported in
KnowledgeBaseModal.tsx. The unhandled error unmounts the entire React tree,
so users see a blank screen ~20s after opening the panel.

Root cause: PR #895 (conditional warnings) rewrote the render() and used
`sourceToDisplayName(record.source)` instead of `record.displayName`, which
KbFileGroup already carries from groupAndSortKbFiles(). PR #895's review
follow-up (cbae48a) compounded this by narrowing the StyledTable generic
from `KbFileGroup` to `{source: string}`, hiding the type drift from tsc.

This restores the post-#892 pattern:
- StyledTable generic back to `KbFileGroup`
- Render uses `record.displayName` (works for both per-file rows and the
  collapsed admin-docs row; calling sourceToDisplayName on the synthetic
  `__admin_docs_group__` would have rendered that literal as the row name).

Also folds in tooltip copy on the three bulk-action buttons (Reset & Rebuild,
Re-embed All, Sync Storage) so the difference in destructiveness is visible
on hover. Uses native `title` attribute via StyledButton's prop pass-through;
no new component dependency.

Inertia tsconfig catches this regression cleanly (TS2304 + TS2339); the
pre-push hook only runs the backend tsconfig which excludes inertia/**, so
the bug shipped. Tracking the typecheck-coverage gap as a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 10:16:00 -07:00