Rebuilt on top of dev's RFC #883 state-machine UI rather than the now-defunct
StoredFile shape:
- Extend StoredFileInfo with fileName/size/uploadedAt/isUserUpload
- Populate metadata from on-disk stats in RagService.getStoredFiles
- Add fileSourceSchema validator + getFileContent/downloadFile endpoints
scoped to the uploads directory only (tighter than the original PR — matches
docs_service traversal pattern)
- KnowledgeBaseModal: sortable Size and Uploaded columns; View/Download
buttons on upload-bucket rows; new FileViewerModal for in-browser text
preview. Bucket grouping preserved — sort applies within each bucket.
- Use formatBytes from ~/lib/util rather than redefining
BullMQ instantiates a fresh ioredis client per Queue/Worker when handed a
plain {host, port} config object, and under sustained ZIM ingestion the
embed pipeline leaked ~1 client/sec until Redis maxclients was exhausted.
Pass a single shared ioredis instance (maxRetriesPerRequest: null, as
required by BullMQ) so all queues and workers reuse one client pool.
Workers still duplicate the connection once for their blocking client,
which is expected and bounded.
Closes#885
* feat: replace legacy Kolibri image default with latest v19 image
* feat(supply-depot): add content migration instructions for Edu Platform Gen 1 to 2
Adds the MeshCore web client to the Supply Depot catalog (host port 8500),
alongside the existing Meshtastic apps. Uses aXistem's prebuilt image of Liam
Cottle's MeshCore client (MeshCore is a sibling LoRa mesh project to Meshtastic).
The image is stock nginx serving a static Flutter build over HTTP, but the
client reaches radios via Web Bluetooth / Web Serial, which browsers only allow
from a secure (HTTPS) context. So we serve it over HTTPS: a new preinstall hook
generates a self-signed cert + a small SSL nginx config into storage/meshcore-web,
both bind-mounted into the container (the config over the image's default.conf),
publishing 443. Same one-time browser-warning approach as Vaultwarden, whose
openssl cert generation is refactored into a shared _ensureSelfSignedCert helper.
Also adds a NOMAD-specific docs section + Manage>Docs anchor, and registers the
IconAntenna icon. Meshtastic Web left unchanged.
Validated on NOMAD3 (v1.33.0-rc.1): the image + SSL config + self-signed cert
serves the MeshCore Flutter app over HTTPS 200 with working SPA fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Curated catalog apps could be installed, stopped, and force-reinstalled,
but never removed — the only path off a device was manual docker + DB
surgery. Custom apps already had delete; this adds the equivalent for
curated apps.
POST /api/system/services/uninstall stops and removes the app's
container (optionally its image, same best-effort semantics as custom
app delete) and flips the record back to not-installed so the card
returns to the available catalog. Host bind-mount data is deliberately
left on disk, so a later reinstall picks the app back up where it left
off — unlike force-reinstall, which clears volumes.
Guards: custom apps are rejected (use delete), dependency services are
rejected, and uninstalling a not-installed app is a 409.
UI: installed curated cards get an Uninstall action in the card menu,
with a confirm modal that explains data is preserved and offers the
same remove-image checkbox as custom app delete.
Refresh the in-app Markdoc docs for the v1.33 feature set:
- Repoint dead /settings/apps links to the Supply Depot (/supply-depot)
across home, getting-started, and faq; reword "Apps page" / "Settings
-> Apps" to "Supply Depot". The old /apps route now redirects to the
Supply Depot.
- Expand supply-depot-apps.md with a "Managing your apps" section (Docs/
Edit/Logs/Stats/Update/Remove, version + update-available visibility,
custom launch URLs, per-app auto-update toggle) and a "Bringing your
own app" section for custom Docker containers.
- Add a new "Updates" doc (updates.md) covering the auto-update trilogy
(core/apps/content), manual updates, and the Early Access channel;
wire it into DOC_ORDER and cross-link from home, getting-started, faq.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Only Wikipedia had version cleanup; every other curated map and non-Wikipedia
ZIM left its prior version on disk when a newer one installed, so users silently
accumulated orphaned content (potentially hundreds of GB). (#634)
The install paths already record each resource via InstalledResource
{resource_id, resource_type, version, file_path}, so the authoritative old-file
path for a resource is known. On install of a new version we now capture the
prior row before updateOrCreate repoints it, then delete the old file — gated
behind a pure, fully unit-tested decision function with strict safety rails:
- tracked-only: requires a prior InstalledResource row for the same
resource_id, so sideloaded/untracked files are never touched
- genuine replacement: old and new file paths must differ
- new-file-verified: the new file must be confirmed on disk first
- strictly-newer: a re-install or downgrade can't wipe a newer file
- within-storage-dir: the old path must resolve under the content store
ZIM cleanup deletes the old file directly (NOT via this.delete(), which would
drop the InstalledResource row by resource_id that updateOrCreate just
repointed) and rebuilds the Kiwix library only if a file was actually removed,
so its XML never references a deleted ZIM. Maps need no library step. Wikipedia
keeps its own existing cleanup path. All deletions are best-effort and logged;
a failure never breaks the install.
Closes#634
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`getChatSuggestions` previously picked the largest installed model by file
size, on the assumption that bigger models give better suggestions. This
is unsafe: if any installed model exceeds available VRAM (e.g.
llama3.1:405b on a 96 GB GPU), Ollama spends minutes trying to load it
and the request 500s — making the chat page unusable for anyone who
happens to keep a flagship-sized model on disk.
Chat suggestions are short prompts that don't benefit from a flagship
model anyway. Prefer the user's selected `chat.lastModel` when set, and
fall back to the smallest installed model otherwise. `OllamaService.getModels()`
already excludes embedders, so the fallback always picks a chat model.
Two card tweaks for the update workflow:
- Show the installed image tag next to the powered-by name (e.g. "Kiwix ·
3.7.0"), so the running version is visible at a glance. Only rendered for
installed apps; falls back to just the version when powered_by is unset.
- Change the "Update available" pill from a muted light-green tint to a solid
desert-orange fill with white text, so an available update actually draws
the eye instead of blending into the card.
listTags() follows the registry's Link-header pagination, but the next-page
URL is relative per the OCI/Docker registry spec (e.g.
"/v2/ollama/ollama/tags/list?last=0.9.3-rc5&n=1000"). The code assigned that
raw relative path straight back to `url` and re-fetched it, so fetch() threw
"Failed to parse URL from /v2/...". Any image repo with more than 1000 tags
paginates, so the entire tag list — and therefore the update check — failed
silently for ollama/ollama and filebrowser/filebrowser.
That's the root cause of #945 ("won't update past 0.24.0"): the Ollama
update check never completed, so no newer version was ever offered.
Resolve the next-page URL against the registry origin with
new URL(next, `https://${registry}`), which also passes absolute next-URLs
through unchanged for registries that return those.
Closes#945
The maps page reset to the default US-wide view on every refresh because
initialViewState was hardcoded. Save the position and zoom to localStorage
on each move-end (key nomad:map-view, matching the existing scale-unit
pattern) and restore it at mount: saved view → default. The restore is
bounds-checked so a corrupt value falls through to the default.
Replaces #815, whose branch had drifted far out of scope (56 files of
stale-base/merged-commit noise plus unrelated map-feature WIP). This is
just the persist-view improvement, ported cleanly onto current dev. The
null-island and coordinate-search parts of #815 targeted URL-param code
that never landed on dev, so they don't apply here.
The map style names each source by its date-stripped region (both
"washington.pmtiles" and "washington_2025-12.pmtiles" -> "washington").
When an old and new copy of the same region are both on disk, the style
emitted two sources with the same key and duplicate layer ids, which
MapLibre rejects outright -- blanking the ENTIRE map, not just that region.
Old copies linger when a newer curated version installs (#634), so a user
who updates maps can silently lose all map rendering until the stale file
is removed by hand.
generateSourcesArray() now keeps only the newest file per region: a dated
build beats an undated legacy file, and between two dated builds the later
YYYY-MM wins. The skipped duplicate is logged. The style stays valid even
when stale files are present.
Complements #981, which removes superseded curated files on install. This
is the runtime safety net that also recovers installs already in the broken
state (which a cleanup-on-install alone can't reach).
Refs #634
Dense source content produces chunks that exceed the embedding model's
context window (nomic-embed-text:v1.5 defaults to 2048 tokens). Two paths
hit this even after the prior pre-cap:
- Older Ollama (e.g. 0.18.1, #944) ignores the num_ctx=8192 we send on
/api/embed, so it stays at the model's 2048 default.
- The OpenAI-compat /v1/embeddings fallback didn't pass num_ctx/truncate
at all, so any Ollama drops to 2048 whenever it lands on the fallback.
When a chunk overflowed, the 400 was swallowed and the chunk was silently
dropped from Qdrant. Worse, the failure propagated to EmbedFileJob, which
re-embeds the entire file on each of its 30 BullMQ attempts — the "endless
queue loop" / "api/embed for weeks" / pegged GPU reported in #944/#959.
Fix:
- OllamaService.embed(): on a context-length error, retry once with an
aggressive 2048-safe cap (EMBED_CONTEXT_SAFE_CHARS = 2000) so the chunk
is embedded (start-of-chunk) instead of dropped. Native-path context
errors now bubble to this retry instead of falling through to the
smaller-context fallback. Split the native+fallback attempt into
_embedWithFallback().
- Pass truncate/num_ctx on the /v1/embeddings fallback too (Ollama's
OpenAI-compat shim forwards them).
- EmbedFileJob: classify "input length exceeds context length" as an
UnrecoverableError so one permanently-oversized chunk can't trigger 30
full-file re-embeds.
- Add OllamaService.isContextLengthError() shared by both.
Graceful degradation: a truncated chunk loses its tail but is kept in the
index, which is strictly better than today's silent drop + retry storm.
Refs #881. Supersedes the #369/#670 symptom closures that never fixed the
fallback path.
reconcileFromFilesystem() skipped every ZIM whose filename starts with
`wikipedia_en_`, on the assumption that all such files are managed by the
WikipediaSelection model. But curated category tiers ship Wikipedia-themed
ZIMs (e.g. Medicine → Comprehensive includes `wikipedia_en_medicine_maxi`),
so those files were skipped during reconcile and their InstalledResource
rows got wiped on every restart — silently downgrading the detected tier.
Skip only the single file actually tracked by WikipediaSelection, matched
by exact filename instead of the `wikipedia_en_` prefix.
Reimplemented in-house from @johno10661's PR #774 (which was trapped on a
stale base); credit to them for the diagnosis and fix.
Closes#774
A multi-GB service update (e.g. nomad_ollama pulling ~6.5 GB) left the
Update button clickable with no feedback, so users clicked again thinking
it was stuck. The second click raced a concurrent updateContainer run into
Docker 304/400 errors (stop/rename on a container the first run had already
moved). The backend lock was in-memory only and never written to the DB, so
nothing durable signaled "update in progress" to the UI, and a page reload
mid-pull re-enabled the button.
Backend (docker_service.updateContainer):
- Set installation_status='installing' when the update starts and reset it
to 'idle' in a finally on every exit path. This mirrors the install path,
survives a page reload, and is visible to other tabs/clients.
- Reject a second update with a clear message when installation_status is
already 'installing', instead of letting it race into Docker errors.
Frontend (settings/apps.tsx):
- Track in-flight updates per service. Seed optimistically on click and
reconcile with the durable installation_status from the server.
- Disable the per-service Update button and show "Updating..." while in
flight. Drop the fullscreen spinner for updates so the table and the
activity feed (live pull/stop/start progress) stay visible.
Closes#931
Kiwix runs in library mode reading kiwix-library.xml via --monitorLibrary.
Today a missing or corrupt XML is only repaired on the download path
(rebuildFromDisk after a completed download), so if the file is lost or
truncated outside that flow — storage relocation, an interrupted write, manual
deletion — Kiwix comes up serving an empty library with no path to recovery.
Add KiwixLibraryService.ensureLibraryXmlHealthy(): reads the XML, and if it's
missing (ENOENT) or fails to parse / lacks a <library> root, rebuilds it from
the ZIM files on disk. A well-formed but empty library is treated as valid (no
spurious rebuild), and filesystem errors other than ENOENT are surfaced rather
than masked. The boot provider calls it on the already-in-library-mode path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Let users override an app's "Open" link with a reverse-proxy or
local-DNS address (e.g. https://jellyfin.myhomelab.net). Falls back to
the default host+port when unset. Metadata-only — no container changes.
Replaces the regex blocklist in assertNotPrivateUrl with ipaddr.js range
classification and normalizes the host before checking it. Consolidates two
community proposals (#930 ipaddr.js parsing, #912 trailing-dot normalization)
into one validator so the SSRF-critical path lives in-house with full tests.
- Classify literal IPs by range (loopback / linkLocal / unspecified) via
ipaddr.js instead of a hand-maintained regex list, which also catches
alternate IPv4 encodings and avoids over-blocking mapped public IPs (the old
`::ffff:` regex blocked every mapped address, including public ones). IPv4-
mapped IPv6 is reduced to its embedded IPv4 before classification.
- Strip a trailing root dot from the host so `localhost.` / `127.0.0.1.` can't
bypass the checks (they resolve to the same target as the dotless form, #911).
- Strip IPv6 brackets and lowercase for the localhost comparison.
- RFC1918, bare LAN hostnames (e.g. `nomad3`), and external FQDNs remain
allowed — LAN appliances need them, and DNS rebinding is a fetch-time concern
outside this guard's scope.
Adds a consolidated unit spec covering loopback/link-local/unspecified literals,
alternate encodings, IPv4-mapped v6, mixed-case + trailing-dot localhost, and
the allowed LAN/FQDN/mapped-public cases.
Resolves#922. Supersedes #930 and #912 (thanks @Gujiassh and @luyua9).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every Docker pull went through `followProgress(pullStream, resolve)`, passing
the Promise's resolve as dockerode's onFinished(err, output) callback — so the
error argument was ignored and a failed pull (dropped/metered connection, bad
manifest, registry error, disk full mid-pull) resolved as if it had succeeded.
The code then tried to create/start a container from a missing or partial
image, surfacing a confusing downstream error rather than the real cause. (#790)
Add a DockerService.pullImage() helper that rejects when followProgress reports
an error, and route all five pull sites through it:
- service install
- AMD ROCm image pull
- service update
- force-reinstall / recreate (forcePull)
- sysbench benchmark image pull
Closes#790
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On ZIMs that pack one logical article as several sub-pages (e.g. iFixit),
iterByPath yields more entries passing our isArticleEntry() filter than
archive.articleCount reports. The inter-batch progress used nextOffset /
articleCount, so the numerator outran the denominator, the ratio overflowed past
100%, and the UI (which clamps at 99%) pinned the file at 99% for the entire
remaining tail, making it look hung.
Grow the denominator to max(articleCount, nextOffset + ZIM_BATCH_SIZE) once we
pass the reported article count, so progress keeps creeping forward monotonically
instead of freezing, and clamp to 99% so only the genuinely-final batch reports
100%.
This is a graceful heuristic, not exact progress (true accuracy would require a
pre-scan to count isArticleEntry matches up front); it removes the user-visible
"stuck at 99%" symptom with no change to batch semantics.
Closes#903
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Stored Files "partial stall" warning compares chunks in Qdrant against an
expected count from the ratio registry. The registry has an empty-pattern
catch-all (100 chunks/MB) that matches any filename, so a ZIM that matches no
specific pattern still gets a size-based estimate. For archives that are mostly
PDFs, images, or link-out stubs (e.g. irp.fas.org military-medicine), byte size
wildly over-predicts embeddable text: a 75 MB ZIM estimates ~7,236 chunks but
produces ~1, tripping a false "ingestion may have stalled" warning that re-embed
can't clear.
The catch-all is fine for rough aggregate disk-cost estimates, but it should not
drive a per-file stall signal. Add an `ignoreCatchAll` option to the ratio
lookup that excludes the empty-pattern row (returning null when only the
fallback would match), and use it in the warnings path so partial_stall only
fires when the registry has a *specific* expectation for the file. Files that
match a real pattern (wikipedia_, devdocs_, ifixit_, ...) are unaffected;
disk-cost/batch estimates keep using the fallback.
Closes#913
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The service update path stopped and renamed the old container aside before
the new one was confirmed running, but only wired up rollback if
createContainer threw or the 5s health check failed. A throw from
newContainer.start() itself (bad device/GPU config, host port already bound,
image incompatibility) bubbled straight to the outer catch, which returned a
generic 400 and never restored the old container, leaving the service down.
Retrying then wedged: the failed new container still held the service name, so
the next attempt's rename to `<name>_old` collided with the leftover from the
first attempt and threw the same error every time.
- Wrap newContainer.start() so a start failure removes the half-created
container and rolls back to the previous one.
- Clear any stale `<name>_old` before renaming so retries can't collide.
- Dedupe the three rollback sites into a single rollbackToOld() helper (also
removes a non-null assertion in the create-failure path that could itself
throw when no `_old` existed).
Refs #949
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a service fails to install because something on the host already binds its
port, the user saw the raw dockerode error ("Bind for 0.0.0.0:11434 failed:
port is already allocated"), which is meaningless to a non-technical user. The
most common case is a native Ollama install holding 11434.
Add _humanizeDockerError() to map host port-conflict errors to an actionable
message that names the port and, for Ollama/11434, points at the likely cause
(a host Ollama service) with the commands to stop it. Unrecognized errors pass
through unchanged. Wired into the install failure broadcast and the thrown
error.
Closes#934
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a self-update path for the NOMAD admin/core image that runs without a
human in the loop, gated by layered safety checks. Recreation remains the
sidecar's job; this only decides *whether* to update now and requests it.
An hourly job (AutoUpdateJob) evaluates a side-effect-free decision pipeline
(AutoUpdateService.evaluate):
- opt-in: disabled by default
- in a user-configured local-time window (handles midnight wrap)
- an eligible release: same major (major bumps stay manual), strictly newer,
past a configurable cool-off, GA only (no drafts/prereleases), strict semver
- pre-flight: sidecar present, no in-flight system update / downloads / app
installs, and sufficient host disk (estimated from the registry manifest)
When all pass, it drives SystemUpdateService.requestUpdate() with a vetted tag.
Failure backoff auto-disables after 3 genuine update-request failures; transient
release-lookup errors are skips (offline-first appliances are routinely without
connectivity). Re-enabling clears the backoff state.
Settings UI exposes the toggle, window, and cool-off, with live status (eligible
target, in/out of window, last result/error). A `node ace auto-update:dry-run`
command exercises the full pipeline — and a deterministic --scenarios suite the
pure decision logic — without ever triggering an update.
New KVStore keys under autoUpdate.* hold config + last-run state.
The continuation dispatch in EmbedFileJob did not pass the running chunk
count forward, so each batch started with job.data.chunks undefined.
On the final batch, totalChunks collapsed to just that batch's result
and KbIngestState.markIndexed stored a value far below what Qdrant
actually held.
Add chunksSoFar to EmbedFileJobParams and thread it through the
continuation chain so the final markIndexed call reflects the true
total across all batches.
Closes#933
Allow operators to select a Redis logical database index via the
REDIS_DB environment variable. Without this, the BullMQ queue and the
@adonisjs/transmit Redis transport both implicitly used db 0, causing
key collisions when sharing a Redis instance across multiple services
or environments.
REDIS_DB is added to the env schema as an optional number; both
config/queue.ts and config/transmit.ts fall back to db 0 when unset,
preserving existing behavior.
cosmistack-bot
jakeaturner
Jake Turner
Benjamin Smith
Henry Estela
Lorenzo Galassi
Chris Sherwood
johno10661
akashsalan
Metbcy