chore(collections): surface FDA Drug Reference in Medicine > Standard (#1167)

HOLD UNTIL 1.34.0 GA. Merging early breaks v1.33.0 installs.

The manifest is fetched live from main
(collection_manifest_service.ts: refs/heads/main/collections/kiwix-categories.json),
so every install on every version picks this up the moment it merges — not on
their next upgrade.

`type: "dataset"` does not exist anywhere in v1.33.0: zero hits across
admin/app, admin/inertia and admin/types. It is new in 1.34.0. On a v1.33.0 box
this entry would render as an installable resource with no code that
understands its type and a URL that is not a ZIM, so the install path breaks
for users who are not running the version that introduced the feature.

Held back until GA for that reason, which also means 1.34.0 would otherwise
ship a working Drug Reference that cannot be discovered from Content Explorer.
Entry is byte-identical to the one already on rc.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -1,5 +1,5 @@
 name: Bug Report
-description: Report a bug or issue with Project N.O.M.A.D.
+description: Report a bug or issue with Project NOMAD
 title: "[Bug]: "
 labels: ["bug", "needs-triage"]
 body:
@@ -10,9 +10,9 @@ body:
         
         **Before submitting:**
         - Search existing issues to avoid duplicates
-        - Ensure you're running the latest version of N.O.M.A.D.
+        - Ensure you're running the latest version of NOMAD
         - Redact any personal or sensitive information from logs/configs
-        - Please don't submit issues related to running N.O.M.A.D. on Unraid or another NAS - we don't have plans to support these kinds of platforms at this time
+        - Please don't submit issues related to running NOMAD on Unraid or another NAS - we don't have plans to support these kinds of platforms at this time
 
   - type: dropdown
     id: issue-category
@@ -75,8 +75,8 @@ body:
   - type: input
     id: nomad-version
     attributes:
-      label: N.O.M.A.D. Version
+      label: NOMAD Version
-      description: What version of N.O.M.A.D. are you running? (Check Settings > Update or run `docker ps` and check nomad_admin image tag)
+      description: What version of NOMAD are you running? (Check Settings > Update or run `docker ps` and check nomad_admin image tag)
       placeholder: "e.g., 1.29.0"
     validations:
       required: true
@@ -85,8 +85,9 @@ body:
     id: os
     attributes:
       label: Operating System
-      description: What OS are you running N.O.M.A.D. on?
+      description: What OS are you running NOMAD on?
       options:
+        - Ubuntu 26.04 (Resolute Raccoon)
         - Ubuntu 24.04
         - Ubuntu 22.04
         - Ubuntu 20.04
@@ -150,7 +151,7 @@ body:
         Include any relevant logs or error messages. **Please redact any personal/sensitive information.**
         
         Useful commands for collecting logs:
-        - N.O.M.A.D. management app: `docker logs nomad_admin`
+        - NOMAD management app: `docker logs nomad_admin`
         - Ollama: `docker logs nomad_ollama`
         - Qdrant: `docker logs nomad_qdrant`
         - Specific service: `docker logs nomad_<service-name>`
@@ -185,9 +186,9 @@ body:
       options:
         - label: I have searched for existing issues that might be related to this bug
           required: true
-        - label: I am running the latest version of Project N.O.M.A.D. (or have noted my version above)
+        - label: I am running the latest version of Project NOMAD (or have noted my version above)
           required: true
         - label: I have redacted any personal or sensitive information from logs and screenshots
           required: true
-        - label: This issue is NOT related to running N.O.M.A.D. on an unsupported/non-Debian-based OS
+        - label: This issue is NOT related to running NOMAD on an unsupported/non-Debian-based OS
           required: false

.github/ISSUE_TEMPLATE/config.yml

@@ -8,10 +8,10 @@ contact_links:
     about: Check the official documentation and guides
   - name: 🏆 Community Leaderboard
     url: https://benchmark.projectnomad.us
-    about: View the N.O.M.A.D. benchmark leaderboard
+    about: View the NOMAD benchmark leaderboard
   - name: 🤝 Contributing Guide
     url: https://github.com/Crosstalk-Solutions/project-nomad/blob/main/CONTRIBUTING.md
-    about: Learn how to contribute to Project N.O.M.A.D.
+    about: Learn how to contribute to Project NOMAD
   - name: 📅 Roadmap
     url: https://roadmap.projectnomad.us
     about: See our public roadmap, vote on features, and suggest new ones

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -1,20 +1,20 @@
 name: Feature Request
-description: Suggest a new feature or enhancement for Project N.O.M.A.D.
+description: Suggest a new feature or enhancement for Project NOMAD
 title: "[Feature]: "
 labels: ["enhancement", "needs-discussion"]
 body:
   - type: markdown
     attributes:
       value: |
-        Thanks for your interest in improving Project N.O.M.A.D.! Before you submit a feature request, consider checking our [roadmap](https://roadmap.projectnomad.us) to see if it's already planned or in progress. You're welcome to suggest new ideas there if you don't plan on opening PRs yourself.
+        Thanks for your interest in improving Project NOMAD! Before you submit a feature request, consider checking our [roadmap](https://roadmap.projectnomad.us) to see if it's already planned or in progress. You're welcome to suggest new ideas there if you don't plan on opening PRs yourself.
 
         
         **Please note:** Feature requests are not guaranteed to be implemented. All requests are evaluated based on alignment with the project's goals, feasibility, and community demand.
         
         **Before submitting:**
         - Search existing feature requests and our [roadmap](https://roadmap.projectnomad.us) to avoid duplicates
-        - Consider if this aligns with N.O.M.A.D.'s mission: offline-first knowledge and education
+        - Consider if this aligns with NOMAD's mission: offline-first knowledge and education
-        - Consider the technical feasibility of the feature: N.O.M.A.D. is designed to be containerized and run on a wide range of hardware, so features that require heavy resources (aside from GPU-intensive tasks) or complex host configurations may be less likely to be implemented
+        - Consider the technical feasibility of the feature: NOMAD is designed to be containerized and run on a wide range of hardware, so features that require heavy resources (aside from GPU-intensive tasks) or complex host configurations may be less likely to be implemented
         - Consider the scope of the feature: Small, focused enhancements that can be implemented incrementally are more likely to be implemented than large, broad features that would require significant development effort or have an unclear path forward
         - If you're able to contribute code, testing, or documentation, that significantly increases the chances of your feature being implemented
 
@@ -95,7 +95,7 @@ body:
     attributes:
       label: How important is this feature to you?
       options:
-        - Critical - Blocking my use of N.O.M.A.D.
+        - Critical - Blocking my use of NOMAD
         - High - Would significantly improve my experience
         - Medium - Would be nice to have
         - Low - Minor convenience
@@ -144,7 +144,7 @@ body:
       options:
         - label: I have searched for existing feature requests that might be similar
           required: true
-        - label: This feature aligns with N.O.M.A.D.'s mission of offline-first knowledge and education
+        - label: This feature aligns with NOMAD's mission of offline-first knowledge and education
           required: true
         - label: I understand that feature requests are not guaranteed to be implemented
           required: true

.github/workflows/build-primary-image.yml

@@ -52,3 +52,4 @@ jobs:
             VERSION=${{ inputs.version }}
             BUILD_DATE=${{ github.event.workflow_run.created_at }}
             VCS_REF=${{ github.sha }}
+            CREATOR_PACKS_APP_KEY=${{ secrets.CREATOR_PACKS_APP_KEY }}

.github/workflows/validate-collection-urls.yml

@@ -28,22 +28,46 @@ jobs:
             CHECKED=$((CHECKED + 1))
             printf "Checking: %s ... " "$url"
 
-            # Use Range: bytes=0-0 to avoid downloading the full file.
+            # HEAD transfers no body at all, so nothing is downloaded even when a
-            # --max-filesize 1 aborts early if the server ignores the Range header
+            # server ignores Range headers. curl can still exit non-zero (DNS,
-            # and returns 200 with the full body. The HTTP status is still captured.
+            # TLS, timeout), so capture the status without letting `set -e`
+            # (bash -e) kill the whole step mid-loop.
+            METHOD="HEAD"
+            CURL_EXIT=0
             HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
-              --range 0-0 \
+              --head \
-              --max-filesize 1 \
               --max-time 30 \
+              --retry 2 \
+              --retry-delay 2 \
               --location \
-              "$url")
+              "$url") || CURL_EXIT=$?
+
+            # Some servers refuse HEAD outright. Fall back to a single-byte
+            # ranged GET for those. --max-filesize caps the damage if the server
+            # ignores the Range header, but has to stay comfortably above a
+            # redirect body: --location applies the limit to the 3xx body too,
+            # and a cap below that aborts on the redirect itself.
+            case "$HTTP_CODE" in
+              403|405|501)
+                METHOD="GET"
+                CURL_EXIT=0
+                HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
+                  --range 0-0 \
+                  --max-filesize 8192 \
+                  --max-time 30 \
+                  --retry 2 \
+                  --retry-delay 2 \
+                  --location \
+                  "$url") || CURL_EXIT=$?
+                ;;
+            esac
 
             if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "206" ]; then
-              echo "OK ($HTTP_CODE)"
+              echo "OK ($HTTP_CODE via $METHOD)"
             else
-              echo "FAILED ($HTTP_CODE)"
+              echo "FAILED (HTTP $HTTP_CODE via $METHOD, curl exit $CURL_EXIT)"
               FAILED=$((FAILED + 1))
-              FAILED_URLS="$FAILED_URLS\n  - $url (HTTP $HTTP_CODE)"
+              FAILED_URLS="$FAILED_URLS\n  - $url (HTTP $HTTP_CODE via $METHOD, curl exit $CURL_EXIT)"
             fi
           done <<< "$URLS"
 

CONTRIBUTING.md

@@ -1,8 +1,8 @@
-# Contributing to Project N.O.M.A.D.
+# Contributing to Project NOMAD
 
-Thank you for your interest in contributing to Project N.O.M.A.D.! Community contributions are what keep this project growing and improving. Please read this guide fully before getting started — it will save you (and the maintainers) a lot of time.
+Thank you for your interest in contributing to Project NOMAD! Community contributions are what keep this project growing and improving. Please read this guide fully before getting started — it will save you (and the maintainers) a lot of time.
 
-> **Note:** Acceptance of contributions is not guaranteed. All pull requests are evaluated based on quality, relevance, and alignment with the project's goals. The maintainers of Project N.O.M.A.D. ("Nomad") reserve the right accept, deny, or modify any pull request at their sole discretion.
+> **Note:** Acceptance of contributions is not guaranteed. All pull requests are evaluated based on quality, relevance, and alignment with the project's goals. The maintainers of Project NOMAD ("NOMAD") reserve the right to accept, deny, or modify any pull request at their sole discretion.
 
 ---
 
@@ -12,6 +12,7 @@ Thank you for your interest in contributing to Project N.O.M.A.D.! Community con
 - [Before You Start](#before-you-start)
 - [Getting Started](#getting-started)
 - [Development Workflow](#development-workflow)
+- [UI Consistency](#ui-consistency)
 - [Commit Messages](#commit-messages)
 - [Release Notes](#release-notes)
 - [Versioning](#versioning)
@@ -48,11 +49,11 @@ When opening an issue:
 ---
 
 ## Getting Started with Contributing
-**Please note**: this is the Getting Started guide for developing and contributing to Nomad, NOT [installing Nomad](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/README.md) for regular use! 
+**Please note**: this is the Getting Started guide for developing and contributing to NOMAD, NOT [installing NOMAD](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/README.md) for regular use! 
 
 ### Prerequisites
 
-- A Debian-based OS (Ubuntu recommended)
+- A Debian-based OS (Ubuntu 26.04 LTS recommended)
 - `sudo`/root privileges
 - Docker installed and running
 - A stable internet connection (required for dependency downloads)
@@ -72,7 +73,7 @@ When opening an issue:
    ```
 
 ### Avoid Installing a Release Version Locally
-Because Nomad relies heavily on Docker, we actually recommend against installing a release version of the project on the same local machine where you are developing. This can lead to conflicts with ports, volumes, and other resources. Instead, you can run your development version in a separate Docker environment while keeping your local machine clean. It certainly __can__ be done, but it adds complexity to your setup and workflow. If you choose to install a release version locally, please ensure you have a clear strategy for managing potential conflicts and resource usage.
+Because NOMAD relies heavily on Docker, we actually recommend against installing a release version of the project on the same local machine where you are developing. This can lead to conflicts with ports, volumes, and other resources. Instead, you can run your development version in a separate Docker environment while keeping your local machine clean. It certainly __can__ be done, but it adds complexity to your setup and workflow. If you choose to install a release version locally, please ensure you have a clear strategy for managing potential conflicts and resource usage.
 
 ---
 
@@ -92,7 +93,7 @@ Because Nomad relies heavily on Docker, we actually recommend against installing
    git checkout -b feature/add-new-tool
    ```
 
-3. **Make your changes.** Follow existing code style and conventions. Test your changes locally against a running N.O.M.A.D. instance before submitting.
+3. **Make your changes.** Follow existing code style and conventions. Test your changes locally against a running NOMAD instance before submitting.
 
 4. **Add release notes** (see [Release Notes](#release-notes) below).
 
@@ -102,6 +103,34 @@ Because Nomad relies heavily on Docker, we actually recommend against installing
 
 ---
 
+## UI Consistency
+
+NOMAD's guiding principle is that **user-friendliness is paramount**: a control that looks or behaves differently from the rest of the app reads as broken to a non-technical user. New frontend (inertia/React) work should be visually and behaviorally uniform with what is already there. Before adding a UI element, look at its neighbors and reuse the shared building blocks rather than hand-rolling a one-off.
+
+**Reuse the shared components** in `admin/inertia/components/` (and `.../components/inputs/`):
+
+| Need | Use | Not |
+|------|-----|-----|
+| Binary on/off setting | `Switch` | a raw `<input type="checkbox">` |
+| Explanatory hover help | `InfoTooltip` | a raw `title=` attribute or a bespoke tooltip |
+| Modal / confirmation dialog | `StyledModal` | a hand-built overlay |
+| Text field | `Input` | a bare `<input>` |
+| Section heading | `StyledSectionHeader` | ad-hoc heading markup |
+
+Grep for an existing component before building a new one.
+
+**Match the neighbors.** Copy the exact classes and conventions of adjacent elements:
+
+- **Labels:** match punctuation and casing of sibling labels. If the field beside yours reads `Model:` (with a colon), yours should read `Thinking:`, not `Thinking`. Reuse the same typography tokens (e.g. `text-sm text-text-secondary`).
+- **Theme:** use design tokens (`text-*`, `bg-*`, `border-*`) so the element works in light and dark mode. Never hardcode colors.
+- **Placement:** make sure popovers and tooltips are not clipped or crushed against a viewport edge, including when the trigger sits near an edge of the screen.
+
+**When a raw control is fine.** These conventions are about matching intent, not banning primitives. A raw checkbox is appropriate for a multi-select list or a consent box; radio groups and native selects are fine where a shared component does not exist. The point is to reach for the shared component when your case matches its intent (a binary setting toggle should be a `Switch`), not to eliminate primitives.
+
+**Test UI changes in a real browser.** Most of these conventions are judgment calls that tooling cannot fully enforce, so the single most important habit is to load your change in a browser against a running instance before submitting. Several classes of issue (clipped or cramped tooltips, layout breaking at different window widths, blank-screen render errors) are invisible to type-checking and only show up when you actually look at the page. Check the states that should appear *and* the states that should be hidden, and try more than one window width when layout or positioning is involved.
+
+---
+
 ## Commit Messages
 
 This project uses [Conventional Commits](https://www.conventionalcommits.org/). All commit messages must follow this format:
@@ -166,10 +195,10 @@ This project uses [Semantic Versioning](https://semver.org/). Versions are manag
 
 Have questions or want to discuss ideas before opening an issue? Join the community:
 
-- **Discord:** [Join the Crosstalk Solutions server](https://discord.com/invite/crosstalksolutions) — the best place to get help, share your builds, and talk with other N.O.M.A.D. users
+- **Discord:** [Join the Crosstalk Solutions server](https://discord.com/invite/crosstalksolutions) — the best place to get help, share your builds, and talk with other NOMAD users
 - **Website:** [www.projectnomad.us](https://www.projectnomad.us)
 - **Benchmark Leaderboard:** [benchmark.projectnomad.us](https://benchmark.projectnomad.us)
 
 ---
 
-*Project N.O.M.A.D. is licensed under the [Apache License 2.0](LICENSE).*
+*Project NOMAD is licensed under the [Apache License 2.0](LICENSE).*

Dockerfile

@@ -4,6 +4,7 @@ FROM node:22-slim AS base
 RUN apt-get update && apt-get install -y \
       bash \
       curl \
+      openssl \
       graphicsmagick \
       libvips-dev \
       build-essential \
@@ -27,6 +28,14 @@ FROM base AS build
 WORKDIR /app
 COPY --from=deps /app/node_modules /app/node_modules
 ADD admin/ ./
+# Regenerate the curated drug-reference data modules
+# (app/data/{conditions,natural_remedies,home_remedies}.ts) from their single
+# source of truth — the repo-root collections/*.json — so the JSON is what gets
+# compiled into the image and the committed .ts can never silently drift from it
+# in a build. The gen script resolves ../../collections relative to admin/scripts,
+# which is /collections once admin/ has been copied to /app.
+COPY collections/ /collections/
+RUN npm run gen:curated-data
 RUN node ace build
 
 # Production stage
@@ -61,8 +70,8 @@ RUN set -eux; \
     /usr/local/bin/pmtiles version
 
 # Labels
-LABEL org.opencontainers.image.title="Project N.O.M.A.D" \
+LABEL org.opencontainers.image.title="Project NOMAD" \
-      org.opencontainers.image.description="The Project N.O.M.A.D Official Docker image" \
+      org.opencontainers.image.description="The Project NOMAD Official Docker image" \
       org.opencontainers.image.version="${VERSION}" \
       org.opencontainers.image.created="${BUILD_DATE}" \
       org.opencontainers.image.revision="${VCS_REF}" \
@@ -72,6 +81,17 @@ LABEL org.opencontainers.image.title="Project N.O.M.A.D" \
       org.opencontainers.image.licenses="Apache-2.0"
 
 ENV NODE_ENV=production
+
+# Creator Packs entitlement key, injected into OFFICIAL release builds at build
+# time (--build-arg CREATOR_PACKS_APP_KEY=... from the CREATOR_PACKS_APP_KEY CI
+# secret; see build-primary-image.yml). Baked as an ENV so admin/start/env.ts
+# reads it at runtime. Empty by default, so builds from source (and any build
+# without the secret) ship UNCONFIGURED and hide the Creator Packs UI. The key
+# lands in this public image layer (extractable — the accepted ceiling); rotate
+# via `wrangler secret put APP_KEY` + a new image if it leaks.
+ARG CREATOR_PACKS_APP_KEY=""
+ENV CREATOR_PACKS_APP_KEY=$CREATOR_PACKS_APP_KEY
+
 WORKDIR /app
 COPY --from=production-deps /app/node_modules /app/node_modules
 COPY --from=build /app/build /app
@@ -84,6 +104,10 @@ RUN echo "{\"version\":\"${VERSION}\"}" > /app/version.json
 COPY admin/docs /app/docs
 COPY README.md /app/README.md
 
+# Empty Calibre library, seeded into storage/books on Calibre-Web install
+# (see DockerService._runPreinstallActions__CalibreWeb)
+COPY install/calibre-empty-library/metadata.db /app/assets/calibre/metadata.db
+
 # Copy entrypoint script and ensure it's executable
 COPY install/entrypoint.sh /usr/local/bin/entrypoint.sh
 RUN chmod +x /usr/local/bin/entrypoint.sh

FAQ.md

@@ -1,6 +1,6 @@
 # Frequently Asked Questions (FAQ)
 
-Find answers to some of the most common questions about Project N.O.M.A.D.
+Find answers to some of the most common questions about Project NOMAD
 
 ## Can I customize the port(s) that NOMAD uses?
 
@@ -26,30 +26,30 @@ Long answer: Custom storage paths, mount points, and external drives (like iSCSI
 
 ## Why does NOMAD require a Debian-based OS?
 
-Project N.O.M.A.D. is currently designed to run on Debian-based Linux distributions (with Ubuntu being the recommended distro) because our installation scripts and Docker configurations are optimized for this environment. While it's technically possible to run the Docker containers on other operating systems that support Docker, we have not tested or optimized the installation process for non-Debian-based systems, so we cannot guarantee a smooth experience on those platforms at this time.
+Project NOMAD is currently designed to run on Debian-based Linux distributions (with Ubuntu 26.04 LTS being the recommended version) because our installation scripts and Docker configurations are optimized for this environment. While it's technically possible to run the Docker containers on other operating systems that support Docker, we have not tested or optimized the installation process for non-Debian-based systems, so we cannot guarantee a smooth experience on those platforms at this time.
 
 Support for other operating systems will come in the future, but because our development resources are limited as a free and open-source project, we needed to prioritize our efforts and focus on a narrower set of supported platforms for the initial release. We chose Debian-based Linux as our starting point because it's widely used, easy to spin up, and provides a stable environment for running Docker containers.
 
-For Windows users, the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) provides a community-supported path. Community members have also published guides for other platforms (e.g. macOS) in our Discord community and [Github Discussions](https://github.com/Crosstalk-Solutions/project-nomad/discussions), so if you're interested in running N.O.M.A.D. on a non-Debian-based system, we recommend checking there for any available resources or guides. However, keep in mind that if you choose to run N.O.M.A.D. on a non-Debian-based system, you may encounter issues that we won't be able to provide support for, and you may need to have a higher level of technical expertise to troubleshoot and resolve any problems that arise.
+For Windows users, the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) provides a community-supported path. Community members have also published guides for other platforms (e.g. macOS) in our Discord community and [Github Discussions](https://github.com/Crosstalk-Solutions/project-nomad/discussions), so if you're interested in running NOMAD on a non-Debian-based system, we recommend checking there for any available resources or guides. However, keep in mind that if you choose to run NOMAD on a non-Debian-based system, you may encounter issues that we won't be able to provide support for, and you may need to have a higher level of technical expertise to troubleshoot and resolve any problems that arise.
 
 ## Can I run NOMAD on a Raspberry Pi or other ARM-based device?
-Project N.O.M.A.D. is currently designed to run on x86-64 architecture, and we have not yet tested or optimized it for ARM-based devices like the Raspberry Pi (and have not published any official images for ARM architecture).
+Project NOMAD is currently designed to run on x86-64 architecture, and we have not yet tested or optimized it for ARM-based devices like the Raspberry Pi (and have not published any official images for ARM architecture).
 
 Support for ARM-based devices is on our roadmap, but our initial focus was on x86-64 hardware due to its widespread use and compatibility with a wide range of applications.
 
-Community members have forked and published their own ARM-compatible images and installation guides for running N.O.M.A.D. on Raspberry Pi and other ARM-based devices in our Discord community and [Github Discussions](https://github.com/Crosstalk-Solutions/project-nomad/discussions), but these are not officially supported by the core development team, and we cannot guarantee their functionality or provide support for any issues that arise when using these community-created resources.
+Community members have forked and published their own ARM-compatible images and installation guides for running NOMAD on Raspberry Pi and other ARM-based devices in our Discord community and [Github Discussions](https://github.com/Crosstalk-Solutions/project-nomad/discussions), but these are not officially supported by the core development team, and we cannot guarantee their functionality or provide support for any issues that arise when using these community-created resources.
 
 ## What are the hardware requirements for running NOMAD?
 
-Project N.O.M.A.D. itself is quite lightweight and can run on even modest x86-64 hardware, but the tools and resources you choose to install with N.O.M.A.D. will determine the specs required for your unique deployment. Please see the [Hardware Guide](https://www.projectnomad.us/hardware) for detailed build recommendations at various price points.
+Project NOMAD itself is quite lightweight and can run on even modest x86-64 hardware, but the tools and resources you choose to install with NOMAD will determine the specs required for your unique deployment. Please see the [Hardware Guide](https://www.projectnomad.us/hardware) for detailed build recommendations at various price points.
 
 ## Does NOMAD support languages other than English?
 
-As of March 2026, Project N.O.M.A.D.'s UI is only available in English, and the majority of the tools and resources available through N.O.M.A.D. are also primarily in English. However, we have multi-language support on our roadmap for a future release, and we are actively working on adding support for additional languages both in the UI and in the available tools/resources. If you're interested in contributing to this effort, please check out our [CONTRIBUTING.md](CONTRIBUTING.md) file for guidelines on how to get involved.
+As of March 2026, Project NOMAD's UI is only available in English, and the majority of the tools and resources available through NOMAD are also primarily in English. However, we have multi-language support on our roadmap for a future release, and we are actively working on adding support for additional languages both in the UI and in the available tools/resources. If you're interested in contributing to this effort, please check out our [CONTRIBUTING.md](CONTRIBUTING.md) file for guidelines on how to get involved.
 
 ## What technologies is NOMAD built with?
 
-Project N.O.M.A.D. is built using a combination of technologies, including:
+Project NOMAD is built using a combination of technologies, including:
 - **Docker:** for containerization of the Command Center and its dependencies
 - **Node.js & TypeScript:** for the backend of the Command Center, particularly the [AdonisJS](https://adonisjs.com/) framework
 - **React:** for the frontend of the Command Center, utilizing [Vite](https://vitejs.dev/) and [Inertia.js](https://inertiajs.com/) under the hood
@@ -59,7 +59,7 @@ Project N.O.M.A.D. is built using a combination of technologies, including:
 NOMAD makes use of the Docker-outside-of-Docker ("DooD") pattern, which allows the Command Center to manage and orchestrate other Docker containers on the host machine without needing to run Docker itself inside a container. This approach provides better performance and compatibility with a wider range of host environments while still allowing for powerful container management capabilities through the Command Center's UI.
 
 ## Can I run NOMAD if I have existing Docker containers on my machine?
-Yes, you can safely run Project N.O.M.A.D. on a machine that already has existing Docker containers. NOMAD is designed to coexist with other Docker containers and will not interfere with them as long as there are no port conflicts or resource constraints.
+Yes, you can safely run Project NOMAD on a machine that already has existing Docker containers. NOMAD is designed to coexist with other Docker containers and will not interfere with them as long as there are no port conflicts or resource constraints.
 
 All of NOMAD's containers are prefixed with `nomad_` in their names, so they can be easily identified and managed separately from any other containers you may have running. Just make sure to review the ports that NOMAD's core services (Command Center, MySQL, Redis) use during installation and adjust them if necessary to avoid conflicts with your existing containers.
 
@@ -76,17 +76,17 @@ NOMAD by default uses Ollama inside of a docker container to run LLM Models for
 No, the AI features in NOMAD (Ollama, Qdrant, custom RAG pipeline, etc.) are all optional and not required to use the core functionality of NOMAD.
 
 ## Is NOMAD actually free? Are there any hidden costs?
-Yes, Project N.O.M.A.D. is completely free and open-source software licensed under the Apache License 2.0. There are no hidden costs or fees associated with using NOMAD itself, and we don't have any plans to introduce "premium" features or paid tiers.
+Yes, Project NOMAD is completely free and open-source software licensed under the Apache License 2.0. There are no hidden costs or fees associated with using NOMAD itself, and we don't have any plans to introduce "premium" features or paid tiers.
 
 Aside from the cost of the hardware you choose to run it on, there are no costs associated with using NOMAD.
 
 ## Do you sell hardware or pre-built devices with NOMAD pre-installed?
 
-No, we do not sell hardware or pre-built devices with NOMAD pre-installed at this time. Project N.O.M.A.D. is a free and open-source software project, and we provide detailed installation instructions and hardware recommendations for users to set up their own NOMAD instances on compatible hardware of their choice. The tradeoff to this DIY approach is some additional setup time and technical know-how required on the user's end, but it also allows for greater flexibility and customization in terms of hardware selection and configuration to best suit each user's unique needs, budget, and preferences.
+No, we do not sell hardware or pre-built devices with NOMAD pre-installed at this time. Project NOMAD is a free and open-source software project, and we provide detailed installation instructions and hardware recommendations for users to set up their own NOMAD instances on compatible hardware of their choice. The tradeoff to this DIY approach is some additional setup time and technical know-how required on the user's end, but it also allows for greater flexibility and customization in terms of hardware selection and configuration to best suit each user's unique needs, budget, and preferences.
 
 ## How quickly are issues resolved when reported?
 
-We strive to address and resolve issues as quickly as possible, but please keep in mind that Project N.O.M.A.D. is a free and open-source project maintained by a small team of volunteers. We prioritize issues based on their severity, impact on users, and the resources required to resolve them. Critical issues that affect a large number of users are typically addressed more quickly, while less severe issues may take longer to resolve. Aside from the development efforts needed to address the issue, we do our best to conduct thorough testing and validation to ensure that any fix we implement doesn't introduce new issues or regressions, which also adds to the time it takes to resolve an issue.
+We strive to address and resolve issues as quickly as possible, but please keep in mind that Project NOMAD is a free and open-source project maintained by a small team of volunteers. We prioritize issues based on their severity, impact on users, and the resources required to resolve them. Critical issues that affect a large number of users are typically addressed more quickly, while less severe issues may take longer to resolve. Aside from the development efforts needed to address the issue, we do our best to conduct thorough testing and validation to ensure that any fix we implement doesn't introduce new issues or regressions, which also adds to the time it takes to resolve an issue.
 
 We also encourage community involvement in troubleshooting and resolving issues, so if you encounter a problem, please consider checking our Discord community and Github Discussions for potential solutions or workarounds while we work on an official fix.
 

README.md

@@ -1,10 +1,8 @@
 <div align="center">
 <img src="admin/public/project_nomad_logo.webp" width="200" height="200"/>
 
-# Project N.O.M.A.D.
+# Project NOMAD
-### Node for Offline Media, Archives, and Data
+### Knowledge That Never Goes Offline
-
-**Knowledge That Never Goes Offline**
 
 [![Website](https://img.shields.io/badge/Website-projectnomad.us-blue)](https://www.projectnomad.us)
 [![Discord](https://img.shields.io/badge/Discord-Join%20Community-5865F2)](https://discord.com/invite/crosstalksolutions)
@@ -14,10 +12,10 @@
 
 ---
 
-Project N.O.M.A.D. is a self-contained, offline-first knowledge and education server packed with critical tools, knowledge, and AI to keep you informed and empowered—anytime, anywhere.
+Project NOMAD is a self-contained, offline-first knowledge and education server packed with critical tools, knowledge, and AI to keep you informed and empowered — anytime, anywhere.
 
 ## Installation & Quickstart
-Project N.O.M.A.D. can be installed on any Debian-based operating system (we recommend Ubuntu). Installation is completely terminal-based, and all tools and resources are designed to be accessed through the browser, so there's no need for a desktop environment if you'd rather setup N.O.M.A.D. as a "server" and access it through other clients.
+Project NOMAD can be installed on any Debian-based operating system (we recommend Ubuntu 26.04 LTS; 24.04 LTS and Debian 12 are also supported). Installation is completely terminal-based, and all tools and resources are designed to be accessed through the browser, so there's no need for a desktop environment if you'd rather setup NOMAD as a "server" and access it through other clients.
 
 *Note: sudo/root privileges are required to run the install script*
 
@@ -30,7 +28,7 @@ curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/r
 sudo bash install_nomad.sh
 ```
 
-Project N.O.M.A.D. is now installed on your device! Open a browser and navigate to `http://localhost:8080` (or `http://DEVICE_IP:8080`) to start exploring!
+Project NOMAD is now installed on your device! Open a browser and navigate to `http://localhost:8080` (or `http://DEVICE_IP:8080`) to start exploring!
 
 For a complete step-by-step walkthrough (including Ubuntu installation), see the [Installation Guide](https://www.projectnomad.us/install). For Windows users, see the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) — community-supported path covering native Docker and Docker Desktop install routes.
 
@@ -38,7 +36,7 @@ For a complete step-by-step walkthrough (including Ubuntu installation), see the
 For more control over the installation process, copy and paste the [Docker Compose template](https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/management_compose.yaml) into a `docker-compose.yml` file and customize it to your liking (be sure to replace any placeholders with your actual values). Then, run `docker compose up -d` to start the Command Center and its dependencies. Note: this method is recommended for advanced users only, as it requires familiarity with Docker and manual configuration before starting.
 
 ## How It Works
-N.O.M.A.D. is a management UI ("Command Center") and API that orchestrates a collection of containerized tools and resources via [Docker](https://www.docker.com/). It handles installation, configuration, and updates for everything — so you don't have to.
+NOMAD is a management UI ("Command Center") and API that orchestrates a collection of containerized tools and resources via [Docker](https://www.docker.com/). It handles installation, configuration, and updates for everything — so you don't have to.
 
 **Built-in capabilities include:**
 - **AI Chat with Knowledge Base** — local AI chat powered by [Ollama](https://ollama.com/) or you can use OpenAI API compatible software such as LM Studio or llama.cpp, with document upload and semantic search (RAG via [Qdrant](https://qdrant.tech/))
@@ -48,9 +46,11 @@ N.O.M.A.D. is a management UI ("Command Center") and API that orchestrates a col
 - **Data Tools** — encryption, encoding, and analysis via [CyberChef](https://gchq.github.io/CyberChef/)
 - **Notes** — local note-taking via [FlatNotes](https://github.com/dullage/flatnotes)
 - **System Benchmark** — hardware scoring with a [community leaderboard](https://benchmark.projectnomad.us)
+- **Supply Depot** — a one-click app catalog (PDF tools, file browser, e-book library, password manager, and more) plus the ability to run your own custom Docker containers
+- **Automatic Updates** — opt-in, hands-off updates for the core software, installed apps, and offline content, on a schedule you control
 - **Easy Setup Wizard** — guided first-time configuration with curated content collections
 
-N.O.M.A.D. also includes built-in tools like a Wikipedia content selector, ZIM library manager, and content explorer.
+NOMAD also includes built-in tools like a Wikipedia content selector, ZIM library manager, and content explorer.
 
 ## What's Included
 
@@ -59,79 +59,116 @@ N.O.M.A.D. also includes built-in tools like a Wikipedia content selector, ZIM l
 | Information Library | Kiwix | Offline Wikipedia, medical references, survival guides, ebooks |
 | AI Assistant | Ollama + Qdrant | Built-in chat with document upload and semantic search |
 | Education Platform | Kolibri | Khan Academy courses, progress tracking, multi-user support |
-| Offline Maps | ProtoMaps | Downloadable regional maps with search and navigation |
+| Offline Maps | ProtoMaps | Downloadable regional maps for offline viewing and search |
 | Data Tools | CyberChef | Encryption, encoding, hashing, and data analysis |
 | Notes | FlatNotes | Local note-taking with markdown support |
 | System Benchmark | Built-in | Hardware scoring, Builder Tags, and community leaderboard |
+| Supply Depot | Built-in | One-click app catalog + bring-your-own custom Docker containers |
 
 ## Device Requirements
-While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project N.O.M.A.D. is quite the opposite. To install and run the
+While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project NOMAD is quite the opposite. To install and run the
 available AI tools, we highly encourage the use of a beefy, GPU-backed device to make the most of your install.
 
-At it's core, however, N.O.M.A.D. is still very lightweight. For a barebones installation of the management application itself, the following minimal specs are required:
+At its core, however, NOMAD is still very lightweight. For a barebones installation of the management application itself, the following minimal specs are required:
 
-*Note: Project N.O.M.A.D. is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The harware listed below is for example/comparison use only*
+*Note: Project NOMAD is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The hardware listed below is for example/comparison use only*
 
 #### Minimum Specs
 - Processor: 2 GHz dual-core processor or better
 - RAM: 4GB system memory
 - Storage: At least 5 GB free disk space
-- OS: Debian-based (Ubuntu recommended)
+- OS: Debian-based (Ubuntu 26.04 LTS recommended)
 - Stable internet connection (required during install only)
 
-To run LLM's and other included AI tools:
+To run LLMs and other included AI tools:
 
 #### Optimal Specs
 - Processor: AMD Ryzen 7 or Intel Core i7 or better
 - RAM: 32 GB system memory
 - Graphics: NVIDIA RTX 3060 or AMD equivalent or better (more VRAM = run larger models)
 - Storage: At least 250 GB free disk space (preferably on SSD)
-- OS: Debian-based (Ubuntu recommended)
+- OS: Debian-based (Ubuntu 26.04 LTS recommended)
 - Stable internet connection (required during install only)
 
 **For detailed build recommendations at three price points ($150–$1,000+), see the [Hardware Guide](https://www.projectnomad.us/hardware).**
 
-Again, Project N.O.M.A.D. itself is quite lightweight - it's the tools and resources you choose to install with N.O.M.A.D. that will determine the specs required for your unique deployment
+Again, Project NOMAD itself is quite lightweight — it's the tools and resources you choose to install with NOMAD that will determine the specs required for your unique deployment
 
 #### Running AI models on a different host
-By default, N.O.M.A.D.'s installer will attempt to setup Ollama on the host when the AI Assistant is installed. However, if you would like to run the AI model on a different host, you can go to the settings of of the AI assistant and input a URL for either an ollama or OpenAI-compatible API server (such as LM Studio).  
+By default, NOMAD's installer will attempt to setup Ollama on the host when the AI Assistant is installed. However, if you would like to run the AI model on a different host, you can go to the settings of the AI assistant and input a URL for either an ollama or OpenAI-compatible API server (such as LM Studio).  
-Note that if you use Ollama on a different host, you must start the server with this option `OLLAMA_HOST=0.0.0.0`.  
+Note that if you use Ollama on a different host, you must start the server with this option: `OLLAMA_HOST=0.0.0.0`.  
-Ollama is the preferred way to use the AI assistant as it has features such as model download that OpenAI API does not support. So when using LM Studio for example, you will have to use LM Studio to download models.
+Ollama is the preferred way to use the AI assistant, as it has features such as model download that OpenAI API does not support. So when using LM Studio, for example, you will have to use LM Studio to download models.
 You are responsible for the setup of Ollama/OpenAI server on the other host.
 
 ## Frequently Asked Questions (FAQ)
-For answers to common questions about Project N.O.M.A.D., please see our [FAQ](FAQ.md) page.
+For answers to common questions about Project NOMAD, please see our [FAQ](FAQ.md) page.
 
 ## About Internet Usage & Privacy
-Project N.O.M.A.D. is designed for offline usage. An internet connection is only required during the initial installation (to download dependencies) and if you (the user) decide to download additional tools and resources at a later time. Otherwise, N.O.M.A.D. does not require an internet connection and has ZERO built-in telemetry.
+Project NOMAD is designed for offline usage. An internet connection is only required during the initial installation (to download dependencies) and if you (the user) decide to download additional tools and resources at a later time. Otherwise, NOMAD does not require an internet connection and has ZERO built-in telemetry.
 
-To test internet connectivity, N.O.M.A.D. attempts to make a request to Cloudflare's utility endpoint, `https://1.1.1.1/cdn-cgi/trace` and checks for a successful response.
+To test internet connectivity, NOMAD first attempts to make a request to Cloudflare's utility endpoint, `https://1.1.1.1/cdn-cgi/trace`. If that endpoint is unreachable (for example, because your network blocks `1.1.1.1`), it falls back to other endpoints the application already contacts (the GitHub API and the Project NOMAD API) and considers the connection online if any of them respond.
+
+You can override the endpoint used for this check in two ways. The connectivity test URL can be configured from the UI under **Settings → Advanced** (stored locally on your instance), or you can set the `INTERNET_STATUS_TEST_URL` environment variable. When set, the environment variable always takes precedence over the UI-configured value. If neither is set, the built-in defaults above are used.
 
 ## About Security
-By design, Project N.O.M.A.D. is intended to be open and available without hurdles - it includes no authentication. If you decide to connect your device to a local network after install (e.g. for allowing other devices to access it's resources), you can block/open ports to control which services are exposed.
+By design, Project NOMAD is intended to be open and available without hurdles — it includes no authentication. If you decide to connect your device to a local network after install (e.g. for allowing other devices to access its resources), you can block/open ports to control which services are exposed.
 
-**Will authentication be added in the future?** Maybe. It's not currently a priority, but if there's enough demand for it, we may consider building in an optional authentication layer in a future release to support uses cases where multiple users need access to the same instance but with different permission levels (e.g. family use with parental controls, classroom use with teacher/admin accounts, etc.). We have a suggestion for this on our public roadmap, so if this is something you'd like to see, please upvote it here: https://roadmap.projectnomad.us/posts/1/user-authentication-please-build-in-user-auth-with-admin-user-roles
+**Will authentication be added in the future?** Maybe. It's not currently a priority, but if there's enough demand for it, we may consider building in an optional authentication layer in a future release to support use cases where multiple users need access to the same instance but with different permission levels (e.g. family use with parental controls, classroom use with teacher/admin accounts, etc.). We have a suggestion for this on our public roadmap, so if this is something you'd like to see, please upvote it here: https://roadmap.projectnomad.us/posts/1/user-authentication-please-build-in-user-auth-with-admin-user-roles
 
-For now, we recommend using network-level controls to manage access if you're planning to expose your N.O.M.A.D. instance to other devices on a local network. N.O.M.A.D. is not designed to be exposed directly to the internet, and we strongly advise against doing so unless you really know what you're doing, have taken appropriate security measures, and understand the risks involved.
+For now, we recommend using network-level controls to manage access if you're planning to expose your NOMAD instance to other devices on a local network. NOMAD is not designed to be exposed directly to the internet, and we strongly advise against doing so unless you really know what you're doing, have taken appropriate security measures, and understand the risks involved.
 
 ## Contributing
 Contributions are welcome and appreciated! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to contribute to the project.
 
+### Testing Auto-Updates (Dry Run)
+
+The Command Center can automatically install **minor/patch** updates of itself during a configurable window, after a cool-off period, and only when pre-flight checks pass (sufficient disk for the new image, no downloads or app installs in progress). Major versions always require a manual update.
+
+Because exercising this logic with real version bumps is impractical, an Ace command runs the **entire decision pipeline without ever triggering an update**. Run it from the `admin/` directory:
+
+```bash
+# 1) Deterministic scenario suite — no network, DB, or Docker required.
+#    Proves every branch (major-only, cool-off, prerelease/draft, window wrap, …)
+#    and exits non-zero on failure, so it's safe to wire into CI.
+node ace auto-update:dry-run --scenarios
+
+# 2) Simulate "what would happen if I were running 1.32.0 right now?"
+#    against the LIVE GitHub releases feed and real pre-flight checks:
+node ace auto-update:dry-run --current=1.32.0 --force-enabled
+
+# 3) Fully offline simulation with a canned release list and a fixed clock:
+node ace auto-update:dry-run --current=1.32.0 --force-enabled \
+  --releases-file=./fixtures/releases.json --now=2026-06-04T21:00:00Z \
+  --window-start=20:00 --window-end=23:00 --cooloff=72 --skip-preflight
+```
+
+It prints the resolved decision — current version, whether the clock is inside the window, the eligible target (if any), and pre-flight blockers — ending in a clear verdict such as `WOULD UPDATE → v1.33.2` or `WOULD NOT UPDATE (outside-window): …`. **No real update is ever requested.**
+
+| Flag | Description |
+|------|-------------|
+| `--scenarios` | Run the built-in deterministic scenario suite and exit |
+| `--current=<version>` | Simulate this currently-running version (e.g. `1.32.0`) |
+| `--force-enabled` | Treat auto-update as enabled, ignoring the saved setting |
+| `--cooloff=<hours>` | Override the cool-off period |
+| `--window-start=<HH:MM>` / `--window-end=<HH:MM>` | Override the update window |
+| `--now=<ISO timestamp>` | Simulate the clock at a specific time |
+| `--releases-file=<path>` | Use a local JSON releases array instead of fetching GitHub (offline) |
+| `--skip-preflight` | Bypass the Docker/disk/queue pre-flight checks |
+
 ## Community & Resources
 
 - **Website:** [www.projectnomad.us](https://www.projectnomad.us) - Learn more about the project
 - **Discord:** [Join the Community](https://discord.com/invite/crosstalksolutions) - Get help, share your builds, and connect with other NOMAD users
 - **Benchmark Leaderboard:** [benchmark.projectnomad.us](https://benchmark.projectnomad.us) - See how your hardware stacks up against other NOMAD builds
-- **Troubleshooting Guide:** [TROUBLESHOOTING.md](TROUBLESHOOTING.md) - Find solutions to common issues
 - **FAQ:** [FAQ.md](FAQ.md) - Find answers to frequently asked questions
 - **Community Add-Ons:** [admin/docs/community-add-ons.md](admin/docs/community-add-ons.md) - Third-party content packs built by the community
 
 ## License
 
-Project N.O.M.A.D. is licensed under the [Apache License 2.0](LICENSE).
+Project NOMAD is licensed under the [Apache License 2.0](LICENSE).
 
 ## Helper Scripts
-Once installed, Project N.O.M.A.D. has a few helper scripts should you ever need to troubleshoot issues or perform maintenance that can't be done through the Command Center. All of these scripts are found in Project N.O.M.A.D.'s install directory, `/opt/project-nomad`
+Once installed, Project NOMAD has a few helper scripts should you ever need to troubleshoot issues or perform maintenance that can't be done through the Command Center. All of these scripts are found in Project NOMAD's install directory, `/opt/project-nomad`
 
 ###
 

admin/.env.example

@@ -1,6 +1,10 @@
 PORT=8080
 HOST=localhost
 LOG_LEVEL=info
+# Optional: override the URL used to test internet connectivity.
+# Defaults to https://1.1.1.1/cdn-cgi/trace with fallbacks to hosts the app
+# already contacts. Leave unset to use the defaults.
+# INTERNET_STATUS_TEST_URL=https://1.1.1.1/cdn-cgi/trace
 APP_KEY=some_random_key
 NODE_ENV=development
 SESSION_DRIVER=cookie
@@ -12,7 +16,15 @@ DB_PASSWORD=password
 DB_SSL=false
 REDIS_HOST=localhost
 REDIS_PORT=6379
+# Optional: Redis logical database index (0-15). Defaults to 0 if unset.
+# Set this when sharing a Redis instance across services to avoid key collisions.
+# REDIS_DB=0
 # Storage path for NOMAD content (ZIM files, maps, etc.)
 # On Windows dev, use an absolute path like: C:/nomad-storage
 # On Linux production, use: /opt/project-nomad/storage
-NOMAD_STORAGE_PATH=/opt/project-nomad/storage
+NOMAD_STORAGE_PATH=/opt/project-nomad/storage
+# Creator Packs (gated content downloads). Official release builds inject
+# CREATOR_PACKS_APP_KEY at build time; leave unset for a build that can't install
+# packs. CREATOR_PACKS_WORKER_BASE overrides the entitlement Worker origin.
+# CREATOR_PACKS_APP_KEY=
+# CREATOR_PACKS_WORKER_BASE=https://nomad-packs-worker.chris-556.workers.dev

admin/app/controllers/benchmark_controller.ts

@@ -184,9 +184,22 @@ export default class BenchmarkController {
       // Pass through the status code from the service if available, otherwise default to 400
       const statusCode = (error as any).statusCode || 400
       logger.error({ err: error }, '[BenchmarkController] Benchmark submit failed')
+
+      // Surface a clear, actionable reason to the UI instead of a generic failure.
+      // The rate limiter (429) is the most common cause, so name it explicitly;
+      // otherwise pass through the underlying detail (repository error or a service
+      // validation message) and fall back to a safe generic only when we have none.
+      let errorMessage: string
+      if (statusCode === 429) {
+        errorMessage = 'You can only submit one benchmark per hour. Please wait a bit and try again.'
+      } else {
+        errorMessage =
+          (error as any).detail || (error as any).message || 'Failed to submit benchmark results.'
+      }
+
       return response.status(statusCode).send({
         success: false,
-        error: 'Failed to submit benchmark results.',
+        error: errorMessage,
       })
     }
   }
@@ -248,6 +261,13 @@ export default class BenchmarkController {
     return this.benchmarkService.getStatus()
   }
 
+  /**
+   * Whether to show the dashboard "re-run under Score v2" banner
+   */
+  async rerunBanner({}: HttpContext) {
+    return { show: await this.benchmarkService.shouldShowRerunBanner() }
+  }
+
   /**
    * Get benchmark settings
    */

admin/app/controllers/conditions_controller.ts

@@ -0,0 +1,114 @@
+import type { HttpContext } from '@adonisjs/core/http'
+import logger from '@adonisjs/core/services/logger'
+import { ConditionService } from '#services/condition_service'
+import { conditionDrugsValidator } from '#validators/conditions'
+import { affirmativeRemediesEnabled } from '../utils/affirmative_remedies.js'
+
+/**
+ * "When to use what" — condition-first HTTP boundary (Phase 1).
+ *
+ * Two Inertia pages (index / show) + a small JSON API (drugs). Mirrors the
+ * DrugReferenceController chain:
+ *   - index/show render Inertia
+ *   - the JSON action returns a plain object
+ *   - slug guard on show (404 when not in the curated spine)
+ *   - never leak exceptions to the UI
+ */
+export default class ConditionsController {
+  private get service() {
+    return new ConditionService()
+  }
+
+  /**
+   * GET /conditions — legacy browse route.
+   * Situation browsing now lives directly on the unified Drug Reference page, so
+   * the standalone browse route permanently redirects there. Any old bookmark or
+   * in-app link lands on the same content. The condition detail route
+   * (/conditions/:slug) is unchanged — situation chips still deep-link to it via
+   * /drug-reference?situation=<slug>.
+   */
+  async index({ response }: HttpContext) {
+    return response.redirect('/drug-reference')
+  }
+
+  /**
+   * GET /conditions/:slug — condition detail page.
+   * 404s when the slug is not a curated condition.
+   */
+  async show({ inertia, params, response }: HttpContext) {
+    const slug = String(params.slug ?? '').trim()
+    if (!slug) {
+      return response.notFound({ error: 'invalid condition' })
+    }
+
+    try {
+      const condition = this.service.findCondition(slug)
+      if (!condition) {
+        return response.notFound({ error: 'Condition not found' })
+      }
+
+      const [result, drugRowCount, remediesOn] = await Promise.all([
+        this.service.drugsForSlug(slug),
+        this.service.drugRowCount(),
+        affirmativeRemediesEnabled(),
+      ])
+
+      return inertia.render('conditions/show', {
+        condition: result?.condition ?? null,
+        drugs: result?.drugs ?? [],
+        // Affirmative remedies gated off by default (#1040); the OTC/condition
+        // match above is regulated label text and stays live.
+        remedies: remediesOn ? (result?.remedies ?? []) : [],
+        drugRowCount,
+      })
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[ConditionsController] show(${slug}) failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not load condition' })
+    }
+  }
+
+  /**
+   * GET /api/conditions/drugs?slug=… | ?q=…
+   * Returns { condition, drugs } for a curated condition or a free-text
+   * situation. Requires exactly one of slug/q.
+   */
+  async drugsApi({ request, response }: HttpContext) {
+    try {
+      const params = await request.validateUsing(conditionDrugsValidator)
+
+      if (params.slug && params.q) {
+        return response.badRequest({ error: 'Provide either slug or q, not both' })
+      }
+
+      const filterOpts = {
+        route: params.route,
+        sort: params.sort,
+      }
+
+      // Strip affirmative remedies from the situation-search response when the
+      // gate is closed (#1040); the OTC drug matches are regulated label text and
+      // are returned either way.
+      const remediesOn = await affirmativeRemediesEnabled()
+
+      if (params.slug) {
+        const result = await this.service.drugsForSlug(params.slug, params.limit, filterOpts)
+        if (!result) {
+          return response.notFound({ error: 'Condition not found' })
+        }
+        return remediesOn ? result : { ...result, remedies: [] }
+      }
+
+      if (params.q) {
+        const result = await this.service.drugsForFreeText(params.q, params.limit, filterOpts)
+        return remediesOn ? result : { ...result, remedies: [] }
+      }
+
+      return response.badRequest({ error: 'Provide a slug or q query parameter' })
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.warn(`[ConditionsController] drugsApi failed: ${msg}`)
+      return response.badRequest({ error: msg })
+    }
+  }
+}

admin/app/controllers/creator_packs_controller.ts

@@ -0,0 +1,62 @@
+import { CreatorPackService } from '#services/creator_pack_service'
+import { DownloadService } from '#services/download_service'
+import { QueueService } from '#services/queue_service'
+import logger from '@adonisjs/core/services/logger'
+import type { HttpContext } from '@adonisjs/core/http'
+
+export default class CreatorPacksController {
+  private creatorPackService = new CreatorPackService()
+
+  /**
+   * Catalog with per-pack status, plus live progress for any in-flight pack
+   * downloads (ZIM download jobs — filtered client-side by collection_ref once
+   * the UI needs it; for now the raw zim job list is enough for progress).
+   */
+  async index({ response }: HttpContext) {
+    try {
+      const configured = this.creatorPackService.isConfigured()
+      const packs = await this.creatorPackService.listPacksWithStatus()
+      const downloadService = new DownloadService(QueueService.getInstance())
+      const downloads = await downloadService.listDownloadJobs('zim')
+      return { configured, packs, downloads }
+    } catch (error: any) {
+      logger.error('[CreatorPacksController] Failed to list creator packs:', error?.message || error)
+      return response.status(500).send({ message: 'Failed to load creator packs' })
+    }
+  }
+
+  async install({ params, response }: HttpContext) {
+    const packId = params.id as string
+    const result = await this.creatorPackService.installPack(packId)
+
+    switch (result.code) {
+      case 'dispatched':
+        return response.status(202).send({
+          message: 'Pack download started',
+          filename: result.filename,
+        })
+      case 'already_installed':
+        return { message: 'Pack is already installed' }
+      case 'already_downloading':
+        return { message: 'Pack download is already in progress' }
+      case 'not_found':
+        return response.status(404).send({ message: `Creator pack not found: ${packId}` })
+      case 'not_configured':
+        return response.status(503).send({
+          message: 'Creator Packs are not configured on this server',
+        })
+    }
+  }
+
+  async uninstall({ params, response }: HttpContext) {
+    const packId = params.id as string
+    const result = await this.creatorPackService.uninstallPack(packId)
+
+    switch (result.code) {
+      case 'uninstalled':
+        return { message: 'Pack uninstalled', filename: result.filename }
+      case 'not_installed':
+        return response.status(404).send({ message: `Creator pack is not installed: ${packId}` })
+    }
+  }
+}

admin/app/controllers/downloads_controller.ts

@@ -24,4 +24,8 @@ export default class DownloadsController {
   async cancelJob({ params }: HttpContext) {
     return this.downloadService.cancelJob(params.jobId)
   }
+
+  async retryJob({ params }: HttpContext) {
+    return this.downloadService.retryFailedJob(params.jobId)
+  }
 }

admin/app/controllers/drug_reference_controller.ts

@@ -0,0 +1,295 @@
+import type { HttpContext } from '@adonisjs/core/http'
+import logger from '@adonisjs/core/services/logger'
+import { DrugReferenceService } from '#services/drug_reference_service'
+import { ConditionService } from '#services/condition_service'
+import { searchDrugValidator, interactionsValidator } from '#validators/drug_reference'
+import { parseCompareIds } from '../../util/compare_ids.js'
+import { situationsForIndications } from '../../util/conditions.js'
+import { affirmativeRemediesEnabled } from '../utils/affirmative_remedies.js'
+
+/**
+ * Drug Reference v1 — HTTP boundary.
+ *
+ * Two Inertia pages (index / show) + a small JSON API (search / status /
+ * download). Mirrors the WorkshopController / InventoryController chain:
+ *   - index/show render Inertia
+ *   - JSON actions return plain objects
+ *   - Integer-id guard on show
+ *   - Never leak exceptions to the UI
+ */
+export default class DrugReferenceController {
+  private get service() {
+    return new DrugReferenceService()
+  }
+
+  /**
+   * GET /drug-reference — unified search page.
+   * Passes the current row count and ingest status so the empty-state
+   * "download first" prompt can render server-side. Also passes the curated
+   * condition spine so the always-visible situation chips (and the situation→
+   * drugs direction of the unified surface) can render server-side.
+   */
+  async index({ inertia }: HttpContext) {
+    try {
+      const conditionService = new ConditionService()
+      const [status, count, remediesOn] = await Promise.all([
+        this.service.getIngestStatus(),
+        this.service.rowCount(),
+        affirmativeRemediesEnabled(),
+      ])
+
+      return inertia.render('drug-reference/index', {
+        ingestStatus: status,
+        rowCount: count,
+        conditions: conditionService.listConditions(),
+        // Affirmative remedy content is gated off by default (#1040): keep it out
+        // of the payload entirely when disabled, and tell the page so it can hide
+        // the "Natural" filter too. Drug search + condition matching are unaffected.
+        remedies: remediesOn ? conditionService.listRemedies() : [],
+        remediesEnabled: remediesOn,
+      })
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] index failed: ${msg}`)
+      return inertia.render('drug-reference/index', {
+        ingestStatus: null,
+        rowCount: 0,
+        conditions: [],
+        remedies: [],
+        remediesEnabled: false,
+      })
+    }
+  }
+
+  /**
+   * GET /drug-reference/:id — detail page.
+   */
+  async show({ inertia, params, response }: HttpContext) {
+    const id = Number(params.id)
+    if (!Number.isInteger(id) || id <= 0) {
+      return response.notFound({ error: 'invalid id' })
+    }
+
+    try {
+      const label = await this.service.find(id)
+      if (!label) {
+        return response.notFound({ error: 'Drug label not found' })
+      }
+
+      // Reverse link — the other direction of the symbiotic relationship: which
+      // curated situations does THIS label's indications text treat? Matched
+      // server-side against the curated spine so searchTerms stay server-only.
+      const situations = situationsForIndications(
+        label.indications,
+        new ConditionService().allConditions()
+      )
+
+      return inertia.render('drug-reference/show', { label, situations })
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] show(${id}) failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not load drug label' })
+    }
+  }
+
+  /**
+   * GET /api/drug-reference/search
+   * Returns a slim collapsed result list (brand+generic pairs).
+   */
+  async search({ request, response }: HttpContext) {
+    try {
+      const params = await request.validateUsing(searchDrugValidator)
+      const results = await this.service.search(params.q, {
+        productType: params.product_type,
+        route: params.route,
+        sort: params.sort,
+        limit: params.limit,
+        offset: params.offset,
+        scope: params.scope,
+      })
+      return { results }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.warn(`[DrugReferenceController] search failed: ${msg}`)
+      return response.badRequest({ error: msg })
+    }
+  }
+
+  /**
+   * GET /api/drug-reference/status
+   * Returns the live ingest status DTO.
+   */
+  async status({ response }: HttpContext) {
+    try {
+      const status = await this.service.getIngestStatus()
+      return status
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] status failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not read ingest status' })
+    }
+  }
+
+  /**
+   * GET /drug-reference/interactions — side-by-side label comparison page.
+   * Passes rowCount + ingestStatus so the empty-state prompt can render,
+   * mirroring the index() pattern. The actual entry data is loaded client-side
+   * via /api/drug-reference/interactions?ids=… so the page is shareable via URL.
+   */
+  async interactions({ inertia }: HttpContext) {
+    try {
+      const [status, count] = await Promise.all([
+        this.service.getIngestStatus(),
+        this.service.rowCount(),
+      ])
+
+      return inertia.render('drug-reference/interactions', {
+        ingestStatus: status,
+        rowCount: count,
+      })
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] interactions page failed: ${msg}`)
+      return inertia.render('drug-reference/interactions', {
+        ingestStatus: null,
+        rowCount: 0,
+      })
+    }
+  }
+
+  /**
+   * GET /api/drug-reference/interactions?ids=1,2,3
+   * Validates → parses → fetches and returns { entries: DrugInteractionEntry[] }.
+   * Never leaks exceptions; integer-guards ids via parseCompareIds.
+   */
+  async interactionsApi({ request, response }: HttpContext) {
+    try {
+      const params = await request.validateUsing(interactionsValidator)
+      const ids = parseCompareIds(params.ids ?? '')
+      const entries = await this.service.getInteractionsFor(ids)
+      return { entries }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.warn(`[DrugReferenceController] interactionsApi failed: ${msg}`)
+      return response.badRequest({ error: msg })
+    }
+  }
+
+  /**
+   * POST /api/drug-reference/download
+   * Triggers the download phase (idempotent — deduped on deterministic jobId).
+   * The download auto-chains the ingest phase on completion.
+   */
+  async download({ response }: HttpContext) {
+    try {
+      const result = await this.service.triggerDownload()
+      return { success: true, created: result.created, message: result.message }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] download trigger failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not trigger download' })
+    }
+  }
+
+  /**
+   * POST /api/drug-reference/ingest
+   * Manually (re-)runs the ingest phase from the already-downloaded on-disk
+   * parts, with no re-download. Returns 404 when nothing is on disk so the UI
+   * can keep its guard honest even if the button is reached out of band.
+   */
+  async ingest({ response }: HttpContext) {
+    try {
+      const result = await this.service.triggerIngestFromDisk()
+      if (result.nothingDownloaded) {
+        return response.notFound({ error: result.message })
+      }
+      return { success: true, created: result.created, message: result.message }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] ingest trigger failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not trigger ingest' })
+    }
+  }
+
+  /**
+   * POST /api/drug-reference/reset-ingest
+   * Force-clears a wedged ingest job (e.g. one left 'active' by a worker killed
+   * mid-ingest during an upgrade) and restarts it from the on-disk parts. The
+   * escape hatch for a stuck "Indexing…" state.
+   */
+  async resetIngest({ response }: HttpContext) {
+    try {
+      const result = await this.service.resetAndReingest()
+      if (result.nothingDownloaded) {
+        return response.notFound({ error: result.message })
+      }
+      return { success: true, created: result.created, message: result.message }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] reset-ingest failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not reset ingest' })
+    }
+  }
+
+  /**
+   * POST /api/drug-reference/uninstall
+   * Uninstall the offline FDA drug dataset: stop in-flight jobs, delete on-disk
+   * parts, TRUNCATE drug_labels, clear KV markers, and remove the install-state
+   * row (which auto-hides the home tiles). The curated-tier "remove" action.
+   * Reports partial failures rather than masking them.
+   */
+  async uninstall({ response }: HttpContext) {
+    try {
+      const result = await this.service.uninstall()
+      if (!result.success) {
+        return response.internalServerError({
+          success: false,
+          rowsDropped: result.rowsDropped,
+          error: result.message,
+        })
+      }
+      return { success: true, rowsDropped: result.rowsDropped, message: result.message }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceController] uninstall failed: ${msg}`)
+      return response.internalServerError({ error: 'Could not uninstall drug reference' })
+    }
+  }
+
+  /**
+   * GET /api/drug-reference/ingest-log
+   * Tail the persisted app log for ingest/download lines. In production the logger
+   * writes JSON to /app/storage/logs/admin.log (both the admin and worker
+   * containers share that volume), so the worker's [IngestDrugDataJob] trace lands
+   * there even when its stdout never reaches the log viewer. This surfaces it over
+   * HTTP so the exact stall stage (zip-open vs first-record vs batch) is visible
+   * without container access. Reads only the last slice of the file.
+   */
+  async ingestLog({ request, response }: HttpContext) {
+    const LOG_PATH = '/app/storage/logs/admin.log'
+    const TAIL_BYTES = 128 * 1024
+    const limit = Math.min(Number(request.input('lines', 400)) || 400, 2000)
+    try {
+      const { stat, open } = await import('node:fs/promises')
+      const st = await stat(LOG_PATH)
+      const start = Math.max(0, st.size - TAIL_BYTES)
+      const fh = await open(LOG_PATH, 'r')
+      try {
+        const buf = Buffer.alloc(st.size - start)
+        await fh.read(buf, 0, buf.length, start)
+        const all = buf.toString('utf8').split('\n')
+        // Keep only ingest/download/worker-relevant lines; for JSON pino lines the
+        // substring match still works against the embedded "msg" field.
+        const re =
+          /IngestDrugDataJob|DownloadDrugDataJob|DrugReference|drug-ingest|drug-download|unhandledRejection|uncaughtException|queue:work|stalled/i
+        const matched = all.filter((l) => re.test(l)).slice(-limit)
+        return { ok: true, path: LOG_PATH, size: st.size, count: matched.length, lines: matched }
+      } finally {
+        await fh.close()
+      }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      return response.notFound({ ok: false, path: LOG_PATH, error: msg })
+    }
+  }
+}

admin/app/controllers/home_controller.ts

@@ -1,4 +1,6 @@
 import { SystemService } from '#services/system_service'
+import { DrugReferenceService } from '#services/drug_reference_service'
+import logger from '@adonisjs/core/services/logger'
 import { inject } from '@adonisjs/core'
 import type { HttpContext } from '@adonisjs/core/http'
 
@@ -18,7 +20,38 @@ export default class HomeController {
         return inertia.render('home', {
             system: {
                 services
-            }
+            },
+            // Gate the Drug Reference / "When to use what" tiles behind the FDA
+            // dataset install state. Installed when the curated-tier ingest has
+            // reached 'ready' OR an install is in flight (downloading/ingesting) —
+            // so the tile appears the moment the user opts in and persists through
+            // the long install, rather than popping in only at the very end.
+            drugReferenceInstalled: await this.computeDrugReferenceInstalled(),
         })
     }
+
+    /**
+     * True when the offline FDA drug dataset is installed or installing. Reads the
+     * two-phase ingest status: ready (fully installed) or an active phase
+     * (downloading/downloaded/ingesting). rowCount > 0 covers a populated table
+     * whose job history was pruned. Never throws — a status read failure hides the
+     * tiles (fail-closed) rather than 500-ing the dashboard.
+     */
+    private async computeDrugReferenceInstalled(): Promise<boolean> {
+        try {
+            const status = await new DrugReferenceService().getIngestStatus()
+            const installing =
+                status.phase === 'downloading' ||
+                status.phase === 'downloaded' ||
+                status.phase === 'ingesting'
+            return status.phase === 'ready' || installing || status.rowCount > 0
+        } catch (err) {
+            logger.error(
+                `[HomeController] drug-reference install check failed: ${
+                    err instanceof Error ? err.message : String(err)
+                }`
+            )
+            return false
+        }
+    }
 }

admin/app/controllers/maps_controller.ts

@@ -19,10 +19,14 @@ export default class MapsController {
 
   async index({ inertia }: HttpContext) {
     const baseAssetsCheck = await this.mapService.ensureBaseAssets()
-    const regionFiles = await this.mapService.listRegions()
+    const [regionFiles, worldBasemapExists] = await Promise.all([
+      this.mapService.listRegions(),
+      this.mapService.checkWorldBasemapExists(),
+    ])
     return inertia.render('maps', {
       maps: {
         baseAssetsExist: baseAssetsCheck,
+        worldBasemapExists,
         regionFiles: regionFiles.files,
       },
     })
@@ -35,6 +39,24 @@ export default class MapsController {
     return { success: true }
   }
 
+  async setupWorldBasemap({ response }: HttpContext) {
+    try {
+      const ready = await this.mapService.provisionWorldBasemap()
+      if (!ready) {
+        return response.status(500).send({
+          message:
+            'Could not download the base map. Please connect this NOMAD to the internet and try again.',
+        })
+      }
+      return { success: true }
+    } catch {
+      return response.status(500).send({
+        message:
+          'Could not download the base map. Please connect this NOMAD to the internet and try again.',
+      })
+    }
+  }
+
   async downloadRemote({ request }: HttpContext) {
     const payload = await request.validateUsing(remoteDownloadValidator)
     assertNotPrivateUrl(payload.url)

admin/app/controllers/nomad_md_controller.ts

@@ -0,0 +1,22 @@
+import { NomadMdService } from '#services/nomad_md_service'
+import { updateNomadMdSchema } from '#validators/nomad_md'
+import { inject } from '@adonisjs/core'
+import type { HttpContext } from '@adonisjs/core/http'
+
+@inject()
+export default class NomadMdController {
+  constructor(private nomadMdService: NomadMdService) {}
+
+  async show({ response }: HttpContext) {
+    const content = await this.nomadMdService.read()
+    return response.status(200).json({ content })
+  }
+
+  async update({ request, response }: HttpContext) {
+    const { content } = await request.validateUsing(updateNomadMdSchema)
+    // Empty request strings arrive as undefined (AdonisJS null-coerces them);
+    // treat that as an explicit "clear the file".
+    await this.nomadMdService.write(content ?? '')
+    return response.status(200).json({ success: true, message: 'NOMAD.md saved successfully' })
+  }
+}

admin/app/controllers/ollama_controller.ts

@@ -1,5 +1,6 @@
 import { ChatService } from '#services/chat_service'
 import { DockerService } from '#services/docker_service'
+import { NomadMdService } from '#services/nomad_md_service'
 import { OllamaService } from '#services/ollama_service'
 import { RagService } from '#services/rag_service'
 import Service from '#models/service'
@@ -20,7 +21,8 @@ export default class OllamaController {
     private chatService: ChatService,
     private dockerService: DockerService,
     private ollamaService: OllamaService,
-    private ragService: RagService
+    private ragService: RagService,
+    private nomadMdService: NomadMdService
   ) { }
 
   async availableModels({ request }: HttpContext) {
@@ -71,16 +73,28 @@ export default class OllamaController {
         reqData.messages.unshift(systemPrompt)
       }
 
+      // Inject the user-managed NOMAD.md as its own leading system message so the
+      // user's persistent instructions take precedence, while the default
+      // formatting prompt and any RAG context below remain intact. A missing or
+      // blank file yields null and changes nothing.
+      const nomadPrompt = await this.nomadMdService.getSystemPrompt()
+      if (nomadPrompt) {
+        logger.debug('[OllamaController] Injecting NOMAD.md system prompt')
+        reqData.messages.unshift({ role: 'system' as const, content: nomadPrompt })
+      }
+
       // Query rewriting for better RAG retrieval with manageable context
       // Will return user's latest message if no rewriting is needed
       const rewrittenQuery = await this.rewriteQueryWithContext(reqData.messages, reqData.model)
 
       logger.debug(`[OllamaController] Rewritten query for RAG: "${rewrittenQuery}"`)
       if (rewrittenQuery) {
+        const collectionFilter: string | null = request.input('collection', null)
         const relevantDocs = await this.ragService.searchSimilarDocuments(
           rewrittenQuery,
           5, // Top 5 most relevant chunks
-          0.3 // Minimum similarity score of 0.3
+          0.3, // Minimum similarity score of 0.3
+          collectionFilter ?? undefined
         )
 
         logger.debug(`[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"`)
@@ -106,8 +120,17 @@ export default class OllamaController {
             `[RAG] Injecting ${trimmedDocs.length}/${relevantDocs.length} results (model: ${reqData.model}, maxResults: ${maxResults}, maxTokens: ${maxTokens || 'unlimited'})`
           )
 
+          // Label each context block with its source title when available (a neutral,
+          // honest provenance signal) but never the raw relevance score — nomic cosine
+          // scores for genuinely relevant passages sit ~0.4-0.6, and surfacing e.g.
+          // "42%" primes the model to distrust correct context. Scores stay in the logs
+          // above for debugging.
           const contextText = trimmedDocs
-            .map((doc, idx) => `[Context ${idx + 1}] (Relevance: ${(doc.score * 100).toFixed(1)}%)\n${doc.text}`)
+            .map((doc, idx) => {
+              const title = doc.metadata?.full_title || doc.metadata?.article_title
+              const label = title ? `[Context ${idx + 1} — ${title}]` : `[Context ${idx + 1}]`
+              return `${label}\n${doc.text}`
+            })
             .join('\n\n')
 
           const systemMessage = {
@@ -135,13 +158,21 @@ export default class OllamaController {
         logger.debug(`[OllamaController] Large system prompt (~${estimatedSystemTokens} tokens), requesting num_ctx: ${numCtx}`)
       }
 
-      // Check if the model supports "thinking" capability for enhanced response generation
+      // Check if the model supports "thinking" capability for enhanced response generation.
+      // Thinking is only enabled when the model supports it AND the user wants it: the explicit
+      // per-request preference wins, otherwise the global default (ai.autoThinking, default OFF).
       // If gpt-oss model, it requires a text param for "think" https://docs.ollama.com/api/chat
       const thinkingCapability = await this.ollamaService.checkModelHasThinking(reqData.model)
-      const think: boolean | 'medium' = thinkingCapability ? (reqData.model.startsWith('gpt-oss') ? 'medium' : true) : false
+      let thinkingEnabled = false
+      if (thinkingCapability) {
+        thinkingEnabled = reqData.think ?? ((await KVStore.getValue('ai.autoThinking')) ?? false)
+      }
+      const think: boolean | 'medium' =
+        thinkingEnabled ? (reqData.model.startsWith('gpt-oss') ? 'medium' : true) : false
 
-      // Separate sessionId from the Ollama request payload — Ollama rejects unknown fields
+      // Separate sessionId and the resolved thinking preference from the Ollama request payload —
-      const { sessionId, ...ollamaRequest } = reqData
+      // Ollama rejects unknown fields, and `think` is re-derived above (not forwarded raw).
+      const { sessionId, think: _thinkPref, ...ollamaRequest } = reqData
 
       // Save user message to DB before streaming if sessionId provided
       let userContent: string | null = null
@@ -155,14 +186,33 @@ export default class OllamaController {
 
       if (reqData.stream) {
         logger.debug(`[OllamaController] Initiating streaming response for model: "${reqData.model}" with think: ${think}`)
-        // Headers already flushed above
+        // Headers already flushed above.
-        const stream = await this.ollamaService.chatStream({ ...ollamaRequest, think, numCtx })
+        // Abort the upstream generation if the client disconnects — otherwise an abandoned
+        // request keeps decoding server-side and, with Ollama's default OLLAMA_NUM_PARALLEL=1,
+        // blocks every later chat/RAG request until the model is manually stopped (#1065).
+        const abortController = new AbortController()
+        response.response.on('close', () => abortController.abort())
+        const stream = await this.ollamaService.chatStream({
+          ...ollamaRequest,
+          think,
+          thinkingCapable: thinkingCapability,
+          numCtx,
+          signal: abortController.signal,
+        })
         let fullContent = ''
-        for await (const chunk of stream) {
+        try {
-          if (chunk.message?.content) {
+          for await (const chunk of stream) {
-            fullContent += chunk.message.content
+            if (chunk.message?.content) {
+              fullContent += chunk.message.content
+            }
+            response.response.write(`data: ${JSON.stringify(chunk)}\n\n`)
           }
-          response.response.write(`data: ${JSON.stringify(chunk)}\n\n`)
+        } catch (err) {
+          if (abortController.signal.aborted) {
+            logger.debug('[OllamaController] Client disconnected; aborted upstream Ollama generation')
+            return
+          }
+          throw err
         }
         response.response.end()
 
@@ -180,7 +230,7 @@ export default class OllamaController {
       }
 
       // Non-streaming (legacy) path
-      const result = await this.ollamaService.chat({ ...ollamaRequest, think, numCtx })
+      const result = await this.ollamaService.chat({ ...ollamaRequest, think, thinkingCapable: thinkingCapability, numCtx })
 
       if (sessionId && result?.message?.content) {
         await this.chatService.addMessage(sessionId, 'assistant', result.message.content)
@@ -360,7 +410,14 @@ export default class OllamaController {
   }
 
   async installedModels({ }: HttpContext) {
-    return await this.ollamaService.getModels()
+    const models = await this.ollamaService.getModels()
+    // Enrich each model with its thinking capability so the chat picker knows which models
+    // to show the per-model thinking toggle for. checkModelHasThinking memoizes /api/show
+    // results, so this stays cheap on repeat loads. Best-effort per model.
+    const thinking = await Promise.all(
+      models.map((m) => this.ollamaService.checkModelHasThinking(m.name))
+    )
+    return models.map((m, i) => ({ ...m, thinking: thinking[i] }))
   }
 
   /**
@@ -447,3 +504,4 @@ export default class OllamaController {
     }
   }
 }
+

admin/app/controllers/openapi_controller.ts

@@ -0,0 +1,74 @@
+import { createRequire } from 'node:module'
+import { readFileSync } from 'node:fs'
+import type { HttpContext } from '@adonisjs/core/http'
+import { buildOpenApiDocument } from '#start/openapi/generator'
+
+const require = createRequire(import.meta.url)
+
+/**
+ * The self-contained Scalar browser bundle, read from the installed
+ * `@scalar/api-reference` package and cached in memory. Served from this app
+ * (not a CDN) so the API reference works on an offline appliance.
+ */
+let scalarBundle: string | null = null
+
+function getScalarBundle(): string {
+  if (scalarBundle === null) {
+    const entry = require.resolve('@scalar/api-reference')
+    const marker = '/node_modules/@scalar/api-reference/'
+    const root = entry.slice(0, entry.lastIndexOf(marker) + marker.length - 1)
+    scalarBundle = readFileSync(`${root}/dist/browser/standalone.js`, 'utf-8')
+  }
+  return scalarBundle
+}
+
+/**
+ * The reference page mounts Scalar against our locally served bundle and spec.
+ * Scalar auto-detects the `#api-reference` element and reads `data-url`.
+ */
+const REFERENCE_HTML = `<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <title>Nomad Admin API Reference</title>
+  </head>
+  <body>
+    <div id="app"></div>
+    <script src="/reference/assets/standalone.js"></script>
+    <script>
+      Scalar.createApiReference('#app', {
+        url: '/api/openapi.json',
+        hideClientButton: true,
+        telemetry: false,
+        theme: 'saturn',
+      })
+    </script>
+  </body>
+</html>`
+
+export default class OpenApiController {
+  /**
+   * The generated OpenAPI 3.1 document.
+   */
+  async spec({ response }: HttpContext) {
+    return response.json(buildOpenApiDocument())
+  }
+
+  /**
+   * The interactive Scalar API reference page.
+   */
+  async reference({ response }: HttpContext) {
+    return response.header('content-type', 'text/html').send(REFERENCE_HTML)
+  }
+
+  /**
+   * The locally bundled Scalar JS (kept off any CDN for offline use).
+   */
+  async standalone({ response }: HttpContext) {
+    return response
+      .header('content-type', 'application/javascript; charset=utf-8')
+      .header('cache-control', 'public, max-age=86400')
+      .send(getScalarBundle())
+  }
+}

admin/app/controllers/rag_controller.ts

@@ -7,8 +7,9 @@ import app from '@adonisjs/core/services/app'
 import { randomBytes } from 'node:crypto'
 import { sanitizeFilename } from '../utils/fs.js'
 import { basename } from 'node:path'
-import { deleteFileSchema, embedFileSchema, estimateBatchSchema, getJobStatusSchema } from '#validators/rag'
+import { deleteFileSchema, embedFileSchema, estimateBatchSchema, fileSourceSchema, getJobStatusSchema } from '#validators/rag'
 import logger from '@adonisjs/core/services/logger'
+import { sanitizeCollectionName } from '../../constants/kb_collections.js'
 
 @inject()
 export default class RagController {
@@ -20,6 +21,8 @@ export default class RagController {
       return response.status(400).json({ error: 'No file uploaded' })
     }
 
+    const collection = sanitizeCollectionName(request.input('collection', null))
+
     const randomSuffix = randomBytes(6).toString('hex')
     const sanitizedName = sanitizeFilename(uploadedFile.clientName)
 
@@ -34,6 +37,7 @@ export default class RagController {
     const result = await EmbedFileJob.dispatch({
       filePath: fullPath,
       fileName,
+      ...(collection ? { collection } : {}),
     })
 
     return response.status(202).json({
@@ -42,9 +46,9 @@ export default class RagController {
       fileName,
       filePath: `/${RagService.UPLOADS_STORAGE_PATH}/${fileName}`,
       alreadyProcessing: !result.created,
+      ...(collection ? { collection } : {}),
     })
   }
-
   public async getActiveJobs({ response }: HttpContext) {
     const jobs = await EmbedFileJob.listActiveJobs()
     return response.status(200).json(jobs)
@@ -68,6 +72,57 @@ export default class RagController {
     return response.status(200).json({ files })
   }
 
+  public async getKnowledgeCollections({ response }: HttpContext) {
+    const collections = await this.ragService.getKnowledgeCollections()
+    return response.status(200).json({ collections })
+  }
+
+  public async updateFileCollection({ request, response }: HttpContext) {
+    const source: string | null = request.input('source', null)
+    // sanitizeCollectionName trims/lowercases/caps length, and returns null
+    // for empty input — which doubles as "clear back to Uncategorized".
+    const collection = sanitizeCollectionName(request.input('collection', null))
+
+    if (!source) {
+      return response.status(400).json({ error: 'source is required.' })
+    }
+
+    const result = await this.ragService.updateFileCollection(source, collection)
+    if (!result.success) {
+      return response.status(500).json({ error: result.message })
+    }
+    return response.status(200).json({ message: result.message })
+  }
+
+  public async renameKnowledgeCollection({ request, response }: HttpContext) {
+    const oldName = sanitizeCollectionName(request.input('oldName', null))
+    const newName = sanitizeCollectionName(request.input('newName', null))
+
+    if (!oldName || !newName) {
+      return response.status(400).json({ error: 'oldName and newName are required.' })
+    }
+
+    const result = await this.ragService.renameKnowledgeCollection(oldName, newName)
+    if (!result.success) {
+      return response.status(500).json({ error: result.message })
+    }
+    return response.status(200).json({ message: result.message })
+  }
+
+  public async deleteKnowledgeCollection({ request, response }: HttpContext) {
+    const name = sanitizeCollectionName(request.input('name', null))
+
+    if (!name) {
+      return response.status(400).json({ error: 'name is required.' })
+    }
+
+    const result = await this.ragService.deleteKnowledgeCollection(name)
+    if (!result.success) {
+      return response.status(500).json({ error: result.message })
+    }
+    return response.status(200).json({ message: result.message })
+  }
+
   public async getFileWarnings({ response }: HttpContext) {
     const result = await this.ragService.computeFileWarnings()
     return response.status(200).json(result)
@@ -110,6 +165,14 @@ export default class RagController {
     })
   }
 
+  public async cancelAllJobs({ response }: HttpContext) {
+    const result = await EmbedFileJob.cancelAllJobs()
+    return response.status(200).json({
+      message: `Cancelled ${result.cancelled} job${result.cancelled !== 1 ? 's' : ''}${result.filesDeleted > 0 ? `, deleted ${result.filesDeleted} file${result.filesDeleted !== 1 ? 's' : ''}` : ''}.`,
+      ...result,
+    })
+  }
+
   public async policyPromptState({ response }: HttpContext) {
     const result = await this.ragService.getPolicyPromptState()
     return response.status(200).json(result)
@@ -162,4 +225,23 @@ export default class RagController {
     const result = await KbRatioRegistry.estimateBatch(normalized)
     return response.status(200).json(result)
   }
+
+  public async getFileContent({ request, response }: HttpContext) {
+    const { source } = await request.validateUsing(fileSourceSchema)
+    const result = await this.ragService.readFileContent(source)
+    if (!result) {
+      return response.status(404).json({ error: 'File not found or not viewable' })
+    }
+    return response.status(200).json(result)
+  }
+
+  public async downloadFile({ request, response }: HttpContext) {
+    const { source } = await request.validateUsing(fileSourceSchema)
+    const filePath = await this.ragService.resolveDownloadPath(source)
+    if (!filePath) {
+      return response.status(404).json({ error: 'File not found' })
+    }
+    const fileName = filePath.split(/[/\\]/).at(-1) ?? 'download'
+    return response.attachment(filePath, fileName)
+  }
 }

admin/app/controllers/settings_controller.ts

@@ -3,9 +3,10 @@ import { BenchmarkService } from '#services/benchmark_service'
 import { MapService } from '#services/map_service'
 import { OllamaService } from '#services/ollama_service'
 import { SystemService } from '#services/system_service'
-import { getSettingSchema, updateSettingSchema } from '#validators/settings'
+import { getSettingSchema, updateSettingSchema, validateSettingValue } from '#validators/settings'
 import { inject } from '@adonisjs/core'
 import type { HttpContext } from '@adonisjs/core/http'
+import env from '#start/env'
 
 @inject()
 export default class SettingsController {
@@ -44,10 +45,14 @@ export default class SettingsController {
 
   async maps({ inertia }: HttpContext) {
     const baseAssetsCheck = await this.mapService.ensureBaseAssets()
-    const regionFiles = await this.mapService.listRegions()
+    const [regionFiles, worldBasemapExists] = await Promise.all([
+      this.mapService.listRegions(),
+      this.mapService.checkWorldBasemapExists(),
+    ])
     return inertia.render('settings/maps', {
       maps: {
         baseAssetsExist: baseAssetsCheck,
+        worldBasemapExists,
         regionFiles: regionFiles.files,
       },
     })
@@ -65,6 +70,7 @@ export default class SettingsController {
     const aiAssistantCustomName = await KVStore.getValue('ai.assistantCustomName')
     const remoteOllamaUrl = await KVStore.getValue('ai.remoteOllamaUrl')
     const ollamaFlashAttention = await KVStore.getValue('ai.ollamaFlashAttention')
+    const autoThinking = await KVStore.getValue('ai.autoThinking')
     return inertia.render('settings/models', {
       models: {
         availableModels: availableModels?.models || [],
@@ -74,6 +80,7 @@ export default class SettingsController {
           aiAssistantCustomName: aiAssistantCustomName ?? '',
           remoteOllamaUrl: remoteOllamaUrl ?? '',
           ollamaFlashAttention: ollamaFlashAttention ?? true,
+          autoThinking: autoThinking ?? false,
         },
       },
     })
@@ -98,6 +105,10 @@ export default class SettingsController {
     return inertia.render('settings/zim/remote-explorer')
   }
 
+  async creatorPacks({ inertia }: HttpContext) {
+    return inertia.render('settings/creator-packs')
+  }
+
   async benchmark({ inertia }: HttpContext) {
     const latestResult = await this.benchmarkService.getLatestResult()
     const status = this.benchmarkService.getStatus()
@@ -110,6 +121,19 @@ export default class SettingsController {
     })
   }
 
+  async advanced({ inertia }: HttpContext) {
+    // When the env var is set it always takes precedence over the stored value,
+    // so surface that to the UI to disable the field and explain the override.
+    const envOverride = Boolean(env.get('INTERNET_STATUS_TEST_URL')?.trim())
+    const internetStatusTestUrl = await KVStore.getValue('system.internetStatusTestUrl')
+    return inertia.render('settings/advanced', {
+      advanced: {
+        internetStatusTestUrl: internetStatusTestUrl ?? '',
+        internetStatusTestUrlEnvOverride: envOverride,
+      },
+    })
+  }
+
   async getSetting({ request, response }: HttpContext) {
     const { key } = await getSettingSchema.validate({ key: request.qs().key });
     const value = await KVStore.getValue(key);
@@ -118,6 +142,10 @@ export default class SettingsController {
 
   async updateSetting({ request, response }: HttpContext) {
     const reqData = await request.validateUsing(updateSettingSchema)
+    const valueError = validateSettingValue(reqData.key, reqData.value)
+    if (valueError) {
+      return response.status(422).send({ success: false, message: valueError })
+    }
     await this.systemService.updateSetting(reqData.key, reqData.value)
     return response.status(200).send({ success: true, message: 'Setting updated successfully' })
   }

admin/app/controllers/supply_depot_controller.ts

@@ -0,0 +1,13 @@
+import { SystemService } from '#services/system_service'
+import { inject } from '@adonisjs/core'
+import type { HttpContext } from '@adonisjs/core/http'
+
+@inject()
+export default class SupplyDepotController {
+  constructor(private systemService: SystemService) {}
+
+  async index({ inertia }: HttpContext) {
+    const services = await this.systemService.getServices({ installedOnly: false })
+    return inertia.render('supply-depot', { system: { services } })
+  }
+}

admin/app/controllers/system_controller.ts

@@ -2,11 +2,38 @@ import { DockerService } from '#services/docker_service';
 import { SystemService } from '#services/system_service'
 import { SystemUpdateService } from '#services/system_update_service'
 import { ContainerRegistryService } from '#services/container_registry_service'
+import { AutoUpdateService } from '#services/auto_update_service'
+import { AppAutoUpdateService } from '#services/app_auto_update_service'
+import { ContentAutoUpdateService } from '#services/content_auto_update_service'
+import { DownloadService } from '#services/download_service'
+import { QueueService } from '#services/queue_service'
 import { CheckServiceUpdatesJob } from '#jobs/check_service_updates_job'
-import { affectServiceValidator, checkLatestVersionValidator, installServiceValidator, subscribeToReleaseNotesValidator, updateServiceValidator } from '#validators/system';
+import {
+  affectServiceValidator,
+  checkLatestVersionValidator,
+  customAppValidator,
+  deleteCustomAppValidator,
+  installServiceValidator,
+  preflightCustomValidator,
+  preflightValidator,
+  serviceLogsValidator,
+  subscribeToReleaseNotesValidator,
+  uninstallServiceValidator,
+  updateCustomAppValidator,
+  updateServiceValidator,
+  setServiceAutoUpdateValidator,
+  setServiceCustomUrlValidator,
+  normalizeCustomUrl,
+} from '#validators/system'
+import {
+  DEFAULT_CPUS,
+  DEFAULT_MEMORY_MB,
+  evaluateCustomApp,
+} from '#services/custom_app_guard'
 import { inject } from '@adonisjs/core'
 import type { HttpContext } from '@adonisjs/core/http'
 import logger from '@adonisjs/core/services/logger'
+import Service from '#models/service'
 
 @inject()
 export default class SystemController {
@@ -108,6 +135,82 @@ export default class SystemController {
         response.send({ logs });
     }
 
+    async getAutoUpdateStatus({ response }: HttpContext) {
+        // Construct inline reusing already-injected singletons + the QueueService
+        // singleton (its constructor is private to prevent Redis connection leaks,
+        // so we must not let the container new a fresh one).
+        const autoUpdateService = new AutoUpdateService(
+            this.dockerService,
+            new DownloadService(QueueService.getInstance()),
+            this.systemService,
+            this.systemUpdateService,
+            this.containerRegistryService
+        )
+
+        try {
+            const status = await autoUpdateService.getStatus()
+            response.send(status)
+        } catch (error) {
+            logger.error({ err: error }, '[SystemController] Failed to get auto-update status')
+            response.status(500).send({ error: 'Failed to retrieve auto-update status' })
+        }
+    }
+
+    async getAppAutoUpdateStatus({ response }: HttpContext) {
+        // Constructed inline reusing already-injected singletons + the QueueService
+        // singleton (its constructor is private to prevent Redis connection leaks),
+        // mirroring getAutoUpdateStatus. Apps need no SystemUpdateService (no sidecar).
+        const appAutoUpdateService = new AppAutoUpdateService(
+            this.dockerService,
+            new DownloadService(QueueService.getInstance()),
+            this.systemService,
+            this.containerRegistryService
+        )
+
+        try {
+            const status = await appAutoUpdateService.getStatus()
+            response.send(status)
+        } catch (error) {
+            logger.error({ err: error }, '[SystemController] Failed to get app auto-update status')
+            response.status(500).send({ error: 'Failed to retrieve app auto-update status' })
+        }
+    }
+
+    async getContentAutoUpdateStatus({ response }: HttpContext) {
+        // Mirrors getAppAutoUpdateStatus. Content auto-update needs only the
+        // DownloadService (for the active-download pre-flight); the catalog and
+        // collection-update services default-construct inside the service.
+        const contentAutoUpdateService = new ContentAutoUpdateService(
+            new DownloadService(QueueService.getInstance())
+        )
+
+        try {
+            const status = await contentAutoUpdateService.getStatus()
+            response.send(status)
+        } catch (error) {
+            logger.error({ err: error }, '[SystemController] Failed to get content auto-update status')
+            response.status(500).send({ error: 'Failed to retrieve content auto-update status' })
+        }
+    }
+
+    async setServiceAutoUpdate({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(setServiceAutoUpdateValidator)
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ error: `Service ${payload.service_name} not found` })
+        }
+
+        service.auto_update_enabled = payload.enabled
+        // Re-enabling clears any prior self-disable so the app gets a fresh start.
+        if (payload.enabled) {
+            service.auto_update_consecutive_failures = 0
+            service.auto_update_disabled_reason = null
+        }
+        await service.save()
+
+        return response.send({ success: true, message: 'App auto-update preference updated' })
+    }
+
 
     async subscribeToReleaseNotes({ request }: HttpContext) {
         const reqData = await request.validateUsing(subscribeToReleaseNotesValidator);
@@ -180,4 +283,525 @@ export default class SystemController {
             return 'amd64'
         }
     }
+
+    /**
+     * Pre-install preflight check: reports port conflicts and resource warnings for a service.
+     * Results are advisory — the UI shows warnings but allows the user to force-proceed.
+     */
+    async preflightCheck({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(preflightValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ error: `Service ${payload.service_name} not found` })
+        }
+
+        // Extract host ports from container_config — the MySQL driver may return JSON columns
+        // as an already-parsed object rather than a string, so guard before calling JSON.parse.
+        const rawConfig = service.container_config
+        const config = rawConfig
+            ? typeof rawConfig === 'object'
+                ? rawConfig
+                : JSON.parse(rawConfig as string)
+            : null
+        const portBindings: Record<string, [{ HostPort: string }]> =
+            config?.HostConfig?.PortBindings ?? {}
+        const hostPorts = Object.values(portBindings)
+            .flat()
+            .map((b) => parseInt(b.HostPort, 10))
+            .filter((p) => !isNaN(p))
+
+        // Parse resource requirements from metadata (same object-guard as container_config)
+        let minMemoryMB = 256
+        let minDiskMB = 512
+        try {
+            const rawMeta = service.metadata
+            const meta = rawMeta
+                ? typeof rawMeta === 'object'
+                    ? rawMeta
+                    : JSON.parse(rawMeta as string)
+                : null
+            if (meta?.minMemoryMB) minMemoryMB = meta.minMemoryMB
+            if (meta?.minDiskMB) minDiskMB = meta.minDiskMB
+        } catch {}
+
+        const [{ conflicts: portConflicts }, resourceWarnings] = await Promise.all([
+            this.dockerService.checkPortConflicts(hostPorts),
+            this.systemService.checkResourceWarnings(minMemoryMB, minDiskMB),
+        ])
+
+        return response.send({ portConflicts, resourceWarnings })
+    }
+
+    /** Return the next suggested host port for a custom app (8600+ range). */
+    async suggestCustomPort({ response }: HttpContext) {
+        const port = await this.systemService.getNextSuggestedCustomPort()
+        return response.send({ port })
+    }
+
+    /**
+     * Service-less preflight for the custom-app form: given host ports, volumes and an image,
+     * report port conflicts, host resource warnings, overridable guard warnings (risky bind
+     * mounts / untrusted or moving-tag images), and hard blocks (docker socket, system dirs,
+     * malformed image). Lets the form give live feedback before a Service record exists.
+     */
+    async preflightCustomApp({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(preflightCustomValidator)
+        const [{ conflicts }, resourceWarnings] = await Promise.all([
+            this.dockerService.checkPortConflicts(payload.ports ?? []),
+            this.systemService.checkResourceWarnings(256, 512),
+        ])
+        // When editing, the app's own container legitimately holds its ports — don't flag those.
+        const portConflicts = payload.exclude_service
+            ? conflicts.filter((c) => c.usedBy !== payload.exclude_service)
+            : conflicts
+        const guard = evaluateCustomApp({ image: payload.image, volumes: payload.volumes })
+        return response.send({
+            portConflicts,
+            resourceWarnings: [...resourceWarnings, ...guard.warnings],
+            blocked: guard.blocked,
+        })
+    }
+
+    /** Create and immediately begin installing a custom app container. */
+    async createCustomApp({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(customAppValidator)
+
+        // Derive a stable service_name from the friendly name
+        const slug = payload.friendly_name
+            .toLowerCase()
+            .replace(/[^a-z0-9]+/g, '_')
+            .replace(/^_+|_+$/g, '')
+        const serviceName = `nomad_custom_${slug}`
+
+        const existing = await Service.query().where('service_name', serviceName).first()
+        if (existing) {
+            return response.status(409).send({
+                success: false,
+                message: `A custom app named "${payload.friendly_name}" already exists. Choose a different name.`,
+            })
+        }
+
+        // Reject duplicate host ports within the request — Docker would otherwise fail at
+        // start time with an opaque "port is already allocated" error.
+        const hostPorts = (payload.ports ?? []).map((p) => p.host)
+        const duplicateHostPorts = [...new Set(hostPorts.filter((p, i) => hostPorts.indexOf(p) !== i))]
+        if (duplicateHostPorts.length) {
+            return response.status(422).send({
+                success: false,
+                message: `Duplicate host port(s): ${duplicateHostPorts.join(', ')}. Each host port can map to only one container.`,
+            })
+        }
+
+        // Security guardrails: hard-block dangerous bind mounts / malformed images regardless of
+        // force; surface overridable warnings (risky paths, untrusted/moving-tag images) unless forced.
+        const guard = evaluateCustomApp({ image: payload.image, volumes: payload.volumes })
+        if (guard.blocked.length) {
+            return response.status(422).send({
+                success: false,
+                message: guard.blocked.join(' '),
+                blocked: guard.blocked,
+            })
+        }
+        if (!payload.force && guard.warnings.length) {
+            return response.status(409).send({
+                success: false,
+                message: guard.warnings.join(' '),
+                warnings: guard.warnings,
+            })
+        }
+
+        // Advisory preflight: surface port conflicts before creating the record so a failed
+        // install doesn't leave a phantom card. The user can re-submit with force=true to override.
+        if (!payload.force && hostPorts.length) {
+            const { conflicts } = await this.dockerService.checkPortConflicts(hostPorts)
+            if (conflicts.length) {
+                return response.status(409).send({
+                    success: false,
+                    message: `Port conflict: ${conflicts
+                        .map((c) => `${c.port} (in use by ${c.usedBy})`)
+                        .join(', ')}.`,
+                    portConflicts: conflicts,
+                })
+            }
+        }
+
+        const { containerConfig, uiLocation } = this.buildCustomContainerConfig(payload)
+
+        await Service.create({
+            service_name: serviceName,
+            friendly_name: payload.friendly_name,
+            container_image: payload.image,
+            container_config: JSON.stringify(containerConfig),
+            ui_location: uiLocation,
+            icon: payload.icon || 'IconBrandDocker',
+            installed: false,
+            installation_status: 'idle',
+            is_dependency_service: false,
+            is_custom: true,
+            category: payload.category ?? 'custom',
+            depends_on: null,
+        })
+
+        const result = await this.dockerService.createContainerPreflight(serviceName)
+        if (result.success) {
+            return response.send({ success: true, message: result.message, service_name: serviceName })
+        }
+        return response.status(400).send({ success: false, message: result.message })
+    }
+
+    /** Delete a custom app: stop + remove its container, then delete the DB record. */
+    async deleteCustomApp({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(deleteCustomAppValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ error: `Service ${payload.service_name} not found` })
+        }
+        if (!service.is_custom) {
+            return response.status(403).send({ error: 'Only custom apps can be deleted.' })
+        }
+
+        await this.dockerService.removeCustomAppContainer(payload.service_name, payload.remove_image ?? false)
+        await service.delete()
+
+        return response.send({ success: true, message: `Custom app ${payload.service_name} deleted` })
+    }
+
+    /** Uninstall a curated catalog app: stop + remove its container (optionally its image) and
+     * return the card to the available catalog. App data under the storage path stays on disk,
+     * so a later reinstall picks it back up. Custom apps are removed via deleteCustomApp instead,
+     * which also drops their DB record. */
+    async uninstallService({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(uninstallServiceValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ error: `Service ${payload.service_name} not found` })
+        }
+        if (service.is_custom) {
+            return response.status(403).send({ error: 'Custom apps are removed via delete.' })
+        }
+        if (service.is_dependency_service) {
+            return response.status(403).send({ error: 'Dependency services cannot be uninstalled directly.' })
+        }
+        if (!service.installed) {
+            return response.status(409).send({ error: `Service ${payload.service_name} is not installed` })
+        }
+
+        const result = await this.dockerService.uninstallService(
+            payload.service_name,
+            payload.remove_image ?? false
+        )
+        if (!result.success) {
+            return response.status(500).send({ success: false, message: result.message })
+        }
+        return response.send({ success: true, message: result.message })
+    }
+
+    /** Set or clear an app's custom launch URL (works for curated and custom apps). Purely a
+     * metadata change — no container is touched. An empty/invalid value clears the override, after
+     * which the default host + port link is used again. */
+    async setServiceCustomUrl({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(setServiceCustomUrlValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: `Service ${payload.service_name} not found` })
+        }
+        // Hidden dependency services (e.g. Qdrant) aren't user-launchable, so they have no link to set.
+        if (service.is_dependency_service) {
+            return response.status(403).send({ success: false, message: 'This service cannot be configured.' })
+        }
+
+        // Reject a non-empty value that isn't a valid http(s) URL; an empty value clears the override.
+        const normalized = normalizeCustomUrl(payload.custom_url)
+        if (payload.custom_url && payload.custom_url.trim() && !normalized) {
+            return response.status(422).send({
+                success: false,
+                message: 'Custom URL must be a valid http(s) address (e.g. https://jellyfin.myhomelab.net).',
+            })
+        }
+
+        service.custom_url = normalized
+        await service.save()
+
+        return response.send({ success: true, custom_url: service.custom_url })
+    }
+
+    /** Re-pull a custom app's image and recreate its container in place (preserving volumes). */
+    async updateCustomApp_pullLatest({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(installServiceValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: `Service ${payload.service_name} not found` })
+        }
+        if (!service.is_custom) {
+            return response.status(403).send({ success: false, message: 'Only custom apps can be updated this way.' })
+        }
+
+        const result = await this.dockerService.recreateCustomAppContainer(payload.service_name, {
+            forcePull: true,
+        })
+        if (result.success) {
+            return response.send({ success: true, message: result.message })
+        }
+        return response.status(400).send({ success: false, message: result.message })
+    }
+
+    /** Return the last N lines of a service container's logs. */
+    async getServiceLogs({ params, request, response }: HttpContext) {
+        // Scope to managed services only — otherwise any sibling container's logs (admin app,
+        // database) would be readable by name on this unauthenticated API surface.
+        const service = await Service.query().where('service_name', params.name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: `Service ${params.name} not found` })
+        }
+        const { tail } = await request.validateUsing(serviceLogsValidator)
+        const result = await this.dockerService.getContainerLogs(params.name, tail ?? 200)
+        if (!result.success) {
+            return response.status(404).send({ success: false, message: result.message })
+        }
+        return response.send({ success: true, logs: result.logs })
+    }
+
+    /** Return a one-shot CPU/memory usage snapshot for a running service container. */
+    async getServiceStats({ params, response }: HttpContext) {
+        // Scope to managed services only (see getServiceLogs).
+        const service = await Service.query().where('service_name', params.name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: `Service ${params.name} not found` })
+        }
+        const result = await this.dockerService.getContainerStats(params.name)
+        if (!result.success) {
+            return response.status(404).send({ success: false, message: result.message })
+        }
+        return response.send({ success: true, running: result.running ?? false, stats: result.stats ?? null })
+    }
+
+    /** Return an app's current configuration in the editable form-shape. */
+    async getCustomApp({ params, response }: HttpContext) {
+        const service = await Service.query().where('service_name', params.name).first()
+        if (!service) {
+            return response.status(404).send({ error: `Service ${params.name} not found` })
+        }
+        // Custom and curated apps are both editable; hidden dependency services (e.g. Qdrant) are not.
+        if (service.is_dependency_service) {
+            return response.status(403).send({ error: 'This service cannot be edited.' })
+        }
+        return response.send({ success: true, app: this.parseCustomContainerConfig(service) })
+    }
+
+    /** Reconfigure an app: validate + guard, persist the new config, then recreate the container.
+     * Works for both custom apps and curated (pre-configured) apps. Editing a curated app marks it
+     * user-modified so the seeder stops overwriting the user's changes. */
+    async updateCustomApp({ request, response }: HttpContext) {
+        const payload = await request.validateUsing(updateCustomAppValidator)
+
+        const service = await Service.query().where('service_name', payload.service_name).first()
+        if (!service) {
+            return response.status(404).send({ success: false, message: `Service ${payload.service_name} not found` })
+        }
+        // Custom and curated apps are both editable; hidden dependency services (e.g. Qdrant) are not.
+        if (service.is_dependency_service) {
+            return response.status(403).send({ success: false, message: 'This service cannot be edited.' })
+        }
+
+        // Reject duplicate host ports within the request.
+        const hostPorts = (payload.ports ?? []).map((p) => p.host)
+        const duplicateHostPorts = [...new Set(hostPorts.filter((p, i) => hostPorts.indexOf(p) !== i))]
+        if (duplicateHostPorts.length) {
+            return response.status(422).send({
+                success: false,
+                message: `Duplicate host port(s): ${duplicateHostPorts.join(', ')}. Each host port can map to only one container.`,
+            })
+        }
+
+        // Security guardrails (same posture as create).
+        const guard = evaluateCustomApp({ image: payload.image, volumes: payload.volumes })
+        if (guard.blocked.length) {
+            return response.status(422).send({ success: false, message: guard.blocked.join(' '), blocked: guard.blocked })
+        }
+        if (!payload.force && guard.warnings.length) {
+            return response.status(409).send({ success: false, message: guard.warnings.join(' '), warnings: guard.warnings })
+        }
+
+        // Port conflicts — but ignore ports already held by this app's own container.
+        if (!payload.force && hostPorts.length) {
+            const { conflicts } = await this.dockerService.checkPortConflicts(hostPorts)
+            const external = conflicts.filter((c) => c.usedBy !== payload.service_name)
+            if (external.length) {
+                return response.status(409).send({
+                    success: false,
+                    message: `Port conflict: ${external
+                        .map((c) => `${c.port} (in use by ${c.usedBy})`)
+                        .join(', ')}.`,
+                    portConflicts: external,
+                })
+            }
+        }
+
+        // Merge the form fields into the app's existing config rather than rebuilding from scratch,
+        // so advanced settings a curated app ships with (GPU device requests, special env, etc.) are
+        // preserved across an edit.
+        // Preserve an explicit scheme (e.g. ui_location "https:8480") across an edit — otherwise a
+        // TLS-serving app's Open link would silently revert to http after any reconfigure.
+        const prevScheme = (service.ui_location || '').match(/^(https?):\d+$/)?.[1]
+        const { containerConfig, uiLocation } = this.mergeCustomContainerConfig(
+            service.container_config,
+            payload
+        )
+        service.friendly_name = payload.friendly_name
+        service.container_image = payload.image
+        service.container_config = JSON.stringify(containerConfig)
+        service.ui_location = prevScheme && uiLocation && /^\d+$/.test(uiLocation)
+            ? `${prevScheme}:${uiLocation}`
+            : uiLocation
+        service.category = payload.category ?? service.category ?? 'custom'
+        if (payload.icon) service.icon = payload.icon
+        // Flag as user-modified so the seeder stops overwriting this app's config on future runs.
+        service.is_user_modified = true
+        await service.save()
+
+        const result = await this.dockerService.recreateCustomAppContainer(payload.service_name)
+        if (result.success) {
+            return response.send({ success: true, message: result.message, service_name: payload.service_name })
+        }
+        return response.status(400).send({ success: false, message: result.message })
+    }
+
+    /**
+     * Build a Docker container config (HostConfig + ExposedPorts + Env) from custom-app form input,
+     * applying default resource caps. Shared by create and update so both stay in lockstep.
+     */
+    private buildCustomContainerConfig(payload: {
+        ports?: { container: number; host: number }[]
+        volumes?: { host_path: string; container_path: string }[]
+        env?: string[]
+        memory_mb?: number
+        cpus?: number
+    }): { containerConfig: Record<string, any>; uiLocation: string | null } {
+        const portBindings: Record<string, [{ HostPort: string }]> = {}
+        const exposedPorts: Record<string, {}> = {}
+        for (const { container, host } of payload.ports ?? []) {
+            portBindings[`${container}/tcp`] = [{ HostPort: String(host) }]
+            exposedPorts[`${container}/tcp`] = {}
+        }
+
+        const binds = (payload.volumes ?? []).map(
+            ({ host_path, container_path }) => `${host_path}:${container_path}`
+        )
+
+        // Resource caps so a runaway custom container can't starve the host. Memory is bytes;
+        // NanoCpus is CPUs × 1e9. Defaults are generous and user-overridable.
+        const memoryBytes = (payload.memory_mb ?? DEFAULT_MEMORY_MB) * 1024 * 1024
+        const nanoCpus = Math.round((payload.cpus ?? DEFAULT_CPUS) * 1e9)
+
+        const containerConfig: Record<string, any> = {
+            HostConfig: {
+                RestartPolicy: { Name: 'unless-stopped' },
+                PortBindings: portBindings,
+                Memory: memoryBytes,
+                NanoCpus: nanoCpus,
+                ...(binds.length ? { Binds: binds } : {}),
+            },
+            ExposedPorts: exposedPorts,
+            ...(payload.env?.length ? { Env: payload.env } : {}),
+        }
+
+        const firstHostPort = payload.ports?.[0]?.host
+        const uiLocation = firstHostPort ? String(firstHostPort) : null
+        return { containerConfig, uiLocation }
+    }
+
+    /**
+     * Merge custom-app form input into an app's *existing* container config. Used by the edit path so
+     * editing a curated app only changes the fields exposed in the form (image/ports/volumes/env and,
+     * if supplied, resource caps) while preserving everything else it ships with (GPU DeviceRequests,
+     * User, custom HostConfig keys, etc.). Unlike buildCustomContainerConfig, resource caps are NOT
+     * defaulted here — a curated app intentionally left uncapped stays uncapped unless the user sets one.
+     */
+    private mergeCustomContainerConfig(
+        existingRaw: string | null,
+        payload: {
+            ports?: { container: number; host: number }[]
+            volumes?: { host_path: string; container_path: string }[]
+            env?: string[]
+            memory_mb?: number
+            cpus?: number
+        }
+    ): { containerConfig: Record<string, any>; uiLocation: string | null } {
+        const parsed = existingRaw
+            ? typeof existingRaw === 'object'
+                ? existingRaw
+                : JSON.parse(existingRaw as string)
+            : {}
+        // Deep clone so we never mutate the parsed source.
+        const containerConfig: Record<string, any> = JSON.parse(JSON.stringify(parsed ?? {}))
+        containerConfig.HostConfig = containerConfig.HostConfig ?? {}
+        // Keep a restart policy if the existing config lacked one.
+        containerConfig.HostConfig.RestartPolicy =
+            containerConfig.HostConfig.RestartPolicy ?? { Name: 'unless-stopped' }
+
+        const portBindings: Record<string, [{ HostPort: string }]> = {}
+        const exposedPorts: Record<string, {}> = {}
+        for (const { container, host } of payload.ports ?? []) {
+            portBindings[`${container}/tcp`] = [{ HostPort: String(host) }]
+            exposedPorts[`${container}/tcp`] = {}
+        }
+        containerConfig.HostConfig.PortBindings = portBindings
+        containerConfig.ExposedPorts = exposedPorts
+
+        const binds = (payload.volumes ?? []).map(
+            ({ host_path, container_path }) => `${host_path}:${container_path}`
+        )
+        if (binds.length) containerConfig.HostConfig.Binds = binds
+        else delete containerConfig.HostConfig.Binds
+
+        if (payload.env?.length) containerConfig.Env = payload.env
+        else delete containerConfig.Env
+
+        // Only touch resource caps when the user explicitly set them — preserve existing/uncapped otherwise.
+        if (payload.memory_mb != null) {
+            containerConfig.HostConfig.Memory = payload.memory_mb * 1024 * 1024
+        }
+        if (payload.cpus != null) {
+            containerConfig.HostConfig.NanoCpus = Math.round(payload.cpus * 1e9)
+        }
+
+        const firstHostPort = payload.ports?.[0]?.host
+        const uiLocation = firstHostPort ? String(firstHostPort) : null
+        return { containerConfig, uiLocation }
+    }
+
+    /** Inverse of buildCustomContainerConfig: turn a stored Service into the editable form-shape. */
+    private parseCustomContainerConfig(service: Service) {
+        const raw = service.container_config
+        const config = raw ? (typeof raw === 'object' ? raw : JSON.parse(raw as string)) : {}
+        const hostConfig = config?.HostConfig ?? {}
+
+        const ports = Object.entries(hostConfig.PortBindings ?? {}).map(([key, val]: [string, any]) => ({
+            container: Number.parseInt(key, 10),
+            host: Number.parseInt(val?.[0]?.HostPort, 10),
+        }))
+
+        const volumes = (hostConfig.Binds ?? []).map((bind: string) => {
+            const idx = bind.indexOf(':')
+            return { host_path: bind.slice(0, idx), container_path: bind.slice(idx + 1) }
+        })
+
+        return {
+            service_name: service.service_name,
+            friendly_name: service.friendly_name,
+            image: service.container_image,
+            category: service.category ?? 'custom',
+            icon: service.icon ?? 'IconBrandDocker',
+            ports,
+            volumes,
+            env: (config?.Env ?? []) as string[],
+            memory_mb: hostConfig.Memory ? Math.round(hostConfig.Memory / (1024 * 1024)) : undefined,
+            cpus: hostConfig.NanoCpus ? hostConfig.NanoCpus / 1e9 : undefined,
+        }
+    }
 }

admin/app/controllers/zim_controller.ts

@@ -8,7 +8,12 @@ import {
 } from '#validators/common'
 import { addCustomLibraryValidator, browseLibraryValidator, idParamValidator, listRemoteZimValidator } from '#validators/zim'
 import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
 import type { HttpContext } from '@adonisjs/core/http'
+import { createWriteStream } from 'fs'
+import { rename } from 'fs/promises'
+import { join, resolve, sep } from 'path'
+import { ZIM_STORAGE_PATH, ensureDirectoryExists, sanitizeFilename } from '../utils/fs.js'
 
 @inject()
 export default class ZimController {
@@ -56,6 +61,14 @@ export default class ZimController {
     }
   }
 
+  async rescanLibrary({}: HttpContext) {
+    const result = await this.zimService.rescanLibrary()
+    return {
+      message: 'Kiwix library rescanned',
+      ...result,
+    }
+  }
+
   async delete({ request, response }: HttpContext) {
     const payload = await request.validateUsing(filenameParamValidator)
 
@@ -75,6 +88,87 @@ export default class ZimController {
     }
   }
 
+  async upload({ request, response }: HttpContext) {
+    let filename: string | null = null
+    let tmpPath: string | null = null
+    let uploadError: string | null = null
+
+    try {
+      const basePath = resolve(join(process.cwd(), ZIM_STORAGE_PATH))
+      await ensureDirectoryExists(basePath)
+
+      request.multipart.onFile('*', {}, async (part) => {
+        const clientName = part.filename || ''
+        if (!clientName.toLowerCase().endsWith('.zim')) {
+          part.resume()
+          uploadError = 'INVALID_TYPE'
+          return
+        }
+
+        const sanitized = sanitizeFilename(clientName)
+        const finalPath = resolve(join(basePath, sanitized))
+
+        if (!finalPath.startsWith(basePath + sep)) {
+          part.resume()
+          uploadError = 'INVALID_FILENAME'
+          return
+        }
+
+        const { access } = await import('fs/promises')
+        const exists = await access(finalPath).then(() => true).catch(() => false)
+        if (exists) {
+          part.resume()
+          uploadError = 'DUPLICATE_FILENAME'
+          return
+        }
+
+        filename = sanitized
+        tmpPath = finalPath + '.tmp'
+        const ws = createWriteStream(tmpPath)
+
+        await new Promise<void>((res, rej) => {
+          ws.on('error', rej)
+          ws.on('finish', res)
+          part.on('error', rej)
+          part.pipe(ws)
+        })
+
+        await rename(tmpPath, finalPath)
+        tmpPath = null
+      })
+
+      await request.multipart.process()
+
+      if (uploadError === 'INVALID_TYPE') {
+        return response.status(422).send({ message: 'Only .zim files are accepted' })
+      }
+      if (uploadError === 'INVALID_FILENAME') {
+        return response.status(422).send({ message: 'Invalid filename' })
+      }
+      if (uploadError === 'DUPLICATE_FILENAME') {
+        return response.status(409).send({ message: 'A ZIM file with that name already exists' })
+      }
+      if (!filename) {
+        return response.status(400).send({ message: 'No file received' })
+      }
+
+      const { added } = await this.zimService.registerLocalUpload(filename)
+
+      return response.status(201).send({
+        message: 'ZIM file uploaded and registered successfully',
+        filename,
+        added,
+      })
+    } catch (error) {
+      logger.error('[ZimController] Upload failed:', error)
+      if (tmpPath) {
+        const { unlink } = await import('fs/promises')
+        await unlink(tmpPath).catch(() => {})
+      }
+      return response.status(500).send({ message: 'Upload failed' })
+    }
+  }
+
   // Wikipedia selector endpoints
 
   async getWikipediaState({}: HttpContext) {

admin/app/data/conditions.ts

@@ -0,0 +1,438 @@
+/* eslint-disable */
+/**
+ * GENERATED FILE — DO NOT EDIT.
+ *
+ * Single source of truth: collections/conditions.json
+ * Regenerate after editing the JSON: `npm run gen:curated-data` (from admin/).
+ * curated_data_sync.standalone.ts fails CI if this file drifts from the JSON.
+ *
+ * Curated "when to use what" condition spine (Phase 1). US-government / public-domain sourcing.
+ */
+import type { ConditionsFile } from '../../types/conditions.js'
+
+export const CONDITIONS_FILE: ConditionsFile = {
+  "version": "2026-06-07",
+  "conditions": [
+    {
+      "slug": "pain",
+      "label": "Pain",
+      "category": "Pain, fever & inflammation",
+      "searchTerms": [
+        "pain",
+        "aches",
+        "minor aches",
+        "pain relief",
+        "analgesic"
+      ]
+    },
+    {
+      "slug": "headache",
+      "label": "Headache",
+      "category": "Pain, fever & inflammation",
+      "searchTerms": [
+        "headache",
+        "migraine",
+        "head pain",
+        "tension headache"
+      ]
+    },
+    {
+      "slug": "muscle-joint-pain",
+      "label": "Muscle & joint pain",
+      "category": "Pain, fever & inflammation",
+      "searchTerms": [
+        "muscular aches",
+        "muscle aches",
+        "backache",
+        "arthritis",
+        "joint pain",
+        "minor pain of arthritis"
+      ]
+    },
+    {
+      "slug": "fever",
+      "label": "Fever",
+      "category": "Pain, fever & inflammation",
+      "searchTerms": [
+        "fever",
+        "reduces fever",
+        "fever reducer",
+        "temporarily reduces fever"
+      ]
+    },
+    {
+      "slug": "menstrual-cramps",
+      "label": "Menstrual cramps",
+      "category": "Pain, fever & inflammation",
+      "searchTerms": [
+        "menstrual cramps",
+        "menstrual pain",
+        "period pain",
+        "premenstrual"
+      ]
+    },
+    {
+      "slug": "cough",
+      "label": "Cough",
+      "category": "Cold, cough & allergy",
+      "searchTerms": [
+        "cough",
+        "cough suppressant",
+        "coughing",
+        "controls cough"
+      ]
+    },
+    {
+      "slug": "nasal-congestion",
+      "label": "Nasal congestion",
+      "category": "Cold, cough & allergy",
+      "searchTerms": [
+        "nasal congestion",
+        "stuffy nose",
+        "sinus congestion",
+        "decongestant",
+        "nasal decongestant"
+      ]
+    },
+    {
+      "slug": "sore-throat",
+      "label": "Sore throat",
+      "category": "Cold, cough & allergy",
+      "searchTerms": [
+        "sore throat",
+        "sore mouth",
+        "minor sore throat",
+        "throat pain"
+      ]
+    },
+    {
+      "slug": "common-cold",
+      "label": "Common cold",
+      "category": "Cold, cough & allergy",
+      "searchTerms": [
+        "common cold",
+        "cold symptoms",
+        "cold",
+        "flu symptoms"
+      ]
+    },
+    {
+      "slug": "allergic-reaction",
+      "label": "Allergies & allergic reactions",
+      "category": "Cold, cough & allergy",
+      "searchTerms": [
+        "allergy",
+        "allergic reactions",
+        "hay fever",
+        "antihistamine",
+        "runny nose",
+        "sneezing",
+        "itchy watery eyes"
+      ]
+    },
+    {
+      "slug": "heartburn",
+      "label": "Heartburn",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "heartburn",
+        "acid indigestion",
+        "acid reducer",
+        "antacid",
+        "sour stomach"
+      ]
+    },
+    {
+      "slug": "indigestion",
+      "label": "Indigestion & upset stomach",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "indigestion",
+        "upset stomach",
+        "gas",
+        "bloating",
+        "fullness",
+        "antacid"
+      ]
+    },
+    {
+      "slug": "nausea-vomiting",
+      "label": "Nausea & vomiting",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "nausea",
+        "vomiting",
+        "upset stomach associated with nausea"
+      ]
+    },
+    {
+      "slug": "diarrhea",
+      "label": "Diarrhea",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "diarrhea",
+        "antidiarrheal",
+        "loose stools",
+        "traveler’s diarrhea"
+      ]
+    },
+    {
+      "slug": "constipation",
+      "label": "Constipation",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "constipation",
+        "laxative",
+        "irregularity",
+        "occasional constipation",
+        "stool softener"
+      ]
+    },
+    {
+      "slug": "motion-sickness",
+      "label": "Motion sickness",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "motion sickness",
+        "travel sickness",
+        "seasickness",
+        "car sickness"
+      ]
+    },
+    {
+      "slug": "gas",
+      "label": "Gas & bloating",
+      "category": "Stomach & digestion",
+      "searchTerms": [
+        "gas",
+        "bloating",
+        "flatulence",
+        "antigas",
+        "pressure"
+      ]
+    },
+    {
+      "slug": "wounds-cuts",
+      "label": "Wounds & cuts",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "minor cuts",
+        "scrapes",
+        "wounds",
+        "first aid antiseptic",
+        "first aid to help prevent infection",
+        "abrasions"
+      ]
+    },
+    {
+      "slug": "burns",
+      "label": "Burns",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "burns",
+        "minor burns",
+        "sunburn",
+        "scald",
+        "minor burn"
+      ]
+    },
+    {
+      "slug": "insect-bites-stings",
+      "label": "Insect bites & stings",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "insect bites",
+        "insect stings",
+        "bug bites",
+        "bee sting",
+        "itching from insect bites"
+      ]
+    },
+    {
+      "slug": "skin-rash-itch",
+      "label": "Rash & itching",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "itching",
+        "rash",
+        "skin irritation",
+        "itchy skin",
+        "minor skin irritations",
+        "eczema"
+      ]
+    },
+    {
+      "slug": "poison-ivy",
+      "label": "Poison ivy & plant rashes",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "poison ivy",
+        "poison oak",
+        "poison sumac",
+        "rashes due to poison ivy"
+      ]
+    },
+    {
+      "slug": "fungal-infection",
+      "label": "Athlete’s foot & ringworm",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "athlete’s foot",
+        "ringworm",
+        "jock itch",
+        "antifungal",
+        "fungal infection",
+        "tinea"
+      ]
+    },
+    {
+      "slug": "dry-skin",
+      "label": "Dry & chapped skin",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "dry skin",
+        "chapped skin",
+        "chapped lips",
+        "skin protectant",
+        "cracked skin"
+      ]
+    },
+    {
+      "slug": "acne",
+      "label": "Acne",
+      "category": "Skin & wounds",
+      "searchTerms": [
+        "acne",
+        "pimples",
+        "blackheads",
+        "acne treatment"
+      ]
+    },
+    {
+      "slug": "eye-irritation",
+      "label": "Eye irritation & dryness",
+      "category": "Eyes, ears & mouth",
+      "searchTerms": [
+        "eye irritation",
+        "dry eyes",
+        "red eyes",
+        "eye redness",
+        "itchy eyes",
+        "lubricant eye"
+      ]
+    },
+    {
+      "slug": "earache",
+      "label": "Earache & ear wax",
+      "category": "Eyes, ears & mouth",
+      "searchTerms": [
+        "earache",
+        "ear wax",
+        "earwax removal",
+        "ear pain",
+        "swimmer’s ear"
+      ]
+    },
+    {
+      "slug": "canker-sores",
+      "label": "Canker & cold sores",
+      "category": "Eyes, ears & mouth",
+      "searchTerms": [
+        "canker sores",
+        "cold sores",
+        "mouth sores",
+        "fever blisters",
+        "oral pain"
+      ]
+    },
+    {
+      "slug": "toothache",
+      "label": "Toothache",
+      "category": "Eyes, ears & mouth",
+      "searchTerms": [
+        "toothache",
+        "tooth pain",
+        "dental pain",
+        "oral analgesic"
+      ]
+    },
+    {
+      "slug": "sleeplessness",
+      "label": "Sleeplessness",
+      "category": "Sleep, stress & general",
+      "searchTerms": [
+        "sleeplessness",
+        "insomnia",
+        "sleep aid",
+        "difficulty falling asleep",
+        "nighttime"
+      ]
+    },
+    {
+      "slug": "dehydration",
+      "label": "Dehydration",
+      "category": "Sleep, stress & general",
+      "searchTerms": [
+        "dehydration",
+        "oral rehydration",
+        "electrolyte",
+        "fluid loss",
+        "replaces electrolytes"
+      ]
+    },
+    {
+      "slug": "hemorrhoids",
+      "label": "Hemorrhoids",
+      "category": "Sleep, stress & general",
+      "searchTerms": [
+        "hemorrhoids",
+        "hemorrhoidal",
+        "anal itching",
+        "rectal"
+      ]
+    },
+    {
+      "slug": "yeast-infection",
+      "label": "Vaginal yeast infection",
+      "category": "Infections (OTC-treatable)",
+      "searchTerms": [
+        "vaginal yeast infection",
+        "yeast infection",
+        "vaginal antifungal",
+        "candidiasis"
+      ]
+    },
+    {
+      "slug": "pinworm",
+      "label": "Pinworm",
+      "category": "Infections (OTC-treatable)",
+      "searchTerms": [
+        "pinworm",
+        "pinworm infection",
+        "pinworm treatment"
+      ]
+    },
+    {
+      "slug": "cold-sore-lip",
+      "label": "Chapped & sun-protected lips",
+      "category": "Sleep, stress & general",
+      "searchTerms": [
+        "lip protectant",
+        "chapped lips",
+        "sunburn protection lips",
+        "lip balm"
+      ]
+    },
+    {
+      "slug": "eye-allergy",
+      "label": "Eye allergies",
+      "category": "Eyes, ears & mouth",
+      "searchTerms": [
+        "eye allergy",
+        "itchy eyes due to allergies",
+        "ocular itching",
+        "allergic conjunctivitis"
+      ]
+    }
+  ]
+}

admin/app/data/home_remedies.ts

@@ -0,0 +1,328 @@
+/* eslint-disable */
+/**
+ * GENERATED FILE — DO NOT EDIT.
+ *
+ * Single source of truth: collections/home_remedies.json
+ * Regenerate after editing the JSON: `npm run gen:curated-data` (from admin/).
+ * curated_data_sync.standalone.ts fails CI if this file drifts from the JSON.
+ *
+ * Non-herbal home-care / self-care measures from US-government public-domain pages (CDC, NIH/NLM MedlinePlus, FDA); each entry carries its own sourceUrl.
+ */
+import type { NaturalRemediesFile } from '../../types/conditions.js'
+
+export const HOME_REMEDIES_FILE: NaturalRemediesFile = {
+  "version": "2026-06-10",
+  "source": {
+    "name": "US government health guidance (CDC, NIH/NHLBI, MedlinePlus/NLM, FDA)",
+    "url": "https://www.cdc.gov",
+    "license": "Public domain (US government works)"
+  },
+  "remedies": [
+    {
+      "slug": "honey-for-cough",
+      "name": "Honey (for cough)",
+      "commonNames": [],
+      "conditions": [
+        "cough",
+        "sore-throat",
+        "common-cold"
+      ],
+      "uses": "Honey may be used to relieve cough in adults and children at least 1 year old. One to two teaspoons can be taken directly or stirred into a warm (not hot) beverage.",
+      "how": "Give one to two teaspoons of honey directly by mouth, or stir it into a warm (not hot) drink. The CDC recommends this as a home measure for cough associated with the common cold.",
+      "evidence": "CDC lists honey among recommended home measures for easing cough and sore throat associated with the common cold; it is one of several non-medication strategies mentioned alongside rest, fluids, and humidifier use.",
+      "cautions": "Never give honey to infants under 1 year old — it can contain Clostridium botulinum spores that cause infant botulism, a rare but serious illness. This warning applies regardless of honey type or brand.",
+      "sourceUrl": "https://www.cdc.gov/common-cold/treatment/index.html"
+    },
+    {
+      "slug": "fluids-and-rest",
+      "name": "Fluids and rest",
+      "commonNames": [],
+      "conditions": [
+        "common-cold",
+        "fever",
+        "diarrhea",
+        "nausea-vomiting"
+      ],
+      "uses": "Getting plenty of rest and drinking adequate fluids (water, clear broths, juice, or sports drinks) supports recovery from colds, fever, diarrhea, and nausea. Adults with diarrhea should drink water, fruit juices, sports drinks, sodas without caffeine, and salty broths.",
+      "how": "Get plenty of rest and drink plenty of fluids. If keeping liquids down is difficult, take small sips of water or suck on ice chips frequently rather than trying to drink large amounts at once.",
+      "evidence": "CDC recommends rest and fluids as primary home care measures for the common cold. MedlinePlus (NIH/NIDDK) similarly lists these as the foundation of diarrhea self-care, and advises nausea patients to take in small amounts of clear liquids often to stay hydrated.",
+      "cautions": "Severely ill individuals, those with signs of dehydration (no urination, sunken eyes, extreme thirst), or those unable to keep any fluid down should seek medical care promptly. Caffeine and alcohol are not effective rehydration choices.",
+      "sourceUrl": "https://www.cdc.gov/common-cold/treatment/index.html"
+    },
+    {
+      "slug": "oral-rehydration",
+      "name": "Oral rehydration solution (ORS)",
+      "commonNames": [
+        "ORS",
+        "rehydration salts"
+      ],
+      "conditions": [
+        "dehydration",
+        "diarrhea"
+      ],
+      "uses": "Oral rehydration solutions replace fluids and electrolytes lost through diarrhea or other causes of dehydration. For mild to moderate dehydration, drinking water is the first step; sports drinks or oral rehydration solutions (such as Pedialyte) are recommended when electrolytes are also depleted, especially for children.",
+      "how": "Use a commercially prepared oral rehydration solution (available without a prescription) and follow the package directions. For adults with electrolyte losses, sports drinks can help; if liquids are hard to keep down, take small sips frequently or suck on ice chips rather than drinking large amounts at once.",
+      "evidence": "MedlinePlus (NIH/NIDDK) states that treatment for dehydration involves replacing lost fluids and electrolytes, and that oral rehydration solutions for children are available without a prescription. The same source recommends sports drinks for adults when electrolytes have been lost alongside fluids.",
+      "cautions": "Seek immediate medical care for signs of severe dehydration: no urination for 8 or more hours, rapid heartbeat, confusion, or inability to keep fluids down. Infants and small children with diarrhea should use formulated ORS (not plain water) to replace electrolytes safely.",
+      "sourceUrl": "https://medlineplus.gov/dehydration.html"
+    },
+    {
+      "slug": "cool-compress-fever",
+      "name": "Cool compress (for fever and insect bites)",
+      "commonNames": [],
+      "conditions": [
+        "fever",
+        "insect-bites-stings",
+        "eye-allergy",
+        "eye-irritation"
+      ],
+      "uses": "A clean cloth soaked in cool (not ice-cold) water and placed on the forehead or bitten area can help reduce discomfort from fever, insect bites, and eye allergy symptoms. For insect stings, ice wrapped in a washcloth should be applied for 10 minutes on and 10 minutes off.",
+      "how": "Soak a clean cloth in cool water, wring it out, and place it on the forehead or affected area. For insect stings, wrap ice in a cloth and apply for 10 minutes on, then 10 minutes off — never place ice directly on bare skin.",
+      "evidence": "MedlinePlus (NIH/NIAID) notes that applying cool compresses is recommended for allergic conjunctivitis and eye burning and irritation. For insect bites and stings, a cool or iced compress is a standard first-line self-care step described in MedlinePlus search guidance consistent with NIH resources.",
+      "cautions": "Do not apply ice or an ice-cold compress directly to bare skin for extended periods — wrap ice in cloth and limit applications to 10–15 minutes to avoid frostbite or tissue damage. For fever, cool compresses supplement (but do not replace) appropriate fever-reducing medicine when indicated; consult a healthcare provider if fever is high, prolonged, or accompanied by severe symptoms.",
+      "sourceUrl": "https://medlineplus.gov/insectbitesandstings.html"
+    },
+    {
+      "slug": "ice-and-elevation",
+      "name": "Ice and elevation (RICE method)",
+      "commonNames": [
+        "RICE",
+        "Rest-Ice-Compression-Elevation"
+      ],
+      "conditions": [
+        "muscle-joint-pain",
+        "insect-bites-stings"
+      ],
+      "uses": "Applying ice wrapped in cloth to a strained muscle, sprain, or bite site — combined with rest, compression, and elevation of the injured area — reduces swelling and pain. Ice should be applied for 10–15 minutes every 1–3 hours during the first few days of injury.",
+      "how": "Wrap ice in a cloth or towel and apply to the injured area for 10–15 minutes at a time. Rest the area, wrap it snugly with a bandage to reduce swelling, and elevate it above heart level when possible.",
+      "evidence": "MedlinePlus (NIH/NIAMS) describes the RICE method as standard first-line treatment for sprains and strains: resting the area, icing it, compressing it with a bandage, and elevating it above heart level when possible. Ice use for the first 3 days is specifically mentioned.",
+      "cautions": "Never apply ice directly to skin — always wrap it in a cloth or towel. If swelling worsens significantly, numbness develops, or you suspect a fracture, seek medical evaluation. Do not use heat during the first 48–72 hours of an acute soft-tissue injury.",
+      "sourceUrl": "https://medlineplus.gov/sprainsandstrains.html"
+    },
+    {
+      "slug": "heating-pad",
+      "name": "Heating pad or warm compress",
+      "commonNames": [],
+      "conditions": [
+        "menstrual-cramps",
+        "muscle-joint-pain",
+        "earache"
+      ],
+      "uses": "Applying a heating pad or hot water bottle to the lower abdomen eases menstrual cramps. A warm compress applied to the ear can relieve earache discomfort. Heat may also be used on strained muscles after the first 48–72 hours of an acute injury.",
+      "how": "Place a heating pad or hot water bottle on the lower abdomen for menstrual cramps, or hold a warm cloth against the affected ear for earache. Use a low or medium heat setting and put a cloth between the pad and skin to prevent burns.",
+      "evidence": "MedlinePlus (NIH/NLM) lists using a heating pad or hot water bottle on the lower abdomen, along with taking a warm bath, as home care measures for period pain. For earache, placing a warm cloth on the affected ear is listed among comfort measures in NIH/NLM resources for acute ear infection self-care.",
+      "cautions": "Never fall asleep with a heating pad on — burns can result. Use a low or medium setting and place a cloth between the pad and skin. For ear pain, do not insert anything into the ear canal; if pain is severe, accompanied by drainage, hearing loss, or fever, see a healthcare provider to rule out infection requiring antibiotics.",
+      "sourceUrl": "https://medlineplus.gov/periodpain.html"
+    },
+    {
+      "slug": "humidifier-and-steam",
+      "name": "Humidifier or steam inhalation",
+      "commonNames": [
+        "cool-mist vaporizer",
+        "steam inhalation"
+      ],
+      "conditions": [
+        "nasal-congestion",
+        "cough",
+        "common-cold"
+      ],
+      "uses": "Using a clean humidifier or cool-mist vaporizer adds moisture to the air and can help relieve nasal congestion and cough. Breathing steam from a bowl of hot water or a running shower 2–4 times daily also loosens nasal secretions.",
+      "how": "Fill a clean cool-mist humidifier or vaporizer with water and run it in the room. Clean the device daily per the manufacturer's instructions to prevent mold and bacteria buildup.",
+      "evidence": "CDC recommends using a clean humidifier or cool-mist vaporizer as a home care measure for common cold symptoms including congestion and cough. MedlinePlus similarly notes that a humidifier can break up mucus and that steam inhalation from a shower is a recognized congestion-relief strategy.",
+      "cautions": "Clean the humidifier daily per manufacturer instructions to prevent mold and bacterial growth. Use cool-mist humidifiers rather than warm-mist (steam) versions for children to avoid burn risk. When inhaling steam from hot water, use caution to avoid scalding; keep a safe distance and do not cover your head over a pot of boiling water.",
+      "sourceUrl": "https://www.cdc.gov/common-cold/treatment/index.html"
+    },
+    {
+      "slug": "saline-nasal-rinse",
+      "name": "Saline nasal rinse",
+      "commonNames": [
+        "neti pot",
+        "nasal irrigation",
+        "saline nasal wash"
+      ],
+      "conditions": [
+        "nasal-congestion",
+        "common-cold"
+      ],
+      "uses": "Saline nasal rinses flush pollen, dust, and excess mucus from the nasal passages and add moisture. They can be performed with a neti pot, squeeze bottle, or bulb syringe using a prepared saline solution.",
+      "how": "Use only distilled, sterile, or previously boiled-and-cooled water — never tap water. After each use, rinse the device with the same safe water, then air-dry it thoroughly or wipe dry before storing.",
+      "evidence": "CDC recommends saline nasal spray or drops as a home care measure for the common cold. The FDA confirms that nasal irrigation devices are 'usually safe and effective products when used and cleaned properly,' with the critical safety requirement being the type of water used.",
+      "cautions": "Use only distilled, sterile, or previously boiled (and cooled) water — never tap water. The FDA warns that tap water can harbor organisms including amoebas that are safe to swallow but can cause serious or potentially fatal infections in the nasal passages. Boiled water should be cooled to lukewarm and stored in a clean, closed container for no more than 24 hours. Always clean and dry the device after each use.",
+      "sourceUrl": "https://www.fda.gov/consumers/consumer-updates/rinsing-your-sinuses-neti-pots-safe"
+    },
+    {
+      "slug": "oatmeal-bath",
+      "name": "Oatmeal or cool bath",
+      "commonNames": [
+        "colloidal oatmeal bath"
+      ],
+      "conditions": [
+        "skin-rash-itch",
+        "poison-ivy",
+        "dry-skin"
+      ],
+      "uses": "Soaking in a lukewarm oatmeal bath or taking a cool bath can relieve itching and skin irritation from rashes, poison ivy, eczema, and dry skin. Colloidal oatmeal bath products are available at drugstores.",
+      "how": "Fill a tub with lukewarm (not hot) water and add a colloidal oatmeal bath product according to package directions, or use plain cool water. After soaking, pat skin dry gently and apply a fragrance-free moisturizer immediately to lock in moisture.",
+      "evidence": "MedlinePlus (NLM) recommends taking 'lukewarm or oatmeal baths' as a self-care measure for itching, alongside cool compresses and moisturizing lotion. Oatmeal bath products are specifically noted to relieve symptoms of eczema and psoriasis; short, cooler baths are described as better than long, hot baths for skin conditions.",
+      "cautions": "Use lukewarm — not hot — water; hot water can worsen skin dryness and irritation. After bathing, pat skin dry gently (do not rub) and immediately apply a fragrance-free moisturizer to lock in moisture. If a rash is spreading rapidly, is accompanied by fever, or involves the face or genitals, consult a healthcare provider.",
+      "sourceUrl": "https://medlineplus.gov/itching.html"
+    },
+    {
+      "slug": "cool-running-water-on-burns",
+      "name": "Cool running water on burns",
+      "commonNames": [],
+      "conditions": [
+        "burns"
+      ],
+      "uses": "For minor burns, immediately run cool (not cold) water slowly over the burned area for 10–15 minutes to stop the burning process and reduce pain. After cooling, cover the burn with a clean, dry cloth or sterile bandage.",
+      "how": "Run cool water slowly over the burned area for several minutes, then cover with a clean, dry cloth or bandage. Do not apply ice, butter, or any creams — these can worsen tissue damage.",
+      "evidence": "CDC burn first-aid materials instruct: run cool water slowly over the burn area for several minutes, then cover with a clean, dry cloth or bandage. MedlinePlus (NIH/NIGMS) similarly specifies cool running water for 10–15 minutes followed by a dry sterile dressing as the appropriate first-aid response for minor burns.",
+      "cautions": "Do not apply ice, ice water, butter, first-aid creams, sprays, or home remedies — these can worsen tissue damage or introduce infection. Do not break blisters unless directed by a healthcare provider. Do not try to remove clothing or debris stuck to the burn. Seek immediate medical care for large burns, burns on the face, eyes, hands, or feet, burns from chemicals or electricity, or any burn with extreme pain, numbness, or deep tissue involvement.",
+      "sourceUrl": "https://medlineplus.gov/burns.html"
+    },
+    {
+      "slug": "clean-and-cover-wounds",
+      "name": "Clean and cover wounds and cuts",
+      "commonNames": [],
+      "conditions": [
+        "wounds-cuts"
+      ],
+      "uses": "For minor cuts and scrapes, rinse the wound thoroughly with cool clean water to remove dirt, apply gentle pressure with gauze to stop bleeding, then cover with a clean bandage. Wash with soap and water to reduce infection risk.",
+      "how": "Apply firm but gentle pressure with gauze to stop bleeding; if blood soaks through, add more gauze on top without removing the first layer. Rinse the wound with cool clean water, then cover it with a clean dry bandage.",
+      "evidence": "CDC guidance on wound care states: put pressure on a bleeding cut until it stops, gently pour clean water over the wound to clean it, then apply a clean, dry bandage. MedlinePlus (NLM) similarly advises rinsing cuts with cool water and applying firm but gentle pressure to stop bleeding.",
+      "cautions": "Watch for signs of infection in the days following — increasing redness, swelling, warmth, pus, or red streaks spreading from the wound require prompt medical evaluation. Seek immediate care for wounds that are deep, gaping, caused by an animal or human bite, or associated with a puncture from a potentially contaminated object (tetanus risk).",
+      "sourceUrl": "https://medlineplus.gov/firstaid.html"
+    },
+    {
+      "slug": "salt-water-gargle",
+      "name": "Salt-water gargle",
+      "commonNames": [],
+      "conditions": [
+        "sore-throat",
+        "common-cold",
+        "canker-sores"
+      ],
+      "uses": "Gargling with warm salt water several times a day can ease sore throat pain and may also help relieve canker sore discomfort. A standard preparation is ½ teaspoon (3 grams) of salt dissolved in 1 cup (240 mL) of warm water.",
+      "how": "Dissolve salt in a cup of warm water, take a mouthful, tilt your head back, and gargle for several seconds before spitting it out. Repeat as needed throughout the day to help ease sore throat pain.",
+      "evidence": "MedlinePlus (NLM) states that gargling may ease sore throat pain, listing it alongside lozenges and fluids. The same resource notes that salt-water rinses may help with canker sore discomfort, though mouthwashes containing alcohol should be avoided as they irritate the tissue.",
+      "cautions": "Gargling with salt water provides symptomatic relief only and does not treat the underlying cause of a sore throat. Sore throat accompanied by high fever, difficulty swallowing or breathing, drooling, a stiff neck, or lasting more than a week should be evaluated by a healthcare provider, as strep throat and other conditions require different treatment.",
+      "sourceUrl": "https://medlineplus.gov/sorethroat.html"
+    },
+    {
+      "slug": "fiber-and-water-constipation",
+      "name": "Dietary fiber and water (for constipation and hemorrhoids)",
+      "commonNames": [],
+      "conditions": [
+        "constipation",
+        "hemorrhoids"
+      ],
+      "uses": "Eating more fruits, vegetables, and whole grains (which are high in fiber) and drinking plenty of water each day are the foundational self-care steps for preventing and relieving constipation and reducing hemorrhoid discomfort.",
+      "how": "Eat more fruits, vegetables, and whole grains each day and drink plenty of fluids. Increase fiber gradually to avoid gas and bloating.",
+      "evidence": "MedlinePlus (NIH/NIDDK) lists increased dietary fiber and adequate fluid intake as primary self-care measures for both constipation and hemorrhoids. The hemorrhoids topic page specifically recommends eating high-fiber foods and drinking enough fluids every day as the first-line home treatment.",
+      "cautions": "Increase dietary fiber gradually to avoid gas and bloating. If constipation is new, severe, accompanied by blood in the stool, or associated with significant weight loss, see a healthcare provider to rule out underlying conditions. Hemorrhoid symptoms persisting beyond one week of home treatment, or any rectal bleeding, warrant medical evaluation.",
+      "sourceUrl": "https://medlineplus.gov/constipation.html"
+    },
+    {
+      "slug": "sitz-bath",
+      "name": "Sitz bath (for hemorrhoids)",
+      "commonNames": [],
+      "conditions": [
+        "hemorrhoids"
+      ],
+      "uses": "A sitz bath — sitting in a few inches of warm water for 10–15 minutes, several times a day — relieves the pain and itching of hemorrhoids. A special sitz bath tub that fits over a toilet is available at pharmacies.",
+      "how": "Fill a tub or sitz bath basin with a few inches of comfortably warm water and sit in it for 10 to 15 minutes. Repeat several times a day, keeping the area clean and dry between baths.",
+      "evidence": "MedlinePlus (NLM) lists taking warm baths several times a day, including sitz baths, as a recommended home care measure to relieve hemorrhoid pain. The recommendation is to sit in warm water for 10 to 15 minutes per session.",
+      "cautions": "The water should be comfortably warm — not hot — to avoid burns. Keep the area clean and dry between baths. If hemorrhoid symptoms do not improve after one week of home treatment, or if there is rectal bleeding, see a healthcare provider.",
+      "sourceUrl": "https://medlineplus.gov/hemorrhoids.html"
+    },
+    {
+      "slug": "elevate-head-heartburn",
+      "name": "Elevate head of bed (for heartburn)",
+      "commonNames": [],
+      "conditions": [
+        "heartburn",
+        "indigestion"
+      ],
+      "uses": "Raising the head of the bed 4–6 inches (using blocks under the bed frame or a wedge support) prevents stomach acid from backing up into the esophagus during sleep, reducing nighttime heartburn and GERD symptoms.",
+      "how": "Place blocks under the legs at the head of the bed frame, or use a foam wedge under the mattress, to raise the sleeping surface about 6 inches. Using extra pillows under only the head is less effective because it bends the body at the waist rather than tilting the whole torso.",
+      "evidence": "MedlinePlus heartburn resources (sourced from NIH/NIDDK) consistently recommend elevating the head during sleep as a lifestyle measure for heartburn and GERD, noting that this position helps prevent reflux. Sleeping with the head raised about 6 inches is a specific recommendation described in the Medical Encyclopedia entry.",
+      "cautions": "Using extra pillows under the head is less effective than raising the entire upper body — pillows can cause neck strain and do not sufficiently change the angle. Heartburn that is frequent, severe, or accompanied by difficulty swallowing, unexplained weight loss, or vomiting blood requires medical evaluation.",
+      "sourceUrl": "https://medlineplus.gov/heartburn.html"
+    },
+    {
+      "slug": "bland-small-meals",
+      "name": "Small, frequent bland meals",
+      "commonNames": [
+        "BRAT diet",
+        "bland diet"
+      ],
+      "conditions": [
+        "nausea-vomiting",
+        "indigestion",
+        "diarrhea"
+      ],
+      "uses": "Eating 6–8 small bland meals throughout the day (crackers, toast, baked chicken, rice, potatoes) instead of 3 large meals reduces nausea and eases indigestion. As diarrhea symptoms improve, soft bland foods can be introduced gradually.",
+      "how": "Eat smaller meals more often and stick to bland foods, avoiding spicy, fatty, or salty options. If you have trouble keeping food down, start with small sips of clear liquids frequently and add bland solids only when tolerated.",
+      "evidence": "MedlinePlus (NLM) recommends small, frequent bland meals and avoiding spicy, fatty, or salty foods as the primary dietary self-care for nausea and vomiting. For indigestion, MedlinePlus notes that avoiding foods and situations that trigger symptoms is the main home strategy. For diarrhea, 'soft, bland food' is recommended as symptoms improve.",
+      "cautions": "The BRAT diet (bananas, rice, applesauce, toast) was historically promoted but MedlinePlus notes there is not strong evidence it is better than a standard bland diet; it probably does not cause harm. If nausea or vomiting persists beyond 24–48 hours, is accompanied by severe pain, or prevents adequate fluid intake, seek medical care.",
+      "sourceUrl": "https://medlineplus.gov/nauseaandvomiting.html"
+    },
+    {
+      "slug": "dark-quiet-room-headache",
+      "name": "Rest in a dark, quiet room (for headache)",
+      "commonNames": [],
+      "conditions": [
+        "headache",
+        "sleeplessness"
+      ],
+      "uses": "Resting with eyes closed in a dark, quiet room is a recommended non-medication self-care measure during a headache or migraine. Drinking water to prevent dehydration and placing a cool cloth on the forehead are often combined with this rest.",
+      "how": "Go to a quiet, darkened room, close your eyes, and rest. Drink water and place a cool damp cloth on your forehead to help ease discomfort.",
+      "evidence": "MedlinePlus (NLM) describes resting in a quiet, darkened room as one of the key things you can do to feel better during a headache or migraine, alongside drinking water and using relaxation techniques. These recommendations are attributed to NIH sources on headache and migraine management.",
+      "cautions": "Sudden severe ('thunderclap') headache, headache with fever, stiff neck, confusion, or vision changes may signal a serious condition and require emergency evaluation. Frequent headaches that disrupt daily life should be discussed with a healthcare provider rather than managed solely at home.",
+      "sourceUrl": "https://medlineplus.gov/headache.html"
+    },
+    {
+      "slug": "sleep-hygiene",
+      "name": "Sleep hygiene practices",
+      "commonNames": [
+        "good sleep habits"
+      ],
+      "conditions": [
+        "sleeplessness"
+      ],
+      "uses": "A consistent set of behavioral practices — consistent bedtime and wake time, a cool and dark bedroom, avoiding screens and caffeine near bedtime, and regular exercise — helps adults achieve and maintain adequate sleep.",
+      "how": "Go to bed and wake up at the same time every day. Keep the bedroom quiet, dark, and cool; turn off electronic devices at least 30 minutes before bedtime; and avoid caffeine in the afternoon and evening and large meals or alcohol before bed.",
+      "evidence": "CDC and NHLBI both recommend these specific practices for healthy sleep: going to bed and waking at the same time daily, keeping the bedroom quiet, cool, and dark, turning off screens at least 30 minutes before bed, avoiding caffeine in the afternoon and evening, and avoiding large meals or alcohol before sleep. NHLBI notes these habits are particularly important for shift workers and people with insomnia.",
+      "cautions": "Good sleep hygiene can help relieve short-term insomnia; persistent insomnia lasting more than a few weeks should be evaluated by a healthcare provider. Sleep difficulties accompanied by snoring, gasping during sleep, or excessive daytime sleepiness may indicate obstructive sleep apnea, which requires medical diagnosis.",
+      "sourceUrl": "https://www.cdc.gov/sleep/about/index.html"
+    },
+    {
+      "slug": "pinworm-hygiene",
+      "name": "Hygiene measures for pinworm",
+      "commonNames": [],
+      "conditions": [
+        "pinworm"
+      ],
+      "uses": "Thorough handwashing with soap and warm water (especially after toilet use and before eating), daily morning bathing with soap and water, daily underwear changes, short and clean fingernails, and laundering of bedding and pajamas in hot water are the key household self-care measures for managing pinworm infection.",
+      "how": "Bathe after waking up each morning and wash hands regularly, especially after using the bathroom. Change underwear daily, wash pajamas and bed sheets often, and avoid nail biting to prevent reinfection.",
+      "evidence": "CDC identifies handwashing as 'the most important way to prevent the spread of pinworms.' MedlinePlus (NIH/NIAID) lists bathe after waking up, wash pajamas and bed sheets often, wash hands regularly, change underwear every day, and avoid nail biting and scratching the anal area as the core preventive hygiene steps.",
+      "cautions": "Hygiene alone typically cannot eliminate an active pinworm infection — over-the-counter antiparasitic medication (pyrantel pamoate) is generally needed, and all household members should be treated simultaneously. Medication is typically repeated after 2 weeks because it kills worms but not eggs; the second dose treats worms that hatched after the first dose.",
+      "sourceUrl": "https://medlineplus.gov/pinworms.html"
+    },
+    {
+      "slug": "keep-feet-dry-athlete-foot",
+      "name": "Keep feet clean and dry (for fungal infection)",
+      "commonNames": [
+        "athlete's foot self-care"
+      ],
+      "conditions": [
+        "fungal-infection"
+      ],
+      "uses": "Keeping the feet clean, dry, and cool — including washing daily with soap and water, drying carefully between the toes, wearing clean cotton socks, and not walking barefoot in public showers or locker rooms — supports treatment and prevents spread of athlete's foot and other tinea infections.",
+      "how": "Keep feet clean, dry, and cool; wear clean socks and avoid walking barefoot in public areas such as locker room showers (use flip-flops instead). Apply an over-the-counter antifungal cream as directed on the package for most cases of athlete's foot.",
+      "evidence": "MedlinePlus (CDC-sourced) advises: keep your feet clean, dry, and cool; wear clean socks; avoid walking barefoot in public areas; wear flip-flops in locker room showers; keep toenails clean and clipped short. Over-the-counter antifungal creams work for most cases of athlete's foot.",
+      "cautions": "If over-the-counter antifungal treatment does not improve the infection within 2–4 weeks, see a healthcare provider. Spreading or worsening redness, warmth, and swelling — especially in people with diabetes or circulatory problems — warrants prompt medical care. Nail fungal infections are more difficult to treat than skin infections and often require prescription therapy.",
+      "sourceUrl": "https://medlineplus.gov/athletesfoot.html"
+    }
+  ]
+}

admin/app/data/natural_remedies.ts

@@ -0,0 +1,316 @@
+/* eslint-disable */
+/**
+ * GENERATED FILE — DO NOT EDIT.
+ *
+ * Single source of truth: collections/natural_remedies.json
+ * Regenerate after editing the JSON: `npm run gen:curated-data` (from admin/).
+ * curated_data_sync.standalone.ts fails CI if this file drifts from the JSON.
+ *
+ * Hand-curated subset of NCCIH "Herbs at a Glance" (nccih.nih.gov), mapped to the condition spine. US-government / public domain; attribution lives in the file `source` field and the UI caveat.
+ */
+import type { NaturalRemediesFile } from '../../types/conditions.js'
+
+export const NATURAL_REMEDIES_FILE: NaturalRemediesFile = {
+  "version": "2026-06-10",
+  "source": {
+    "name": "NCCIH — Herbs at a Glance",
+    "url": "https://www.nccih.nih.gov/health/herbsataglance",
+    "license": "Public domain (US government work; NCCIH)"
+  },
+  "remedies": [
+    {
+      "slug": "aloe-vera",
+      "name": "Aloe Vera",
+      "commonNames": [
+        "Aloe barbadensis miller"
+      ],
+      "conditions": [
+        "burns",
+        "acne",
+        "dry-skin"
+      ],
+      "uses": "Aloe vera gel is applied topically for burns, acne, and various skin conditions including psoriasis and radiation-related skin damage. Oral use is promoted for digestive conditions, though topical applications have more research support.",
+      "how": "Apply gel from the inner leaf of the aloe plant (or a commercially prepared aloe gel) directly to the affected skin area. Topical use is the application with the most research support; oral aloe products carry more serious risks and should not be used without consulting a healthcare provider.",
+      "evidence": "Research suggests topical aloe gel may speed burn healing and reduce burn-related pain. Two small studies indicate aloe gel, combined with other treatments, may improve acne; evidence for other skin uses is limited.",
+      "cautions": "Topical use is generally well tolerated, though occasional burning or itching may occur. Oral use carries more serious risks: the latex can cause abdominal pain and diarrhea, oral extracts have been linked to cases of acute hepatitis, and animal studies associated non-decolorized extracts with gastrointestinal cancer. Likely unsafe during pregnancy; may interact with medications such as digoxin.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/aloe-vera"
+    },
+    {
+      "slug": "boswellia",
+      "name": "Boswellia",
+      "commonNames": [
+        "Boswellia serrata",
+        "Indian frankincense"
+      ],
+      "conditions": [
+        "muscle-joint-pain"
+      ],
+      "uses": "Boswellia resin extract is traditionally used to reduce inflammation and pain, and is promoted as a dietary supplement to support joint health and mobility, particularly for osteoarthritis.",
+      "how": "Boswellia is taken orally as a dietary supplement. Clinical trials have used doses up to 1,000 mg daily for up to 6 months; follow the product label and consult a healthcare provider before starting.",
+      "evidence": "Some studies suggest oral boswellia may help reduce inflammation and pain associated with osteoarthritis, but larger rigorous trials are needed; topical use lacks sufficient evidence of effectiveness.",
+      "cautions": "Extracts up to 1,000 mg daily have been used safely in clinical trials lasting up to 6 months. Consult a healthcare provider before use, especially if pregnant, breastfeeding, managing asthma, or taking medications, as interactions remain unclear.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/boswellia"
+    },
+    {
+      "slug": "bromelain",
+      "name": "Bromelain",
+      "commonNames": [
+        "pineapple enzyme",
+        "Ananas comosus extract"
+      ],
+      "conditions": [
+        "muscle-joint-pain",
+        "nasal-congestion"
+      ],
+      "uses": "Bromelain, an enzyme from pineapple, is promoted for reducing postoperative pain and swelling, sinusitis, osteoarthritis, and exercise-induced muscle soreness. A topical formulation has FDA approval for debridement of severe burns.",
+      "evidence": "Some studies suggest oral bromelain may reduce certain symptoms after wisdom tooth surgery; evidence for sinusitis is insufficient. The topical formulation has been successfully used by health professionals for burn debridement as an alternative to surgical debridement.",
+      "cautions": "Oral bromelain is generally well tolerated; the most common side effects are stomach upset and diarrhea. Talk with a healthcare provider before use alongside any medications, as harmful interactions are possible. Safety during pregnancy and breastfeeding is unclear.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/bromelain"
+    },
+    {
+      "slug": "butterbur",
+      "name": "Butterbur",
+      "commonNames": [
+        "Petasites hybridus"
+      ],
+      "conditions": [
+        "headache",
+        "allergic-reaction"
+      ],
+      "uses": "Butterbur root extract is used for migraine prevention and for reducing symptoms of allergic rhinitis (hay fever). It has been studied for reducing migraine frequency in both adults and children.",
+      "how": "Butterbur is taken orally as a root extract for migraine prevention or as a leaf extract for allergic rhinitis symptoms. Use only products that are certified and labeled as free of pyrrolizidine alkaloids (PA-free), as the untreated plant contains compounds that can damage the liver.",
+      "evidence": "Studies of a butterbur root extract suggest it may reduce migraine frequency; a leaf extract may help with allergic rhinitis symptoms. However, the American Academy of Neurology withdrew its 2012 recommendation in 2015 due to safety concerns.",
+      "cautions": "The plant naturally contains pyrrolizidine alkaloids (PAs) that can damage the liver and lungs and may cause cancer; only PA-free certified products should be used. Even PA-free products have been linked to rare cases of liver injury. Side effects include belching, diarrhea, drowsiness, rash, and stomach upset. Avoid during pregnancy; people allergic to ragweed or related plants are at higher risk of reactions.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/butterbur"
+    },
+    {
+      "slug": "chamomile",
+      "name": "Chamomile",
+      "commonNames": [
+        "Matricaria chamomilla",
+        "German chamomile",
+        "Chamaemelum nobile"
+      ],
+      "conditions": [
+        "sleeplessness",
+        "indigestion",
+        "common-cold",
+        "sore-throat",
+        "skin-rash-itch"
+      ],
+      "uses": "Chamomile is promoted for insomnia, indigestion, anxiety, the common cold, and infant colic; it is also used topically for skin conditions and as a mouthwash for oral inflammation.",
+      "how": "Chamomile is commonly consumed as a tea (considered safe in amounts typically found in teas), taken as an oral supplement, applied topically to skin, or used as a mouthwash for oral inflammation. Consult a healthcare provider before using oral supplements rather than tea.",
+      "evidence": "Evidence is limited: some preliminary studies suggest chamomile supplements may help with anxiety, and combination products may help childhood diarrhea and infant colic. A 2019 review found minimal evidence for insomnia, with one study showing no benefit. Evidence for cold, sore throat, and skin uses in people is insufficient.",
+      "cautions": "Generally considered safe in typical amounts. Side effects may include nausea, dizziness, and allergic reactions, including severe hypersensitivity; risk is higher for those sensitive to ragweed, chrysanthemums, marigolds, or daisies. May interact with blood thinners, birth control pills, sedatives, and liver-metabolized drugs. Safety during pregnancy and breastfeeding is unknown.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/chamomile"
+    },
+    {
+      "slug": "echinacea",
+      "name": "Echinacea",
+      "commonNames": [
+        "Echinacea purpurea",
+        "purple coneflower"
+      ],
+      "conditions": [
+        "common-cold"
+      ],
+      "uses": "Echinacea is primarily marketed for the common cold and upper respiratory tract infections, based on the idea that it may support immune system function.",
+      "evidence": "Studies indicate that taking echinacea may slightly reduce the chances of catching a cold, though evidence regarding whether it shortens cold duration is inconclusive. Evidence for other conditions, including eczema, is unclear.",
+      "cautions": "E. purpurea extracts appear likely safe for short periods in adults; allergic reactions can occur, with digestive symptoms most common. Children may experience rashes potentially linked to allergic reactions. Theoretical interactions with immunosuppressants and certain other drugs; consult a healthcare provider before use with medications. Limited data on safety in early pregnancy.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/echinacea"
+    },
+    {
+      "slug": "elderberry",
+      "name": "Elderberry",
+      "commonNames": [
+        "Sambucus nigra"
+      ],
+      "conditions": [
+        "common-cold"
+      ],
+      "uses": "Elderberry has been used in folk medicine to treat colds and flu, and is promoted as a dietary supplement for colds, flu, and other upper respiratory infections.",
+      "how": "Elderberry is used as a prepared dietary supplement (syrup, capsule, or lozenge) rather than raw fruit. Never eat raw or unripe elderberries — they contain cyanide-producing substances that cause nausea, vomiting, and severe diarrhea; cooking the berries eliminates this toxin.",
+      "evidence": "A small number of studies suggest elderberry may relieve symptoms of flu, colds, or other upper respiratory infections; however, the overall evidence is limited and insufficient for most other claimed benefits.",
+      "cautions": "Raw or unripe elderberries contain cyanide-producing substances that can cause nausea, vomiting, and severe diarrhea; cooking eliminates this toxin. Consult a healthcare provider before use, especially if taking medications. Limited safety data for pregnancy and breastfeeding.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/elderberry"
+    },
+    {
+      "slug": "feverfew",
+      "name": "Feverfew",
+      "commonNames": [
+        "Tanacetum parthenium"
+      ],
+      "conditions": [
+        "headache"
+      ],
+      "uses": "Feverfew is promoted for migraine headache prevention and for minor head and tension pain. Topically, it is marketed for itching and skin irritation.",
+      "evidence": "A 2020 systematic review of seven migraine studies found inconsistent results, though some evidence suggests feverfew may reduce migraine frequency and associated symptoms such as nausea and light sensitivity. There is little or no evidence supporting feverfew for other health conditions.",
+      "cautions": "Side effects include nausea, digestive issues, bloating, and mouth sores from chewing fresh leaves. People sensitive to ragweed may experience allergic reactions. Feverfew may slow blood clotting and should be stopped at least 2 weeks before scheduled surgery. It may interact with migraine medications and should be avoided during pregnancy due to potential effects on uterine contractions.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/feverfew"
+    },
+    {
+      "slug": "ginger",
+      "name": "Ginger",
+      "commonNames": [
+        "Zingiber officinale"
+      ],
+      "conditions": [
+        "nausea-vomiting",
+        "menstrual-cramps",
+        "indigestion"
+      ],
+      "uses": "Ginger is traditionally and commonly used for nausea and vomiting, indigestion, menstrual cramps, and osteoarthritis.",
+      "evidence": "Research shows ginger may be helpful for nausea and vomiting associated with pregnancy. Studies suggest ginger dietary supplements might be helpful for reducing the severity of menstrual cramps and for knee osteoarthritis symptoms; effectiveness for chemotherapy- or post-surgery-related nausea remains uncertain.",
+      "cautions": "Side effects when taken orally include abdominal discomfort, heartburn, diarrhea, and mouth and throat irritation. Possible interactions with medications exist; consult a healthcare provider before use, especially during pregnancy or breastfeeding.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/ginger"
+    },
+    {
+      "slug": "goldenseal",
+      "name": "Goldenseal",
+      "commonNames": [
+        "Hydrastis canadensis"
+      ],
+      "conditions": [
+        "common-cold",
+        "diarrhea",
+        "wounds-cuts"
+      ],
+      "uses": "Goldenseal is marketed for the common cold and upper respiratory infections, diarrhea, constipation, and other digestive conditions. Historically, Native Americans used it for digestive disorders, wounds, and skin conditions.",
+      "evidence": "There is not enough evidence to determine whether goldenseal is useful for any health condition; no rigorous studies have been done in people. Very little of the active compound berberine is absorbed when goldenseal is taken orally, making findings from berberine studies potentially inapplicable.",
+      "cautions": "Short-term use at approximately 3 grams daily appears to be without serious adverse effects, though longer-term safety is uncertain. Goldenseal significantly affects drug metabolism; one study found it decreased metformin levels by about 25%. Some commercial products contain unlisted ingredients or substitute herbs. Avoid during pregnancy, breastfeeding, and in infants due to potential harm from berberine.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/goldenseal"
+    },
+    {
+      "slug": "horse-chestnut",
+      "name": "Horse Chestnut",
+      "commonNames": [
+        "Aesculus hippocastanum"
+      ],
+      "conditions": [
+        "hemorrhoids"
+      ],
+      "uses": "Horse chestnut seed extract is promoted for chronic venous insufficiency (leg pain and swelling from poor circulation) and has historically been used for hemorrhoids, arthritis, and menstrual cramps.",
+      "how": "Use only standardized horse chestnut seed extract (a dietary supplement) — never raw seeds, bark, flowers, or leaves, which are toxic when taken orally. Research has used standardized extract for up to 12 weeks; follow the product label and consult a healthcare provider before use.",
+      "evidence": "A 2012 systematic review found horse chestnut seed extract can improve symptoms of chronic venous insufficiency, comparable to compression stockings in one study; however, more rigorous trials are needed. Evidence for other uses, including hemorrhoids, is insufficient.",
+      "cautions": "Raw seeds, bark, flowers, and leaves are unsafe to take orally due to toxic components; only standardized extracts with the toxin removed should be used, and research shows safety for up to 12 weeks. Side effects include dizziness, digestive upset, headache, and itching. Safety during pregnancy and breastfeeding is unknown; consult a healthcare provider before use.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/horse-chestnut"
+    },
+    {
+      "slug": "lavender",
+      "name": "Lavender",
+      "commonNames": [
+        "Lavandula angustifolia"
+      ],
+      "conditions": [
+        "sleeplessness"
+      ],
+      "uses": "Lavender is promoted as an oral supplement and aromatherapy agent for calming anxiety, stress, and sleep difficulties; it is also used topically and in aromatherapy for depression symptoms and pain.",
+      "how": "Lavender is used as an oral supplement (capsule or tea), as an aromatherapy oil (inhaled or diffused), or applied topically to skin. Oral lavender oil products have been studied for anxiety; aromatherapy involves inhaling the scent from a diffuser or a few drops on a cloth.",
+      "evidence": "Studies suggest oral lavender oil might be beneficial for anxiety, including anxiety with co-occurring depression; evidence for sleep improvement is insufficient, and it is unclear whether aromatherapy with lavender benefits anxiety, stress, or depression.",
+      "cautions": "Oral lavender products may cause diarrhea, headache, nausea, or burping. Topical application can trigger allergic skin reactions. Potential interactions with sedative drugs warrant discussion with a healthcare provider. Safety during pregnancy and breastfeeding is unknown.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/lavender"
+    },
+    {
+      "slug": "licorice-root",
+      "name": "Licorice Root",
+      "commonNames": [
+        "Glycyrrhiza glabra"
+      ],
+      "conditions": [
+        "canker-sores",
+        "sore-throat",
+        "burns",
+        "skin-rash-itch"
+      ],
+      "uses": "Licorice root is promoted for digestive and respiratory support and is used topically for skin conditions. Research has focused on mouth rinses for canker sores, gargles or lozenges for sore throat prevention, and topical gels for eczema and burn healing.",
+      "how": "For canker sores, licorice is used as a mouth rinse or gargle; for sore throat prevention around surgery, as a gargle or lozenge; for eczema or burn healing, as a topical gel applied to the skin. Avoid long-term or high-dose oral use due to serious cardiovascular risks.",
+      "evidence": "Preliminary studies suggest licorice mouth rinses may reduce canker sore pain and size, and lozenges or gargles may prevent sore throat after surgical intubation. Topical gels show some promise for eczema symptoms and burn healing, though more research is needed. There is not enough high-quality evidence to support its use for most conditions.",
+      "cautions": "Licorice contains glycyrrhizin, which can cause serious adverse effects including irregular heartbeat and cardiac arrest, especially with long-term or large-dose use. Even small amounts can be risky for people with high blood pressure, high salt intake, or heart or kidney conditions. Large amounts during pregnancy increase miscarriage risk and are considered unsafe. Interactions with corticosteroids have been documented; consult a healthcare provider before use.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/licorice-root"
+    },
+    {
+      "slug": "passionflower",
+      "name": "Passionflower",
+      "commonNames": [
+        "Passiflora incarnata"
+      ],
+      "conditions": [
+        "sleeplessness"
+      ],
+      "uses": "Passionflower is marketed as a dietary supplement for anxiety, sleep problems, and stress.",
+      "how": "Passionflower is taken as a tea (studied for up to 7 nights) or as an oral extract supplement (studied for up to 8 weeks). Follow package directions and consult a healthcare provider before use, especially if you take other medications or are scheduled for surgery.",
+      "evidence": "Some research suggests oral passionflower may help reduce anxiety symptoms and improve total sleep time in adults with insomnia, though findings on falling or staying asleep are mixed and conclusions are not definite.",
+      "cautions": "Taking passionflower alongside anesthesia or other surgical medications may slow the nervous system too much; avoid use before surgery. Should not be used during pregnancy as it may induce uterine contractions. Possible side effects include drowsiness, dizziness, and confusion. Discuss with a healthcare provider before combining with medications.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/passionflower"
+    },
+    {
+      "slug": "peppermint-oil",
+      "name": "Peppermint Oil",
+      "commonNames": [
+        "Mentha x piperita"
+      ],
+      "conditions": [
+        "headache",
+        "nausea-vomiting",
+        "indigestion"
+      ],
+      "uses": "Peppermint oil is promoted for irritable bowel syndrome, indigestion, headaches, muscle tension, and nausea. Topical application is used for tension headaches, and inhaled peppermint oil is used for nausea.",
+      "how": "For IBS, peppermint oil is taken as enteric-coated capsules (which reduce heartburn compared to plain capsules). For tension headaches, apply peppermint oil topically to the skin. For nausea, peppermint oil is inhaled as aromatherapy.",
+      "evidence": "A 2022 review found peppermint oil was better than placebo at improving overall IBS symptoms and abdominal pain. A 2024 review concluded that inhaling peppermint oil was particularly effective at reducing nausea and vomiting in patients with cancer. Evidence suggests potential benefit for tension headaches when applied topically, though peppermint oil taken alone may worsen indigestion in some people.",
+      "cautions": "Oral side effects may include heartburn, nausea, abdominal pain, and dry mouth; enteric-coated capsules reduce heartburn risk. Never apply topically to infants' or young children's faces due to menthol's respiratory effects. Skin irritation is possible with topical use. Consult a healthcare provider before use with medications, as interactions may occur.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/peppermint-oil"
+    },
+    {
+      "slug": "tea-tree-oil",
+      "name": "Tea Tree Oil",
+      "commonNames": [
+        "Melaleuca alternifolia"
+      ],
+      "conditions": [
+        "fungal-infection",
+        "acne",
+        "wounds-cuts",
+        "insect-bites-stings",
+        "burns"
+      ],
+      "uses": "Tea tree oil is marketed for external use including acne, athlete's foot, toenail fungus, and lice. Traditional Aboriginal uses included treating wounds, burns, and insect bites.",
+      "how": "Apply tea tree oil topically to the affected skin area only — for acne or athlete's foot, use a product containing tea tree oil as directed on the label. Never swallow tea tree oil; oral ingestion can cause serious symptoms including confusion, unsteadiness, and coma.",
+      "evidence": "A small amount of research suggests tea tree oil might be helpful for acne and athlete's foot (fungal infection), though more studies are needed. Evidence for toenail fungus, lice, eyelid inflammation, and oral health uses is insufficient or uncertain. Traditional wound, burn, and insect bite uses lack rigorous clinical research.",
+      "cautions": "Critical warning: tea tree oil must not be swallowed — oral ingestion can cause serious symptoms including confusion, unsteadiness, inability to walk, and coma. Topical use is generally safe for most adults, though some experience skin redness or irritation. Older products or those exposed to heat, light, or air may increase reaction risk. Products appear safe during pregnancy and breastfeeding when used topically only.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/tea-tree-oil"
+    },
+    {
+      "slug": "turmeric",
+      "name": "Turmeric",
+      "commonNames": [
+        "Curcuma longa",
+        "curcumin"
+      ],
+      "conditions": [
+        "muscle-joint-pain",
+        "indigestion",
+        "skin-rash-itch"
+      ],
+      "uses": "Turmeric is promoted for osteoarthritis, itching, depression, allergies, and high cholesterol; topical applications target joint pain. Historically, it was used in traditional medicine for indigestion, colds, skin infections, and liver disease.",
+      "how": "Turmeric is taken orally as a dietary supplement or applied topically to the skin. Conventionally formulated oral turmeric or curcumin supplements are considered likely safe in recommended amounts for up to 2 to 3 months; follow the product label and consult a healthcare provider before use.",
+      "evidence": "For osteoarthritis, initial evidence is positive for knee pain relief and joint function, though higher-quality research is needed. For most other claimed uses, researchers cannot definitively confirm turmeric's effectiveness, and overall evidence remains inconclusive.",
+      "cautions": "Standard turmeric formulations are likely safe in recommended amounts for up to 2–3 months. Common side effects include nausea, vomiting, acid reflux, and digestive issues. Enhanced bioavailability products have caused liver damage in some users; symptoms include fatigue, dark urine, and jaundice. Use during pregnancy may be unsafe; breastfeeding safety is unclear. Consult a healthcare provider before use with medications.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/turmeric"
+    },
+    {
+      "slug": "valerian",
+      "name": "Valerian",
+      "commonNames": [
+        "Valeriana officinalis"
+      ],
+      "conditions": [
+        "sleeplessness"
+      ],
+      "uses": "Valerian is promoted for insomnia, anxiety, stress, and depression; historically, it was also used for migraine, fatigue, and stomach issues.",
+      "how": "Valerian is taken orally as a dietary supplement; clinical studies have used doses of 300 to 600 mg daily for up to 6 weeks. Follow the product label and consult a healthcare provider before use, particularly if combining with sedatives or alcohol.",
+      "evidence": "Evidence is limited and inconsistent. The American Academy of Sleep Medicine recommended against using valerian for chronic insomnia in adults (2017). Studies on anxiety, menstrual cramps, and other conditions show inadequate evidence of benefit.",
+      "cautions": "Appears generally safe for short-term use at 300–600 mg daily for up to 6 weeks; long-term safety is unknown. Common side effects include headache, stomach upset, mental dullness, and vivid dreams. Abrupt discontinuation after long-term use may cause withdrawal symptoms including anxiety and insomnia. Avoid combining with alcohol or sedatives. Safety during pregnancy and breastfeeding is unclear.",
+      "sourceUrl": "https://www.nccih.nih.gov/health/valerian"
+    }
+  ]
+}

admin/app/jobs/app_auto_update_job.ts

@@ -0,0 +1,78 @@
+import { Job } from 'bullmq'
+import { QueueService } from '#services/queue_service'
+import { DockerService } from '#services/docker_service'
+import { DownloadService } from '#services/download_service'
+import { SystemService } from '#services/system_service'
+import { ContainerRegistryService } from '#services/container_registry_service'
+import { AppAutoUpdateService } from '#services/app_auto_update_service'
+import logger from '@adonisjs/core/services/logger'
+
+/**
+ * Hourly job that evaluates whether any opted-in installed apps should auto-update
+ * right now and, if so, updates them. All gating (master switch, per-app opt-in,
+ * window, cool-off, pre-flight, per-app backoff) lives in {@link AppAutoUpdateService};
+ * this job is just the scheduled trigger. Runs hourly so it can act anywhere inside a
+ * user's window regardless of the window's length. Mirrors {@link AutoUpdateJob}.
+ */
+export class AppAutoUpdateJob {
+  static get queue() {
+    return 'system'
+  }
+
+  static get key() {
+    return 'app-auto-update'
+  }
+
+  async handle(_job: Job) {
+    logger.info('[AppAutoUpdateJob] Evaluating app auto-updates...')
+
+    const dockerService = new DockerService()
+    const appAutoUpdateService = new AppAutoUpdateService(
+      dockerService,
+      new DownloadService(QueueService.getInstance()),
+      new SystemService(dockerService),
+      new ContainerRegistryService()
+    )
+
+    const result = await appAutoUpdateService.attempt()
+    logger.info(`[AppAutoUpdateJob] ${result.updated} updated: ${result.reason}`)
+    return result
+  }
+
+  static async schedule() {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    await queue.upsertJobScheduler(
+      'hourly-app-auto-update',
+      { pattern: '0 * * * *' }, // Top of every hour; attempt() gates on the window
+      {
+        name: this.key,
+        opts: {
+          removeOnComplete: { count: 12 },
+          removeOnFail: { count: 5 },
+        },
+      }
+    )
+
+    logger.info('[AppAutoUpdateJob] App auto-update evaluation scheduled with cron: 0 * * * *')
+  }
+
+  static async dispatch() {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    const job = await queue.add(
+      this.key,
+      {},
+      {
+        attempts: 1,
+        removeOnComplete: { count: 12 },
+        removeOnFail: { count: 5 },
+      }
+    )
+
+    logger.info(`[AppAutoUpdateJob] Dispatched ad-hoc app auto-update evaluation job ${job.id}`)
+    return job
+  }
+}

admin/app/jobs/auto_update_job.ts

@@ -0,0 +1,80 @@
+import { Job } from 'bullmq'
+import { QueueService } from '#services/queue_service'
+import { DockerService } from '#services/docker_service'
+import { DownloadService } from '#services/download_service'
+import { SystemService } from '#services/system_service'
+import { SystemUpdateService } from '#services/system_update_service'
+import { ContainerRegistryService } from '#services/container_registry_service'
+import { AutoUpdateService } from '#services/auto_update_service'
+import logger from '@adonisjs/core/services/logger'
+
+/**
+ * Hourly job that evaluates whether the NOMAD application should auto-update right
+ * now and, if so, requests it. All gating (opt-in, window, eligibility, cool-off,
+ * pre-flight, backoff) lives in {@link AutoUpdateService}; this job is just the
+ * scheduled trigger. Runs hourly so it can act anywhere inside a user's window
+ * regardless of the window's length.
+ */
+export class AutoUpdateJob {
+  static get queue() {
+    return 'system'
+  }
+
+  static get key() {
+    return 'auto-update'
+  }
+
+  async handle(_job: Job) {
+    logger.info('[AutoUpdateJob] Evaluating auto-update...')
+
+    const dockerService = new DockerService()
+    const autoUpdateService = new AutoUpdateService(
+      dockerService,
+      new DownloadService(QueueService.getInstance()),
+      new SystemService(dockerService),
+      new SystemUpdateService(),
+      new ContainerRegistryService()
+    )
+
+    const result = await autoUpdateService.attempt()
+    logger.info(`[AutoUpdateJob] ${result.updated ? 'Updating' : 'No update'}: ${result.reason}`)
+    return result
+  }
+
+  static async schedule() {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    await queue.upsertJobScheduler(
+      'hourly-auto-update',
+      { pattern: '0 * * * *' }, // Top of every hour; attempt() gates on the window
+      {
+        name: this.key,
+        opts: {
+          removeOnComplete: { count: 12 },
+          removeOnFail: { count: 5 },
+        },
+      }
+    )
+
+    logger.info('[AutoUpdateJob] Auto-update evaluation scheduled with cron: 0 * * * *')
+  }
+
+  static async dispatch() {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    const job = await queue.add(
+      this.key,
+      {},
+      {
+        attempts: 1,
+        removeOnComplete: { count: 12 },
+        removeOnFail: { count: 5 },
+      }
+    )
+
+    logger.info(`[AutoUpdateJob] Dispatched ad-hoc auto-update evaluation job ${job.id}`)
+    return job
+  }
+}

admin/app/jobs/check_service_updates_job.ts

@@ -39,6 +39,14 @@ export class CheckServiceUpdatesJob {
 
         const latestUpdate = updates.length > 0 ? updates[0].tag : null
 
+        // Stamp/clear the cool-off anchor only when the available version *changes*.
+        // Registry tags carry no publish date, so the auto-update cool-off is measured
+        // from when a version was first detected; leaving the timestamp untouched while
+        // the same version persists keeps the cool-off clock running.
+        if (latestUpdate !== service.available_update_version) {
+          service.available_update_first_seen_at = latestUpdate ? DateTime.now() : null
+        }
+
         service.available_update_version = latestUpdate
         service.update_checked_at = DateTime.now()
         await service.save()

admin/app/jobs/content_auto_update_job.ts

@@ -0,0 +1,82 @@
+import { Job } from 'bullmq'
+import { QueueService } from '#services/queue_service'
+import { DownloadService } from '#services/download_service'
+import { ContentAutoUpdateService } from '#services/content_auto_update_service'
+import logger from '@adonisjs/core/services/logger'
+
+/**
+ * Hourly job that evaluates whether any installed content (ZIM/map) should
+ * auto-update right now and, if so, dispatches the downloads. All gating (master
+ * switch, content window, cool-off, per-window data cap, pre-flight, backoff)
+ * lives in {@link ContentAutoUpdateService}; this job is just the scheduled
+ * trigger. Runs hourly so it can act anywhere inside a user's window regardless
+ * of the window's length. Mirrors {@link AppAutoUpdateJob}.
+ */
+export class ContentAutoUpdateJob {
+  static get queue() {
+    return 'system'
+  }
+
+  static get key() {
+    return 'content-auto-update'
+  }
+
+  async handle(_job: Job) {
+    logger.info('[ContentAutoUpdateJob] Evaluating content auto-updates...')
+
+    const contentAutoUpdateService = new ContentAutoUpdateService(
+      new DownloadService(QueueService.getInstance())
+    )
+
+    const result = await contentAutoUpdateService.attempt()
+    logger.info(`[ContentAutoUpdateJob] ${result.started} started: ${result.reason}`)
+
+    // The FDA drug dataset refreshes on the same cycle and master switch as the
+    // ZIM/map catalog (its apply path differs, so it's a separate step, not part
+    // of attempt()). Governed by the same `contentAutoUpdate.*` settings.
+    const drugResult = await contentAutoUpdateService.attemptDrugDataset()
+    logger.info(`[ContentAutoUpdateJob] ${drugResult.started} started: ${drugResult.reason}`)
+
+    return {
+      started: result.started + drugResult.started,
+      reason: `${result.reason}; ${drugResult.reason}`,
+    }
+  }
+
+  static async schedule() {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    await queue.upsertJobScheduler(
+      'hourly-content-auto-update',
+      { pattern: '0 * * * *' }, // Top of every hour; attempt() gates on the window
+      {
+        name: this.key,
+        opts: {
+          removeOnComplete: { count: 12 },
+          removeOnFail: { count: 5 },
+        },
+      }
+    )
+
+    logger.info('[ContentAutoUpdateJob] Content auto-update evaluation scheduled with cron: 0 * * * *')
+  }
+
+  static async dispatch() {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    const job = await queue.add(
+      this.key,
+      {},
+      {
+        attempts: 1,
+        removeOnComplete: { count: 12 },
+        removeOnFail: { count: 5 },
+      }
+    )
+
+    logger.info(`[ContentAutoUpdateJob] Dispatched ad-hoc content auto-update evaluation job ${job.id}`)
+    return job
+  }
+}

admin/app/jobs/download_drug_data_job.ts

@@ -0,0 +1,345 @@
+import { Job } from 'bullmq'
+import { access, mkdir, constants } from 'node:fs/promises'
+import logger from '@adonisjs/core/services/logger'
+import { QueueService } from '#services/queue_service'
+import { doResumableDownload } from '../utils/downloads.js'
+import { parseDrugLabelManifest, partZipPath, manifestBytesTotal } from '../../util/drug_labels.js'
+import type {
+  DownloadDrugDataJobParams,
+  DrugLabelManifest,
+  DownloadStateMarker,
+  DrugDatasetResourceMeta,
+} from '../../types/drug_reference.js'
+
+/** Where all part zips are staged on the bind-mounted storage volume. */
+export const STORAGE_BASE = '/app/storage/drug-data'
+const MANIFEST_URL = 'https://api.fda.gov/download.json'
+
+/**
+ * Phase A — Download (network-only failure domain).
+ *
+ * Pass 0 fetches the openFDA manifest; each later pass downloads ONE part to
+ * disk (resumable via doResumableDownload, so a worker restart resumes from the
+ * .tmp rather than re-downloading). Continuations use queue.add with NO jobId —
+ * the same rule as the embed/ingest chains, so BullMQ dedupe doesn't swallow the
+ * next part against the lingering parent. After the last part lands we write the
+ * `drugReference.downloadState` KV marker and, when auto-chaining, dispatch the
+ * ingest phase. Parts are NEVER deleted here — they persist until a full ingest
+ * succeeds so the manual "Ingest into search" path can re-run from disk.
+ */
+export class DownloadDrugDataJob {
+  static get queue() {
+    return 'drug-download'
+  }
+
+  static get key() {
+    return 'download-drug-data'
+  }
+
+  /** Deterministic jobId — only one download at a time, re-runnable. */
+  static get jobId() {
+    return 'drug-data-download'
+  }
+
+  // ─── Public API ────────────────────────────────────────────────────────────
+
+  /**
+   * Dispatch the initial download (pass 0). Idempotent on the deterministic
+   * jobId. A finished/failed prior job under that id is cleared first so the
+   * "Download FDA data" button can always restart (resume is handled at the
+   * file level by doResumableDownload).
+   *
+   * @param autoChain - dispatch the ingest phase after the last part. Default true.
+   * @param resourceMeta - install-state identity when the install came through a
+   *   curated tier. Carried through the chain so the ingest job writes the
+   *   `installed_resources` row on `ready`. Omit for a manual download (no row).
+   */
+  static async dispatch(autoChain = true, resourceMeta?: DrugDatasetResourceMeta) {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    const existing = await queue.getJob(this.jobId)
+    if (existing) {
+      const state = await existing.getState()
+      if (state === 'active' || state === 'waiting' || state === 'delayed') {
+        return { job: existing, created: false, message: 'Drug data download already running' }
+      }
+      try {
+        await existing.remove()
+      } catch {
+        // Best-effort: fall through to add, which surfaces any genuine conflict.
+      }
+    }
+
+    try {
+      const job = await queue.add(
+        this.key,
+        { autoChain, resourceMeta } satisfies DownloadDrugDataJobParams,
+        {
+          jobId: this.jobId,
+          attempts: 3,
+          backoff: { type: 'exponential', delay: 5000 },
+          removeOnComplete: { count: 5 },
+          removeOnFail: { count: 5 },
+        }
+      )
+      return { job, created: true, message: 'Drug data download dispatched' }
+    } catch (error: unknown) {
+      const msg = error instanceof Error ? error.message : String(error)
+      if (msg.includes('job already exists')) {
+        const stillThere = await queue.getJob(this.jobId)
+        return { job: stillThere, created: false, message: 'Drug data download already running' }
+      }
+      throw error
+    }
+  }
+
+  static async getJob(): Promise<Job | undefined> {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+    return await queue.getJob(this.jobId)
+  }
+
+  // ─── Job handler ───────────────────────────────────────────────────────────
+
+  async handle(job: Job) {
+    const params = job.data as DownloadDrugDataJobParams
+    const partIndex = params.partIndex ?? 0
+    const autoChain = params.autoChain ?? true
+    const startedAt = params.startedAt ?? Date.now()
+    const resourceMeta = params.resourceMeta
+
+    logger.info(`[DownloadDrugDataJob] Starting pass partIndex=${partIndex}`)
+
+    // Pre-flight: storage drive writable.
+    await this.verifyStorageAvailable(job)
+
+    // Pass 0: fetch the manifest.
+    let manifest: DrugLabelManifest
+    if (partIndex === 0 || !params.manifest) {
+      await job.updateData({
+        ...job.data,
+        phase: 'manifest',
+        partIndex: 0,
+        totalParts: 0,
+        currentPartName: null,
+        autoChain,
+        startedAt,
+      })
+      await job.updateProgress(0)
+
+      logger.info('[DownloadDrugDataJob] Fetching manifest from api.fda.gov/download.json')
+      manifest = await this.fetchManifest()
+      logger.info(
+        `[DownloadDrugDataJob] Manifest: export_date=${manifest.export_date} ` +
+          `total_records=${manifest.total_records} parts=${manifest.partitions.length}`
+      )
+    } else {
+      manifest = params.manifest
+    }
+
+    const totalParts = params.totalParts ?? manifest.partitions.length
+
+    if (partIndex >= totalParts) {
+      logger.warn(
+        `[DownloadDrugDataJob] partIndex ${partIndex} >= totalParts ${totalParts}, nothing to do`
+      )
+      return
+    }
+
+    const partition = manifest.partitions[partIndex]
+    const zipPath = partZipPath(STORAGE_BASE, partition)
+    const partName = partition.display_name || partition.file
+
+    logger.info(
+      `[DownloadDrugDataJob] Downloading part ${partIndex + 1}/${totalParts}: ${partName}`
+    )
+
+    await job.updateData({
+      ...job.data,
+      phase: 'downloading',
+      partIndex,
+      totalParts,
+      currentPartName: partName,
+      manifest,
+      autoChain,
+      startedAt,
+      bytesDownloaded: 0,
+    })
+    await job.updateProgress(Math.floor((partIndex / totalParts) * 100))
+
+    await mkdir(STORAGE_BASE, { recursive: true })
+
+    // Aggregate-across-parts byte accounting for the Active Downloads card. The
+    // drug job fans the manifest's N partitions into BullMQ continuations, but
+    // the user sees ONE download — so progress is reported as bytes across the
+    // whole set, not the current part. `priorPartsBytes` is the actual bytes of
+    // the parts already on disk (from the running recordedParts list);
+    // `manifestTotalBytes` is the sum of every partition's manifest size_mb.
+    const priorRecorded =
+      (params as { recordedParts?: DownloadStateMarker['parts'] }).recordedParts ?? []
+    const priorPartsBytes = priorRecorded.reduce((acc, p) => acc + (p.bytes || 0), 0)
+    const manifestTotalBytes = manifestBytesTotal(manifest)
+
+    logger.info(`[DownloadDrugDataJob] ${partition.file} → ${zipPath}`)
+    let partBytes = 0
+    await doResumableDownload({
+      url: partition.file,
+      filepath: zipPath,
+      timeout: 300_000, // 5-minute per-chunk timeout
+      allowedMimeTypes: [], // skip MIME check — zip content-type varies across CDNs
+      onProgress: (progress) => {
+        partBytes = progress.downloadedBytes
+        const downloadFraction = progress.downloadedBytes / (progress.totalBytes || 1)
+        const pct = Math.floor(((partIndex + downloadFraction) / totalParts) * 100)
+        const downloadedBytes = priorPartsBytes + progress.downloadedBytes
+        // Emit the canonical {percent, downloadedBytes, totalBytes} object the
+        // Active Downloads aggregator + the byte/speed readout expect (the old
+        // bare-int progress couldn't carry bytes). parseProgress in
+        // DownloadService still tolerates a bare int for back-compat, but only
+        // the object surfaces a live byte/speed readout. Fire-and-forget; swallow
+        // transient reject so it can't crash the worker.
+        void job
+          .updateProgress({
+            percent: pct,
+            downloadedBytes,
+            totalBytes: manifestTotalBytes,
+            lastProgressTime: Date.now(),
+          })
+          .catch(() => {})
+        void job.updateData({ ...job.data, bytesDownloaded: progress.downloadedBytes }).catch(() => {})
+      },
+    })
+    logger.info(`[DownloadDrugDataJob] Download complete: ${zipPath} (${partBytes} bytes)`)
+
+    // Record this part for the download-state marker (written after the last one).
+    const recordedParts =
+      (params as { recordedParts?: DownloadStateMarker['parts'] }).recordedParts ?? []
+    recordedParts.push({ index: partIndex, name: partName, path: zipPath, bytes: partBytes })
+
+    const nextIndex = partIndex + 1
+
+    if (nextIndex < totalParts) {
+      // Continuation — NO jobId (let BullMQ auto-generate). The critical rule.
+      const queueService = QueueService.getInstance()
+      const queue = queueService.getQueue(DownloadDrugDataJob.queue)
+
+      const continuationParams: DownloadDrugDataJobParams & {
+        recordedParts: DownloadStateMarker['parts']
+      } = {
+        partIndex: nextIndex,
+        manifest,
+        totalParts,
+        autoChain,
+        startedAt,
+        recordedParts,
+        resourceMeta,
+      }
+
+      await queue.add(DownloadDrugDataJob.key, continuationParams, {
+        attempts: 3,
+        backoff: { type: 'exponential', delay: 5000 },
+        removeOnComplete: { count: 5 },
+        removeOnFail: { count: 5 },
+      })
+      logger.info(
+        `[DownloadDrugDataJob] Dispatched continuation for part ${nextIndex + 1}/${totalParts}`
+      )
+    } else {
+      // Last part — write the download-state marker. Parts stay on disk until a
+      // full ingest succeeds; do NOT delete them here.
+      await this.writeDownloadState(manifest, totalParts, recordedParts)
+      await job.updateData({
+        ...job.data,
+        phase: 'downloaded',
+        partIndex,
+        totalParts,
+        currentPartName: null,
+      })
+      await job.updateProgress(100)
+      logger.info(
+        `[DownloadDrugDataJob] All ${totalParts} parts downloaded. ` +
+          `export_date=${manifest.export_date}`
+      )
+
+      if (autoChain) {
+        const { IngestDrugDataJob } = await import('#jobs/ingest_drug_data_job')
+        // Forward the install-state identity so the ingest job writes the
+        // `installed_resources` row on `ready`. undefined for a manual download.
+        await IngestDrugDataJob.dispatch(resourceMeta)
+        logger.info('[DownloadDrugDataJob] Auto-chained ingest phase')
+      }
+    }
+
+    return { partIndex, totalParts }
+  }
+
+  // ─── Private helpers ───────────────────────────────────────────────────────
+
+  private async verifyStorageAvailable(job: Job): Promise<void> {
+    try {
+      await access(STORAGE_BASE, constants.W_OK)
+    } catch {
+      try {
+        await mkdir(STORAGE_BASE, { recursive: true })
+      } catch (mkdirErr) {
+        await job.updateData({ ...job.data, phase: 'failed' })
+        throw new Error(
+          `Storage drive not available: cannot write to ${STORAGE_BASE} (${
+            mkdirErr instanceof Error ? mkdirErr.message : String(mkdirErr)
+          })`
+        )
+      }
+    }
+  }
+
+  private async fetchManifest(): Promise<DrugLabelManifest> {
+    return DownloadDrugDataJob.fetchManifest()
+  }
+
+  /**
+   * Fetch + parse the openFDA download manifest. The SINGLE source of truth for
+   * the openFDA manifest call (Maxim 4): the download job uses it on pass 0, and
+   * the freshness check (DrugReferenceService.checkForUpdate, driven by
+   * attemptAutoUpdate) reuses it so there is exactly one place that knows the URL and the
+   * offline-error translation.
+   */
+  static async fetchManifest(): Promise<DrugLabelManifest> {
+    let json: unknown
+    try {
+      const resp = await fetch(MANIFEST_URL)
+      if (!resp.ok) {
+        throw new Error(`HTTP ${resp.status} from ${MANIFEST_URL}`)
+      }
+      json = await resp.json()
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      if (
+        msg.includes('ENOTFOUND') ||
+        msg.includes('ECONNREFUSED') ||
+        msg.includes('ECONNRESET') ||
+        msg.includes('fetch failed')
+      ) {
+        throw new Error(`No internet — connect to download FDA drug data. (${msg})`)
+      }
+      throw err
+    }
+    return parseDrugLabelManifest(json)
+  }
+
+  private async writeDownloadState(
+    manifest: DrugLabelManifest,
+    totalParts: number,
+    parts: DownloadStateMarker['parts']
+  ): Promise<void> {
+    const KVStore = (await import('#models/kv_store')).default
+    const marker: DownloadStateMarker = {
+      export_date: manifest.export_date,
+      totalParts,
+      totalRecords: manifest.total_records,
+      parts,
+      completedAtMs: Date.now(),
+    }
+    await KVStore.setValue('drugReference.downloadState', JSON.stringify(marker))
+  }
+}

admin/app/jobs/download_model_job.ts

@@ -37,7 +37,7 @@ export class DownloadModelJob {
     const queueService = QueueService.getInstance()
     const queue = queueService.getQueue(this.queue)
     const client = await queue.client
-    await client.set(this.cancelKey(jobId), '1', 'EX', 300) // 5 min TTL
+    await client.set(this.cancelKey(jobId), '1', { EX: 300 }) // 5 min TTL
   }
 
   async handle(job: Job) {

admin/app/jobs/embed_file_job.ts

@@ -18,6 +18,12 @@ export interface EmbedFileJobParams {
   batchOffset?: number  // Current batch offset (for ZIM files)
   totalArticles?: number // Total articles in ZIM (for progress tracking)
   isFinalBatch?: boolean // Whether this is the last batch (prevents premature deletion)
+  // Running total of chunks embedded across prior batches in this dispatch chain.
+  // Carried forward so the final batch can persist an accurate `chunks_embedded`
+  // count via KbIngestState.markIndexed (see #933 -- without this, only the last
+  // batch's chunk count was stored while Qdrant held the full set).
+  chunksSoFar?: number
+  collection?: string
 }
 
 export class EmbedFileJob {
@@ -51,7 +57,16 @@ export class EmbedFileJob {
   }
 
   async handle(job: Job) {
-    const { filePath, fileName, batchOffset, totalArticles } = job.data as EmbedFileJobParams
+    const { filePath, fileName, batchOffset, totalArticles, collection } = job.data as EmbedFileJobParams
+
+    // Only the direct KB-upload controller passes `collection` on dispatch; the other
+    // six dispatch sites (download auto-index, scan/sync, re-embed, local ZIM upload,
+    // replaced-file reconcile, and this job's own ZIM batch continuation) do not. Fall
+    // back to whatever the file is already assigned to, so an assignment made *before*
+    // the file was indexed still reaches the vectors. Resolving it here rather than at
+    // each dispatch site keeps one source of truth and covers batch continuations too.
+    const effectiveCollection =
+      collection ?? (await KbIngestState.findBy('file_path', filePath))?.collection ?? undefined
 
     const isZimBatch = batchOffset !== undefined
     const batchInfo = isZimBatch ? ` (batch offset: ${batchOffset})` : ''
@@ -131,7 +146,8 @@ export class EmbedFileJob {
         filePath,
         allowDeletion,
         batchOffset,
-        onProgress
+        onProgress,
+        effectiveCollection
       )
 
       if (!result.success) {
@@ -159,18 +175,49 @@ export class EmbedFileJob {
           await new Promise((resolve) => setTimeout(resolve, EmbedFileJob.CPU_BATCH_DELAY_MS))
         }
 
-        // Dispatch next batch (not final yet)
+        // Bail before re-populating the queue if this job was cancelled mid-batch.
+        // cancelAllJobs() obliterates the queue (including this active job), but a
+        // worker already inside handle() would otherwise dispatch its continuation
+        // afterwards and silently revive a cancelled ZIM ingestion. If our own job
+        // key is gone, the cancel happened — skip the dispatch. Mirrors the
+        // "tolerate external removal" handling in safeUpdateProgress above.
+        const stillQueued = await QueueService.getInstance()
+          .getQueue(EmbedFileJob.queue)
+          .getJob(job.id!)
+        if (!stillQueued) {
+          logger.info(
+            `[EmbedFileJob] Job ${fileName} was cancelled; skipping continuation dispatch`
+          )
+          return { success: false, cancelled: true, fileName, filePath }
+        }
+
+        // Dispatch next batch (not final yet). Carry forward the running
+        // chunk count so the final batch can persist an accurate total (#933).
+        const chunksSoFarNext = (job.data.chunksSoFar || 0) + (result.chunks || 0)
         await EmbedFileJob.dispatch({
           filePath,
           fileName,
           batchOffset: nextOffset,
           totalArticles: totalArticles || result.totalArticles,
           isFinalBatch: false, // Explicitly not final
+          chunksSoFar: chunksSoFarNext,
+          // Carry the collection across batches, otherwise only batch 1 of a ZIM
+          // would be tagged and the rest would land uncategorized.
+          ...(effectiveCollection ? { collection: effectiveCollection } : {}),
         })
 
-        // Calculate progress based on articles processed
+        // Calculate progress based on articles processed.
+        //
+        // nextOffset counts entries passing our isArticleEntry() filter, but the
+        // denominator (totalArticles = archive.articleCount) uses libzim's
+        // narrower article definition. On ZIMs that pack one logical article as
+        // several sub-pages (e.g. iFixit), nextOffset outruns articleCount and a
+        // raw ratio overflows past 100%, which the UI pins at 99% for the entire
+        // tail so the file looks stuck (#903). Grow the denominator once we pass
+        // the reported count so the gauge keeps creeping forward monotonically,
+        // and never report 100% before the genuinely-final batch (handled below).
         const progress = totalArticles
-          ? Math.round((nextOffset / totalArticles) * 100)
+          ? Math.min(99, Math.round((nextOffset / Math.max(totalArticles, nextOffset + ZIM_BATCH_SIZE)) * 100))
           : 50
 
         await this.safeUpdateProgress(job, progress)
@@ -178,7 +225,7 @@ export class EmbedFileJob {
           ...job.data,
           status: 'batch_completed',
           lastBatchAt: Date.now(),
-          chunks: (job.data.chunks || 0) + (result.chunks || 0),
+          chunks: chunksSoFarNext,
         })
 
         return {
@@ -192,8 +239,11 @@ export class EmbedFileJob {
         }
       }
 
-      // Final batch or non-batched file - mark as complete
+      // Final batch or non-batched file - mark as complete.
-      const totalChunks = (job.data.chunks || 0) + (result.chunks || 0)
+      // chunksSoFar carries the accumulated count from prior dispatched batches
+      // (each continuation passes it forward — see EmbedFileJobParams). For a
+      // non-batched file it is undefined and we just count this single result.
+      const totalChunks = (job.data.chunksSoFar || 0) + (result.chunks || 0)
       await this.safeUpdateProgress(job, 100)
       await job.updateData({
         ...job.data,
@@ -206,7 +256,7 @@ export class EmbedFileJob {
       // BullMQ's :completed retention (50 jobs) ages out, so the state row is
       // the only durable record of "this file finished embedding".
       try {
-        await KbIngestState.markIndexed(filePath, totalChunks)
+        await KbIngestState.markIndexed(filePath, totalChunks, effectiveCollection)
       } catch (stateErr) {
         logger.warn(
           `[EmbedFileJob] Failed to persist ingest state for ${fileName}: %s`,
@@ -227,23 +277,38 @@ export class EmbedFileJob {
         message: `Successfully embedded ${result.chunks} chunks`,
       }
     } catch (error) {
-      logger.error(`[EmbedFileJob] Error embedding file ${fileName}:`, error)
+      // A chunk that still exceeds the model's context after OllamaService's truncate-and-retry is
+      // permanently oversized for this install (e.g. a model whose context is smaller than our safe
+      // cap). Re-embedding the whole file 30x re-processes everything and can never succeed — that is
+      // the "endless queue loop" / "api/embed for weeks" (#881/#944/#959). Mark it unrecoverable so
+      // BullMQ stops after one pass instead of storming.
+      let normalizedError = error
+      if (!(error instanceof UnrecoverableError) && OllamaService.isContextLengthError(error)) {
+        logger.warn(
+          `[EmbedFileJob] Context-length overflow persisted for ${fileName} after truncation; not retrying.`
+        )
+        normalizedError = new UnrecoverableError(
+          error instanceof Error ? error.message : 'Embedding input exceeds the model context length'
+        )
+      }
+
+      logger.error(`[EmbedFileJob] Error embedding file ${fileName}:`, normalizedError)
 
       await job.updateData({
         ...job.data,
         status: 'failed',
         failedAt: Date.now(),
-        error: error instanceof Error ? error.message : 'Unknown error',
+        error: normalizedError instanceof Error ? normalizedError.message : 'Unknown error',
       })
 
       // Only persist `failed` for unrecoverable errors. Retryable errors get
       // automatic BullMQ retries (30 attempts); marking state failed on every
       // transient blip would suppress the retry-driven recovery path.
-      if (error instanceof UnrecoverableError) {
+      if (normalizedError instanceof UnrecoverableError) {
         try {
           await KbIngestState.markFailed(
             filePath,
-            error instanceof Error ? error.message : 'Unknown error'
+            normalizedError instanceof Error ? normalizedError.message : 'Unknown error'
           )
         } catch (stateErr) {
           logger.warn(
@@ -253,7 +318,7 @@ export class EmbedFileJob {
         }
       }
 
-      throw error
+      throw normalizedError
     }
   }
 
@@ -408,6 +473,46 @@ export class EmbedFileJob {
     return { cleaned, filesDeleted }
   }
 
+  /** Unconditionally clear every embedding job regardless of state.
+   *
+   *  cleanupFailedJobs only removes jobs explicitly tagged status === 'failed',
+   *  which leaves stuck jobs (waiting / active / delayed / paused that never
+   *  reached 'failed') unreachable from the UI — the operator's only recourse was
+   *  flushing Redis by hand. This wipes the whole queue, including a locked active
+   *  job, via obliterate({ force: true }) (plain obliterate/job.remove throw on a
+   *  locked job). It touches only Redis, so it is safe while Qdrant/Ollama are
+   *  offline — which is exactly when jobs pile up and wedge. */
+  static async cancelAllJobs(): Promise<{ cancelled: number; filesDeleted: number }> {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+    const jobs = await queue.getJobs(['waiting', 'active', 'delayed', 'paused', 'failed'])
+
+    let filesDeleted = 0
+    for (const job of jobs) {
+      const filePath = (job.data as EmbedFileJobParams).filePath
+      // Same guard as cleanupFailedJobs: only delete user uploads, never ZIM
+      // library files or Nomad docs that live outside the uploads path.
+      if (filePath && filePath.includes(RagService.UPLOADS_STORAGE_PATH)) {
+        try {
+          await fs.unlink(filePath)
+          filesDeleted++
+        } catch {
+          // File may already be deleted — that's fine
+        }
+      }
+    }
+
+    const cancelled = jobs.length
+
+    // force: true removes the locked/active job too. An in-flight worker may keep
+    // running its current batch in memory; the self-exists guard in handle()
+    // prevents it from dispatching a continuation back into the cleared queue.
+    await queue.obliterate({ force: true })
+
+    logger.info(`[EmbedFileJob] Cancelled ${cancelled} jobs, deleted ${filesDeleted} files`)
+    return { cancelled, filesDeleted }
+  }
+
   static async getStatus(filePath: string): Promise<{
     exists: boolean
     status?: string

admin/app/jobs/ingest_drug_data_job.ts

@@ -0,0 +1,837 @@
+import { Job } from 'bullmq'
+import { promises as fsPromises } from 'node:fs'
+import { access, constants } from 'node:fs/promises'
+import { Writable } from 'node:stream'
+import logger from '@adonisjs/core/services/logger'
+import { QueueService } from '#services/queue_service'
+import { mapDrugLabelRecord, parseDownloadState, partZipPath } from '../../util/drug_labels.js'
+import { STORAGE_BASE } from '#jobs/download_drug_data_job'
+import type {
+  IngestDrugDataJobParams,
+  DrugLabelManifest,
+  DrugLabelPartition,
+  DrugDatasetResourceMeta,
+} from '../../types/drug_reference.js'
+
+const BATCH_SIZE = 500
+
+/**
+ * Dedupe a batch by set_id, CASE-INSENSITIVELY (last occurrence wins). MySQL's
+ * uniq_drug_labels_set_id index uses a case-insensitive collation, so two ids
+ * differing only in case are ONE row to the database — the dedupe key must agree
+ * with the database's notion of equality or a batch can still collide with
+ * itself.
+ */
+function dedupeBySetId(
+  rows: ReturnType<typeof mapDrugLabelRecord>[]
+): NonNullable<ReturnType<typeof mapDrugLabelRecord>>[] {
+  const bySetId = new Map<string, NonNullable<ReturnType<typeof mapDrugLabelRecord>>>()
+  for (const r of rows) {
+    if (r) bySetId.set(r.set_id.toLowerCase(), r)
+  }
+  return [...bySetId.values()]
+}
+
+/**
+ * Upsert one batch with MySQL-native `INSERT … ON DUPLICATE KEY UPDATE`
+ * (knex onConflict().merge()) instead of Lucid's updateOrCreateMany.
+ *
+ * WHY: updateOrCreateMany SELECTs existing rows and matches them to incoming
+ * rows IN JAVASCRIPT — a case-sensitive string compare. The DB's unique key on
+ * set_id is case-INsensitive, so when openFDA ships the same set_id with
+ * different casing across parts (seen live: part 5's '93A0696B-…' colliding
+ * with an earlier part's variant), Lucid misses the match, INSERTs, and the
+ * unique key rejects it — aborting the run. A native upsert makes the unique
+ * key itself the arbiter: same-key rows update, new rows insert, intra-batch
+ * duplicates take the update path. No JS equality anywhere.
+ *
+ * `ingested_at` is stamped explicitly (raw knex bypasses Lucid's autoCreate);
+ * merge() refreshes every inserted column on conflict, preserving the previous
+ * "re-ingest updates the row + timestamp" behavior.
+ */
+async function upsertDrugLabelBatch(
+  rows: NonNullable<ReturnType<typeof mapDrugLabelRecord>>[]
+): Promise<number> {
+  if (rows.length === 0) return 0
+  const { default: db } = await import('@adonisjs/lucid/services/db')
+  const now = new Date()
+  const withTs = rows.map((r) => ({ ...r, ingested_at: now }))
+  await db.knexQuery().table('drug_labels').insert(withTs).onConflict('set_id').merge()
+  return rows.length
+}
+
+// A single 500-row updateOrCreateMany should finish in seconds even on the
+// FULLTEXT-indexed drug_labels table. If one batch exceeds this, the DB is
+// locked/overloaded — reject loudly so the ingest FAILS VISIBLY (and retries via
+// BullMQ) instead of hanging forever with the worker's lock still renewing,
+// which reads in the UI as a frozen "part 1 of 13, 0 rows". 0.2.714 removed the
+// un-awaited-update worker crash; this removes the remaining silent hang.
+const UPSERT_TIMEOUT_MS = 120_000
+
+// No record parsed within this window (the zip-open + JSON-parse stage, BEFORE the
+// first batch) means the part is almost certainly corrupt/truncated: yauzl's
+// inflate stream hangs with no 'end' and no 'error', so the ingest sat on
+// "part 1, 0 rows" forever with the worker's lock still renewing. Fail loud
+// instead. Once records flow, the per-batch upsert timeout governs.
+const STALL_MS = 90_000
+
+/**
+ * updateOrCreateMany with a hard timeout. mysql2 has no default query timeout, so
+ * a stuck DB call (metadata lock, exhausted connection pool, FULLTEXT stall)
+ * would otherwise never settle and the part-stream would back-pressure to a halt.
+ * Promise.race turns that into a rejection the caller can log + fail + retry.
+ */
+async function withUpsertTimeout<T>(
+  work: Promise<T>,
+  rowCount: number,
+  timeoutMs: number
+): Promise<T> {
+  let timer: ReturnType<typeof setTimeout> | undefined
+  const timeout = new Promise<never>((_, reject) => {
+    timer = setTimeout(
+      () =>
+        reject(
+          new Error(
+            `updateOrCreateMany timed out after ${timeoutMs}ms (batch of ${rowCount} rows) — ` +
+              'the database may be locked or overloaded'
+          )
+        ),
+      timeoutMs
+    )
+  })
+  try {
+    return await Promise.race([work, timeout])
+  } finally {
+    if (timer) clearTimeout(timer)
+  }
+}
+
+// ─── Local type aliases for yauzl callbacks ───────────────────────────────────
+// yauzl/stream-json are loaded via dynamic import() with @ts-ignore (see
+// streamIngestPart). The real @types ship in devDependencies and resolve on the
+// target machine; loading them lazily keeps the pure util/ helpers importable in
+// tests without the streaming deps. These local interfaces give the callback
+// parameters explicit types without importing yauzl's own types (which aren't
+// resolvable in the inertia tsconfig context).
+import type { Readable } from 'node:stream'
+
+interface YauzlEntry { fileName: string }
+interface YauzlZipFile {
+  readEntry(): void
+  openReadStream(entry: YauzlEntry, cb: (err: Error | null, stream: Readable | null) => void): void
+  on(event: 'entry', listener: (entry: YauzlEntry) => void): this
+  on(event: 'end', listener: () => void): this
+  on(event: 'error', listener: (err: Error) => void): this
+}
+
+/**
+ * Phase B — Ingest (parse/DB-only failure domain, ZERO network I/O).
+ *
+ * Each pass reads ONE on-disk part and streams it into drug_labels via the
+ * memory-safe streamIngestPart pipeline (yauzl → stream-json → batched
+ * updateOrCreateMany). The part list comes from the manifest in job data OR, for
+ * a manual "Ingest into search" run with no manifest, is rebuilt from the
+ * `drugReference.downloadState` KV marker. A missing on-disk part fails loudly
+ * ("run Download first") rather than silently under-ingesting. Continuations use
+ * queue.add with NO jobId. After the LAST part: write the final KV status, then
+ * delete the downloaded parts and clear the download-state marker (the per-part
+ * unlink that used to run during download moves here — parts persist until a
+ * full ingest succeeds).
+ */
+export class IngestDrugDataJob {
+  static get queue() {
+    return 'drug-ingest'
+  }
+
+  static get key() {
+    return 'ingest-drug-data'
+  }
+
+  /** Deterministic jobId — only one ingest at a time, re-runnable. */
+  static get jobId() {
+    return 'drug-labels-ingest'
+  }
+
+  // ─── Public API ────────────────────────────────────────────────────────────
+
+  /**
+   * Dispatch the initial ingest (pass 0). Idempotent on the deterministic jobId.
+   * A finished/failed prior job under that id is cleared first so a re-ingest can
+   * always restart (upserts are idempotent on set_id).
+   *
+   * @param resourceMeta - install-state identity, present only when the install
+   *   came through a curated tier. Carried through the chain so the final pass
+   *   writes the `installed_resources` row on `ready`. Absent on a manual ingest.
+   */
+  static async dispatch(resourceMeta?: DrugDatasetResourceMeta) {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+
+    const existing = await queue.getJob(this.jobId)
+    if (existing) {
+      const state = await existing.getState()
+      if (state === 'active' || state === 'waiting' || state === 'delayed') {
+        return { job: existing, created: false, message: 'Drug label ingest already running' }
+      }
+      try {
+        await existing.remove()
+      } catch {
+        // Best-effort: fall through to add.
+      }
+    }
+
+    try {
+      const job = await queue.add(
+        this.key,
+        { resourceMeta } satisfies IngestDrugDataJobParams,
+        {
+          jobId: this.jobId,
+          attempts: 3,
+          backoff: { type: 'exponential', delay: 5000 },
+          removeOnComplete: { count: 5 },
+          removeOnFail: { count: 5 },
+        }
+      )
+      return { job, created: true, message: 'Drug label ingest dispatched' }
+    } catch (error: unknown) {
+      const msg = error instanceof Error ? error.message : String(error)
+      if (msg.includes('job already exists')) {
+        const stillThere = await queue.getJob(this.jobId)
+        return { job: stillThere, created: false, message: 'Drug label ingest already running' }
+      }
+      throw error
+    }
+  }
+
+  static async getJob(): Promise<Job | undefined> {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(this.queue)
+    return await queue.getJob(this.jobId)
+  }
+
+  // ─── Job handler ───────────────────────────────────────────────────────────
+
+  async handle(job: Job) {
+    const params = job.data as IngestDrugDataJobParams
+    const partIndex = params.partIndex ?? 0
+    const runningIngested = params.recordsIngested ?? 0
+    const runningSkipped = params.recordsSkipped ?? 0
+    const startedAt = params.startedAt ?? Date.now()
+    const resourceMeta = params.resourceMeta
+
+    // Progress baseline (pass 0 only): the table's row count BEFORE this run.
+    // Without it, a re-ingest into a populated table shows ~100% from second
+    // zero because the live row count dominates the shown counter.
+    let startRowCount = params.startRowCount
+    if (startRowCount === undefined) {
+      try {
+        const { default: db } = await import('@adonisjs/lucid/services/db')
+        const result = await db.rawQuery('SELECT COUNT(*) AS cnt FROM drug_labels')
+        const rows = result[0] as Array<{ cnt: number | string }>
+        startRowCount = Number(rows[0]?.cnt ?? 0)
+      } catch {
+        startRowCount = 0
+      }
+    }
+
+    logger.info(`[IngestDrugDataJob] Starting pass partIndex=${partIndex}`)
+
+    // Resolve the part list: manifest in job data, else the KV download marker.
+    const { manifest, exportDate } = await this.resolvePartSource(params)
+    const totalParts = params.totalParts ?? manifest.partitions.length
+
+    if (partIndex >= totalParts) {
+      logger.warn(
+        `[IngestDrugDataJob] partIndex ${partIndex} >= totalParts ${totalParts}, nothing to do`
+      )
+      return
+    }
+
+    const partition = manifest.partitions[partIndex]
+    const zipPath = partZipPath(STORAGE_BASE, partition)
+    const partName = partition.display_name || partition.file
+
+    // Guard: the part MUST already be on disk. No re-download here — fail loud so
+    // a missing part can't silently produce a "ready" status with fewer rows.
+    try {
+      await access(zipPath, constants.R_OK)
+    } catch {
+      await job.updateData({ ...job.data, phase: 'failed' })
+      throw new Error(
+        `Part ${partIndex + 1}/${totalParts} not downloaded (${zipPath}). ` +
+          'Run Download FDA data first.'
+      )
+    }
+
+    logger.info(
+      `[IngestDrugDataJob] Ingesting part ${partIndex + 1}/${totalParts}: ${partName}`
+    )
+
+    await job.updateData({
+      ...job.data,
+      phase: 'ingesting',
+      partIndex,
+      totalParts,
+      currentPartName: partName,
+      recordsIngested: runningIngested,
+      recordsSkipped: runningSkipped,
+      manifest,
+      startedAt,
+      startRowCount,
+    })
+    await job.updateProgress(Math.floor((partIndex / totalParts) * 100))
+
+    const { recordsIngested: partIngested, recordsSkipped: partSkipped } =
+      await this.streamIngestPart(
+        job,
+        zipPath,
+        partIndex,
+        totalParts,
+        runningIngested,
+        runningSkipped
+      )
+
+    const totalIngested = runningIngested + partIngested
+    const totalSkipped = runningSkipped + partSkipped
+
+    logger.info(
+      `[IngestDrugDataJob] Part ${partIndex + 1} done: ` +
+        `ingested=${partIngested} skipped=${partSkipped} running_total=${totalIngested}`
+    )
+
+    const nextIndex = partIndex + 1
+
+    if (nextIndex < totalParts) {
+      // Continuation — NO jobId. The critical rule.
+      const queueService = QueueService.getInstance()
+      const queue = queueService.getQueue(IngestDrugDataJob.queue)
+
+      const continuationParams: IngestDrugDataJobParams = {
+        partIndex: nextIndex,
+        manifest,
+        totalParts,
+        recordsIngested: totalIngested,
+        recordsSkipped: totalSkipped,
+        startedAt,
+        startRowCount,
+        resourceMeta,
+      }
+
+      await queue.add(IngestDrugDataJob.key, continuationParams, {
+        attempts: 3,
+        backoff: { type: 'exponential', delay: 5000 },
+        removeOnComplete: { count: 5 },
+        removeOnFail: { count: 5 },
+      })
+      logger.info(`[IngestDrugDataJob] Dispatched continuation for part ${nextIndex + 1}/${totalParts}`)
+
+      await job.updateData({
+        ...job.data,
+        phase: 'ingesting',
+        partIndex,
+        totalParts,
+        recordsIngested: totalIngested,
+        recordsSkipped: totalSkipped,
+      })
+    } else {
+      // Final part — write KV status, mark ready, THEN reclaim disk.
+      await this.writeFinalStatus(exportDate, resourceMeta)
+
+      await job.updateData({
+        ...job.data,
+        phase: 'ready',
+        partIndex,
+        totalParts,
+        currentPartName: null,
+        recordsIngested: totalIngested,
+        recordsSkipped: totalSkipped,
+      })
+      await job.updateProgress(100)
+
+      logger.info(
+        `[IngestDrugDataJob] Ingest complete. ` +
+          `total_ingested=${totalIngested} total_skipped=${totalSkipped} ` +
+          `export_date=${exportDate}`
+      )
+
+      // Reclaim disk only after a FULL ingest succeeds.
+      await this.deleteDownloadedParts(manifest, totalParts)
+    }
+
+    return { partIndex, totalIngested, totalSkipped }
+  }
+
+  // ─── Private helpers ───────────────────────────────────────────────────────
+
+  /**
+   * Resolve the ordered partition list + export_date for this ingest run.
+   *
+   * Prefers the manifest carried in job data (auto-chained from the download job,
+   * or a continuation pass). Falls back to the KV download-state marker so a
+   * manual "Ingest into search" with no manifest still works — the marker stores
+   * each part's on-disk path and manifest index, which is enough to drive
+   * streamIngestPart without re-fetching the manifest from the network. Fails
+   * loudly if neither source is present.
+   */
+  private async resolvePartSource(
+    params: IngestDrugDataJobParams
+  ): Promise<{ manifest: DrugLabelManifest; exportDate: string }> {
+    if (params.manifest) {
+      return { manifest: params.manifest, exportDate: params.manifest.export_date }
+    }
+
+    const KVStore = (await import('#models/kv_store')).default
+    const marker = parseDownloadState(await KVStore.getValue('drugReference.downloadState'))
+    if (!marker) {
+      throw new Error('Nothing downloaded — run Download FDA data first.')
+    }
+
+    // Rebuild a manifest-shaped partition list from the marker. partZipPath uses
+    // path.basename(partition.file), so feeding the recorded path as `file`
+    // resolves back to the same on-disk path.
+    const ordered = [...marker.parts].sort((a, b) => a.index - b.index)
+    const partitions: DrugLabelPartition[] = ordered.map((p) => ({
+      display_name: p.name,
+      file: p.path,
+      size_mb: '0',
+      records: 0,
+    }))
+
+    const manifest: DrugLabelManifest = {
+      export_date: marker.export_date,
+      // The marker persists the real manifest total (~259k) so a rebuilt
+      // manifest carries the same label-count denominator the auto-chained run
+      // would have had — keeping the "X of ~259k" counter, the records-based
+      // progress %, and the ETA alive on the manual-ingest path. Pre-totalRecords
+      // markers parse back as 0; the service treats 0 as unknown and falls back.
+      total_records: marker.totalRecords,
+      partitions,
+    }
+    return { manifest, exportDate: marker.export_date }
+  }
+
+  /**
+   * Stream-unzip the part, stream-parse the JSON, batch-upsert into drug_labels.
+   *
+   * Memory-safe: never loads the full JSON into memory.
+   * Pipeline: yauzl entry read-stream → stream-json Pick+StreamArray → Writable batching.
+   * Back-pressure: the Writable's `write()` method calls `callback()` only after
+   * the DB upsert resolves, so Node's stream machinery naturally pauses the upstream
+   * pipe chain when BATCH_SIZE is reached and an upsert is in flight.
+   */
+  private async streamIngestPart(
+    job: Job,
+    zipPath: string,
+    partIndex: number,
+    totalParts: number,
+    runningIngested: number,
+    runningSkipped: number
+  ): Promise<{ recordsIngested: number; recordsSkipped: number }> {
+    // Dynamic imports for the streaming deps (yauzl, stream-json). @ts-ignore
+    // covers local dev where node_modules hasn't been refreshed with the new
+    // deps yet; it is a no-op once the real @types are installed on the target
+    // machine, and it also covers stream-json's deep `.js` subpaths that the
+    // @types package doesn't map. Importing here (not at module top) keeps the
+    // pure util/ helpers loadable in tests without these deps.
+    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+    // @ts-ignore — yauzl resolved at runtime from dependencies
+    const yauzl = await import('yauzl')
+    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+    // @ts-ignore — stream-json resolved at runtime from dependencies
+    const createParser = (await import('stream-json')).default.parser
+    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+    // @ts-ignore — stream-json deep subpath resolved at runtime
+    const createPick = (await import('stream-json/filters/Pick.js')).default.pick
+    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+    // @ts-ignore — stream-json deep subpath resolved at runtime
+    const createStreamArray = (await import('stream-json/streamers/StreamArray.js')).default.streamArray
+
+    // Defensive: stream-json 1.9.1 (CJS) exposes its factories only as
+    // `.default.parser` / `.default.pick` / `.default.streamArray` under ESM
+    // dynamic import — there is NO `parser`/`pick`/`streamArray` named export, so
+    // destructuring those silently yielded `undefined`. Calling an undefined
+    // "factory" then threw inside the yauzl openReadStream callback (not a
+    // promise), the process-level backstop swallowed the uncaughtException, the
+    // streamIngestPart promise never settled, and the 90s watchdog fired in a
+    // retry loop with zero records. THIS was the real ingest hang. Validate the
+    // imports here so any future interop slip fails loud in the async body.
+    if (
+      typeof createParser !== 'function' ||
+      typeof createPick !== 'function' ||
+      typeof createStreamArray !== 'function'
+    ) {
+      throw new Error(
+        'stream-json factory imports did not resolve to functions ' +
+          `(parser=${typeof createParser}, pick=${typeof createPick}, streamArray=${typeof createStreamArray})`
+      )
+    }
+
+    let recordsIngested = 0
+    let recordsSkipped = 0
+    let batch: ReturnType<typeof mapDrugLabelRecord>[] = []
+    let batchNum = 0
+    let firstRecordSeen = false
+
+    // Cast to `any` so callback parameters get explicit annotations below rather
+    // than triggering implicit-any in tsconfigs that don't find the yauzl types.
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+    const yauzlOpen = (yauzl as any).open as (
+      path: string,
+      opts: { lazyEntries: boolean; autoClose: boolean },
+      cb: (err: Error | null, zipFile: YauzlZipFile | null) => void
+    ) => void
+
+    return new Promise<{ recordsIngested: number; recordsSkipped: number }>((resolveRaw, rejectRaw) => {
+      // Stall watchdog. The per-batch upsert has a timeout, but the stage BEFORE
+      // the first record (zip-open + JSON parse) had none — a corrupt/truncated
+      // part makes yauzl's inflate stream hang with no 'end' and no 'error', so
+      // the ingest froze on "part 1, 0 rows". If no record is parsed within
+      // STALL_MS, fail the part loudly (the reason lands in the job's failedReason
+      // → the status panel) and destroy the stuck stream. resolve/reject are
+      // wrapped (shadowing the raw executor params) so every existing handler
+      // routes through the once-guard + watchdog cleanup.
+      let settled = false
+      let activeReadStream: Readable | null = null
+      let watchdog: ReturnType<typeof setInterval> | undefined
+      const watchStart = Date.now()
+      const settle = () => {
+        settled = true
+        if (watchdog) clearInterval(watchdog)
+        try {
+          activeReadStream?.destroy()
+        } catch {
+          // best-effort stream teardown
+        }
+      }
+      const resolve = (v: { recordsIngested: number; recordsSkipped: number }) => {
+        if (settled) return
+        settle()
+        resolveRaw(v)
+      }
+      const reject = (e: Error) => {
+        if (settled) return
+        settle()
+        rejectRaw(e)
+      }
+      watchdog = setInterval(() => {
+        if (!settled && !firstRecordSeen && Date.now() - watchStart > STALL_MS) {
+          reject(
+            new Error(
+              `Ingest stalled: no records parsed from part ${partIndex + 1}/${totalParts} ` +
+                `within ${Math.round(STALL_MS / 1000)}s. The downloaded part is likely corrupt ` +
+                'or truncated — re-download FDA data, then ingest again.'
+            )
+          )
+        }
+      }, 10_000)
+
+      yauzlOpen(zipPath, { lazyEntries: true, autoClose: true }, (err, zipFile) => {
+        if (err || !zipFile) {
+          reject(err ?? new Error(`Failed to open zip: ${zipPath}`))
+          return
+        }
+
+        zipFile.on('error', reject)
+        zipFile.readEntry()
+
+        zipFile.on('entry', (entry) => {
+          // Skip directory entries
+          if (/\/$/.test(entry.fileName)) {
+            zipFile.readEntry()
+            return
+          }
+
+          // Open the single JSON entry as a read stream — never buffer it
+          zipFile.openReadStream(entry, (streamErr, readStream) => {
+            if (streamErr || !readStream) {
+              reject(streamErr ?? new Error(`Could not open zip entry ${entry.fileName}`))
+              return
+            }
+
+            logger.info(
+              `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: reading zip entry ${entry.fileName}`
+            )
+            activeReadStream = readStream
+
+            // The JSON envelope is { meta, results: [...] }
+            // Pick the `results` path → StreamArray emits one record at a time
+            const jsonParser = createParser({ jsonStreaming: false })
+            const pick = createPick({ filter: 'results' })
+            const streamArray = createStreamArray()
+
+            // Writable that accumulates batches and flushes with back-pressure.
+            // The callback is called only after the async upsert completes, which
+            // naturally applies back-pressure via the pipe chain.
+            const batchWriter = new Writable({
+              objectMode: true,
+              write(chunk: { value: unknown }, _encoding: BufferEncoding, callback: (err?: Error | null) => void) {
+                const record = chunk.value
+                if (!firstRecordSeen) {
+                  firstRecordSeen = true
+                  logger.info(
+                    `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: first record received from parser`
+                  )
+                }
+
+                // Map the record
+                let row: ReturnType<typeof mapDrugLabelRecord>
+                try {
+                  row = mapDrugLabelRecord(record as Parameters<typeof mapDrugLabelRecord>[0])
+                } catch (mapErr) {
+                  logger.warn(
+                    `[IngestDrugDataJob] mapDrugLabelRecord threw: ${mapErr instanceof Error ? mapErr.message : String(mapErr)}`
+                  )
+                  recordsSkipped++
+                  callback()
+                  return
+                }
+
+                if (!row) {
+                  recordsSkipped++
+                  callback()
+                  return
+                }
+
+                batch.push(row)
+
+                if (batch.length < BATCH_SIZE) {
+                  // Not full yet — don't block the stream
+                  callback()
+                  return
+                }
+
+                // Batch full — flush and hold the callback until the upsert
+                // resolves (this is the back-pressure point).
+                const currentBatch = dedupeBySetId(batch)
+                batch = []
+                const myBatch = ++batchNum
+                const t0 = Date.now()
+                logger.info(
+                  `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: upserting batch ${myBatch} (${currentBatch.length} rows)…`
+                )
+
+                withUpsertTimeout(
+                  upsertDrugLabelBatch(currentBatch),
+                  currentBatch.length,
+                  UPSERT_TIMEOUT_MS
+                )
+                  .then((rowCount) => {
+                    recordsIngested += rowCount
+                    logger.info(
+                      `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: batch ${myBatch} ok in ${Date.now() - t0}ms ` +
+                        `(${rowCount} rows; part running ${recordsIngested})`
+                    )
+
+                    // Update progress: parts-done fraction + within-part fraction
+                    const withinFraction = recordsIngested / Math.max(1, 20000)
+                    const pct = Math.floor(
+                      ((partIndex + withinFraction) / totalParts) * 100
+                    )
+                    // Fire-and-forget progress writes. Swallow transient Redis/
+                    // job-update rejections: an un-awaited reject would otherwise
+                    // bubble to an unhandledRejection and crash the worker, which
+                    // BullMQ then reports as "job stalled more than allowable
+                    // limit" (a dead worker stops renewing its lock).
+                    void job
+                      .updateProgress(
+                        Math.min(pct, Math.floor(((partIndex + 1) / totalParts) * 100) - 1)
+                      )
+                      .catch(() => {})
+                    void job
+                      .updateData({
+                        ...job.data,
+                        recordsIngested: runningIngested + recordsIngested,
+                        recordsSkipped: runningSkipped + recordsSkipped,
+                      })
+                      .catch(() => {})
+                    callback()
+                  })
+                  .catch((upsertErr: unknown) => {
+                    const msg = upsertErr instanceof Error ? upsertErr.message : String(upsertErr)
+                    if (/timed out/i.test(msg)) {
+                      // Systemic DB hang — fail the part loudly so BullMQ retries.
+                      logger.error(
+                        `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: batch ${myBatch} TIMED OUT after ${Date.now() - t0}ms: ${msg}`
+                      )
+                      callback(upsertErr instanceof Error ? upsertErr : new Error(msg))
+                      return
+                    }
+                    // Per-batch data error (a row the schema rejects, etc.) — log,
+                    // count as skipped, and CONTINUE. One bad batch must not abort
+                    // the whole ~259k ingest (over-correcting to fail-loud here is
+                    // what let a single duplicate set_id kill the run at part 5).
+                    logger.error(
+                      `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: batch ${myBatch} skipped after error (${Date.now() - t0}ms): ${msg}`
+                    )
+                    recordsSkipped += currentBatch.length
+                    callback()
+                  })
+              },
+              final(callback: (err?: Error | null) => void) {
+                // Flush the last partial batch
+                if (batch.length === 0) {
+                  callback()
+                  return
+                }
+
+                const remainingBatch = dedupeBySetId(batch)
+                batch = []
+                const t0 = Date.now()
+                logger.info(
+                  `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: upserting final batch (${remainingBatch.length} rows)…`
+                )
+
+                withUpsertTimeout(
+                  upsertDrugLabelBatch(remainingBatch),
+                  remainingBatch.length,
+                  UPSERT_TIMEOUT_MS
+                )
+                  .then((rowCount) => {
+                    recordsIngested += rowCount
+                    logger.info(
+                      `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: final batch ok in ${Date.now() - t0}ms (${rowCount} rows)`
+                    )
+                    callback()
+                  })
+                  .catch((upsertErr: unknown) => {
+                    const msg = upsertErr instanceof Error ? upsertErr.message : String(upsertErr)
+                    if (/timed out/i.test(msg)) {
+                      logger.error(
+                        `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: final batch TIMED OUT after ${Date.now() - t0}ms: ${msg}`
+                      )
+                      callback(upsertErr instanceof Error ? upsertErr : new Error(msg))
+                      return
+                    }
+                    logger.error(
+                      `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: final batch skipped after error (${Date.now() - t0}ms): ${msg}`
+                    )
+                    recordsSkipped += remainingBatch.length
+                    callback()
+                  })
+              },
+            })
+
+            batchWriter.on('finish', () => {
+              logger.info(
+                `[IngestDrugDataJob] part ${partIndex + 1}/${totalParts}: stream finished — ` +
+                  `ingested=${recordsIngested} skipped=${recordsSkipped} batches=${batchNum}`
+              )
+              resolve({ recordsIngested, recordsSkipped })
+            })
+
+            batchWriter.on('error', reject)
+            readStream.on('error', reject)
+            jsonParser.on('error', reject)
+            pick.on('error', reject)
+            streamArray.on('error', reject)
+
+            // Pipeline: readStream → jsonParser → pick → streamArray → batchWriter
+            readStream.pipe(jsonParser).pipe(pick).pipe(streamArray).pipe(batchWriter)
+          })
+        })
+
+        zipFile.on('end', () => {
+          // All zip entries enumerated. The batchWriter 'finish' event resolves
+          // the promise once the last batch flushes.
+        })
+      })
+    })
+  }
+
+  private async writeFinalStatus(
+    exportDate: string,
+    resourceMeta?: DrugDatasetResourceMeta
+  ): Promise<void> {
+    // Lazy import to keep module top level free of Lucid
+    const KVStore = (await import('#models/kv_store')).default
+    await KVStore.setValue('drugReference.lastUpdatedExportDate', exportDate)
+
+    // Install-state write-back: when this ingest was kicked off by a curated-tier
+    // install, record an `installed_resources` row so the tier-status math (which
+    // is row-driven for ZIM/map) recognizes the dataset uniformly — the home-tile
+    // gate and the tier "installed" badge both read these rows. resource_type
+    // 'dataset' was widened onto the model in the foundational slice. The row's
+    // `version` is the openFDA export_date (the real freshness key), not the
+    // manifest placeholder. Manual downloads pass no resourceMeta → no row, which
+    // is correct: install-state belongs to the curated-tier path only.
+    if (!resourceMeta) return
+    try {
+      const { default: InstalledResource } = await import('#models/installed_resource')
+      const { DateTime } = await import('luxon')
+      const totalBytes = await this.totalDownloadedBytes()
+      await InstalledResource.updateOrCreate(
+        { resource_id: resourceMeta.resourceId, resource_type: 'dataset' },
+        {
+          version: exportDate,
+          collection_ref: resourceMeta.collectionRef,
+          url: 'https://api.fda.gov/download.json',
+          // No single on-disk file — the parts are deleted after ingest; the
+          // installed artifact is the DB table. Record the staging dir for
+          // provenance; uninstall keys off resource_id, never this path.
+          file_path: STORAGE_BASE,
+          file_size_bytes: totalBytes,
+          installed_at: DateTime.now(),
+        }
+      )
+      logger.info(
+        `[IngestDrugDataJob] Wrote installed_resources row for ${resourceMeta.resourceId} (export_date=${exportDate})`
+      )
+    } catch (err) {
+      // A failed row write must NOT abort a completed ingest — the data is
+      // already searchable. Log loud; the tier badge will simply read
+      // not-installed until the next install reconcile.
+      logger.error(
+        `[IngestDrugDataJob] Failed to write installed_resources row for ${resourceMeta.resourceId}: ${
+          err instanceof Error ? err.message : String(err)
+        }`
+      )
+    }
+  }
+
+  /**
+   * Sum the recorded part sizes from the download-state KV marker, for the
+   * `installed_resources.file_size_bytes` column. Best-effort: the parts are
+   * deleted right after a full ingest, so the marker (written before deletion) is
+   * the only durable size source. Returns null when the marker is absent/empty so
+   * the column stays NULL rather than reporting a wrong 0.
+   */
+  private async totalDownloadedBytes(): Promise<number | null> {
+    try {
+      const KVStore = (await import('#models/kv_store')).default
+      const { parseDownloadState } = await import('../../util/drug_labels.js')
+      const marker = parseDownloadState(await KVStore.getValue('drugReference.downloadState'))
+      if (!marker || marker.parts.length === 0) return null
+      const sum = marker.parts.reduce((acc, p) => acc + (p.bytes || 0), 0)
+      return sum > 0 ? sum : null
+    } catch {
+      return null
+    }
+  }
+
+  /**
+   * Delete the downloaded part zips and clear the download-state marker, run once
+   * after a full ingest succeeds (reclaims ~1.7 GB). A failed unlink is logged
+   * but never aborts a completed ingest.
+   */
+  private async deleteDownloadedParts(
+    manifest: DrugLabelManifest,
+    totalParts: number
+  ): Promise<void> {
+    for (let i = 0; i < totalParts; i++) {
+      const partition = manifest.partitions[i]
+      if (!partition) continue
+      const zipPath = partZipPath(STORAGE_BASE, partition)
+      try {
+        await fsPromises.unlink(zipPath)
+        logger.info(`[IngestDrugDataJob] Deleted zip: ${zipPath}`)
+      } catch (err) {
+        logger.warn(
+          `[IngestDrugDataJob] Could not delete zip ${zipPath}: ${err instanceof Error ? err.message : String(err)}`
+        )
+      }
+    }
+
+    const KVStore = (await import('#models/kv_store')).default
+    await KVStore.clearValue('drugReference.downloadState')
+  }
+}

admin/app/jobs/run_download_job.ts

@@ -1,12 +1,41 @@
 import { Job, UnrecoverableError } from 'bullmq'
 import { RunDownloadJobParams, DownloadProgressData } from '../../types/downloads.js'
 import { QueueService } from '#services/queue_service'
-import { doResumableDownload } from '../utils/downloads.js'
+import { doResumableDownload, GatedContentAuthError } from '../utils/downloads.js'
 import { createHash } from 'crypto'
 import { DockerService } from '#services/docker_service'
 import { ZimService } from '#services/zim_service'
 import { MapService } from '#services/map_service'
+import { RagService } from '#services/rag_service'
+import { OllamaService } from '#services/ollama_service'
 import { EmbedFileJob } from './embed_file_job.js'
+import { basename, join, resolve, sep } from 'node:path'
+import { ZIM_STORAGE_PATH } from '../utils/fs.js'
+
+/** Maps live under `<cwd>/storage/maps/pmtiles`; no shared constant exists. */
+const MAP_STORAGE_PATH = '/storage/maps'
+
+/**
+ * Guard for the outdated-file deletion in {@link RunDownloadJob} `onComplete`:
+ * returns true only when `oldFilePath` sits under the expected content storage
+ * root for its type AND its filename carries this resource's id prefix. This
+ * makes the delete explicit and bounded — we only ever remove the replaced
+ * resource's own previous file, never another file, even if the
+ * InstalledResource row is stale or malformed.
+ */
+function isSafeOldContentPath(
+  oldFilePath: string,
+  resourceId: string,
+  filetype: string
+): boolean {
+  const root =
+    filetype === 'zim'
+      ? join(process.cwd(), ZIM_STORAGE_PATH)
+      : join(process.cwd(), MAP_STORAGE_PATH)
+  const resolved = resolve(oldFilePath)
+  if (!resolved.startsWith(root + sep)) return false
+  return basename(resolved).startsWith(`${resourceId}_`)
+}
 
 export class RunDownloadJob {
   static get queue() {
@@ -34,11 +63,11 @@ export class RunDownloadJob {
     const queueService = QueueService.getInstance()
     const queue = queueService.getQueue(this.queue)
     const client = await queue.client
-    await client.set(this.cancelKey(jobId), '1', 'EX', 300) // 5 min TTL
+    await client.set(this.cancelKey(jobId), '1', { EX: 300 }) // 5 min TTL
   }
 
   async handle(job: Job) {
-    const { url, filepath, timeout, allowedMimeTypes, forceNew, filetype, resourceMetadata } =
+    const { url, filepath, timeout, allowedMimeTypes, forceNew, filetype, resourceMetadata, requestHeaders } =
       job.data as RunDownloadJobParams
 
     // Register abort controller for this job
@@ -81,6 +110,7 @@ export class RunDownloadJob {
         timeout,
         allowedMimeTypes,
         forceNew,
+        requestHeaders,
         signal: abortController.signal,
         onProgress(progress) {
           const progressPercent = (progress.downloadedBytes / (progress.totalBytes || 1)) * 100
@@ -98,6 +128,11 @@ export class RunDownloadJob {
           lastKnownProgress = { downloadedBytes: progress.downloadedBytes, totalBytes: progress.totalBytes }
         },
         async onComplete(url) {
+          // The previous file recorded for this resource (if any). Hoisted out of
+          // the metadata block below so the ZIM branch can decide whether this
+          // download is a content UPDATE (replacing a prior file) vs a fresh
+          // install, which changes how we reconcile the knowledge base.
+          let oldFilePath: string | null = null
           try {
             // Create InstalledResource entry if metadata was provided
             if (resourceMetadata) {
@@ -111,9 +146,9 @@ export class RunDownloadJob {
                 .where('resource_id', resourceMetadata.resource_id)
                 .where('resource_type', filetype as 'zim' | 'map')
                 .first()
-              const oldFilePath = oldEntry?.file_path ?? null
+              oldFilePath = oldEntry?.file_path ?? null
 
-              await InstalledResource.updateOrCreate(
+              const installed = await InstalledResource.updateOrCreate(
                 { resource_id: resourceMetadata.resource_id, resource_type: filetype as 'zim' | 'map' },
                 {
                   version: resourceMetadata.version,
@@ -125,15 +160,45 @@ export class RunDownloadJob {
                 }
               )
 
-              // Delete the old file if it differs from the new one
+              // A completed auto-update is the authoritative success signal for the
-              if (oldFilePath && oldFilePath !== filepath) {
+              // per-resource backoff — clear it here (NOT at dispatch time, which
+              // would reset the counter every window and defeat self-disable). The
+              // matching terminal-failure increment lives in the worker `failed`
+              // handler (commands/queue/work.ts). Manual downloads (auto !== true)
+              // never touch the counter.
+              if (resourceMetadata.auto === true) {
                 try {
-                  await deleteFileIfExists(oldFilePath)
+                  const { recordResourceUpdateSuccess } = await import(
-                  console.log(`[RunDownloadJob] Deleted old file: ${oldFilePath}`)
+                    '../utils/content_auto_update_backoff.js'
-                } catch (deleteError) {
+                  )
+                  await recordResourceUpdateSuccess(installed)
+                } catch (error) {
+                  console.error(
+                    `[RunDownloadJob] Error clearing auto-update backoff for ${resourceMetadata.resource_id}:`,
+                    error
+                  )
+                }
+              }
+
+              // Step 1: delete the OUTDATED file if it differs from the new one.
+              // Guarded by isSafeOldContentPath so we can ONLY ever delete the
+              // replaced resource's own previous file — never another resource's
+              // file, even if the InstalledResource row is stale/malformed.
+              if (oldFilePath && oldFilePath !== filepath) {
+                if (isSafeOldContentPath(oldFilePath, resourceMetadata.resource_id, filetype)) {
+                  try {
+                    await deleteFileIfExists(oldFilePath)
+                    console.log(`[RunDownloadJob] Deleted old file: ${oldFilePath}`)
+                  } catch (deleteError) {
+                    console.warn(
+                      `[RunDownloadJob] Failed to delete old file ${oldFilePath}:`,
+                      deleteError
+                    )
+                  }
+                } else {
                   console.warn(
-                    `[RunDownloadJob] Failed to delete old file ${oldFilePath}:`,
+                    `[RunDownloadJob] Refusing to delete unexpected old file path for ` +
-                    deleteError
+                      `${resourceMetadata.resource_id} (${filetype}): ${oldFilePath}`
                   )
                 }
               }
@@ -144,42 +209,78 @@ export class RunDownloadJob {
               const zimService = new ZimService(dockerService)
               await zimService.downloadRemoteSuccessCallback([url], true)
 
-              // Only dispatch embedding job if AI Assistant (Ollama) is installed
+              // Only touch the knowledge base if AI Assistant (Ollama) is installed.
+              // skip_embedding opts a ZIM out entirely — Creator Pack video ZIMs are
+              // media galleries, not text, so they must never be embedded or KB-reconciled.
               const ollamaUrl = await dockerService.getServiceURL('nomad_ollama')
-              if (ollamaUrl) {
+              if (ollamaUrl && !resourceMetadata?.skip_embedding) {
-                // Respect the global ingest policy. Under Manual, record the file
+                // A content UPDATE replaces a prior file at a DIFFERENT path
-                // as pending_decision so the KB panel surfaces the per-file Index
+                // (version is in the filename). A fresh install has no prior row;
-                // affordance (PR #909) instead of silently auto-embedding behind
+                // a same-version re-download keeps the same path. The two cases
-                // the user's back. Unset is treated as Always to preserve legacy
+                // reconcile the KB differently.
-                // behavior — mirrors rag_service.ts:1587-1588.
+                const isReplacement = !!oldFilePath && oldFilePath !== filepath
-                const { default: KVStore } = await import('#models/kv_store')
-                const { default: KbIngestState } = await import('#models/kb_ingest_state')
-                const policyRaw = await KVStore.getValue('rag.defaultIngestPolicy')
-                const policy: 'Always' | 'Manual' = policyRaw === 'Manual' ? 'Manual' : 'Always'
 
-                if (policy === 'Manual') {
+                if (isReplacement) {
+                  // CONTENT UPDATE: mirror the REPLACED file's prior indexed state
+                  // rather than the global Always/Manual policy. reconcileReplaced-
+                  // ContentFile removes the old file's points and re-queues the new
+                  // file IFF the old one was indexed and Qdrant is running; it is a
+                  // no-op otherwise (not installed / old not indexed / Qdrant down).
+                  // The user already chose whether this content is in the KB, so we
+                  // honor that choice in both directions. See the method for the
+                  // full 5-step contract.
                   try {
-                    // firstOrCreate so a re-download doesn't demote an existing
+                    const ragService = new RagService(dockerService, new OllamaService())
-                    // indexed/failed row — user keeps prior state and can re-index
+                    const outcome = await ragService.reconcileReplacedContentFile({
-                    // explicitly from the KB panel if they want fresh content.
+                      oldFilePath: oldFilePath!,
-                    await KbIngestState.firstOrCreate(
+                      newFilePath: filepath,
-                      { file_path: filepath },
+                      fileName: url.split('/').pop() || '',
-                      { file_path: filepath, state: 'pending_decision', chunks_embedded: 0 }
+                    })
+                    console.log(
+                      `[RunDownloadJob] KB reconciliation for replaced ${filepath}: ${outcome}`
                     )
                   } catch (error) {
                     console.error(
-                      `[RunDownloadJob] Error recording pending_decision state for ${filepath}:`,
+                      `[RunDownloadJob] Error reconciling knowledge base for replaced file ${filepath}:`,
                       error
                     )
                   }
                 } else {
-                  try {
+                  // FRESH INSTALL (or same-version re-download): respect the global
-                    await EmbedFileJob.dispatch({
+                  // ingest policy. Under Manual, record the file as pending_decision
-                      fileName: url.split('/').pop() || '',
+                  // so the KB panel surfaces the per-file Index affordance (PR #909)
-                      filePath: filepath,
+                  // instead of silently auto-embedding behind the user's back. Unset
-                    })
+                  // is treated as Always to preserve legacy behavior — mirrors
-                  } catch (error) {
+                  // rag_service.ts:1587-1588.
-                    console.error(`[RunDownloadJob] Error dispatching EmbedFileJob for URL ${url}:`, error)
+                  const { default: KVStore } = await import('#models/kv_store')
+                  const { default: KbIngestState } = await import('#models/kb_ingest_state')
+                  const policyRaw = await KVStore.getValue('rag.defaultIngestPolicy')
+                  const policy: 'Always' | 'Manual' = policyRaw === 'Manual' ? 'Manual' : 'Always'
+
+                  if (policy === 'Manual') {
+                    try {
+                      // firstOrCreate so a re-download doesn't demote an existing
+                      // indexed/failed row — user keeps prior state and can re-index
+                      // explicitly from the KB panel if they want fresh content.
+                      await KbIngestState.firstOrCreate(
+                        { file_path: filepath },
+                        { file_path: filepath, state: 'pending_decision', chunks_embedded: 0 }
+                      )
+                    } catch (error) {
+                      console.error(
+                        `[RunDownloadJob] Error recording pending_decision state for ${filepath}:`,
+                        error
+                      )
+                    }
+                  } else {
+                    try {
+                      await EmbedFileJob.dispatch({
+                        fileName: url.split('/').pop() || '',
+                        filePath: filepath,
+                      })
+                    } catch (error) {
+                      console.error(`[RunDownloadJob] Error dispatching EmbedFileJob for URL ${url}:`, error)
+                    }
                   }
                 }
               }
@@ -215,6 +316,13 @@ export class RunDownloadJob {
       if (userCancelled || abortController.signal.reason === 'user-cancel') {
         throw new UnrecoverableError(`Download cancelled: ${error.message}`)
       }
+      // A rejected entitlement is permanent - this build either has the key or it
+      // doesn't. Left as a plain Error it consumes all 10 attempts with exponential
+      // backoff from 30s, so the job reads `delayed` for ~4h15m and the user sees a
+      // stuck download instead of the message above.
+      if (error instanceof GatedContentAuthError) {
+        throw new UnrecoverableError(error.message)
+      }
       throw error
     } finally {
       if (cancelPollInterval !== null) {

admin/app/jobs/run_extract_pmtiles_job.ts

@@ -52,7 +52,7 @@ export class RunExtractPmtilesJob {
     const queueService = QueueService.getInstance()
     const queue = queueService.getQueue(this.queue)
     const client = await queue.client
-    await client.set(this.cancelKey(jobId), '1', 'EX', 300)
+    await client.set(this.cancelKey(jobId), '1', { EX: 300 })
   }
 
   /** Awaits job.updateProgress and swallows BullMQ stale-job errors (code -1),

admin/app/models/benchmark_result.ts

@@ -56,6 +56,79 @@ export default class BenchmarkResult extends BaseModel {
   @column()
   declare ai_time_to_first_token: number | null
 
+  // Harness forensic metadata (nullable — added in Score v2 Phase 1)
+  @column()
+  declare sysbench_digest: string | null
+
+  @column()
+  declare ollama_version: string | null
+
+  // Platform metadata (nullable). Sourced from the Docker daemon, not
+  // systeminformation: inside the admin container si.osInfo()/os.arch()
+  // describe the container, not the host.
+  @column()
+  declare cpu_architecture: string | null
+
+  @column()
+  declare os_name: string | null
+
+  @column()
+  declare os_version: string | null
+
+  // NOMAD Score v2 raw channels (nullable — added in Score v2 Phase 4). Populated
+  // on full benchmarks under benchmark_version >= 2.0.0; the leaderboard recomputes
+  // the score from these on submit. cpu_events_multi is measured at
+  // cpu_benchmark_threads; memory_ops_per_sec at memory_threads. Disk figures are
+  // O_DIRECT MB/s. cpu_total_events/cpu_total_time are the W6 consistency companions.
+  @column()
+  declare cpu_events_single: number | null
+
+  @column()
+  declare cpu_events_multi: number | null
+
+  @column()
+  declare cpu_benchmark_threads: number | null
+
+  @column()
+  declare cpu_total_events: number | null
+
+  @column()
+  declare cpu_total_time: number | null
+
+  @column()
+  declare memory_ops_per_sec: number | null
+
+  @column()
+  declare memory_threads: number | null
+
+  @column()
+  declare disk_read_mb_per_sec: number | null
+
+  @column()
+  declare disk_write_mb_per_sec: number | null
+
+  // Uncapped NOMAD Score v2 (null for pre-v2 rows and system-only runs). The
+  // legacy nomad_score below is retained in parallel for display continuity.
+  // columnName + serializeAs pinned: the snake_case strategy would otherwise map
+  // this property to `nomad_score_v_2` (splitting the digit) for both the DB column
+  // (which the migration doesn't create) and the JSON key (which the frontend reads
+  // as nomad_score_v2). Pin both so DB, API, and UI all agree.
+  @column({ columnName: 'nomad_score_v2', serializeAs: 'nomad_score_v2' })
+  declare nomad_score_v2: number | null
+
+  // Best-effort run environment metadata (issue #1016)
+  @column()
+  declare run_environment: string | null
+
+  @column()
+  declare storage_path_type: string | null
+
+  @column({
+    // Nullable tri-state; coerce the SQLite 0/1 int to a real boolean when present.
+    consume: (value: number | null) => (value === null || value === undefined ? null : Boolean(value)),
+  })
+  declare gpu_compute_detected: boolean | null
+
   // Composite NOMAD score (0-100)
   @column()
   declare nomad_score: number

admin/app/models/drug_label.ts

@@ -0,0 +1,116 @@
+import { DateTime } from 'luxon'
+import { BaseModel, column, SnakeCaseNamingStrategy } from '@adonisjs/lucid/orm'
+
+/**
+ * Drug Reference v1 — openFDA drug label catalog entry.
+ *
+ * One row per FDA `set_id` (stable GUID for a labeling, across all revisions).
+ * Re-ingesting updates existing rows idempotently via `set_id` UNIQUE key.
+ *
+ * Enum-ish columns (product_type) are plain varchars validated at the edge —
+ * the stl_files / inventory_items convention: no native DB enums, so the
+ * schema can evolve without ALTER TABLE.
+ *
+ * All section-text columns are optional: label records frequently omit
+ * sections (e.g. OTC labels have no `boxed_warning`). The mapper returns
+ * null for any absent or empty section.
+ */
+export default class DrugLabel extends BaseModel {
+  static table = 'drug_labels'
+  static namingStrategy = new SnakeCaseNamingStrategy()
+
+  @column({ isPrimary: true })
+  declare id: number
+
+  /** openFDA set_id — stable GUID across label revisions. Idempotent upsert key. */
+  @column()
+  declare set_id: string
+
+  /** openFDA id — per-revision GUID for provenance. */
+  @column()
+  declare spl_id: string | null
+
+  @column()
+  declare version: string | null
+
+  /** openfda.brand_name[0] — first element only. */
+  @column()
+  declare brand_name: string | null
+
+  /** openfda.generic_name joined with ", " (labels can list several). */
+  @column()
+  declare generic_name: string | null
+
+  /** openfda.manufacturer_name[0]. */
+  @column()
+  declare manufacturer: string | null
+
+  /** openfda.product_ndc joined with ", ". */
+  @column()
+  declare product_ndc: string | null
+
+  /** openfda.route joined with ", ". */
+  @column()
+  declare route: string | null
+
+  /**
+   * openfda.product_type[0]. Drives OTC vs Rx badge + filter.
+   * Expected values: 'HUMAN OTC DRUG' | 'HUMAN PRESCRIPTION DRUG'.
+   */
+  @column()
+  declare product_type: string | null
+
+  /**
+   * Normalized brand+generic blob for FULLTEXT and LIKE search.
+   * Built by normalizeDrugName() at ingest time; never re-computed on read.
+   * Max 768 chars — stays inside InnoDB utf8mb4 index key-length budget.
+   */
+  @column()
+  declare searchable_name: string | null
+
+  /** Flattened indications_and_usage sections, joined with \n\n. */
+  @column()
+  declare indications: string | null
+
+  /** Flattened dosage_and_administration sections. */
+  @column()
+  declare dosage: string | null
+
+  /** Flattened warnings sections. */
+  @column()
+  declare warnings: string | null
+
+  /** Flattened boxed_warning sections (absent on most OTC labels). */
+  @column()
+  declare boxed_warning: string | null
+
+  /**
+   * Flattened drug_interactions label text.
+   * Single-drug label info only — NOT a pairwise cross-drug checker.
+   */
+  @column()
+  declare drug_interactions: string | null
+
+  /** Flattened contraindications sections. */
+  @column()
+  declare contraindications: string | null
+
+  /** Flattened when_using sections (common on OTC labels). */
+  @column()
+  declare when_using: string | null
+
+  /** Flattened stop_use sections (common on OTC labels). */
+  @column()
+  declare stop_use: string | null
+
+  /**
+   * Parsed from effective_time (YYYYMMDD → YYYY-MM-DD).
+   * Stored as a plain date string, not a Luxon DateTime, because
+   * the source format is just a date (no time component).
+   */
+  @column()
+  declare source_updated_at: string | null
+
+  @column.dateTime({ autoCreate: true, autoUpdate: true, columnName: 'ingested_at' })
+  declare ingested_at: DateTime
+}

admin/app/models/installed_resource.ts

@@ -11,7 +11,7 @@ export default class InstalledResource extends BaseModel {
   declare resource_id: string
 
   @column()
-  declare resource_type: 'zim' | 'map'
+  declare resource_type: 'zim' | 'map' | 'dataset'
 
   @column()
   declare collection_ref: string | null
@@ -30,4 +30,25 @@ export default class InstalledResource extends BaseModel {
 
   @column.dateTime()
   declare installed_at: DateTime
+
+  // ── Content auto-update state (global opt-in; gated by `contentAutoUpdate.enabled`) ──
+
+  /** Newest catalog version (YYYY-MM) detected, or null when already current. */
+  @column()
+  declare available_update_version: string | null
+
+  /** Size (bytes) of the available update, captured from the catalog. */
+  @column()
+  declare available_update_size_bytes: number | null
+
+  /** Cool-off anchor: when the current available update was first detected. */
+  @column.dateTime()
+  declare available_update_first_seen_at: DateTime | null
+
+  /** Per-resource failure backoff so one flapping download self-disables. */
+  @column()
+  declare auto_update_consecutive_failures: number
+
+  @column()
+  declare auto_update_disabled_reason: string | null
 }

admin/app/models/kb_ingest_state.ts

@@ -28,6 +28,9 @@ export default class KbIngestState extends BaseModel {
   @column()
   declare chunks_embedded: number
 
+  @column()
+  declare collection: string | null
+
   @column()
   declare last_error: string | null
 
@@ -37,18 +40,19 @@ export default class KbIngestState extends BaseModel {
   @column.dateTime({ autoCreate: true, autoUpdate: true })
   declare updated_at: DateTime
 
-  static async getOrCreate(filePath: string): Promise<KbIngestState> {
+  static async getOrCreate(filePath: string, collection?: string): Promise<KbIngestState> {
     return this.firstOrCreate(
       { file_path: filePath },
-      { file_path: filePath, state: 'pending_decision', chunks_embedded: 0 }
+      { file_path: filePath, state: 'pending_decision', chunks_embedded: 0, collection: collection ?? null }
     )
   }
 
-  static async markIndexed(filePath: string, chunksEmbedded: number): Promise<void> {
+  static async markIndexed(filePath: string, chunksEmbedded: number, collection?: string): Promise<void> {
-    const row = await this.getOrCreate(filePath)
+    const row = await this.getOrCreate(filePath, collection)
     row.state = 'indexed'
     row.chunks_embedded = chunksEmbedded
     row.last_error = null
+    if (collection) row.collection = collection
     await row.save()
   }
 

admin/app/models/kb_ratio_registry.ts

@@ -49,10 +49,22 @@ export default class KbRatioRegistry extends BaseModel {
     return findChunksPerMb(filename, rows)
   }
 
-  /** Estimate total chunks for a file of the given size on disk. */
+  /**
-  static async estimateChunks(filename: string, fileSizeBytes: number): Promise<number | null> {
+   * Estimate total chunks for a file of the given size on disk.
+   *
+   * `ignoreCatchAll` excludes the empty-pattern fallback, returning `null` for
+   * filenames that only the catch-all would match. The partial_stall warning
+   * uses this so it never flags ZIMs the registry can't specifically
+   * characterize (e.g. PDF/link-out-heavy archives whose byte size wildly
+   * over-predicts embeddable chunks). See #913.
+   */
+  static async estimateChunks(
+    filename: string,
+    fileSizeBytes: number,
+    opts: { ignoreCatchAll?: boolean } = {}
+  ): Promise<number | null> {
     const rows = await this.all()
-    return estimateChunkCount(filename, fileSizeBytes, rows)
+    return estimateChunkCount(filename, fileSizeBytes, rows, opts)
   }
 
   /**

admin/app/models/service.ts

@@ -59,9 +59,42 @@ export default class Service extends BaseModel {
   @column()
   declare ui_location: string | null
 
+  // User-set override for the launch ("Open") link (e.g. a reverse-proxy/local-DNS host like
+  // https://jellyfin.myhomelab.net). When null, the default host + port link derived from
+  // ui_location is used. Only affects user-facing links — never internal service-to-service URLs.
+  @column()
+  declare custom_url: string | null
+
   @column()
   declare metadata: string | null
 
+  @column({
+    serialize(value) {
+      return Boolean(value)
+    },
+  })
+  declare is_custom: boolean
+
+  @column({
+    serialize(value) {
+      return Boolean(value)
+    },
+  })
+  declare is_user_modified: boolean
+
+  @column()
+  declare category: string | null
+
+  // When true the service is sunset: hidden from the install catalog unless it is already
+  // installed (see SystemService.getServices). Lets a deprecated app stay manageable for users who
+  // still run it while keeping new users from installing it.
+  @column({
+    serialize(value) {
+      return Boolean(value)
+    },
+  })
+  declare is_deprecated: boolean
+
   @column()
   declare source_repo: string | null
 
@@ -71,6 +104,28 @@ export default class Service extends BaseModel {
   @column.dateTime()
   declare update_checked_at: DateTime | null
 
+  // Per-app opt-in for automatic updates. An app auto-updates only when both this
+  // and the global `appAutoUpdate.enabled` master switch are on.
+  @column({
+    serialize(value) {
+      return Boolean(value)
+    },
+  })
+  declare auto_update_enabled: boolean
+
+  // When the current `available_update_version` was first detected — the anchor for
+  // the auto-update cool-off (registry tags carry no publish timestamp).
+  @column.dateTime()
+  declare available_update_first_seen_at: DateTime | null
+
+  // Per-app auto-update failure backoff; at the threshold the app self-disables via
+  // `auto_update_disabled_reason` without affecting other apps.
+  @column()
+  declare auto_update_consecutive_failures: number
+
+  @column()
+  declare auto_update_disabled_reason: string | null
+
   @column.dateTime({ autoCreate: true })
   declare created_at: DateTime
 

admin/app/services/app_auto_update_service.ts

@@ -0,0 +1,398 @@
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import { DateTime } from 'luxon'
+import KVStore from '#models/kv_store'
+import Service from '#models/service'
+import { DockerService } from '#services/docker_service'
+import { DownloadService } from '#services/download_service'
+import { SystemService } from '#services/system_service'
+import { ContainerRegistryService } from '#services/container_registry_service'
+import { isNewerVersion, parseMajorVersion } from '../utils/version.js'
+import { isWithinWindow } from '../utils/update_window.js'
+import {
+  checkImageDiskSpace,
+  type Blocker,
+  type PreflightResult,
+} from '../utils/image_disk_preflight.js'
+
+/**
+ * Defaults shared with the core auto-update. App auto-updates intentionally reuse
+ * the SAME window/cool-off settings (`autoUpdate.windowStart/windowEnd/cooloffHours`);
+ * only the enable flag (`appAutoUpdate.enabled`) is separate.
+ */
+const DEFAULT_WINDOW_START = '02:00'
+const DEFAULT_WINDOW_END = '05:00'
+const DEFAULT_COOLOFF_HOURS = 72
+
+/** Per-app genuine failures before that app self-disables (others keep running). */
+const MAX_CONSECUTIVE_FAILURES = 3
+
+export interface AppAutoUpdateConfig {
+  /** Global master switch (`appAutoUpdate.enabled`). */
+  enabled: boolean
+  windowStart: string
+  windowEnd: string
+  cooloffHours: number
+}
+
+/** An installed app that should be auto-updated this run. */
+export interface AppUpdateTarget {
+  service: Service
+  /** Exact registry tag to update to (the value in `available_update_version`). */
+  targetVersion: string
+}
+
+/** Per-app eligibility verdict (drives both selection and the status UI). */
+export interface AppEligibility {
+  eligible: boolean
+  reason: string
+  cooloffRemainingHours: number | null
+}
+
+export interface AppAutoUpdateAppStatus {
+  service_name: string
+  friendly_name: string | null
+  auto_update_enabled: boolean
+  current_version: string
+  available_update_version: string | null
+  first_seen_at: string | null
+  eligible: boolean
+  reason: string
+  cooloff_remaining_hours: number | null
+  consecutive_failures: number
+  auto_disabled_reason: string | null
+}
+
+export interface AppAutoUpdateStatus extends AppAutoUpdateConfig {
+  withinWindow: boolean
+  lastAttemptAt: string | null
+  lastResult: string | null
+  apps: AppAutoUpdateAppStatus[]
+}
+
+/**
+ * Decision + safety layer for automatic updates of installed sibling apps (the
+ * containers NOMAD deploys via the Docker socket and manages in Supply Depot).
+ *
+ * This is the app-side counterpart to {@link AutoUpdateService} and intentionally
+ * reuses its generic window/disk pre-flight helpers. Unlike the core update, an
+ * app update needs no sidecar — the admin container recreates its siblings directly
+ * via {@link DockerService.updateContainer} (in-process pull → rename → health-check
+ * → rollback). Auto-update only decides *whether* each opted-in app should update now
+ * (master switch on + per-app toggle on + in window + an eligible minor/patch past
+ * its cool-off + pre-flight passes) and then drives the existing update path.
+ *
+ * Minor/patch-only is already guaranteed upstream by
+ * {@link ContainerRegistryService.getAvailableUpdates} (same-major filter); the
+ * major-version check here is defense-in-depth.
+ */
+@inject()
+export class AppAutoUpdateService {
+  constructor(
+    private dockerService: DockerService,
+    private downloadService: DownloadService,
+    private systemService: SystemService,
+    private containerRegistryService: ContainerRegistryService
+  ) {}
+
+  /** Read the global master switch plus the shared window/cool-off settings. */
+  async getConfig(): Promise<AppAutoUpdateConfig> {
+    const [enabled, windowStart, windowEnd, cooloffHours] = await Promise.all([
+      KVStore.getValue('appAutoUpdate.enabled'),
+      KVStore.getValue('autoUpdate.windowStart'),
+      KVStore.getValue('autoUpdate.windowEnd'),
+      KVStore.getValue('autoUpdate.cooloffHours'),
+    ])
+
+    const parsedCooloff = Number(cooloffHours)
+    return {
+      enabled: enabled ?? false,
+      windowStart: windowStart || DEFAULT_WINDOW_START,
+      windowEnd: windowEnd || DEFAULT_WINDOW_END,
+      // `Number(null) === 0`, so an unset value must fall through to the default
+      // rather than silently resolving to a zero cool-off. An explicit 0 is honored.
+      cooloffHours:
+        cooloffHours !== null && Number.isFinite(parsedCooloff) && parsedCooloff >= 0
+          ? parsedCooloff
+          : DEFAULT_COOLOFF_HOURS,
+    }
+  }
+
+  /**
+   * Pure per-app eligibility verdict. An app is eligible when it has a detected
+   * update that is the same major (defense-in-depth), strictly newer, not self-
+   * disabled, and past its cool-off (measured from first-detected).
+   */
+  appEligibility(service: Service, cooloffHours: number, now: DateTime): AppEligibility {
+    if (!service.available_update_version) {
+      return { eligible: false, reason: 'Up to date', cooloffRemainingHours: null }
+    }
+    if (service.auto_update_disabled_reason) {
+      return {
+        eligible: false,
+        reason: 'Auto-update disabled after repeated failures',
+        cooloffRemainingHours: null,
+      }
+    }
+
+    const currentTag = this.containerRegistryService.parseImageReference(
+      service.container_image
+    ).tag
+    if (currentTag === 'latest') {
+      return {
+        eligible: false,
+        reason: 'Pinned to :latest — cannot version-check',
+        cooloffRemainingHours: null,
+      }
+    }
+    if (parseMajorVersion(service.available_update_version) !== parseMajorVersion(currentTag)) {
+      return {
+        eligible: false,
+        reason: 'Major version — manual update required',
+        cooloffRemainingHours: null,
+      }
+    }
+    if (!isNewerVersion(service.available_update_version, currentTag)) {
+      return { eligible: false, reason: 'Up to date', cooloffRemainingHours: null }
+    }
+    if (!service.available_update_first_seen_at) {
+      return { eligible: false, reason: 'Cool-off pending', cooloffRemainingHours: cooloffHours }
+    }
+
+    const ageHours = now.diff(service.available_update_first_seen_at, 'hours').hours
+    const remaining = cooloffHours - ageHours
+    if (remaining > 0) {
+      const rounded = Math.ceil(remaining)
+      return {
+        eligible: false,
+        reason: `In cool-off (${rounded}h remaining)`,
+        cooloffRemainingHours: rounded,
+      }
+    }
+
+    return {
+      eligible: true,
+      reason: `Eligible → ${service.available_update_version}`,
+      cooloffRemainingHours: 0,
+    }
+  }
+
+  /** Installed, opted-in apps that are eligible to update right now. */
+  async getEligibleApps(config: AppAutoUpdateConfig, now: DateTime): Promise<AppUpdateTarget[]> {
+    const apps = await Service.query().where('installed', true).where('auto_update_enabled', true)
+    const targets: AppUpdateTarget[] = []
+    for (const service of apps) {
+      const verdict = this.appEligibility(service, config.cooloffHours, now)
+      if (verdict.eligible) {
+        targets.push({ service, targetVersion: service.available_update_version! })
+      }
+    }
+    return targets
+  }
+
+  /**
+   * Run-wide pre-flight checked once per attempt (independent of any single app):
+   * never auto-update while content/model downloads are running. Transient → `skip`.
+   */
+  async runGlobalPreflight(): Promise<PreflightResult> {
+    const blockers: Blocker[] = []
+    try {
+      const downloads = await this.downloadService.listDownloadJobs()
+      const active = downloads.filter(
+        (d) => !!d.status && ['waiting', 'active', 'delayed'].includes(d.status)
+      )
+      if (active.length > 0) {
+        blockers.push({ reason: `${active.length} download(s) in progress`, severity: 'skip' })
+      }
+    } catch (error) {
+      logger.warn(`[AppAutoUpdateService] Could not check active downloads: ${error.message}`)
+    }
+    return { ok: blockers.length === 0, blockers }
+  }
+
+  /** Per-app pre-flight: not already mid-operation (`skip`) and enough disk (`failure`). */
+  async runAppPreflight(target: AppUpdateTarget): Promise<PreflightResult> {
+    const blockers: Blocker[] = []
+    const service = target.service
+
+    if (service.installation_status !== 'idle') {
+      blockers.push({
+        reason: `App has an operation in progress (status: ${service.installation_status})`,
+        severity: 'skip',
+      })
+    }
+
+    const hostArch = await this.getHostArch()
+    const targetImage = `${this.imageBase(service.container_image)}:${target.targetVersion}`
+    const diskBlocker = await checkImageDiskSpace({
+      image: targetImage,
+      hostArch,
+      containerRegistryService: this.containerRegistryService,
+      systemService: this.systemService,
+    })
+    if (diskBlocker) blockers.push(diskBlocker)
+
+    return { ok: blockers.length === 0, blockers }
+  }
+
+  /**
+   * Entry point invoked by AppAutoUpdateJob. Gates on the master switch + window,
+   * then runs each eligible app through pre-flight and {@link DockerService.updateContainer}.
+   * A failing app self-disables after repeated failures without affecting the others.
+   */
+  async attempt(): Promise<{ updated: number; reason: string }> {
+    const config = await this.getConfig()
+    const now = DateTime.now()
+
+    if (!config.enabled) {
+      return { updated: 0, reason: 'App auto-update is disabled' }
+    }
+    if (!isWithinWindow(config.windowStart, config.windowEnd, now)) {
+      const reason = `Outside update window (${config.windowStart}-${config.windowEnd})`
+      await this.recordRun(reason)
+      return { updated: 0, reason }
+    }
+
+    const eligible = await this.getEligibleApps(config, now)
+    if (eligible.length === 0) {
+      const reason = 'No eligible app updates (all current, in cool-off, or major-only)'
+      await this.recordRun(reason)
+      return { updated: 0, reason }
+    }
+
+    const global = await this.runGlobalPreflight()
+    if (!global.ok) {
+      const reason = `Pre-flight blocked: ${global.blockers.map((b) => b.reason).join('; ')}`
+      await this.recordRun(reason)
+      return { updated: 0, reason }
+    }
+
+    let updated = 0
+    let failed = 0
+    let skipped = 0
+
+    for (const target of eligible) {
+      const name = target.service.service_name
+      const preflight = await this.runAppPreflight(target)
+      if (!preflight.ok) {
+        const summary = preflight.blockers.map((b) => b.reason).join('; ')
+        if (preflight.blockers.some((b) => b.severity === 'failure')) {
+          await this.recordAppFailure(target.service, summary)
+          failed++
+        } else {
+          logger.info(`[AppAutoUpdateService] Skipped ${name}: ${summary}`)
+          skipped++
+        }
+        continue
+      }
+
+      logger.info(`[AppAutoUpdateService] Updating ${name} → ${target.targetVersion}`)
+      const result = await this.dockerService.updateContainer(name, target.targetVersion)
+      if (result.success) {
+        await this.recordAppSuccess(target.service)
+        updated++
+      } else {
+        await this.recordAppFailure(target.service, result.message)
+        failed++
+      }
+    }
+
+    const reason = `${updated} updated, ${failed} failed, ${skipped} skipped`
+    await this.recordRun(reason)
+    logger.info(`[AppAutoUpdateService] Run complete: ${reason}`)
+    return { updated, reason }
+  }
+
+  /** Clear an app's failure backoff after a successful auto-update. */
+  private async recordAppSuccess(service: Service): Promise<void> {
+    // updateContainer already advanced container_image and cleared
+    // available_update_version on its own (fresh) row; here we only touch the
+    // backoff fields, so Lucid persists just those dirty columns.
+    service.auto_update_consecutive_failures = 0
+    service.auto_update_disabled_reason = null
+    await service.save()
+  }
+
+  /** Record an app failure and self-disable it once the threshold is reached. */
+  private async recordAppFailure(service: Service, reason: string): Promise<void> {
+    const failures = (service.auto_update_consecutive_failures || 0) + 1
+    service.auto_update_consecutive_failures = failures
+    if (failures >= MAX_CONSECUTIVE_FAILURES) {
+      service.auto_update_disabled_reason = `Auto-update disabled after ${failures} consecutive failures. Last error: ${reason}`
+      logger.error(
+        `[AppAutoUpdateService] ${service.service_name} auto-disabled after ${failures} failures`
+      )
+    }
+    await service.save()
+    logger.error(
+      `[AppAutoUpdateService] ${service.service_name} failure ${failures}/${MAX_CONSECUTIVE_FAILURES}: ${reason}`
+    )
+  }
+
+  /** Record the global last-attempt summary for the settings UI. */
+  private async recordRun(reason: string): Promise<void> {
+    await KVStore.setValue('appAutoUpdate.lastAttemptAt', DateTime.now().toISO()!)
+    await KVStore.setValue('appAutoUpdate.lastResult', reason)
+  }
+
+  /** Full state snapshot for the settings UI (opted-in apps + their eligibility). */
+  async getStatus(): Promise<AppAutoUpdateStatus> {
+    const config = await this.getConfig()
+    const now = DateTime.now()
+
+    const apps = await Service.query().where('installed', true).where('auto_update_enabled', true)
+    const appStatuses: AppAutoUpdateAppStatus[] = apps.map((service) => {
+      const verdict = this.appEligibility(service, config.cooloffHours, now)
+      return {
+        service_name: service.service_name,
+        friendly_name: service.friendly_name,
+        auto_update_enabled: service.auto_update_enabled,
+        current_version: this.containerRegistryService.parseImageReference(service.container_image)
+          .tag,
+        available_update_version: service.available_update_version,
+        first_seen_at: service.available_update_first_seen_at?.toISO() ?? null,
+        eligible: verdict.eligible,
+        reason: verdict.reason,
+        cooloff_remaining_hours: verdict.cooloffRemainingHours,
+        consecutive_failures: service.auto_update_consecutive_failures || 0,
+        auto_disabled_reason: service.auto_update_disabled_reason,
+      }
+    })
+
+    const [lastAttemptAt, lastResult] = await Promise.all([
+      KVStore.getValue('appAutoUpdate.lastAttemptAt'),
+      KVStore.getValue('appAutoUpdate.lastResult'),
+    ])
+
+    return {
+      ...config,
+      withinWindow: isWithinWindow(config.windowStart, config.windowEnd, now),
+      lastAttemptAt: lastAttemptAt || null,
+      lastResult: lastResult || null,
+      apps: appStatuses,
+    }
+  }
+
+  /** Strip the tag from an image reference, leaving "registry/namespace/repo". */
+  private imageBase(image: string): string {
+    return image.includes(':') ? image.substring(0, image.lastIndexOf(':')) : image
+  }
+
+  /** Map the Docker daemon's architecture string to OCI naming (amd64/arm64/...). */
+  private async getHostArch(): Promise<string> {
+    try {
+      const info = await this.dockerService.docker.info()
+      const arch = info.Architecture || ''
+      const archMap: Record<string, string> = {
+        x86_64: 'amd64',
+        aarch64: 'arm64',
+        armv7l: 'arm',
+        amd64: 'amd64',
+        arm64: 'arm64',
+      }
+      return archMap[arch] || arch.toLowerCase()
+    } catch {
+      return 'amd64'
+    }
+  }
+}

admin/app/services/auto_update_service.ts

@@ -0,0 +1,580 @@
+import { inject } from '@adonisjs/core'
+import logger from '@adonisjs/core/services/logger'
+import axios from 'axios'
+import { DateTime } from 'luxon'
+import KVStore from '#models/kv_store'
+import Service from '#models/service'
+import { DockerService } from '#services/docker_service'
+import { DownloadService } from '#services/download_service'
+import { SystemService } from '#services/system_service'
+import { SystemUpdateService } from '#services/system_update_service'
+import { ContainerRegistryService } from '#services/container_registry_service'
+import { isNewerVersion, parseMajorVersion } from '../utils/version.js'
+import { isWithinWindow as isWithinWindowUtil } from '../utils/update_window.js'
+import {
+  checkImageDiskSpace,
+  type Blocker,
+  type PreflightResult,
+} from '../utils/image_disk_preflight.js'
+
+/** Docker image repository for the NOMAD admin/core image (tag applied per-release). */
+const NOMAD_IMAGE_REPO = 'ghcr.io/crosstalk-solutions/project-nomad'
+const RELEASES_URL = 'https://api.github.com/repos/Crosstalk-Solutions/project-nomad/releases'
+
+/** Defaults for user-configurable settings (server-local time window + cool-off). */
+const DEFAULT_WINDOW_START = '02:00'
+const DEFAULT_WINDOW_END = '05:00'
+const DEFAULT_COOLOFF_HOURS = 72
+
+/** Genuine failures before auto-update disables itself to avoid an update loop. */
+const MAX_CONSECUTIVE_FAILURES = 3
+
+/**
+ * Only tags matching strict semver are eligible. Defense-in-depth: the selected
+ * tag becomes `target_tag`, which the sidecar interpolates into a host-side `sed`
+ * (install/sidecar-updater/update-watcher.sh) — so a malformed tag must never be
+ * able to reach it, even though releases come from a trusted repo.
+ */
+const SEMVER_TAG = /^\d+\.\d+\.\d+$/
+
+/** Cache the GitHub releases feed in-process to avoid hammering the API. */
+const RELEASES_CACHE_TTL_MS = 15 * 60 * 1000
+/** Briefly remember a failed fetch so repeated calls don't each block on the timeout. */
+const RELEASES_FAILURE_TTL_MS = 60 * 1000
+
+export interface AutoUpdateConfig {
+  enabled: boolean
+  windowStart: string
+  windowEnd: string
+  cooloffHours: number
+}
+
+export interface EligibleTarget {
+  version: string
+  tag: string
+  publishedAt: string
+}
+
+// Pre-flight types/primitives are shared with AppAutoUpdateService; re-exported
+// here for back-compat with existing imports of this module.
+export type { Blocker, BlockerSeverity, PreflightResult } from '../utils/image_disk_preflight.js'
+
+/** Minimal shape of a GitHub release entry we depend on. */
+export interface GithubRelease {
+  tag_name?: string
+  published_at?: string
+  draft?: boolean
+  prerelease?: boolean
+}
+
+/**
+ * Inputs that can be injected to exercise the decision pipeline deterministically
+ * (used by the dry-run command/tests). All are optional; when omitted the real
+ * settings/clock/GitHub feed/pre-flight are used, exactly as production runs.
+ */
+export interface EvaluateOverrides {
+  currentVersion?: string
+  releases?: GithubRelease[]
+  now?: DateTime
+  forceEnabled?: boolean
+  windowStart?: string
+  windowEnd?: string
+  cooloffHours?: number
+  /** Treat pre-flight as passing without touching Docker/disk/queues. */
+  skipPreflight?: boolean
+  /** Substitute a canned pre-flight result. */
+  fakePreflight?: PreflightResult
+}
+
+export type DecisionOutcome =
+  | 'disabled'
+  | 'outside-window'
+  | 'eligibility-error'
+  | 'no-eligible'
+  | 'blocked'
+  | 'ready'
+
+/** Side-effect-free verdict of the decision pipeline. */
+export interface AutoUpdateDecision {
+  enabled: boolean
+  currentVersion: string
+  config: AutoUpdateConfig
+  withinWindow: boolean
+  eligibleTarget: EligibleTarget | null
+  preflight: PreflightResult | null
+  outcome: DecisionOutcome
+  reason: string
+}
+
+export interface AutoUpdateStatus extends AutoUpdateConfig {
+  currentVersion: string
+  withinWindow: boolean
+  eligibleTarget: EligibleTarget | null
+  lastAttemptAt: string | null
+  lastResult: string | null
+  lastError: string | null
+  consecutiveFailures: number
+  autoDisabledReason: string | null
+}
+
+/**
+ * Decision + safety layer for automatic updates of the NOMAD application itself.
+ *
+ * It does NOT recreate containers — that remains the sidecar's job. This service
+ * decides *whether* an update should run right now (opt-in, in-window, an eligible
+ * minor/patch release exists past its cool-off, pre-flight checks pass) and, if so,
+ * drives the existing {@link SystemUpdateService.requestUpdate} with an explicit,
+ * eligibility-vetted image tag.
+ *
+ * The window/pre-flight helpers are intentionally generic so a future PR can reuse
+ * them to auto-update installed apps (driving DockerService.updateContainer instead).
+ */
+@inject()
+export class AutoUpdateService {
+  constructor(
+    private dockerService: DockerService,
+    private downloadService: DownloadService,
+    private systemService: SystemService,
+    private systemUpdateService: SystemUpdateService,
+    private containerRegistryService: ContainerRegistryService
+  ) {}
+
+  /** In-process cache of the last successful releases fetch (per-process). */
+  private static releasesCache: { releases: GithubRelease[]; at: number } | null = null
+  /** Timestamp of the last failed fetch, for short-lived negative caching. */
+  private static releasesFailureAt = 0
+
+  /** Read user-configurable settings, applying defaults. */
+  async getConfig(): Promise<AutoUpdateConfig> {
+    const [enabled, windowStart, windowEnd, cooloffHours] = await Promise.all([
+      KVStore.getValue('autoUpdate.enabled'),
+      KVStore.getValue('autoUpdate.windowStart'),
+      KVStore.getValue('autoUpdate.windowEnd'),
+      KVStore.getValue('autoUpdate.cooloffHours'),
+    ])
+
+    const parsedCooloff = Number(cooloffHours)
+    return {
+      enabled: enabled ?? false,
+      windowStart: windowStart || DEFAULT_WINDOW_START,
+      windowEnd: windowEnd || DEFAULT_WINDOW_END,
+      // `Number(null) === 0`, so an *unset* value must fall through to the default
+      // rather than silently resolving to a zero cool-off. An explicit 0 is honored.
+      cooloffHours:
+        cooloffHours != null && Number.isFinite(parsedCooloff) && parsedCooloff >= 0
+          ? parsedCooloff
+          : DEFAULT_COOLOFF_HOURS,
+    }
+  }
+
+  /**
+   * Determine whether `now` falls inside the configured update window. The window
+   * is interpreted in the container's local time (set via the TZ env var). Windows
+   * that wrap past midnight (start > end, e.g. 22:00-02:00) are handled.
+   */
+  isWithinWindow(config: AutoUpdateConfig, now: DateTime = DateTime.now()): boolean {
+    return isWithinWindowUtil(config.windowStart, config.windowEnd, now)
+  }
+
+  /**
+   * Find the newest release that is safe to auto-apply: same major version as the
+   * running build (major bumps are deliberately left for manual update), strictly
+   * newer than current, and published at least `cooloffHours` ago. Prereleases and
+   * drafts are ignored — auto-update never rides early access.
+   *
+   * Returns null when nothing qualifies (e.g. only a major bump is newer, or the
+   * newest eligible release is still inside its cool-off window).
+   */
+  async getEligibleTarget(config: AutoUpdateConfig): Promise<EligibleTarget | null> {
+    const releases = await this.fetchReleases()
+    return this.selectEligibleTarget(
+      releases,
+      SystemService.getAppVersion(),
+      config.cooloffHours,
+      DateTime.now()
+    )
+  }
+
+  /**
+   * Fetch the published GitHub releases for the NOMAD repo, cached in-process.
+   * A successful result is reused for {@link RELEASES_CACHE_TTL_MS} so repeated
+   * status-page loads don't each hit (and risk rate-limiting) the unauthenticated
+   * GitHub API. A recent failure is negatively cached for {@link RELEASES_FAILURE_TTL_MS}
+   * so back-to-back calls while offline don't each block on the request timeout.
+   */
+  async fetchReleases(): Promise<GithubRelease[]> {
+    const now = Date.now()
+    const cached = AutoUpdateService.releasesCache
+    if (cached && now - cached.at < RELEASES_CACHE_TTL_MS) {
+      return cached.releases
+    }
+    if (now - AutoUpdateService.releasesFailureAt < RELEASES_FAILURE_TTL_MS) {
+      throw new Error('GitHub releases fetch recently failed; backing off')
+    }
+
+    try {
+      const response = await axios.get(RELEASES_URL, {
+        headers: { Accept: 'application/vnd.github+json' },
+        timeout: 5000,
+      })
+      if (!Array.isArray(response.data)) {
+        throw new Error('Unexpected response from GitHub releases API')
+      }
+      AutoUpdateService.releasesCache = { releases: response.data, at: now }
+      return response.data
+    } catch (error) {
+      AutoUpdateService.releasesFailureAt = now
+      throw error
+    }
+  }
+
+  /**
+   * Pure selection of the newest auto-applicable release from a release list.
+   * Extracted so the dry-run command and tests can drive it with fixtures.
+   * Same major as `currentVersion`, strictly newer, published on/before
+   * `now - cooloffHours`, prereleases/drafts excluded. Returns null for dev
+   * builds or when nothing qualifies.
+   */
+  selectEligibleTarget(
+    releases: GithubRelease[],
+    currentVersion: string,
+    cooloffHours: number,
+    now: DateTime
+  ): EligibleTarget | null {
+    if (currentVersion === 'dev' || currentVersion === '0.0.0') {
+      return null
+    }
+    const currentMajor = parseMajorVersion(currentVersion)
+    const cutoff = now.minus({ hours: cooloffHours })
+
+    const candidates = releases
+      .filter((r) => r && !r.draft && !r.prerelease && r.tag_name && r.published_at)
+      .map((r) => ({
+        version: String(r.tag_name).replace(/^v/, '').trim(),
+        publishedAt: String(r.published_at),
+      }))
+      .filter((r) => SEMVER_TAG.test(r.version))
+      .filter((r) => parseMajorVersion(r.version) === currentMajor)
+      .filter((r) => isNewerVersion(r.version, currentVersion))
+      .filter((r) => DateTime.fromISO(r.publishedAt) <= cutoff)
+      .sort((a, b) => (isNewerVersion(a.version, b.version) ? -1 : 1))
+
+    const best = candidates[0]
+    if (!best) return null
+
+    return {
+      version: best.version,
+      tag: `v${best.version}`,
+      publishedAt: best.publishedAt,
+    }
+  }
+
+  /**
+   * Pre-flight checks that gate an auto-update. `skip` blockers are transient
+   * (retry next window, no penalty); `failure` blockers count toward the backoff
+   * that eventually auto-disables auto-update.
+   */
+  async runPreflight(targetTag: string): Promise<PreflightResult> {
+    const blockers: Blocker[] = []
+
+    // 1. Sidecar must be present to perform the update.
+    if (!this.systemUpdateService.isSidecarAvailable()) {
+      blockers.push({ reason: 'Update sidecar is not available', severity: 'failure' })
+    }
+
+    // 2. No system update already running.
+    const updateStatus = this.systemUpdateService.getUpdateStatus()
+    if (updateStatus && !['idle', 'complete', 'error'].includes(updateStatus.stage)) {
+      blockers.push({
+        reason: `A system update is already in progress (stage: ${updateStatus.stage})`,
+        severity: 'skip',
+      })
+    }
+
+    // 3. No content/model downloads in progress.
+    try {
+      const downloads = await this.downloadService.listDownloadJobs()
+      const active = downloads.filter(
+        (d) => !!d.status && ['waiting', 'active', 'delayed'].includes(d.status)
+      )
+      if (active.length > 0) {
+        blockers.push({
+          reason: `${active.length} download(s) in progress`,
+          severity: 'skip',
+        })
+      }
+    } catch (error) {
+      logger.warn(`[AutoUpdateService] Could not check active downloads: ${error.message}`)
+    }
+
+    // 4. No app (container) install/update in progress.
+    try {
+      const installing = await Service.query().whereNot('installation_status', 'idle')
+      if (installing.length > 0) {
+        blockers.push({
+          reason: `${installing.length} app install/update(s) in progress`,
+          severity: 'skip',
+        })
+      }
+    } catch (error) {
+      logger.warn(`[AutoUpdateService] Could not check app installations: ${error.message}`)
+    }
+
+    // 5. Sufficient host storage for the new image.
+    const diskBlocker = await this.checkDiskSpace(targetTag)
+    if (diskBlocker) blockers.push(diskBlocker)
+
+    return { ok: blockers.length === 0, blockers }
+  }
+
+  /** Returns a disk blocker if free space is insufficient, otherwise null. */
+  private async checkDiskSpace(targetTag: string): Promise<Blocker | null> {
+    const hostArch = await this.getHostArch()
+    return checkImageDiskSpace({
+      image: `${NOMAD_IMAGE_REPO}:${targetTag}`,
+      hostArch,
+      containerRegistryService: this.containerRegistryService,
+      systemService: this.systemService,
+    })
+  }
+
+  /** Map the Docker daemon's architecture string to OCI naming (amd64/arm64/...). */
+  private async getHostArch(): Promise<string> {
+    try {
+      const info = await this.dockerService.docker.info()
+      const arch = info.Architecture || ''
+      const archMap: Record<string, string> = {
+        x86_64: 'amd64',
+        aarch64: 'arm64',
+        armv7l: 'arm',
+        amd64: 'amd64',
+        arm64: 'arm64',
+      }
+      return archMap[arch] || arch.toLowerCase()
+    } catch {
+      return 'amd64'
+    }
+  }
+
+  /**
+   * Side-effect-free core of the decision pipeline. Resolves the effective config
+   * (settings, overridable), checks the window, finds an eligible target, and runs
+   * pre-flight — returning a verdict WITHOUT requesting an update or mutating any
+   * persisted state. Both {@link attempt} (production) and {@link dryRun} (testing)
+   * are built on this so a dry run faithfully reflects what a real run would do.
+   */
+  async evaluate(overrides: EvaluateOverrides = {}): Promise<AutoUpdateDecision> {
+    const baseConfig = await this.getConfig()
+    const config: AutoUpdateConfig = {
+      enabled: overrides.forceEnabled ?? baseConfig.enabled,
+      windowStart: overrides.windowStart ?? baseConfig.windowStart,
+      windowEnd: overrides.windowEnd ?? baseConfig.windowEnd,
+      cooloffHours: overrides.cooloffHours ?? baseConfig.cooloffHours,
+    }
+    const now = overrides.now ?? DateTime.now()
+    const currentVersion = overrides.currentVersion ?? SystemService.getAppVersion()
+
+    const base = {
+      enabled: config.enabled,
+      currentVersion,
+      config,
+      withinWindow: false,
+      eligibleTarget: null as EligibleTarget | null,
+      preflight: null as PreflightResult | null,
+    }
+
+    if (!config.enabled) {
+      return { ...base, outcome: 'disabled', reason: 'Auto-update is disabled' }
+    }
+
+    const withinWindow = this.isWithinWindow(config, now)
+    if (!withinWindow) {
+      return {
+        ...base,
+        outcome: 'outside-window',
+        reason: `Outside update window (${config.windowStart}-${config.windowEnd})`,
+      }
+    }
+
+    let eligibleTarget: EligibleTarget | null
+    try {
+      const releases = overrides.releases ?? (await this.fetchReleases())
+      eligibleTarget = this.selectEligibleTarget(releases, currentVersion, config.cooloffHours, now)
+    } catch (error) {
+      return {
+        ...base,
+        withinWindow,
+        outcome: 'eligibility-error',
+        reason: `Failed to determine eligible version: ${error.message}`,
+      }
+    }
+
+    if (!eligibleTarget) {
+      return {
+        ...base,
+        withinWindow,
+        outcome: 'no-eligible',
+        reason: 'No eligible minor/patch update available (or still in cool-off)',
+      }
+    }
+
+    const preflight = overrides.fakePreflight
+      ? overrides.fakePreflight
+      : overrides.skipPreflight
+        ? { ok: true, blockers: [] }
+        : await this.runPreflight(eligibleTarget.tag)
+
+    if (!preflight.ok) {
+      const summary = preflight.blockers.map((b) => b.reason).join('; ')
+      return {
+        ...base,
+        withinWindow,
+        eligibleTarget,
+        preflight,
+        outcome: 'blocked',
+        reason: `Pre-flight blocked: ${summary}`,
+      }
+    }
+
+    return {
+      ...base,
+      withinWindow,
+      eligibleTarget,
+      preflight,
+      outcome: 'ready',
+      reason: `Ready to update to ${eligibleTarget.tag}`,
+    }
+  }
+
+  /**
+   * Run the full decision pipeline WITHOUT requesting an update or recording any
+   * state. Accepts the same injectable overrides as {@link evaluate}, so callers can
+   * simulate any scenario (a given current version, a canned release list, a fixed
+   * clock, a forced window) and see exactly what a real run would decide.
+   */
+  async dryRun(overrides: EvaluateOverrides = {}): Promise<AutoUpdateDecision> {
+    return this.evaluate(overrides)
+  }
+
+  /**
+   * The entry point invoked by AutoUpdateJob. Evaluates the decision pipeline and,
+   * when everything passes, requests the update with the vetted tag — recording the
+   * outcome to the KVStore (for the UI) and applying failure backoff.
+   */
+  async attempt(): Promise<{ updated: boolean; reason: string }> {
+    const decision = await this.evaluate()
+
+    switch (decision.outcome) {
+      case 'disabled':
+        return { updated: false, reason: decision.reason }
+
+      case 'outside-window':
+      case 'no-eligible':
+      // A failed release lookup is transient (offline-first appliances are
+      // routinely without connectivity) — treat as a skip so it never trips the
+      // backoff that auto-disables the feature. Only real update-request failures
+      // (the `ready` case below) count toward MAX_CONSECUTIVE_FAILURES.
+      case 'eligibility-error':
+        await this.recordSkip(decision.reason)
+        return { updated: false, reason: decision.reason }
+
+      case 'blocked': {
+        const hasFailure = decision.preflight!.blockers.some((b) => b.severity === 'failure')
+        if (hasFailure) {
+          await this.recordFailure(decision.reason)
+        } else {
+          await this.recordSkip(decision.reason)
+        }
+        return { updated: false, reason: decision.reason }
+      }
+
+      case 'ready': {
+        const target = decision.eligibleTarget!
+        const result = await this.systemUpdateService.requestUpdate({
+          targetTag: target.tag,
+          requester: 'auto-update',
+        })
+
+        if (result.success) {
+          await this.recordSuccess(target)
+          logger.info(`[AutoUpdateService] Auto-update requested: ${target.tag}`)
+          return { updated: true, reason: `Update requested: ${target.tag}` }
+        }
+
+        await this.recordFailure(`Update request failed: ${result.message}`)
+        return { updated: false, reason: result.message }
+      }
+    }
+  }
+
+  // --- Outcome recording -----------------------------------------------------
+
+  private async recordSuccess(target: EligibleTarget): Promise<void> {
+    await KVStore.setValue('autoUpdate.lastAttemptAt', DateTime.now().toISO()!)
+    await KVStore.setValue('autoUpdate.lastResult', `Update requested: ${target.tag}`)
+    await KVStore.clearValue('autoUpdate.lastError')
+    await KVStore.setValue('autoUpdate.consecutiveFailures', '0')
+  }
+
+  private async recordSkip(reason: string): Promise<void> {
+    await KVStore.setValue('autoUpdate.lastAttemptAt', DateTime.now().toISO()!)
+    await KVStore.setValue('autoUpdate.lastResult', reason)
+    logger.info(`[AutoUpdateService] Skipped: ${reason}`)
+  }
+
+  private async recordFailure(reason: string): Promise<void> {
+    await KVStore.setValue('autoUpdate.lastAttemptAt', DateTime.now().toISO()!)
+    await KVStore.setValue('autoUpdate.lastResult', reason)
+    await KVStore.setValue('autoUpdate.lastError', reason)
+
+    const prior = Number(await KVStore.getValue('autoUpdate.consecutiveFailures')) || 0
+    const failures = prior + 1
+    await KVStore.setValue('autoUpdate.consecutiveFailures', String(failures))
+    logger.error(`[AutoUpdateService] Failure ${failures}/${MAX_CONSECUTIVE_FAILURES}: ${reason}`)
+
+    if (failures >= MAX_CONSECUTIVE_FAILURES) {
+      await KVStore.setValue('autoUpdate.enabled', false)
+      await KVStore.setValue(
+        'autoUpdate.autoDisabledReason',
+        `Auto-update disabled after ${failures} consecutive failures. Last error: ${reason}`
+      )
+      logger.error(
+        `[AutoUpdateService] Auto-update auto-disabled after ${failures} consecutive failures`
+      )
+    }
+  }
+
+  /** Full state snapshot for the settings UI. */
+  async getStatus(): Promise<AutoUpdateStatus> {
+    const config = await this.getConfig()
+    const currentVersion = SystemService.getAppVersion()
+
+    let eligibleTarget: EligibleTarget | null = null
+    try {
+      eligibleTarget = await this.getEligibleTarget(config)
+    } catch (error) {
+      logger.warn(`[AutoUpdateService] getStatus eligibility lookup failed: ${error.message}`)
+    }
+
+    const [lastAttemptAt, lastResult, lastError, consecutiveFailures, autoDisabledReason] =
+      await Promise.all([
+        KVStore.getValue('autoUpdate.lastAttemptAt'),
+        KVStore.getValue('autoUpdate.lastResult'),
+        KVStore.getValue('autoUpdate.lastError'),
+        KVStore.getValue('autoUpdate.consecutiveFailures'),
+        KVStore.getValue('autoUpdate.autoDisabledReason'),
+      ])
+
+    return {
+      ...config,
+      currentVersion,
+      withinWindow: this.isWithinWindow(config),
+      eligibleTarget,
+      lastAttemptAt: lastAttemptAt || null,
+      lastResult: lastResult || null,
+      lastError: lastError || null,
+      consecutiveFailures: Number(consecutiveFailures) || 0,
+      autoDisabledReason: autoDisabledReason || null,
+    }
+  }
+}

admin/app/services/benchmark_telemetry.ts

@@ -0,0 +1,102 @@
+import transmit from '@adonisjs/transmit/services/main'
+import si from 'systeminformation'
+import logger from '@adonisjs/core/services/logger'
+import { BROADCAST_CHANNELS } from '../../constants/broadcast.js'
+import type { BenchmarkStatus, BenchmarkTelemetry } from '../../types/benchmark.js'
+
+const SAMPLE_INTERVAL_MS = 1000
+
+/**
+ * Samples host telemetry (per-core CPU load, temperature, disk throughput) on a
+ * timer and broadcasts it over SSE while a benchmark runs. Read-only: the sampler
+ * runs in the orchestration process, never inside the sysbench container, so it
+ * cannot influence the scored numbers.
+ *
+ * Signals come from `systeminformation`, which reads the host's /proc and /sys
+ * (not namespaced for these counters), so this works from inside a container too.
+ * Temperature legitimately isn't available on every host (VMs, some hwmon-less
+ * boards) and is reported as null rather than faked.
+ */
+export class BenchmarkTelemetrySampler {
+  private timer: NodeJS.Timeout | null = null
+  private readonly benchmarkId: string | null
+  private status: BenchmarkStatus = 'starting'
+  private startedAt = Date.now()
+  private stageMetric: BenchmarkTelemetry['stage_metric'] | undefined
+  private gpu: BenchmarkTelemetry['gpu'] | undefined
+  private sampling = false
+
+  constructor(benchmarkId: string | null) {
+    this.benchmarkId = benchmarkId
+  }
+
+  start() {
+    if (this.timer) return
+    this.startedAt = Date.now()
+    // Prime systeminformation's per-second deltas and emit an initial frame,
+    // then continue on the interval.
+    void this._sample()
+    this.timer = setInterval(() => void this._sample(), SAMPLE_INTERVAL_MS)
+  }
+
+  /** Tag subsequent frames with the current stage; clears any prior in-test state. */
+  setStage(status: BenchmarkStatus) {
+    this.status = status
+    this.stageMetric = undefined
+    this.gpu = undefined
+  }
+
+  /** Inject an in-test metric (e.g. live AI tokens/sec) into subsequent frames. */
+  setStageMetric(kind: NonNullable<BenchmarkTelemetry['stage_metric']>['kind'], value: number, ttftMs?: number) {
+    this.stageMetric = { kind, value, ...(ttftMs !== undefined ? { ttft_ms: ttftMs } : {}) }
+  }
+
+  /** Inject NVIDIA GPU stats into subsequent frames (null clears them). */
+  setGpuStats(gpu: NonNullable<BenchmarkTelemetry['gpu']> | null) {
+    this.gpu = gpu ?? undefined
+  }
+
+  stop() {
+    if (this.timer) {
+      clearInterval(this.timer)
+      this.timer = null
+    }
+  }
+
+  private async _sample() {
+    // Skip if the previous sample is still in flight (currentLoad can take a
+    // couple hundred ms); never let samples pile up.
+    if (this.sampling) return
+    this.sampling = true
+    try {
+      const [load, temp, fs] = await Promise.all([
+        si.currentLoad().catch(() => null),
+        si.cpuTemperature().catch(() => null),
+        si.fsStats().catch(() => null),
+      ])
+
+      const overall = load ? Math.max(0, Math.round(load.currentLoad)) : 0
+      const perCore = load?.cpus?.map((c) => Math.max(0, Math.round(c.load))) ?? []
+      const tempC = temp && typeof temp.main === 'number' && temp.main > 0 ? Math.round(temp.main) : null
+      const readMb = fs && typeof fs.rx_sec === 'number' && fs.rx_sec >= 0 ? Number((fs.rx_sec / 1e6).toFixed(1)) : 0
+      const writeMb = fs && typeof fs.wx_sec === 'number' && fs.wx_sec >= 0 ? Number((fs.wx_sec / 1e6).toFixed(1)) : 0
+
+      const payload: BenchmarkTelemetry = {
+        benchmark_id: this.benchmarkId,
+        status: this.status,
+        t: Date.now() - this.startedAt,
+        cpu: { overall, per_core: perCore },
+        temp_c: tempC,
+        disk: { read_mb_s: readMb, write_mb_s: writeMb },
+        ...(this.stageMetric ? { stage_metric: this.stageMetric } : {}),
+        ...(this.gpu ? { gpu: this.gpu } : {}),
+      }
+
+      transmit.broadcast(BROADCAST_CHANNELS.BENCHMARK_TELEMETRY, payload)
+    } catch (err) {
+      logger.debug(`[BenchmarkTelemetry] sample failed: ${err.message}`)
+    } finally {
+      this.sampling = false
+    }
+  }
+}

admin/app/services/chat_service.ts

@@ -1,5 +1,6 @@
 import ChatSession from '#models/chat_session'
 import ChatMessage from '#models/chat_message'
+import KVStore from '#models/kv_store'
 import logger from '@adonisjs/core/services/logger'
 import { DateTime } from 'luxon'
 import { inject } from '@adonisjs/core'
@@ -36,17 +37,23 @@ export class ChatService {
         return [] // If no models are available, return empty suggestions
       }
 
-      // Larger models generally give "better" responses, so pick the largest one
+      // Prefer the user's selected chat model. Fall back to the smallest
-      const largestModel = models.reduce((prev, current) => {
+      // installed model — picking the largest by file size is unsafe: if any
-        return prev.size > current.size ? prev : current
+      // installed model exceeds available VRAM (e.g. llama3.1:405b on a 96 GB
-      })
+      // GPU), Ollama spends minutes trying to load it and the request 500s.
+      // Suggestions are short prompts that don't benefit from a flagship model.
+      const lastModel = await KVStore.getValue('chat.lastModel')
+      const preferred = lastModel ? models.find((m) => m.name === lastModel) : undefined
+      const chosen =
+        preferred ??
+        models.reduce((prev, current) => (prev.size < current.size ? prev : current))
 
-      if (!largestModel) {
+      if (!chosen) {
         return []
       }
 
       const response = await this.ollamaService.chat({
-        model: largestModel.name,
+        model: chosen.name,
         messages: [
           {
             role: 'user',

admin/app/services/collection_manifest_service.ts

@@ -5,9 +5,11 @@ import { DateTime } from 'luxon'
 import { join } from 'path'
 import CollectionManifest from '#models/collection_manifest'
 import InstalledResource from '#models/installed_resource'
+import WikipediaSelection from '#models/wikipedia_selection'
 import { QueueService } from './queue_service.js'
 import { RunDownloadJob } from '#jobs/run_download_job'
-import { zimCategoriesSpecSchema, mapsSpecSchema, wikipediaSpecSchema } from '#validators/curated_collections'
+import { zimCategoriesSpecSchema, mapsSpecSchema, wikipediaSpecSchema, creatorPacksSpecSchema } from '#validators/curated_collections'
+import { isGatedResource } from '../utils/hosted_content.js'
 import {
   ensureDirectoryExists,
   listDirectoryContents,
@@ -18,8 +20,10 @@ import type {
   ManifestType,
   ZimCategoriesSpec,
   MapsSpec,
+  CreatorPacksSpec,
   CategoryWithStatus,
   CollectionWithStatus,
+  CreatorPackWithStatus,
   SpecResource,
   SpecTier,
 } from '../../types/collections.js'
@@ -28,12 +32,14 @@ const SPEC_URLS: Record<ManifestType, string> = {
   zim_categories: 'https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/collections/kiwix-categories.json',
   maps: 'https://github.com/Crosstalk-Solutions/project-nomad/raw/refs/heads/main/collections/maps.json',
   wikipedia: 'https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/collections/wikipedia.json',
+  creator_packs: 'https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/collections/creator-packs.json',
 }
 
 const VALIDATORS: Record<ManifestType, any> = {
   zim_categories: zimCategoriesSpecSchema,
   maps: mapsSpecSchema,
   wikipedia: wikipediaSpecSchema,
+  creator_packs: creatorPacksSpecSchema,
 }
 
 export class CollectionManifestService {
@@ -97,15 +103,28 @@ export class CollectionManifestService {
     const spec = await this.getSpecWithFallback<ZimCategoriesSpec>('zim_categories')
     if (!spec) return []
 
-    const installedResources = await InstalledResource.query().where('resource_type', 'zim')
+    // Include 'dataset' rows alongside 'zim' so a curated tier carrying the FDA
+    // drug dataset reads "installed" once the ingest writes its row (the tier-
+    // status math below treats every resolved resource id uniformly — a dataset
+    // resource is just another id to account for).
+    const installedResources = await InstalledResource.query().whereIn('resource_type', [
+      'zim',
+      'dataset',
+    ])
     const installedMap = new Map(installedResources.map((r) => [r.resource_id, r]))
 
-    // In-flight ZIM download resource IDs from the BullMQ queue. Used to
+    // In-flight ZIM + dataset download resource IDs from the BullMQ queues. Used
-    // surface the user's tier intent immediately on submit, before any single
+    // to surface the user's tier intent immediately on submit, before any single
-    // file has finished downloading. Failed jobs are excluded so a stuck
+    // file has finished downloading. Failed jobs are excluded so a stuck queue
-    // queue entry doesn't keep claiming the user's pick forever.
+    // entry doesn't keep claiming the user's pick forever.
     const inFlightIds = await this.getInFlightZimResourceIds()
 
+    // Whether the in-flight drug dataset is in its INGEST (indexing) phase — the
+    // download finished and the heavy ingest is running. Lets the wizard card
+    // flip "(downloading)" → "(indexing)" at the handoff (Req 7) instead of
+    // showing a stale "downloading" through the long index.
+    const drugIndexing = await this.isDrugDatasetIndexing()
+
     return spec.categories.map((category) => {
       const installedTierSlug = this.getInstalledTierForCategory(category.tiers, installedMap)
       const downloadingTierSlug = this.getDownloadingTierForCategory(
@@ -114,10 +133,47 @@ export class CollectionManifestService {
         inFlightIds,
         installedTierSlug
       )
-      return { ...category, installedTierSlug, downloadingTierSlug }
+      // Only mark "indexing" when this category's downloading tier actually
+      // carries a dataset resource that is the one indexing.
+      const downloadingTierIndexing =
+        drugIndexing && downloadingTierSlug
+          ? CollectionManifestService.resolveTierResources(
+              category.tiers.find((t) => t.slug === downloadingTierSlug)!,
+              category.tiers
+            ).some((r) => r.type === 'dataset')
+          : false
+      return { ...category, installedTierSlug, downloadingTierSlug, downloadingTierIndexing }
     })
   }
 
+  /**
+   * True when the FDA drug dataset's INGEST is in flight while its DOWNLOAD is
+   * not — i.e. the handoff into the indexing phase. Drives the wizard card's
+   * "(indexing)" label. Defensive: any queue read failure returns false (the card
+   * just keeps showing "(downloading)") rather than breaking the categories list.
+   */
+  private async isDrugDatasetIndexing(): Promise<boolean> {
+    try {
+      const { DownloadDrugDataJob } = await import('#jobs/download_drug_data_job')
+      const { IngestDrugDataJob } = await import('#jobs/ingest_drug_data_job')
+      const queueService = QueueService.getInstance()
+
+      const ingestQueue = queueService.getQueue(IngestDrugDataJob.queue)
+      const ingestJobs = await ingestQueue.getJobs(['active', 'waiting', 'delayed'])
+      if (ingestJobs.length === 0) return false
+
+      const downloadQueue = queueService.getQueue(DownloadDrugDataJob.queue)
+      const downloadJobs = await downloadQueue.getJobs(['active', 'waiting', 'delayed'])
+      return downloadJobs.length === 0
+    } catch (error: any) {
+      logger.warn(
+        '[CollectionManifestService] Could not determine drug indexing state:',
+        error?.message || error
+      )
+      return false
+    }
+  }
+
   private async getInFlightZimResourceIds(): Promise<Set<string>> {
     const ids = new Set<string>()
     try {
@@ -133,9 +189,47 @@ export class CollectionManifestService {
       // unreachable — just report no in-flight downloads.
       logger.warn('[CollectionManifestService] Could not read download queue:', error?.message || error)
     }
+
+    // Also surface an in-flight curated-tier drug-dataset install. The drug
+    // download/ingest run on their own queues (not RunDownloadJob), carrying the
+    // dataset's resource id in resourceMeta — read it so the wizard shows the
+    // Medicine tier as "downloading" the moment the user opts in, mirroring the
+    // ZIM behaviour. Scanned independently so a missing drug queue can't blank
+    // the ZIM in-flight set above.
+    await this.addInFlightDrugDatasetId(ids)
+
     return ids
   }
 
+  /**
+   * Add the FDA drug dataset's manifest resource id to `ids` when its download or
+   * ingest is in flight. The dataset only counts as "downloading" while no
+   * install-state row exists yet — once ingest writes the row it is "installed"
+   * (handled by the installedMap), so a still-running ingest correctly reads as
+   * the in-flight tier intent here. resourceMeta is only present on a curated-tier
+   * install, so a manual (non-tier) drug download never claims a tier slug.
+   */
+  private async addInFlightDrugDatasetId(ids: Set<string>): Promise<void> {
+    try {
+      const { DownloadDrugDataJob } = await import('#jobs/download_drug_data_job')
+      const { IngestDrugDataJob } = await import('#jobs/ingest_drug_data_job')
+      const queueService = QueueService.getInstance()
+      for (const queueName of [DownloadDrugDataJob.queue, IngestDrugDataJob.queue]) {
+        const queue = queueService.getQueue(queueName)
+        const jobs = await queue.getJobs(['waiting', 'active', 'delayed'])
+        for (const job of jobs) {
+          const resourceId = job.data?.resourceMeta?.resourceId
+          if (typeof resourceId === 'string') ids.add(resourceId)
+        }
+      }
+    } catch (error: any) {
+      logger.warn(
+        '[CollectionManifestService] Could not read drug dataset queue:',
+        error?.message || error
+      )
+    }
+  }
+
   /**
    * Highest tier whose every resource is installed OR has an in-flight
    * download. Returns undefined when there are no in-flight downloads for this
@@ -188,6 +282,72 @@ export class CollectionManifestService {
     })
   }
 
+  /**
+   * Per-pack install status for Creator Packs. Packs are single ZIMs, so this is
+   * a one-resource-per-pack join (simpler than the tiered category logic):
+   * catalog ⋈ InstalledResource (resource_type 'zim', matched on resource_id) ⋈
+   * the in-flight download queue. `available_update_version` is set when an
+   * installed pack's version trails the catalog (a creator published a rebuild).
+   */
+  async getCreatorPacksWithStatus(): Promise<CreatorPackWithStatus[]> {
+    const spec = await this.getSpecWithFallback<CreatorPacksSpec>('creator_packs')
+    if (!spec) return []
+
+    const installedResources = await InstalledResource.query().where('resource_type', 'zim')
+    const installedMap = new Map(installedResources.map((r) => [r.resource_id, r]))
+    const inFlightIds = await this.getInFlightZimResourceIds()
+
+    return spec.packs.map((pack) => {
+      const installed = installedMap.get(pack.resource_id)
+      if (installed) {
+        const hasUpdate = installed.version !== pack.version
+        return {
+          ...pack,
+          status: 'installed' as const,
+          installed_version: installed.version,
+          ...(hasUpdate ? { available_update_version: pack.version } : {}),
+        }
+      }
+      if (inFlightIds.has(pack.resource_id)) {
+        return { ...pack, status: 'downloading' as const }
+      }
+      return { ...pack, status: 'available' as const }
+    })
+  }
+
+  /**
+   * Resource ids in the ZIM manifest that we host ourselves behind the
+   * entitlement Worker (`auth: 'nomad_app_key'`).
+   *
+   * Used to keep gated content out of the Kiwix-catalog update path. Those
+   * resources are not in the openzim catalog, so they can never legitimately
+   * match there — but a resource-id collision would otherwise let a third-party
+   * mirror present itself as a newer version and overwrite our content. Their
+   * versions come from the manifest instead.
+   *
+   * Reads the CACHED spec rather than refetching: this sits on the scheduled
+   * update-check path and does not need a network round-trip. A gated resource
+   * cannot be installed without the manifest having been fetched first, so the
+   * cache is always populated by the time it matters.
+   *
+   * Returns an empty set if the manifest has never been cached, which correctly
+   * degrades to current behaviour rather than skipping every update.
+   */
+  async getGatedZimResourceIds(): Promise<Set<string>> {
+    const ids = new Set<string>()
+    const spec = await this.getCachedSpec<ZimCategoriesSpec>('zim_categories')
+    if (!spec) return ids
+
+    for (const category of spec.categories) {
+      for (const tier of category.tiers) {
+        for (const resource of tier.resources) {
+          if (isGatedResource(resource)) ids.add(resource.id)
+        }
+      }
+    }
+    return ids
+  }
+
   // ---- Tier resolution ----
 
   static resolveTierResources(tier: SpecTier, allTiers: SpecTier[]): SpecResource[] {
@@ -284,10 +444,17 @@ export class CollectionManifestService {
 
       const seenZimIds = new Set<string>()
 
+      // Only skip the single Wikipedia file tracked by WikipediaSelection — not every file
+      // starting with `wikipedia_en_`. Curated category tiers (e.g. Medicine → Comprehensive)
+      // ship Wikipedia-themed ZIMs like `wikipedia_en_medicine_maxi` that must reconcile
+      // normally; otherwise their InstalledResource row gets wiped on every restart and the
+      // tier detection silently downgrades.
+      const wikipediaSelection = await WikipediaSelection.query().first()
+      const managedWikipediaFilename = wikipediaSelection?.filename ?? null
+
       for (const file of zimFiles) {
         console.log(`Processing ZIM file: ${file.name}`)
-        // Skip Wikipedia files (managed by WikipediaSelection model)
+        if (managedWikipediaFilename && file.name === managedWikipediaFilename) continue
-        if (file.name.startsWith('wikipedia_en_')) continue
 
         const parsed = CollectionManifestService.parseZimFilename(file.name)
         console.log(`Parsed ZIM filename:`, parsed)

admin/app/services/collection_update_service.ts

@@ -1,16 +1,16 @@
 import logger from '@adonisjs/core/services/logger'
-import env from '#start/env'
 import axios from 'axios'
+import { DateTime } from 'luxon'
 import InstalledResource from '#models/installed_resource'
 import { RunDownloadJob } from '../jobs/run_download_job.js'
 import { ZIM_STORAGE_PATH } from '../utils/fs.js'
 import { join } from 'path'
 import type {
-  ResourceUpdateCheckRequest,
   ResourceUpdateInfo,
   ContentUpdateCheckResult,
 } from '../../types/collections.js'
-import { NOMAD_API_DEFAULT_BASE_URL } from '../../constants/misc.js'
+import { KiwixCatalogService, reconcileResourceUpdateState } from './kiwix_catalog_service.js'
+import { CollectionManifestService } from './collection_manifest_service.js'
 
 const MAP_STORAGE_PATH = '/storage/maps'
 
@@ -18,17 +18,28 @@ const ZIM_MIME_TYPES = ['application/x-zim', 'application/x-openzim', 'applicati
 const PMTILES_MIME_TYPES = ['application/vnd.pmtiles', 'application/octet-stream']
 
 export class CollectionUpdateService {
+  /**
+   * Check every installed resource against the upstream catalogs locally (Kiwix
+   * OPDS for ZIMs, GitHub for maps) — no longer routed through the external
+   * project-nomad-api. Side-effect: persists each resource's available-update
+   * state (version + cool-off anchor) so the auto-updater can act on it later.
+   */
   async checkForUpdates(): Promise<ContentUpdateCheckResult> {
-    const nomadAPIURL = env.get('NOMAD_API_URL') || NOMAD_API_DEFAULT_BASE_URL
+    // ZIM/map catalog update path only — exclude `dataset` resources (e.g. the
-    if (!nomadAPIURL) {
+    // FDA drug labels), which are not filename-versioned and get their own
-      return {
+    // freshness path. No-op today (no dataset rows are written in this slice).
-        updates: [],
+    const allInstalled = await InstalledResource.query().whereNot('resource_type', 'dataset')
-        checked_at: new Date().toISOString(),
+
-        error: 'Nomad API is not configured. Set the NOMAD_API_URL environment variable.',
+    // Content we host ourselves is versioned by the manifest, not by the Kiwix
-      }
+    // catalog, so it has no business in this check. Excluding it also means a
-    }
+    // resource-id collision can't let a third-party mirror present itself as a
+    // newer version and overwrite our content. See resolveZimDownload, which
+    // pins the same resources to their manifest URL on the install path.
+    const gatedIds = await new CollectionManifestService().getGatedZimResourceIds()
+    const installed = allInstalled.filter(
+      (r) => !(r.resource_type === 'zim' && gatedIds.has(r.resource_id))
+    )
 
-    const installed = await InstalledResource.all()
     if (installed.length === 0) {
       return {
         updates: [],
@@ -36,53 +47,57 @@ export class CollectionUpdateService {
       }
     }
 
-    const requestBody: ResourceUpdateCheckRequest = {
-      resources: installed.map((r) => ({
-        resource_id: r.resource_id,
-        resource_type: r.resource_type,
-        installed_version: r.version,
-      })),
-    }
-
     try {
-      const response = await axios.post<ResourceUpdateInfo[]>(`${nomadAPIURL}/api/v1/resources/check-updates`, requestBody, {
+      const catalog = new KiwixCatalogService()
-        timeout: 15000,
+      const latestByKey = await catalog.getLatestForResources(
-      })
+        // `dataset` rows are filtered out above, so the type narrows to ZIM/map.
-
+        installed.map((r) => ({
-      logger.info(
+          resource_id: r.resource_id,
-        `[CollectionUpdateService] Update check complete: ${response.data.length} update(s) available`
+          resource_type: r.resource_type as 'zim' | 'map',
+        }))
       )
 
-      const updates = await this.enrichWithSizes(response.data)
+      const now = DateTime.now()
+      const updates: ResourceUpdateInfo[] = []
+      for (const resource of installed) {
+        const latest = latestByKey.get(`${resource.resource_type}:${resource.resource_id}`) ?? null
+        await reconcileResourceUpdateState(resource, latest, now)
 
+        if (latest && latest.version > resource.version) {
+          updates.push({
+            resource_id: resource.resource_id,
+            resource_type: resource.resource_type as 'zim' | 'map',
+            installed_version: resource.version,
+            latest_version: latest.version,
+            download_url: latest.download_url,
+            size_bytes: latest.size_bytes || undefined,
+          })
+        }
+      }
+
+      logger.info(
+        `[CollectionUpdateService] Local update check complete: ${updates.length} update(s) available`
+      )
+
+      const enriched = await this.enrichWithSizes(updates)
       return {
-        updates,
+        updates: enriched,
         checked_at: new Date().toISOString(),
       }
     } catch (error) {
-      if (axios.isAxiosError(error) && error.response) {
+      const message = error instanceof Error ? error.message : 'Unknown error during update check'
-        logger.error(
-          `[CollectionUpdateService] Nomad API returned ${error.response.status}: ${JSON.stringify(error.response.data)}`
-        )
-        return {
-          updates: [],
-          checked_at: new Date().toISOString(),
-          error: 'Failed to check for content updates. The update service may be temporarily unavailable.',
-        }
-      }
-      const message =
-        error instanceof Error ? error.message : 'Unknown error contacting Nomad API'
       logger.error(`[CollectionUpdateService] Failed to check for updates: ${message}`)
       return {
         updates: [],
         checked_at: new Date().toISOString(),
-        error: 'Failed to contact the update service. Please try again later.',
+        error: 'Failed to check for content updates. Please try again later.',
       }
     }
   }
 
   async applyUpdate(
-    update: ResourceUpdateInfo
+    update: ResourceUpdateInfo,
+    options?: { auto?: boolean }
   ): Promise<{ success: boolean; jobId?: string; error?: string }> {
     // Check if a download is already in progress for this URL
     const existingJob = await RunDownloadJob.getByUrl(update.download_url)
@@ -105,7 +120,6 @@ export class CollectionUpdateService {
       timeout: 30000,
       allowedMimeTypes:
         update.resource_type === 'zim' ? ZIM_MIME_TYPES : PMTILES_MIME_TYPES,
-      forceNew: true,
       filetype: update.resource_type,
       title: update.resource_id,
       totalBytes: update.size_bytes,
@@ -113,6 +127,7 @@ export class CollectionUpdateService {
         resource_id: update.resource_id,
         version: update.latest_version,
         collection_ref: null,
+        auto: options?.auto ?? false,
       },
     })
 

admin/app/services/condition_service.ts

@@ -0,0 +1,329 @@
+import db from '@adonisjs/lucid/services/db'
+import logger from '@adonisjs/core/services/logger'
+import { CONDITIONS_FILE } from '../data/conditions.js'
+import { NATURAL_REMEDIES_FILE } from '../data/natural_remedies.js'
+import { HOME_REMEDIES_FILE } from '../data/home_remedies.js'
+import {
+  parseConditionsFile,
+  parseNaturalRemediesFile,
+  findConditionBySlug,
+  toConditionSummary,
+  buildIndicationQuery,
+  orderOtcFirst,
+  remediesForCondition,
+  remediesForFreeText,
+} from '../../util/conditions.js'
+import { PRODUCT_TYPES } from '../../types/drug_reference.js'
+import type {
+  Condition,
+  ConditionSummary,
+  ConditionDrugsResult,
+  NaturalRemedy,
+  NaturalRemediesFile,
+} from '../../types/conditions.js'
+import type { DrugSearchResult } from '../../types/drug_reference.js'
+
+/**
+ * "When to use what" — condition-first service (Phase 1 + Phase 2).
+ *
+ * Resolves a curated condition (or a free-text situation) to the OTC drugs whose
+ * FDA label indications match it, plus the natural remedies (NCCIH) whose curated
+ * condition mapping includes the resolved slug.
+ *
+ * Reuses the Drug Reference indication-search machinery: the same combined-FULLTEXT
+ * index (ft_drug_labels_name_indications), the same MAX(MATCH …) aggregate pattern
+ * required under MySQL 8 ONLY_FULL_GROUP_BY, and the same brand+generic collapse —
+ * then re-buckets results OTC-first.
+ *
+ * Natural remedies are in-memory (no DB table, no migration) — one module-level
+ * parse of NATURAL_REMEDIES_FILE, reused across all requests.
+ */
+
+/**
+ * Module-level merged remedies corpus (fail-soft): the NCCIH herbs plus the
+ * non-herbal home-care measures (CDC/NIH/FDA), each entry tagged with its kind
+ * so the UI can badge them apart. Slugs are disjoint between the two files
+ * (validated at curation time).
+ */
+const HERB_FILE = parseNaturalRemediesFile(NATURAL_REMEDIES_FILE)
+const HOME_FILE = parseNaturalRemediesFile(HOME_REMEDIES_FILE)
+const REMEDIES_FILE: NaturalRemediesFile = {
+  version: HERB_FILE.version,
+  source: HERB_FILE.source,
+  remedies: [
+    ...HERB_FILE.remedies.map((r) => ({ ...r, kind: 'herb' as const })),
+    ...HOME_FILE.remedies.map((r) => ({ ...r, kind: 'self-care' as const })),
+  ],
+}
+
+export class ConditionService {
+  /** Parsed (fail-soft) curated spine — bad entries are dropped, not fatal. */
+  private get spine(): Condition[] {
+    return parseConditionsFile(CONDITIONS_FILE).conditions
+  }
+
+  /** All curated conditions as client-facing summaries (no searchTerms). */
+  listConditions(): ConditionSummary[] {
+    return this.spine.map(toConditionSummary)
+  }
+
+  /**
+   * The full curated natural-remedies list (NCCIH herbs). Small enough to ship
+   * as page props, which lets the Drug Reference search match remedies by name
+   * client-side and offer a "Natural" browse with no extra round trip.
+   */
+  listRemedies(): NaturalRemedy[] {
+    return REMEDIES_FILE.remedies
+  }
+
+  /**
+   * The full curated spine (WITH searchTerms) for server-side matching such as
+   * the drug-detail reverse link (situationsForIndications). Stays server-only —
+   * searchTerms are a search-implementation detail the client never receives.
+   */
+  allConditions(): Condition[] {
+    return this.spine
+  }
+
+  /** Find a curated condition by slug. Returns null when absent. */
+  findCondition(slug: string): Condition | null {
+    return findConditionBySlug(this.spine, slug)
+  }
+
+  /**
+   * Resolve a curated condition (by slug) to its matching OTC drugs and natural
+   * remedies. Returns null when the slug is not in the curated spine so the
+   * controller can 404. Drugs are OTC-only and OTC-first-ordered.
+   */
+  async drugsForSlug(
+    slug: string,
+    limit = 50,
+    opts?: { route?: string; sort?: 'relevance' | 'name' }
+  ): Promise<ConditionDrugsResult | null> {
+    const condition = this.findCondition(slug)
+    if (!condition) return null
+
+    const drugs = await this.searchIndications(condition.searchTerms, limit, opts)
+    const remedies: NaturalRemedy[] = remediesForCondition(REMEDIES_FILE, slug)
+    return { condition: toConditionSummary(condition), drugs, remedies }
+  }
+
+  /**
+   * Resolve a free-text situation (off-list condition) to matching OTC drugs and
+   * natural remedies. Treats the raw query as a single search term. Returns a
+   * synthetic condition summary echoing the query so the UI can render a consistent
+   * header.
+   *
+   * Remedy resolution for free text:
+   *   1. Try to resolve the query to a curated condition slug (exact or substring
+   *      match on slug/label). If found, use remediesForCondition — this is the
+   *      primary path (e.g. "burns" typed free-form still finds aloe/tea-tree).
+   *   2. Secondary fallback: remediesForFreeText searches remedy name/uses by
+   *      substring so an unmapped query ("athlete's foot") can still surface a
+   *      relevant remedy. Results from both paths are unioned and de-duped by slug.
+   */
+  async drugsForFreeText(
+    query: string,
+    limit = 50,
+    opts?: { route?: string; sort?: 'relevance' | 'name' }
+  ): Promise<ConditionDrugsResult> {
+    const trimmed = query.trim()
+    const drugs = trimmed.length > 0 ? await this.searchIndications([trimmed], limit, opts) : []
+
+    // Phase 2: union condition-mapped + free-text-matched remedies.
+    const remedyMap = new Map<string, NaturalRemedy>()
+    if (trimmed.length > 0) {
+      // Primary: resolve to a curated condition slug the same way the client-side
+      // matchSituation helper does (case-insensitive exact/substring on slug+label).
+      const q = trimmed.toLowerCase()
+      const matchedCondition =
+        this.spine.find((c) => c.slug.toLowerCase() === q || c.label.toLowerCase() === q) ??
+        this.spine.find(
+          (c) =>
+            c.label.toLowerCase().includes(q) || c.slug.replace(/-/g, ' ').toLowerCase().includes(q)
+        ) ??
+        null
+      if (matchedCondition) {
+        for (const r of remediesForCondition(REMEDIES_FILE, matchedCondition.slug)) {
+          remedyMap.set(r.slug, r)
+        }
+      }
+      // Secondary: name/uses substring fallback.
+      for (const r of remediesForFreeText(REMEDIES_FILE, trimmed)) {
+        if (!remedyMap.has(r.slug)) remedyMap.set(r.slug, r)
+      }
+    }
+
+    return {
+      condition: { slug: '', label: trimmed, category: 'Search' },
+      drugs,
+      remedies: Array.from(remedyMap.values()),
+    }
+  }
+
+  /**
+   * Core search: FULLTEXT over (searchable_name, indications) for the OR-expanded
+   * searchTerms, OTC-filtered, collapsed by brand+generic, OTC-first-ordered.
+   *
+   * Strategy mirrors DrugReferenceService:
+   *   1. FULLTEXT NATURAL LANGUAGE MODE on the built query (requires the query
+   *      to have a >= 3-char token; innodb_ft_min_token_size = 3).
+   *   2. LIKE fallback when the built query is empty/too short OR FULLTEXT throws
+   *      (e.g. the index is absent) — runs each term as a LIKE clause.
+   * Both paths force product_type = OTC and then orderOtcFirst (a no-op on the
+   * already-OTC set, but kept so a future relaxation of the OTC filter stays
+   * correctly ordered).
+   */
+  async searchIndications(
+    searchTerms: string[],
+    limit = 50,
+    opts?: { route?: string; sort?: 'relevance' | 'name' }
+  ): Promise<DrugSearchResult[]> {
+    const ftQuery = buildIndicationQuery(searchTerms)
+    // A FULLTEXT query needs a token >= innodb_ft_min_token_size (3). If the
+    // longest bare token is shorter, NATURAL LANGUAGE MODE returns nothing, so
+    // go straight to LIKE.
+    const longestToken = ftQuery
+      .replace(/"/g, ' ')
+      .split(/\s+/)
+      .reduce((max, t) => Math.max(max, t.length), 0)
+    const useFulltext = ftQuery.length > 0 && longestToken >= 3
+
+    if (useFulltext) {
+      try {
+        const rows = await this.searchIndicationFulltext(ftQuery, limit, opts)
+        return orderOtcFirst(rows)
+      } catch (err) {
+        logger.warn(
+          `[ConditionService] FULLTEXT indication search failed, falling back to LIKE: ${
+            err instanceof Error ? err.message : String(err)
+          }`
+        )
+      }
+    }
+
+    const rows = await this.searchIndicationLike(searchTerms, limit, opts)
+    return orderOtcFirst(rows)
+  }
+
+  /**
+   * FULLTEXT OTC indication search.
+   *
+   * MATCHes over (searchable_name, indications) — must exactly match the
+   * ft_drug_labels_name_indications index column list. The MAX(MATCH …) wrapper
+   * is LOAD-BEARING: MySQL 8 ONLY_FULL_GROUP_BY rejects a bare MATCH() in SELECT
+   * under GROUP BY; wrapping it in MAX() makes it an aggregate. Copied verbatim
+   * from DrugReferenceService.searchIndicationFulltext, with a fixed OTC filter.
+   */
+  private async searchIndicationFulltext(
+    ftQuery: string,
+    limit: number,
+    opts?: { route?: string; sort?: 'relevance' | 'name' }
+  ): Promise<DrugSearchResult[]> {
+    let sql = `
+      SELECT
+        MIN(id) AS id,
+        brand_name,
+        generic_name,
+        MIN(manufacturer) AS manufacturer,
+        MIN(route) AS route,
+        MIN(product_type) AS product_type,
+        COUNT(*) AS labelCount,
+        MAX(MATCH(searchable_name, indications) AGAINST(? IN NATURAL LANGUAGE MODE)) AS relevance
+      FROM drug_labels
+      WHERE MATCH(searchable_name, indications) AGAINST(? IN NATURAL LANGUAGE MODE)
+        AND product_type = ?
+    `
+    const bindings: unknown[] = [ftQuery, ftQuery, PRODUCT_TYPES.OTC]
+
+    if (opts?.route) {
+      sql += ' AND route LIKE ?'
+      bindings.push(`%${opts.route.toUpperCase()}%`)
+    }
+
+    sql += `
+      GROUP BY brand_name, generic_name
+      ORDER BY ${opts?.sort === 'name' ? 'COALESCE(brand_name, generic_name) ASC' : 'relevance DESC'}
+      LIMIT ?
+    `
+    bindings.push(limit)
+
+    const rows = await db.rawQuery(sql, bindings)
+    return this.mapSearchRows(rows[0])
+  }
+
+  /**
+   * LIKE OTC indication fallback (FULLTEXT unavailable or query too short).
+   *
+   * ORs a LIKE clause per search term over `indications` so any synonym can
+   * surface a match. OTC-filtered and collapsed identically to the FULLTEXT path.
+   */
+  private async searchIndicationLike(
+    searchTerms: string[],
+    limit: number,
+    opts?: { route?: string; sort?: 'relevance' | 'name' }
+  ): Promise<DrugSearchResult[]> {
+    const terms = searchTerms.map((t) => t.trim()).filter((t) => t.length > 0)
+    if (terms.length === 0) return []
+
+    const likeClauses = terms.map(() => 'indications LIKE ?').join(' OR ')
+    let sql = `
+      SELECT
+        MIN(id) AS id,
+        brand_name,
+        generic_name,
+        MIN(manufacturer) AS manufacturer,
+        MIN(route) AS route,
+        MIN(product_type) AS product_type,
+        COUNT(*) AS labelCount
+      FROM drug_labels
+      WHERE (${likeClauses})
+        AND product_type = ?
+    `
+    const bindings: unknown[] = [...terms.map((t) => `%${t}%`), PRODUCT_TYPES.OTC]
+
+    if (opts?.route) {
+      sql += ' AND route LIKE ?'
+      bindings.push(`%${opts.route.toUpperCase()}%`)
+    }
+
+    sql += `
+      GROUP BY brand_name, generic_name
+      ORDER BY ${opts?.sort === 'name' ? 'COALESCE(brand_name, generic_name) ASC' : 'brand_name ASC'}
+      LIMIT ?
+    `
+    bindings.push(limit)
+
+    const rows = await db.rawQuery(sql, bindings)
+    return this.mapSearchRows(rows[0])
+  }
+
+  /** Map raw collapsed rows to DrugSearchResult — mirrors DrugReferenceService. */
+  private mapSearchRows(rows: any[]): DrugSearchResult[] {
+    if (!Array.isArray(rows)) return []
+    return rows.map((row) => ({
+      id: Number(row.id),
+      brand_name: row.brand_name ?? null,
+      generic_name: row.generic_name ?? null,
+      manufacturer: row.manufacturer ?? null,
+      route: row.route ?? null,
+      product_type: row.product_type ?? null,
+      labelCount: Number(row.labelCount ?? row.labelcount ?? 1),
+    }))
+  }
+
+  /**
+   * Current drug_labels row count — drives the index page empty-state (no data →
+   * point the user to Drug Reference to download first). Mirrors
+   * DrugReferenceService.rowCount; returns 0 on any error.
+   */
+  async drugRowCount(): Promise<number> {
+    try {
+      const result = await db.rawQuery('SELECT COUNT(*) AS cnt FROM drug_labels')
+      const rows = result[0] as Array<{ cnt: number | string }>
+      return Number(rows[0]?.cnt ?? 0)
+    } catch {
+      return 0
+    }
+  }
+}

admin/app/services/container_registry_service.ts

@@ -139,11 +139,17 @@ export class ContainerRegistryService {
         allTags.push(...data.tags)
       }
 
-      // Handle pagination via Link header
+      // Handle pagination via Link header. Per the OCI/Docker registry spec the next-page
+      // URL is relative (e.g. "/v2/<repo>/tags/list?last=<tag>&n=1000"), so it must be
+      // resolved against the registry origin before re-fetching — assigning the raw relative
+      // path to `url` makes fetch() throw "Failed to parse URL". This silently broke update
+      // checks for any repo with >1000 tags (e.g. ollama/ollama, filebrowser/filebrowser),
+      // which is the root cause of #945. new URL(relative, base) also passes absolute
+      // next-URLs through unchanged, so it's safe for registries that return those.
       const linkHeader = response.headers.get('link')
       if (linkHeader) {
         const match = linkHeader.match(/<([^>]+)>;\s*rel="next"/)
-        url = match ? match[1] : ''
+        url = match ? new URL(match[1], `https://${parsed.registry}`).toString() : ''
       } else {
         url = ''
       }
@@ -200,6 +206,66 @@ export class ContainerRegistryService {
     }
   }
 
+  /**
+   * Estimate the compressed download size (in bytes) of an image tag for the
+   * given host architecture by summing its layer sizes from the manifest.
+   *
+   * Resolves a multi-arch manifest list/index down to the platform-specific
+   * manifest before summing `layers[].size`. Returns null on any failure so
+   * callers can fall back to a conservative fixed threshold rather than
+   * silently skipping a disk pre-flight check.
+   */
+  async getImageDownloadSize(
+    parsed: ParsedImageReference,
+    tag: string,
+    hostArch: string
+  ): Promise<number | null> {
+    try {
+      const token = await this.getToken(parsed.registry, parsed.fullName)
+      const manifestAccept = [
+        'application/vnd.oci.image.index.v1+json',
+        'application/vnd.docker.distribution.manifest.list.v2+json',
+        'application/vnd.oci.image.manifest.v1+json',
+        'application/vnd.docker.distribution.manifest.v2+json',
+      ].join(', ')
+
+      const fetchManifest = async (ref: string) =>
+        this.fetchWithRetry(`https://${parsed.registry}/v2/${parsed.fullName}/manifests/${ref}`, {
+          headers: { Authorization: `Bearer ${token}`, Accept: manifestAccept },
+        })
+
+      const topRes = await fetchManifest(tag)
+      if (!topRes.ok) return null
+
+      let manifest = (await topRes.json()) as {
+        mediaType?: string
+        layers?: Array<{ size?: number }>
+        manifests?: Array<{ digest?: string; platform?: { architecture?: string } }>
+      }
+
+      // Multi-arch manifest list/index — resolve to the host-arch child manifest.
+      if (manifest.manifests?.length) {
+        const match =
+          manifest.manifests.find((m) => m.platform?.architecture === hostArch) ||
+          manifest.manifests[0]
+        if (!match?.digest) return null
+
+        const childRes = await fetchManifest(match.digest)
+        if (!childRes.ok) return null
+        manifest = (await childRes.json()) as { layers?: Array<{ size?: number }> }
+      }
+
+      if (!manifest.layers?.length) return null
+
+      return manifest.layers.reduce((total, layer) => total + (layer.size || 0), 0)
+    } catch (error) {
+      logger.warn(
+        `[ContainerRegistryService] Failed to get image size for ${parsed.fullName}:${tag}: ${error.message}`
+      )
+      return null
+    }
+  }
+
   /**
    * Extract the source repository URL from an image's OCI labels.
    * Uses the standardized `org.opencontainers.image.source` label.

admin/app/services/content_auto_update_service.ts

@@ -0,0 +1,599 @@
+import logger from '@adonisjs/core/services/logger'
+import { DateTime } from 'luxon'
+import KVStore from '#models/kv_store'
+import InstalledResource from '#models/installed_resource'
+import { DownloadService } from '#services/download_service'
+import { CollectionUpdateService } from '#services/collection_update_service'
+import { DrugReferenceService } from '#services/drug_reference_service'
+import {
+  KiwixCatalogService,
+  reconcileResourceUpdateState,
+  type CatalogResult,
+} from '#services/kiwix_catalog_service'
+import { isWithinWindow, parseWindowMinutes } from '../utils/update_window.js'
+import { recordResourceUpdateFailure } from '../utils/content_auto_update_backoff.js'
+import type { Blocker, PreflightResult } from '../utils/image_disk_preflight.js'
+
+/**
+ * Content auto-update is opt-in via a single global master switch and runs on
+ * its OWN window + per-window data cap (deliberately separate from the core/app
+ * `autoUpdate.*` window, since ZIM downloads are multi-GB and bandwidth
+ * sensitive). Defaults err toward an overnight window with no cap; the UI
+ * strongly recommends setting a cap.
+ */
+const DEFAULT_WINDOW_START = '02:00'
+const DEFAULT_WINDOW_END = '05:00'
+const DEFAULT_COOLOFF_HOURS = 72
+
+/** Whole-feature failures (e.g. catalog unreachable) before it self-disables. */
+const MAX_FEATURE_FAILURES = 3
+
+export interface ContentAutoUpdateConfig {
+  /** Global master switch (`contentAutoUpdate.enabled`). */
+  enabled: boolean
+  windowStart: string
+  windowEnd: string
+  cooloffHours: number
+  /** Max NEW bytes initiated per window instance. 0 = unlimited. */
+  maxBytesPerWindow: number
+}
+
+/** Per-resource eligibility verdict (drives both selection and the status UI). */
+export interface ContentEligibility {
+  eligible: boolean
+  reason: string
+  cooloffRemainingHours: number | null
+}
+
+/** An eligible resource paired with the catalog facts needed to download it. */
+export interface ContentCandidate {
+  resource: InstalledResource
+  version: string
+  download_url: string
+  size_bytes: number
+  installed_at: DateTime
+}
+
+/** Outcome of the cap-bounded greedy selection. */
+export interface ContentSelection {
+  selected: ContentCandidate[]
+  /** Single files larger than the whole cap — never auto-started (manual only). */
+  skippedOversize: ContentCandidate[]
+  /** Fit the cap but not this window's remaining budget — retried next window. */
+  deferred: ContentCandidate[]
+}
+
+export interface ContentAutoUpdateResourceStatus {
+  resource_id: string
+  resource_type: 'zim' | 'map'
+  current_version: string
+  available_update_version: string | null
+  size_bytes: number | null
+  eligible: boolean
+  reason: string
+  cooloff_remaining_hours: number | null
+  exceeds_cap: boolean
+  consecutive_failures: number
+  auto_disabled_reason: string | null
+}
+
+export interface ContentAutoUpdateStatus extends ContentAutoUpdateConfig {
+  withinWindow: boolean
+  windowBytesUsed: number
+  lastAttemptAt: string | null
+  lastResult: string | null
+  lastError: string | null
+  autoDisabledReason: string | null
+  resources: ContentAutoUpdateResourceStatus[]
+}
+
+/**
+ * Decision + safety layer for automatic content (ZIM/map) updates. This is the
+ * content-side counterpart to {@link AppAutoUpdateService}: it decides *whether*
+ * each installed resource with an available update should be downloaded now
+ * (master switch on + in the content window + past cool-off + within the data
+ * cap) and then drives the existing manual download path
+ * ({@link CollectionUpdateService.applyUpdate} → {@link RunDownloadJob}).
+ *
+ * It never installs synchronously — it dispatches resumable download jobs and
+ * lets the existing job-completion path advance the installed version and
+ * rebuild the Kiwix library.
+ */
+export class ContentAutoUpdateService {
+  constructor(
+    private downloadService: DownloadService,
+    private catalog: KiwixCatalogService = new KiwixCatalogService(),
+    private collectionUpdateService: CollectionUpdateService = new CollectionUpdateService()
+  ) {}
+
+  /** Read the master switch plus the content-specific window/cool-off/cap. */
+  async getConfig(): Promise<ContentAutoUpdateConfig> {
+    const [enabled, windowStart, windowEnd, cooloffHours, maxBytes] = await Promise.all([
+      KVStore.getValue('contentAutoUpdate.enabled'),
+      KVStore.getValue('contentAutoUpdate.windowStart'),
+      KVStore.getValue('contentAutoUpdate.windowEnd'),
+      KVStore.getValue('contentAutoUpdate.cooloffHours'),
+      KVStore.getValue('contentAutoUpdate.maxBytesPerWindow'),
+    ])
+
+    const parsedCooloff = Number(cooloffHours)
+    const parsedCap = Number(maxBytes)
+    return {
+      enabled: enabled ?? false,
+      windowStart: windowStart || DEFAULT_WINDOW_START,
+      windowEnd: windowEnd || DEFAULT_WINDOW_END,
+      // `Number(null) === 0`, so an unset value must fall through to the default
+      // rather than silently resolving to a zero cool-off. An explicit 0 is honored.
+      cooloffHours:
+        cooloffHours !== null && Number.isFinite(parsedCooloff) && parsedCooloff >= 0
+          ? parsedCooloff
+          : DEFAULT_COOLOFF_HOURS,
+      maxBytesPerWindow:
+        maxBytes !== null && Number.isFinite(parsedCap) && parsedCap >= 0 ? parsedCap : 0,
+    }
+  }
+
+  /**
+   * Pure per-resource eligibility verdict. A resource is eligible when it has a
+   * detected newer version, is not self-disabled, and is past its cool-off
+   * (measured from first-detected). Version comparison is a lexicographic
+   * compare of the YYYY-MM stamps, which sorts chronologically.
+   */
+  resourceEligibility(
+    resource: InstalledResource,
+    cooloffHours: number,
+    now: DateTime
+  ): ContentEligibility {
+    if (!resource.available_update_version) {
+      return { eligible: false, reason: 'Up to date', cooloffRemainingHours: null }
+    }
+    if (resource.auto_update_disabled_reason) {
+      return {
+        eligible: false,
+        reason: 'Auto-update disabled after repeated failures',
+        cooloffRemainingHours: null,
+      }
+    }
+    if (!(resource.available_update_version > resource.version)) {
+      return { eligible: false, reason: 'Up to date', cooloffRemainingHours: null }
+    }
+    if (!resource.available_update_first_seen_at) {
+      return { eligible: false, reason: 'Cool-off pending', cooloffRemainingHours: cooloffHours }
+    }
+
+    const ageHours = now.diff(resource.available_update_first_seen_at, 'hours').hours
+    const remaining = cooloffHours - ageHours
+    if (remaining > 0) {
+      const rounded = Math.ceil(remaining)
+      return {
+        eligible: false,
+        reason: `In cool-off (${rounded}h remaining)`,
+        cooloffRemainingHours: rounded,
+      }
+    }
+
+    return {
+      eligible: true,
+      reason: `Eligible → ${resource.available_update_version}`,
+      cooloffRemainingHours: 0,
+    }
+  }
+
+  /**
+   * Pure cap-bounded greedy selection. Oldest-installed first (stale content is
+   * prioritized), tie-broken smallest-first for predictability.
+   *
+   * - size unknown (0) → deferred (can't budget safely)
+   * - size > the WHOLE cap → skippedOversize (never auto-started; manual only)
+   * - size ≤ remaining budget → selected
+   * - otherwise → deferred (fits the cap, not this window)
+   */
+  selectUnderCap(
+    candidates: ContentCandidate[],
+    capBytes: number,
+    usedBytes: number
+  ): ContentSelection {
+    const cap = capBytes > 0 ? capBytes : Number.POSITIVE_INFINITY
+    let remaining = Math.max(0, cap - usedBytes)
+
+    const selected: ContentCandidate[] = []
+    const skippedOversize: ContentCandidate[] = []
+    const deferred: ContentCandidate[] = []
+
+    const ordered = [...candidates].sort((a, b) => {
+      const at = a.installed_at?.toMillis?.() ?? 0
+      const bt = b.installed_at?.toMillis?.() ?? 0
+      if (at !== bt) return at - bt
+      return a.size_bytes - b.size_bytes
+    })
+
+    for (const candidate of ordered) {
+      if (candidate.size_bytes <= 0) {
+        deferred.push(candidate)
+      } else if (candidate.size_bytes > cap) {
+        skippedOversize.push(candidate)
+      } else if (candidate.size_bytes <= remaining) {
+        selected.push(candidate)
+        remaining -= candidate.size_bytes
+      } else {
+        deferred.push(candidate)
+      }
+    }
+
+    return { selected, skippedOversize, deferred }
+  }
+
+  /**
+   * Run-wide pre-flight: never auto-update content while ANY download is already
+   * running. Because content downloads are multi-GB and resumable, an in-flight
+   * download from a prior window naturally blocks new starts here — exactly the
+   * "let in-flight finish, don't start new" behavior we want. Transient → `skip`.
+   */
+  async runGlobalPreflight(): Promise<PreflightResult> {
+    const blockers: Blocker[] = []
+    try {
+      const downloads = await this.downloadService.listDownloadJobs()
+      const active = downloads.filter(
+        (d) => !!d.status && ['waiting', 'active', 'delayed'].includes(d.status)
+      )
+      if (active.length > 0) {
+        blockers.push({ reason: `${active.length} download(s) in progress`, severity: 'skip' })
+      }
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error)
+      logger.warn(`[ContentAutoUpdateService] Could not check active downloads: ${message}`)
+    }
+    return { ok: blockers.length === 0, blockers }
+  }
+
+  /**
+   * Entry point invoked by ContentAutoUpdateJob. Gates on the master switch +
+   * window, runs the local catalog check, then downloads as many eligible
+   * resources as fit under the per-window data cap.
+   */
+  async attempt(): Promise<{ started: number; reason: string }> {
+    const config = await this.getConfig()
+    const now = DateTime.now()
+
+    if (!config.enabled) {
+      return { started: 0, reason: 'Content auto-update is disabled' }
+    }
+    if (!isWithinWindow(config.windowStart, config.windowEnd, now)) {
+      const reason = `Outside update window (${config.windowStart}-${config.windowEnd})`
+      await this.recordRun(reason)
+      return { started: 0, reason }
+    }
+
+    try {
+      // Reset the per-window budget once per window instance (the cron fires
+      // hourly but a window can span several hours).
+      await this.maybeResetWindowBudget(config, now)
+
+      // Local catalog check + persist available-update state for every resource.
+      // ZIM/map catalog path only — `dataset` resources are excluded here because
+      // their freshness key (openFDA `export_date`) and apply path (the drug
+      // download/ingest chain) don't fit the catalog-version model. They refresh
+      // via `attemptDrugDataset()`, which the same content-update job runs under
+      // the same master switch + window.
+      const installed = await InstalledResource.query().whereNot('resource_type', 'dataset')
+      const latestByKey = await this.catalog.getLatestForResources(
+        // `dataset` rows are filtered out above, so the type narrows to ZIM/map.
+        installed.map((r) => ({
+          resource_id: r.resource_id,
+          resource_type: r.resource_type as 'zim' | 'map',
+        }))
+      )
+      for (const resource of installed) {
+        const latest = latestByKey.get(`${resource.resource_type}:${resource.resource_id}`) ?? null
+        await reconcileResourceUpdateState(resource, latest, now)
+      }
+
+      const eligible = installed.filter(
+        (r) => this.resourceEligibility(r, config.cooloffHours, now).eligible
+      )
+      if (eligible.length === 0) {
+        await this.recordFeatureSuccess()
+        const reason = 'No eligible content updates'
+        await this.recordRun(reason)
+        return { started: 0, reason }
+      }
+
+      const global = await this.runGlobalPreflight()
+      if (!global.ok) {
+        await this.recordFeatureSuccess()
+        const reason = `Pre-flight blocked: ${global.blockers.map((b) => b.reason).join('; ')}`
+        await this.recordRun(reason)
+        return { started: 0, reason }
+      }
+
+      const candidates: ContentCandidate[] = eligible.map((r) => {
+        const latest = latestByKey.get(`${r.resource_type}:${r.resource_id}`) as CatalogResult
+        return {
+          resource: r,
+          version: latest.version,
+          download_url: latest.download_url,
+          size_bytes: r.available_update_size_bytes ?? latest.size_bytes ?? 0,
+          installed_at: r.installed_at,
+        }
+      })
+
+      const usedBytes = await this.getWindowBytesUsed()
+      const { selected, skippedOversize, deferred } = this.selectUnderCap(
+        candidates,
+        config.maxBytesPerWindow,
+        usedBytes
+      )
+
+      let started = 0
+      let failed = 0
+      let initiatedBytes = 0
+      for (const candidate of selected) {
+        const result = await this.collectionUpdateService.applyUpdate(
+          {
+            resource_id: candidate.resource.resource_id,
+            // `dataset` rows are filtered out of `installed` above, so ZIM/map.
+            resource_type: candidate.resource.resource_type as 'zim' | 'map',
+            installed_version: candidate.resource.version,
+            latest_version: candidate.version,
+            download_url: candidate.download_url,
+            size_bytes: candidate.size_bytes || undefined,
+          },
+          { auto: true }
+        )
+
+        if (result.success) {
+          // Success is NOT recorded here: applyUpdate only enqueues a resumable
+          // download. The per-resource backoff is cleared once the download
+          // actually completes (RunDownloadJob.onComplete) and incremented when it
+          // fails terminally (the worker `failed` handler). Recording success on
+          // dispatch would reset the counter every window and defeat self-disable.
+          initiatedBytes += candidate.size_bytes
+          started++
+          logger.info(
+            `[ContentAutoUpdateService] Started ${candidate.resource.resource_id} → ${candidate.version}`
+          )
+        } else {
+          // A failure to even enqueue is a genuine auto-update failure; no job runs,
+          // so no terminal `failed` event will follow — count it here.
+          await recordResourceUpdateFailure(candidate.resource, result.error ?? 'dispatch failed')
+          failed++
+        }
+      }
+
+      if (initiatedBytes > 0) {
+        await this.addWindowBytesUsed(initiatedBytes)
+      }
+
+      const parts = [`${started} started`]
+      if (failed) parts.push(`${failed} failed`)
+      if (skippedOversize.length) parts.push(`${skippedOversize.length} skipped (exceeds cap)`)
+      if (deferred.length) parts.push(`${deferred.length} deferred (over budget)`)
+      const reason = parts.join(', ')
+
+      await this.recordFeatureSuccess()
+      await this.recordRun(reason)
+      logger.info(`[ContentAutoUpdateService] Run complete: ${reason}`)
+      return { started, reason }
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error)
+      await this.recordFeatureFailure(message)
+      await this.recordRun(`Failed: ${message}`)
+      logger.error(`[ContentAutoUpdateService] Run failed: ${message}`)
+      return { started: 0, reason: `Failed: ${message}` }
+    }
+  }
+
+  /**
+   * Freshness pass for the FDA drug dataset (`resource_type` 'dataset'), run by
+   * the same hourly ContentAutoUpdateJob as the ZIM/map `attempt()` and gated on
+   * the SAME master switch + window. The dataset's apply path differs from the
+   * catalog loop (openFDA `export_date` + the drug download/ingest chain rather
+   * than an InstalledResource catalog-version row), so it runs as its own step
+   * instead of through `selectUnderCap` — but sharing `contentAutoUpdate.*` means
+   * it never updates while content auto-update is off, and refreshes alongside
+   * ZIMs and maps when it is on. Isolated so a drug-side failure can't affect the
+   * catalog run.
+   */
+  async attemptDrugDataset(): Promise<{ started: number; reason: string }> {
+    const config = await this.getConfig()
+    if (!config.enabled) {
+      return { started: 0, reason: 'Content auto-update is disabled' }
+    }
+    if (!isWithinWindow(config.windowStart, config.windowEnd, DateTime.now())) {
+      return {
+        started: 0,
+        reason: `Outside update window (${config.windowStart}-${config.windowEnd})`,
+      }
+    }
+
+    try {
+      const result = await new DrugReferenceService().attemptAutoUpdate()
+      return { started: result.started ? 1 : 0, reason: `drug dataset: ${result.reason}` }
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error)
+      logger.warn(`[ContentAutoUpdateService] Drug dataset freshness check failed: ${message}`)
+      return { started: 0, reason: `drug dataset check failed: ${message}` }
+    }
+  }
+
+  /**
+   * Evaluate what the next run *would* do, without hitting the network,
+   * persisting state, or dispatching anything. Operates on the available-update
+   * state already persisted by the last check (manual or auto), so run a "Check
+   * for Content Updates" first if you want fresh catalog data. Used by the
+   * `content-auto-update:dry-run` command.
+   */
+  async dryRun(
+    overrides: {
+      now?: DateTime
+      forceEnabled?: boolean
+      cooloffHours?: number
+      windowStart?: string
+      windowEnd?: string
+      maxBytesPerWindow?: number
+      windowBytesUsed?: number
+    } = {}
+  ): Promise<{
+    enabled: boolean
+    withinWindow: boolean
+    config: ContentAutoUpdateConfig
+    eligibleCount: number
+    selection: ContentSelection
+  }> {
+    const base = await this.getConfig()
+    const config: ContentAutoUpdateConfig = {
+      enabled: overrides.forceEnabled ? true : base.enabled,
+      windowStart: overrides.windowStart ?? base.windowStart,
+      windowEnd: overrides.windowEnd ?? base.windowEnd,
+      cooloffHours: overrides.cooloffHours ?? base.cooloffHours,
+      maxBytesPerWindow: overrides.maxBytesPerWindow ?? base.maxBytesPerWindow,
+    }
+    const now = overrides.now ?? DateTime.now()
+    const withinWindow = isWithinWindow(config.windowStart, config.windowEnd, now)
+
+    const pending = await InstalledResource.query()
+      .whereNotNull('available_update_version')
+      .whereNot('resource_type', 'dataset')
+    const eligible = pending.filter(
+      (r) => this.resourceEligibility(r, config.cooloffHours, now).eligible
+    )
+    const candidates: ContentCandidate[] = eligible.map((r) => ({
+      resource: r,
+      version: r.available_update_version!,
+      download_url: '(dry-run)',
+      size_bytes: r.available_update_size_bytes ?? 0,
+      installed_at: r.installed_at,
+    }))
+
+    const usedBytes = overrides.windowBytesUsed ?? (await this.getWindowBytesUsed())
+    const selection = this.selectUnderCap(candidates, config.maxBytesPerWindow, usedBytes)
+
+    return {
+      enabled: config.enabled,
+      withinWindow,
+      config,
+      eligibleCount: eligible.length,
+      selection,
+    }
+  }
+
+  // ── Per-window budget ─────────────────────────────────────────────────────────
+
+  /** Most-recent window-open boundary as an absolute timestamp (handles wrap). */
+  windowStartBoundary(windowStart: string, now: DateTime): DateTime {
+    const minutes = parseWindowMinutes(windowStart) ?? 0
+    const todayStart = now.startOf('day').plus({ minutes })
+    return now >= todayStart ? todayStart : todayStart.minus({ days: 1 })
+  }
+
+  /** Reset the window budget exactly once per entry into the window. */
+  private async maybeResetWindowBudget(
+    config: ContentAutoUpdateConfig,
+    now: DateTime
+  ): Promise<void> {
+    const boundary = this.windowStartBoundary(config.windowStart, now)
+    const resetAtRaw = await KVStore.getValue('contentAutoUpdate.windowResetAt')
+    const resetAt = resetAtRaw ? DateTime.fromISO(resetAtRaw) : null
+    if (!resetAt || !resetAt.isValid || resetAt < boundary) {
+      await KVStore.setValue('contentAutoUpdate.windowBytesUsed', '0')
+      await KVStore.setValue('contentAutoUpdate.windowResetAt', now.toISO()!)
+    }
+  }
+
+  private async getWindowBytesUsed(): Promise<number> {
+    const raw = await KVStore.getValue('contentAutoUpdate.windowBytesUsed')
+    const num = Number(raw)
+    return Number.isFinite(num) && num > 0 ? num : 0
+  }
+
+  private async addWindowBytesUsed(bytes: number): Promise<void> {
+    const used = await this.getWindowBytesUsed()
+    await KVStore.setValue('contentAutoUpdate.windowBytesUsed', String(used + bytes))
+  }
+
+  // ── Backoff + run recording ───────────────────────────────────────────────────
+  // Per-resource backoff lives in ../utils/content_auto_update_backoff.ts so the
+  // job-completion path and the worker `failed` handler can share it without an
+  // import cycle. The feature-level backoff below stays here.
+
+  /** Clear the feature-level backoff after a clean run. */
+  private async recordFeatureSuccess(): Promise<void> {
+    await KVStore.setValue('contentAutoUpdate.consecutiveFailures', '0')
+    await KVStore.clearValue('contentAutoUpdate.lastError')
+  }
+
+  /** Record a whole-feature failure and self-disable the feature at the threshold. */
+  private async recordFeatureFailure(reason: string): Promise<void> {
+    const raw = await KVStore.getValue('contentAutoUpdate.consecutiveFailures')
+    const failures = (Number(raw) || 0) + 1
+    await KVStore.setValue('contentAutoUpdate.consecutiveFailures', String(failures))
+    await KVStore.setValue('contentAutoUpdate.lastError', reason)
+    if (failures >= MAX_FEATURE_FAILURES) {
+      await KVStore.setValue('contentAutoUpdate.enabled', false)
+      await KVStore.setValue(
+        'contentAutoUpdate.autoDisabledReason',
+        `Content auto-update disabled after ${failures} consecutive failures. Last error: ${reason}`
+      )
+      logger.error(
+        `[ContentAutoUpdateService] Feature auto-disabled after ${failures} consecutive failures`
+      )
+    }
+  }
+
+  private async recordRun(reason: string): Promise<void> {
+    await KVStore.setValue('contentAutoUpdate.lastAttemptAt', DateTime.now().toISO()!)
+    await KVStore.setValue('contentAutoUpdate.lastResult', reason)
+  }
+
+  // ── Status snapshot ───────────────────────────────────────────────────────────
+
+  /** Full state snapshot for the settings UI (resources with pending updates). */
+  async getStatus(): Promise<ContentAutoUpdateStatus> {
+    const config = await this.getConfig()
+    const now = DateTime.now()
+
+    const pending = await InstalledResource.query()
+      .whereNotNull('available_update_version')
+      .whereNot('resource_type', 'dataset')
+    const resources: ContentAutoUpdateResourceStatus[] = pending.map((resource) => {
+      const verdict = this.resourceEligibility(resource, config.cooloffHours, now)
+      const size = resource.available_update_size_bytes ?? null
+      const exceedsCap =
+        config.maxBytesPerWindow > 0 && size !== null && size > config.maxBytesPerWindow
+      return {
+        resource_id: resource.resource_id,
+        // `dataset` rows are filtered out of `pending` above, so ZIM/map.
+        resource_type: resource.resource_type as 'zim' | 'map',
+        current_version: resource.version,
+        available_update_version: resource.available_update_version,
+        size_bytes: size,
+        eligible: verdict.eligible && !exceedsCap,
+        reason: exceedsCap ? 'Exceeds data cap — update manually' : verdict.reason,
+        cooloff_remaining_hours: verdict.cooloffRemainingHours,
+        exceeds_cap: exceedsCap,
+        consecutive_failures: resource.auto_update_consecutive_failures || 0,
+        auto_disabled_reason: resource.auto_update_disabled_reason,
+      }
+    })
+
+    const [lastAttemptAt, lastResult, lastError, autoDisabledReason, windowBytesUsed] =
+      await Promise.all([
+        KVStore.getValue('contentAutoUpdate.lastAttemptAt'),
+        KVStore.getValue('contentAutoUpdate.lastResult'),
+        KVStore.getValue('contentAutoUpdate.lastError'),
+        KVStore.getValue('contentAutoUpdate.autoDisabledReason'),
+        this.getWindowBytesUsed(),
+      ])
+
+    return {
+      ...config,
+      withinWindow: isWithinWindow(config.windowStart, config.windowEnd, now),
+      windowBytesUsed,
+      lastAttemptAt: lastAttemptAt || null,
+      lastResult: lastResult || null,
+      lastError: lastError || null,
+      autoDisabledReason: autoDisabledReason || null,
+      resources,
+    }
+  }
+}

admin/app/services/creator_pack_service.ts

@@ -0,0 +1,177 @@
+import env from '#start/env'
+import logger from '@adonisjs/core/services/logger'
+import { join } from 'node:path'
+import { DockerService } from '#services/docker_service'
+import { ZimService } from '#services/zim_service'
+import { CollectionManifestService } from '#services/collection_manifest_service'
+import { RunDownloadJob } from '#jobs/run_download_job'
+import { ZIM_STORAGE_PATH } from '../utils/fs.js'
+import { SERVICE_NAMES } from '../../constants/service_names.js'
+import type { CreatorPacksSpec, CreatorPackWithStatus } from '../../types/collections.js'
+
+// ZIMs report as octet-stream from the Worker (Content-Type is set explicitly to
+// application/octet-stream in worker/src/index.js). Mirror the ZimService allowlist.
+const ZIM_MIME_TYPES = ['application/x-zim', 'application/x-openzim', 'application/octet-stream']
+
+// Default entitlement Worker origin. Overridable via CREATOR_PACKS_WORKER_BASE so
+// we can move to a branded packs.projectnomad.us domain without a code change.
+const DEFAULT_WORKER_BASE = 'https://nomad-packs-worker.chris-556.workers.dev'
+
+/**
+ * Outcome of an install request. Discriminated on `code` so the controller can
+ * map each case to an HTTP status without string-matching thrown messages.
+ * (See feedback: typed failure codes over ad-hoc Error parsing.)
+ */
+export type InstallPackResult =
+  | { code: 'dispatched'; filename: string }
+  | { code: 'already_installed' }
+  | { code: 'already_downloading' }
+  | { code: 'not_found' }
+  | { code: 'not_configured' }
+
+export type UninstallPackResult =
+  | { code: 'uninstalled'; filename: string }
+  | { code: 'not_installed' }
+
+/**
+ * Creator Packs install rail. Diverges from the curated-collections rail in
+ * exactly one way: instead of a static manifest URL, the ZIM is fetched from the
+ * entitlement Worker with a bearer `Authorization` header. From RunDownloadJob
+ * onward everything is the shared rail (resumable download → InstalledResource →
+ * Kiwix rebuildFromDisk). Video ZIMs opt out of KB embedding via skip_embedding.
+ */
+export class CreatorPackService {
+  private dockerService: DockerService
+  private manifestService: CollectionManifestService
+
+  constructor(dockerService?: DockerService, manifestService?: CollectionManifestService) {
+    this.dockerService = dockerService ?? new DockerService()
+    this.manifestService = manifestService ?? new CollectionManifestService()
+  }
+
+  private get workerBase(): string {
+    return (env.get('CREATOR_PACKS_WORKER_BASE') || DEFAULT_WORKER_BASE).replace(/\/+$/, '')
+  }
+
+  /**
+   * Whether this build can actually install packs — i.e. the release-injected
+   * app key is present. Forks built from source have no key; the UI uses this to
+   * HIDE the Creator Packs surfaces entirely rather than show install buttons
+   * that would 503. The public catalog still lists, but nothing is installable.
+   */
+  isConfigured(): boolean {
+    return !!env.get('CREATOR_PACKS_APP_KEY')
+  }
+
+  /** Stable, per-version download URL. Not signed — the gate is the auth header. */
+  packUrl(id: string, version: string): string {
+    return `${this.workerBase}/packs/${id}_${version}.zim`
+  }
+
+  async listPacksWithStatus(): Promise<CreatorPackWithStatus[]> {
+    return this.manifestService.getCreatorPacksWithStatus()
+  }
+
+  /**
+   * Install a pack by catalog id: ensure Kiwix is present (auto-install if not),
+   * then dispatch the gated ZIM download onto the shared rail.
+   */
+  async installPack(packId: string): Promise<InstallPackResult> {
+    const appKey = env.get('CREATOR_PACKS_APP_KEY')
+    if (!appKey) {
+      logger.error('[CreatorPackService] CREATOR_PACKS_APP_KEY is not set; cannot install packs')
+      return { code: 'not_configured' }
+    }
+
+    const spec = await this.manifestService.getSpecWithFallback<CreatorPacksSpec>('creator_packs')
+    const pack = spec?.packs.find((p) => p.id === packId)
+    if (!pack) {
+      return { code: 'not_found' }
+    }
+
+    const { default: InstalledResource } = await import('#models/installed_resource')
+    const existing = await InstalledResource.query()
+      .where('resource_type', 'zim')
+      .where('resource_id', pack.resource_id)
+      .first()
+    if (existing && existing.version === pack.version) {
+      return { code: 'already_installed' }
+    }
+
+    const url = this.packUrl(pack.id, pack.version)
+    if (await RunDownloadJob.getActiveByUrl(url)) {
+      return { code: 'already_downloading' }
+    }
+
+    // Kiwix is a hard prerequisite — it's what serves the pack. Auto-install it if
+    // absent (its own preinstall bootstraps a mini-wiki ZIM so the container can
+    // start). Best-effort: if the install can't be kicked off we still download the
+    // pack so the bytes are on disk; Kiwix hot-reloads it once installed.
+    await this.ensureKiwixInstalled()
+
+    const filename = `${pack.id}_${pack.version}.zim`
+    const filepath = join(process.cwd(), ZIM_STORAGE_PATH, filename)
+
+    await RunDownloadJob.dispatch({
+      url,
+      filepath,
+      timeout: 30000,
+      allowedMimeTypes: ZIM_MIME_TYPES,
+      filetype: 'zim',
+      title: pack.name,
+      totalBytes: pack.size_mb ? pack.size_mb * 1024 * 1024 : undefined,
+      requestHeaders: { Authorization: `Bearer ${appKey}` },
+      resourceMetadata: {
+        resource_id: pack.resource_id,
+        version: pack.version,
+        collection_ref: 'creator-packs',
+        skip_embedding: true,
+      },
+    })
+
+    logger.info(`[CreatorPackService] Dispatched download for pack ${pack.id} (${filename})`)
+    return { code: 'dispatched', filename }
+  }
+
+  /**
+   * Uninstall an installed pack: delete the ZIM (which via ZimService.delete also
+   * removes it from the Kiwix library and clears its InstalledResource row).
+   * Uses the INSTALLED version (not the catalog's) so we remove the file that's
+   * actually on disk even if a newer version has since been published.
+   */
+  async uninstallPack(packId: string): Promise<UninstallPackResult> {
+    const { default: InstalledResource } = await import('#models/installed_resource')
+    const installed = await InstalledResource.query()
+      .where('resource_type', 'zim')
+      .where('resource_id', packId)
+      .first()
+    if (!installed) {
+      return { code: 'not_installed' }
+    }
+
+    const filename = `${packId}_${installed.version}.zim`
+    const zimService = new ZimService(this.dockerService)
+    await zimService.delete(filename)
+
+    logger.info(`[CreatorPackService] Uninstalled pack ${packId} (${filename})`)
+    return { code: 'uninstalled', filename }
+  }
+
+  private async ensureKiwixInstalled(): Promise<void> {
+    try {
+      const kiwixUrl = await this.dockerService.getServiceURL(SERVICE_NAMES.KIWIX)
+      if (kiwixUrl) return // already installed and resolvable
+
+      logger.info('[CreatorPackService] Kiwix not installed; auto-installing before pack download')
+      const result = await this.dockerService.createContainerPreflight(SERVICE_NAMES.KIWIX)
+      if (!result.success) {
+        // e.g. "already installing" — not fatal, the pack download proceeds regardless.
+        logger.warn(`[CreatorPackService] Kiwix preflight did not start: ${result.message}`)
+      }
+    } catch (error: any) {
+      logger.warn(
+        `[CreatorPackService] Kiwix auto-install failed (continuing with pack download): ${error?.message || error}`
+      )
+    }
+  }
+}

admin/app/services/custom_app_guard.ts

@@ -0,0 +1,167 @@
+import { dirname, normalize } from 'node:path'
+import env from '#start/env'
+
+/**
+ * Security guardrails for user-defined ("custom app") containers.
+ *
+ * project-nomad runs containers as host siblings via the mounted Docker socket (DooD), so a
+ * misconfigured bind mount or image is a real host-takeover vector. The posture here is
+ * "guardrails with warnings": hard-block the genuinely catastrophic, warn-but-allow the merely
+ * risky so a trusted admin keeps their power without an easy foot-gun.
+ */
+
+export interface GuardEvaluation {
+  /** Hard rejections — the install cannot proceed until these are fixed. */
+  blocked: string[]
+  /** Advisory warnings — overridable via the "install anyway" force flag. */
+  warnings: string[]
+}
+
+/** Absolute host directories that must never be bind-mounted into a custom container. */
+const SYSTEM_BLOCK_PREFIXES = ['/etc', '/proc', '/sys', '/boot', '/dev', '/run', '/var/run']
+
+/** Registries we ship curated apps from; anything else is allowed but warned on. */
+const TRUSTED_REGISTRIES = ['docker.io', 'registry-1.docker.io', 'ghcr.io', 'lscr.io', 'quay.io']
+
+/** Resolve the managed storage root (where bind mounts are expected to live). */
+export function getStorageRoot(): string {
+  return normalize(env.get('NOMAD_STORAGE_PATH', '/opt/project-nomad/storage')).replace(/\/+$/, '')
+}
+
+/** Normalize an absolute path: collapse `..`/`.` segments and strip any trailing slash. */
+function normalizeHostPath(p: string): string {
+  return normalize(p).replace(/\/+$/, '') || '/'
+}
+
+/** True when `child` equals `ancestor` or sits beneath it. */
+function isWithin(child: string, ancestor: string): boolean {
+  return child === ancestor || child.startsWith(ancestor + '/')
+}
+
+/**
+ * Evaluate user-supplied bind mounts. Hard-blocks the Docker socket, core system directories,
+ * and any mount at or above project-nomad's own install tree (which would expose its code/data).
+ * Warns on any host path outside the managed storage root.
+ */
+export function evaluateBindMounts(
+  volumes: { host_path: string; container_path: string }[]
+): GuardEvaluation {
+  const blocked: string[] = []
+  const warnings: string[] = []
+
+  const storageRoot = getStorageRoot()
+  // The install tree is the parent of the storage root (e.g. /opt/project-nomad). Mounting it —
+  // or any ancestor, up to and including `/` — would hand a container project-nomad's own files.
+  const installRoot = dirname(storageRoot)
+
+  for (const { host_path: hostPath, container_path: containerPath } of volumes) {
+    const host = normalizeHostPath(hostPath)
+
+    if (!hostPath.startsWith('/')) {
+      blocked.push(`Volume host path "${hostPath}" must be an absolute path.`)
+      continue
+    }
+    if (!containerPath.startsWith('/')) {
+      blocked.push(`Volume container path "${containerPath}" must be an absolute path.`)
+      continue
+    }
+
+    // A colon is Docker's bind delimiter (host:container:options). A path containing one would be
+    // re-split by Docker into a different mount than the one validated here — reject it outright so
+    // the checks below can't be bypassed by a parse-differential. (The validator blocks this too;
+    // this keeps the guard self-defending for any caller that skips validation.)
+    if (hostPath.includes(':') || containerPath.includes(':')) {
+      blocked.push(`Volume paths must not contain a colon (":"): "${hostPath}" → "${containerPath}".`)
+      continue
+    }
+
+    // The Docker socket is the most dangerous mount of all — full control of the host daemon.
+    if (host.endsWith('docker.sock') || /\/docker\.sock$/.test(host)) {
+      blocked.push(
+        `Mounting the Docker socket ("${hostPath}") is not allowed — it grants full host control.`
+      )
+      continue
+    }
+
+    // Core system directories.
+    if (host === '/' || SYSTEM_BLOCK_PREFIXES.some((p) => isWithin(host, p))) {
+      blocked.push(`Mounting system directory "${hostPath}" is not allowed.`)
+      continue
+    }
+
+    // At or above project-nomad's own install tree (covers `/`, `/opt`, `/opt/project-nomad`).
+    if (host === installRoot || isWithin(installRoot, host)) {
+      blocked.push(
+        `Mounting "${hostPath}" would expose project-nomad's own files and is not allowed.`
+      )
+      continue
+    }
+
+    // Anything outside the managed storage root is allowed but flagged.
+    if (!isWithin(host, storageRoot)) {
+      warnings.push(
+        `Volume "${hostPath}" is outside the managed storage root (${storageRoot}). Make sure you trust this image with access to that path.`
+      )
+    }
+  }
+
+  return { blocked, warnings }
+}
+
+/**
+ * Evaluate a Docker image reference. Hard-blocks malformed references; warns on moving tags
+ * (`:latest`/untagged) and images from registries outside the trusted set.
+ */
+export function evaluateImageReference(image: string): GuardEvaluation {
+  const blocked: string[] = []
+  const warnings: string[] = []
+
+  const ref = image.trim()
+  // Loose validity check: no whitespace/control chars, and a sane character set for an image ref.
+  if (!ref || /\s/.test(ref) || !/^[\w./:@-]+$/.test(ref)) {
+    blocked.push(`"${image}" is not a valid image reference.`)
+    return { blocked, warnings }
+  }
+
+  // Split off any digest, then any tag, to inspect the registry and tag.
+  const [nameAndTag] = ref.split('@')
+  const firstSegment = nameAndTag.split('/')[0]
+  const hasRegistryHost =
+    nameAndTag.includes('/') && (firstSegment.includes('.') || firstSegment.includes(':'))
+  const registry = hasRegistryHost ? firstSegment.split(':')[0] : 'docker.io'
+
+  if (!TRUSTED_REGISTRIES.includes(registry)) {
+    warnings.push(
+      `Image is from "${registry}", which is outside project-nomad's trusted registries. Only install images you trust.`
+    )
+  }
+
+  // Determine the tag (ignore a colon that's part of a registry host:port in the first segment).
+  const remainder = hasRegistryHost ? nameAndTag.slice(firstSegment.length + 1) : nameAndTag
+  const tag = remainder.includes(':') ? remainder.split(':').pop() : undefined
+  const hasDigest = ref.includes('@sha256:')
+  if (!hasDigest && (!tag || tag === 'latest')) {
+    warnings.push(
+      `Image "${image}" uses a moving tag (${tag ? ':latest' : 'no tag'}). Pin a specific version for reproducible installs.`
+    )
+  }
+
+  return { blocked, warnings }
+}
+
+/** Combine bind-mount and image evaluations into a single result. */
+export function evaluateCustomApp(input: {
+  image?: string
+  volumes?: { host_path: string; container_path: string }[]
+}): GuardEvaluation {
+  const bind = evaluateBindMounts(input.volumes ?? [])
+  const img = input.image ? evaluateImageReference(input.image) : { blocked: [], warnings: [] }
+  return {
+    blocked: [...bind.blocked, ...img.blocked],
+    warnings: [...bind.warnings, ...img.warnings],
+  }
+}
+
+/** Default resource caps applied to custom containers unless the user overrides them. */
+export const DEFAULT_MEMORY_MB = 1024
+export const DEFAULT_CPUS = 1

admin/app/services/docs_service.ts

@@ -12,10 +12,13 @@ export class DocsService {
     'home': 1,
     'getting-started': 2,
     'use-cases': 3,
-    'community-add-ons': 4,
+    'supply-depot-apps': 4,
-    'faq': 5,
+    'drug-reference': 5,
-    'about': 6,
+    'community-add-ons': 6,
-    'release-notes': 7,
+    'updates': 7,
+    'faq': 8,
+    'about': 9,
+    'release-notes': 10,
   }
 
   async getDocs() {

admin/app/services/download_service.ts

@@ -4,7 +4,8 @@ import { RunDownloadJob } from '#jobs/run_download_job'
 import { RunExtractPmtilesJob } from '#jobs/run_extract_pmtiles_job'
 import type { RunExtractPmtilesJobParams } from '#jobs/run_extract_pmtiles_job'
 import { DownloadModelJob } from '#jobs/download_model_job'
-import { DownloadJobWithProgress, DownloadProgressData } from '../../types/downloads.js'
+import { DownloadDrugDataJob } from '#jobs/download_drug_data_job'
+import { DownloadJobWithProgress, DownloadProgressData, RunDownloadJobParams } from '../../types/downloads.js'
 import type { Job, Queue } from 'bullmq'
 import { normalize } from 'path'
 import { deleteFileIfExists } from '../utils/fs.js'
@@ -46,15 +47,21 @@ export class DownloadService {
       ...active.map((j) => ({ job: j, state: 'active' as const })),
       ...delayed.map((j) => ({ job: j, state: 'delayed' as const })),
       ...failed.map((j) => ({ job: j, state: 'failed' as const })),
-    ]
+      // A job id can outlive its payload hash — BullMQ still returns an entry for
+      // it, with `data` empty. One of those in the failed set used to throw on
+      // every poll of this endpoint (normalize(undefined)), and because failed
+      // jobs are never evicted the endpoint stayed broken until Redis was cleared
+      // by hand. Drop them: with no payload there is nothing to show anyway.
+    ].filter(({ job }) => job?.id != null && job.data != null)
   }
 
   async listDownloadJobs(filetype?: string): Promise<DownloadJobWithProgress[]> {
     const modelQueue = this.queueService.getQueue(DownloadModelJob.queue)
-    const [fileTagged, extractTagged, modelJobs] = await Promise.all([
+    const [fileTagged, extractTagged, modelJobs, drugTagged] = await Promise.all([
       this.fetchJobsWithStates(RunDownloadJob.queue),
       this.fetchJobsWithStates(RunExtractPmtilesJob.queue),
       modelQueue.getJobs(['waiting', 'active', 'delayed', 'failed']),
+      this.fetchJobsWithStates(DownloadDrugDataJob.queue),
     ])
 
     const fileDownloads = fileTagged.map(({ job, state }) => {
@@ -63,7 +70,7 @@ export class DownloadService {
         jobId: job.id!.toString(),
         url: job.data.url,
         progress: parsed.percent,
-        filepath: normalize(job.data.filepath),
+        filepath: job.data.filepath ? normalize(job.data.filepath) : '',
         filetype: job.data.filetype,
         title: job.data.title || undefined,
         downloadedBytes: parsed.downloadedBytes,
@@ -80,7 +87,7 @@ export class DownloadService {
         jobId: job.id!.toString(),
         url: job.data.sourceUrl,
         progress: parsed.percent,
-        filepath: normalize(job.data.outputFilepath),
+        filepath: job.data.outputFilepath ? normalize(job.data.outputFilepath) : '',
         filetype: job.data.filetype || 'map',
         title: job.data.title || undefined,
         downloadedBytes: parsed.downloadedBytes,
@@ -101,7 +108,40 @@ export class DownloadService {
       failedReason: job.failedReason || undefined,
     }))
 
-    const allDownloads = [...fileDownloads, ...extractDownloads, ...modelDownloads]
+    // FDA drug dataset — DOWNLOAD phase only, collapsed to ONE card. The job fans
+    // the manifest's N partitions into continuations under AUTO-GENERATED jobIds
+    // (only part 0 runs under the deterministic jobId), and the queue is
+    // concurrency 1, so at most one part is ever in flight. Filtering to the
+    // deterministic jobId would track only part 0 and then drop the card while
+    // parts 2..N keep downloading. Instead represent the whole download with the
+    // single in-flight job's aggregate progress (the progress emit already spans
+    // all parts), and always report the deterministic jobId so the cancel/remove
+    // button routes to _cancelDrugDownloadJob whichever part is active. The heavy
+    // ingest is EXCLUDED here — it stays on the IngestStatus surface.
+    const drugInFlight =
+      drugTagged.find(({ state }) => state === 'active') ??
+      drugTagged.find(({ state }) => state === 'waiting' || state === 'delayed') ??
+      drugTagged.find(({ state }) => state === 'failed')
+    const drugDownloads = drugInFlight
+      ? [drugInFlight].map(({ job, state }) => {
+          const parsed = this.parseProgress(job.progress)
+          return {
+            jobId: DownloadDrugDataJob.jobId,
+            url: 'https://api.fda.gov/download.json',
+            progress: parsed.percent,
+            filepath: job.data.currentPartName || 'FDA Drug Reference',
+            filetype: 'drug-data',
+            title: 'FDA Drug Reference',
+            downloadedBytes: parsed.downloadedBytes,
+            totalBytes: parsed.totalBytes,
+            lastProgressTime: parsed.lastProgressTime,
+            status: state,
+            failedReason: job.failedReason || undefined,
+          }
+        })
+      : []
+
+    const allDownloads = [...fileDownloads, ...extractDownloads, ...modelDownloads, ...drugDownloads]
     const filtered = allDownloads.filter((job) => !filetype || job.filetype === filetype)
 
     return filtered.sort((a, b) => {
@@ -116,6 +156,7 @@ export class DownloadService {
       RunDownloadJob.queue,
       RunExtractPmtilesJob.queue,
       DownloadModelJob.queue,
+      DownloadDrugDataJob.queue,
     ]) {
       const queue = this.queueService.getQueue(queueName)
       const job = await queue.getJob(jobId)
@@ -137,6 +178,40 @@ export class DownloadService {
     }
   }
 
+  async retryFailedJob(jobId: string): Promise<{ success: boolean; message: string }> {
+    // Search both the file download queue and the model download queue
+    for (const queueName of [RunDownloadJob.queue, DownloadModelJob.queue]) {
+      const queue = this.queueService.getQueue(queueName)
+      const job = await queue.getJob(jobId)
+
+      if (job) {
+        // For Ollama model downloads, re-dispatch with the model name
+        if (queueName === DownloadModelJob.queue) {
+          const modelName = job.data.modelName
+          if (!modelName) {
+            return { success: false, message: 'Cannot retry: model name not found in job data' }
+          }
+          await DownloadModelJob.dispatch({ modelName })
+          await job.remove().catch(() => {})
+          return { success: true, message: `Retrying download for model ${modelName}` }
+        }
+
+        // For file downloads (zim, map, etc.), re-dispatch with original params
+        const params = job.data as RunDownloadJobParams
+        if (!params.url || !params.filepath) {
+          return { success: false, message: 'Cannot retry: missing URL or filepath in job data' }
+        }
+
+        // Remove the old failed job, then dispatch a fresh one
+        await job.remove().catch(() => {})
+        await RunDownloadJob.dispatch(params)
+        return { success: true, message: `Retrying download for ${params.url}` }
+      }
+    }
+
+    return { success: false, message: 'Failed job not found. It may have already been dismissed.' }
+  }
+
   async cancelJob(jobId: string): Promise<{ success: boolean; message: string }> {
     const queue = this.queueService.getQueue(RunDownloadJob.queue)
     const job = await queue.getJob(jobId)
@@ -159,9 +234,42 @@ export class DownloadService {
       return await this._cancelModelDownloadJob(jobId, modelJob, modelQueue)
     }
 
+    // FDA drug dataset: cancel is matched on the deterministic jobId only (the
+    // single card the aggregator shows). The continuation parts run under
+    // auto-generated ids, so cancelling must stop the whole CHAIN, not just the
+    // current part — otherwise the next continuation fires after we remove one.
+    if (jobId === DownloadDrugDataJob.jobId) {
+      return await this._cancelDrugDownloadJob()
+    }
+
     return { success: true, message: 'Job not found (may have already completed)' }
   }
 
+  /**
+   * Cancel the FDA drug-data download.
+   *
+   * The drug job has no Redis cancel-signal / AbortController (unlike
+   * RunDownloadJob), and it self-continues into the next part under a fresh jobId.
+   * So a v1 cancel that only removed the current job would leave the next
+   * continuation to fire. Instead we obliterate the single-purpose drug-download
+   * queue (force = removes the active/locked job too), which drops the current
+   * part AND every queued continuation in one shot. Scoped to the drug-download
+   * queue only — the ingest queue and everything else are untouched. The on-disk
+   * parts are intentionally LEFT in place: they're resumable, and a re-trigger
+   * picks up from the .tmp rather than re-downloading. (Tracked as the v1 choice
+   * for cancel depth — full cross-process signal cancel is a follow-up.)
+   */
+  private async _cancelDrugDownloadJob(): Promise<{ success: boolean; message: string }> {
+    const queue = this.queueService.getQueue(DownloadDrugDataJob.queue)
+    try {
+      await queue.obliterate({ force: true })
+      return { success: true, message: 'Drug data download cancelled' }
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      return { success: false, message: `Could not cancel drug data download: ${msg}` }
+    }
+  }
+
   private async _cancelExtractJob(
     jobId: string,
     job: Job<RunExtractPmtilesJobParams>,

admin/app/services/drug_reference_service.ts

@@ -0,0 +1,830 @@
+import db from '@adonisjs/lucid/services/db'
+import logger from '@adonisjs/core/services/logger'
+import { rm } from 'node:fs/promises'
+import type { Job } from 'bullmq'
+import { QueueService } from './queue_service.js'
+import { DownloadDrugDataJob, STORAGE_BASE } from '#jobs/download_drug_data_job'
+import { IngestDrugDataJob } from '#jobs/ingest_drug_data_job'
+
+/**
+ * Manifest resource id for the FDA drug dataset — the `installed_resources`
+ * row's `resource_id`, matching the `medicine-standard` tier manifest entry.
+ * Single source of truth so the install write-back, the tier-status math, and
+ * uninstall all agree on the same id.
+ */
+export const DRUG_DATASET_RESOURCE_ID = 'openfda-drug-labels'
+import {
+  normalizeDrugName,
+  parseDownloadState,
+  deriveIngestPhase,
+  resolveExpectedTotal,
+  resolveIngestRecordsShown,
+  summarizeJobError,
+  isExportDateNewer,
+} from '../../util/drug_labels.js'
+import KVStore from '#models/kv_store'
+import { parseCompareIds, MAX_COMPARE } from '../../util/compare_ids.js'
+import type {
+  DrugSearchResult,
+  DrugLabelDetail,
+  DrugIngestStatus,
+  DrugPhaseState,
+  DrugDownloadStatus,
+  DrugIngestPhaseStatus,
+  DrugInteractionEntry,
+} from '../../types/drug_reference.js'
+
+/**
+ * Drug Reference v1 — service layer.
+ *
+ * Exposes search (collapsed by brand+generic), detail fetch, ingest trigger,
+ * and ingest status. Mirrors the shape of DownloadService/ZimService.
+ */
+export class DrugReferenceService {
+  /**
+   * Search for drug labels, collapsed by (brand_name, generic_name).
+   *
+   * Each distinct (brand_name, generic_name) pair returns ONE result — a
+   * representative row id (MIN(id)) and a labelCount of how many set_ids
+   * collapsed into it. This is the locked UX decision.
+   *
+   * Scope:
+   *   'name'       (default) — existing path: MATCH(searchable_name) on brand+generic.
+   *   'indication' — new path: MATCH(searchable_name, indications) on the combined
+   *                  ft_drug_labels_name_indications index so users can search by
+   *                  what a drug treats ("heartburn", "high blood pressure").
+   *
+   * Strategy (both scopes):
+   *   1. FULLTEXT path: MATCH(cols) AGAINST(? IN NATURAL LANGUAGE MODE)
+   *      — relevance-ranked, requires >= 3 chars (innodb_ft_min_token_size = 3).
+   *   2. LIKE fallback: query < 3 chars OR FULLTEXT throws → LIKE '%term%'.
+   *   3. Both paths apply the optional product_type filter and GROUP BY collapse.
+   */
+  async search(
+    query: string,
+    options: {
+      productType?: string
+      route?: string
+      sort?: 'relevance' | 'name'
+      limit?: number
+      offset?: number
+      scope?: 'name' | 'indication'
+    }
+  ): Promise<DrugSearchResult[]> {
+    const limit = options.limit ?? 50
+    const offset = options.offset ?? 0
+    const scope = options.scope ?? 'name'
+    const normalized = normalizeDrugName(query, null) ?? query.trim()
+
+    if (!normalized || normalized.length === 0) return []
+
+    const useLike = normalized.length < 3
+
+    if (scope === 'indication') {
+      if (!useLike) {
+        try {
+          return await this.searchIndicationFulltext(normalized, options.productType, limit, offset)
+        } catch (err) {
+          logger.warn(
+            `[DrugReferenceService] FULLTEXT indication search failed, falling back to LIKE: ${
+              err instanceof Error ? err.message : String(err)
+            }`
+          )
+        }
+      }
+      return await this.searchIndicationLike(normalized, options.productType, limit, offset)
+    }
+
+    if (!useLike) {
+      // FULLTEXT path
+      try {
+        return await this.searchFulltext(normalized, options, limit, offset)
+      } catch (err) {
+        logger.warn(
+          `[DrugReferenceService] FULLTEXT search failed, falling back to LIKE: ${
+            err instanceof Error ? err.message : String(err)
+          }`
+        )
+      }
+    }
+
+    // LIKE fallback
+    return await this.searchLike(normalized, options, limit, offset)
+  }
+
+  private async searchFulltext(
+    normalized: string,
+    opts: { productType?: string; route?: string; sort?: 'relevance' | 'name' },
+    limit: number,
+    offset: number
+  ): Promise<DrugSearchResult[]> {
+    let sql = `
+      SELECT
+        MIN(id) AS id,
+        brand_name,
+        generic_name,
+        MIN(manufacturer) AS manufacturer,
+        MIN(route) AS route,
+        MIN(product_type) AS product_type,
+        COUNT(*) AS labelCount,
+        MAX(MATCH(searchable_name) AGAINST(? IN NATURAL LANGUAGE MODE)) AS relevance
+      FROM drug_labels
+      WHERE MATCH(searchable_name) AGAINST(? IN NATURAL LANGUAGE MODE)
+    `
+    const bindings: unknown[] = [normalized, normalized]
+
+    if (opts.productType) {
+      sql += ' AND product_type = ?'
+      bindings.push(opts.productType)
+    }
+    if (opts.route) {
+      // `route` is a comma-joined uppercase list (e.g. "ORAL, TOPICAL").
+      sql += ' AND route LIKE ?'
+      bindings.push(`%${opts.route.toUpperCase()}%`)
+    }
+
+    sql += `
+      GROUP BY brand_name, generic_name
+      ORDER BY ${opts.sort === 'name' ? 'COALESCE(brand_name, generic_name) ASC' : 'relevance DESC'}
+      LIMIT ? OFFSET ?
+    `
+    bindings.push(limit, offset)
+
+    const rows = await db.rawQuery(sql, bindings)
+    return this.mapSearchRows(rows[0])
+  }
+
+  private async searchLike(
+    normalized: string,
+    opts: { productType?: string; route?: string; sort?: 'relevance' | 'name' },
+    limit: number,
+    offset: number
+  ): Promise<DrugSearchResult[]> {
+    const term = `%${normalized}%`
+    let sql = `
+      SELECT
+        MIN(id) AS id,
+        brand_name,
+        generic_name,
+        MIN(manufacturer) AS manufacturer,
+        MIN(route) AS route,
+        MIN(product_type) AS product_type,
+        COUNT(*) AS labelCount
+      FROM drug_labels
+      WHERE (searchable_name LIKE ? OR brand_name LIKE ?)
+    `
+    const bindings: unknown[] = [term, term]
+
+    if (opts.productType) {
+      sql += ' AND product_type = ?'
+      bindings.push(opts.productType)
+    }
+    if (opts.route) {
+      sql += ' AND route LIKE ?'
+      bindings.push(`%${opts.route.toUpperCase()}%`)
+    }
+
+    sql += `
+      GROUP BY brand_name, generic_name
+      ORDER BY brand_name ASC
+      LIMIT ? OFFSET ?
+    `
+    bindings.push(limit, offset)
+
+    const rows = await db.rawQuery(sql, bindings)
+    return this.mapSearchRows(rows[0])
+  }
+
+  /**
+   * FULLTEXT indication-scope search.
+   *
+   * MATCHes over (searchable_name, indications) — must exactly match the
+   * ft_drug_labels_name_indications index column list. The MAX(MATCH …) pattern
+   * is LOAD-BEARING: MySQL 8.0 ONLY_FULL_GROUP_BY rejects a bare MATCH() in
+   * SELECT when GROUP BY is in effect; wrapping in MAX() makes it an aggregate
+   * and satisfies the mode constraint.
+   */
+  private async searchIndicationFulltext(
+    normalized: string,
+    productType: string | undefined,
+    limit: number,
+    offset: number
+  ): Promise<DrugSearchResult[]> {
+    let sql = `
+      SELECT
+        MIN(id) AS id,
+        brand_name,
+        generic_name,
+        MIN(manufacturer) AS manufacturer,
+        MIN(route) AS route,
+        MIN(product_type) AS product_type,
+        COUNT(*) AS labelCount,
+        MAX(MATCH(searchable_name, indications) AGAINST(? IN NATURAL LANGUAGE MODE)) AS relevance
+      FROM drug_labels
+      WHERE MATCH(searchable_name, indications) AGAINST(? IN NATURAL LANGUAGE MODE)
+    `
+    const bindings: unknown[] = [normalized, normalized]
+
+    if (productType) {
+      sql += ' AND product_type = ?'
+      bindings.push(productType)
+    }
+
+    sql += `
+      GROUP BY brand_name, generic_name
+      ORDER BY relevance DESC
+      LIMIT ? OFFSET ?
+    `
+    bindings.push(limit, offset)
+
+    const rows = await db.rawQuery(sql, bindings)
+    return this.mapSearchRows(rows[0])
+  }
+
+  /**
+   * LIKE indication-scope fallback (query < 3 chars or FULLTEXT unavailable).
+   *
+   * Searches searchable_name OR indications so short queries still return
+   * useful results without requiring the FULLTEXT index.
+   */
+  private async searchIndicationLike(
+    normalized: string,
+    productType: string | undefined,
+    limit: number,
+    offset: number
+  ): Promise<DrugSearchResult[]> {
+    const term = `%${normalized}%`
+    let sql = `
+      SELECT
+        MIN(id) AS id,
+        brand_name,
+        generic_name,
+        MIN(manufacturer) AS manufacturer,
+        MIN(route) AS route,
+        MIN(product_type) AS product_type,
+        COUNT(*) AS labelCount
+      FROM drug_labels
+      WHERE (searchable_name LIKE ? OR indications LIKE ?)
+    `
+    const bindings: unknown[] = [term, term]
+
+    if (productType) {
+      sql += ' AND product_type = ?'
+      bindings.push(productType)
+    }
+
+    sql += `
+      GROUP BY brand_name, generic_name
+      ORDER BY brand_name ASC
+      LIMIT ? OFFSET ?
+    `
+    bindings.push(limit, offset)
+
+    const rows = await db.rawQuery(sql, bindings)
+    return this.mapSearchRows(rows[0])
+  }
+
+  private mapSearchRows(rows: any[]): DrugSearchResult[] {
+    if (!Array.isArray(rows)) return []
+    return rows.map((row) => ({
+      id: Number(row.id),
+      brand_name: row.brand_name ?? null,
+      generic_name: row.generic_name ?? null,
+      manufacturer: row.manufacturer ?? null,
+      route: row.route ?? null,
+      product_type: row.product_type ?? null,
+      labelCount: Number(row.labelCount ?? row.labelcount ?? 1),
+    }))
+  }
+
+  /**
+   * Load the full detail for a single drug label row by its surrogate id.
+   * Returns null if the row doesn't exist.
+   */
+  async find(id: number): Promise<DrugLabelDetail | null> {
+    const { default: DrugLabel } = await import('#models/drug_label')
+    const row = await DrugLabel.find(id)
+    if (!row) return null
+
+    return {
+      id: row.id,
+      set_id: row.set_id,
+      spl_id: row.spl_id,
+      version: row.version,
+      brand_name: row.brand_name,
+      generic_name: row.generic_name,
+      manufacturer: row.manufacturer,
+      product_ndc: row.product_ndc,
+      route: row.route,
+      product_type: row.product_type,
+      indications: row.indications,
+      dosage: row.dosage,
+      warnings: row.warnings,
+      boxed_warning: row.boxed_warning,
+      drug_interactions: row.drug_interactions,
+      contraindications: row.contraindications,
+      when_using: row.when_using,
+      stop_use: row.stop_use,
+      source_updated_at: row.source_updated_at,
+      ingested_at: row.ingested_at.toISO() ?? '',
+    }
+  }
+
+  /**
+   * Return the drug interaction text for a set of label ids.
+   *
+   * - Dedupes and caps the id list via parseCompareIds / MAX_COMPARE.
+   * - One query: SELECT id, brand_name, generic_name, product_type,
+   *   drug_interactions FROM drug_labels WHERE id IN (?).
+   * - Re-orders the rows to match the requested id order so the caller's
+   *   column positions are stable even if MySQL returns rows in a different
+   *   order. Missing ids (non-existent in the table) are silently omitted.
+   * - Returns [] for an empty or entirely-invalid id list.
+   */
+  async getInteractionsFor(ids: number[]): Promise<DrugInteractionEntry[]> {
+    if (ids.length === 0) return []
+
+    // Dedupe + cap (caller may already have done this, but be defensive).
+    const safeIds = parseCompareIds(ids.join(',')).slice(0, MAX_COMPARE)
+    if (safeIds.length === 0) return []
+
+    const placeholders = safeIds.map(() => '?').join(', ')
+    const sql = `
+      SELECT id, brand_name, generic_name, product_type, drug_interactions
+      FROM drug_labels
+      WHERE id IN (${placeholders})
+    `
+    const rows = await db.rawQuery(sql, safeIds)
+    const rawRows: Array<{
+      id: number | string
+      brand_name: string | null
+      generic_name: string | null
+      product_type: string | null
+      drug_interactions: string | null
+    }> = Array.isArray(rows[0]) ? rows[0] : []
+
+    // Build a lookup by id so we can re-order to match the request order.
+    const byId = new Map<number, DrugInteractionEntry>()
+    for (const row of rawRows) {
+      const entry: DrugInteractionEntry = {
+        id: Number(row.id),
+        brand_name: row.brand_name ?? null,
+        generic_name: row.generic_name ?? null,
+        product_type: row.product_type ?? null,
+        drug_interactions: row.drug_interactions ?? null,
+      }
+      byId.set(entry.id, entry)
+    }
+
+    // Re-order: iterate safeIds, include only ids that exist in the table.
+    const ordered: DrugInteractionEntry[] = []
+    for (const id of safeIds) {
+      const entry = byId.get(id)
+      if (entry) ordered.push(entry)
+    }
+    return ordered
+  }
+
+  /**
+   * Get current row count — what's searchable right now.
+   */
+  async rowCount(): Promise<number> {
+    try {
+      const result = await db.rawQuery('SELECT COUNT(*) AS cnt FROM drug_labels')
+      const rows = result[0] as Array<{ cnt: number | string }>
+      return Number(rows[0]?.cnt ?? 0)
+    } catch {
+      return 0
+    }
+  }
+
+  /**
+   * Dispatch the download phase (idempotent — deduped by deterministic jobId).
+   * Auto-chains the ingest phase on completion. Returns "already running" if the
+   * download job is active/waiting.
+   */
+  async triggerDownload() {
+    return DownloadDrugDataJob.dispatch(true)
+  }
+
+  /**
+   * Freshness check for the content-auto-update path: compare the live openFDA
+   * manifest `export_date` against the last-ingested one (KV
+   * `drugReference.lastUpdatedExportDate`). Reuses the job's single manifest
+   * fetch (Maxim 4 — one openFDA call site) and the pure `isExportDateNewer`
+   * compare (defensive about the unconfirmed date format; see its TODO).
+   *
+   * Returns `{ updateAvailable, latestExportDate, currentExportDate }`. Never
+   * triggers a download itself — `attemptAutoUpdate` decides whether to, after
+   * its own gating (only when installed, no active job).
+   */
+  async checkForUpdate(): Promise<{
+    updateAvailable: boolean
+    latestExportDate: string | null
+    currentExportDate: string | null
+  }> {
+    const current = await KVStore.getValue('drugReference.lastUpdatedExportDate')
+    const manifest = await DownloadDrugDataJob.fetchManifest()
+    const latest = manifest.export_date ?? null
+    return {
+      updateAvailable: isExportDateNewer(latest ?? '', current),
+      latestExportDate: latest,
+      currentExportDate: current,
+    }
+  }
+
+  /**
+   * Freshness pass for the drug dataset, invoked from
+   * `ContentAutoUpdateService.attemptDrugDataset()` inside the hourly content
+   * loop — so the dataset refreshes alongside ZIMs and maps under the shared
+   * `contentAutoUpdate.*` master switch and window, not on a rogue schedule.
+   *
+   * Gating (unchanged from the prior standalone job): only acts when the dataset
+   * is installed, and never while a drug download or ingest is already in flight,
+   * so it can't stack a refresh on a running install. A failed manifest fetch is
+   * a transient miss (offline), reported rather than thrown.
+   */
+  async attemptAutoUpdate(): Promise<{
+    started: boolean
+    reason: string
+    latestExportDate?: string | null
+  }> {
+    // Only act when the dataset is installed.
+    const rowCount = await this.rowCount()
+    if (rowCount === 0) {
+      return { started: false, reason: 'not-installed' }
+    }
+
+    // Don't stack on top of an in-flight download/ingest.
+    const [dlJob, ingJob] = await Promise.all([
+      DownloadDrugDataJob.getJob(),
+      IngestDrugDataJob.getJob(),
+    ])
+    const activeStates = ['active', 'waiting', 'delayed']
+    const dlState = dlJob ? await dlJob.getState() : undefined
+    const ingState = ingJob ? await ingJob.getState() : undefined
+    if (
+      (dlState && activeStates.includes(dlState)) ||
+      (ingState && activeStates.includes(ingState))
+    ) {
+      return { started: false, reason: 'job-in-flight' }
+    }
+
+    let check: Awaited<ReturnType<DrugReferenceService['checkForUpdate']>>
+    try {
+      check = await this.checkForUpdate()
+    } catch (err) {
+      // Offline or manifest fetch failed — transient, retried next run.
+      logger.warn(
+        `[DrugReferenceService] Freshness check failed (will retry next run): ${
+          err instanceof Error ? err.message : String(err)
+        }`
+      )
+      return { started: false, reason: 'check-failed' }
+    }
+
+    if (!check.updateAvailable) {
+      return {
+        started: false,
+        reason: `up-to-date (current=${check.currentExportDate ?? 'none'}, latest=${
+          check.latestExportDate ?? 'unknown'
+        })`,
+        latestExportDate: check.latestExportDate,
+      }
+    }
+
+    logger.info(
+      `[DrugReferenceService] Newer export_date available ` +
+        `(current=${check.currentExportDate ?? 'none'} → latest=${check.latestExportDate}); ` +
+        'triggering re-download.'
+    )
+    const result = await this.triggerDownload()
+    return {
+      started: result.created,
+      reason: result.created ? 'update-dispatched' : result.message,
+      latestExportDate: check.latestExportDate,
+    }
+  }
+
+  /**
+   * Dispatch the ingest phase from the already-downloaded on-disk parts (the
+   * manual "Ingest into search" path). Guards on the KV download-state marker so
+   * it fails fast with a typed result when nothing has been downloaded, rather
+   * than dispatching a job that would immediately fail in the worker.
+   */
+  async triggerIngestFromDisk() {
+    const marker = parseDownloadState(await KVStore.getValue('drugReference.downloadState'))
+    if (!marker) {
+      return {
+        job: undefined,
+        created: false,
+        message: 'Nothing downloaded — run Download FDA data first.',
+        nothingDownloaded: true,
+      }
+    }
+    const result = await IngestDrugDataJob.dispatch()
+    return { ...result, nothingDownloaded: false }
+  }
+
+  /**
+   * Force-clear a wedged ingest and restart it from the on-disk parts.
+   *
+   * A worker killed mid-ingest (e.g. during a `nomad upgrade`) leaves its job
+   * 'active' holding a lock BullMQ won't reclaim until lockDuration elapses, so
+   * the normal dispatch refuses to start a new ingest and the UI button stays
+   * disabled ("Indexing…") with no way out. Obliterating the single-purpose
+   * drug-ingest queue removes the stuck job (force = even active/locked); we then
+   * re-dispatch from disk. The downloaded parts and the download-state marker are
+   * left untouched, so this restarts ingest WITHOUT re-downloading.
+   */
+  async resetAndReingest() {
+    const marker = parseDownloadState(await KVStore.getValue('drugReference.downloadState'))
+    if (!marker) {
+      return {
+        job: undefined,
+        created: false,
+        message: 'Nothing downloaded — run Download FDA data first.',
+        nothingDownloaded: true,
+      }
+    }
+
+    const queue = QueueService.getInstance().getQueue(IngestDrugDataJob.queue)
+    try {
+      // force: true removes the active/locked stuck job too. Scoped to the
+      // single-purpose ingest queue, so nothing else is affected.
+      await queue.obliterate({ force: true })
+      logger.info('[DrugReferenceService] drug-ingest queue obliterated for restart')
+    } catch (err) {
+      logger.warn(
+        `[DrugReferenceService] ingest queue obliterate failed (continuing to dispatch): ${
+          err instanceof Error ? err.message : String(err)
+        }`
+      )
+    }
+
+    const result = await IngestDrugDataJob.dispatch()
+    return { ...result, nothingDownloaded: false }
+  }
+
+  /**
+   * Uninstall the offline FDA drug dataset — the curated-tier "remove" path.
+   *
+   * Mirrors how ZimService.delete() reverses a ZIM install: stop anything that
+   * could re-create the data, delete the on-disk artifacts, drop the data, and
+   * remove the install-state row so the tier reads not-installed and the home
+   * tiles auto-hide (the same cascade as a ZIM delete).
+   *
+   * Order matters:
+   *   1. Obliterate BOTH drug queues (force = removes active/locked jobs too) so a
+   *      running download/ingest can't write rows back in after we clear them.
+   *      Scoped to the two single-purpose drug queues — nothing else is touched.
+   *   2. Delete the on-disk parts dir (STORAGE_BASE is a FIXED constant, never a
+   *      client value — no path-traversal surface). Usually empty after a full
+   *      ingest; non-empty only when uninstalling a downloaded-not-yet-ingested
+   *      state.
+   *   3. TRUNCATE drug_labels (not DROP) — preserves the schema + FULLTEXT
+   *      indexes so a later reinstall re-ingests without a migration.
+   *   4. Clear the KV markers (download-state + last-updated export_date).
+   *   5. Delete the `installed_resources` 'dataset' row.
+   *
+   * Every step is best-effort and logged: a partial failure still removes as much
+   * as it can and reports what it did, rather than leaving a half-uninstalled
+   * state with no signal.
+   */
+  async uninstall(): Promise<{
+    success: boolean
+    rowsDropped: number
+    message: string
+  }> {
+    const rowsBefore = await this.rowCount()
+    const errors: string[] = []
+
+    // 1. Stop in-flight jobs on both drug queues (force removes locked/active).
+    for (const queueName of [DownloadDrugDataJob.queue, IngestDrugDataJob.queue]) {
+      try {
+        await QueueService.getInstance().getQueue(queueName).obliterate({ force: true })
+        logger.info(`[DrugReferenceService] obliterated ${queueName} for uninstall`)
+      } catch (err) {
+        const msg = err instanceof Error ? err.message : String(err)
+        logger.warn(`[DrugReferenceService] obliterate ${queueName} failed: ${msg}`)
+        errors.push(`queue ${queueName}: ${msg}`)
+      }
+    }
+
+    // 2. Delete the on-disk parts directory. STORAGE_BASE is a fixed module const
+    // (never a client filename), so this is not a path-traversal surface.
+    try {
+      await rm(STORAGE_BASE, { recursive: true, force: true })
+      logger.info(`[DrugReferenceService] removed on-disk parts dir ${STORAGE_BASE}`)
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.warn(`[DrugReferenceService] could not remove ${STORAGE_BASE}: ${msg}`)
+      errors.push(`storage: ${msg}`)
+    }
+
+    // 3. Drop the searchable data. TRUNCATE keeps schema + the FULLTEXT indexes
+    // so reinstall re-ingests cleanly with no migration.
+    try {
+      await db.rawQuery('TRUNCATE TABLE drug_labels')
+      logger.info('[DrugReferenceService] truncated drug_labels')
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.error(`[DrugReferenceService] TRUNCATE drug_labels failed: ${msg}`)
+      errors.push(`truncate: ${msg}`)
+    }
+
+    // 4. Clear KV markers.
+    try {
+      await KVStore.clearValue('drugReference.downloadState')
+      await KVStore.clearValue('drugReference.lastUpdatedExportDate')
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.warn(`[DrugReferenceService] clearing KV markers failed: ${msg}`)
+      errors.push(`kv: ${msg}`)
+    }
+
+    // 5. Remove the install-state row so the tier reads not-installed.
+    try {
+      const { default: InstalledResource } = await import('#models/installed_resource')
+      await InstalledResource.query()
+        .where('resource_type', 'dataset')
+        .where('resource_id', DRUG_DATASET_RESOURCE_ID)
+        .delete()
+      logger.info('[DrugReferenceService] deleted installed_resources dataset row')
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err)
+      logger.warn(`[DrugReferenceService] deleting install row failed: ${msg}`)
+      errors.push(`install-row: ${msg}`)
+    }
+
+    const rowsDropped = rowsBefore - (await this.rowCount())
+    const success = errors.length === 0
+    return {
+      success,
+      rowsDropped,
+      message: success
+        ? `Uninstalled FDA drug reference (${rowsDropped} labels removed).`
+        : `Uninstall completed with ${errors.length} issue(s): ${errors.join('; ')}`,
+    }
+  }
+
+  /**
+   * Resolve the canonical deterministic job for a phase's queue, falling back to
+   * the most-progressed auto-id continuation when the deterministic job is
+   * absent or completed (passes > 0 use auto-generated ids). Each phase has its
+   * own queue + jobId, so this is called once per queue with the matching ids.
+   */
+  private async resolvePhaseJob(
+    queueName: string,
+    deterministicJobId: string
+  ): Promise<Job | undefined> {
+    const queueService = QueueService.getInstance()
+    const queue = queueService.getQueue(queueName)
+
+    let job = await queue.getJob(deterministicJobId)
+    if (!job || (await job.getState()) === 'completed') {
+      const activeJobs = await queue.getJobs(['active', 'waiting', 'delayed'])
+      const continuation = activeJobs
+        .filter((j) => j.id !== deterministicJobId)
+        .sort((a, b) => (b.data?.partIndex ?? 0) - (a.data?.partIndex ?? 0))[0]
+      if (continuation) {
+        job = continuation
+      } else {
+        // No live continuation. If the chain FAILED partway, surface the failed
+        // job so the status reads 'failed' instead of falsely 'ready' with only
+        // part 1's count. BUT removeOnFail keeps history: a STALE failure from a
+        // prior run must not mask a newer successful one (seen live: a fixed
+        // re-run completed, yet the panel stayed 'failed' on last night's
+        // duplicate-key job). Compare finish times and only surface the failed
+        // job when it is the most recent terminal outcome.
+        const [failedJobs, completedJobs] = await Promise.all([
+          queue.getJobs(['failed']),
+          queue.getJobs(['completed']),
+        ])
+        const newestFailed = failedJobs.sort((a, b) => (b.finishedOn ?? 0) - (a.finishedOn ?? 0))[0]
+        const newestCompletedFinish = completedJobs.reduce(
+          (max, j) => Math.max(max, j.finishedOn ?? 0),
+          0
+        )
+        if (newestFailed && (newestFailed.finishedOn ?? 0) > newestCompletedFinish) {
+          job = newestFailed
+        }
+      }
+    }
+    return job
+  }
+
+  /** Map a BullMQ job state to the per-phase run state. */
+  private phaseStateFor(state: string | undefined): DrugPhaseState {
+    if (state === 'failed') return 'failed'
+    if (state === 'completed') return 'completed'
+    if (state === 'active' || state === 'waiting' || state === 'delayed') return 'running'
+    return 'idle'
+  }
+
+  /**
+   * Return the two-phase ingest status for the UI panel. Reads the download job
+   * (drug-download queue) and the ingest job (drug-ingest queue) independently,
+   * merges the KV download-state marker + last-updated marker + live row count,
+   * and derives the top-level phase from the two sub-phases.
+   */
+  async getIngestStatus(): Promise<DrugIngestStatus> {
+    const [downloadJob, ingestJob] = await Promise.all([
+      this.resolvePhaseJob(DownloadDrugDataJob.queue, DownloadDrugDataJob.jobId),
+      this.resolvePhaseJob(IngestDrugDataJob.queue, IngestDrugDataJob.jobId),
+    ])
+
+    const [lastUpdated, rawMarker, count] = await Promise.all([
+      KVStore.getValue('drugReference.lastUpdatedExportDate'),
+      KVStore.getValue('drugReference.downloadState'),
+      this.rowCount(),
+    ])
+    const marker = parseDownloadState(rawMarker)
+
+    // ── Download sub-status ─────────────────────────────────────────────────
+    const dlState = downloadJob ? await downloadJob.getState() : undefined
+    const dlData = downloadJob?.data ?? {}
+    let downloadPhaseState = this.phaseStateFor(dlState)
+    // A finished download job is pruned (removeOnComplete) but the marker proves
+    // the parts are on disk — treat that as a completed download phase so the
+    // manual "Ingest into search" button stays available.
+    if (downloadPhaseState === 'idle' && marker) downloadPhaseState = 'completed'
+
+    const download: DrugDownloadStatus = {
+      state: downloadPhaseState,
+      partsDone: downloadPhaseState === 'completed' ? (marker?.totalParts ?? dlData.totalParts ?? 0) : (dlData.partIndex ?? 0),
+      totalParts: dlData.totalParts ?? marker?.totalParts ?? 0,
+      bytesDownloaded: dlData.bytesDownloaded,
+      currentPartName: dlData.currentPartName ?? null,
+      failedReason:
+        dlState === 'failed' ? summarizeJobError(downloadJob?.failedReason) : undefined,
+    }
+
+    // ── Ingest sub-status ───────────────────────────────────────────────────
+    const ingState = ingestJob ? await ingestJob.getState() : undefined
+    const ingData = ingestJob?.data ?? {}
+    let ingestPhaseState = this.phaseStateFor(ingState)
+    // A pruned-but-successful ingest leaves rows + the last-updated marker and
+    // clears the download marker; reflect that as a completed ingest phase.
+    if (ingestPhaseState === 'idle' && !marker && count > 0) ingestPhaseState = 'completed'
+
+    // total_records 0 means "unknown" (e.g. a manifest rebuilt from an older
+    // marker) — resolveExpectedTotal falls back to a parts estimate, then the
+    // live row count, so the counter/%/ETA never silently vanish.
+    const expectedTotal = resolveExpectedTotal(
+      ingData.manifest?.total_records,
+      ingData.totalParts,
+      count
+    )
+    const jobRecords = ingData.recordsIngested ?? 0
+    // While ingesting, the per-job recordsIngested lags across the per-part
+    // continuation handoff (continuations run under auto jobIds). Drive the shown
+    // count from max(jobRecords, live rowCount) so a first ingest tracks the table
+    // filling 0 → ~259k, while a re-ingest into a populated table still rides the
+    // per-job counter. Outside the running phase, trust the job's own total.
+    // Always reconcile against the live row count (the per-job recordsIngested can
+    // be a partial/stale total — a completed pass-0 job only counted part 1; a
+    // failed continuation stops mid-run). WHILE RUNNING, additionally subtract the
+    // run's start-row baseline so a re-ingest into a populated table shows THIS
+    // run's progress (0 → ~259k) instead of reading ~100% from second zero.
+    // Outside running, start=0 so 'ready'/'failed' reflect total searchable rows.
+    const records =
+      ingestPhaseState === 'running'
+        ? resolveIngestRecordsShown(jobRecords, count, expectedTotal, ingData.startRowCount ?? 0)
+        : resolveIngestRecordsShown(jobRecords, count, expectedTotal)
+
+    const ingest: DrugIngestPhaseStatus = {
+      state: ingestPhaseState,
+      records,
+      expectedTotal,
+      partsDone: ingData.partIndex ?? 0,
+      totalParts: ingData.totalParts ?? marker?.totalParts ?? 0,
+      currentPartName: ingData.currentPartName ?? null,
+      failedReason: ingState === 'failed' ? summarizeJobError(ingestJob?.failedReason) : undefined,
+    }
+
+    const phase = deriveIngestPhase(download, ingest, count)
+
+    // The active phase drives elapsed/ETA: ingest start when ingesting, else the
+    // download start.
+    const startedAtMs =
+      phase === 'ingesting'
+        ? (ingData.startedAt ?? null)
+        : phase === 'downloading'
+          ? (dlData.startedAt ?? null)
+          : null
+
+    const error =
+      phase === 'failed' ? (ingest.failedReason ?? download.failedReason) : undefined
+
+    return {
+      phase,
+      download,
+      ingest,
+      startedAtMs,
+      lastUpdated: lastUpdated ?? null,
+      rowCount: count,
+      error,
+    }
+  }
+}

admin/app/services/kiwix_catalog_service.ts

@@ -0,0 +1,338 @@
+import axios from 'axios'
+import { XMLParser } from 'fast-xml-parser'
+import { DateTime } from 'luxon'
+import logger from '@adonisjs/core/services/logger'
+import InstalledResource from '#models/installed_resource'
+import { isRawListRemoteZimFilesResponse } from '../../util/zim.js'
+
+/**
+ * Local, in-process freshness check for installed content (Kiwix ZIM files +
+ * PMTiles maps). This replaces the former dependency on the external
+ * project-nomad-api `/api/v1/resources/check-updates` endpoint — every NOMAD
+ * instance now queries the upstream catalogs directly.
+ *
+ * Downloads have always gone straight to the Kiwix/GitHub mirrors regardless of
+ * who performed the check, so moving the check in-process only shifts the
+ * lightweight *catalog* lookup. To stay mirror-respectful the auto-updater gates
+ * these calls behind the update window and bounds their concurrency; sizes come
+ * from the catalog metadata so we avoid per-file HEAD requests.
+ *
+ * Robustness over the old API: ZIM lookups use the OPDS exact `name=` filter
+ * (no lossy keyword stripping) and every returned link is still validated
+ * against the authoritative `^<id>_YYYY-MM\.zim$` filename regex, so a substring
+ * match in the catalog can never resolve to the wrong book. Parsing is fully
+ * defensive — a malformed entry is skipped, never thrown.
+ */
+
+const KIWIX_CATALOG_URL = 'https://browse.library.kiwix.org/catalog/v2/entries'
+const GITHUB_PMTILES_URL =
+  'https://api.github.com/repos/Crosstalk-Solutions/project-nomad-maps/contents/pmtiles'
+
+const CATALOG_TIMEOUT_MS = 15000
+/** Bounded paginated fallback scan when the exact `name=` lookup comes up empty. */
+const KIWIX_PAGE_SIZE = 60
+const MAX_KIWIX_FETCHES = 5
+/** Concurrent ZIM catalog lookups — keep small to avoid hammering the mirror. */
+const ZIM_CHECK_CONCURRENCY = 4
+
+/** The newest available version of a single resource (a YYYY-MM date stamp). */
+export interface CatalogResult {
+  version: string
+  download_url: string
+  size_bytes: number
+}
+
+interface CatalogZimEntry {
+  name: string | null
+  download_url: string
+  file_name: string
+  size_bytes: number
+}
+
+interface GithubContentEntry {
+  name: string
+  download_url: string | null
+  size: number
+}
+
+function escapeRegex(input: string): string {
+  return input.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
+}
+
+export class KiwixCatalogService {
+  private parser = new XMLParser({
+    ignoreAttributes: false,
+    attributeNamePrefix: '',
+    textNodeName: '#text',
+  })
+
+  /**
+   * Resolve the newest available version for a batch of installed resources.
+   * Returns a map keyed by `"<resource_type>:<resource_id>"`; resources with no
+   * available newer version (or a failed lookup) are simply absent.
+   *
+   * ZIMs are checked one OPDS request each (bounded concurrency). All maps are
+   * resolved from a single GitHub directory listing.
+   */
+  async getLatestForResources(
+    resources: Array<{ resource_id: string; resource_type: 'zim' | 'map' }>
+  ): Promise<Map<string, CatalogResult>> {
+    const result = new Map<string, CatalogResult>()
+    const zims = resources.filter((r) => r.resource_type === 'zim')
+    const maps = resources.filter((r) => r.resource_type === 'map')
+
+    await this.forEachWithConcurrency(zims, ZIM_CHECK_CONCURRENCY, async (r) => {
+      try {
+        const latest = await this.getLatestZim(r.resource_id)
+        if (latest) result.set(`zim:${r.resource_id}`, latest)
+      } catch (error) {
+        logger.warn(
+          `[KiwixCatalogService] ZIM check failed for ${r.resource_id}: ${error instanceof Error ? error.message : error}`
+        )
+      }
+    })
+
+    if (maps.length > 0) {
+      try {
+        const listing = await this.fetchMapListing()
+        for (const r of maps) {
+          const latest = this.pickNewestMap(listing, r.resource_id)
+          if (latest) result.set(`map:${r.resource_id}`, latest)
+        }
+      } catch (error) {
+        logger.warn(
+          `[KiwixCatalogService] Map listing fetch failed: ${error instanceof Error ? error.message : error}`
+        )
+      }
+    }
+
+    return result
+  }
+
+  /** Newest catalog version of a single ZIM book, or null if none/older. */
+  async getLatestZim(resourceId: string): Promise<CatalogResult | null> {
+    const pattern = new RegExp(`^${escapeRegex(resourceId)}_(\\d{4}-\\d{2})\\.zim$`)
+
+    // 1. Exact-name lookup (the robust path).
+    const named = await this.fetchZimEntries({ name: resourceId, count: 50, start: 0 })
+    const exact = this.pickNewestZim(named, pattern)
+    if (exact) return exact
+
+    // 2. Fallback: bounded keyword scan in case the catalog ignored `name=` or
+    //    indexes the book under a slightly different name.
+    return this.scanZimByQuery(resourceId, pattern)
+  }
+
+  /** Newest catalog version of a single PMTiles map, or null if none/older. */
+  async getLatestMap(resourceId: string): Promise<CatalogResult | null> {
+    const listing = await this.fetchMapListing()
+    return this.pickNewestMap(listing, resourceId)
+  }
+
+  // ── ZIM internals ───────────────────────────────────────────────────────────
+
+  private pickNewestZim(entries: CatalogZimEntry[], pattern: RegExp): CatalogResult | null {
+    let latest: CatalogResult | null = null
+    for (const entry of entries) {
+      const match = entry.file_name.match(pattern)
+      if (!match) continue
+      const version = match[1]
+      if (!latest || version > latest.version) {
+        latest = { version, download_url: entry.download_url, size_bytes: entry.size_bytes }
+      }
+    }
+    return latest
+  }
+
+  private async scanZimByQuery(
+    resourceId: string,
+    pattern: RegExp
+  ): Promise<CatalogResult | null> {
+    let start = 0
+    let total = 0
+    let latest: CatalogResult | null = null
+
+    for (let i = 0; i < MAX_KIWIX_FETCHES; i++) {
+      const { entries, totalResults } = await this.fetchZimEntriesPage({
+        q: resourceId,
+        count: KIWIX_PAGE_SIZE,
+        start,
+      })
+      total = totalResults
+      if (entries.length === 0) break
+      start += entries.length
+
+      const candidate = this.pickNewestZim(entries, pattern)
+      if (candidate && (!latest || candidate.version > latest.version)) {
+        latest = candidate
+      }
+      if (start >= total) break
+    }
+    return latest
+  }
+
+  private async fetchZimEntries(params: {
+    name?: string
+    q?: string
+    count: number
+    start: number
+  }): Promise<CatalogZimEntry[]> {
+    const { entries } = await this.fetchZimEntriesPage(params)
+    return entries
+  }
+
+  private async fetchZimEntriesPage(params: {
+    name?: string
+    q?: string
+    count: number
+    start: number
+  }): Promise<{ entries: CatalogZimEntry[]; totalResults: number }> {
+    const res = await axios.get(KIWIX_CATALOG_URL, {
+      params: {
+        start: params.start,
+        count: params.count,
+        lang: 'eng',
+        ...(params.name ? { name: params.name } : {}),
+        ...(params.q ? { q: params.q } : {}),
+      },
+      responseType: 'text',
+      timeout: CATALOG_TIMEOUT_MS,
+    })
+    return this.parseZimEntries(res.data)
+  }
+
+  private parseZimEntries(xml: string): { entries: CatalogZimEntry[]; totalResults: number } {
+    let parsed: any
+    try {
+      parsed = this.parser.parse(xml)
+    } catch {
+      return { entries: [], totalResults: 0 }
+    }
+    if (!isRawListRemoteZimFilesResponse(parsed)) {
+      return { entries: [], totalResults: 0 }
+    }
+
+    const feed = parsed.feed
+    const totalResults = Number(feed?.totalResults)
+    const rawEntries = feed?.entry
+      ? Array.isArray(feed.entry)
+        ? feed.entry
+        : [feed.entry]
+      : []
+
+    const entries: CatalogZimEntry[] = []
+    for (const raw of rawEntries) {
+      if (!raw || typeof raw !== 'object') continue
+      const links = Array.isArray(raw.link) ? raw.link : raw.link ? [raw.link] : []
+      const downloadLink = links.find(
+        (link: any) =>
+          link &&
+          typeof link === 'object' &&
+          link.type === 'application/x-zim' &&
+          typeof link.href === 'string'
+      )
+      if (!downloadLink) continue
+
+      // The OPDS href ends with `.meta4`; strip it to get the real .zim URL.
+      const href: string = downloadLink.href
+      const download_url = href.endsWith('.meta4') ? href.slice(0, -'.meta4'.length) : href
+      const file_name = download_url.split('/').pop() || ''
+      if (!file_name) continue
+
+      const size_bytes = Number.parseInt(downloadLink.length, 10) || 0
+      entries.push({
+        name: typeof raw.name === 'string' ? raw.name : null,
+        download_url,
+        file_name,
+        size_bytes,
+      })
+    }
+
+    return { entries, totalResults: Number.isFinite(totalResults) ? totalResults : 0 }
+  }
+
+  // ── Map internals ────────────────────────────────────────────────────────────
+
+  private async fetchMapListing(): Promise<GithubContentEntry[]> {
+    const res = await axios.get(GITHUB_PMTILES_URL, {
+      headers: { Accept: 'application/vnd.github+json' },
+      timeout: CATALOG_TIMEOUT_MS,
+    })
+    return Array.isArray(res.data) ? res.data : []
+  }
+
+  private pickNewestMap(listing: GithubContentEntry[], resourceId: string): CatalogResult | null {
+    const pattern = new RegExp(`^${escapeRegex(resourceId)}_(\\d{4}-\\d{2})\\.pmtiles$`)
+    let latest: CatalogResult | null = null
+    for (const file of listing) {
+      if (!file || typeof file.name !== 'string' || !file.download_url) continue
+      const match = file.name.match(pattern)
+      if (!match) continue
+      const version = match[1]
+      if (!latest || version > latest.version) {
+        latest = {
+          version,
+          download_url: file.download_url,
+          size_bytes: typeof file.size === 'number' ? file.size : 0,
+        }
+      }
+    }
+    return latest
+  }
+
+  // ── Shared ───────────────────────────────────────────────────────────────────
+
+  private async forEachWithConcurrency<T>(
+    items: T[],
+    concurrency: number,
+    worker: (item: T) => Promise<void>
+  ): Promise<void> {
+    let cursor = 0
+    const runners = Array.from({ length: Math.min(concurrency, items.length) }, async () => {
+      while (cursor < items.length) {
+        const index = cursor++
+        await worker(items[index])
+      }
+    })
+    await Promise.all(runners)
+  }
+}
+
+/**
+ * Persist the latest-known available-update state onto an installed resource.
+ * Shared by the manual check and the auto-updater so both keep the cool-off
+ * anchor consistent.
+ *
+ * The first-seen anchor is reset **only** when the available version string
+ * actually changes, so a manual "Check for updates" never resets the auto
+ * cool-off clock. State is cleared entirely once the resource is current (the
+ * update got installed, or the upstream release was withdrawn).
+ */
+export async function reconcileResourceUpdateState(
+  resource: InstalledResource,
+  latest: CatalogResult | null,
+  now: DateTime
+): Promise<void> {
+  const hasUpdate = latest !== null && latest.version > resource.version
+
+  if (hasUpdate) {
+    if (resource.available_update_version !== latest!.version) {
+      resource.available_update_version = latest!.version
+      resource.available_update_first_seen_at = now
+    }
+    // Keep the cached size fresh even when the version is unchanged (the catalog
+    // may report a size it lacked on a previous check).
+    const size = latest!.size_bytes || null
+    if (resource.available_update_size_bytes !== size) {
+      resource.available_update_size_bytes = size
+    }
+  } else if (resource.available_update_version !== null) {
+    resource.available_update_version = null
+    resource.available_update_size_bytes = null
+    resource.available_update_first_seen_at = null
+  }
+
+  if (resource.$isDirty) {
+    await resource.save()
+  }
+}

admin/app/services/kiwix_library_service.ts

@@ -187,7 +187,71 @@ export class KiwixLibraryService {
       .filter((b) => b.id && b.path)
   }
 
-  async rebuildFromDisk(opts?: { excludeFilenames?: string[] }): Promise<void> {
+  /**
+   * Returns the number of books currently listed in the library XML, or 0 if the
+   * file doesn't exist yet. Used to report a before/after delta on a manual rescan.
+   */
+  async getBookCount(): Promise<number> {
+    try {
+      const content = await readFile(this.getLibraryFilePath(), 'utf-8')
+      return this._parseExistingBooks(content).length
+    } catch (err: any) {
+      if (err.code === 'ENOENT') return 0
+      throw err
+    }
+  }
+
+  /**
+   * True if the library XML parses and has a <library> root. A truncated or
+   * corrupt file (e.g. an interrupted write) fails this even though it exists,
+   * so the caller can rebuild rather than leave Kiwix serving a broken library.
+   * An empty-but-well-formed library is considered valid (nothing to repair).
+   */
+  private _isValidLibraryXml(xmlContent: string): boolean {
+    try {
+      const parser = new XMLParser({
+        ignoreAttributes: false,
+        attributeNamePrefix: '@_',
+        isArray: (name) => name === 'book',
+      })
+      const parsed = parser.parse(xmlContent)
+      return parsed?.library !== undefined && parsed?.library !== null
+    } catch {
+      return false
+    }
+  }
+
+  /**
+   * Boot-time safety net: if the library XML is missing or unparseable, rebuild
+   * it from the ZIM files on disk so Kiwix (running in library mode with
+   * --monitorLibrary) doesn't come up serving an empty/broken library with no
+   * path to recovery. This covers files lost or corrupted outside the normal
+   * download flow (storage relocation, interrupted write, manual deletion).
+   *
+   * Returns true if a rebuild was performed. Filesystem errors other than
+   * "not found" are surfaced rather than masked by a rebuild.
+   */
+  async ensureLibraryXmlHealthy(): Promise<boolean> {
+    let content: string
+    try {
+      content = await readFile(this.getLibraryFilePath(), 'utf-8')
+    } catch (err: any) {
+      if (err?.code === 'ENOENT') {
+        logger.warn('[KiwixLibraryService] Library XML missing on startup; rebuilding from disk.')
+        await this.rebuildFromDisk()
+        return true
+      }
+      throw err
+    }
+
+    if (this._isValidLibraryXml(content)) return false
+
+    logger.warn('[KiwixLibraryService] Library XML present but invalid; rebuilding from disk.')
+    await this.rebuildFromDisk()
+    return true
+  }
+
+  async rebuildFromDisk(opts?: { excludeFilenames?: string[] }): Promise<number> {
     const dirPath = join(process.cwd(), ZIM_STORAGE_PATH)
     await ensureDirectoryExists(dirPath)
 
@@ -221,6 +285,7 @@ export class KiwixLibraryService {
     const xml = this._buildXml(books)
     await this._atomicWrite(xml)
     logger.info(`[KiwixLibraryService] Rebuilt library XML with ${books.length} book(s).`)
+    return books.length
   }
 
   async addBook(filename: string): Promise<void> {

admin/app/services/map_service.ts

@@ -21,6 +21,7 @@ import logger from '@adonisjs/core/services/logger'
 import { assertNotPrivateUrl } from '#validators/common'
 import InstalledResource from '#models/installed_resource'
 import { CollectionManifestService } from './collection_manifest_service.js'
+import { decideSupersededDeletion } from '../utils/superseded_resource.js'
 import type { CollectionWithStatus, MapsSpec } from '../../types/collections.js'
 import type { Country, CountryCode, CountryGroup, MapExtractPreflight } from '../../types/maps.js'
 import {
@@ -199,10 +200,18 @@ export class MapService implements IMapService {
       const parsed = CollectionManifestService.parseMapFilename(filename)
       if (!parsed) continue
 
-      const filepath = join(process.cwd(), this.mapStoragePath, 'pmtiles', filename)
+      const pmtilesDir = join(process.cwd(), this.mapStoragePath, 'pmtiles')
+      const filepath = join(pmtilesDir, filename)
       const stats = await getFileStatsIfExists(filepath)
 
       try {
+        // Capture the prior install for this resource_id before updateOrCreate
+        // overwrites it, so we know the old file to clean up (#634).
+        const prior = await InstalledResource.query()
+          .where('resource_id', parsed.resource_id)
+          .where('resource_type', 'map')
+          .first()
+
         const { DateTime } = await import('luxon')
         await InstalledResource.updateOrCreate(
           { resource_id: parsed.resource_id, resource_type: 'map' },
@@ -215,6 +224,31 @@ export class MapService implements IMapService {
           }
         )
         logger.info(`[MapService] Created InstalledResource entry for: ${parsed.resource_id}`)
+
+        // Remove the superseded prior version's pmtiles file if every safety
+        // rail passes (see decideSupersededDeletion). Maps have no library index,
+        // so a direct delete of the recorded old file is sufficient.
+        const decision = decideSupersededDeletion({
+          existing: prior ? { file_path: prior.file_path, version: prior.version } : null,
+          newFilePath: filepath,
+          newVersion: parsed.version,
+          newFileExists: !!stats,
+          storageBaseDir: pmtilesDir,
+        })
+        if (decision.delete && decision.path) {
+          try {
+            await deleteFileIfExists(decision.path)
+            logger.info(
+              `[MapService] Removed superseded ${parsed.resource_id} file: ${decision.path}`
+            )
+          } catch (err) {
+            logger.warn(`[MapService] Failed to remove superseded file ${decision.path}:`, err)
+          }
+        } else if (decision.reason !== 'first_install' && decision.reason !== 'same_file') {
+          logger.info(
+            `[MapService] Kept prior ${parsed.resource_id} file (reason: ${decision.reason})`
+          )
+        }
       } catch (error) {
         logger.error(`[MapService] Failed to create InstalledResource for ${filename}:`, error)
       }
@@ -301,7 +335,7 @@ export class MapService implements IMapService {
       }
 
       const contentLength = response.headers['content-length']
-      const size = contentLength ? parseInt(contentLength, 10) : 0
+      const size = contentLength ? parseInt(contentLength.toString(), 10) : 0
 
       return { filename, size }
     } catch (error: any) {
@@ -370,6 +404,34 @@ export class MapService implements IMapService {
     return true
   }
 
+  /**
+   * Whether the low-zoom world basemap is present on disk. Checked directly (rather than trusting
+   * the in-process `worldBasemapReady` flag) so callers get an accurate answer regardless of whether
+   * `ensureWorldBasemap()` has run yet this process. Used to warn the user instead of showing a
+   * silent grey map when the basemap was never provisioned (e.g. installed straight offline, #1030).
+   */
+  async checkWorldBasemapExists(): Promise<boolean> {
+    const basePath = resolve(join(this.baseDirPath, 'pmtiles'))
+    const filepath = resolve(join(basePath, WORLD_BASEMAP_FILENAME))
+    if (!filepath.startsWith(basePath + sep)) return false
+    const stats = await getFileStatsIfExists(filepath)
+    const exists = !!stats && Number(stats.size) > 0
+    if (exists) this.worldBasemapReady = true
+    return exists
+  }
+
+  /**
+   * Explicitly (re)provision the world basemap on demand. Ensures base assets exist, then extracts
+   * the basemap if it isn't present yet. Requires internet — surfaced to the user as a "download the
+   * base map" action so the one network dependency can be satisfied deliberately while online (#1030).
+   */
+  async provisionWorldBasemap(): Promise<boolean> {
+    const baseAssetsExist = await this.ensureBaseAssets()
+    if (!baseAssetsExist) return false
+    await this.ensureWorldBasemap()
+    return this.checkWorldBasemapExists()
+  }
+
   /**
    * Extract a low-zoom global basemap once so the map isn't grey outside a
    * regional extract's polygon. Cheap (~15 MB, a handful of HTTP range
@@ -457,6 +519,18 @@ export class MapService implements IMapService {
     return await listDirectoryContentsRecursive(this.baseDirPath)
   }
 
+  /**
+   * Compare two map-file versions (YYYY-MM, or null for an undated legacy file). Returns >0 if
+   * `a` is newer than `b`, <0 if older, 0 if equal. A dated build is always newer than an undated
+   * legacy file; two dated builds compare lexicographically (correct for zero-padded YYYY-MM).
+   */
+  private static compareMapVersions(a: string | null, b: string | null): number {
+    if (a === b) return 0
+    if (a === null) return -1
+    if (b === null) return 1
+    return a < b ? -1 : 1
+  }
+
   private generateSourcesArray(host: string | null, regions: FileEntry[], protocol: string = 'http'): BaseStylesFile['sources'][] {
     const sources: BaseStylesFile['sources'][] = []
     const baseUrl = this.getPublicFileBaseUrl(host, 'pmtiles', protocol)
@@ -474,23 +548,47 @@ export class MapService implements IMapService {
       sources.push(worldSource)
     }
 
+    // Dedupe by region name, keeping only the newest file per region. The source name is the
+    // date-stripped region (e.g. both "washington.pmtiles" and "washington_2025-12.pmtiles" map
+    // to "washington"). Emitting both produces duplicate source keys and duplicate layer ids,
+    // which MapLibre rejects outright — blanking the ENTIRE map, not just that region. Old copies
+    // linger when a newer curated version installs (#634), so guard against it here so the style
+    // stays valid even if cleanup hasn't run. A dated build beats an undated legacy file; between
+    // two dated builds the later YYYY-MM wins (lexicographic compare is correct for that format).
+    const bestByRegion = new Map<string, { region: FileEntry; version: string | null }>()
     for (const region of regions) {
       if (region.type === 'file' && region.name.endsWith('.pmtiles')) {
-        // Strip .pmtiles and date suffix (e.g. "alaska_2025-12" -> "alaska") for stable source names
         const parsed = CollectionManifestService.parseMapFilename(region.name)
         const regionName = parsed ? parsed.resource_id : region.name.replace('.pmtiles', '')
-        const source: BaseStylesFile['sources'] = {}
+        const version = parsed?.version ?? null
-        const sourceUrl = urlJoin(baseUrl, region.name)
+        const existing = bestByRegion.get(regionName)
-
+        if (!existing || MapService.compareMapVersions(version, existing.version) > 0) {
-        source[regionName] = {
+          if (existing) {
-          type: 'vector',
+            logger.warn(
-          attribution: PMTILES_ATTRIBUTION,
+              `[MapService] Duplicate map region "${regionName}": using "${region.name}" over "${existing.region.name}" (keeping newest)`
-          url: `pmtiles://${sourceUrl}`,
+            )
+          }
+          bestByRegion.set(regionName, { region, version })
+        } else {
+          logger.warn(
+            `[MapService] Duplicate map region "${regionName}": skipping "${region.name}" in favor of "${existing.region.name}" (keeping newest)`
+          )
         }
-        sources.push(source)
       }
     }
 
+    for (const [regionName, { region }] of bestByRegion) {
+      const source: BaseStylesFile['sources'] = {}
+      const sourceUrl = urlJoin(baseUrl, region.name)
+
+      source[regionName] = {
+        type: 'vector',
+        attribution: PMTILES_ATTRIBUTION,
+        url: `pmtiles://${sourceUrl}`,
+      }
+      sources.push(source)
+    }
+
     return sources
   }
 

admin/app/services/nomad_md_service.ts

@@ -0,0 +1,51 @@
+import { rename, writeFile } from 'node:fs/promises'
+import { dirname } from 'node:path'
+import { randomUUID } from 'node:crypto'
+import app from '@adonisjs/core/services/app'
+import { ensureDirectoryExists, getFile } from '../utils/fs.js'
+
+/**
+ * Manages the user-editable `NOMAD.md` file that is injected as a system prompt
+ * during chat completions. The file lives in the admin storage directory
+ * (`/app/storage/NOMAD.md` in the container, `/opt/project-nomad/storage/NOMAD.md`
+ * on the host), so an end user can also edit it directly on disk. The file is
+ * created lazily on the first save — a missing or empty file simply means no
+ * custom prompt is injected.
+ */
+export class NomadMdService {
+  static STORAGE_PATH = 'storage/NOMAD.md'
+
+  private get filePath(): string {
+    return app.makePath(NomadMdService.STORAGE_PATH)
+  }
+
+  /**
+   * Read the raw file contents. Returns an empty string when the file does not
+   * exist yet (the editor renders a template client-side in that case).
+   */
+  async read(): Promise<string> {
+    const content = await getFile(this.filePath, 'string')
+    return content ?? ''
+  }
+
+  /**
+   * Persist the file. Written atomically (tmp file + rename) so a concurrent
+   * on-disk edit never observes a half-written file.
+   */
+  async write(content: string): Promise<void> {
+    const filePath = this.filePath
+    await ensureDirectoryExists(dirname(filePath))
+    const tmpPath = `${filePath}.tmp.${randomUUID()}`
+    await writeFile(tmpPath, content, 'utf-8')
+    await rename(tmpPath, filePath)
+  }
+
+  /**
+   * The trimmed contents suitable for injection as a system message, or `null`
+   * when the file is missing or blank (so chat behaviour is unchanged).
+   */
+  async getSystemPrompt(): Promise<string | null> {
+    const content = (await this.read()).trim()
+    return content.length > 0 ? content : null
+  }
+}

admin/app/services/ollama_service.ts

@@ -43,8 +43,14 @@ type ChatInput = {
   model: string
   messages: Array<{ role: 'system' | 'user' | 'assistant'; content: string }>
   think?: boolean | 'medium'
+  // Whether the target model supports thinking. Lets chat()/chatStream() tell "capable but
+  // disabled" (send reasoning_effort:'none') apart from "not capable" (send nothing).
+  thinkingCapable?: boolean
   stream?: boolean
   numCtx?: number
+  // Aborts the upstream request when the client disconnects, so an abandoned generation
+  // doesn't keep decoding server-side and block Ollama's single parallel slot (#1065).
+  signal?: AbortSignal
 }
 
 @inject()
@@ -54,6 +60,9 @@ export class OllamaService {
   private initPromise: Promise<void> | null = null
   private isOllamaNative: boolean | null = null
   private activeDownloads: Map<string, Promise<{ success: boolean; message: string; retryable?: boolean }>> = new Map()
+  // Memoized `thinking` capability per model name (see checkModelHasThinking). Only successful
+  // /api/show lookups are cached; transient failures are left uncached so they can be retried.
+  private thinkingCapabilityCache: Map<string, boolean> = new Map()
 
   constructor() {}
 
@@ -329,17 +338,28 @@ export class OllamaService {
     if (chatRequest.think) {
       params.think = chatRequest.think
     }
+    // The /v1 (OpenAI-compat) endpoint ignores `think`; `reasoning_effort` is the actual lever.
+    // Only touch it for thinking-capable models so non-Ollama backends never get an unexpected
+    // param. gpt-oss requires an explicit level; a capable-but-disabled model gets 'none' to
+    // suppress thinking (capable models default thinking ON otherwise, so think===true is a no-op).
+    if (chatRequest.think === 'medium') {
+      params.reasoning_effort = 'medium'
+    } else if (chatRequest.thinkingCapable && chatRequest.think === false) {
+      params.reasoning_effort = 'none'
+    }
     if (chatRequest.numCtx) {
       params.num_ctx = chatRequest.numCtx
     }
 
-    const response = await this.openai.chat.completions.create(params)
+    const response = await this.openai.chat.completions.create(params, { signal: chatRequest.signal })
     const choice = response.choices[0]
 
     return {
       message: {
         content: choice.message.content ?? '',
-        thinking: (choice.message as any).thinking ?? undefined,
+        // Ollama's OpenAI-compat endpoint (/v1) emits thinking as `reasoning`; its native
+        // shape uses `thinking`. Read both so thinking is never silently dropped (#1065).
+        thinking: (choice.message as any).thinking ?? (choice.message as any).reasoning ?? undefined,
       },
       done: true,
       model: response.model,
@@ -360,11 +380,22 @@ export class OllamaService {
     if (chatRequest.think) {
       params.think = chatRequest.think
     }
+    // The /v1 (OpenAI-compat) endpoint ignores `think`; `reasoning_effort` is the actual lever.
+    // Only touch it for thinking-capable models so non-Ollama backends never get an unexpected
+    // param. gpt-oss requires an explicit level; a capable-but-disabled model gets 'none' to
+    // suppress thinking (capable models default thinking ON otherwise, so think===true is a no-op).
+    if (chatRequest.think === 'medium') {
+      params.reasoning_effort = 'medium'
+    } else if (chatRequest.thinkingCapable && chatRequest.think === false) {
+      params.reasoning_effort = 'none'
+    }
     if (chatRequest.numCtx) {
       params.num_ctx = chatRequest.numCtx
     }
 
-    const stream = (await this.openai.chat.completions.create(params)) as unknown as Stream<ChatCompletionChunk>
+    const stream = (await this.openai.chat.completions.create(params, {
+      signal: chatRequest.signal,
+    })) as unknown as Stream<ChatCompletionChunk>
 
     // Returns how many trailing chars of `text` could be the start of `tag`
     function partialTagSuffix(tag: string, text: string): number {
@@ -383,7 +414,8 @@ export class OllamaService {
 
       for await (const chunk of stream) {
         const delta = chunk.choices[0]?.delta
-        const nativeThinking: string = (delta as any)?.thinking ?? ''
+        // /v1 emits thinking as `reasoning`; native Ollama uses `thinking`. Read both (#1065).
+        const nativeThinking: string = (delta as any)?.thinking ?? (delta as any)?.reasoning ?? ''
         const rawContent: string = delta?.content ?? ''
 
         // Parse <think> tags out of the content stream
@@ -436,13 +468,22 @@ export class OllamaService {
     await this._ensureDependencies()
     if (!this.baseUrl) return false
 
+    // A model's capabilities don't change at runtime, so memoize the /api/show result. Without
+    // this, loading the chat picker fires one /api/show per installed model and every chat send
+    // fires another — this collapses those to a single call per model per process.
+    const cached = this.thinkingCapabilityCache.get(modelName)
+    if (cached !== undefined) return cached
+
     try {
       const response = await axios.post(
         `${this.baseUrl}/api/show`,
         { model: modelName },
         { timeout: 5000 }
       )
-      return Array.isArray(response.data?.capabilities) && response.data.capabilities.includes('thinking')
+      const hasThinking =
+        Array.isArray(response.data?.capabilities) && response.data.capabilities.includes('thinking')
+      this.thinkingCapabilityCache.set(modelName, hasThinking)
+      return hasThinking
     } catch {
       // Non-Ollama backends don't expose /api/show — assume no thinking support
       return false
@@ -481,9 +522,43 @@ export class OllamaService {
    */
   public static readonly EMBED_MAX_INPUT_CHARS = 4000
 
+  /**
+   * Aggressive 2048-safe character cap, applied only on a context-length retry. nomic-embed-text:v1.5
+   * defaults to a 2048-token context, and on the OpenAI-compat fallback path (or an older Ollama that
+   * ignores num_ctx for embeddings) we cannot widen it at request time. 2000 chars stays under 2048
+   * tokens even for the densest content (~1 char/token code/markup), so an oversized chunk gets
+   * truncated-and-kept instead of silently dropped from Qdrant — and the embed job stops re-embedding
+   * the whole file 30x on the one bad chunk (#881).
+   */
+  public static readonly EMBED_CONTEXT_SAFE_CHARS = 2000
+
+  /**
+   * True if the error is the model rejecting input that exceeds its context window
+   * ("input length exceeds the context length"). Matches both the native /api/embed axios error
+   * shape and the OpenAI-compat BadRequestError. Drives the truncate-and-retry here and the
+   * non-retryable classification in EmbedFileJob (#881).
+   */
+  public static isContextLengthError(err: unknown): boolean {
+    const parts: string[] = []
+    if (err instanceof Error && err.message) parts.push(err.message)
+    const anyErr = err as any
+    const data = anyErr?.response?.data
+    if (data) parts.push(typeof data === 'string' ? data : JSON.stringify(data))
+    if (anyErr?.error) parts.push(typeof anyErr.error === 'string' ? anyErr.error : JSON.stringify(anyErr.error))
+    const haystack = parts.join(' ').toLowerCase()
+    return (
+      (haystack.includes('context length') && haystack.includes('exceed')) ||
+      haystack.includes('input length exceeds')
+    )
+  }
+
   /**
    * Generate embeddings for the given input strings.
    * Tries the Ollama native /api/embed endpoint first, falls back to /v1/embeddings.
+   *
+   * If the first attempt fails because a chunk exceeds the model's context window, retries once
+   * with an aggressive 2048-safe truncation (EMBED_CONTEXT_SAFE_CHARS) so the chunk is embedded
+   * (start-of-chunk) rather than silently dropped from Qdrant (#881).
    */
   public async embed(model: string, input: string[]): Promise<{ embeddings: number[][] }> {
     await this._ensureDependencies()
@@ -491,41 +566,49 @@ export class OllamaService {
       throw new Error('AI service is not initialized.')
     }
 
-    // Runtime safety net (#881). The OpenAI-compat fallback has no equivalent of
+    const cap = (arr: string[], max: number) => arr.map((s) => (s.length > max ? s.slice(0, max) : s))
-    // truncate:true, so a chunk that exceeds the model's loaded context_length
+
-    // (often 2048 for nomic-embed-text:v1.5) returns 400 and the chunk is silently
+    // Generous pre-cap (#881): fine for the native path (num_ctx=8192) but can still exceed a
-    // dropped from Qdrant. Pre-capping at the input layer protects both paths.
+    // 2048-context fallback on dense content. The context-length retry below is the hard backstop.
-    const safeInput = input.map((s) =>
+    const safeInput = cap(input, OllamaService.EMBED_MAX_INPUT_CHARS)
-      s.length > OllamaService.EMBED_MAX_INPUT_CHARS
-        ? s.slice(0, OllamaService.EMBED_MAX_INPUT_CHARS)
-        : s
-    )
-    const truncatedCount = input.reduce(
-      (n, s) => (s.length > OllamaService.EMBED_MAX_INPUT_CHARS ? n + 1 : n),
-      0
-    )
-    if (truncatedCount > 0) {
-      logger.debug(
-        '[OllamaService] embed: pre-capped %d/%d inputs at %d chars',
-        truncatedCount,
-        input.length,
-        OllamaService.EMBED_MAX_INPUT_CHARS
-      )
-    }
 
     try {
-      // Prefer Ollama native endpoint (supports batch input natively).
+      return await this._embedWithFallback(model, safeInput)
-      // Pass num_ctx explicitly so we don't depend on the embedding model's
+    } catch (err) {
-      // modelfile defaults. Some installs ship nomic-embed-text:v1.5 with
+      if (!OllamaService.isContextLengthError(err)) throw err
-      // num_ctx=2048, which our chunker (sized for ~1500 tokens) can exceed
+      // One or more chunks exceeded the model's context even after the pre-cap — typically an
-      // on dense content, causing "input length exceeds context length" errors.
+      // older Ollama that ignores num_ctx for embeddings, or the OpenAI-compat fallback path.
-      // truncate:true is a runtime safety net for any chunk that still overshoots.
+      // Retry once, truncated hard enough to fit a 2048-token context at any density, so the
-      // 8192 matches nomic-embed-text:v1.5's RoPE-extrapolated max.
+      // chunk is embedded (truncated) instead of dropped and the job doesn't storm.
+      const hardCapped = cap(input, OllamaService.EMBED_CONTEXT_SAFE_CHARS)
+      const reduced = hardCapped.reduce((n, s, i) => (s.length < safeInput[i].length ? n + 1 : n), 0)
+      logger.warn(
+        '[OllamaService] embed: context-length overflow; retrying %d/%d inputs hard-capped at %d chars',
+        reduced,
+        input.length,
+        OllamaService.EMBED_CONTEXT_SAFE_CHARS
+      )
+      return await this._embedWithFallback(model, hardCapped)
+    }
+  }
+
+  /**
+   * Single embed attempt: native /api/embed first, then the OpenAI-compat /v1/embeddings fallback.
+   * Both paths request num_ctx/truncate (Ollama's OpenAI-compat shim forwards them). A context-length
+   * error from the native path is re-thrown rather than falling back, because the fallback has a
+   * smaller effective context and would only fail the same way — the caller (embed) retries it
+   * truncated instead.
+   */
+  private async _embedWithFallback(model: string, input: string[]): Promise<{ embeddings: number[][] }> {
+    try {
+      // Pass num_ctx explicitly so we don't depend on the embedding model's modelfile defaults.
+      // Some installs ship nomic-embed-text:v1.5 with num_ctx=2048; 8192 matches its RoPE-extrapolated
+      // max. truncate:true is a server-side net for any chunk that still overshoots.
       const response = await axios.post(
         `${this.baseUrl}/api/embed`,
         {
           model,
-          input: safeInput,
+          input,
           truncate: true,
           options: { num_ctx: 8192 },
         },
@@ -538,22 +621,26 @@ export class OllamaService {
       }
       return { embeddings: response.data.embeddings }
     } catch (err) {
-      // Capture the original error so we know *why* we fell back. Earlier bare
+      // Let context-length errors bubble so embed() can retry with a smaller cap; the fallback
-      // catches here masked recurring "input length exceeds context length"
+      // endpoint (smaller effective context, no num_ctx honored on older Ollama) can't help here.
-      // failures for months (#369, #670, #881) — without this log we have no
+      if (OllamaService.isContextLengthError(err)) throw err
-      // signal that /api/embed is the broken path vs the fallback.
+      // Log the original error so we know *why* we fell back. Earlier bare catches here masked
+      // recurring failures for months (#369, #670, #881).
       logger.warn(
         '[OllamaService] /api/embed failed, falling back to /v1/embeddings: %s',
         err instanceof Error ? err.message : String(err)
       )
-      // Fall back to OpenAI-compatible /v1/embeddings.
+      // Fall back to OpenAI-compatible /v1/embeddings. Explicitly request float format — some
-      // Explicitly request float format — some backends (e.g. LM Studio) don't reliably
+      // backends (e.g. LM Studio) don't reliably implement the base64 the OpenAI SDK defaults to.
-      // implement the base64 encoding the OpenAI SDK requests by default.
+      // truncate/num_ctx are forwarded by Ollama's OpenAI-compat shim; the SDK types omit them,
-      const results = await this.openai.embeddings.create({
+      // hence the cast. We only ever talk to a local Ollama here, not real OpenAI.
+      const results = await this.openai!.embeddings.create({
         model,
-        input: safeInput,
+        input,
         encoding_format: 'float',
-      })
+        truncate: true,
+        options: { num_ctx: 8192 },
+      } as any)
       return { embeddings: results.data.map((e) => e.embedding as number[]) }
     }
   }
@@ -718,7 +805,7 @@ export class OllamaService {
   ): Promise<{ models: NomadOllamaModel[]; hasMore: boolean } | null> {
     try {
       const models = await this.retrieveAndRefreshModels(sort, force)
-      if (!models) {
+      if (!models || models.length === 0) {
         logger.warn(
           '[OllamaService] Returning fallback recommended models due to failure in fetching available models'
         )
@@ -773,7 +860,10 @@ export class OllamaService {
     try {
       if (!force) {
         const cachedModels = await this.readModelsFromCache()
-        if (cachedModels) {
+        // An empty cached array (e.g. written from a transient empty upstream
+        // response) must not be treated as valid data — fall through to a
+        // fresh fetch and, failing that, the fallback list.
+        if (cachedModels && cachedModels.length > 0) {
           logger.info('[OllamaService] Using cached available models data')
           return this.sortModels(cachedModels, sort)
         }
@@ -786,7 +876,7 @@ export class OllamaService {
       const baseUrl = env.get('NOMAD_API_URL') || NOMAD_API_DEFAULT_BASE_URL
       const fullUrl = new URL(NOMAD_MODELS_API_PATH, baseUrl).toString()
 
-      const response = await axios.get(fullUrl)
+      const response = await axios.get(fullUrl, { timeout: 10000 })
       if (!response.data || !Array.isArray(response.data.models)) {
         logger.warn(
           `[OllamaService] Invalid response format when fetching available models: ${JSON.stringify(response.data)}`
@@ -803,6 +893,16 @@ export class OllamaService {
         }))
         .filter((model) => model.tags.length > 0)
 
+      // A successful-but-empty upstream response (0 models, or all filtered out
+      // as cloud-only) is a soft failure: return null so the caller serves the
+      // fallback list, and don't poison the 24h cache with an empty array.
+      if (noCloud.length === 0) {
+        logger.warn(
+          '[OllamaService] Nomad API returned no usable (non-cloud) models; using fallback'
+        )
+        return null
+      }
+
       await this.writeModelsToCache(noCloud)
       return this.sortModels(noCloud, sort)
     } catch (error) {

admin/app/services/queue_service.ts

@@ -1,11 +1,11 @@
 import { Queue } from 'bullmq'
 import queueConfig from '#config/queue'
 
-// Process-wide singleton. Each `Queue` opens two ioredis connections (one for
+// Process-wide singleton. Instantiating a fresh QueueService per dispatch /
-// commands, one blocking). Instantiating a fresh QueueService per dispatch /
+// status lookup leaks connections, and under sustained job churn (e.g.
-// status lookup leaks both, and under sustained job churn (e.g. multi-batch ZIM
+// multi-batch ZIM ingestion enqueueing a continuation every few seconds) it
-// ingestion enqueueing a continuation every few seconds) it saturates Redis's
+// saturates Redis's maxclients within hours. All queues additionally reuse the
-// maxclients within hours.
+// single shared ioredis instance exported from #config/queue (#885).
 export class QueueService {
   private queues: Map<string, Queue> = new Map()
 

admin/app/services/rag_service.ts

@@ -10,6 +10,7 @@ import { createWorker } from 'tesseract.js'
 import { fromBuffer } from 'pdf2pic'
 import JSZip from 'jszip'
 import * as cheerio from 'cheerio'
+import mammoth from 'mammoth'
 import { OllamaService } from './ollama_service.js'
 import { SERVICE_NAMES } from '../../constants/service_names.js'
 import { removeStopwords } from 'stopword'
@@ -18,6 +19,7 @@ import { join, resolve, sep } from 'node:path'
 import KVStore from '#models/kv_store'
 import KbIngestState from '#models/kb_ingest_state'
 import { decideScanAction, type IngestPolicy } from '../utils/kb_ingest_decision.js'
+import { decideContentReindex, type ReindexOutcome } from '../utils/content_reindex_decision.js'
 import KbRatioRegistry from '#models/kb_ratio_registry'
 import { decideWarnings } from '../utils/kb_warning_decision.js'
 import type { FileWarning, FileWarningsResult, StoredFileInfo } from '../../types/rag.js'
@@ -43,6 +45,10 @@ export class RagService {
   private qdrantInitPromise: Promise<void> | null = null
   private embeddingModelVerified = false
   private resolvedEmbeddingModel: string | null = null
+  // Collections already verified this session (created + payload indexes in place).
+  // Skips the getCollections/createPayloadIndex round-trips that otherwise run on
+  // every embed call — ~45% of per-document Qdrant time on large ingestions (#1129)
+  private ensuredCollections = new Set<string>()
   public static UPLOADS_STORAGE_PATH = 'storage/kb_uploads'
   public static CONTENT_COLLECTION_NAME = 'nomad_knowledge_base'
   public static EMBEDDING_DIMENSION = 768 // Nomic Embed Text v1.5 dimension is 768
@@ -92,6 +98,8 @@ export class RagService {
     } catch {
       this.qdrant = null
       this.qdrantInitPromise = null
+      // Qdrant may have restarted (or been recreated) — re-verify collections on reconnect
+      this.ensuredCollections.clear()
       return {
         online: false,
         message: 'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.',
@@ -111,6 +119,11 @@ export class RagService {
   ) {
     try {
       await this._ensureDependencies()
+
+      if (this.ensuredCollections.has(collectionName)) {
+        return
+      }
+
       const collections = await this.qdrant!.getCollections()
       const collectionExists = collections.collections.some((col) => col.name === collectionName)
 
@@ -132,6 +145,13 @@ export class RagService {
         field_name: 'content_type',
         field_schema: 'keyword',
       })
+      await this.qdrant!.createPayloadIndex(collectionName, {
+        field_name: 'collection',
+        field_schema: 'keyword',
+      })
+
+      // Only memoize after every step succeeded, so a partial failure is retried
+      this.ensuredCollections.add(collectionName)
     } catch (error) {
       logger.error('Error ensuring Qdrant collection:', error)
       throw error
@@ -509,7 +529,8 @@ export class RagService {
     filepath: string,
     deleteAfterEmbedding: boolean,
     batchOffset?: number,
-    onProgress?: (percent: number) => Promise<void>
+    onProgress?: (percent: number) => Promise<void>,
+    collection?: string
   ): Promise<ProcessZIMFileResponse> {
     const zimExtractionService = new ZIMExtractionService()
 
@@ -536,6 +557,9 @@ export class RagService {
       const result = await this.embedAndStoreText(zimChunk.text, {
         source: filepath,
         content_type: 'zim_article',
+        // Without this the ZIM path writes points with no `collection` at all, so
+        // getKnowledgeCollections() (which facets on it) never sees them.
+        ...(collection ? { collection } : {}),
 
         // Article-level context
         article_title: zimChunk.articleTitle,
@@ -610,6 +634,16 @@ export class RagService {
     return await this.extractTXTText(fileBuffer)
   }
 
+  /**
+   * Extract text content from a DOCX file using mammoth. DOCX is a ZIP-based
+   * XML format, so raw-text extraction (extractTXTText) would return garbage —
+   * this parses the document XML properly and returns clean plain text.
+   */
+  private async processDocxFile(fileBuffer: Buffer): Promise<string> {
+    const { value: text } = await mammoth.extractRawText({ buffer: fileBuffer })
+    return text
+  }
+
   /**
    * Extract text content from an EPUB file.
    * EPUBs are ZIP archives containing XHTML content files.
@@ -694,14 +728,16 @@ export class RagService {
     extractedText: string,
     filepath: string,
     deleteAfterEmbedding: boolean = false,
-    onProgress?: (percent: number) => Promise<void>
+    onProgress?: (percent: number) => Promise<void>,
+    collection?: string
   ): Promise<{ success: boolean; message: string; chunks?: number }> {
     if (!extractedText || extractedText.trim().length === 0) {
       return { success: false, message: 'Process completed succesfully, but no text was found to embed.' }
     }
 
     const embedResult = await this.embedAndStoreText(extractedText, {
-      source: filepath
+      source: filepath,
+      ...(collection ? { collection } : {})
     }, onProgress)
 
     if (!embedResult) {
@@ -731,7 +767,8 @@ export class RagService {
     filepath: string,
     deleteAfterEmbedding: boolean = false,
     batchOffset?: number,
-    onProgress?: (percent: number) => Promise<void>
+    onProgress?: (percent: number) => Promise<void>,
+    collection?: string
   ): Promise<ProcessAndEmbedFileResponse> {
     try {
       const fileType = determineFileType(filepath)
@@ -750,7 +787,7 @@ export class RagService {
       // Process based on file type
       // ZIM files are handled specially since they have their own embedding workflow
       if (fileType === 'zim') {
-        return await this.processZIMFile(filepath, deleteAfterEmbedding, batchOffset, onProgress)
+        return await this.processZIMFile(filepath, deleteAfterEmbedding, batchOffset, onProgress, collection)
       }
 
       // Extract text based on file type
@@ -764,6 +801,9 @@ export class RagService {
         case 'pdf':
           extractedText = await this.processPDFFile(fileBuffer!)
           break
+        case 'docx':
+          extractedText = await this.processDocxFile(fileBuffer!)
+          break
         case 'epub':
           extractedText = await this.processEPUBFile(fileBuffer!)
           break
@@ -780,7 +820,7 @@ export class RagService {
         : undefined
 
       // Embed extracted text and cleanup
-      return await this.embedTextAndCleanup(extractedText, filepath, deleteAfterEmbedding, scaledProgress)
+      return await this.embedTextAndCleanup(extractedText, filepath, deleteAfterEmbedding, scaledProgress, collection)
     } catch (error) {
       logger.error('[RAG] Error processing and embedding file:', error)
       return { success: false, message: 'Error processing and embedding file.' }
@@ -799,7 +839,8 @@ export class RagService {
   public async searchSimilarDocuments(
     query: string,
     limit: number = 5,
-    scoreThreshold: number = 0.3 // Lower default threshold - was 0.7, now 0.3
+    scoreThreshold: number = 0.3, // Lower default threshold - was 0.7, now 0.3
+    collection?: string
   ): Promise<Array<{ text: string; score: number; metadata?: Record<string, any> }>> {
     try {
       logger.debug(`[RAG] Starting similarity search for query: "${query}"`)
@@ -872,6 +913,7 @@ export class RagService {
         limit: searchLimit,
         score_threshold: scoreThreshold,
         with_payload: true,
+        ...(collection ? { filter: { must: [{ key: 'collection', match: { value: collection } }] } } : {}),
       })
 
       logger.debug(`[RAG] Found ${searchResults.length} results above threshold ${scoreThreshold}`)
@@ -1011,6 +1053,29 @@ export class RagService {
           }
         }
 
+        // Boost when query keywords match the chunk's section/article heading. ZIM
+        // content carries this structural metadata (already fetched, no extra cost),
+        // and a query term appearing in a heading is a strong relevance signal that
+        // body-text matching alone misses. Same conservative, score-scaled, diminishing
+        // -returns shape as the boosts above, so it can't promote a weak match.
+        const headingText = [result.full_title, result.section_title, result.article_title]
+          .filter(Boolean)
+          .join(' ')
+          .toLowerCase()
+        if (headingText) {
+          const headingHits = queryKeywords.filter((kw) =>
+            headingText.includes(kw.toLowerCase())
+          ).length
+          if (headingHits > 0) {
+            const headingRatio = headingHits / Math.max(queryKeywords.length, 1)
+            const headingBoost = Math.sqrt(headingRatio) * 0.1 * result.score
+            logger.debug(
+              `[RAG] Heading match: ${headingHits}/${queryKeywords.length} - Boost: ${headingBoost.toFixed(3)}`
+            )
+            finalScore += headingBoost
+          }
+        }
+
         finalScore = Math.min(1.0, finalScore + keywordBoost)
 
         return {
@@ -1097,14 +1162,15 @@ export class RagService {
       // in particular) have no row to attach to. The state machine is the
       // authoritative "what's on disk?" view; Qdrant is "what made it into
       // the vector store?". Both are needed to render the KB UI honestly.
-      const stateByPath = new Map<string, { state: KbIngestStateValue; chunks_embedded: number }>()
+      const stateByPath = new Map<string, { state: KbIngestStateValue; chunks_embedded: number; collection: string | null }>()
       try {
-        const stateRows = await KbIngestState.query().select('file_path', 'state', 'chunks_embedded')
+        const stateRows = await KbIngestState.query().select('file_path', 'state', 'chunks_embedded', 'collection')
         for (const row of stateRows) {
           sources.add(row.file_path)
           stateByPath.set(row.file_path, {
             state: row.state,
             chunks_embedded: row.chunks_embedded,
+            collection: row.collection,
           })
         }
       } catch (error) {
@@ -1116,20 +1182,201 @@ export class RagService {
         )
       }
 
-      return Array.from(sources).map((source) => {
+      const uploadsAbsPath = resolve(join(process.cwd(), RagService.UPLOADS_STORAGE_PATH))
-        const row = stateByPath.get(source)
+      return await Promise.all(
-        return {
+        Array.from(sources).map(async (source) => {
-          source,
+          const row = stateByPath.get(source)
-          state: row?.state ?? null,
+          const fileName = source.split(/[/\\]/).at(-1) ?? source
-          chunksEmbedded: row?.chunks_embedded ?? 0,
+          const isUserUpload = resolve(source).startsWith(uploadsAbsPath + sep)
-        }
+          const stats = await getFileStatsIfExists(source)
-      })
+          return {
+            source,
+            state: row?.state ?? null,
+            chunksEmbedded: row?.chunks_embedded ?? 0,
+            fileName,
+            size: stats?.size ?? null,
+            uploadedAt: stats?.modifiedTime.toISOString() ?? null,
+            isUserUpload,
+            collection: row?.collection ?? null,
+          }
+        })
+      )
     } catch (error) {
       logger.error('Error retrieving stored files:', error)
       return []
     }
   }
 
+  /**
+   * Enumerate distinct `collection` values currently in the knowledge base,
+   * for populating a subject-picker in the upload/chat UI. Mirrors the
+   * `source` facet pattern used elsewhere in this file (see getStoredFiles).
+   */
+  public async getKnowledgeCollections(): Promise<string[]> {
+    await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION)
+    const facetResult = await this.qdrant!.facet(RagService.CONTENT_COLLECTION_NAME, {
+      key: 'collection',
+      limit: RagService.FACET_SOURCE_LIMIT,
+      exact: true,
+    })
+    const collections = new Set<string>()
+    for (const hit of facetResult.hits) {
+      if (typeof hit.value === 'string') collections.add(hit.value)
+    }
+    return Array.from(collections).sort()
+  }
+
+  /**
+   * Reassign a stored file's collection after the fact. Updates the `collection`
+   * payload field on every existing Qdrant point for this source in place (no
+   * re-chunking or re-embedding needed), then mirrors the change onto the
+   * KbIngestState row so getStoredFiles() reflects it immediately.
+   */
+  public async updateFileCollection(
+    source: string,
+    collection: string | null
+  ): Promise<{ success: boolean; message: string }> {
+    try {
+      await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION)
+
+      await this.qdrant!.setPayload(RagService.CONTENT_COLLECTION_NAME, {
+        payload: { collection },
+        filter: { must: [{ key: 'source', match: { value: source } }] },
+      })
+
+      // The setPayload above only reaches points that already exist, so for a file
+      // that has not been indexed yet it matches nothing. The row is therefore the
+      // only durable record of the choice until EmbedFileJob picks it up -- create
+      // it when absent rather than reporting success and storing the value nowhere.
+      const row = await KbIngestState.getOrCreate(source)
+      row.collection = collection
+      await row.save()
+
+      return { success: true, message: collection ? `Moved to "${collection}".` : 'Moved to Uncategorized.' }
+    } catch (error) {
+      logger.error('[RAG] Error updating file collection:', error)
+      return { success: false, message: 'Error updating file collection.' }
+    }
+  }
+
+  /**
+   * Rename a knowledge-base collection everywhere it's referenced: updates every
+   * Qdrant point tagged with the old name in place, and mirrors the change onto
+   * any matching KbIngestState rows so getStoredFiles() reflects it immediately.
+   */
+  public async renameKnowledgeCollection(
+    oldName: string,
+    newName: string
+  ): Promise<{ success: boolean; message: string }> {
+    try {
+      if (!oldName || !newName || oldName === newName) {
+        return { success: false, message: 'Invalid collection names.' }
+      }
+      await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION)
+
+      await this.qdrant!.setPayload(RagService.CONTENT_COLLECTION_NAME, {
+        payload: { collection: newName },
+        filter: { must: [{ key: 'collection', match: { value: oldName } }] },
+      })
+
+      await KbIngestState.query().where('collection', oldName).update({ collection: newName })
+
+      return { success: true, message: `Renamed "${oldName}" to "${newName}".` }
+    } catch (error) {
+      logger.error('[RAG] Error renaming knowledge collection:', error)
+      return { success: false, message: 'Error renaming collection.' }
+    }
+  }
+
+  /**
+   * Remove a collection by reassigning every file tagged with it back to
+   * Uncategorized (collection: null). Non-destructive — no files or embeddings
+   * are deleted, only the grouping label is cleared so items can be
+   * recategorized later.
+   */
+  public async deleteKnowledgeCollection(
+    name: string
+  ): Promise<{ success: boolean; message: string }> {
+    try {
+      if (!name) {
+        return { success: false, message: 'Invalid collection name.' }
+      }
+      await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION)
+
+      await this.qdrant!.setPayload(RagService.CONTENT_COLLECTION_NAME, {
+        payload: { collection: null },
+        filter: { must: [{ key: 'collection', match: { value: name } }] },
+      })
+
+      await KbIngestState.query().where('collection', name).update({ collection: null })
+
+      return { success: true, message: `"${name}" removed. Files moved to Uncategorized.` }
+    } catch (error) {
+      logger.error('[RAG] Error deleting knowledge collection:', error)
+      return { success: false, message: 'Error deleting collection.' }
+    }
+  }
+
+  /**
+   * Resolve a stored-file `source` to an absolute disk path, but only if the
+   * path lives under the uploads directory. Mirrors the docs_service traversal
+   * guard: resolve, then require the resolved path to be strictly inside the
+   * base + path separator (so siblings of `kb_uploads` can't slip through).
+   * Returns null for anything else — ZIMs, admin docs, README, or paths
+   * outside the app entirely. The viewer/download endpoints lean on this so
+   * they don't need to repeat the check.
+   */
+  private resolveUploadPath(source: string): string | null {
+    const uploadsAbsPath = resolve(join(process.cwd(), RagService.UPLOADS_STORAGE_PATH))
+    const resolved = resolve(source)
+    if (!resolved.startsWith(uploadsAbsPath + sep)) return null
+    return resolved
+  }
+
+  private static readonly VIEWABLE_TEXT_EXTENSIONS: ReadonlySet<string> = new Set([
+    'md', 'txt', 'csv', 'json', 'yaml', 'yml', 'toml', 'xml', 'html',
+  ])
+
+  /**
+   * Read the text content of a user-uploaded file for in-browser viewing.
+   * Returns null when the source is outside uploads, missing, or not a
+   * recognized text extension. The extension allowlist is intentionally narrow
+   * — PDFs/EPUBs/ZIMs round-trip through Download, not the viewer.
+   */
+  public async readFileContent(
+    source: string
+  ): Promise<{ content: string; extension: string; fileName: string } | null> {
+    const resolved = this.resolveUploadPath(source)
+    if (!resolved) return null
+
+    const extension = resolved.split('.').at(-1)?.toLowerCase() ?? ''
+    if (!RagService.VIEWABLE_TEXT_EXTENSIONS.has(extension)) return null
+
+    const stats = await getFileStatsIfExists(resolved)
+    if (!stats) return null
+
+    try {
+      const { readFile } = await import('node:fs/promises')
+      const content = await readFile(resolved, 'utf-8')
+      const fileName = resolved.split(/[/\\]/).at(-1) ?? resolved
+      return { content, extension, fileName }
+    } catch (error) {
+      logger.warn({ err: error, source }, '[RagService.readFileContent] read failed')
+      return null
+    }
+  }
+
+  /**
+   * Resolve a download-target path for a stored upload. Returns null if the
+   * source isn't an upload or the file is missing on disk.
+   */
+  public async resolveDownloadPath(source: string): Promise<string | null> {
+    const resolved = this.resolveUploadPath(source)
+    if (!resolved) return null
+    const stats = await getFileStatsIfExists(resolved)
+    return stats ? resolved : null
+  }
+
   /**
    * Compute whether the first-chat JIT prompt should fire and surface the file
    * count the banner uses in its copy ("Index your N existing files?"). The
@@ -1219,9 +1466,14 @@ export class RagService {
         const fileSizeBytes = sizeByPath.get(source) ?? 0
         const chunksInQdrant = chunksBySource.get(source) ?? 0
         const fileName = source.split(/[/\\]/).pop() ?? source
+        // ignoreCatchAll: the partial_stall warning must only fire when the
+        // registry has a *specific* expectation for this file. The empty-pattern
+        // fallback (100 chunks/MB) over-predicts wildly for atypical ZIMs that
+        // are mostly PDFs/images/link-outs (e.g. military-medicine), producing
+        // false "ingestion stalled" warnings. Suppress Warning B in that case. (#913)
         const expectedChunks =
           fileSizeBytes > 0
-            ? await KbRatioRegistry.estimateChunks(fileName, fileSizeBytes)
+            ? await KbRatioRegistry.estimateChunks(fileName, fileSizeBytes, { ignoreCatchAll: true })
             : null
 
         const warnings = decideWarnings({ fileSizeBytes, chunksInQdrant, expectedChunks })
@@ -1281,6 +1533,98 @@ export class RagService {
     }
   }
 
+  /**
+   * Reconcile the knowledge base after a curated content file (a ZIM) is
+   * replaced on disk by a newer downloaded version. Called from
+   * `RunDownloadJob.onComplete` once the new file is written and the old file
+   * has been deleted from disk.
+   *
+   * The decision (see `decideContentReindex` for the exhaustive, tested
+   * contract) mirrors the REPLACED file's prior indexed state instead of the
+   * global `rag.defaultIngestPolicy`. On a content update the user has already
+   * chosen whether this content belongs in the AI knowledge base, so we honor
+   * that choice in both directions:
+   *
+   *   1. (caller already deleted the outdated file from disk)
+   *   2. Qdrant not installed          → no-op (no knowledge base exists)
+   *   3. Old file WAS indexed + Qdrant
+   *      running                       → delete ONLY the old file's points
+   *                                       (filter `source == oldFilePath`), drop
+   *                                       its ingest-state row, and queue the new
+   *                                       file for embedding — BYPASSING the
+   *                                       Always/Manual policy on purpose.
+   *   4. Old file NOT indexed          → no-op (respect the prior un-indexed /
+   *                                       browse-only choice; do NOT auto-embed
+   *                                       even under an Always policy)
+   *   5. Old indexed but Qdrant NOT
+   *      currently running             → no-op. We can't remove the stale points,
+   *                                       and a queued embed job could be cleared
+   *                                       before Qdrant returns. Acting half-way is
+   *                                       wasteful, so we defer entirely. Accepted
+   *                                       tradeoff: the old file's points linger in
+   *                                       Qdrant until a future re-index; they are
+   *                                       NOT auto-reaped here.
+   *
+   * Point deletion is exact: ZIM chunks are stored with `source` equal to the
+   * full file path (see embedAndStoreText callers), so filtering on the old path
+   * can only ever remove the replaced file's own points.
+   *
+   * Returns the decision outcome for logging/tests; never throws on a Qdrant
+   * hiccup mid-reindex (logged and surfaced as `qdrant_not_running` semantics by
+   * the caller's try/catch).
+   */
+  public async reconcileReplacedContentFile(params: {
+    oldFilePath: string
+    newFilePath: string
+    fileName: string
+  }): Promise<ReindexOutcome> {
+    const { oldFilePath, newFilePath, fileName } = params
+
+    const isReplacement = !!oldFilePath && oldFilePath !== newFilePath
+
+    // Step 2: is the knowledge base even installed? Short-circuits before any
+    // KB-state lookup, per the spec ordering.
+    const qdrantInstalled = isReplacement
+      ? !!(await this.dockerService.getServiceURL(SERVICE_NAMES.QDRANT))
+      : false
+
+    // Steps 3/4: was the OUTDATED file actually indexed? The state row is the
+    // authoritative signal (RFC #883) — chunk presence alone can't distinguish
+    // a fully-indexed file from a stalled ingestion.
+    let oldFileWasIndexed = false
+    if (isReplacement && qdrantInstalled) {
+      const oldState = await KbIngestState.query().where('file_path', oldFilePath).first()
+      oldFileWasIndexed = oldState?.state === 'indexed'
+    }
+
+    // Step 5: only check liveness once we know we'd otherwise act. The health
+    // check is a real network round-trip, so we avoid it on the common no-op
+    // paths above.
+    let qdrantRunning = false
+    if (isReplacement && qdrantInstalled && oldFileWasIndexed) {
+      qdrantRunning = (await this.checkQdrantHealth()).online
+    }
+
+    const outcome = decideContentReindex({
+      isReplacement,
+      qdrantInstalled,
+      oldFileWasIndexed,
+      qdrantRunning,
+    })
+
+    if (outcome === 'reindex') {
+      // Order matters: remove the stale points and state row BEFORE queueing the
+      // new embed so a fresh index can't be conflated with the old one. Each step
+      // targets only `oldFilePath` / `newFilePath` — never another resource.
+      await this._deletePointsBySource(oldFilePath)
+      await KbIngestState.remove(oldFilePath)
+      const { EmbedFileJob } = await import('#jobs/embed_file_job')
+      await EmbedFileJob.dispatch({ fileName, filePath: newFilePath })
+    }
+
+    return outcome
+  }
+
   public async discoverNomadDocs(force?: boolean): Promise<{ success: boolean; message: string }> {
     try {
       const README_PATH = join(process.cwd(), 'README.md')
@@ -1780,6 +2124,10 @@ export class RagService {
         logger.warn(`[RAG] deleteCollection failed (may not exist): ${(err as Error).message}`)
       }
 
+      // The collection is gone — drop it from the ensured cache so the
+      // _ensureCollection call below actually recreates it
+      this.ensuredCollections.delete(RagService.CONTENT_COLLECTION_NAME)
+
       await this._ensureCollection(
         RagService.CONTENT_COLLECTION_NAME,
         RagService.EMBEDDING_DIMENSION

admin/app/services/system_service.ts

@@ -1,4 +1,5 @@
 import Service from '#models/service'
+import InstalledResource from '#models/installed_resource'
 import { inject } from '@adonisjs/core'
 import { DockerService } from '#services/docker_service'
 import { ServiceSlim } from '../../types/services.js'
@@ -21,6 +22,7 @@ import KVStore from '#models/kv_store'
 import { KV_STORE_SCHEMA, KVStoreKey } from '../../types/kv_store.js'
 import { isNewerVersion } from '../utils/version.js'
 import { invalidateAssistantNameCache } from '../../config/inertia.js'
+import { KiwixLibraryService } from '#services/kiwix_library_service'
 
 @inject()
 export class SystemService {
@@ -35,29 +37,54 @@ export class SystemService {
   }
 
   async getInternetStatus(): Promise<boolean> {
-    const DEFAULT_TEST_URL = 'https://1.1.1.1/cdn-cgi/trace'
+    // Primary endpoint stays Cloudflare's privacy-respecting utility endpoint.
+    // The fallbacks are hosts the application already contacts elsewhere
+    // (GitHub API for update checks, the Project NOMAD API for release-note
+    // subscriptions), so no new third-party services are introduced. They exist
+    // to avoid false "offline" reports on networks that block 1.1.1.1.
+    const DEFAULT_TEST_URLS = [
+      'https://1.1.1.1/cdn-cgi/trace',
+      'https://api.github.com',
+      'https://api.projectnomad.us',
+    ]
     const MAX_ATTEMPTS = 3
 
-    let testUrl = DEFAULT_TEST_URL
+    let testUrls = DEFAULT_TEST_URLS
-    let customTestUrl = env.get('INTERNET_STATUS_TEST_URL')?.trim()
 
-    // check that customTestUrl is a valid URL, if provided
+    // Resolve the test endpoint in priority order: the INTERNET_STATUS_TEST_URL
+    // env var always wins (legacy override for operators who intentionally point
+    // connectivity checks at a specific endpoint), then the UI-configurable value
+    // stored in KVStore, and finally the built-in defaults.
+    const envTestUrl = env.get('INTERNET_STATUS_TEST_URL')?.trim()
+    const kvTestUrl = (await KVStore.getValue('system.internetStatusTestUrl'))?.trim()
+    const customTestUrl = envTestUrl || kvTestUrl
+
+    // If a custom test URL is provided and valid, use it exclusively.
     if (customTestUrl && customTestUrl !== '') {
       try {
         new URL(customTestUrl)
-        testUrl = customTestUrl
+        testUrls = [customTestUrl]
       } catch (error) {
         logger.warn(
-          `Invalid INTERNET_STATUS_TEST_URL: ${customTestUrl}. Falling back to default URL.`
+          `Invalid internet status test URL: ${customTestUrl}. Falling back to default URLs.`
         )
       }
     }
 
     for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
       try {
-        const res = await axios.get(testUrl, { timeout: 5000 })
+        // Probe all test endpoints in parallel and resolve as soon as the first one
-        return res.status === 200
+        // responds. Any HTTP response (including non-2xx) means we reached the
+        // internet, so accept all status codes rather than requiring a strict 200.
+        await Promise.any(
+          testUrls.map((testUrl) => {
+            logger.debug(`[SystemService] Checking internet connectivity via: ${testUrl}`)
+            return axios.get(testUrl, { timeout: 5000, validateStatus: () => true })
+          })
+        )
+        return true
       } catch (error) {
+        // Promise.any only rejects (with an AggregateError) when every endpoint failed.
         logger.warn(
           `Internet status check attempt ${attempt}/${MAX_ATTEMPTS} failed: ${error instanceof Error ? error.message : error}`
         )
@@ -302,15 +329,26 @@ export class SystemService {
         'installed',
         'installation_status',
         'ui_location',
+        'custom_url',
         'friendly_name',
         'description',
         'icon',
         'powered_by',
         'display_order',
         'container_image',
-        'available_update_version'
+        'available_update_version',
+        'auto_update_enabled',
+        'is_custom',
+        'is_user_modified',
+        'is_deprecated',
+        'category'
       )
       .where('is_dependency_service', false)
+      // Deprecated/sunset apps stay visible only while still installed, so the user can manage and
+      // uninstall them — they never reappear in the install catalog once removed.
+      .where((q) => {
+        q.where('is_deprecated', false).orWhere('installed', true)
+      })
     if (installedOnly) {
       query.where('installed', true)
     }
@@ -334,10 +372,16 @@ export class SystemService {
         installation_status: service.installation_status,
         status: status ? status.status : 'unknown',
         ui_location: service.ui_location || '',
+        custom_url: service.custom_url,
         powered_by: service.powered_by,
         display_order: service.display_order,
         container_image: service.container_image,
         available_update_version: service.available_update_version,
+        auto_update_enabled: service.auto_update_enabled,
+        is_custom: service.is_custom,
+        is_user_modified: service.is_user_modified,
+        is_deprecated: service.is_deprecated,
+        category: service.category,
       })
     }
 
@@ -714,6 +758,28 @@ export class SystemService {
       this.checkLatestVersion().catch(() => null),
     ])
 
+    // Diagnostics tied to common support cases: storage relocation (#1050),
+    // container/updater issues (#858), GPU passthrough (#755/#878), and the
+    // auto-update trilogy. All best-effort so a single failure never blanks the
+    // whole bundle.
+    const [dockerVersion, hostStorageRoot, kiwixBookCount, gpuType] = await Promise.all([
+      this.dockerService.docker
+        .version()
+        .then((v: any) => v?.Version ?? null)
+        .catch(() => null),
+      this.dockerService.getHostStorageRoot().catch(() => null),
+      new KiwixLibraryService().getBookCount().catch(() => null),
+      KVStore.getValue('gpu.type').catch(() => null),
+    ])
+    const [autoUpdateCore, autoUpdateApps, autoUpdateContent, autoDisabledReason] =
+      await Promise.all([
+        KVStore.getValue('autoUpdate.enabled').catch(() => null),
+        KVStore.getValue('appAutoUpdate.enabled').catch(() => null),
+        KVStore.getValue('contentAutoUpdate.enabled').catch(() => null),
+        KVStore.getValue('autoUpdate.autoDisabledReason').catch(() => null),
+      ])
+    const isEnabled = (v: any) => v === true || v === 'true'
+
     const lines: string[] = [
       'Project NOMAD Debug Info',
       '========================',
@@ -722,7 +788,7 @@ export class SystemService {
     ]
 
     if (systemInfo) {
-      const { cpu, mem, os, disk, fsSize, uptime, graphics } = systemInfo
+      const { cpu, mem, os, disk, fsSize, uptime, graphics, gpuHealth } = systemInfo
 
       lines.push('')
       lines.push('System:')
@@ -730,6 +796,7 @@ export class SystemService {
       if (os.hostname) lines.push(`  Hostname: ${os.hostname}`)
       if (os.kernel) lines.push(`  Kernel: ${os.kernel}`)
       if (os.arch) lines.push(`  Architecture: ${os.arch}`)
+      if (dockerVersion) lines.push(`  Docker Engine: ${dockerVersion}`)
       if (uptime?.uptime) lines.push(`  Uptime: ${this._formatUptime(uptime.uptime)}`)
 
       lines.push('')
@@ -751,6 +818,10 @@ export class SystemService {
       } else {
         lines.push('  GPU: None detected')
       }
+      if (gpuHealth?.status) {
+        const vendor = gpuType || gpuHealth.gpuVendor
+        lines.push(`  GPU Passthrough: ${gpuHealth.status}${vendor ? ` (${vendor})` : ''}`)
+      }
 
       // Disk info — try disk array first, fall back to fsSize
       const diskEntries = disk.filter((d) => d.totalSize > 0)
@@ -771,6 +842,20 @@ export class SystemService {
       }
     }
 
+    lines.push('')
+    lines.push('Storage:')
+    lines.push(`  Host storage root: ${hostStorageRoot ?? 'unknown'}`)
+    lines.push(`  Container path: ${DockerService.ADMIN_STORAGE_DEST}`)
+    const storageEnv = process.env.NOMAD_STORAGE_PATH
+    lines.push(
+      `  NOMAD_STORAGE_PATH: ${storageEnv ? storageEnv : 'not set (auto-detected from admin mount)'}`
+    )
+    if (kiwixBookCount !== null) {
+      lines.push(
+        `  Kiwix library: ${kiwixBookCount === 0 ? 'empty (0 books)' : `${kiwixBookCount} book(s)`}`
+      )
+    }
+
     const installed = services.filter((s) => s.installed)
     lines.push('')
     if (installed.length > 0) {
@@ -794,6 +879,15 @@ export class SystemService {
       lines.push(`Update Available: ${updateMsg}`)
     }
 
+    lines.push('')
+    lines.push('Auto-Update:')
+    lines.push(`  Core: ${isEnabled(autoUpdateCore) ? 'Enabled' : 'Disabled'}`)
+    lines.push(`  Apps: ${isEnabled(autoUpdateApps) ? 'Enabled' : 'Disabled'}`)
+    lines.push(`  Content: ${isEnabled(autoUpdateContent) ? 'Enabled' : 'Disabled'}`)
+    if (autoDisabledReason) {
+      lines.push(`  Auto-disabled reason: ${autoDisabledReason}`)
+    }
+
     return lines.join('\n')
   }
 
@@ -826,6 +920,31 @@ export class SystemService {
     if (key === 'ai.assistantCustomName') {
       invalidateAssistantNameCache()
     }
+    // Re-enabling auto-update after a backoff-driven auto-disable clears the
+    // failure state so it gets a fresh start instead of immediately re-tripping.
+    if (key === 'autoUpdate.enabled' && (value === true || value === 'true')) {
+      await KVStore.setValue('autoUpdate.consecutiveFailures', '0')
+      await KVStore.clearValue('autoUpdate.autoDisabledReason')
+    }
+    // Re-enabling the global app auto-update master switch clears every app's
+    // per-app failure backoff so previously self-disabled apps get a fresh start.
+    if (key === 'appAutoUpdate.enabled' && (value === true || value === 'true')) {
+      await Service.query().update({
+        auto_update_consecutive_failures: 0,
+        auto_update_disabled_reason: null,
+      })
+    }
+    // Re-enabling content auto-update clears the feature-level backoff and every
+    // resource's per-resource backoff so previously self-disabled content gets a
+    // fresh start.
+    if (key === 'contentAutoUpdate.enabled' && (value === true || value === 'true')) {
+      await KVStore.setValue('contentAutoUpdate.consecutiveFailures', '0')
+      await KVStore.clearValue('contentAutoUpdate.autoDisabledReason')
+      await InstalledResource.query().update({
+        auto_update_consecutive_failures: 0,
+        auto_update_disabled_reason: null,
+      })
+    }
   }
 
   /**
@@ -928,4 +1047,83 @@ export class SystemService {
         }
       })
   }
+
+  /**
+   * Check whether the host has enough free memory and disk to comfortably run an app.
+   * Returns an array of human-readable warning strings; an empty array means no concerns.
+   * These are advisory only — the caller decides whether to block or warn.
+   */
+  async checkResourceWarnings(minMemoryMB: number, minDiskMB: number): Promise<string[]> {
+    const warnings: string[] = []
+
+    try {
+      const mem = await si.mem()
+      const availableMB = Math.floor(mem.available / 1024 / 1024)
+      if (availableMB < minMemoryMB) {
+        warnings.push(
+          `Low memory: ${availableMB} MB available, this app recommends at least ${minMemoryMB} MB free.`
+        )
+      }
+    } catch (err: any) {
+      logger.warn(`[SystemService] checkResourceWarnings mem check failed: ${err.message}`)
+    }
+
+    try {
+      const storagePath = env.get('NOMAD_STORAGE_PATH', '/opt/project-nomad/storage')
+      const fsSizes = await si.fsSize()
+      // Find the filesystem whose mount point is the longest prefix of storagePath
+      const fs = fsSizes
+        .filter((f) => storagePath.startsWith(f.mount))
+        .sort((a, b) => b.mount.length - a.mount.length)[0]
+
+      if (fs) {
+        const availableDiskMB = Math.floor((fs.size - fs.used) / 1024 / 1024)
+        if (availableDiskMB < minDiskMB) {
+          warnings.push(
+            `Low disk space: ${availableDiskMB} MB available on ${fs.mount}, this app recommends at least ${minDiskMB} MB free.`
+          )
+        }
+      }
+    } catch (err: any) {
+      logger.warn(`[SystemService] checkResourceWarnings disk check failed: ${err.message}`)
+    }
+
+    return warnings
+  }
+
+  /**
+   * Return the next suggested host port for a custom app in the 8600+ range.
+   * Looks at existing custom service records and all Docker container port bindings.
+   */
+  async getNextSuggestedCustomPort(): Promise<number> {
+    const CUSTOM_PORT_START = 8600
+    const occupied = new Set<number>()
+
+    try {
+      // Ports used by existing custom services in the DB
+      const customServices = await Service.query().where('is_custom', true)
+      for (const svc of customServices) {
+        const config = svc.container_config ? JSON.parse(svc.container_config) : null
+        const bindings = config?.HostConfig?.PortBindings ?? {}
+        for (const binding of Object.values(bindings) as any[]) {
+          const port = parseInt(binding?.[0]?.HostPort, 10)
+          if (!isNaN(port)) occupied.add(port)
+        }
+      }
+
+      // Ports used by any running Docker container in the 8600+ range
+      const containers = await this.dockerService.docker.listContainers({ all: true })
+      for (const c of containers) {
+        for (const p of c.Ports) {
+          if (p.PublicPort && p.PublicPort >= CUSTOM_PORT_START) occupied.add(p.PublicPort)
+        }
+      }
+    } catch (err: any) {
+      logger.warn(`[SystemService] getNextSuggestedCustomPort probe failed: ${err.message}`)
+    }
+
+    let candidate = CUSTOM_PORT_START
+    while (occupied.has(candidate)) candidate += 10
+    return candidate
+  }
 }

admin/app/services/system_update_service.ts

@@ -18,9 +18,18 @@ export class SystemUpdateService {
   private static LOG_FILE = join(SystemUpdateService.SHARED_DIR, 'update-log')
 
   /**
-   * Requests a system update by creating a request file that the sidecar will detect
+   * Requests a system update by creating a request file that the sidecar will detect.
+   *
+   * @param options.targetTag - Explicit Docker image tag to install (e.g. "v1.33.2").
+   *   When omitted, falls back to the cached `system.latestVersion` (manual-update
+   *   behavior). Auto-update passes an eligibility-vetted tag here, which may differ
+   *   from `system.latestVersion` when the newest release is a major bump.
+   * @param options.requester - Identifier recorded in the request file for auditing.
    */
-  async requestUpdate(): Promise<{ success: boolean; message: string }> {
+  async requestUpdate(options?: {
+    targetTag?: string
+    requester?: string
+  }): Promise<{ success: boolean; message: string }> {
     try {
       const currentStatus = this.getUpdateStatus()
       if (currentStatus && !['idle', 'complete', 'error'].includes(currentStatus.stage)) {
@@ -30,13 +39,18 @@ export class SystemUpdateService {
         }
       }
 
-      // Determine the Docker image tag to install.
+      // Determine the Docker image tag to install. Prefer an explicit caller-supplied
-      const latestVersion = await KVStore.getValue('system.latestVersion')
+      // tag; otherwise use the cached latest version.
+      let targetTag = options?.targetTag
+      if (!targetTag) {
+        const latestVersion = await KVStore.getValue('system.latestVersion')
+        targetTag = latestVersion ? `v${latestVersion}` : 'latest'
+      }
 
       const requestData = {
         requested_at: new Date().toISOString(),
-        requester: 'admin-api',
+        requester: options?.requester ?? 'admin-api',
-        target_tag: latestVersion ? `v${latestVersion}` : 'latest',
+        target_tag: targetTag,
       }
 
       await writeFile(SystemUpdateService.REQUEST_FILE, JSON.stringify(requestData, null, 2))

admin/app/services/zim_extraction_service.ts

@@ -1,6 +1,7 @@
 import { Archive, Entry } from '@openzim/libzim'
 import * as cheerio from 'cheerio'
 import { HTML_SELECTORS_TO_REMOVE, NON_CONTENT_HEADING_PATTERNS } from '../../constants/zim_extraction.js'
+import { extractStructuredContent } from '../utils/zim_html.js'
 import logger from '@adonisjs/core/services/logger'
 import { ExtractZIMChunkingStrategy, ExtractZIMContentOptions, ZIMContentChunk, ZIMArchiveMetadata } from '../../types/zim.js'
 import { randomUUID } from 'node:crypto'
@@ -115,7 +116,7 @@ export class ZIMExtractionService {
                 let chunks: ZIMContentChunk[]
 
                 if (strategy === 'structured') {
-                    const structured = this.extractStructuredContent(html)
+                    const structured = extractStructuredContent(html)
                     chunks = structured.sections.map(s => ({
                         text: s.text,
                         articleTitle,
@@ -210,76 +211,6 @@ export class ZIMExtractionService {
         }
     }
 
-    private extractStructuredContent(html: string) {
-        const $ = cheerio.load(html);
-
-        const title = $('h1').first().text().trim() || $('title').text().trim();
-
-        // Extract sections with their headings and heading levels
-        const sections: Array<{ heading: string; text: string; level: number }> = [];
-        let currentSection = { heading: 'Introduction', content: [] as string[], level: 2 };
-
-        // Walk the full DOM rather than only direct children of <body>. Modern ZIMs (Devdocs,
-        // Wikipedia, FreeCodeCamp, etc.) wrap article content in a container div, which under
-        // .children() would be a single non-heading/non-paragraph element and yield zero sections.
-        $('body').find('h2, h3, h4, p, ul, ol, dl, table').each((_, element) => {
-            const $el = $(element);
-            const tagName = element.tagName?.toLowerCase();
-
-            if (['h2', 'h3', 'h4'].includes(tagName)) {
-                // Save current section if it has content
-                if (currentSection.content.length > 0) {
-                    sections.push({
-                        heading: currentSection.heading,
-                        text: currentSection.content.join(' ').replace(/\s+/g, ' ').trim(),
-                        level: currentSection.level,
-                    });
-                }
-                // Start new section
-                const level = parseInt(tagName.substring(1)); // Extract number from h2, h3, h4
-                currentSection = {
-                    heading: $el.text().replace(/\[edit\]/gi, '').trim(),
-                    content: [],
-                    level,
-                };
-            } else if (['p', 'ul', 'ol', 'dl', 'table'].includes(tagName)) {
-                const text = $el.text().trim();
-                if (text.length > 0) {
-                    currentSection.content.push(text);
-                }
-            }
-        });
-
-        // Push the last section if it has content
-        if (currentSection.content.length > 0) {
-            sections.push({
-                heading: currentSection.heading,
-                text: currentSection.content.join(' ').replace(/\s+/g, ' ').trim(),
-                level: currentSection.level,
-            });
-        }
-
-        // Fallback: if the selector walk produced no sections but the body has meaningful
-        // text (unusual structure, minimal markup), emit one section with the full body text
-        // so the article still contributes to the knowledge base.
-        if (sections.length === 0) {
-            const bodyText = $('body').text().replace(/\s+/g, ' ').trim();
-            if (bodyText.length > 0) {
-                sections.push({
-                    heading: title || 'Content',
-                    text: bodyText,
-                    level: 2,
-                });
-            }
-        }
-
-        return {
-            title,
-            sections,
-            fullText: sections.map(s => `${s.heading}\n${s.text}`).join('\n\n'),
-        };
-    }
-
     private hasStructuredHeadings(html: string): boolean {
         const $ = cheerio.load(html);
 

admin/app/services/zim_service.ts

@@ -8,6 +8,7 @@ import * as cheerio from 'cheerio'
 import { XMLParser } from 'fast-xml-parser'
 import { isRawListRemoteZimFilesResponse, isRawRemoteZimFileEntry } from '../../util/zim.js'
 import { findReplacedWikipediaFiles } from '../utils/zim_filename.js'
+import { decideSupersededDeletion } from '../utils/superseded_resource.js'
 import logger from '@adonisjs/core/services/logger'
 import { DockerService } from './docker_service.js'
 import { inject } from '@adonisjs/core'
@@ -24,13 +25,19 @@ import vine from '@vinejs/vine'
 import { wikipediaOptionsFileSchema } from '#validators/curated_collections'
 import WikipediaSelection from '#models/wikipedia_selection'
 import InstalledResource from '#models/installed_resource'
+import CollectionManifest from '#models/collection_manifest'
 import { RunDownloadJob } from '#jobs/run_download_job'
+import { DownloadDrugDataJob } from '#jobs/download_drug_data_job'
+import { DrugReferenceService } from './drug_reference_service.js'
 import { SERVICE_NAMES } from '../../constants/service_names.js'
 import { CollectionManifestService } from './collection_manifest_service.js'
+import { KiwixCatalogService } from './kiwix_catalog_service.js'
 import { KiwixLibraryService } from './kiwix_library_service.js'
 import type { CategoryWithStatus } from '../../types/collections.js'
 import CustomLibrarySource from '#models/custom_library_source'
 import { assertNotPrivateUrl } from '#validators/common'
+import { resolveZimDownload } from '../utils/zim_download_resolution.js'
+import { getHostedContentHeaders } from '../utils/hosted_content_auth.js'
 
 const ZIM_MIME_TYPES = ['application/x-zim', 'application/x-openzim', 'application/octet-stream']
 const WIKIPEDIA_OPTIONS_URL = 'https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/collections/wikipedia.json'
@@ -162,6 +169,13 @@ export class ZimService {
           download_url,
           author: entry.author.name,
           file_name,
+          language: entry.language,
+          category: entry.category,
+          tags: entry.tags,
+          article_count: entry.articleCount,
+          media_count: entry.mediaCount,
+          publisher: entry.publisher?.name,
+          issued: entry['dc:issued'],
         })
       }
 
@@ -208,7 +222,6 @@ export class ZimService {
       filepath,
       timeout: 30000,
       allowedMimeTypes: ZIM_MIME_TYPES,
-      forceNew: true,
       filetype: 'zim',
       title: metadata?.title,
       totalBytes: metadata?.size_bytes,
@@ -251,40 +264,80 @@ export class ZimService {
 
     const allResources = CollectionManifestService.resolveTierResources(tier, category.tiers)
 
-    // Filter out already installed
+    // Filter out already installed. Includes 'dataset' rows (the FDA drug labels)
-    const installed = await InstalledResource.query().where('resource_type', 'zim')
+    // alongside 'zim' so an installed dataset is filtered out here — without it,
+    // a re-select of an installed Medicine tier would re-dispatch the ~1.7 GB drug
+    // download every time (the dataset branch's own getIngestStatus guard is a
+    // second line of defence, but this keeps the filter symmetric with the row-
+    // driven tier-status math).
+    const installed = await InstalledResource.query().whereIn('resource_type', ['zim', 'dataset'])
     const installedIds = new Set(installed.map((r) => r.resource_id))
     const toDownload = allResources.filter((r) => !installedIds.has(r.id))
 
     if (toDownload.length === 0) return null
 
+    const latestByResource = await new KiwixCatalogService().getLatestForResources(
+      toDownload.map((resource) => ({ resource_id: resource.id, resource_type: 'zim' }))
+    )
     const downloadFilenames: string[] = []
 
     for (const resource of toDownload) {
-      const existingJob = await RunDownloadJob.getActiveByUrl(resource.url)
+      // A `dataset` resource (e.g. the FDA drug labels) is DB-ingested, not a
-      if (existingJob) {
+      // ZIM file — route it to the drug download+ingest pipeline instead of
-        logger.warn(`[ZimService] Download already in progress for ${resource.url}, skipping.`)
+      // RunDownloadJob. The install-state row written on ingest 'ready' (below,
+      // via resourceMeta) is what the installed-filter above and the tier-status
+      // math key off; until that row exists the getIngestStatus guard prevents
+      // re-dispatching the ~1.7 GB download on every tier select.
+      if (resource.type === 'dataset') {
+        const drugReferenceService = new DrugReferenceService()
+        const status = await drugReferenceService.getIngestStatus()
+        if (status.phase === 'ready' || status.rowCount > 0) {
+          logger.info('[ZimService] Drug dataset already ingested, skipping dispatch.')
+          continue
+        }
+        // DownloadDrugDataJob.dispatch() is idempotent on its deterministic
+        // jobId — a concurrent in-flight download returns "already running"
+        // without re-adding, so this is safe to call repeatedly. The resourceMeta
+        // is threaded through download → ingest so the final ingest pass writes
+        // the `installed_resources` 'dataset' row, making the tier read installed.
+        await DownloadDrugDataJob.dispatch(true, {
+          resourceId: resource.id,
+          version: resource.version,
+          collectionRef: categorySlug,
+        })
+        logger.info('[ZimService] Dispatched drug data download for dataset resource.')
         continue
       }
 
-      const filename = resource.url.split('/').pop()
+      const resolved = resolveZimDownload(
+        resource,
+        latestByResource.get(`zim:${resource.id}`) ?? null
+      )
+      const existingJob = await RunDownloadJob.getActiveByUrl(resolved.url)
+      if (existingJob) {
+        logger.warn(`[ZimService] Download already in progress for ${resolved.url}, skipping.`)
+        continue
+      }
+
+      const filename = resolved.url.split('/').pop()
       if (!filename) continue
 
       downloadFilenames.push(filename)
       const filepath = join(process.cwd(), ZIM_STORAGE_PATH, filename)
 
       await RunDownloadJob.dispatch({
-        url: resource.url,
+        url: resolved.url,
         filepath,
         timeout: 30000,
         allowedMimeTypes: ZIM_MIME_TYPES,
-        forceNew: true,
         filetype: 'zim',
         title: (resource as any).title || undefined,
-        totalBytes: (resource as any).size_mb ? (resource as any).size_mb * 1024 * 1024 : undefined,
+        totalBytes: resolved.sizeBytes,
+        // Undefined for every ungated resource, so the existing flow is untouched.
+        requestHeaders: getHostedContentHeaders(resource),
         resourceMetadata: {
           resource_id: resource.id,
-          version: resource.version,
+          version: resolved.version,
           collection_ref: categorySlug,
         },
       })
@@ -358,6 +411,8 @@ export class ZimService {
     }
 
     // Create InstalledResource entries for downloaded files
+    const zimStorageDir = join(process.cwd(), ZIM_STORAGE_PATH)
+    let removedSupersededZim = false
     for (const url of urls) {
       // Skip Wikipedia files (managed separately)
       if (url.includes('wikipedia_en_')) continue
@@ -368,10 +423,17 @@ export class ZimService {
       const parsed = CollectionManifestService.parseZimFilename(filename)
       if (!parsed) continue
 
-      const filepath = join(process.cwd(), ZIM_STORAGE_PATH, filename)
+      const filepath = join(zimStorageDir, filename)
       const stats = await getFileStatsIfExists(filepath)
 
       try {
+        // Capture the prior install for this resource_id BEFORE updateOrCreate
+        // overwrites it, so we know the old file path to clean up (#634).
+        const prior = await InstalledResource.query()
+          .where('resource_id', parsed.resource_id)
+          .where('resource_type', 'zim')
+          .first()
+
         const { DateTime } = await import('luxon')
         await InstalledResource.updateOrCreate(
           { resource_id: parsed.resource_id, resource_type: 'zim' },
@@ -384,10 +446,196 @@ export class ZimService {
           }
         )
         logger.info(`[ZimService] Created InstalledResource entry for: ${parsed.resource_id}`)
+
+        // Remove the superseded prior version's file if (and only if) every
+        // safety rail passes — see decideSupersededDeletion. The InstalledResource
+        // row already points at the new file, so we delete the old file directly
+        // (NOT via this.delete(), which would drop the row by resource_id).
+        const decision = decideSupersededDeletion({
+          existing: prior ? { file_path: prior.file_path, version: prior.version } : null,
+          newFilePath: filepath,
+          newVersion: parsed.version,
+          newFileExists: !!stats,
+          storageBaseDir: zimStorageDir,
+        })
+        if (decision.delete && decision.path) {
+          try {
+            await deleteFileIfExists(decision.path)
+            removedSupersededZim = true
+            logger.info(
+              `[ZimService] Removed superseded ${parsed.resource_id} file: ${decision.path}`
+            )
+          } catch (err) {
+            logger.warn(`[ZimService] Failed to remove superseded file ${decision.path}:`, err)
+          }
+        } else if (decision.reason !== 'first_install' && decision.reason !== 'same_file') {
+          logger.info(
+            `[ZimService] Kept prior ${parsed.resource_id} file (reason: ${decision.reason})`
+          )
+        }
       } catch (error) {
         logger.error(`[ZimService] Failed to create InstalledResource for ${filename}:`, error)
       }
     }
+
+    // If we removed any superseded ZIM, rebuild the Kiwix library so its XML no
+    // longer references the deleted file. The earlier rebuild in this flow ran
+    // while both versions were still on disk.
+    if (removedSupersededZim) {
+      try {
+        await new KiwixLibraryService().rebuildFromDisk()
+        logger.info('[ZimService] Rebuilt Kiwix library after removing superseded ZIM(s).')
+      } catch (err) {
+        logger.error('[ZimService] Failed to rebuild Kiwix library after cleanup:', err)
+      }
+    }
+  }
+
+  /**
+   * Rebuilds the kiwix library XML from whatever ZIM files are currently on disk.
+   *
+   * This is the manual counterpart to the automatic rebuilds that run after a
+   * download or delete. It exists for the sideload case: a user copies a .zim file
+   * onto the box (USB, SSH, network share) outside the download flow, and kiwix has
+   * no way to discover it without regenerating the library index.
+   *
+   * In library mode (--monitorLibrary) kiwix-serve hot-reloads the XML on its own, so
+   * no restart is needed. Only legacy glob-mode containers are restarted to pick up
+   * the change. Returns the book count before and after plus the number added.
+   */
+  async rescanLibrary(): Promise<{ before: number; after: number; added: number }> {
+    const kiwixLibraryService = new KiwixLibraryService()
+    const before = await kiwixLibraryService.getBookCount()
+    const after = await kiwixLibraryService.rebuildFromDisk()
+
+    const isLegacy = await this.dockerService.isKiwixOnLegacyConfig()
+    if (isLegacy) {
+      logger.info('[ZimService] Kiwix in legacy mode — restarting container after rescan.')
+      await this.dockerService
+        .affectContainer(SERVICE_NAMES.KIWIX, 'restart')
+        .catch((error) => {
+          logger.error('[ZimService] Failed to restart KIWIX container after rescan:', error)
+        })
+    }
+
+    return { before, after, added: Math.max(0, after - before) }
+  }
+
+  async registerLocalUpload(filename: string): Promise<{ added: number }> {
+    let added = 0
+    try {
+      const result = await this.rescanLibrary()
+      added = result.added
+    } catch (err) {
+      logger.error('[ZimService] Failed to rebuild kiwix library after local upload:', err)
+    }
+
+    const parsed = CollectionManifestService.parseZimFilename(filename)
+    if (parsed) {
+      const filepath = join(process.cwd(), ZIM_STORAGE_PATH, filename)
+      const stats = await getFileStatsIfExists(filepath)
+      try {
+        const { DateTime } = await import('luxon')
+        await InstalledResource.updateOrCreate(
+          { resource_id: parsed.resource_id, resource_type: 'zim' },
+          {
+            version: parsed.version,
+            url: `local-upload://${filename}`,
+            file_path: filepath,
+            file_size_bytes: stats ? Number(stats.size) : null,
+            installed_at: DateTime.now(),
+          }
+        )
+      } catch (error) {
+        logger.error(`[ZimService] Failed to create InstalledResource for ${filename}:`, error)
+      }
+    }
+
+    // If the uploaded file matches a known Wikipedia option, mark it as installed
+    try {
+      const manifest = await CollectionManifest.find('wikipedia')
+      if (manifest) {
+        const spec = manifest.spec_data as { options: Array<{ id: string; url: string | null }> }
+        const matchedOption = spec.options.find(
+          (opt) => opt.url && opt.url.split('/').pop() === filename
+        )
+        if (matchedOption && matchedOption.url) {
+          const existing = await WikipediaSelection.query().first()
+          if (existing) {
+            existing.option_id = matchedOption.id
+            existing.url = matchedOption.url
+            existing.filename = filename
+            existing.status = 'installed'
+            await existing.save()
+          } else {
+            await WikipediaSelection.create({
+              option_id: matchedOption.id,
+              url: matchedOption.url,
+              filename,
+              status: 'installed',
+            })
+          }
+          logger.info(`[ZimService] Marked Wikipedia option '${matchedOption.id}' as installed from local upload`)
+
+          // Remove any other wikipedia_en_*.zim files, same as the download flow
+          const allFiles = await this.list()
+          const staleWikipediaFiles = allFiles.files.filter(
+            (f) => f.name.startsWith('wikipedia_en_') && f.name !== filename
+          )
+          for (const stale of staleWikipediaFiles) {
+            try {
+              await this.delete(stale.name)
+              logger.info(`[ZimService] Deleted stale Wikipedia file after upload: ${stale.name}`)
+            } catch (err) {
+              logger.warn(`[ZimService] Could not delete stale Wikipedia file: ${stale.name}`, err)
+            }
+          }
+        }
+      }
+    } catch (error) {
+      logger.error(`[ZimService] Failed to update WikipediaSelection for ${filename}:`, error)
+    }
+
+    const ollamaUrl = await this.dockerService.getServiceURL('nomad_ollama')
+    if (ollamaUrl) {
+      // Respect the global ingest policy, same as the post-download path (PR #919).
+      // This used to dispatch unconditionally, so a user who deliberately chose
+      // Manual still got sideloaded ZIMs embedded behind their back.
+      //
+      // Reuses decideScanAction rather than re-inlining the Always/Manual check,
+      // so an existing browse_only or pending_decision row is honored too instead
+      // of being overridden by the act of re-uploading the file.
+      const filePath = join(process.cwd(), ZIM_STORAGE_PATH, filename)
+      try {
+        const { default: KVStore } = await import('#models/kv_store')
+        const { default: KbIngestState } = await import('#models/kb_ingest_state')
+        const { decideScanAction } = await import('../utils/kb_ingest_decision.js')
+
+        // Unset is treated as Always, preserving legacy behavior — mirrors
+        // rag_service.ts and run_download_job.ts.
+        const policyRaw = await KVStore.getValue('rag.defaultIngestPolicy')
+        const policy = policyRaw === 'Manual' ? 'Manual' : 'Always'
+
+        const existing = await KbIngestState.findBy('file_path', filePath)
+        const action = decideScanAction(existing, false, policy)
+
+        if (action.kind === 'dispatch') {
+          const { EmbedFileJob } = await import('#jobs/embed_file_job')
+          await EmbedFileJob.dispatch({ fileName: filename, filePath })
+        } else if (action.kind === 'create_pending') {
+          // firstOrCreate so the KB panel surfaces the per-file Index affordance
+          // without demoting a row that already exists.
+          await KbIngestState.getOrCreate(filePath)
+        }
+        // 'skip' and 'backfill_indexed' need no action here: the file was just
+        // written to disk, so there is nothing to backfill and a settled state
+        // row means the user has already decided about this file.
+      } catch (error) {
+        logger.error(`[ZimService] KB ingest decision failed after local upload:`, error)
+      }
+    }
+
+    return { added }
   }
 
   async delete(file: string): Promise<void> {
@@ -426,6 +674,21 @@ export class ZimService {
         .delete()
       logger.info(`[ZimService] Deleted InstalledResource entry for: ${parsed.resource_id}`)
     }
+
+    // If this file was the active Wikipedia selection, clear the selection
+    try {
+      const selection = await WikipediaSelection.query().first()
+      if (selection && selection.filename === fileName) {
+        selection.option_id = 'none'
+        selection.status = 'none'
+        selection.filename = null
+        selection.url = null
+        await selection.save()
+        logger.info(`[ZimService] Cleared WikipediaSelection after deleting ${fileName}`)
+      }
+    } catch (error) {
+      logger.error(`[ZimService] Failed to clear WikipediaSelection after deleting ${fileName}:`, error)
+    }
   }
 
   // Wikipedia selector methods
@@ -565,7 +828,6 @@ export class ZimService {
       filepath,
       timeout: 30000,
       allowedMimeTypes: ZIM_MIME_TYPES,
-      forceNew: true,
       filetype: 'zim',
       title: selectedOption.name,
       totalBytes: selectedOption.size_mb ? selectedOption.size_mb * 1024 * 1024 : undefined,

admin/app/utils/affirmative_remedies.ts

@@ -0,0 +1,23 @@
+import KVStore from '#models/kv_store'
+
+/**
+ * Affirmative-content gate for the drug-reference feature (upstream #1040).
+ *
+ * The verbatim FDA label search and the condition→OTC matching are grounded in
+ * regulated label text and ship with the tier. The hand-authored self-care and
+ * herbal REMEDY content is guidance we author, so it stays gated off by default
+ * until a clinician has done the content pass; the maintainer flips this on in a
+ * follow-up once that's signed off.
+ *
+ * Independent of the tier install-state (a `medicine-standard` install alone does
+ * NOT enable remedies). Defaults off — a missing/null KV value reads as false —
+ * and there is deliberately no user-facing toggle, so un-reviewed medical
+ * guidance can't be self-enabled.
+ *
+ * Read at every HTTP boundary that could emit remedy data (the drug-reference
+ * page prop and the conditions show / drugs API), so no affirmative content is
+ * serialized to the client while the gate is closed.
+ */
+export async function affirmativeRemediesEnabled(): Promise<boolean> {
+  return (await KVStore.getValue('drugReference.remediesEnabled')) === true
+}

admin/app/utils/amd_hsa_override.ts

@@ -0,0 +1,52 @@
+/**
+ * Map an AMD GPU's gfx target to the `HSA_OVERRIDE_GFX_VERSION` value the ollama:rocm
+ * container needs, or `null` when the card is discovered natively and no override should
+ * be applied.
+ *
+ * This is intentionally a pure function so the mapping is unit-testable without
+ * constructing the Docker service or touching the container runtime. `DockerService`
+ * delegates its private `_mapGfxToHsaOverride` to this.
+ *
+ * The bundled `ollama/ollama:rocm` rocblas ships kernels for a fixed allowlist — as seen
+ * in ollama's own startup log:
+ *   supported=[gfx1030, gfx1100/1101/1102, gfx1150/1151, gfx1200/1201, gfx908/90a/942/950]
+ * A target NOT in that list is dropped to CPU unless we coerce it onto a supported one via
+ * HSA_OVERRIDE_GFX_VERSION.
+ *
+ * Mapping:
+ *  - gfx1030 / gfx1100 / gfx1101 / gfx1102 → none. Discrete RDNA 2/3 on the allowlist;
+ *    forcing an override here breaks GPU discovery.
+ *  - gfx1150 / gfx1151 (Strix 890M, Strix Halo) → none. RDNA 3.5 iGPUs that ARE on the
+ *    allowlist under the bundled ROCm, so native discovery works. (#1076 got this right.)
+ *  - gfx1103 (Phoenix/Hawk Point 780M/760M) → '11.0.0'. RDNA 3 iGPU that is NOT on the
+ *    allowlist, so it must be coerced onto gfx1100's kernels. #1076 wrongly grouped it with
+ *    gfx1150/1151 and dropped the override, silently sending the 780M to CPU (the very
+ *    common iGPU this regression hit). 11.0.0 is the value that worked on v1.33.0 and that
+ *    restores full GPU offload in the field; #1076's "gfx1100 WMMA fault" theory did not
+ *    hold up.
+ *  - gfx1031..gfx1036 (RDNA 2 iGPUs, e.g. Rembrandt 680M) → '10.3.0'. Not on the allowlist;
+ *    coerce onto gfx1030.
+ *  - anything else (unknown/newer target) → none. Prefer native discovery over a coercion
+ *    that's likely wrong; a hardcoded default gets more wrong as ROCm adds native targets.
+ */
+export function mapGfxToHsaOverride(gfx: string): string | null {
+  // Officially supported by the bundled ROCm — no override needed.
+  if (gfx === 'gfx1030' || gfx === 'gfx1100' || gfx === 'gfx1101' || gfx === 'gfx1102') {
+    return null
+  }
+  // RDNA 3.5 iGPUs (Strix 890M = gfx1150, Strix Halo = gfx1151) — natively supported.
+  if (gfx === 'gfx1150' || gfx === 'gfx1151') {
+    return null
+  }
+  // RDNA 3 Phoenix/Hawk Point (780M/760M = gfx1103) — NOT on the rocblas allowlist; coerce
+  // to gfx1100 kernels or ollama drops it to CPU.
+  if (gfx === 'gfx1103') {
+    return '11.0.0'
+  }
+  // RDNA 2 variants + iGPUs (gfx1031..gfx1036, e.g. Rembrandt 680M) — coerce to gfx1030.
+  if (/^gfx103[1-6]$/.test(gfx)) {
+    return '10.3.0'
+  }
+  // Unknown/newer target: prefer native discovery over a coercion that's likely wrong.
+  return null
+}

admin/app/utils/content_auto_update_backoff.ts

@@ -0,0 +1,51 @@
+import logger from '@adonisjs/core/services/logger'
+import type InstalledResource from '#models/installed_resource'
+
+/**
+ * Per-resource failure backoff for content (ZIM/map) auto-updates, shared by the
+ * three places that observe an auto-update's real lifecycle:
+ *
+ *   - {@link ContentAutoUpdateService.attempt} — a dispatch that fails to even
+ *     enqueue (counts as a failure; no job runs so no terminal event follows).
+ *   - `RunDownloadJob.onComplete` — a download that actually finished (success).
+ *   - the worker `failed` handler in `commands/queue/work.ts` — a download that
+ *     exhausted its retries (terminal failure).
+ *
+ * Kept in a dependency-light util (not on ContentAutoUpdateService) on purpose:
+ * RunDownloadJob is imported by CollectionUpdateService, which is imported by
+ * ContentAutoUpdateService, so importing the service back into the job would
+ * close an import cycle. Only the InstalledResource model and the logger are
+ * touched here.
+ */
+
+/** Genuine consecutive auto-update failures before a resource self-disables. */
+export const MAX_CONSECUTIVE_FAILURES = 3
+
+/** Clear a resource's failure backoff after a successful auto-update. */
+export async function recordResourceUpdateSuccess(resource: InstalledResource): Promise<void> {
+  if (resource.auto_update_consecutive_failures === 0 && !resource.auto_update_disabled_reason) {
+    return
+  }
+  resource.auto_update_consecutive_failures = 0
+  resource.auto_update_disabled_reason = null
+  await resource.save()
+}
+
+/** Record an auto-update failure and self-disable the resource at the threshold. */
+export async function recordResourceUpdateFailure(
+  resource: InstalledResource,
+  reason: string
+): Promise<void> {
+  const failures = (resource.auto_update_consecutive_failures || 0) + 1
+  resource.auto_update_consecutive_failures = failures
+  if (failures >= MAX_CONSECUTIVE_FAILURES) {
+    resource.auto_update_disabled_reason = `Auto-update disabled after ${failures} consecutive failures. Last error: ${reason}`
+    logger.error(
+      `[ContentAutoUpdate] ${resource.resource_id} auto-disabled after ${failures} failures`
+    )
+  }
+  await resource.save()
+  logger.error(
+    `[ContentAutoUpdate] ${resource.resource_id} failure ${failures}/${MAX_CONSECUTIVE_FAILURES}: ${reason}`
+  )
+}

admin/app/utils/content_reindex_decision.ts

@@ -0,0 +1,58 @@
+/**
+ * Decision for reconciling the AI knowledge base (Qdrant) after a curated
+ * content file (a ZIM) is replaced by a newer downloaded version.
+ *
+ * This is the pure, I/O-free core of `RagService.reconcileReplacedContentFile`.
+ * Keeping the branching here (mirrors `decideScanAction` in
+ * `kb_ingest_decision.ts`) makes the contract exhaustively testable without a
+ * database or a live Qdrant.
+ *
+ * The logic deliberately MIRRORS the replaced file's prior indexed state rather
+ * than applying the global `rag.defaultIngestPolicy`: on a content *update* the
+ * user has already made an indexing choice for this content, so we honor it in
+ * both directions (re-index a previously-indexed file even under Manual; leave a
+ * previously-unindexed file alone even under Always). Fresh installs still go
+ * through the normal policy path — those return `not_a_replacement` here.
+ *
+ * Outcomes (evaluated top-down, short-circuiting):
+ * - `not_a_replacement` — no prior file, or the new file has the same path
+ *   (same-version re-download). Caller defers to normal ingest policy.
+ * - `qdrant_not_installed` (step 2) — no knowledge base exists; nothing to do.
+ * - `old_not_indexed` (step 4) — the replaced file was never embedded
+ *   (no state row, or state ≠ `indexed`); leave the new file un-indexed.
+ * - `qdrant_not_running` (step 5) — the replaced file WAS indexed but Qdrant is
+ *   currently offline. We do nothing: we can't remove the stale points, and a
+ *   queued embed job could be dropped before Qdrant returns. Acting half-way is
+ *   wasteful, so we defer entirely (accepted tradeoff: stale points linger).
+ * - `reindex` (step 3) — the replaced file was indexed and Qdrant is running:
+ *   delete ONLY the old file's points, drop its state row, and queue the new
+ *   file for embedding.
+ *
+ * Note the install-before-indexed ordering: step 2 short-circuits before any KB
+ * state lookup, matching the spec.
+ */
+export type ReindexOutcome =
+  | 'not_a_replacement'
+  | 'qdrant_not_installed'
+  | 'old_not_indexed'
+  | 'qdrant_not_running'
+  | 'reindex'
+
+export interface ContentReindexInput {
+  /** The replaced file existed AND its path differs from the new file's path. */
+  isReplacement: boolean
+  /** `nomad_qdrant` service exists (installed), regardless of running state. */
+  qdrantInstalled: boolean
+  /** The replaced file's `KbIngestState.state === 'indexed'`. */
+  oldFileWasIndexed: boolean
+  /** Qdrant answered a live health check (currently reachable). */
+  qdrantRunning: boolean
+}
+
+export function decideContentReindex(input: ContentReindexInput): ReindexOutcome {
+  if (!input.isReplacement) return 'not_a_replacement'
+  if (!input.qdrantInstalled) return 'qdrant_not_installed'
+  if (!input.oldFileWasIndexed) return 'old_not_indexed'
+  if (!input.qdrantRunning) return 'qdrant_not_running'
+  return 'reindex'
+}

admin/app/utils/downloads.ts

@@ -8,6 +8,32 @@ import { deleteFileIfExists, ensureDirectoryExists, getFileStatsIfExists } from
 import { createWriteStream } from 'fs'
 import { rename } from 'fs/promises'
 import path from 'path'
+import logger from '@adonisjs/core/services/logger'
+
+/**
+ * A gated source rejected this install's credentials (401/403).
+ *
+ * Permanent by nature: whether the entitlement key is baked in is a property of
+ * the build, so no amount of retrying changes the answer. Declared here rather
+ * than thrown as an UnrecoverableError directly so this module stays free of a
+ * BullMQ dependency — RunDownloadJob translates it at the queue boundary.
+ */
+export class GatedContentAuthError extends Error {
+  constructor(message: string) {
+    super(message)
+    this.name = 'GatedContentAuthError'
+  }
+}
+
+// Some upstream mirrors reject requests with a missing or generic User-Agent.
+// Notably, download.kiwix.org routes the large Wikimedia-family ZIMs (Wikipedia,
+// Wikiversity, Wikibooks — including the flagship full Wikipedia) to
+// dumps.wikimedia.org, which returns HTTP 403 for a default `axios/x` (or empty)
+// User-Agent per Wikimedia's UA policy. Identify ourselves descriptively so
+// those downloads succeed.
+const DOWNLOAD_HEADERS: Record<string, string> = {
+  'User-Agent': 'ProjectNOMAD/1.0 (+https://projectnomad.us)',
+}
 
 /**
  * Perform a resumable download with progress tracking
@@ -24,6 +50,7 @@ export async function doResumableDownload({
   onComplete,
   forceNew = false,
   allowedMimeTypes,
+  requestHeaders,
 }: DoResumableDownloadParams): Promise<string> {
   const dirname = path.dirname(filepath)
   await ensureDirectoryExists(dirname)
@@ -41,11 +68,33 @@ export async function doResumableDownload({
     appendMode = true
   }
 
-  // Get file info with HEAD request first
+  // Merge default headers with any caller-supplied headers (e.g. Creator Packs' Authorization)
-  const headResponse = await axios.head(url, {
+  const headers: Record<string, string> = { ...DOWNLOAD_HEADERS, ...requestHeaders }
-    signal,
+
-    timeout,
+  // Get file info with HEAD request first. Gated sources (Creator Packs) require
-  })
+  // the auth header on the HEAD too, or the probe 401s before the GET is reached.
+  let headResponse
+  try {
+    headResponse = await axios.head(url, {
+      signal,
+      timeout,
+      headers,
+    })
+  } catch (error: any) {
+    // A 401/403 from a gated source is not a network problem and the raw axios
+    // message ("Request failed with status code 401") reads like our server is
+    // broken. Translate it, because the actual cause is almost always a build
+    // without the entitlement key baked in — i.e. not an official release.
+    // failedReason is surfaced verbatim on the downloads UI.
+    const status = error?.response?.status
+    if (status === 401 || status === 403) {
+      throw new GatedContentAuthError(
+        'This content is hosted by Project NOMAD and requires an official release build. ' +
+          `The download server rejected this install's credentials (HTTP ${status}).`
+      )
+    }
+    throw error
+  }
 
   // Some upstream hosts (notably download.kiwix.org for .zim files) don't set a
   // Content-Type header at all. Per RFC 7231 §3.1.1.5, "if no Content-Type is
@@ -54,8 +103,8 @@ export async function doResumableDownload({
   // Without this default, the validator below throws `MIME type  is not allowed`
   // and breaks all downloads from kiwix's primary host (#848).
   const contentType =
-    headResponse.headers['content-type'] || 'application/octet-stream'
+    headResponse.headers['content-type']?.toString() || 'application/octet-stream'
-  const totalBytes = parseInt(headResponse.headers['content-length'] || '0')
+  const totalBytes = parseInt(headResponse.headers['content-length']?.toString() || '0', 10)
   const supportsRangeRequests = headResponse.headers['accept-ranges'] === 'bytes'
 
   // If allowedMimeTypes is provided, check content type
@@ -88,13 +137,32 @@ export async function doResumableDownload({
     appendMode = false
   }
 
-  const headers: Record<string, string> = {}
+  // A .tmp bigger than the file now on the server cannot be a prefix of it — the
+  // publisher replaced the file under the same name (openZIM rolls builds forward,
+  // see #1189/#1187). Resuming would ask for a range past the end and get a 416 on
+  // every attempt, with nothing deleting the .tmp, so the download could never
+  // recover on its own. Discard and start clean.
+  if (startByte > totalBytes && totalBytes > 0) {
+    logger.warn(
+      `[Download] Discarding stale partial for ${filepath}: .tmp is ${startByte}B but the server reports ${totalBytes}B`
+    )
+    await deleteFileIfExists(tempPath)
+    startByte = 0
+    appendMode = false
+  }
+
+  // Add Range header if resuming
   if (supportsRangeRequests && startByte > 0) {
     headers.Range = `bytes=${startByte}-`
   }
 
-  const fetchStream = (hdrs: Record<string, string>) =>
+  const fetchStream = (headers: Record<string, string>) =>
-    axios.get(url, { responseType: 'stream', headers: hdrs, signal, timeout })
+    axios.get(url, {
+      responseType: 'stream',
+      headers,
+      signal,
+      timeout,
+    })
 
   let response = await fetchStream(headers)
 

admin/app/utils/fs.ts

@@ -6,6 +6,22 @@ import { LSBlockDevice, NomadDiskInfoRaw } from '../../types/system.js'
 
 export const ZIM_STORAGE_PATH = '/storage/zim'
 export const KIWIX_LIBRARY_XML_PATH = '/storage/zim/kiwix-library.xml'
+export const BOOKS_STORAGE_PATH = '/storage/books'
+// Shared media root (Jellyfin reads it as /media; File Browser shows it as "media"). Per-type
+// subfolders are pre-created on Jellyfin install — see _runPreinstallActions__Jellyfin.
+export const MEDIA_STORAGE_PATH = '/storage/media'
+export const JELLYFIN_MEDIA_SUBFOLDERS = ['Movies', 'TV Shows', 'Music', 'Photos']
+// Empty Calibre library bundled into the admin image (see install/calibre-empty-library/).
+// Seeded into storage/books on Calibre-Web install so it doesn't dead-end at db config.
+export const CALIBRE_EMPTY_LIBRARY_ASSET_PATH = 'assets/calibre/metadata.db'
+// Vaultwarden's /data volume. A self-signed TLS cert is generated here on install so the
+// web vault has the secure context (HTTPS) it requires — see _runPreinstallActions__Vaultwarden.
+export const VAULTWARDEN_STORAGE_PATH = '/storage/vaultwarden'
+// MeshCore Web's working dir. On install a self-signed cert (certs/) and an SSL nginx config
+// (nginx-ssl.conf) are generated here, then bind-mounted into the container so the static client is
+// served over HTTPS — required for its Web Bluetooth/Serial connections. See
+// _runPreinstallActions__MeshCoreWeb.
+export const MESHCORE_WEB_STORAGE_PATH = '/storage/meshcore-web'
 
 export async function listDirectoryContents(path: string): Promise<FileEntry[]> {
   const entries = await readdir(path, { withFileTypes: true })
@@ -174,13 +190,15 @@ export function matchesDevice(fsPath: string, deviceName: string): boolean {
   return false
 }
 
-export function determineFileType(filename: string): 'image' | 'pdf' | 'text' | 'epub' | 'zim' | 'unknown' {
+export function determineFileType(filename: string): 'image' | 'pdf' | 'text' | 'docx' | 'epub' | 'zim' | 'unknown' {
   const ext = path.extname(filename).toLowerCase()
   if (['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp'].includes(ext)) {
     return 'image'
   } else if (ext === '.pdf') {
     return 'pdf'
-  } else if (['.txt', '.md', '.docx', '.rtf'].includes(ext)) {
+  } else if (ext === '.docx') {
+    return 'docx'
+  } else if (['.txt', '.md', '.rtf'].includes(ext)) {
     return 'text'
   } else if (ext === '.epub') {
     return 'epub'

admin/app/utils/hosted_content.ts

@@ -0,0 +1,17 @@
+import type { SpecResource } from '../../types/collections.js'
+
+/**
+ * Pure predicate for "is this a resource we host behind the entitlement Worker?"
+ *
+ * Deliberately kept free of any `#start/env` import. `zim_download_resolution` is
+ * a pure, unit-tested module, and importing the env-reading side of this (see
+ * hosted_content_auth.ts) would trigger env validation at module load and break
+ * those tests outside a configured app context.
+ */
+
+/** The only gating scheme we support today. See SpecResource.auth. */
+export const NOMAD_APP_KEY_AUTH = 'nomad_app_key' as const
+
+export function isGatedResource(resource: Pick<SpecResource, 'auth'>): boolean {
+  return resource.auth === NOMAD_APP_KEY_AUTH
+}

admin/app/utils/hosted_content_auth.ts

@@ -0,0 +1,38 @@
+import env from '#start/env'
+import type { SpecResource } from '../../types/collections.js'
+import { isGatedResource } from './hosted_content.js'
+
+/**
+ * Auth for curated content that WE host and pay egress for.
+ *
+ * Content we host sits in a private R2 bucket behind the entitlement Worker,
+ * which requires a bearer key that only official release builds bake in (see the
+ * Dockerfile ARG/ENV pair, fed from the CI secret). That is the whole point: a
+ * fork rebuilt from source cannot point at our bucket and spend our bandwidth.
+ *
+ * A manifest resource opts in with `auth: 'nomad_app_key'`. Everything else keeps
+ * downloading unauthenticated exactly as before.
+ *
+ * Note on the key name: this deliberately reuses CREATOR_PACKS_APP_KEY rather
+ * than minting a second secret. The question it answers ("is this an official
+ * build?") is identical for Creator Packs and for our own hosted content, so a
+ * second CI secret plus a second Dockerfile ARG would be real cost for no
+ * security gain. The name is narrower than the use; this comment is cheaper than
+ * the churn of renaming it across CI, the Dockerfile and the Worker.
+ *
+ * The pure `isGatedResource` predicate lives in hosted_content.ts so that
+ * modules which must not pull in env validation can still use it.
+ */
+export function getHostedContentHeaders(
+  resource: Pick<SpecResource, 'auth'>
+): Record<string, string> | undefined {
+  if (!isGatedResource(resource)) return undefined
+
+  const appKey = env.get('CREATOR_PACKS_APP_KEY')
+  if (!appKey) return undefined
+
+  // Deliberately still dispatches with no header when the key is absent: the
+  // Worker answers 401 and the download surfaces "official release build
+  // required", which is a more useful signal than a silent no-op.
+  return { Authorization: `Bearer ${appKey}` }
+}

admin/app/utils/image_disk_preflight.ts

@@ -0,0 +1,83 @@
+import logger from '@adonisjs/core/services/logger'
+import type { ContainerRegistryService } from '#services/container_registry_service'
+import type { SystemService } from '#services/system_service'
+
+/**
+ * Shared pre-flight primitives for update flows (core app + installed apps).
+ * Kept framework-light (plain functions + injected service instances) so both
+ * {@link AutoUpdateService} and {@link AppAutoUpdateService} reuse one implementation.
+ */
+
+export type BlockerSeverity = 'skip' | 'failure'
+
+export interface Blocker {
+  reason: string
+  severity: BlockerSeverity
+}
+
+export interface PreflightResult {
+  ok: boolean
+  blockers: Blocker[]
+}
+
+/** Require free space >= imageSize * factor to cover decompressed layers + headroom. */
+export const DISK_SAFETY_FACTOR = 2
+/** Conservative fallback when the registry image size can't be determined. */
+export const MIN_FREE_BYTES = 5 * 1024 * 1024 * 1024 // 5 GiB
+
+/** Free bytes on the root filesystem (best-effort, falls back to max available). */
+export async function getFreeBytes(systemService: SystemService): Promise<number | null> {
+  const info = await systemService.getSystemInfo()
+  if (!info?.fsSize?.length) return null
+  const root = info.fsSize.find((f) => f.mount === '/')
+  if (root) return root.available
+  return Math.max(...info.fsSize.map((f) => f.available))
+}
+
+function gib(bytes: number): string {
+  return `${(bytes / 1024 / 1024 / 1024).toFixed(1)} GiB`
+}
+
+/**
+ * Returns a `failure` disk blocker if free space is insufficient for the given
+ * image reference, otherwise null. Mirrors the core update's behavior: estimate
+ * the image's compressed download size from the registry manifest, require
+ * `size * DISK_SAFETY_FACTOR` (or {@link MIN_FREE_BYTES} when size is unknown),
+ * and never block on transient lookup errors (returns null on failure).
+ *
+ * @param image Full image reference INCLUDING tag (e.g. "ollama/ollama:0.23.2").
+ */
+export async function checkImageDiskSpace(params: {
+  image: string
+  hostArch: string
+  containerRegistryService: ContainerRegistryService
+  systemService: SystemService
+}): Promise<Blocker | null> {
+  const { image, hostArch, containerRegistryService, systemService } = params
+  try {
+    const parsed = containerRegistryService.parseImageReference(image)
+    const imageSize = await containerRegistryService.getImageDownloadSize(
+      parsed,
+      parsed.tag,
+      hostArch
+    )
+    const required = imageSize !== null ? imageSize * DISK_SAFETY_FACTOR : MIN_FREE_BYTES
+
+    const free = await getFreeBytes(systemService)
+    if (free === null) {
+      logger.warn('[ImageDiskPreflight] Could not determine free disk space; skipping disk check')
+      return null
+    }
+
+    if (free < required) {
+      return {
+        reason: `Insufficient disk space: ${gib(free)} free, ${gib(required)} required`,
+        severity: 'failure',
+      }
+    }
+    return null
+  } catch (error) {
+    logger.warn(`[ImageDiskPreflight] Disk space check failed: ${error.message}`)
+    return null
+  }
+}

admin/app/utils/kb_ratio_lookup.ts

@@ -66,10 +66,21 @@ export function estimateBatch(
  *
  * Returns `null` if no row matches and no empty-string fallback is present —
  * caller decides whether to surface "unknown" or use its own default.
+ *
+ * `ignoreCatchAll` excludes the empty-string catch-all row from matching, so a
+ * filename that only the fallback would have matched returns `null` instead.
+ * Callers that need a *specific* expectation (e.g. the partial_stall warning,
+ * which must not fire on atypical ZIMs the registry can't actually characterize)
+ * pass this; rough aggregate estimates (disk cost) leave it off. See #913.
  */
-export function findChunksPerMb(filename: string, rows: RatioRow[]): number | null {
+export function findChunksPerMb(
+  filename: string,
+  rows: RatioRow[],
+  opts: { ignoreCatchAll?: boolean } = {}
+): number | null {
   let best: RatioRow | null = null
   for (const row of rows) {
+    if (opts.ignoreCatchAll && row.pattern === '') continue
     if (!filename.startsWith(row.pattern)) continue
     if (best === null || row.pattern.length > best.pattern.length) {
       best = row
@@ -88,9 +99,10 @@ export function findChunksPerMb(filename: string, rows: RatioRow[]): number | nu
 export function estimateChunkCount(
   filename: string,
   fileSizeBytes: number,
-  rows: RatioRow[]
+  rows: RatioRow[],
+  opts: { ignoreCatchAll?: boolean } = {}
 ): number | null {
-  const ratio = findChunksPerMb(filename, rows)
+  const ratio = findChunksPerMb(filename, rows, opts)
   if (ratio === null) return null
   const megabytes = fileSizeBytes / (1024 * 1024)
   return Math.round(ratio * megabytes)

admin/app/utils/platform_metadata.ts

@@ -0,0 +1,68 @@
+/**
+ * Pure helpers for turning the Docker daemon's platform strings into the fields
+ * the benchmark submission carries.
+ *
+ * These read the HOST's platform, which is the entire point: inside the admin
+ * container `os.arch()` and `si.osInfo()` describe the container, not the
+ * machine being benchmarked. `BenchmarkService` delegates to these so the string
+ * handling is unit-testable without a Docker daemon.
+ *
+ * Observed daemon output across the test fleet:
+ *
+ *   Architecture      'x86_64'                  | 'aarch64'
+ *   OSVersion         '24.04'                   | '26.04'
+ *   OperatingSystem   'Ubuntu 24.04.4 LTS'      | 'Ubuntu 26.04 LTS'
+ */
+
+/**
+ * Canonicalise the daemon's architecture string to the OCI platform names used
+ * everywhere else in the project (image manifests, install docs, the
+ * leaderboard).
+ *
+ * Docker reports `x86_64` / `aarch64`; images and the board talk in `amd64` /
+ * `arm64`. A fixed two-way map rather than a general normalisation table: these
+ * are the only architectures NOMAD targets, and anything unrecognised passes
+ * through verbatim rather than being guessed at, so an unexpected platform shows
+ * up honestly instead of mislabelled.
+ */
+export function normalizeArchitecture(raw: string): string {
+  const map: Record<string, string> = {
+    x86_64: 'amd64',
+    amd64: 'amd64',
+    aarch64: 'arm64',
+    arm64: 'arm64',
+  }
+  const key = raw.trim().toLowerCase()
+  return map[key] ?? raw.trim()
+}
+
+/**
+ * Split the distro name out of the daemon's free-form OperatingSystem string.
+ *
+ * `OperatingSystem` is a description ('Ubuntu 24.04.4 LTS') while `OSVersion` is
+ * structured ('24.04'). Taking the text before the version yields the name
+ * without hand-maintaining a list of distributions:
+ *
+ *   'Ubuntu 24.04.4 LTS'             + '24.04' -> 'Ubuntu'
+ *   'Ubuntu 26.04 LTS'               + '26.04' -> 'Ubuntu'
+ *   'Debian GNU/Linux 12 (bookworm)' + '12'    -> 'Debian GNU/Linux'
+ *
+ * Falls back to the full description whenever the version is missing, empty, or
+ * doesn't appear in the string. An over-long name is harmless; a wrong one is
+ * not, and silently truncating an unfamiliar distro would be worse than leaving
+ * it verbose.
+ */
+export function deriveOsName(operatingSystem: string, osVersion: string | null): string {
+  const description = operatingSystem.trim()
+  if (!osVersion) return description
+
+  const version = osVersion.trim()
+  if (version === '') return description
+
+  const idx = description.indexOf(version)
+  // idx === 0 means the string starts with the version and has no name to take.
+  if (idx <= 0) return description
+
+  const name = description.slice(0, idx).trim()
+  return name.length > 0 ? name : description
+}

admin/app/utils/superseded_resource.ts

@@ -0,0 +1,72 @@
+import { resolve, sep } from 'node:path'
+
+/**
+ * Decides whether a curated resource's PREVIOUSLY-installed file should be
+ * deleted now that a newer version has been downloaded (issue #634 — old map
+ * and ZIM versions accumulated on disk indefinitely because only Wikipedia had
+ * version cleanup).
+ *
+ * This is intentionally a pure function so every safety rail is unit-testable
+ * without touching the DB or filesystem. The caller looks up the prior
+ * `InstalledResource` row, records the new version, then asks this whether the
+ * old file is safe to remove.
+ *
+ * Safety rails (a "delete" decision requires ALL of these):
+ *  - There was a prior install for this exact resource_id (`existing` non-null).
+ *    Untracked / sideloaded files have no row and are therefore never touched.
+ *  - The old file path actually differs from the new one (a genuine version
+ *    swap, not a re-download of the same file).
+ *  - The new file is confirmed present on disk — we never remove the old copy
+ *    before the replacement is verified.
+ *  - The new version is strictly newer than the recorded one, so a re-install
+ *    or downgrade can't wipe a newer file.
+ *  - The old path resolves to within the resource's storage directory, so a
+ *    malformed DB value can't direct a delete outside the content store.
+ */
+
+export interface SupersededInputs {
+  /** Prior InstalledResource row for this resource_id, or null on first install. */
+  existing: { file_path: string; version: string } | null
+  /** Absolute path of the newly downloaded file. */
+  newFilePath: string
+  /** Version of the newly downloaded file (e.g. "2026-05"). */
+  newVersion: string
+  /** Whether the new file is confirmed present on disk. */
+  newFileExists: boolean
+  /** Absolute storage directory the old file must live under to be eligible. */
+  storageBaseDir: string
+}
+
+export type SupersededReason =
+  | 'first_install'
+  | 'same_file'
+  | 'new_file_missing'
+  | 'not_newer'
+  | 'outside_storage'
+  | 'superseded'
+
+export interface SupersededDecision {
+  delete: boolean
+  /** Resolved old path to delete — set only when `delete` is true. */
+  path?: string
+  reason: SupersededReason
+}
+
+export function decideSupersededDeletion(inputs: SupersededInputs): SupersededDecision {
+  const { existing, newFilePath, newVersion, newFileExists, storageBaseDir } = inputs
+
+  if (!existing) return { delete: false, reason: 'first_install' }
+  if (existing.file_path === newFilePath) return { delete: false, reason: 'same_file' }
+  if (!newFileExists) return { delete: false, reason: 'new_file_missing' }
+  // Versions are zero-padded date strings (YYYY-MM / YYYY-MM-DD), so a lexical
+  // compare orders them correctly. Require strictly newer.
+  if (!(newVersion > existing.version)) return { delete: false, reason: 'not_newer' }
+
+  const resolvedOld = resolve(existing.file_path)
+  const base = resolve(storageBaseDir)
+  if (resolvedOld !== base && !resolvedOld.startsWith(base + sep)) {
+    return { delete: false, reason: 'outside_storage' }
+  }
+
+  return { delete: true, path: resolvedOld, reason: 'superseded' }
+}

admin/app/utils/update_window.ts

@@ -0,0 +1,35 @@
+import { DateTime } from 'luxon'
+
+/**
+ * Shared update-window helpers used by both the core auto-update
+ * ({@link AutoUpdateService}) and the per-app auto-update ({@link AppAutoUpdateService}).
+ *
+ * The window is interpreted in the container's local time (set via the TZ env var).
+ * Windows that wrap past midnight (start > end, e.g. 22:00-02:00) are supported.
+ */
+
+/** Parse an "HH:MM" 24-hour string into minutes-since-midnight, or null if malformed. */
+export function parseWindowMinutes(hhmm: string): number | null {
+  const match = /^([01]\d|2[0-3]):([0-5]\d)$/.exec(hhmm)
+  if (!match) return null
+  return Number(match[1]) * 60 + Number(match[2])
+}
+
+/** Whether `now` falls inside the [windowStart, windowEnd) window (handles midnight wrap). */
+export function isWithinWindow(
+  windowStart: string,
+  windowEnd: string,
+  now: DateTime = DateTime.now()
+): boolean {
+  const start = parseWindowMinutes(windowStart)
+  const end = parseWindowMinutes(windowEnd)
+  if (start === null || end === null) return false
+
+  const current = now.hour * 60 + now.minute
+  if (start === end) return false // zero-length window
+  if (start < end) {
+    return current >= start && current < end
+  }
+  // Wraps midnight
+  return current >= start || current < end
+}

admin/app/utils/zim_download_resolution.ts

@@ -0,0 +1,64 @@
+import type { CatalogResult } from '../services/kiwix_catalog_service.js'
+import type { SpecResource } from '../../types/collections.js'
+import { isGatedResource } from './hosted_content.js'
+
+export type ResolvedZimDownload = {
+  url: string
+  version: string
+  sizeBytes: number | undefined
+}
+
+function compareZimVersions(left: string, right: string): number {
+  const parse = (value: string): [number, number] | null => {
+    const match = /^(\d{4})-(\d{1,2})$/.exec(value)
+    if (!match) return null
+
+    const month = Number.parseInt(match[2], 10)
+    if (month < 1 || month > 12) return null
+
+    return [Number.parseInt(match[1], 10), month]
+  }
+
+  const leftParts = parse(left)
+  const rightParts = parse(right)
+  if (!leftParts || !rightParts) return left.localeCompare(right)
+
+  return leftParts[0] - rightParts[0] || leftParts[1] - rightParts[1]
+}
+
+export function resolveZimDownload(
+  resource: SpecResource,
+  latest: CatalogResult | null
+): ResolvedZimDownload {
+  const manifestSizeBytes = resource.size_mb > 0 ? resource.size_mb * 1024 * 1024 : undefined
+
+  // Content we host ourselves is pinned to the manifest URL, never the Kiwix
+  // catalog. It isn't in the openzim catalog at all, so this is normally a no-op
+  // — but a resource-id collision would otherwise silently redirect a gated
+  // download to a third-party mirror, losing both the auth header and any
+  // guarantee about what the bytes are.
+  //
+  // Consequence, stated rather than implied: gated content does NOT participate
+  // in catalog-driven auto-update. New versions ship by bumping the manifest.
+  if (isGatedResource(resource)) {
+    return {
+      url: resource.url,
+      version: resource.version,
+      sizeBytes: manifestSizeBytes,
+    }
+  }
+
+  if (!latest || compareZimVersions(latest.version, resource.version) < 0) {
+    return {
+      url: resource.url,
+      version: resource.version,
+      sizeBytes: manifestSizeBytes,
+    }
+  }
+
+  return {
+    url: latest.download_url,
+    version: latest.version,
+    sizeBytes: latest.size_bytes > 0 ? latest.size_bytes : manifestSizeBytes,
+  }
+}

admin/app/utils/zim_html.ts

@@ -0,0 +1,127 @@
+import * as cheerio from 'cheerio'
+import { NON_CONTENT_HEADING_PATTERNS } from '../../constants/zim_extraction.js'
+
+export interface ZIMSection {
+  heading: string
+  text: string
+  level: number
+}
+
+export interface StructuredContent {
+  title: string
+  sections: ZIMSection[]
+  fullText: string
+}
+
+/**
+ * True when a section heading is one of the low-signal boilerplate headings
+ * (See also / References / External links / etc.). Sections under these
+ * headings are reference apparatus, not article content, and shouldn't reach
+ * the embedder. (#902)
+ */
+export function isNonContentHeading(heading: string): boolean {
+  return NON_CONTENT_HEADING_PATTERNS.some((pattern) => pattern.test(heading))
+}
+
+/**
+ * Render an HTML <table> into delimited text. cheerio's `.text()` concatenates
+ * every cell with no separators ("AgeDoseAdult500mg" word salad), which is
+ * unsearchable and pollutes embeddings. Instead, join cells with " | " and rows
+ * with newlines so row/column structure survives into the chunk. (#902)
+ */
+export function tableToText($: cheerio.CheerioAPI, table: any): string {
+  const rows: string[] = []
+  $(table)
+    .find('tr')
+    .each((_, tr) => {
+      const cells = $(tr)
+        .find('th, td')
+        .map((__, cell) => $(cell).text().replace(/\s+/g, ' ').trim())
+        .get()
+        .filter((cell) => cell.length > 0)
+      if (cells.length > 0) {
+        rows.push(cells.join(' | '))
+      }
+    })
+  return rows.join('\n')
+}
+
+/**
+ * Break a cleaned article's HTML into heading-delimited sections for chunking.
+ * Skips non-content sections (References, See also, ...) at emit time and
+ * renders tables as delimited text rather than concatenated cell soup. (#902)
+ */
+export function extractStructuredContent(html: string): StructuredContent {
+  const $ = cheerio.load(html)
+
+  const title = $('h1').first().text().trim() || $('title').text().trim()
+
+  const sections: ZIMSection[] = []
+  let currentSection = { heading: 'Introduction', content: [] as string[], level: 2, skip: false }
+
+  const flushSection = () => {
+    if (!currentSection.skip && currentSection.content.length > 0) {
+      sections.push({
+        heading: currentSection.heading,
+        text: currentSection.content.join(' ').replace(/\s+/g, ' ').trim(),
+        level: currentSection.level,
+      })
+    }
+  }
+
+  // Walk the full DOM rather than only direct children of <body>. Modern ZIMs (Devdocs,
+  // Wikipedia, FreeCodeCamp, etc.) wrap article content in a container div, which under
+  // .children() would be a single non-heading/non-paragraph element and yield zero sections.
+  $('body')
+    .find('h2, h3, h4, p, ul, ol, dl, table')
+    .each((_, element) => {
+      const $el = $(element)
+      const tagName = element.tagName?.toLowerCase()
+
+      if (['h2', 'h3', 'h4'].includes(tagName)) {
+        // Save the section we just finished, then open the next one.
+        flushSection()
+        const heading = $el
+          .text()
+          .replace(/\[edit\]/gi, '')
+          .trim()
+        const level = Number.parseInt(tagName.substring(1)) // Extract number from h2, h3, h4
+        currentSection = {
+          heading,
+          content: [],
+          level,
+          skip: isNonContentHeading(heading),
+        }
+      } else if (['p', 'ul', 'ol', 'dl', 'table'].includes(tagName)) {
+        // Don't bother collecting content for a section we're going to drop.
+        if (currentSection.skip) return
+        const text = tagName === 'table' ? tableToText($, element) : $el.text().trim()
+        if (text.length > 0) {
+          currentSection.content.push(text)
+        }
+      }
+    })
+
+  // Push the last section if it has content
+  flushSection()
+
+  // Fallback: if the selector walk produced no sections but the body has meaningful
+  // text (unusual structure, minimal markup), emit one section with the full body text
+  // so the article still contributes to the knowledge base.
+  if (sections.length === 0) {
+    const bodyText = $('body').text().replace(/\s+/g, ' ').trim()
+    if (bodyText.length > 0) {
+      sections.push({
+        heading: title || 'Content',
+        text: bodyText,
+        level: 2,
+      })
+    }
+  }
+
+  return {
+    title,
+    sections,
+    fullText: sections.map((s) => `${s.heading}\n${s.text}`).join('\n\n'),
+  }
+}

admin/app/validators/common.ts

@@ -14,25 +14,43 @@ import ipaddr from 'ipaddr.js'
  */
 export function assertNotPrivateUrl(urlString: string): void {
   const parsed = new URL(urlString)
-  const hostname = parsed.hostname.toLowerCase()
 
-  // `URL.hostname` strips the surrounding brackets from IPv6 literals
+  // Normalize the host before classifying it:
-  // (e.g. `http://[::1]/` → hostname `::1`), so IPv6 patterns must match
+  //  - lowercase for the `localhost` comparison
-  // the unbracketed form.
+  //  - strip the surrounding brackets `URL.hostname` leaves on IPv6 literals
-  const blockedPatterns = [
+  //    (`http://[::1]/` → `::1`)
-    /^localhost$/,
+  //  - strip any trailing root dot(s): `localhost.` and `127.0.0.1.` resolve to
-    /^127\.\d+\.\d+\.\d+$/,
+  //    the same target as the dotless form, so they must not slip past the
-    /^0\.0\.0\.0$/,
+  //    checks below (#911).
-    /^169\.254\.\d+\.\d+$/, // Link-local / cloud metadata
+  const hostname = parsed.hostname
-    /^::1$/, // IPv6 loopback
+    .toLowerCase()
-    /^fe80:/i, // IPv6 link-local
+    .replace(/^\[|\]$/g, '')
-    /^::ffff:/i, // IPv4-mapped IPv6 (e.g. ::ffff:7f00:1 = 127.0.0.1)
+    .replace(/\.+$/, '')
-    /^::$/, // IPv6 all-zeros (equivalent to 0.0.0.0)
-  ]
 
-  if (blockedPatterns.some((re) => re.test(hostname))) {
+  if (hostname === 'localhost') {
     throw new Error(`Download URL must not point to a loopback or link-local address: ${hostname}`)
   }
+
+  // Anything that isn't a literal IP (DNS names, bare LAN hostnames like
+  // `nomad3`, external FQDNs) is allowed — LAN appliances need them, and DNS
+  // rebinding is a fetch-time concern outside this guard's scope. Classifying
+  // literal addresses with ipaddr.js (rather than a regex list) catches
+  // alternate encodings and normalizes address ranges correctly (#922).
+  if (!ipaddr.isValid(hostname)) return
+
+  let addr = ipaddr.parse(hostname)
+  if (addr.kind() === 'ipv6' && (addr as ipaddr.IPv6).isIPv4MappedAddress()) {
+    // e.g. ::ffff:127.0.0.1 — classify by the embedded IPv4 so a mapped
+    // loopback/link-local is blocked while a mapped public IP is allowed.
+    addr = (addr as ipaddr.IPv6).toIPv4Address()
+  }
+
+  const range = addr.range()
+  if (range === 'loopback' || range === 'linkLocal' || range === 'unspecified') {
+    throw new Error(
+      `Download URL must not point to a loopback or link-local address: ${addr.toNormalizedString()}`
+    )
+  }
 }
 
 /**

admin/app/validators/conditions.ts

@@ -0,0 +1,25 @@
+import vine from '@vinejs/vine'
+
+/**
+ * "When to use what" — request validators.
+ *
+ * Mirrors the drug_reference validators: vine, minimal, typed at the edge.
+ */
+
+/**
+ * GET /api/conditions/drugs
+ *
+ * Resolve OTC drugs for either a curated condition (`slug`) or a free-text
+ * situation (`q`). Both are optional at the schema level; the controller
+ * requires exactly one and 400s otherwise, so the error message is specific
+ * ("provide slug or q") rather than a generic vine union failure.
+ */
+export const conditionDrugsValidator = vine.compile(
+  vine.object({
+    slug: vine.string().trim().minLength(1).maxLength(80).optional(),
+    q: vine.string().trim().minLength(1).maxLength(200).optional(),
+    limit: vine.number().min(1).max(200).optional(),
+    route: vine.string().trim().minLength(1).maxLength(40).optional(),
+    sort: vine.enum(['relevance', 'name']).optional(),
+  })
+)

admin/app/validators/curated_collections.ts

@@ -9,6 +9,14 @@ export const specResourceValidator = vine.object({
   description: vine.string(),
   url: vine.string().url(),
   size_mb: vine.number().min(0).optional(),
+  // Resource-type discriminator (absent == 'zim'). Required here because VineJS
+  // strips unknown keys, which would silently drop the field on manifest fetch.
+  type: vine.enum(['zim', 'dataset']).optional(),
+  // Gated-download discriminator (absent == unauthenticated). Declared here for
+  // the same reason as `type`: VineJS strips unknown keys, so omitting it would
+  // silently drop the field on manifest fetch and every gated download would go
+  // out with no Authorization header and 401.
+  auth: vine.enum(['nomad_app_key']).optional(),
 })
 
 // ---- ZIM Categories spec (versioned) ----
@@ -68,6 +76,31 @@ export const wikipediaSpecSchema = vine.object({
   ).minLength(1),
 })
 
+// ---- Creator Packs spec (versioned) ----
+//
+// Display metadata only — deliberately NO `url` field. The ZIM bytes are resolved
+// through the entitlement Worker at install time (CreatorPackService), never from
+// this public catalog. See project_content_creator_packs.
+export const creatorPacksSpecSchema = vine.object({
+  spec_version: vine.string(),
+  packs: vine.array(
+    vine.object({
+      id: vine.string(),
+      name: vine.string(),
+      creator: vine.string(),
+      description: vine.string(),
+      version: vine.string(),
+      resource_id: vine.string(),
+      video_count: vine.number().min(0),
+      size_mb: vine.number().min(0),
+      license_id: vine.string(),
+      banner_url: vine.string().url().optional(),
+      poster_url: vine.string().url().optional(),
+      logo_url: vine.string().url().optional(),
+    })
+  ).minLength(1),
+})
+
 // ---- Wikipedia validators (used by ZimService) ----
 
 export const wikipediaOptionSchema = vine.object({

admin/app/validators/drug_reference.ts

@@ -0,0 +1,40 @@
+import vine from '@vinejs/vine'
+import { PRODUCT_TYPES } from '../../types/drug_reference.js'
+
+/**
+ * Drug Reference v1 — request validators.
+ *
+ * Mirrors the stl_library validators: vine, minimal, typed at the edge.
+ */
+
+const PRODUCT_TYPE_VALUES = Object.values(PRODUCT_TYPES) as [string, ...string[]]
+
+/** GET /api/drug-reference/search */
+export const searchDrugValidator = vine.compile(
+  vine.object({
+    q: vine.string().trim().minLength(1).maxLength(200),
+    product_type: vine.enum(PRODUCT_TYPE_VALUES).optional(),
+    // Administration-route filter (openFDA `route`, e.g. ORAL, TOPICAL). Matched
+    // with LIKE because the column holds a comma-joined list; the UI sends
+    // curated values, the cap just bounds free input.
+    route: vine.string().trim().minLength(2).maxLength(40).optional(),
+    sort: vine.enum(['relevance', 'name'] as const).optional(),
+    limit: vine.number().min(1).max(200).optional(),
+    offset: vine.number().min(0).optional(),
+    scope: vine.enum(['name', 'indication'] as const).optional(),
+  })
+)
+
+/**
+ * GET /api/drug-reference/interactions?ids=1,2,3
+ *
+ * Accepts `ids` as a comma-separated string. The controller parses the actual
+ * id values via parseCompareIds (dedupe, drop non-positive-int, cap at 5).
+ * Vine validates only that `ids` is a non-empty string; the pure helper does
+ * the real semantic validation so it can be unit-tested without a running app.
+ */
+export const interactionsValidator = vine.compile(
+  vine.object({
+    ids: vine.string().trim().minLength(1).maxLength(200).optional(),
+  })
+)

admin/app/validators/nomad_md.ts

@@ -0,0 +1,11 @@
+import vine from '@vinejs/vine'
+
+// Allow an empty/absent value so the user can clear their NOMAD.md — AdonisJS
+// converts empty request strings to null, so `optional()` (coerced to '' in the
+// controller) is what lets a "clear" through. The cap keeps a single system
+// prompt from growing unbounded.
+export const updateNomadMdSchema = vine.compile(
+  vine.object({
+    content: vine.string().maxLength(100_000).optional(),
+  })
+)

admin/app/validators/ollama.ts

@@ -11,6 +11,9 @@ export const chatSchema = vine.compile(
     ),
     stream: vine.boolean().optional(),
     sessionId: vine.number().positive().optional(),
+    // Effective per-request thinking preference (per-model override or global default),
+    // resolved client-side. Omitted -> server falls back to the ai.autoThinking KV default.
+    think: vine.boolean().optional(),
   })
 )
 

admin/app/validators/rag.ts

@@ -19,6 +19,12 @@ export const embedFileSchema = vine.compile(
   })
 )
 
+export const fileSourceSchema = vine.compile(
+  vine.object({
+    source: vine.string().minLength(1),
+  })
+)
+
 export const estimateBatchSchema = vine.compile(
   vine.object({
     files: vine

admin/app/validators/responses/chat.ts

@@ -0,0 +1,33 @@
+/*
+|--------------------------------------------------------------------------
+| Chat response schemas
+|--------------------------------------------------------------------------
+|
+| Shapes mirror the serialized `ChatSession` / `ChatMessage` Lucid models
+| (app/models/chat_session.ts, app/models/chat_message.ts). `messages` is only
+| present when the relation is preloaded, so it is optional.
+|
+*/
+import vine from '@vinejs/vine'
+
+const chatMessage = vine.object({
+  id: vine.number(),
+  session_id: vine.number(),
+  role: vine.enum(['system', 'user', 'assistant'] as const),
+  content: vine.string(),
+  created_at: vine.string(),
+  updated_at: vine.string(),
+})
+
+const chatSession = vine.object({
+  id: vine.number(),
+  title: vine.string(),
+  model: vine.string().nullable(),
+  created_at: vine.string(),
+  updated_at: vine.string(),
+  messages: vine.array(chatMessage).optional(),
+})
+
+export const chatSessionResponse = vine.compile(chatSession)
+export const chatSessionListResponse = vine.compile(vine.array(chatSession.clone()))
+export const chatMessageResponse = vine.compile(chatMessage.clone())

admin/app/validators/responses/common.ts

@@ -0,0 +1,37 @@
+/*
+|--------------------------------------------------------------------------
+| Shared response schemas
+|--------------------------------------------------------------------------
+|
+| Response schemas are authored as VineJS validators purely so the OpenAPI
+| generator can turn them into documented response bodies via `toJSONSchema()`.
+| They are not (currently) used to validate outgoing responses — though they
+| could be asserted against in tests.
+|
+| Only add a schema here once its shape is verified against the controller /
+| model it documents; a wrong response schema is worse than none.
+|
+*/
+import vine from '@vinejs/vine'
+
+/** `{ status: 'ok' }` health probes. */
+export const healthResponse = vine.compile(
+  vine.object({
+    status: vine.string(),
+  })
+)
+
+/** The `{ error: '...' }` envelope returned on most failure paths. */
+export const errorResponse = vine.compile(
+  vine.object({
+    error: vine.string(),
+  })
+)
+
+/** The `{ success, message? }` envelope used by many mutation endpoints. */
+export const successMessageResponse = vine.compile(
+  vine.object({
+    success: vine.boolean(),
+    message: vine.string().optional(),
+  })
+)

admin/app/validators/settings.ts

@@ -1,5 +1,6 @@
 import vine from "@vinejs/vine";
 import { SETTINGS_KEYS } from "../../constants/kv_store.js";
+import type { KVStoreKey } from "../../types/kv_store.js";
 
 export const getSettingSchema = vine.compile(vine.object({
     key: vine.enum(SETTINGS_KEYS),
@@ -8,4 +9,60 @@ export const getSettingSchema = vine.compile(vine.object({
 export const updateSettingSchema = vine.compile(vine.object({
     key: vine.enum(SETTINGS_KEYS),
     value: vine.any().optional(),
-}))
+}))
+
+const HHMM_PATTERN = /^([01]\d|2[0-3]):[0-5]\d$/
+
+/**
+ * Validate the *value* for keys that have format constraints beyond the generic
+ * enum/any check (the generic validator only constrains the key). Returns an
+ * error message string when invalid, or null when the value is acceptable.
+ */
+export function validateSettingValue(key: KVStoreKey, value: unknown): string | null {
+    switch (key) {
+        case 'autoUpdate.windowStart':
+        case 'autoUpdate.windowEnd':
+        case 'contentAutoUpdate.windowStart':
+        case 'contentAutoUpdate.windowEnd':
+            if (typeof value !== 'string' || !HHMM_PATTERN.test(value)) {
+                return 'Time window values must be in 24-hour HH:MM format (e.g. "20:00").'
+            }
+            return null
+        case 'autoUpdate.cooloffHours':
+        case 'contentAutoUpdate.cooloffHours': {
+            const num = Number(value)
+            if (!Number.isInteger(num) || num < 0 || num > 8760) {
+                return 'Cool-off must be a whole number of hours between 0 and 8760.'
+            }
+            return null
+        }
+        case 'system.internetStatusTestUrl': {
+            // Empty clears the setting (reverts to env var / built-in defaults).
+            if (value === '' || value === undefined || value === null) {
+                return null
+            }
+            if (typeof value !== 'string') {
+                return 'Test URL must be a string.'
+            }
+            try {
+                const url = new URL(value)
+                if (url.protocol !== 'http:' && url.protocol !== 'https:') {
+                    return 'Test URL must use http or https.'
+                }
+            } catch {
+                return 'Test URL must be a valid URL (e.g. "https://example.com").'
+            }
+            return null
+        }
+        case 'contentAutoUpdate.maxBytesPerWindow': {
+            // Per-window download budget in bytes. 0 = unlimited.
+            const num = Number(value)
+            if (!Number.isInteger(num) || num < 0) {
+                return 'The per-window data cap must be a whole number of bytes (0 = unlimited).'
+            }
+            return null
+        }
+        default:
+            return null
+    }
+}

admin/app/validators/system.ts

@@ -31,3 +31,142 @@ export const updateServiceValidator = vine.compile(
     target_version: vine.string().trim(),
   })
 )
+
+export const preflightValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+  })
+)
+
+// Toggle per-app automatic updates (opt-in). The global master switch lives in
+// the KVStore (`appAutoUpdate.enabled`) and flows through the settings endpoint.
+export const setServiceAutoUpdateValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+    enabled: vine.boolean(),
+  })
+)
+
+// Shared sub-schema for a volume bind mapping. A colon is Docker's bind delimiter
+// (host:container:options) — forbid it in either field so a path can't smuggle in an
+// extra segment that the guard reads as safe but Docker re-parses as a different mount.
+const volumeSchema = vine.object({
+  host_path: vine.string().trim().regex(/^[^:]+$/),
+  container_path: vine.string().trim().regex(/^[^:]+$/),
+})
+
+// Environment variables must be KEY=value (value may be empty), matching Docker's Env format.
+const envVarSchema = vine.string().trim().regex(/^[A-Za-z_][A-Za-z0-9_]*=[\s\S]*$/)
+
+// Service-less preflight for the custom-app form: evaluates ports, volumes and image together.
+export const preflightCustomValidator = vine.compile(
+  vine.object({
+    image: vine.string().trim().optional(),
+    ports: vine.array(vine.number().min(1).max(65535)).optional(),
+    volumes: vine.array(volumeSchema).optional(),
+    // When editing, ignore port conflicts caused by this app's own running container.
+    exclude_service: vine.string().trim().optional(),
+  })
+)
+
+export const customAppValidator = vine.compile(
+  vine.object({
+    friendly_name: vine.string().trim().minLength(1).maxLength(100),
+    image: vine.string().trim().minLength(1),
+    ports: vine
+      .array(
+        vine.object({
+          container: vine.number().min(1).max(65535),
+          host: vine.number().min(1024).max(65535),
+        })
+      )
+      .optional(),
+    volumes: vine.array(volumeSchema).optional(),
+    env: vine.array(envVarSchema).optional(),
+    category: vine
+      .enum(['productivity', 'media', 'security', 'networking', 'utility', 'ai', 'education', 'custom'])
+      .optional(),
+    icon: vine.string().trim().optional(),
+    // Optional resource caps (advanced). Default caps are applied when omitted.
+    memory_mb: vine.number().min(64).optional(),
+    cpus: vine.number().min(0.1).max(64).optional(),
+    // When true, bypass advisory preflight (port conflicts / guard warnings) and install anyway.
+    force: vine.boolean().optional(),
+  })
+)
+
+// Set or clear an app's custom launch URL. A null/empty value clears the override; a non-empty
+// value is normalized + validated to a http(s) URL by normalizeCustomUrl in the controller.
+export const setServiceCustomUrlValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+    custom_url: vine.string().trim().nullable(),
+  })
+)
+
+/**
+ * Normalize a user-supplied custom app URL (backend twin of the inertia helper in
+ * lib/navigation.ts). Accepts a bare host or a full URL; prepends http:// when no scheme is
+ * present. Returns the normalized href, or null when empty (clears the override) or not a valid
+ * http(s) URL. Restricting to http/https blocks javascript:/data: from ever being stored.
+ */
+export function normalizeCustomUrl(input: string | null | undefined): string | null {
+  const trimmed = (input ?? '').trim()
+  if (!trimmed) return null
+  const withScheme = /^https?:\/\//i.test(trimmed) ? trimmed : `http://${trimmed}`
+  try {
+    const url = new URL(withScheme)
+    if (url.protocol !== 'http:' && url.protocol !== 'https:') return null
+    return url.href
+  } catch {
+    return null
+  }
+}
+
+export const deleteCustomAppValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+    // When true, also remove the backing Docker image (best-effort).
+    remove_image: vine.boolean().optional(),
+  })
+)
+
+export const uninstallServiceValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+    // When true, also remove the backing Docker image (best-effort).
+    remove_image: vine.boolean().optional(),
+  })
+)
+
+export const serviceLogsValidator = vine.compile(
+  vine.object({
+    tail: vine.number().min(1).max(2000).optional(),
+  })
+)
+
+// Reconfigure an existing custom app: the create shape plus the target service_name.
+export const updateCustomAppValidator = vine.compile(
+  vine.object({
+    service_name: vine.string().trim(),
+    friendly_name: vine.string().trim().minLength(1).maxLength(100),
+    image: vine.string().trim().minLength(1),
+    ports: vine
+      .array(
+        vine.object({
+          container: vine.number().min(1).max(65535),
+          host: vine.number().min(1024).max(65535),
+        })
+      )
+      .optional(),
+    volumes: vine.array(volumeSchema).optional(),
+    env: vine.array(envVarSchema).optional(),
+    category: vine
+      .enum(['productivity', 'media', 'security', 'networking', 'utility', 'ai', 'education', 'custom'])
+      .optional(),
+    icon: vine.string().trim().optional(),
+    memory_mb: vine.number().min(64).optional(),
+    cpus: vine.number().min(0.1).max(64).optional(),
+    force: vine.boolean().optional(),
+  })
+)

admin/commands/app_auto_update/dry_run.ts

@@ -0,0 +1,227 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+
+/**
+ * Exercise the app auto-update decision logic WITHOUT ever triggering a real update.
+ *
+ *   # Prove the per-app eligibility + window logic deterministically (no DB/Docker):
+ *   node ace app-auto-update:dry-run --scenarios
+ *
+ *   # Show, against the live DB, which opted-in apps WOULD update right now:
+ *   node ace app-auto-update:dry-run
+ */
+export default class AppAutoUpdateDryRun extends BaseCommand {
+  static commandName = 'app-auto-update:dry-run'
+  static description = 'Dry-run the app auto-update decision logic (never triggers an update)'
+
+  @flags.boolean({ description: 'Run the built-in deterministic scenario suite and exit' })
+  declare scenarios: boolean
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { DateTime } = await import('luxon')
+    const { DockerService } = await import('#services/docker_service')
+    const { DownloadService } = await import('#services/download_service')
+    const { SystemService } = await import('#services/system_service')
+    const { ContainerRegistryService } = await import('#services/container_registry_service')
+    const { QueueService } = await import('#services/queue_service')
+    const { AppAutoUpdateService } = await import('#services/app_auto_update_service')
+    const { isWithinWindow } = await import('../../app/utils/update_window.js')
+
+    const dockerService = new DockerService()
+    const svc = new AppAutoUpdateService(
+      dockerService,
+      new DownloadService(QueueService.getInstance()),
+      new SystemService(dockerService),
+      new ContainerRegistryService()
+    )
+
+    if (this.scenarios) {
+      const ok = this.runScenarios(svc, DateTime, isWithinWindow)
+      if (!ok) this.exitCode = 1
+      return
+    }
+
+    // --- Live read-only snapshot (no update triggered) ----------------------
+    const status = await svc.getStatus()
+    this.logger.log('')
+    this.logger.log(`  Master switch   : ${status.enabled ? 'enabled' : 'disabled'}`)
+    this.logger.log(
+      `  Window          : ${status.windowStart}-${status.windowEnd} ` +
+        `(currently ${status.withinWindow ? 'inside' : 'outside'})`
+    )
+    this.logger.log(`  Cool-off hours  : ${status.cooloffHours}`)
+    this.logger.log('')
+    if (status.apps.length === 0) {
+      this.logger.info('No apps are opted into auto-update.')
+      return
+    }
+    this.logger.log('Opted-in apps:')
+    for (const app of status.apps) {
+      const tag = app.eligible ? this.colors.green('WOULD UPDATE') : this.colors.dim('skip')
+      this.logger.log(
+        `  ${tag}  ${app.friendly_name || app.service_name}: ${app.current_version}` +
+          `${app.available_update_version ? ' → ' + app.available_update_version : ''} — ${app.reason}`
+      )
+    }
+  }
+
+  /**
+   * Deterministic acceptance suite over the pure decision helpers — no DB or Docker.
+   * Uses ContainerRegistryService.parseImageReference (pure) via appEligibility.
+   */
+  private runScenarios(svc: any, DateTime: any, isWithinWindow: any): boolean {
+    const now = DateTime.fromISO('2026-06-04T12:00:00Z')
+    const daysAgo = (d: number) => now.minus({ days: d })
+    const hoursAgo = (h: number) => now.minus({ hours: h })
+
+    const mk = (o: Record<string, any>) => ({
+      service_name: 'nomad_test',
+      container_image: 'ollama/ollama:0.18.1',
+      available_update_version: null,
+      available_update_first_seen_at: null,
+      auto_update_disabled_reason: null,
+      ...o,
+    })
+
+    type Case = { name: string; service: any; cooloff: number; expect: boolean }
+    const cases: Case[] = [
+      { name: 'no update → not eligible', service: mk({}), cooloff: 72, expect: false },
+      {
+        name: 'major bump → not eligible',
+        service: mk({
+          available_update_version: '1.0.0',
+          available_update_first_seen_at: daysAgo(10),
+        }),
+        cooloff: 72,
+        expect: false,
+      },
+      {
+        name: 'minor newer inside cool-off → not eligible',
+        service: mk({
+          available_update_version: '0.19.0',
+          available_update_first_seen_at: hoursAgo(10),
+        }),
+        cooloff: 72,
+        expect: false,
+      },
+      {
+        name: 'minor newer past cool-off → eligible',
+        service: mk({
+          available_update_version: '0.19.0',
+          available_update_first_seen_at: daysAgo(5),
+        }),
+        cooloff: 72,
+        expect: true,
+      },
+      {
+        name: 'null first-seen → not eligible',
+        service: mk({ available_update_version: '0.19.0', available_update_first_seen_at: null }),
+        cooloff: 72,
+        expect: false,
+      },
+      {
+        name: 'self-disabled → not eligible',
+        service: mk({
+          available_update_version: '0.19.0',
+          available_update_first_seen_at: daysAgo(30),
+          auto_update_disabled_reason: 'disabled',
+        }),
+        cooloff: 72,
+        expect: false,
+      },
+      {
+        name: ':latest pinned → not eligible',
+        service: mk({
+          container_image: 'foo/bar:latest',
+          available_update_version: '1.2.3',
+          available_update_first_seen_at: daysAgo(30),
+        }),
+        cooloff: 72,
+        expect: false,
+      },
+      {
+        name: 'cool-off 0 applies immediately',
+        service: mk({
+          available_update_version: '0.18.2',
+          available_update_first_seen_at: hoursAgo(1),
+        }),
+        cooloff: 0,
+        expect: true,
+      },
+    ]
+
+    type WinCase = { name: string; start: string; end: string; at: string; expect: boolean }
+    const at = (hhmm: string) => `2026-06-04T${hhmm}:00`
+    const windows: WinCase[] = [
+      {
+        name: 'normal 20:00-23:00 @ 21:00 → in',
+        start: '20:00',
+        end: '23:00',
+        at: at('21:00'),
+        expect: true,
+      },
+      {
+        name: 'normal 20:00-23:00 @ 19:00 → out',
+        start: '20:00',
+        end: '23:00',
+        at: at('19:00'),
+        expect: false,
+      },
+      {
+        name: 'wrap 22:00-02:00 @ 01:00 → in',
+        start: '22:00',
+        end: '02:00',
+        at: at('01:00'),
+        expect: true,
+      },
+      {
+        name: 'wrap 22:00-02:00 @ 12:00 → out',
+        start: '22:00',
+        end: '02:00',
+        at: at('12:00'),
+        expect: false,
+      },
+    ]
+
+    let passed = 0
+    let failed = 0
+
+    this.logger.log('')
+    this.logger.log('Eligibility scenarios:')
+    for (const c of cases) {
+      const got = svc.appEligibility(c.service, c.cooloff, now).eligible
+      const ok = got === c.expect
+      this.report(ok, `${c.name} (expected ${c.expect}, got ${got})`)
+      ok ? passed++ : failed++
+    }
+
+    this.logger.log('')
+    this.logger.log('Window scenarios:')
+    for (const c of windows) {
+      const got = isWithinWindow(c.start, c.end, DateTime.fromISO(c.at))
+      const ok = got === c.expect
+      this.report(ok, `${c.name} (expected ${c.expect}, got ${got})`)
+      ok ? passed++ : failed++
+    }
+
+    this.logger.log('')
+    if (failed === 0) {
+      this.logger.success(`All ${passed} scenarios passed`)
+    } else {
+      this.logger.error(`${failed} scenario(s) failed, ${passed} passed`)
+    }
+    return failed === 0
+  }
+
+  private report(ok: boolean, message: string) {
+    if (ok) {
+      this.logger.log(`  ${this.colors.green('✓')} ${message}`)
+    } else {
+      this.logger.log(`  ${this.colors.red('✗')} ${message}`)
+    }
+  }
+}

admin/commands/auto_update/dry_run.ts

@@ -0,0 +1,302 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+import { readFile } from 'node:fs/promises'
+
+/**
+ * Exercise the auto-update decision pipeline WITHOUT ever triggering a real update.
+ *
+ *   # Prove the core logic deterministically (no network/DB/Docker):
+ *   node ace auto-update:dry-run --scenarios
+ *
+ *   # Simulate "what would happen if I were running 1.32.0 right now"
+ *   # against the live GitHub releases feed and real pre-flight checks:
+ *   node ace auto-update:dry-run --current=1.32.0 --force-enabled
+ *
+ *   # Fully offline simulation with a canned release list + fixed clock:
+ *   node ace auto-update:dry-run --current=1.32.0 --force-enabled \
+ *     --releases-file=./fixtures/releases.json --now=2026-06-04T21:00:00Z \
+ *     --window-start=20:00 --window-end=23:00 --skip-preflight
+ */
+export default class AutoUpdateDryRun extends BaseCommand {
+  static commandName = 'auto-update:dry-run'
+  static description = 'Dry-run the auto-update decision pipeline (never triggers an update)'
+
+  @flags.boolean({ description: 'Run the built-in deterministic scenario suite and exit' })
+  declare scenarios: boolean
+
+  @flags.string({ description: 'Simulate this currently-running version (e.g. 1.32.0)' })
+  declare current: string
+
+  @flags.boolean({ description: 'Ignore the persisted enabled setting and treat as enabled' })
+  declare forceEnabled: boolean
+
+  @flags.string({ description: 'Override cool-off hours' })
+  declare cooloff: string
+
+  @flags.string({ description: 'Override window start (HH:MM)' })
+  declare windowStart: string
+
+  @flags.string({ description: 'Override window end (HH:MM)' })
+  declare windowEnd: string
+
+  @flags.string({ description: 'Simulate the clock at this ISO timestamp' })
+  declare now: string
+
+  @flags.string({ description: 'Path to a JSON file with a GitHub releases array (offline)' })
+  declare releasesFile: string
+
+  @flags.boolean({ description: 'Bypass Docker/disk/queue pre-flight checks' })
+  declare skipPreflight: boolean
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { DateTime } = await import('luxon')
+    const { DockerService } = await import('#services/docker_service')
+    const { DownloadService } = await import('#services/download_service')
+    const { SystemService } = await import('#services/system_service')
+    const { SystemUpdateService } = await import('#services/system_update_service')
+    const { ContainerRegistryService } = await import('#services/container_registry_service')
+    const { QueueService } = await import('#services/queue_service')
+    const { AutoUpdateService } = await import('#services/auto_update_service')
+
+    const dockerService = new DockerService()
+    const svc = new AutoUpdateService(
+      dockerService,
+      new DownloadService(QueueService.getInstance()),
+      new SystemService(dockerService),
+      new SystemUpdateService(),
+      new ContainerRegistryService()
+    )
+
+    if (this.scenarios) {
+      const ok = this.runScenarios(svc, DateTime)
+      if (!ok) {
+        this.exitCode = 1
+      }
+      return
+    }
+
+    // --- Live / simulated single dry run ------------------------------------
+    const overrides: Record<string, any> = {}
+    if (this.current) overrides.currentVersion = this.current
+    if (this.forceEnabled) overrides.forceEnabled = true
+    if (this.cooloff) overrides.cooloffHours = Number(this.cooloff)
+    if (this.windowStart) overrides.windowStart = this.windowStart
+    if (this.windowEnd) overrides.windowEnd = this.windowEnd
+    if (this.skipPreflight) overrides.skipPreflight = true
+    if (this.now) overrides.now = DateTime.fromISO(this.now)
+    if (this.releasesFile) {
+      const raw = await readFile(this.releasesFile, 'utf-8')
+      overrides.releases = JSON.parse(raw)
+    }
+
+    this.logger.info('Running auto-update dry run (no update will be triggered)...')
+    const decision = await svc.dryRun(overrides)
+
+    this.logger.log('')
+    this.logger.log(`  Current version : ${decision.currentVersion}`)
+    this.logger.log(`  Enabled         : ${decision.enabled}`)
+    this.logger.log(
+      `  Window          : ${decision.config.windowStart}-${decision.config.windowEnd} ` +
+        `(currently ${decision.withinWindow ? 'inside' : 'outside'})`
+    )
+    this.logger.log(`  Cool-off hours  : ${decision.config.cooloffHours}`)
+    this.logger.log(
+      `  Eligible target : ${decision.eligibleTarget ? decision.eligibleTarget.tag + ' (published ' + decision.eligibleTarget.publishedAt + ')' : '—'}`
+    )
+    if (decision.preflight) {
+      if (decision.preflight.ok) {
+        this.logger.log(`  Pre-flight      : ok`)
+      } else {
+        this.logger.log(`  Pre-flight      : BLOCKED`)
+        for (const b of decision.preflight.blockers) {
+          this.logger.log(`      - [${b.severity}] ${b.reason}`)
+        }
+      }
+    } else {
+      this.logger.log(`  Pre-flight      : (not reached)`)
+    }
+    this.logger.log('')
+
+    const verdict =
+      decision.outcome === 'ready'
+        ? `WOULD UPDATE → ${decision.eligibleTarget!.tag}`
+        : `WOULD NOT UPDATE (${decision.outcome}): ${decision.reason}`
+    if (decision.outcome === 'ready') {
+      this.logger.success(verdict)
+    } else {
+      this.logger.info(verdict)
+    }
+  }
+
+  /**
+   * Deterministic acceptance suite over the pure decision helpers — no network,
+   * DB, or Docker. Proves every branch reviewers care about.
+   */
+  private runScenarios(svc: any, DateTime: any): boolean {
+    const NOW = '2026-06-04T12:00:00Z'
+    const now = DateTime.fromISO(NOW)
+    const daysAgo = (d: number) => now.minus({ days: d }).toISO()
+    const hoursAgo = (h: number) => now.minus({ hours: h }).toISO()
+    const rel = (tag: string, published: string, extra: object = {}) => ({
+      tag_name: tag,
+      published_at: published,
+      ...extra,
+    })
+
+    type EligCase = {
+      name: string
+      releases: any[]
+      current: string
+      cooloff: number
+      expect: string | null
+    }
+
+    const eligibility: EligCase[] = [
+      {
+        name: 'only a major bump is newer → none (major requires manual)',
+        releases: [rel('v2.0.0', daysAgo(10))],
+        current: '1.32.0',
+        cooloff: 72,
+        expect: null,
+      },
+      {
+        name: 'same-major minor newer but inside cool-off → none',
+        releases: [rel('v1.33.0', hoursAgo(10))],
+        current: '1.32.0',
+        cooloff: 72,
+        expect: null,
+      },
+      {
+        name: 'same-major patch past cool-off → selected',
+        releases: [rel('v1.32.1', daysAgo(5))],
+        current: '1.32.0',
+        cooloff: 72,
+        expect: '1.32.1',
+      },
+      {
+        name: 'mixed: newest same-major past cool-off wins; major/in-cooloff/prerelease ignored',
+        releases: [
+          rel('v2.0.0', daysAgo(30)),
+          rel('v1.34.0', hoursAgo(5)),
+          rel('v1.33.2', daysAgo(4)),
+          rel('v1.33.5', daysAgo(1), { prerelease: true }),
+          rel('v1.33.0', daysAgo(8)),
+        ],
+        current: '1.32.9',
+        cooloff: 72,
+        expect: '1.33.2',
+      },
+      {
+        name: 'draft releases ignored',
+        releases: [rel('v1.33.0', daysAgo(5), { draft: true })],
+        current: '1.32.0',
+        cooloff: 72,
+        expect: null,
+      },
+      {
+        name: 'malformed tag with injection chars → ignored (M2)',
+        releases: [rel('v1.33.0|; e reboot', daysAgo(10))],
+        current: '1.32.0',
+        cooloff: 72,
+        expect: null,
+      },
+      {
+        name: 'dev build never updates',
+        releases: [rel('v1.33.0', daysAgo(10))],
+        current: 'dev',
+        cooloff: 72,
+        expect: null,
+      },
+      {
+        name: 'cool-off of 0 applies immediately',
+        releases: [rel('v1.32.1', hoursAgo(1))],
+        current: '1.32.0',
+        cooloff: 0,
+        expect: '1.32.1',
+      },
+    ]
+
+    type WinCase = { name: string; start: string; end: string; at: string; expect: boolean }
+    const at = (hhmm: string) => `2026-06-04T${hhmm}:00`
+    const windows: WinCase[] = [
+      {
+        name: 'normal 20:00-23:00 @ 21:00 → in',
+        start: '20:00',
+        end: '23:00',
+        at: at('21:00'),
+        expect: true,
+      },
+      {
+        name: 'normal 20:00-23:00 @ 19:00 → out',
+        start: '20:00',
+        end: '23:00',
+        at: at('19:00'),
+        expect: false,
+      },
+      {
+        name: 'wrap 22:00-02:00 @ 23:00 → in',
+        start: '22:00',
+        end: '02:00',
+        at: at('23:00'),
+        expect: true,
+      },
+      {
+        name: 'wrap 22:00-02:00 @ 01:00 → in',
+        start: '22:00',
+        end: '02:00',
+        at: at('01:00'),
+        expect: true,
+      },
+      {
+        name: 'wrap 22:00-02:00 @ 12:00 → out',
+        start: '22:00',
+        end: '02:00',
+        at: at('12:00'),
+        expect: false,
+      },
+    ]
+
+    let passed = 0
+    let failed = 0
+
+    this.logger.log('')
+    this.logger.log('Eligibility scenarios:')
+    for (const c of eligibility) {
+      const got = svc.selectEligibleTarget(c.releases, c.current, c.cooloff, now)
+      const gotVersion = got ? got.version : null
+      const ok = gotVersion === c.expect
+      this.report(ok, `${c.name} (expected ${c.expect ?? 'none'}, got ${gotVersion ?? 'none'})`)
+      ok ? passed++ : failed++
+    }
+
+    this.logger.log('')
+    this.logger.log('Window scenarios:')
+    for (const c of windows) {
+      const cfg = { enabled: true, windowStart: c.start, windowEnd: c.end, cooloffHours: 72 }
+      const got = svc.isWithinWindow(cfg, DateTime.fromISO(c.at))
+      const ok = got === c.expect
+      this.report(ok, `${c.name} (expected ${c.expect}, got ${got})`)
+      ok ? passed++ : failed++
+    }
+
+    this.logger.log('')
+    if (failed === 0) {
+      this.logger.success(`All ${passed} scenarios passed`)
+    } else {
+      this.logger.error(`${failed} scenario(s) failed, ${passed} passed`)
+    }
+    return failed === 0
+  }
+
+  private report(ok: boolean, message: string) {
+    if (ok) {
+      this.logger.log(`  ${this.colors.green('✓')} ${message}`)
+    } else {
+      this.logger.log(`  ${this.colors.red('✗')} ${message}`)
+    }
+  }
+}

admin/commands/content_auto_update/dry_run.ts

@@ -0,0 +1,218 @@
+import { BaseCommand, flags } from '@adonisjs/core/ace'
+import type { CommandOptions } from '@adonisjs/core/types/ace'
+import { isWithinWindow } from '../../app/utils/update_window.js'
+
+/**
+ * Exercise the content auto-update decision pipeline WITHOUT ever dispatching a
+ * real download.
+ *
+ *   # Prove the core selection/eligibility/window logic deterministically
+ *   # (no network/DB):
+ *   node ace content-auto-update:dry-run --scenarios
+ *
+ *   # Evaluate what the next run would do against the currently-persisted
+ *   # available-update state (run a "Check for Content Updates" first to refresh
+ *   # it), forcing the feature on and overriding the cap:
+ *   node ace content-auto-update:dry-run --force-enabled --cap=20 --window-start=00:00 --window-end=23:59
+ */
+export default class ContentAutoUpdateDryRun extends BaseCommand {
+  static commandName = 'content-auto-update:dry-run'
+  static description = 'Dry-run the content auto-update decision pipeline (never dispatches a download)'
+
+  @flags.boolean({ description: 'Run the built-in deterministic scenario suite and exit' })
+  declare scenarios: boolean
+
+  @flags.boolean({ description: 'Ignore the persisted enabled setting and treat as enabled' })
+  declare forceEnabled: boolean
+
+  @flags.string({ description: 'Override cool-off hours' })
+  declare cooloff: string
+
+  @flags.string({ description: 'Override window start (HH:MM)' })
+  declare windowStart: string
+
+  @flags.string({ description: 'Override window end (HH:MM)' })
+  declare windowEnd: string
+
+  @flags.string({ description: 'Override per-window data cap in GB (0 = unlimited)' })
+  declare cap: string
+
+  @flags.string({ description: 'Override bytes already used this window' })
+  declare usedBytes: string
+
+  @flags.string({ description: 'Simulate the clock at this ISO timestamp' })
+  declare now: string
+
+  static options: CommandOptions = {
+    startApp: true,
+  }
+
+  async run() {
+    const { DateTime } = await import('luxon')
+    const { DownloadService } = await import('#services/download_service')
+    const { QueueService } = await import('#services/queue_service')
+    const { ContentAutoUpdateService } = await import('#services/content_auto_update_service')
+
+    const svc = new ContentAutoUpdateService(new DownloadService(QueueService.getInstance()))
+
+    if (this.scenarios) {
+      const ok = this.runScenarios(svc, DateTime)
+      if (!ok) this.exitCode = 1
+      return
+    }
+
+    const BYTES_PER_GB = 1024 * 1024 * 1024
+    const overrides: Record<string, any> = {}
+    if (this.forceEnabled) overrides.forceEnabled = true
+    if (this.cooloff) overrides.cooloffHours = Number(this.cooloff)
+    if (this.windowStart) overrides.windowStart = this.windowStart
+    if (this.windowEnd) overrides.windowEnd = this.windowEnd
+    if (this.cap) overrides.maxBytesPerWindow = Math.round(Number(this.cap) * BYTES_PER_GB)
+    if (this.usedBytes) overrides.windowBytesUsed = Number(this.usedBytes)
+    if (this.now) overrides.now = DateTime.fromISO(this.now)
+
+    this.logger.info('Running content auto-update dry run (no download will be dispatched)...')
+    const d = await svc.dryRun(overrides)
+
+    this.logger.log('')
+    this.logger.log(`  Enabled         : ${d.enabled}`)
+    this.logger.log(
+      `  Window          : ${d.config.windowStart}-${d.config.windowEnd} ` +
+        `(currently ${d.withinWindow ? 'inside' : 'outside'})`
+    )
+    this.logger.log(`  Cool-off hours  : ${d.config.cooloffHours}`)
+    this.logger.log(
+      `  Data cap        : ${d.config.maxBytesPerWindow > 0 ? d.config.maxBytesPerWindow + ' bytes' : 'unlimited'}`
+    )
+    this.logger.log(`  Eligible        : ${d.eligibleCount}`)
+    this.logger.log(`  Would start     : ${d.selection.selected.map((c) => c.resource.resource_id).join(', ') || '—'}`)
+    this.logger.log(
+      `  Skipped (cap)   : ${d.selection.skippedOversize.map((c) => c.resource.resource_id).join(', ') || '—'}`
+    )
+    this.logger.log(
+      `  Deferred (budget): ${d.selection.deferred.map((c) => c.resource.resource_id).join(', ') || '—'}`
+    )
+    this.logger.log('')
+  }
+
+  /**
+   * Deterministic acceptance suite over the pure decision helpers — no network
+   * or dispatch. Mirrors the per-resource eligibility, cap selection, and window
+   * branches reviewers care about.
+   */
+  private runScenarios(svc: any, DateTime: any): boolean {
+    const now = DateTime.fromISO('2026-06-04T03:00:00Z')
+    const daysAgo = (d: number) => now.minus({ days: d })
+    const hoursAgo = (h: number) => now.minus({ hours: h })
+
+    const res = (o: Record<string, any> = {}) => ({
+      resource_id: 'res',
+      version: '2024-01',
+      available_update_version: null,
+      available_update_size_bytes: null,
+      available_update_first_seen_at: null,
+      auto_update_disabled_reason: null,
+      auto_update_consecutive_failures: 0,
+      installed_at: daysAgo(100),
+      ...o,
+    })
+    const cand = (id: string, size: number, installedAt: any = daysAgo(100)) => ({
+      resource: res({ resource_id: id }),
+      version: '2024-06',
+      download_url: `(test)`,
+      size_bytes: size,
+      installed_at: installedAt,
+    })
+
+    let passed = 0
+    let failed = 0
+    const report = (ok: boolean, message: string) => {
+      this.logger.log(`  ${ok ? this.colors.green('✓') : this.colors.red('✗')} ${message}`)
+      ok ? passed++ : failed++
+    }
+
+    this.logger.log('')
+    this.logger.log('Eligibility scenarios:')
+    report(
+      svc.resourceEligibility(res(), 72, now).eligible === false,
+      'no available update → not eligible'
+    )
+    report(
+      svc.resourceEligibility(
+        res({ available_update_version: '2024-06', available_update_first_seen_at: hoursAgo(10) }),
+        72,
+        now
+      ).eligible === false,
+      'inside cool-off → not eligible'
+    )
+    report(
+      svc.resourceEligibility(
+        res({ available_update_version: '2024-06', available_update_first_seen_at: daysAgo(5) }),
+        72,
+        now
+      ).eligible === true,
+      'past cool-off → eligible'
+    )
+    report(
+      svc.resourceEligibility(
+        res({
+          available_update_version: '2024-06',
+          available_update_first_seen_at: daysAgo(30),
+          auto_update_disabled_reason: 'disabled',
+        }),
+        72,
+        now
+      ).eligible === false,
+      'self-disabled → not eligible'
+    )
+
+    this.logger.log('')
+    this.logger.log('Cap selection scenarios:')
+    {
+      const s = svc.selectUnderCap([cand('a', 1000), cand('b', 2000)], 10000, 0)
+      report(s.selected.length === 2, 'under cap selects all')
+    }
+    {
+      const s = svc.selectUnderCap([cand('huge', 50000)], 20000, 0)
+      report(
+        s.selected.length === 0 && s.skippedOversize.length === 1,
+        'oversize file → skipped, never selected'
+      )
+    }
+    {
+      const s = svc.selectUnderCap([cand('mid', 8000)], 10000, 5000)
+      report(s.selected.length === 0 && s.deferred.length === 1, 'over remaining budget → deferred')
+    }
+    {
+      const s = svc.selectUnderCap([cand('a', 0)], 10000, 0)
+      report(s.selected.length === 0 && s.deferred.length === 1, 'unknown size → deferred')
+    }
+    {
+      const s = svc.selectUnderCap([cand('big', 9_999_999_999)], 0, 0)
+      report(s.selected.length === 1, 'cap 0 → unlimited')
+    }
+
+    this.logger.log('')
+    this.logger.log('Window scenarios:')
+    report(
+      isWithinWindow('02:00', '05:00', DateTime.fromISO('2026-06-04T03:00:00')) === true,
+      'normal 02:00-05:00 @ 03:00 → in'
+    )
+    report(
+      isWithinWindow('22:00', '02:00', DateTime.fromISO('2026-06-04T01:00:00')) === true,
+      'wrap 22:00-02:00 @ 01:00 → in'
+    )
+    report(
+      isWithinWindow('22:00', '02:00', DateTime.fromISO('2026-06-04T12:00:00')) === false,
+      'wrap 22:00-02:00 @ 12:00 → out'
+    )
+
+    this.logger.log('')
+    if (failed === 0) {
+      this.logger.success(`All ${passed} scenarios passed`)
+    } else {
+      this.logger.error(`${failed} scenario(s) failed, ${passed} passed`)
+    }
+    return failed === 0
+  }
+}