Adds an optional `auth: 'nomad_app_key'` discriminator to curated manifest
resources so a curated collection tier can carry content we host ourselves,
gated to official release builds. Without this, only Creator Packs could use
the entitlement Worker; curated tier installs always downloaded
unauthenticated.
No behaviour change for any existing manifest entry: absent `auth` means
unauthenticated, exactly as today.
- `auth` declared on both the type and the VineJS validator. It has to be on
the validator or VineJS strips it silently on fetch, and the gated download
would then go out with no header and 401 for everyone. A dedicated spec
guards that regression.
- Gated resources are pinned to their manifest URL (resolveZimDownload skips
the catalog comparison) and excluded from catalog update checks, so a
resource-id collision cannot let a third-party mirror overwrite our content.
Consequence, commented rather than implied: gated content does not
auto-update; new versions ship via the manifest.
- 401/403 on a download now reports that an official build is required instead
of a raw axios status, which is what a fork build will hit.
- The pure `isGatedResource` predicate is deliberately split from the
env-reading header builder: importing `#start/env` into
zim_download_resolution triggers env validation at import time and breaks its
unit tests.
Reuses CREATOR_PACKS_APP_KEY rather than minting a second secret — the question
it answers ("is this an official build?") is identical for both content types.
Verified end to end on a test server: `auth` survives validation into the
cached spec, the Bearer header attaches to only the gated resource, the file
lands byte-exact with an installed_resources row and a Kiwix library entry, and
an entry with a gated URL but no `auth` field fails with the intended message.
No catalog entry is included here. Manifests are fetched live from `main`, so a
gated entry must not merge until this ships and is adopted — pre-`auth` builds
strip the field and 401.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(creator-packs): backend rail for gated creator content packs
Phase 1 of Creator Packs: the app-side download rail for per-creator
video ZIMs served from the private R2 bucket behind the entitlement
Worker. No UI yet (Phase 2).
- Thread optional requestHeaders through the download util + job so the
ZIM fetch can carry `Authorization: Bearer <APP_KEY>` on both the HEAD
probe and the GET stream (Worker streams the ZIM directly, gated).
- Add resourceMetadata.skip_embedding and gate the RAG/EmbedFileJob
branch on it so video ZIMs are never sent to the knowledge base.
- New creator_packs ManifestType + creatorPacksSpecSchema (display
metadata only, no url) + CollectionManifestService wiring and
getCreatorPacksWithStatus() status join.
- collections/creator-packs.json catalog with the two seed packs.
- CreatorPackService.installPack: ensure Kiwix (auto-install if absent),
resolve the stable Worker URL, dispatch the gated download with
skip_embedding. Typed result codes.
- CreatorPacksController + GET/POST /api/creator-packs routes.
- CREATOR_PACKS_APP_KEY / CREATOR_PACKS_WORKER_BASE env (release-injected
key; app degrades to a clear not_configured error when unset).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): Phase 2 UI — Content Explorer, settings page, wizard step
Adds the three user-facing surfaces for Creator Packs on top of the Phase 1
backend rail. All surfaces HIDE entirely on builds without the release-injected
key (configured=false) so a fork never shows a broken install button.
- Backend: CreatorPackService.isConfigured() + `configured` flag on the
GET /api/creator-packs response; /settings/creator-packs route + controller
action.
- api client + useCreatorPacks hook (shared cached source of configured/packs/
downloads for every surface) + CreatorPackCard (status badges, art fallback,
update pill) + CreatorPacksSection (install-on-click grid + confirm modal with
the license note), reused by the Content Explorer block and settings page.
- Content Explorer: Creator Packs section before Additional Content.
- Settings: new /settings/creator-packs manage page + conditional nav item.
- Easy Setup wizard: new Creator Packs step after Content. Replaces the fragile
hardcoded AI-skip math with a computed activeSteps list so the two optional
steps (Creator Packs + AI) navigate correctly in every on/off combination;
selection state, storage projection, review summary, and finish-install wired.
Browser-tested on NOMAD3 across all surfaces (configured + fork paths), incl.
live install→download→cancel and AI-on/off wizard renumbering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): use branded pack banners on the cards
Replace the generic colored-block cards with the branded 1060x175 banners we
build into each pack ZIM, shown on both the settings page and the Easy Setup
wizard step (and the Content Explorer block, via the shared card).
- Bundle the seed-pack banners as local webp under admin/public/creator-packs/
so they render offline (no external image dependency, matching the existing
/rogue-support-banner.webp convention).
- CreatorPackCard is now banner-forward: the banner is the hero, with a compact
status/metadata footer (videos · size, Installed/Downloading/Selected/Install,
update pill). Falls back to a simple header if an image is ever missing.
- Prefer an optional catalog `banner_url` (future remote creators) over the
bundled-by-id path; add banner_url to the CreatorPack type + validator.
- Widen the card grids to 1–2 columns since banners are wide.
Browser-tested on NOMAD3: banners render on the settings page and wizard step;
selection/installed/downloading states and storage projection still correct.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(creator-packs): Phase 3 — uninstall, license, release key-injection
Completes the Creator Packs feature.
- Uninstall: CreatorPackService.uninstallPack reuses ZimService.delete (removes
the ZIM, drops it from the Kiwix library, clears the InstalledResource) behind
DELETE /api/creator-packs/:id. A trash control on installed cards appears only
on the settings "manage" surface (allowUninstall) — never on the Content
Explorer block or the wizard — with a danger confirm modal.
- License: draft "Project NOMAD Creator Pack License v1.0" at
collections/creator-pack-license.md (marked DRAFT — pending legal review;
personal-use, no-redistribution, official-channel-only, creators retain
copyright). The install modal links to it ("View license").
- Release key-injection: Dockerfile ARG/ENV CREATOR_PACKS_APP_KEY in the runtime
stage (empty default → source/CI builds ship unconfigured and hide the UI) +
build-primary-image.yml passes it from the CREATOR_PACKS_APP_KEY CI secret.
NOTE FOR JAKE: add that repo secret = the entitlement Worker's key.
- Update flow was already functional (installed-with-update cards → "Update
pack"); verified.
Browser-tested on NOMAD3: uninstall round-trip (file + DB row + kiwix library
cleaned, then reinstalled), settings-only uninstall control, install-modal
license link, and the "Update available" pill via a temporary catalog bump.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(downloads): clean up header merge logic
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: jakeaturner <jturner@cosmistack.com>
download.kiwix.org routes the large Wikimedia-family ZIMs (Wikipedia,
Wikiversity, Wikibooks — including the flagship full Wikipedia) to
dumps.wikimedia.org, which enforces a User-Agent policy and returns HTTP
403 for requests with a missing or generic (axios/x) User-Agent. Because
doResumableDownload sent no User-Agent, every Wikimedia-hosted ZIM failed
to download while Kiwix-mirror-hosted ZIMs succeeded — so a curated set or
Easy Setup run would silently stall on exactly the highest-value content.
Add a descriptive User-Agent to the HEAD and GET requests. Verified
against the live mirror: default/empty UA -> 403, ProjectNOMAD UA -> 200/206.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
download.kiwix.org (and some of its mirrors) don't always set a
Content-Type header on .zim responses. The MIME validator was reading
`headers['content-type'] || ''`, then running each allowlist entry
through `''.includes(...)` which is always false, so every download
from those hosts was rejected with `MIME type is not allowed`.
RFC 7231 §3.1.1.5 says missing Content-Type may be treated as
application/octet-stream by the recipient, and that's already in every
binary-content allowlist we use (ZIM, PMTILES, base assets). Default
the missing case to that and the validator does the right thing.
Strict callers that don't list octet-stream still reject as before.
Downloads are now written to `filepath + '.tmp'` and atomically renamed
to the final path only on successful completion. Kiwix globs for `*.zim`
and ZimService filters `.endsWith('.zim')`, so `.tmp` files are invisible
to both during download. The same staging applies to `.pmtiles` map files.
Ref #372
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three bugs caused downloads to hang, disappear, or leave stuck spinners:
1. Wikipedia downloads that failed never updated the DB status from 'downloading',
leaving the spinner stuck forever. Now the worker's failed handler marks them as failed.
2. No stall detection on streaming downloads - if data stopped flowing mid-download,
the job hung indefinitely. Added a 5-minute stall timer that triggers retry.
3. Failed jobs were invisible to users since only waiting/active/delayed states were
queried. Now failed jobs appear with error indicators in the download list.
Closes#364, closes#216
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chriscrosstalk
jakeaturner
Ben Gauger
Aaron Bird
Chris Sherwood