project-nomad/admin/app
Chris Sherwood e214bf8cc3 fix(downloads): send a descriptive User-Agent so Wikimedia mirrors don't 403
download.kiwix.org routes the large Wikimedia-family ZIMs (Wikipedia,
Wikiversity, Wikibooks — including the flagship full Wikipedia) to
dumps.wikimedia.org, which enforces a User-Agent policy and returns HTTP
403 for requests with a missing or generic (axios/x) User-Agent. Because
doResumableDownload sent no User-Agent, every Wikimedia-hosted ZIM failed
to download while Kiwix-mirror-hosted ZIMs succeeded — so a curated set or
Easy Setup run would silently stall on exactly the highest-value content.

Add a descriptive User-Agent to the HEAD and GET requests. Verified
against the live mirror: default/empty UA -> 403, ProjectNOMAD UA -> 200/206.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:19:44 -07:00
..
controllers feat(RAG): adds the ability to cancel all embedding jobs (#1034) 2026-06-23 04:47:11 +00:00
exceptions fix(Docs): documentation renderer fixes 2025-12-23 16:00:33 -08:00
jobs feat(RAG): adds the ability to cancel all embedding jobs (#1034) 2026-06-23 04:47:11 +00:00
middleware fix(API): skip compression for Server-Sent Events (#798) 2026-05-20 10:16:00 -07:00
models feat: replace legacy Kolibri image default with latest v19 image (#1019) 2026-06-23 04:47:07 +00:00
services feat: configurable internet test url override in new Advanced Settings page 2026-06-23 04:47:10 +00:00
utils fix(downloads): send a descriptive User-Agent so Wikimedia mirrors don't 403 2026-07-16 15:19:44 -07:00
validators feat: configurable internet test url override in new Advanced Settings page 2026-06-23 04:47:10 +00:00