ci: make collection URL validator robust to Range-ignoring mirrors

download.kiwix.org redirects to mirrors, some of which ignore the Range
header and stream the full body, so `curl --max-filesize 1` aborts with exit
63 (CURLE_FILESIZE_EXCEEDED). Under the step's default `bash -eo pipefail`,
that non-zero exit in the `HTTP_CODE=$(...)` assignment killed the whole job
on the first such URL, regardless of the URL being valid. The failure was
non-deterministic (depended which mirror the redirect picked).

Add `|| true` so the assignment can't trip `set -e`; curl still writes the
real %{http_code} (200/206) for the existing check, and a genuine 404 still
returns 404 with exit 0 and is still caught.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Sherwood 2026-06-24 13:09:23 -07:00
parent 05c324ba6c
commit 330774312c
1 changed files with 8 additions and 2 deletions

View File

@ -30,13 +30,19 @@ jobs:
# Use Range: bytes=0-0 to avoid downloading the full file.
# --max-filesize 1 aborts early if the server ignores the Range header
# and returns 200 with the full body. The HTTP status is still captured.
# and returns 200 with the full body. download.kiwix.org redirects to
# mirrors, some of which ignore Range and stream the body, making curl
# exit 63 (CURLE_FILESIZE_EXCEEDED). That just means the file EXISTS, so
# it must not fail the step: `|| true` keeps the step's `set -e` from
# aborting on that exit code, while curl still writes the real
# %{http_code} (200/206) for the check below. A genuine 404 still
# returns 404 with exit 0 and is still caught.
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
--range 0-0 \
--max-filesize 1 \
--max-time 30 \
--location \
"$url")
"$url" || true)
if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "206" ]; then
echo "OK ($HTTP_CODE)"