Replaces the AppBrain scraper (Cloudflare-blocked 403, confirmed dead).
play.google.com server-rendered chart HTML works: live-validated, real
Android packages + names + ratings from both /store/apps/top and
/store/apps/category/<CAT>. CLI: <top|category> [--category CAT]
[--region R] [--limit N] [--html-file F]; output source:"google-play",
entries {rank,name,package,rating}. Package link is the stable field;
title (Epkrse class) and rating are best-effort.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NOTE: AppBrain returns a Cloudflare JS challenge (HTTP 403) for
automated requests; live fetch degrades gracefully to an error (the
skill falls back to Apple RSS + web search). Offline test passes
against synthetic fixture — that is the test gate per the brief.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the hand-rolled two-step APKCombo curl/grep flow with apkeep
(default source apk-pure): no auth, no JavaScript, handles XAPK split
bundling and retries itself, and the per-mirror HTML parsing is no longer
ours to maintain. The script still prints a stable app.apk/app.xapk path.
- download-apk.sh: invoke apkeep, rename artifact to app.<ext>, clear error
when the binary is missing; CLONE_APP_APKEEP (test stub) and
CLONE_APP_APKEEP_SOURCE (mirror override) env hooks
- test-download-apk.sh: rewritten against an apkeep stub (xapk/apk/fail/
missing-binary cases)
- SKILL.md Phase 1: prose updated for apkeep install + source override
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The APKPure direct endpoint (d.apkpure.com/b/APK/<pkg>) now returns an HTTP
403 Cloudflare bot challenge for every package, so downloads always failed —
which also blocked the entire reverse-engineering chain downstream (it never
ran because Phase 1 produced no APK).
- download-apk.sh: fetch the APKCombo download page with a browser UA, extract
the embedded /r2?u=<signed-url> link, and download that. No JS needed; the
URL slug is ignored so a literal 'app' works for any package. Verified
end-to-end against com.wonder (Elevate) -> 126MB xapk -> RE fingerprint.
- test-download-apk.sh: model the two-step flow and the page-without-link case.
- resolve-re-scripts.sh: warn (stderr) when bash < 4, since the upstream RE
scripts use ${VAR,,} and fail with 'bad substitution' on macOS bash 3.2.
- SKILL.md / README.md / CLAUDE.md: document the APKCombo source and bash 4+ req.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hardening patch widened STRICT_URL to recover IPv4 literals, apex
2-label domains and internal hosts that the PR's strict-only regex
discarded as collateral while killing Kotlin-stdlib dictionary noise.
Widening alone reopened a narrow noise class: 'word.word' fragments such
as "www.this" / "this.introduction" pass as apex domains.
Keep extraction permissive and add a small awk pass that decides per host:
- IPv4 literal: always keep (dict fragments are words, never dotted-quads)
- >=3 labels: always keep (any TLD; same tolerance as the original regex)
- any host with a :port or /path: always keep (structured = high signal)
- bare 2-label apex: keep only when the TLD is a real one, matched as a
whole field (so "introduction" != "in" — the prefix-match bug a single
mega-regex would have)
Trade-off documented inline: a first-party host referenced bare with an
uncommon TLD (e.g. https://foo.store with no path) is dropped; a path or
port keeps it. awk is POSIX (sub/split/~/print) — more portable than the
bash>=4 'declare -A' already used in the summary header.
Verified: dictionary noise dropped; IPs, apex, internal and subdomain
hosts kept; --all on a zero-match tree still exits 0; host list and
full-URL list stay consistent (no orphan hosts).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- find-api-calls.sh: add missing '|| true' on the --paths inventory and
--urls extraction pipelines; with set -euo pipefail a zero-match grep
aborted the whole script (including the default --all run) with exit 1.
- find-api-calls.sh: widen STRICT_URL to also match IPv4 literals, apex
2-label domains and bare single-label hosts followed by :port or /path
(localhost, internal backends) while still rejecting dictionary-fragment
noise from the Kotlin stdlib.
- recover-kotlin-names.sh: sanitize the by_package/ filename with
os.path.basename; a crafted absolute path in untrusted @DebugMetadata
package names could otherwise escape the output directory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fatih.bulut
Simone Avogadro