From b1135b82a2c7a23d1e55434b446996d155703d92 Mon Sep 17 00:00:00 2001 From: "fatih.bulut" Date: Sun, 21 Jun 2026 04:16:25 +0300 Subject: [PATCH] fix(clone-app): switch APK source to APKCombo and guard bash version MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The APKPure direct endpoint (d.apkpure.com/b/APK/) now returns an HTTP 403 Cloudflare bot challenge for every package, so downloads always failed — which also blocked the entire reverse-engineering chain downstream (it never ran because Phase 1 produced no APK). - download-apk.sh: fetch the APKCombo download page with a browser UA, extract the embedded /r2?u= link, and download that. No JS needed; the URL slug is ignored so a literal 'app' works for any package. Verified end-to-end against com.wonder (Elevate) -> 126MB xapk -> RE fingerprint. - test-download-apk.sh: model the two-step flow and the page-without-link case. - resolve-re-scripts.sh: warn (stderr) when bash < 4, since the upstream RE scripts use ${VAR,,} and fail with 'bad substitution' on macOS bash 3.2. - SKILL.md / README.md / CLAUDE.md: document the APKCombo source and bash 4+ req. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 70 +++++++++++++++++++ plugins/clone-app/README.md | 9 +++ plugins/clone-app/skills/clone-app/SKILL.md | 13 ++-- .../skills/clone-app/scripts/download-apk.sh | 55 +++++++++++++-- .../clone-app/scripts/resolve-re-scripts.sh | 13 ++++ plugins/clone-app/tests/test-download-apk.sh | 55 +++++++++++---- 6 files changed, 194 insertions(+), 21 deletions(-) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..3adecb0 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,70 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this repo is + +A Claude Code **plugin marketplace** (a fork of `SimoneAvogadro/android-reverse-engineering-skill`) hosting two plugins: + +- `plugins/android-reverse-engineering/` — **upstream, vendored. Never modify.** Decompiles APK/XAPK/JAR/AAR with jadx, extracts HTTP APIs, recovers R8-obfuscated Kotlin names. +- `plugins/clone-app/` — **this project's work.** A `/clone-app` skill that takes a Google Play URL, drives the RE plugin to reverse engineer the app, scrapes store metrics, estimates AI-assisted clone effort + infra cost, writes a viability report, and optionally hands off to the `writing-plans` skill. + +There is no compiled artifact and no application runtime. "The code" is bash + Python helper scripts plus Markdown skill/command/reference docs that Claude Code loads at session time. + +## The hard constraint + +`plugins/android-reverse-engineering/` is the upstream tree. Keeping it byte-identical is what makes `git pull upstream master` conflict-free. Before committing, this must print nothing: + +```bash +git status --porcelain plugins/android-reverse-engineering/ +``` + +Two remotes: `origin` = `masa2146/clone-app-skill` (this fork), `upstream` = `SimoneAvogadro/android-reverse-engineering-skill`. Sync upstream with `git pull upstream master`. The only shared files clone-app touches are root `.claude-plugin/marketplace.json` (added a second plugin entry) and—if ever—root `README.md`; everything else lives under `plugins/clone-app/`. + +## Commands + +```bash +# Full clone-app test suite (5 suites, ~28 assertions). Run from repo root. +bash plugins/clone-app/tests/run-all.sh + +# A single test +bash plugins/clone-app/tests/test-extract-package.sh +python3 plugins/clone-app/tests/test-scrape-play-store.py + +# Structural smoke test: files present, scripts executable, JSON valid, both plugins in marketplace +bash plugins/clone-app/tests/smoke-structure.sh + +# Validate the two JSON manifests +python3 -c "import json; json.load(open('.claude-plugin/marketplace.json')); json.load(open('plugins/clone-app/.claude-plugin/plugin.json'))" +``` + +Shell is zsh but every script uses `#!/usr/bin/env bash` — invoke with `bash `, not `sh`. `shopt` etc. fail under zsh; wrap in `bash -c '...'`. Python is stdlib-only (`urllib`, `json`, `re`); no pip, no virtualenv. + +**bash 4+ is required at runtime.** The upstream RE scripts (`fingerprint.sh`, `find-api-calls.sh`) use `${VAR,,}` and break with "bad substitution" on macOS's stock bash 3.2. You cannot patch them (upstream-untouched rule), so the fix is `brew install bash`; `#!/usr/bin/env bash` then resolves to the modern one. `resolve-re-scripts.sh` emits a stderr WARNING when it detects bash < 4. + +**APK source is APKCombo, not APKPure.** The old `d.apkpure.com/b/APK/` endpoint now returns an HTTP 403 Cloudflare challenge for every package. `download-apk.sh` instead GETs `apkcombo.com/app//download/apk` with a browser User-Agent, greps the embedded `/r2?u=` link out of the HTML, and downloads that (no JavaScript needed; the URL slug segment is ignored by the server so a literal `app` works for any package). + +## How clone-app is wired + +`plugins/clone-app/skills/clone-app/SKILL.md` is the orchestrator — an 8-phase (0–7) prose workflow Claude executes. Phases split into two kinds: + +- **Deterministic steps → helper scripts** under `skills/clone-app/scripts/`, each independently testable: `extract-package.sh` (Play URL → package), `download-apk.sh` (APKCombo two-step download, 3 retries, apk-vs-xapk by ZIP contents), `resolve-re-scripts.sh` (locates the sibling RE plugin's `scripts/` dir + warns if `bash` is < 4), `scrape-play-store.py` (Play page → metrics JSON via embedded ld+json), `check-appstore.py` (iTunes Search → iOS presence). +- **AI-judgment steps → reference rubrics** under `skills/clone-app/references/` (`stack-recommendation-guide`, `effort-estimation-guide`, `infra-cost-guide`, `report-template`). These keep effort/cost/verdict output consistent; edit the rubric, not the prose, to change how estimates are produced. + +clone-app never hardcodes RE script paths. It calls `resolve-re-scripts.sh`, which from `plugins/clone-app/` walks to `plugins/android-reverse-engineering/skills/android-reverse-engineering/scripts/` (honoring `$CLAUDE_PLUGIN_ROOT`, falling back to deriving its own location). Phase 2 then invokes that dir's `fingerprint.sh`, `check-deps.sh`, `decompile.sh -o "$WORK/output"`, `recover-kotlin-names.sh`, `find-api-calls.sh`. + +Two runtime conventions the scripts and SKILL.md share: +- **Working dir** is `./work/{package}/` relative to the user's cwd (never inside the plugin). Decompile output must land at `$WORK/output/` via `decompile.sh -o` — `decompile.sh` otherwise defaults to a relative `-decompiled` dir in cwd. +- **Effort is measured in "AI Sprints"** (one focused Claude session, ~2–4h review), never calendar time. + +SKILL.md pauses for the user at exactly two points: Phase 4 (choose clone stack) and Phase 7 (proceed to implementation plan?). Phase 7's "yes" path invokes `superpowers:writing-plans`. + +## Test pattern + +Bash tests use `set -uo pipefail` (intentionally not `-e`) and aggregate failures into a `fail` var so every assertion runs; the script exits non-zero if any failed. Python scrapers are tested offline against `tests/fixtures/` via `--html-file` / `--json-file` flags — never hitting the network. New scrape logic needs a fixture, not a live call. + +## Conventions + +- Plugin/skill identity is layered: `marketplace.json` top-level `name` (`clone-app-skill`) is what `/plugin install clone-app@clone-app-skill` resolves against; each plugin's `.claude-plugin/plugin.json` carries its own `name`. The upstream plugin entry inside `marketplace.json` stays attributed to Simone Avogadro. +- Commits follow Conventional Commits scoped to the plugin: `feat(clone-app): …`, `test(clone-app): …`, `chore(clone-app): …`. +- `docs/superpowers/specs/` and `docs/superpowers/plans/` hold the design spec and the task-by-task implementation plan this plugin was built from — read them for the rationale behind a phase or script. diff --git a/plugins/clone-app/README.md b/plugins/clone-app/README.md index 66fe261..9ec68c7 100644 --- a/plugins/clone-app/README.md +++ b/plugins/clone-app/README.md @@ -10,6 +10,15 @@ NO GO), and — if you approve — generates a full implementation plan. - The `android-reverse-engineering` plugin (ships in this same repo). - Java JDK 17+ and jadx (the RE plugin auto-installs jadx if missing). - `curl`, Python 3 (stdlib only), `unzip`. +- **bash 4+**. macOS ships bash 3.2, but the RE scripts use bash-4 syntax + (`${VAR,,}`) and fail with "bad substitution" on 3.2. Install a modern bash + with `brew install bash` — `#!/usr/bin/env bash` then picks it up. + +## APK source +APKs/XAPKs are fetched from **APKCombo**. The previous APKPure direct endpoint +(`d.apkpure.com/b/APK/`) now returns an HTTP 403 Cloudflare bot challenge +for every package, so it is no longer usable from a plain `curl`. If an app is +not on APKCombo, pass a local `.apk`/`.xapk` path instead. ## Install ```text diff --git a/plugins/clone-app/skills/clone-app/SKILL.md b/plugins/clone-app/skills/clone-app/SKILL.md index b261525..a6c16e8 100644 --- a/plugins/clone-app/skills/clone-app/SKILL.md +++ b/plugins/clone-app/skills/clone-app/SKILL.md @@ -34,9 +34,11 @@ Create the working dir: `WORK="./work/$PKG"` and `mkdir -p "$WORK"`. ```bash APK="$(bash ${CLAUDE_PLUGIN_ROOT}/skills/clone-app/scripts/download-apk.sh "$PKG" "$WORK")" ``` -The script retries 3× and prints the path (`app.apk` or `app.xapk`). If it exits -non-zero, tell the user the download failed and ask for a local APK/XAPK path; -set `APK` to that path. +The script downloads from APKCombo (the old APKPure direct endpoint is now behind +a Cloudflare bot challenge), retries 3×, and prints the path (`app.apk` or +`app.xapk`). If it exits non-zero, the app may not be on APKCombo or the page +format changed — tell the user the download failed and ask for a local APK/XAPK +path; set `APK` to that path. ## Phase 2: Reverse Engineering @@ -44,7 +46,10 @@ Resolve the sibling RE scripts: ```bash RE="$(bash ${CLAUDE_PLUGIN_ROOT}/skills/clone-app/scripts/resolve-re-scripts.sh)" ``` -If it exits non-zero, show its error (RE plugin not installed) and stop. +If it exits non-zero, show its error (RE plugin not installed) and stop. If it +prints a `WARNING:` about bash version, the RE scripts need **bash 4+** (macOS +ships 3.2; their `${VAR,,}` syntax fails as "bad substitution" otherwise) — +install one with `brew install bash` before continuing, then re-run Phase 2. Run, in order, reading each output before the next: 1. `bash "$RE/fingerprint.sh" "$APK"` — framework, HTTP stack, obfuscation, SDKs, native libs. diff --git a/plugins/clone-app/skills/clone-app/scripts/download-apk.sh b/plugins/clone-app/skills/clone-app/scripts/download-apk.sh index 55a653f..b4296d0 100755 --- a/plugins/clone-app/skills/clone-app/scripts/download-apk.sh +++ b/plugins/clone-app/skills/clone-app/scripts/download-apk.sh @@ -1,35 +1,80 @@ #!/usr/bin/env bash set -euo pipefail +# Download an APK/XAPK for a package by its Android package name. +# +# Source: APKCombo. The historical APKPure endpoint +# (d.apkpure.com/b/APK/?version=latest) is now behind a Cloudflare +# bot challenge and returns HTTP 403 "Just a moment..." for every package, +# so a plain curl can no longer use it. +# +# APKCombo serves the real artifact in two steps, both reachable with a +# normal browser User-Agent and no JavaScript: +# 1. GET https://apkcombo.com/app//download/apk -> an HTML page that +# embeds the signed download as a relative /r2?u= +# link (the URL slug segment is ignored by the server, so a fixed "app" +# works for any package). +# 2. GET https://apkcombo.com -> the apk/xapk bytes. +# +# CLONE_APP_CURL overrides the curl binary (used by the tests to inject a stub). + package="${1:-}" out_dir="${2:-}" curl_bin="${CLONE_APP_CURL:-curl}" +UA="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36" +REFERER="https://apkcombo.com/" +page_url="https://apkcombo.com/app/${package}/download/apk" + if [[ -z "$package" || -z "$out_dir" ]]; then echo "ERROR: usage: download-apk.sh " >&2 exit 1 fi mkdir -p "$out_dir" +page_file="$out_dir/download-page.html" tmp_file="$out_dir/app.download" -url="https://d.apkpure.com/b/APK/${package}?version=latest" +# --- Step 1: fetch the download page and extract the signed /r2 link --------- +r2_path="" +for attempt in 1 2 3; do + if "$curl_bin" -sL -A "$UA" -H "Referer: $REFERER" "$page_url" --output "$page_file" 2>/dev/null; then + r2_path="$(grep -oE '/r2\?u=[^"]*' "$page_file" | head -1 || true)" + [[ -n "$r2_path" ]] && break + fi + echo "download-page attempt $attempt failed (or no link yet), retrying..." >&2 + sleep 1 +done +rm -f "$page_file" + +if [[ -z "$r2_path" ]]; then + echo "ERROR: no download link found for '$package' on APKCombo ($page_url)." >&2 + echo " The app may not be available there, or the page format changed." >&2 + exit 1 +fi + +# Links in the HTML use raw '&'; decode '&' defensively in case that changes. +r2_path="${r2_path//&/&}" +artifact_url="https://apkcombo.com${r2_path}" + +# --- Step 2: download the actual artifact ------------------------------------ ok=0 for attempt in 1 2 3; do - if "$curl_bin" -L --fail "$url" --output "$tmp_file" 2>/dev/null; then + if "$curl_bin" -sL --fail -A "$UA" -H "Referer: $REFERER" "$artifact_url" --output "$tmp_file" 2>/dev/null; then ok=1; break fi - echo "download attempt $attempt failed, retrying..." >&2 + echo "artifact download attempt $attempt failed, retrying..." >&2 sleep 1 done if [[ "$ok" -ne 1 ]]; then - echo "ERROR: failed to download $package after 3 attempts from $url" >&2 + echo "ERROR: failed to download $package artifact after 3 attempts." >&2 rm -f "$tmp_file" exit 1 fi -# Decide extension: both APK and XAPK are ZIPs. XAPK = zip containing manifest.json + >=2 .apk entries. +# Decide extension: both APK and XAPK are ZIPs. XAPK = a zip bundle containing +# manifest.json plus at least one split .apk entry; a plain APK has neither. ext="apk" if entries="$(unzip -Z1 "$tmp_file" 2>/dev/null)"; then apk_count="$(grep -c '\.apk$' <<<"$entries" || true)" diff --git a/plugins/clone-app/skills/clone-app/scripts/resolve-re-scripts.sh b/plugins/clone-app/skills/clone-app/scripts/resolve-re-scripts.sh index 7b884f0..276e8fe 100755 --- a/plugins/clone-app/skills/clone-app/scripts/resolve-re-scripts.sh +++ b/plugins/clone-app/skills/clone-app/scripts/resolve-re-scripts.sh @@ -20,4 +20,17 @@ if [[ ! -d "$re_scripts" ]]; then exit 1 fi +# The RE scripts (fingerprint.sh, find-api-calls.sh) use bash 4+ syntax such as +# ${VAR,,}. macOS ships bash 3.2, where those fail with "bad substitution". +# SKILL.md invokes the RE scripts as `bash "$RE/..."`, so the version that +# matters is whatever `bash` resolves to on PATH — warn (not fatal: the path +# still resolves, and a caller may have a newer bash elsewhere) so the user +# gets an actionable message instead of a cryptic substitution error. +re_bash_major="$(bash -c 'echo "${BASH_VERSINFO[0]:-0}"' 2>/dev/null || echo 0)" +if [[ "$re_bash_major" -lt 4 ]]; then + echo "WARNING: 'bash' on PATH is version ${re_bash_major}.x; the reverse-engineering" >&2 + echo " scripts need bash 4+ (they use \${VAR,,} etc.) and will fail otherwise." >&2 + echo " Install a modern bash, e.g.: brew install bash" >&2 +fi + echo "$re_scripts" diff --git a/plugins/clone-app/tests/test-download-apk.sh b/plugins/clone-app/tests/test-download-apk.sh index 18560ce..eda5ed6 100644 --- a/plugins/clone-app/tests/test-download-apk.sh +++ b/plugins/clone-app/tests/test-download-apk.sh @@ -6,16 +6,34 @@ check() { [[ "$2" == "$3" ]] && echo "PASS: $1" || { echo "FAIL: $1 — expected tmp="$(mktemp -d)" -# Stub curl: writes a minimal zip containing manifest.json + two apk entries → xapk -cat > "$tmp/fakecurl-xapk.sh" <<'EOF' +# Find the --output target in a curl arg list, and the request URL (http... arg). +# Shared helper sourced by the stubs below. +cat > "$tmp/argparse.sh" <<'EOF' +out=""; url=""; prev="" +for a in "$@"; do + [[ "$prev" == "--output" || "$prev" == "-o" ]] && out="$a" + [[ "$a" == http*://* ]] && url="$a" + prev="$a" +done +EOF + +# Stub curl modelling the real two-step APKCombo flow: +# call 1 (URL contains /download/apk) -> emit HTML page with an /r2?u= link +# call 2 (URL contains /r2?u=) -> emit a zip (manifest.json + 2 apk) => xapk +cat > "$tmp/fakecurl-xapk.sh" < "$workdir/manifest.json" -echo 'a' > "$workdir/base.apk"; echo 'b' > "$workdir/config.apk" -( cd "$workdir" && zip -q -r "$out" . ) -exit 0 +source "$tmp/argparse.sh" +if [[ "\$url" == *"/download/apk"* ]]; then + printf '%s' 'Download' > "\$out" + exit 0 +elif [[ "\$url" == *"/r2?u="* ]]; then + workdir="\$(mktemp -d)" + echo '{}' > "\$workdir/manifest.json" + echo 'a' > "\$workdir/base.apk"; echo 'b' > "\$workdir/config.apk" + ( cd "\$workdir" && zip -q -r "\$out" . ) + exit 0 +fi +exit 1 EOF chmod +x "$tmp/fakecurl-xapk.sh" @@ -23,14 +41,27 @@ path="$(CLONE_APP_CURL="$tmp/fakecurl-xapk.sh" bash "$SCRIPT" com.example.app "$ check "xapk exit 0" "0" "$rc" check "xapk extension" "xapk" "${path##*.}" -# Stub curl that always fails → exit 1 after retries +# Stub where the download page loads but contains NO /r2 link -> exit 1 +cat > "$tmp/fakecurl-nolink.sh" <no link here' > "\$out"; exit 0 +fi +exit 1 +EOF +chmod +x "$tmp/fakecurl-nolink.sh" +out2="$(CLONE_APP_CURL="$tmp/fakecurl-nolink.sh" bash "$SCRIPT" com.example.app "$tmp/out2" 2>/dev/null)"; rc2=$? +check "no-link exit 1" "1" "$rc2" + +# Stub curl that always fails -> exit 1 after retries cat > "$tmp/fakecurl-fail.sh" <<'EOF' #!/usr/bin/env bash exit 22 EOF chmod +x "$tmp/fakecurl-fail.sh" -out2="$(CLONE_APP_CURL="$tmp/fakecurl-fail.sh" bash "$SCRIPT" com.example.app "$tmp/out2" 2>/dev/null)"; rc2=$? -check "fail exit 1" "1" "$rc2" +out3="$(CLONE_APP_CURL="$tmp/fakecurl-fail.sh" bash "$SCRIPT" com.example.app "$tmp/out3" 2>/dev/null)"; rc3=$? +check "fail exit 1" "1" "$rc3" rm -rf "$tmp" exit $fail