fix(clone-app): switch APK source to APKCombo and guard bash version
The APKPure direct endpoint (d.apkpure.com/b/APK/<pkg>) now returns an HTTP
403 Cloudflare bot challenge for every package, so downloads always failed —
which also blocked the entire reverse-engineering chain downstream (it never
ran because Phase 1 produced no APK).
- download-apk.sh: fetch the APKCombo download page with a browser UA, extract
the embedded /r2?u=<signed-url> link, and download that. No JS needed; the
URL slug is ignored so a literal 'app' works for any package. Verified
end-to-end against com.wonder (Elevate) -> 126MB xapk -> RE fingerprint.
- test-download-apk.sh: model the two-step flow and the page-without-link case.
- resolve-re-scripts.sh: warn (stderr) when bash < 4, since the upstream RE
scripts use ${VAR,,} and fail with 'bad substitution' on macOS bash 3.2.
- SKILL.md / README.md / CLAUDE.md: document the APKCombo source and bash 4+ req.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
fatih.bulut
parent
b7b62d5f14
commit
b1135b82a2
|
|
@ -0,0 +1,70 @@
|
|||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this repo is
|
||||
|
||||
A Claude Code **plugin marketplace** (a fork of `SimoneAvogadro/android-reverse-engineering-skill`) hosting two plugins:
|
||||
|
||||
- `plugins/android-reverse-engineering/` — **upstream, vendored. Never modify.** Decompiles APK/XAPK/JAR/AAR with jadx, extracts HTTP APIs, recovers R8-obfuscated Kotlin names.
|
||||
- `plugins/clone-app/` — **this project's work.** A `/clone-app` skill that takes a Google Play URL, drives the RE plugin to reverse engineer the app, scrapes store metrics, estimates AI-assisted clone effort + infra cost, writes a viability report, and optionally hands off to the `writing-plans` skill.
|
||||
|
||||
There is no compiled artifact and no application runtime. "The code" is bash + Python helper scripts plus Markdown skill/command/reference docs that Claude Code loads at session time.
|
||||
|
||||
## The hard constraint
|
||||
|
||||
`plugins/android-reverse-engineering/` is the upstream tree. Keeping it byte-identical is what makes `git pull upstream master` conflict-free. Before committing, this must print nothing:
|
||||
|
||||
```bash
|
||||
git status --porcelain plugins/android-reverse-engineering/
|
||||
```
|
||||
|
||||
Two remotes: `origin` = `masa2146/clone-app-skill` (this fork), `upstream` = `SimoneAvogadro/android-reverse-engineering-skill`. Sync upstream with `git pull upstream master`. The only shared files clone-app touches are root `.claude-plugin/marketplace.json` (added a second plugin entry) and—if ever—root `README.md`; everything else lives under `plugins/clone-app/`.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Full clone-app test suite (5 suites, ~28 assertions). Run from repo root.
|
||||
bash plugins/clone-app/tests/run-all.sh
|
||||
|
||||
# A single test
|
||||
bash plugins/clone-app/tests/test-extract-package.sh
|
||||
python3 plugins/clone-app/tests/test-scrape-play-store.py
|
||||
|
||||
# Structural smoke test: files present, scripts executable, JSON valid, both plugins in marketplace
|
||||
bash plugins/clone-app/tests/smoke-structure.sh
|
||||
|
||||
# Validate the two JSON manifests
|
||||
python3 -c "import json; json.load(open('.claude-plugin/marketplace.json')); json.load(open('plugins/clone-app/.claude-plugin/plugin.json'))"
|
||||
```
|
||||
|
||||
Shell is zsh but every script uses `#!/usr/bin/env bash` — invoke with `bash <path>`, not `sh`. `shopt` etc. fail under zsh; wrap in `bash -c '...'`. Python is stdlib-only (`urllib`, `json`, `re`); no pip, no virtualenv.
|
||||
|
||||
**bash 4+ is required at runtime.** The upstream RE scripts (`fingerprint.sh`, `find-api-calls.sh`) use `${VAR,,}` and break with "bad substitution" on macOS's stock bash 3.2. You cannot patch them (upstream-untouched rule), so the fix is `brew install bash`; `#!/usr/bin/env bash` then resolves to the modern one. `resolve-re-scripts.sh` emits a stderr WARNING when it detects bash < 4.
|
||||
|
||||
**APK source is APKCombo, not APKPure.** The old `d.apkpure.com/b/APK/<pkg>` endpoint now returns an HTTP 403 Cloudflare challenge for every package. `download-apk.sh` instead GETs `apkcombo.com/app/<pkg>/download/apk` with a browser User-Agent, greps the embedded `/r2?u=<signed-url>` link out of the HTML, and downloads that (no JavaScript needed; the URL slug segment is ignored by the server so a literal `app` works for any package).
|
||||
|
||||
## How clone-app is wired
|
||||
|
||||
`plugins/clone-app/skills/clone-app/SKILL.md` is the orchestrator — an 8-phase (0–7) prose workflow Claude executes. Phases split into two kinds:
|
||||
|
||||
- **Deterministic steps → helper scripts** under `skills/clone-app/scripts/`, each independently testable: `extract-package.sh` (Play URL → package), `download-apk.sh` (APKCombo two-step download, 3 retries, apk-vs-xapk by ZIP contents), `resolve-re-scripts.sh` (locates the sibling RE plugin's `scripts/` dir + warns if `bash` is < 4), `scrape-play-store.py` (Play page → metrics JSON via embedded ld+json), `check-appstore.py` (iTunes Search → iOS presence).
|
||||
- **AI-judgment steps → reference rubrics** under `skills/clone-app/references/` (`stack-recommendation-guide`, `effort-estimation-guide`, `infra-cost-guide`, `report-template`). These keep effort/cost/verdict output consistent; edit the rubric, not the prose, to change how estimates are produced.
|
||||
|
||||
clone-app never hardcodes RE script paths. It calls `resolve-re-scripts.sh`, which from `plugins/clone-app/` walks to `plugins/android-reverse-engineering/skills/android-reverse-engineering/scripts/` (honoring `$CLAUDE_PLUGIN_ROOT`, falling back to deriving its own location). Phase 2 then invokes that dir's `fingerprint.sh`, `check-deps.sh`, `decompile.sh -o "$WORK/output"`, `recover-kotlin-names.sh`, `find-api-calls.sh`.
|
||||
|
||||
Two runtime conventions the scripts and SKILL.md share:
|
||||
- **Working dir** is `./work/{package}/` relative to the user's cwd (never inside the plugin). Decompile output must land at `$WORK/output/` via `decompile.sh -o` — `decompile.sh` otherwise defaults to a relative `<basename>-decompiled` dir in cwd.
|
||||
- **Effort is measured in "AI Sprints"** (one focused Claude session, ~2–4h review), never calendar time.
|
||||
|
||||
SKILL.md pauses for the user at exactly two points: Phase 4 (choose clone stack) and Phase 7 (proceed to implementation plan?). Phase 7's "yes" path invokes `superpowers:writing-plans`.
|
||||
|
||||
## Test pattern
|
||||
|
||||
Bash tests use `set -uo pipefail` (intentionally not `-e`) and aggregate failures into a `fail` var so every assertion runs; the script exits non-zero if any failed. Python scrapers are tested offline against `tests/fixtures/` via `--html-file` / `--json-file` flags — never hitting the network. New scrape logic needs a fixture, not a live call.
|
||||
|
||||
## Conventions
|
||||
|
||||
- Plugin/skill identity is layered: `marketplace.json` top-level `name` (`clone-app-skill`) is what `/plugin install clone-app@clone-app-skill` resolves against; each plugin's `.claude-plugin/plugin.json` carries its own `name`. The upstream plugin entry inside `marketplace.json` stays attributed to Simone Avogadro.
|
||||
- Commits follow Conventional Commits scoped to the plugin: `feat(clone-app): …`, `test(clone-app): …`, `chore(clone-app): …`.
|
||||
- `docs/superpowers/specs/` and `docs/superpowers/plans/` hold the design spec and the task-by-task implementation plan this plugin was built from — read them for the rationale behind a phase or script.
|
||||
|
|
@ -10,6 +10,15 @@ NO GO), and — if you approve — generates a full implementation plan.
|
|||
- The `android-reverse-engineering` plugin (ships in this same repo).
|
||||
- Java JDK 17+ and jadx (the RE plugin auto-installs jadx if missing).
|
||||
- `curl`, Python 3 (stdlib only), `unzip`.
|
||||
- **bash 4+**. macOS ships bash 3.2, but the RE scripts use bash-4 syntax
|
||||
(`${VAR,,}`) and fail with "bad substitution" on 3.2. Install a modern bash
|
||||
with `brew install bash` — `#!/usr/bin/env bash` then picks it up.
|
||||
|
||||
## APK source
|
||||
APKs/XAPKs are fetched from **APKCombo**. The previous APKPure direct endpoint
|
||||
(`d.apkpure.com/b/APK/<pkg>`) now returns an HTTP 403 Cloudflare bot challenge
|
||||
for every package, so it is no longer usable from a plain `curl`. If an app is
|
||||
not on APKCombo, pass a local `.apk`/`.xapk` path instead.
|
||||
|
||||
## Install
|
||||
```text
|
||||
|
|
|
|||
|
|
@ -34,9 +34,11 @@ Create the working dir: `WORK="./work/$PKG"` and `mkdir -p "$WORK"`.
|
|||
```bash
|
||||
APK="$(bash ${CLAUDE_PLUGIN_ROOT}/skills/clone-app/scripts/download-apk.sh "$PKG" "$WORK")"
|
||||
```
|
||||
The script retries 3× and prints the path (`app.apk` or `app.xapk`). If it exits
|
||||
non-zero, tell the user the download failed and ask for a local APK/XAPK path;
|
||||
set `APK` to that path.
|
||||
The script downloads from APKCombo (the old APKPure direct endpoint is now behind
|
||||
a Cloudflare bot challenge), retries 3×, and prints the path (`app.apk` or
|
||||
`app.xapk`). If it exits non-zero, the app may not be on APKCombo or the page
|
||||
format changed — tell the user the download failed and ask for a local APK/XAPK
|
||||
path; set `APK` to that path.
|
||||
|
||||
## Phase 2: Reverse Engineering
|
||||
|
||||
|
|
@ -44,7 +46,10 @@ Resolve the sibling RE scripts:
|
|||
```bash
|
||||
RE="$(bash ${CLAUDE_PLUGIN_ROOT}/skills/clone-app/scripts/resolve-re-scripts.sh)"
|
||||
```
|
||||
If it exits non-zero, show its error (RE plugin not installed) and stop.
|
||||
If it exits non-zero, show its error (RE plugin not installed) and stop. If it
|
||||
prints a `WARNING:` about bash version, the RE scripts need **bash 4+** (macOS
|
||||
ships 3.2; their `${VAR,,}` syntax fails as "bad substitution" otherwise) —
|
||||
install one with `brew install bash` before continuing, then re-run Phase 2.
|
||||
|
||||
Run, in order, reading each output before the next:
|
||||
1. `bash "$RE/fingerprint.sh" "$APK"` — framework, HTTP stack, obfuscation, SDKs, native libs.
|
||||
|
|
|
|||
|
|
@ -1,35 +1,80 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Download an APK/XAPK for a package by its Android package name.
|
||||
#
|
||||
# Source: APKCombo. The historical APKPure endpoint
|
||||
# (d.apkpure.com/b/APK/<pkg>?version=latest) is now behind a Cloudflare
|
||||
# bot challenge and returns HTTP 403 "Just a moment..." for every package,
|
||||
# so a plain curl can no longer use it.
|
||||
#
|
||||
# APKCombo serves the real artifact in two steps, both reachable with a
|
||||
# normal browser User-Agent and no JavaScript:
|
||||
# 1. GET https://apkcombo.com/app/<pkg>/download/apk -> an HTML page that
|
||||
# embeds the signed download as a relative /r2?u=<url-encoded-signed-url>
|
||||
# link (the URL slug segment is ignored by the server, so a fixed "app"
|
||||
# works for any package).
|
||||
# 2. GET https://apkcombo.com<that /r2 path> -> the apk/xapk bytes.
|
||||
#
|
||||
# CLONE_APP_CURL overrides the curl binary (used by the tests to inject a stub).
|
||||
|
||||
package="${1:-}"
|
||||
out_dir="${2:-}"
|
||||
curl_bin="${CLONE_APP_CURL:-curl}"
|
||||
|
||||
UA="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36"
|
||||
REFERER="https://apkcombo.com/"
|
||||
page_url="https://apkcombo.com/app/${package}/download/apk"
|
||||
|
||||
if [[ -z "$package" || -z "$out_dir" ]]; then
|
||||
echo "ERROR: usage: download-apk.sh <package> <out-dir>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p "$out_dir"
|
||||
page_file="$out_dir/download-page.html"
|
||||
tmp_file="$out_dir/app.download"
|
||||
url="https://d.apkpure.com/b/APK/${package}?version=latest"
|
||||
|
||||
# --- Step 1: fetch the download page and extract the signed /r2 link ---------
|
||||
r2_path=""
|
||||
for attempt in 1 2 3; do
|
||||
if "$curl_bin" -sL -A "$UA" -H "Referer: $REFERER" "$page_url" --output "$page_file" 2>/dev/null; then
|
||||
r2_path="$(grep -oE '/r2\?u=[^"]*' "$page_file" | head -1 || true)"
|
||||
[[ -n "$r2_path" ]] && break
|
||||
fi
|
||||
echo "download-page attempt $attempt failed (or no link yet), retrying..." >&2
|
||||
sleep 1
|
||||
done
|
||||
rm -f "$page_file"
|
||||
|
||||
if [[ -z "$r2_path" ]]; then
|
||||
echo "ERROR: no download link found for '$package' on APKCombo ($page_url)." >&2
|
||||
echo " The app may not be available there, or the page format changed." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Links in the HTML use raw '&'; decode '&' defensively in case that changes.
|
||||
r2_path="${r2_path//&/&}"
|
||||
artifact_url="https://apkcombo.com${r2_path}"
|
||||
|
||||
# --- Step 2: download the actual artifact ------------------------------------
|
||||
ok=0
|
||||
for attempt in 1 2 3; do
|
||||
if "$curl_bin" -L --fail "$url" --output "$tmp_file" 2>/dev/null; then
|
||||
if "$curl_bin" -sL --fail -A "$UA" -H "Referer: $REFERER" "$artifact_url" --output "$tmp_file" 2>/dev/null; then
|
||||
ok=1; break
|
||||
fi
|
||||
echo "download attempt $attempt failed, retrying..." >&2
|
||||
echo "artifact download attempt $attempt failed, retrying..." >&2
|
||||
sleep 1
|
||||
done
|
||||
|
||||
if [[ "$ok" -ne 1 ]]; then
|
||||
echo "ERROR: failed to download $package after 3 attempts from $url" >&2
|
||||
echo "ERROR: failed to download $package artifact after 3 attempts." >&2
|
||||
rm -f "$tmp_file"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Decide extension: both APK and XAPK are ZIPs. XAPK = zip containing manifest.json + >=2 .apk entries.
|
||||
# Decide extension: both APK and XAPK are ZIPs. XAPK = a zip bundle containing
|
||||
# manifest.json plus at least one split .apk entry; a plain APK has neither.
|
||||
ext="apk"
|
||||
if entries="$(unzip -Z1 "$tmp_file" 2>/dev/null)"; then
|
||||
apk_count="$(grep -c '\.apk$' <<<"$entries" || true)"
|
||||
|
|
|
|||
|
|
@ -20,4 +20,17 @@ if [[ ! -d "$re_scripts" ]]; then
|
|||
exit 1
|
||||
fi
|
||||
|
||||
# The RE scripts (fingerprint.sh, find-api-calls.sh) use bash 4+ syntax such as
|
||||
# ${VAR,,}. macOS ships bash 3.2, where those fail with "bad substitution".
|
||||
# SKILL.md invokes the RE scripts as `bash "$RE/..."`, so the version that
|
||||
# matters is whatever `bash` resolves to on PATH — warn (not fatal: the path
|
||||
# still resolves, and a caller may have a newer bash elsewhere) so the user
|
||||
# gets an actionable message instead of a cryptic substitution error.
|
||||
re_bash_major="$(bash -c 'echo "${BASH_VERSINFO[0]:-0}"' 2>/dev/null || echo 0)"
|
||||
if [[ "$re_bash_major" -lt 4 ]]; then
|
||||
echo "WARNING: 'bash' on PATH is version ${re_bash_major}.x; the reverse-engineering" >&2
|
||||
echo " scripts need bash 4+ (they use \${VAR,,} etc.) and will fail otherwise." >&2
|
||||
echo " Install a modern bash, e.g.: brew install bash" >&2
|
||||
fi
|
||||
|
||||
echo "$re_scripts"
|
||||
|
|
|
|||
|
|
@ -6,16 +6,34 @@ check() { [[ "$2" == "$3" ]] && echo "PASS: $1" || { echo "FAIL: $1 — expected
|
|||
|
||||
tmp="$(mktemp -d)"
|
||||
|
||||
# Stub curl: writes a minimal zip containing manifest.json + two apk entries → xapk
|
||||
cat > "$tmp/fakecurl-xapk.sh" <<'EOF'
|
||||
# Find the --output target in a curl arg list, and the request URL (http... arg).
|
||||
# Shared helper sourced by the stubs below.
|
||||
cat > "$tmp/argparse.sh" <<'EOF'
|
||||
out=""; url=""; prev=""
|
||||
for a in "$@"; do
|
||||
[[ "$prev" == "--output" || "$prev" == "-o" ]] && out="$a"
|
||||
[[ "$a" == http*://* ]] && url="$a"
|
||||
prev="$a"
|
||||
done
|
||||
EOF
|
||||
|
||||
# Stub curl modelling the real two-step APKCombo flow:
|
||||
# call 1 (URL contains /download/apk) -> emit HTML page with an /r2?u= link
|
||||
# call 2 (URL contains /r2?u=) -> emit a zip (manifest.json + 2 apk) => xapk
|
||||
cat > "$tmp/fakecurl-xapk.sh" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
out=""; prev=""
|
||||
for a in "$@"; do [[ "$prev" == "--output" ]] && out="$a"; prev="$a"; done
|
||||
workdir="$(mktemp -d)"
|
||||
echo '{}' > "$workdir/manifest.json"
|
||||
echo 'a' > "$workdir/base.apk"; echo 'b' > "$workdir/config.apk"
|
||||
( cd "$workdir" && zip -q -r "$out" . )
|
||||
exit 0
|
||||
source "$tmp/argparse.sh"
|
||||
if [[ "\$url" == *"/download/apk"* ]]; then
|
||||
printf '%s' '<a href="/r2?u=https%3A%2F%2Fexample%2Ftest.xapk">Download</a>' > "\$out"
|
||||
exit 0
|
||||
elif [[ "\$url" == *"/r2?u="* ]]; then
|
||||
workdir="\$(mktemp -d)"
|
||||
echo '{}' > "\$workdir/manifest.json"
|
||||
echo 'a' > "\$workdir/base.apk"; echo 'b' > "\$workdir/config.apk"
|
||||
( cd "\$workdir" && zip -q -r "\$out" . )
|
||||
exit 0
|
||||
fi
|
||||
exit 1
|
||||
EOF
|
||||
chmod +x "$tmp/fakecurl-xapk.sh"
|
||||
|
||||
|
|
@ -23,14 +41,27 @@ path="$(CLONE_APP_CURL="$tmp/fakecurl-xapk.sh" bash "$SCRIPT" com.example.app "$
|
|||
check "xapk exit 0" "0" "$rc"
|
||||
check "xapk extension" "xapk" "${path##*.}"
|
||||
|
||||
# Stub curl that always fails → exit 1 after retries
|
||||
# Stub where the download page loads but contains NO /r2 link -> exit 1
|
||||
cat > "$tmp/fakecurl-nolink.sh" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
source "$tmp/argparse.sh"
|
||||
if [[ "\$url" == *"/download/apk"* ]]; then
|
||||
printf '%s' '<html>no link here</html>' > "\$out"; exit 0
|
||||
fi
|
||||
exit 1
|
||||
EOF
|
||||
chmod +x "$tmp/fakecurl-nolink.sh"
|
||||
out2="$(CLONE_APP_CURL="$tmp/fakecurl-nolink.sh" bash "$SCRIPT" com.example.app "$tmp/out2" 2>/dev/null)"; rc2=$?
|
||||
check "no-link exit 1" "1" "$rc2"
|
||||
|
||||
# Stub curl that always fails -> exit 1 after retries
|
||||
cat > "$tmp/fakecurl-fail.sh" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
exit 22
|
||||
EOF
|
||||
chmod +x "$tmp/fakecurl-fail.sh"
|
||||
out2="$(CLONE_APP_CURL="$tmp/fakecurl-fail.sh" bash "$SCRIPT" com.example.app "$tmp/out2" 2>/dev/null)"; rc2=$?
|
||||
check "fail exit 1" "1" "$rc2"
|
||||
out3="$(CLONE_APP_CURL="$tmp/fakecurl-fail.sh" bash "$SCRIPT" com.example.app "$tmp/out3" 2>/dev/null)"; rc3=$?
|
||||
check "fail exit 1" "1" "$rc3"
|
||||
|
||||
rm -rf "$tmp"
|
||||
exit $fail
|
||||
|
|
|
|||
Loading…
Reference in New Issue