fix(ci): isolate chaos verification by caller workflow (#13208)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud deployments require verified artifacts for the merged source
commit.
> - Cloud readiness and the npm release independently run the same
source checks.
> - Their shared chaos workflow used only the source ref as its
concurrency key.
> - One caller could cancel the other caller's required job for the same
commit.
> - This pull request scopes that key to the caller workflow and source
ref.
> - Both callers can finish their checks without blocking deployment
readiness.

## Linked Issues or Issue Description

Refs #13192 and #13205. Searched for related open issues and PRs; no
duplicate fix was found.

**What happened?**

The master push for `398d304e15739d1ee6105633bd8a0e42c929d33f` started
Cloud readiness and Release together. GitHub cancelled the Cloud
readiness chaos job before it acquired a runner. Its annotation reported
a higher-priority waiting request for the same concurrency group. The
required readiness gate cannot pass after that cancellation.

**Expected behavior**

Cloud readiness and Release must each finish source verification for the
same SHA. Standalone chaos evals must also have a separate group.

**Steps to reproduce**

Merge a commit to master while the npm release queue is empty. Both
callers reach the reusable chaos workflow with the same source SHA. See
[the cancelled
job](https://github.com/paperclipai/paperclip/actions/runs/34569569760/job/103168603926).

**Paperclip version or commit**

`398d304e15739d1ee6105633bd8a0e42c929d33f`.

**Deployment mode**

GitHub Actions on master.

## What Changed

- Add the caller workflow name to the chaos workflow concurrency group.
Retain source isolation and cancellation of duplicate calls within the
same workflow.
- Add a regression test that evaluates the group for Cloud readiness,
Release, and standalone evals at the same source SHA.
- Document the concurrency boundary in the readiness runbook.

## Verification

- `node --test scripts/preview-artifacts.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs` passed: 26 tests.
- The new regression test fails against the previous concurrency key and
passes with this fix.
- `actionlint -shellcheck= -pyflakes=
.github/workflows/runner-chaos-evals.yml
.github/workflows/release-verify.yml
.github/workflows/cloud-readiness.yml` passed.
- `git diff --check` passed.
- The full local typecheck passed for the same application source in
#13205. Its macOS general-server test phase had 10,471 passes and 70
failures in seven unchanged application test files: missing Cargo/Runner
test binaries, filesystem permissions, timeouts, a port conflict, and a
load-test count mismatch. Linux CI test checks passed. The full local
build passed with Cargo on PATH. This PR changes workflow configuration,
its test, and documentation only.
- All CI checks pass on the final head, including typecheck, tests,
browser suites, build, and canary dry run. Greptile is 5/5 with no open
findings. After merge, verify both callers' chaos jobs complete for the
same master SHA and record the resulting readiness time.

## Risks

- Two callers may now run chaos tests at the same time. This uses two
existing GitHub runners, which is the intended cost of independent
verification.
- Renaming a caller changes its concurrency group. The fixed prefix
keeps this child group separate from caller-level concurrency groups.
- The readiness gate continues to require every verification
prerequisite. No gate is bypassed.

## Model Used

- OpenAI GPT-6 / Codex, with reasoning, repository editing, and
command/API tools. Exact serving model ID and context-window size are
not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (26 focused
workflow/artifact tests)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>

.github/workflows/runner-chaos-evals.yml

@@ -12,7 +12,9 @@ on:
         type: string
 
 concurrency:
-  group: runner-chaos-evals-${{ inputs.ref || github.ref }}
+  # Reusable calls inherit the caller's workflow name. Cloud readiness and
+  # Release verify the same SHA independently and must not cancel each other.
+  group: runner-chaos-evals-${{ github.workflow }}-${{ inputs.ref || github.ref }}
   cancel-in-progress: true
 
 jobs:

doc/cloud-build-readiness.md

@@ -50,6 +50,10 @@ successfully verified a real master commit.
 
 ## Timing and rollout
 
+The reusable Runner chaos workflow scopes concurrency to the caller workflow
+and source ref. Cloud readiness and the npm release can verify the same commit
+at the same time. They must not cancel each other's required test job.
+
 Measure the complete path from a master merge to a healthy target running that
 exact commit. Keep readiness and deployment as separate milestones:
 

scripts/__tests__/release-verify-workflow.test.mjs

@@ -14,6 +14,27 @@ function readWorkflow(name) {
   return readFileSync(path.join(repoRoot, ".github/workflows", name), "utf8");
 }
 
+test("chaos verification isolates callers that verify the same source commit", () => {
+  const chaosWorkflow = readWorkflow("runner-chaos-evals.yml");
+  const group = chaosWorkflow.match(/^  group: (.+)$/m)?.[1];
+  assert.ok(group, "chaos verification must define its concurrency group");
+
+  // GitHub supplies the top-level caller's workflow name to reusable calls.
+  const resolveGroup = (caller, ref) => group
+    .replaceAll("${{ github.workflow }}", readWorkflow(caller).match(/^name: (.+)$/m)[1])
+    .replaceAll("${{ inputs.ref || github.ref }}", ref)
+    .toLowerCase();
+  const sha = "a".repeat(40);
+  const callers = ["cloud-readiness.yml", "release.yml", "runner-chaos-evals.yml"];
+  const groups = callers.map((caller) => resolveGroup(caller, sha));
+  assert.equal(new Set(groups).size, callers.length,
+    "Cloud readiness, Release, and standalone evals must not cancel each other");
+  assert.ok(groups.every((value) => !value.includes("${{")), "resolve every group input");
+  assert.notEqual(resolveGroup("cloud-readiness.yml", sha),
+    resolveGroup("cloud-readiness.yml", "b".repeat(40)), "different sources remain independent");
+  assert.match(chaosWorkflow, /cancel-in-progress: true/);
+});
+
 test("release workflow delegates stable and canary verification to the reusable workflow", () => {
   const releaseWorkflow = readWorkflow("release.yml");