From 03609aa6ecc9a047ed53d6b6469d8be554fbc46d Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Fri, 4 Sep 2026 16:39:13 -0700 Subject: [PATCH] ci: keep traceability regression tests in the Docker build context (#12858) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - GitHub Actions builds the Docker images that ship Paperclip, and the image build re-runs the runner's committed-artifact checks. > - PR #12855 restored the capability-contract files that PR #12769's context slimming stripped, and image builds then progressed one step further in the chain. > - The next check, `check:runner-workflow-traceability`, access()es every regression test its spec names — `src/**/*.test.ts` files that the same slimming block also strips. > - Every image build since #12855 merged now fails there with ENOENT, so image publishing is still down. > - This pull request restores those files with one more narrow exception and teaches the context probe to derive the required paths from the spec itself. > - The benefit is that image publishing recovers, and the probe now covers this input class without a hand-maintained path list that could rot. ## Linked Issues or Issue Description Refs #12855 (first restoration from the same incident) and #12769 (the context-slimming change). **What happened?** After #12855 merged, every `Docker` workflow run on master still failed, now inside `check:runner-workflow-traceability`: `Error: ENOENT ... access '/app/packages/paperclip-runner/src/contracts/native-execution.test.ts'`. The check access()es all 29 regression tests named by `spec/evals/stress-workflow-traceability.json`; they are `src/**/*.test.ts` files, and the `packages/paperclip-runner/**/*.test.ts` ignore rule strips them from the build context. **Expected behavior** The Docker build context must contain every file the image build reads. The context-integrity probe must catch this class on the pull request, including inputs named dynamically by a spec. **Steps to reproduce** 1. Check out master after #12855. 2. Run `docker buildx build -f .github/docker-context-checks.Dockerfile .` with this PR's probe, or the real `Docker` workflow build. 3. Observe the ENOENT above; with this PR's `.dockerignore` exception, both pass. **Paperclip version or commit** `bb920fb8` (first post-#12855 failing image build) through master tip. **Deployment mode** GitHub Actions image builds (`docker.yml`), consumed by managed cloud deployments. ## What Changed - `.dockerignore`: re-include `packages/paperclip-runner/src/**/*.test.ts` and `.tsx` — the traceability spec references only files under `src`, so the remaining test exclusions stay. - `.github/docker-context-checks.Dockerfile`: new spec-driven existence walk that replicates the traceability check's own access() loop against the exact build context. The path list comes from the spec at probe time, so a future spec change is covered automatically; the check itself still runs only inside the real image build, where `dist/` exists. ## Verification - `docker buildx build -f .github/docker-context-checks.Dockerfile .` without the `.dockerignore` exception: fails with the exact production ENOENT (`src/contracts/native-execution.test.ts`). - Same command with the exception: passes end to end (all probe stages, including the drift checks from #12855). - Static re-sweep of the remaining image-build chain steps (`build:binary`, replay goldens, semantic-action catalog) against the ignore rules: their inputs are all in the context; cargo needs no `tests` directories (no crate declares an explicit `[[test]]` target). ## Risks - Low. The exception re-adds source test files to the build context only; image contents do not change (tests are neither compiled into the production output nor run in the image build — the check only requires that the referenced files exist). - The probe addition is one dependency-free Node one-liner. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Claude Fable 5 (Anthropic, model id `claude-fable-5`), extended thinking, agentic tool use in Claude Code: GitHub Actions log forensics, spec-driven path inventory, and local docker buildx verification in both failing and fixed states. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (the probe in both failing-before and passing-after states) - [x] I have added or updated tests where applicable (the spec-driven probe walk is the regression test) - [x] I have updated relevant documentation to reflect my changes (inline comments explain the invariant) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- .dockerignore | 5 +++++ .github/docker-context-checks.Dockerfile | 17 +++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/.dockerignore b/.dockerignore index 1b9aa11ca5..ee773610d5 100644 --- a/.dockerignore +++ b/.dockerignore @@ -37,3 +37,8 @@ packages/paperclip-runner/scripts/*-smoke.mjs # now guards this in PR CI. !packages/paperclip-runner/generated/** !packages/paperclip-runner/docs/capability-contract.md +# check:runner-workflow-traceability access()es every regression test the +# stress-traceability spec names — those are src/**/*.test.ts files, so +# they must survive the *.test.ts rule above. +!packages/paperclip-runner/src/**/*.test.ts +!packages/paperclip-runner/src/**/*.test.tsx diff --git a/.github/docker-context-checks.Dockerfile b/.github/docker-context-checks.Dockerfile index 48f5921d9c..b259714bb7 100644 --- a/.github/docker-context-checks.Dockerfile +++ b/.github/docker-context-checks.Dockerfile @@ -31,6 +31,23 @@ RUN test -f packages/paperclip-runner/generated/capability/semantic-tool-contrac && test -f packages/paperclip-runner/generated/semantic-action-catalog.json \ && test -f packages/paperclip-runner/spec/evals/stress-workflow-traceability.json \ && test -d packages/paperclip-runner/protocol/fixtures/replay +# check:runner-workflow-traceability access()es every regression test its +# spec names (it needs dist/ to RUN, so it cannot run here) — replicate +# exactly its existence walk, driven by the spec itself so this never +# needs a hand-maintained path list. (2026-09-04, second unmasking: the +# *.test.ts ignore rule stripped src/contracts/native-execution.test.ts +# and the image build failed there once the capability checks were fixed.) +RUN node -e ' \ + const manifest = require("/context/packages/paperclip-runner/spec/evals/stress-workflow-traceability.json"); \ + const { accessSync } = require("node:fs"); \ + const { resolve } = require("node:path"); \ + let count = 0; \ + for (const finding of manifest.findings) \ + for (const path of finding.regressionTests) { \ + accessSync(resolve("/context/packages/paperclip-runner", path)); \ + count += 1; \ + } \ + console.log(`traceability regression-test paths present: ${count}`);' # ajv is installed in an isolated directory (the runner's own package.json # uses workspace: ranges npm cannot install from) and symlinked in so ESM # resolution finds it from the scripts' location.