bfb98aff5d
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip supports direct adapters and an experimental native Runner path. > - These paths need one stable compatibility matrix. > - The matrix must not launch providers or load credentials during normal tests. > - Result handling must reject incomplete output and sensitive values. > - This pull request adds a credential-free acceptance foundation. > - The benefit is a reviewable contract for later end-to-end executors. ## Linked Issues or Issue Description **What existing behavior does this improve?** This improves verification for direct adapters and Paperclip Runner providers. **Subsystem affected** Cross-cutting test infrastructure for adapters, the server runtime, and the task thread. **Current behavior** The repository has subsystem tests. It does not have one declarative matrix for direct and native compatibility. **Proposed behavior** Add a pure acceptance catalog, result validator, redaction helpers, and failure classification. Keep all execution authority outside this change. **Reason and benefit** The matrix makes legacy isolation and native recovery requirements explicit. The helpers let later executors report safe and complete results. **Breaking changes** None. This change does not alter production runtime selection or start any provider. ## What Changed - Add a catalog for built-in direct adapters and qualified native provider profiles. - Add compatibility cases for runtime selection, task threads, questions, and flag-change recovery. - Add pure redaction and transient-failure classification helpers. - Add fail-closed Markdown and JUnit report aggregation. - Add isolated test and type-check commands. - Document the credential-free boundary and deferred live execution work. ## Verification GitHub Actions must run: - `pnpm test:runner-acceptance` - `pnpm test:runner-acceptance:typecheck` - The repository test, type-check, build, policy, and security gates. No local test command was run. The repository owner requested GitHub-only verification. ## Risks Low production risk. The change adds test-only files and root scripts. The catalog can drift when a built-in adapter changes. Its validation fails closed on that drift. ## Model Used OpenAI Codex with the GPT-5 agent model. The work used high reasoning, repository inspection, tool use, and parallel code review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|---|---|---|
| .. | ||
| README.md | ||
| catalog.test.ts | ||
| catalog.ts | ||
| failure-classifier.ts | ||
| redaction.ts | ||
| report.test.ts | ||
| report.ts | ||
| support.test.ts | ||
| tsconfig.json | ||
| types.ts | ||
| vitest.config.ts | ||
README.md
Credential-free Runner acceptance foundation
This directory defines the reviewable, deterministic foundation for Runner acceptance checks. It contains no launcher and grants no authority to start Paperclip, a provider process, a browser, a remote environment, or a billable model request.
The catalog covers:
- every current built-in direct adapter except the explicitly deferred Pi adapter;
- the external-plugin direct-adapter compatibility contract;
- Paperclip Runner with Codex;
- Paperclip Runner with the qualified OpenCode model; and
- Paperclip Runner with the qualified ACPX Claude and Codex profiles.
The direct-adapter cells assert the legacy boundary: runnerd does not start, native records are not created, direct finalization remains authoritative, and classic task controls remain available. Native cells assert persisted provider identity, runtime authority, structured-question behavior, and recovery of an already-recorded run after the rollout flag changes.
Registered compatibility-only adapters, including the retired acpx_local
entry, remain in the direct catalog so a future selection check can prove they
never fall into native execution. Their presence is not a claim that the
adapter can start a provider session.
Commands
Run the isolated unit suite:
pnpm test:runner-acceptance
Check the standalone TypeScript boundary:
pnpm test:runner-acceptance:typecheck
These commands are credential-free. They validate the catalog and pure support utilities; they do not claim that a provider was contacted.
Result boundary
Future fixture executors may emit paperclip.runner-acceptance.result/v1
objects and pass them to buildRunnerAcceptanceReport. Results must name a
catalog cell, report every expected assertion, carry successful redaction, and
contain no sensitive-looking structured values. The aggregator produces only
in-memory normalized data plus optional Markdown or JUnit strings. It does not
write evidence, screenshots, history, or public reports.
Deliberate exclusions
This foundation does not include:
- Pi or any Pi transitive package;
- Claude Managed or AWS AgentCore;
- Daytona, remote images, live provider execution, or cost accounting;
- provider-key loading, secret references, auth-file discovery, or tracked env templates;
- screenshots, traces, raw logs, evidence archives, dashboards, history, or publication workflows; or
- the eval kernel, scenario explorer, browser SDK, or package/runtime replacement work.
Add an executor only in a later, explicitly authorized change. Keep its launch authority separate from this catalog, and make any billable or secret-bearing mode opt-in and independently reviewed.