Commit Graph

1 Commits

Author SHA1 Message Date
Dotta bfb98aff5d
test(runner): add credential-free acceptance foundation (#12652)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip supports direct adapters and an experimental native Runner
path.
> - These paths need one stable compatibility matrix.
> - The matrix must not launch providers or load credentials during
normal tests.
> - Result handling must reject incomplete output and sensitive values.
> - This pull request adds a credential-free acceptance foundation.
> - The benefit is a reviewable contract for later end-to-end executors.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves verification for direct adapters and Paperclip Runner
providers.

**Subsystem affected**

Cross-cutting test infrastructure for adapters, the server runtime, and
the task thread.

**Current behavior**

The repository has subsystem tests. It does not have one declarative
matrix for direct and native compatibility.

**Proposed behavior**

Add a pure acceptance catalog, result validator, redaction helpers, and
failure classification. Keep all execution authority outside this
change.

**Reason and benefit**

The matrix makes legacy isolation and native recovery requirements
explicit. The helpers let later executors report safe and complete
results.

**Breaking changes**

None. This change does not alter production runtime selection or start
any provider.

## What Changed

- Add a catalog for built-in direct adapters and qualified native
provider profiles.
- Add compatibility cases for runtime selection, task threads,
questions, and flag-change recovery.
- Add pure redaction and transient-failure classification helpers.
- Add fail-closed Markdown and JUnit report aggregation.
- Add isolated test and type-check commands.
- Document the credential-free boundary and deferred live execution
work.

## Verification

GitHub Actions must run:

- `pnpm test:runner-acceptance`
- `pnpm test:runner-acceptance:typecheck`
- The repository test, type-check, build, policy, and security gates.

No local test command was run. The repository owner requested
GitHub-only verification.

## Risks

Low production risk. The change adds test-only files and root scripts.
The catalog can drift when a built-in adapter changes. Its validation
fails closed on that drift.

## Model Used

OpenAI Codex with the GPT-5 agent model. The work used high reasoning,
repository inspection, tool use, and parallel code review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 04:46:04 -05:00