Commit Graph

4 Commits

Author SHA1 Message Date
Dotta 5bddff0920
feat(runner): add guarded API search and call fallback (#13003)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The new runner gives agents dedicated tools for common tasks.
> - Some API operations and parameters have no dedicated tool.
> - Agents need a controlled way to find and use those operations.
> - This pull request adds API search and calls through the real server
routes.
> - Existing tools remain the preferred path. The new tools are disabled
by default.
> - Paired tests measure correctness, tool choice, cost and time.

## Linked Issues or Issue Description

**Subsystem affected**

Paperclip Runner contracts, production tool authority and the server API
catalog.

**Problem or motivation**

The runner cannot use much of the API described by the old Paperclip
skill. A generic HTTP client would also let agents bypass runner control
rules.

**Proposed solution**

Add `search_api` and `call_api`. Resolve calls from the mounted API
catalog. Use server-held, run-bound credentials. Preserve route checks
and runner lifecycle rules. Keep the tools disabled until an operator
enables selected companies.

**Alternatives considered**

A dedicated tool for every endpoint would add a large initial prompt. An
unrestricted HTTP tool would weaken authorization and replay controls.

**Roadmap alignment**

This extends the native runner tooling. The repository owner requested
this design and implementation. The roadmap and related open PRs were
checked. No duplicate API escape-hatch PR was found.

## What Changed

- Register two compact fallback tools in canonical contracts and
provider projections.
- Build deterministic API discovery from OpenAPI, mounted experimental
routes and the old skill reference.
- Execute bounded JSON, text, file and download requests through
authenticated HTTP routes.
- Recheck active runs, company access and work modes. Block runner
lifecycle, scheduling, credential and approval bypasses. Keep routine
annotation collaboration available.
- Retain mutation receipts. Report uncertain outcomes without blindly
repeating writes.
- Add a company rollout gate and a durable eval worker with complete
cost accounting checks.
- Record child-task creation in the activity log with the agent and run.
- Add contract, authorization, file, replay and real runnerd/PRP/HTTP
tests.
- Document rollout gates and paid coverage limits. The companion eval
repository retains immutable attempts and reports.

## Verification

- Final app commit `da58370524c3626a744eec20164397c5fb6ba9ef`: all 32
checks passed; the unrelated Storybook visual check was skipped.
Greptile 5/5; no unresolved review threads.

- Full Linux build and recursive typecheck passed. Repository tests were
run by project and serialized shard; all 143 serialized server suites
passed.
- Runner TypeScript: 1,599 passed, two skipped. Rust release: 451
passing test reports. Conformance and replay parity passed. The required
API check passed 837 tests, including runnerd → PRP → authority → real
HTTP.
- Bindings cannot enable API tools without the explicit deployment flag.
Unit and real-authority tests prove the default-off boundary.
- The standalone API check builds and stages its own binary. It passed
after existing staged and debug binaries were removed from the test
container.
- UI and CLI tests passed. Initial environment failures (missing jq,
Docker overlay file identity, and parallel linker memory pressure) and
focused passing reruns are retained. The macOS full runner suite has
platform-specific failures; Linux is the qualified full-check platform.
- Eval harness: 27 tests passed; existing CI discovery ran 86 tests with
two unrelated skips. Credential export rejection is tested against the
actual report command.
- Luna and OpenRouter Sonnet each passed 60 common-workflow runs: ten
workflows, three repetitions per arm, zero unnecessary API fallback.
- Sonnet passed 11 selected capability/contract cases after fixes.
Gemini passed three smoke cases. DeepSeek exceeded the 120-second limit
and remains unqualified.
- Luna's two cost flags received focused follow-up. The original flags
and a later n=1 latency flag remain visible. Sonnet had no cost or
latency increase above 20%.
- The catalog contains 785 entries; 58 were exercised across all stages.
Most operation probes remain unrun and some need additional fixtures.
Authored probes do not establish successful coverage.
- Total conservative accounted cost: $9.875960. Active paid-campaign
time: 88.16/90 minutes. No missing accounting. Later security and
harness fixes have provider-free verification; no paid validation is
claimed for those revisions.
- Inspect the [qualification
report](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/READINESS.md)
and [verification
record](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/verification.json).

## Risks

- This is a broad authenticated API surface. Keep the default-off gate
until an operator selects initial rollout companies.
- Paid coverage is incomplete. Small regression samples do not prove all
workflows are unchanged.
- A timeout or server failure can follow a committed mutation. The
result reports an unknown outcome and requires state inspection.
- The new definitions add prompt tokens. The report retains cost flags
and cache variation.
- No database migration is required.
- Repository rules require code-owner approval before merge. Technical
CI and automated review are complete.

## Model Used

OpenAI Codex based on GPT-6 assisted with code, tests and review. The
exact serving model ID and context window are not exposed in this
session. It used reasoning, tool calls and code execution.

Eval models: `gpt-5.6-luna` with low reasoning,
`openrouter/anthropic/claude-sonnet-5`,
`openrouter/google/gemini-3.8-flash`, and
`openrouter/deepseek/deepseek-v4-flash-0731`. Attempts retain runtime
versions, model identity, usage and source provenance.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-07 14:14:43 -05:00
Dotta e11b457fd9
feat(runner): complete the canonical action catalog (#12360)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip Runner needs a stable list of semantic actions
> - Earlier pull requests define the core and collaboration action
groups
> - The remaining domain and administration actions need the same
contract form
> - The complete catalog must remain data only until a later pull
request binds actions to services
> - This pull request completes and validates the canonical action
inventory
> - The benefit is one frozen source of truth for later discovery and
authorization work

## Linked Issues or Issue Description

**Subsystem affected**

`packages/paperclip-runner` protocol contracts.

**Problem or motivation**

The runner does not yet have a complete canonical inventory for domain
and administration actions. Later authorization code cannot project a
stable operation catalog without this inventory.

**Proposed solution**

Add the remaining action contracts and aggregate all 41 actions.
Validate every declared live and scenario projection with its own
schema.

**Alternatives considered**

Binding these actions directly to production services would combine data
contracts with authority. This pull request keeps every production
service binding unbound.

**Roadmap alignment**

This change supports the experimental Paperclip Runner. It does not
enable the runner or change an existing adapter.

## What Changed

- Added 14 domain and administration action contracts.
- Added the complete immutable 41-action catalog.
- Added validation for live inputs and outputs.
- Added validation for scenario inputs and capability-result envelopes.
- Kept every production service binding unbound.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner test:typescript`
- `pnpm -r typecheck`
- `pnpm build`
- The focused catalog test validates 41 actions.

## Risks

Low risk. This pull request adds package-local contract data and tests.
It does not authorize an action or change an adapter.

## Model Used

OpenAI Codex with GPT-5.6 and repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-29 23:23:05 -05:00
Dotta 4eb32d7532
Add collaboration action contracts (#12359)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents collaborate through discovery, delegation, dependencies, and
governed approvals
> - These optional operations need stable contracts before any run may
discover them
> - The core action contracts are already isolated in the lower stack
> - This pull request adds the 13 collaboration and governance action
definitions
> - It does not grant claims or bind production services
> - The benefit is a reviewable contract layer for later run-scoped
authorization

## Linked Issues or Issue Description

**Subsystem affected**

packages/paperclip-runner protocol action contracts

**Problem or motivation**

Optional collaboration operations need one provider-neutral definition
for policy metadata, documentation, live and scenario schemas, and
canonical examples.

**Proposed solution**

Add one immutable module for each discovery, delegation, dependency, and
governance action. Add a collaboration aggregate and validate all
examples with JSON Schema.

**Alternatives considered**

These contracts could land with executable authorization. That would
make contract review depend on runtime policy and server service
bindings.

**Roadmap alignment**

This supports the existing experimental Paperclip Runner rollout. It
does not make any optional operation discoverable or invocable.

## What Changed

- Add 13 collaboration and governance action definitions.
- Add immutable policy, schema, documentation, and example data.
- Add a collaboration-only aggregate.
- Validate every action example against its declared schema.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner test:typescript`
- `pnpm --filter @paperclipai/paperclip-runner typecheck:typescript`
- `pnpm -r typecheck`
- `pnpm build`
- The branch changes 15 files relative to its declared base.

## Risks

Low production risk. The declarations do not grant claims or contain
executable service bindings. Tests require optional placement, immutable
metadata, stable identities, and schema-valid examples.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, `gpt-5`, with agentic reasoning, tool use, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked an existing public issue or described the
issue in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-29 23:13:49 -05:00
Dotta 08b5d6bd9b
Add core runner action contracts (#12358)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Runner tools need stable provider-neutral action contracts
> - Core task work must use the same names, policy metadata, schemas,
and examples on every provider surface
> - A single large 41-action review would hide contract drift
> - This pull request adds only the 14 always-present core action
definitions
> - It does not grant discovery or execution authority
> - The benefit is a reviewable immutable contract base for later
authorization wiring

## Linked Issues or Issue Description

**Subsystem affected**

packages/paperclip-runner protocol action contracts

**Problem or motivation**

The runner has an older compact catalog, but it does not single-source
the full policy metadata, documentation, live shape, scenario shape, and
examples for each core action.

**Proposed solution**

Add one immutable definition module per core action. Add an internal
core aggregate and validate every input and output example against its
declared JSON Schema. Keep all production authorization and execution
unavailable.

**Alternatives considered**

All 41 action definitions could land in one pull request. That would
create more than 7,000 changed lines and reduce useful review depth.

**Roadmap alignment**

This supports the existing experimental Paperclip Runner rollout. It
does not enable the adapter or expose any new production tool.

## What Changed

- Add 14 core protocol action definitions.
- Add immutable policy, schema, documentation, and example data for each
action.
- Add a deep-freeze helper and a core-only aggregate.
- Validate all core examples with JSON Schema.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner test:typescript`
- `pnpm --filter @paperclipai/paperclip-runner typecheck:typescript`
- `pnpm -r typecheck`
- `pnpm build`
- The branch changes 17 files relative to its declared base.

## Risks

Low production risk. These declarations do not contain executable
authorization or service bindings. The main risk is schema drift. Tests
validate operation identity, placement, immutability, and canonical
examples.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, `gpt-5`, with agentic reasoning, tool use, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked an existing public issue or described the
issue in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-29 23:05:05 -05:00