Commit Graph

1280 Commits

Author SHA1 Message Date
Devin Foley d1b9448b57
fix(server): stamp the real build version into images instead of the package.json placeholder (#10257)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work; it ships as a Docker image that self-hosters and managed
deployments run.
> - The server resolves its own version at runtime in
`server/src/version.ts` (`resolveServerVersion()`), which feeds
analytics and the server debug panel.
> - That resolver derives the real version from `git describe`, and
falls back to `server/package.json`'s `version` when git isn't
available.
> - But `server/package.json`'s version is a static placeholder — CI
only stamps the real CalVer at publish, so in source it is never the
real version (currently `0.3.1`).
> - A Docker image has no `.git` (it's dockerignored), so `git describe`
can't run inside it. Every image therefore falls back to the placeholder
and reports `0.3.1` in analytics and the debug panel, regardless of
which commit it was built from.
> - This PR computes the real version once on the CI build runner (where
`.git` and tags exist), bakes it into the image, and has
`resolveServerVersion()` prefer that stamp when `git describe` is
unavailable.
> - The benefit: self-hosted and cloud images report their true version
instead of a misleading placeholder, with no change to dev checkouts,
`git describe`-based resolution, or local `docker build`.

## Linked Issues or Issue Description

No public issue exists — describing the bug inline (per the bug report
template).

**What happened?**
Docker images built from `master` (and release tags) report the server
version as the `0.3.1` placeholder in analytics and the server debug
panel, instead of the real version of the commit the image was built
from.

**Expected behavior**
An image reports the real version of its build commit (e.g.
`2026.722.0+51.git.<sha>`), so operators can tell which build is
running.

**Steps to reproduce**
1. Build the server Docker image from any `master` commit (the `Docker`
workflow, `production` target).
2. Run the image and open the server debug panel (or inspect the version
reported to analytics).
3. Observe the version is `0.3.1` rather than the commit's real version.

**Root cause**
`resolveServerVersion()` derives the real version from `git describe`,
but the image has no `.git` (dockerignored), so it falls back to
`server/package.json`'s `version` — a static placeholder CI only
replaces with the real CalVer at publish time. Nothing bakes the real
version into the image.

**Paperclip version or commit:** reproduces on `master` (`4c55f0d8`) and
any published image.
**Deployment mode:** self-hosted and managed (both the `production` and
`-cloud` images).
**Installation method:** Docker image (`ghcr.io/paperclipai/paperclip`).

**Related PRs (dedup search):** #9103 (merged — added the `git
describe`-based source-install resolution this builds on) and #9637
(closed). Neither bakes a version into the image; this PR closes that
gap. No duplicate found.

## What Changed

- **`.github/workflows/docker.yml`** — checkout with full history + tags
(`fetch-depth: 0`), and a new `Compute build version` step that runs
`git describe --tags --match 'v*' --long --dirty` on the pristine runner
checkout. The result is passed as a `PAPERCLIP_BUILD_VERSION` build-arg
to both the `production` and `-cloud` image builds.
- **`Dockerfile`** — the `production` stage takes an `ARG
PAPERCLIP_BUILD_VERSION` (default empty) and bakes it into the runtime
`ENV`; the `cloud` stage inherits it via `FROM production`.
- **`server/src/build-version.ts`** (new) — `readBuildVersion()` /
`parseBuildVersion()`, mirroring `build-commit.ts`: reads
`PAPERCLIP_BUILD_VERSION` (or a `.paperclip-build-version` file) as a
single-token stamp.
- **`server/src/version.ts`** — `resolveServerVersion()` prefers the
baked build version when `git describe` is unavailable, parsing it with
the same rules as a live checkout (`parseGitDescribeVersion`), and
falling through to the existing `build-commit` stamp and package version
when unset. A live checkout's `git describe` still wins over any stamp.
- Tests for the new behavior and the precedence.

## Verification

- `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && tsc
--noEmit` in `server/` — clean.
- `vitest run server/src/__tests__/version.test.ts
server/src/__tests__/build-version.test.ts` — **23 tests pass**,
covering: stamped version used when git describe fails, stamp parsed to
real CalVer, stamp preferred over the build-commit fallback, on-tag
stamp collapses to the release version, a pre-resolved stamp used
verbatim, and a live git describe still winning over a stamp.
- `git describe --tags --match 'v*' --long` for this commit →
`v2026.722.0-51-g<sha>`, which `resolveServerVersion()` reports as
`2026.722.0+51.git.<sha>` — no longer `0.3.1`.
- Not run locally: the full multi-arch image build (CI-only). The
workflow change is verified by inspection; the version is computed on
the pristine checkout before any lockfile refresh, so it carries no
spurious `-dirty`.

## Risks

Low. Additive and image-only:
- No runtime behavior changes for dev checkouts (git describe still
primary and wins over any stamp) or for local `docker build` (empty arg
→ server keeps its existing fallbacks).
- Not a breaking change; no schema or API surface. The stamp is
informational (version reporting only).
- `fetch-depth: 0` makes the release-image checkout fetch full
history/tags — a modest cost on a workflow that already runs at release
cadence with a 60-minute budget.
- Rollback: revert the commit; images simply return to reporting the
placeholder.

## Model Used

Claude Opus 4.8 (`claude-opus-4-8`, 1M-context variant), extended
thinking, with tool use / code execution — agentic edits, `tsc` +
`vitest` runs, and a `git describe` resolution check.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work (bugfix, not core feature work)
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (#9103, #9637 — related, not duplicates)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (`fix/build-version-stamp`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
user-facing docs affected; behavior is documented inline in `version.ts`
/ `build-version.ts` and the workflow/Dockerfile)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 10:06:28 -07:00
Dotta c481be44e3
fix(task-watchdogs): deduplicate unchanged stopped-state wakes (#10207)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Task watchdogs review issue subtrees when no run or queued wake
keeps work live
> - Pending human interactions and approvals are valid stopped states
that still need one watchdog review
> - The existing fingerprint included volatile activity timestamps, so
unchanged stopped trees could wake repeatedly after comments, documents,
work products, or sibling completions
> - This pull request fingerprints only review-material leaf and wait
state, persists the reviewed snapshot, and suppresses shrink-only
repeats
> - The benefit is one review per materially new stopped state without
weakening liveness classification or hiding human waits

## Linked Issues or Issue Description

### What happened?

Task-watchdog stop fingerprints changed for metadata-only activity and
completed siblings, producing duplicate wakes after an unchanged stop
had already been reviewed.

### Expected behavior

Pending interactions and approvals remain classified as stopped, but a
reviewed stopped state only wakes again when waits, non-terminal leaves,
status, assignment, or blockers gain material changes.

### Steps to reproduce

1. Review a stopped watched subtree with a pending human wait or
multiple non-terminal stopped leaves.
2. Add only comment/document/work-product activity, or complete one
stopped sibling without changing the wait set.
3. Observe a duplicate wake from the timestamp-heavy fingerprint.

Related public work: Refs #9452 for overlapping task-watchdog service
edits and #10043 for related no-op fingerprint suppression.

## What Changed

- Added fingerprint v2 over non-terminal material leaves plus
subtree-wide pending wait ids, excluding volatile timestamps while
retaining them in wake context.
- Added nullable observed/reviewed JSONB stop snapshots and shrink-only
reviewed-state suppression with legacy exact-fingerprint fallback.
- Added pending interaction kinds and approval ids to watchdog wake
context, review comments, and comment metadata.
- Added classifier and scheduler coverage for waiting-leaf liveness,
metadata stability, sibling shrink suppression, material changes,
snapshot promotion, legacy rows, and unchanged idempotency keys.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/task-watchdogs-classifier.test.ts
src/__tests__/task-watchdogs-scheduler.test.ts` — 2 files, 36 tests
passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `git diff --check origin/master...HEAD` — passed.

## Risks

- Fingerprint version 2 intentionally re-fingerprints every currently
stopped watched tree once after deployment, causing a one-time wake
burst before the new reviewed snapshots are established.
- Migration `0191_task_watchdog_stop_snapshots.sql` only adds two
nullable JSONB columns with no backfill; legacy rows continue
exact-fingerprint behavior until a post-deploy review promotes a
snapshot.
- PR #9452 edits the same service file; whichever lands second may need
a trivial rebase.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with
repository tool use and code execution. The runtime did not expose a
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 08:34:41 -05:00
Dotta 4e00818574
fix(runtime): support in-place workspace realization (#10230)
## Thinking Path

> - Paperclip is the control plane people use to coordinate AI agents
and their execution environments.
> - Environment realization decides where an agent runs and which
filesystem and toolchain are authoritative.
> - Copy-based realization is unsafe for container-anchored tasks
because absolute paths such as `/app` can point outside the synchronized
tree and task-specific binaries may be absent.
> - That mismatch can let an agent successfully verify work in a phantom
writable path while sync-back silently discards the result.
> - Existing task environments already provide the authoritative
filesystem and toolchain, so they should be executed in place rather
than copied.
> - Copy mode still needs explicit confinement rules so aliases target
the synchronized workspace and unsynchronized writable paths fail
visibly.
> - This pull request adds typed realization metadata, propagates the
authoritative root through orchestration, and teaches Codex to honor it.
> - The benefit is that container-anchored tasks operate on
verifier-visible state with the intended tools, while copy mode remains
safe and backward compatible.

## Linked Issues or Issue Description

No public GitHub issue exists for this defect.

GitHub duplicate searches for in-place execution, workspace realization,
and authoritative workspace roots found no related pull request to link.

### What happened?

Environment-backed agent runs were always realized through a copied
workspace. Tasks anchored to absolute container paths could therefore
write outside the synchronized tree, and task-provided toolchains were
unavailable in the copy. A run could report success even though
sync-back discarded its output.

### Expected behavior

Existing task environments should run against their real authoritative
root and toolchain. Copy-mode runs should map declared absolute aliases
into the synchronized tree and reject writable paths that cannot be
restored.

### Steps to reproduce

1. Run a Codex task environment whose required files live under `/app`
or `/workspace` and whose required binary exists only in the task
container.
2. Observe that copy realization changes the effective
filesystem/toolchain or permits writes outside the synchronized root.
3. Complete and verify the task inside the agent sandbox.
4. Observe that the verifier cannot see out-of-tree artifacts or that
task-specific commands were unavailable.

### Reproduction context

- Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924`
- Deployment: built from source in a task-container execution
environment
- Adapter: Codex local
- Database: not database-related
- Access context: agent execution

## What Changed

- Added typed `copy | in_place` workspace-realization metadata,
authoritative roots, confined aliases, and outbound restore paths to
shared execution-target contracts.
- Selected in-place realization for existing task environments and
skipped archive prepare/restore when the authoritative environment is
used directly.
- Propagated the authoritative root into adapter context so Codex uses
it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP
execution.
- Bound copy-mode aliases such as `/app` to the synchronized workspace
and rejected writable out-of-tree paths without explicit restore
mappings.
- Added focused regression coverage while preserving existing copy-mode
archive restore behavior.

## Verification

- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm exec vitest run
packages/adapter-utils/src/local-process-sandbox.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
packages/adapters/codex-local/src/server/execute.remote.test.ts
server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed,
4 skipped.
- `pnpm -r typecheck` — passed.
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u
AWS_SESSION_TOKEN pnpm test:run` — passed across all general and
serialized Vitest shards.
- `pnpm build` — passed.
- Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC
with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`,
`openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate`
each received terminal grade 1.0 against real task-environment paths and
toolchains.

## Risks

- In-place mode deliberately exposes the authoritative task root to the
adapter; incorrect environment metadata could point execution at the
wrong root. Typed metadata and focused orchestration tests cover
selection and propagation.
- Copy-mode writable-path validation is stricter and may reject
previously accepted unsafe configurations. The rejection is intentional
and produces a visible error instead of silently losing output.
- The acceptance run is focused on four Codex task-environment
workloads, not a broad cross-adapter benchmark. Existing copy-mode
archive tests and the full repository suite remain green.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI coding agent; exact model ID and context-window size
were not exposed to this runtime. Capabilities used: extended reasoning,
repository editing, shell execution, test/build execution, Git, GitHub
CLI, and Paperclip API tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 08:29:26 -05:00
Nicky Leach cca2806e57
test(tool-gateway): make idle-down slot test deterministic via injected clock (#10226)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server has a gateway layer that coordinates tool execution and
runtime slots
> - The idle-down test for the local stdio fixture slot was relying on
real wall-clock timing
> - On slower runners, that made the test nondeterministic because the
slot could be reaped before the presence assertion ran
> - This pull request switches the test to use the existing injectable
clock seam so time only advances when the test says it should
> - The benefit is that the idle-down behavior stays covered while the
test becomes deterministic and no longer flakes under load

## Linked Issues or Issue Description

This PR fixes a flaky gateway test in the server test suite. The
`tool-gateway` idle-down scenario was asserting slot presence while also
depending on a very short real-time idle TTL and a later sleep-based
reap. On loaded runners, the intervening work could exceed the TTL,
which caused the slot to disappear early and the assertion to see an
empty list.

The fix keeps the production code path unchanged and drives the test
from the supervisor's existing injectable clock. The test now holds time
steady through the presence check, then advances the clock past the idle
deadline to trigger the reap deterministically. The original behavioral
assertions stay intact: slot reuse, counter increments, metadata, and
stop status still get verified.

## What Changed

- Replaced the real-time idle-down wait in the `tool-gateway` test with
the runtime supervisor's injectable clock seam.
- Kept the existing assertions for slot reuse, slot identity, counters,
metadata, and stop behavior.
- Removed the test's dependency on wall-clock timing so the idle-down
path is deterministic under load.

## Verification

- Targeted server typecheck passed with `tsc --noEmit`.
- `tool-gateway.test.ts` passed in full: 49/49.
- The targeted idle-down scenario passed 50/50 in a tight loop with 0
failures after the clock injection change.

## Risks

- Low risk: this is a test-only change and does not modify production
gateway logic.
- The test now exercises the idle-down logic through a controlled clock
rather than real elapsed time, which is the point of the fix but does
slightly reduce wall-clock realism in the test itself.

## Model Used

OpenAI Codex (GPT-5), tool-using coding agent; context window not
surfaced in the workspace.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 16:29:47 -07:00
Dotta 30ff3d7c58
feat(routines): expose activity gate API (#9438)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Scheduled routines provide recurring control-plane work without
manual intervention
> - The new activity gate can suppress scheduled runs when no external
work occurred
> - The core scheduler and database support landed without a public
create/update contract
> - Agents, operators, and managed plugins need validated fields plus
discoverable semantics to opt in safely
> - This pull request exposes the activity gate through routine APIs,
revisions, plugin contracts, tests, and skill documentation
> - The benefit is backward-compatible control over idle scheduled work
without losing activity-triggered follow-up

## Linked Issues or Issue Description

- Refs #8534

## What Changed

- Added shared activity-gate policy and scope enums with create/PATCH
validation.
- Persisted activity-gate fields through routine creation, updates,
revision snapshots, pipeline snapshots, and revision restores.
- Defaulted legacy revision snapshots during restore and added
regression coverage for pre-field snapshots.
- Extended managed-plugin routine declarations, production
reconciliation, and the SDK test harness to preserve non-default gate
settings.
- Added end-to-end API coverage for create/PATCH/list/detail
round-trips, defaults, and invalid enum rejection.
- Documented schedule-only semantics, activity windows,
own-run/read-action exclusions, scopes, and an hourly quiet-night
watcher example.

## Verification

- `pnpm exec vitest run packages/shared/src/validators/routine.test.ts
server/src/__tests__/routines-service.test.ts
server/src/__tests__/routines-e2e.test.ts`
- `pnpm exec vitest run packages/shared/src/validators/plugin.test.ts
packages/plugins/sdk/tests/testing-actions.test.ts
server/src/__tests__/plugin-managed-routines.test.ts
server/src/__tests__/routines-service.test.ts -t 'activity
gate|preserves declared activity gate settings|resolves routine agent
and project refs'`
- `pnpm exec vitest run ui/src/lib/workspace-routines.test.ts
ui/src/pages/Routines.test.tsx`
- `pnpm --filter @paperclipai/shared typecheck`
- `pnpm --filter @paperclipai/plugin-sdk typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/ui typecheck`
- `pnpm check:token-gates`
- GitHub CI: all final-head checks green; Storybook visual regression
skipped by path rules.
- Greptile: 5/5 with no unresolved review threads.

## Risks

- Low risk: defaults remain `always` and `company`, preserving existing
routine behavior and old revision snapshots.
- Managed plugin manifests can now declare the same validated gate
settings as the public routine API; omitted values retain core defaults.
- Revision snapshots now include the new fields so policy changes are
not lost or treated as no-ops during restore.

> For core feature work, checked `ROADMAP.md`: this extends the existing
Scheduled Routines roadmap item and does not duplicate a separate
planned capability.

## Model Used

- OpenAI GPT-5.5 via Codex CLI, with repository tool use and code
execution; context-window size was not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 16:47:24 -05:00
Dotta 8f08ec5ce6
feat(status-cards): join summary-mentioned issues to watched set (#10205)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Status cards summarize changing company work and watch issues so
later changes can produce useful deltas
> - A summary can explicitly reference issues that are important to the
update even when those issues do not match the card's configured queries
> - Previously, those referenced issues were not retained in the watched
set, so their later status, assignee, or comment changes could be missed
> - The watched snapshot must avoid artificial additions or removals
caused only by a summary changing which issues it references
> - This pull request resolves issue references when a summary is
written, persists them, and joins them to the watched snapshot with
stable delta semantics
> - The benefit is that status cards continue tracking the exact issues
their latest update called out while keeping follow-up updates relevant
and non-duplicative

## Linked Issues or Issue Description

### Pre-submission checklist

- [x] I have searched existing open and closed issues and this is not a
duplicate.
- [x] I am on the latest released version of Paperclip (or can reproduce
on `master`).
- [x] I have confirmed the error originates in Paperclip itself — not in
my agent adapter, API provider, or local configuration.

### What happened?

When a status-card summary explicitly referenced an issue by identifier
or `/issues/<uuid>` URL, that issue was not automatically retained in
the card's watched set unless it independently matched a configured
query. Later status, assignee, or comment changes to an issue
highlighted by the latest update could therefore be omitted.

### Expected behavior

References in the latest summary should resolve only within the card's
company, appear in dry runs and the watched-issues UI, count and
fingerprint like query matches, and enter or leave the watched set
without artificial added/removed deltas already represented by the
summary change.

### Steps to reproduce

1. Create a status card whose query does not match a second issue in the
same company.
2. Write a summary that references the second issue by identifier or
issue URL.
3. Inspect the card's watched count or Watched issues tab.
4. Change the referenced issue's status, assignee, or comments and run
the next update.
5. Before this change, the referenced issue is absent from the watched
snapshot and its later change does not produce the expected delta.

### Paperclip version or commit

- Reproduced on `master` before this PR (base commit `762ce5b4ef`).

### Deployment mode

- Local dev (`pnpm dev`), built from source.

### Agent adapter(s) involved

- Not adapter-specific (core bug).

### Database mode

- Embedded Postgres test environment; the schema change uses standard
PostgreSQL JSONB.

### Access context

- Board (human operator).

## What Changed

- Added migration `0191` and schema support for persisted
`status_cards.mentioned_issue_ids`.
- Resolved summary references by issue identifier or `/issues/<uuid>`
URL within the status card's company when summaries are written.
- Joined mentioned issues into watched counts and fingerprints so later
status, assignee, and comment changes generate normal update deltas.
- Suppressed artificial added/removed deltas when the latest summary
starts or stops mentioning an issue.
- Added `mentionedIssues` to dry-run responses and a “Mentioned in the
latest update” group in the Watched issues tab.
- Updated the summarizer prompt to explain that referenced issues
automatically join the watched set.
- Added focused server and UI coverage for reference resolution,
snapshot behavior, deltas, API responses, and rendering.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/status-cards.test.ts
src/__tests__/status-card-update-engine.test.ts` — 31 tests passed.
- `pnpm --filter @paperclipai/ui exec vitest run
src/pages/StatusCards/StatusCardTile.test.tsx` — 11 tests passed.
- Earlier implementation verification also passed database/shared/server
typechecks, UI `tsc -b`, the broader StatusCards UI test set, and
embedded-Postgres migration application.

### Visual Verification

- Greptile T-Rex ran Playwright browser checks successfully and captured
the Status Card drawer Watched tab showing the new “Mentioned in the
latest update” grouping:
https://app.greptile.com/trex/runs/15796101/artifacts

## Risks

- The migration adds a nullable JSONB column and is backward-compatible;
existing cards have no mentioned issues until their next summary write.
- Reference extraction is company-scoped to prevent cross-company issue
association.
- Watched counts and future fingerprints change for cards whose latest
summaries reference issues; tests cover additions, removals, and
suppression of spurious deltas.
- This targeted status-card fix does not introduce a new roadmap
subsystem or external integration.

## Model Used

- Anthropic Claude Fable 5 (Paperclip model alias; exact underlying
provider model ID and context window were not recorded in the
implementation task metadata), with extended reasoning, tool use, and
code execution.
- OpenAI Codex coding agent (runtime model identifier and context window
not exposed to this task) prepared the PR, rebased the branch, and ran
focused verification with terminal tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 16:44:56 -05:00
Dotta 665408c6d0
fix(codex): classify mid-turn harness crashes structurally as retriable infra (#10210)
## Thinking Path

> - Paperclip is the open source control plane people use to manage AI
agents and their work
> - Heartbeat execution relies on adapters distinguishing agent failures
from failures in the harness running beneath the agent
> - Codex MCP transport crashes can kill the CLI after the JSONL
protocol has started but before it emits a protocol-terminal event
> - Those interrupted streams were left unclassified, so the control
plane terminalized the heartbeat as `heartbeat_failed` / `agent_failure`
with no continuation
> - Agent-level failure is already expressible through the JSONL
protocol via an `error` event, `turn.failed`, or `turn.completed`, so an
interrupted nonzero exit can be classified structurally without
inspecting unstable error strings
> - This pull request reports that shape as `codex_harness_crash` in the
`transient_upstream` family and routes it through Paperclip's existing
bounded retry and recovery-continuation paths
> - The benefit is that transient Codex harness failures recover safely
without misclassifying quoted agent output or depending on
transport-specific wording

## Linked Issues or Issue Description

- **What happened:** Codex MCP transport failures, including rmcp worker
death, could terminate the CLI mid-turn after protocol output began but
before any terminal JSONL event. The run then became an unclassified
terminal heartbeat failure with `continuationCount: 0`; this occurred in
3 of 44 L3 Codex-lane trials during the associated benchmark
investigation.
- **Expected behavior:** a nonzero Codex exit after the protocol starts
but before an `error`, `turn.failed`, or `turn.completed` event should
be treated as a harness/infrastructure crash and enter the existing
bounded retry policy.
- **Why structural classification:** transport error strings vary, and
stdout may quote agent output that merely discusses network failures.
The protocol boundary identifies whether the agent itself produced a
terminal result without regex matching.
- **Recovery behavior:** `codex_harness_crash` maps to `errorFamily:
transient_upstream`, using the existing `same_session` →
`safer_invocation` → `fresh_session` ladder plus the
recovery-continuation transient-infrastructure path.
- Supersedes the regex-based approach in #10150, which is closed.

## What Changed

- Added protocol-state tracking that identifies a nonzero exit after
protocol start and before any protocol-terminal event as
`codex_harness_crash`.
- Propagated the structural classification as `transient_upstream`
through the Codex adapter.
- Added parse unit coverage, including a faithful crash-shaped stream,
without matching stderr transport strings.
- Added adapter execution coverage using a fake Codex process that emits
a protocol prefix and then dies with the observed rmcp stderr line.
- Added heartbeat bounded-retry coverage, including the `errorCode`-only
fallback, and recovery-continuation classification coverage.

## Verification

- `parse.test.ts` — 16 passed.
- `codex-local-execute.test.ts` — 16 passed.
- `heartbeat-retry-scheduling.test.ts` — 30 passed.
- `service.pause-durability.test.ts` — 6 passed.
- Server and Codex adapter TypeScript checks passed.
- The branch commit is unchanged from the tested and pushed `88f5464d40`
handoff.

## Risks

- Low risk: the classification requires a nonzero exit after protocol
start and before any protocol-terminal event, so normal agent-declared
failures and completed turns keep their existing behavior.
- The change intentionally broadens recovery for structurally
interrupted Codex runs; bounded retry limits still prevent indefinite
continuation loops.
- No schema, migration, public API, UI, lockfile, or workflow changes.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agent. The exact runtime model ID and
context-window size were not exposed by the execution environment;
capabilities used for the implementation included repository analysis,
reasoning, code editing, and terminal-based test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
— the pre-existing, already-pushed branch name was explicitly prescribed
for this replacement PR
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 16:40:47 -05:00
Dotta b996b71a38
Deduplicate wake-payload issue descriptions and compact resume deltas (#10216)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Heartbeat wake payloads and the task-context markdown are the two
channels that deliver an issue's brief into an agent's prompt
> - #10151 fixed wake-prompt-only adapter lanes waking without the issue
description by adding it to the structured wake payload
> - That left the description delivered twice per prompt on lanes that
also inject the task-context markdown, and re-delivered in full on every
resume wake, permanently bloating persistent-session context
> - This pull request makes the task markdown the single description
carrier on lanes that use it, and omits the description from
non-assignment resume deltas on all lanes while keeping it for
assignment-shaped and recovery wakes
> - The benefit is that every lane receives the brief exactly once when
it needs it, and long-lived sessions stop re-paying the full brief in
tokens on every wake

## Linked Issues or Issue Description

Refs #10151

Related prior work: #2883, #8402 (earlier description-delivery attempts
referenced by #10151). I searched the PR list for open work on
wake-payload description handling and found none besides the merged
#10151.

**Bug:** After #10151, adapters that inject the `Paperclip task context`
markdown (ACPX engine lanes, claude-local CLI, hermes server and
gateway) receive the issue description twice in a single prompt — once
in the wake prompt's `Issue description:` block and once in the task
markdown. Separately, resume deltas re-send the full description (up to
12k characters) on every wake even though the persistent session already
received it.

**Expected behavior:** The description appears exactly once per prompt
on every lane, and resume deltas only carry it when the resuming session
may not have seen the brief (assignment-shaped or recovery wakes),
leaving an explicit fetch breadcrumb otherwise.

**Reproduction:** Wake a claude-local or ACPX agent on an issue with a
description and inspect the assembled prompt: the description text
appears in both the wake-payload block and the task-context block. Wake
the same session again via a comment: the full description is present
again in the resume delta.

**Affected version:** Current `master` (with #10151 merged).

**Deployment mode:** Adapter-backed heartbeat execution, local and
sandboxed lanes.

## What Changed

- `renderPaperclipWakePrompt` accepts `suppressIssueDescription`; the
four task-markdown lanes pass it so the task markdown stays the single,
uncapped description carrier there.
- Non-assignment resume deltas omit the description and emit `- issue
description: omitted from this resume delta; fetch the issue if you need
the latest brief`. Assignment-shaped reasons (`issue_assigned`,
`issue_reopened_via_comment`, `issue_recovery_action_restored`,
`issue_tree_restored`) and recovery wakes still deliver the full brief.
- `buildPaperclipTaskMarkdown` gains `includeDescription`; the server
now also publishes `context.paperclipTaskMarkdownCompact` (description
stripped, directives and wake comment kept), and the new
`selectPaperclipTaskMarkdown` helper picks the right variant under the
same resume rules, falling back to the full markdown when no compact
variant exists (version skew safety).
- The wake prompt's description block now carries the same user-authored
trust framing the task markdown already had.

## Verification

- `npx vitest run packages/adapter-utils/src/server-utils.test.ts
packages/adapters/claude-local/src/server/acp.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
server/src/__tests__/heartbeat-context-summary.test.ts` — 137 tests
passed, including new coverage for suppression, resume omission plus
breadcrumb, assignment-shaped resume inclusion, compact-variant
building, variant selection, and an end-to-end ACPX prompt-assembly test
asserting the description appears exactly once on fresh wakes and not at
all on comment resumes.
- `npx vitest run` in `packages/adapters/hermes` — 59 tests passed,
including a gateway execute-level test asserting the brief is sent
exactly once on fresh runs and not re-sent on stable-session resumes.
- `tsc --noEmit` in `packages/adapter-utils`,
`packages/adapters/claude-local`, `packages/adapters/hermes` — clean;
`server` matches the `master` baseline exactly (pre-existing plugin-sdk
resolution errors only, none in touched files).
- Pre-existing failures confirmed identical on clean `master`:
claude-local `execute.remote.test.ts` / `test.probe.test.ts`,
adapter-utils `mcp-isolation.integration.test.ts` (requires a newer
local Claude CLI).

## Risks

- Behavioral shift, prompt-only: a resumed session woken by a comment on
an issue it never handled (rare — assignment wakes normally precede
comment wakes) would not get the inline description; the breadcrumb plus
the standard issue-fetch path covers it.
- Additive context key (`paperclipTaskMarkdownCompact`); older adapters
ignore it and newer adapters fall back to the full markdown when it is
absent, so mixed-version deployments degrade to current behavior.
- No schema, migration, or API changes; the structured wake-payload JSON
shape is unchanged.
- Known follow-up deliberately out of scope: openclaw embeds the raw
wake-payload JSON (which still contains the description) in prompt text
for machine parsing. The hermes-gateway lane is handled: it detects
stable-session resumes (issue/agent session-key strategy plus a stored
prior session id), compacts the task markdown, and omits the description
from its prompt-embedded JSON copy.

> This is a focused correctness/efficiency fix to existing wake plumbing
and does not overlap with planned roadmap feature work.

## Model Used

- Anthropic Claude Fable 5 (`claude-fable-5`), extended thinking
enabled, with repository tool use, shell execution, and local test
execution via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
(execution-workspace branch, same convention as merged #10202)
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
(code-level docs; no user-facing docs affected)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 16:35:07 -05:00
Dotta d3c004d1b8
Fix issue descriptions in structured wake payloads (#10151)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Heartbeat wake payloads provide the scoped task context an agent
needs before it can act safely
> - Assignment wakes already loaded the issue description for task
markdown, but the structured wake-payload builder dropped it
> - Agents reading `PAPERCLIP_WAKE_PAYLOAD_JSON` could therefore see a
missing brief while also being told no fallback fetch was needed
> - Long descriptions also need a bounded representation so wake
environments and prompts remain safe
> - This pull request carries the description through the server and
adapter contract, and marks truncated descriptions as requiring fallback
fetch
> - The benefit is that agents receive the actual brief instead of
inventing requirements from the title

## Linked Issues or Issue Description

Fixes: #5844
Fixes: #2882

Related prior attempts: #2883 and #8402. This change adds focused
regression coverage and enforces the missing long-description fallback
invariant.

**Bug:** Issue-assignment wake payloads omitted the issue description
from the structured payload even when the issue had a populated
description.

**Expected behavior:** The structured wake payload includes the issue
description. If the description must be truncated for payload size,
`fallbackFetchNeeded` is `true`.

**Reproduction:** Assign an issue with a description to an agent and
inspect `PAPERCLIP_WAKE_PAYLOAD_JSON`; before this change,
`issue.description` was absent while `fallbackFetchNeeded` could remain
`false`.

**Affected version:** Reproduced on current `master` before this patch.

**Deployment mode:** Adapter-backed heartbeat execution, including local
Codex agents.

## What Changed

- Include `issues.description` in the server wake-payload query and
supplied issue summaries.
- Bound inline descriptions at 12,000 characters and force fallback
fetch when truncation occurs.
- Preserve and render description metadata through shared adapter
normalization and prompt rendering.
- Add focused tests for long-description fallback and exact brief-string
rendering.

## Verification

- `pnpm exec vitest run
server/src/__tests__/heartbeat-agent-session-message.test.ts
packages/adapter-utils/src/server-utils.test.ts` — 81 tests passed.
- `pnpm --filter @paperclipai/adapter-utils typecheck` — passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `git diff --check` — passed.

## Risks

- Low risk: the payload shape is additive.
- Very long descriptions are truncated at 12,000 characters; the payload
explicitly requests a fallback fetch for the full brief.
- Prompt size increases by the issue-description length for scoped
wakes, bounded by the same limit.

> This is a focused correctness fix and does not overlap with planned
roadmap feature work.

## Model Used

- OpenAI GPT-5.4 via Codex CLI, with reasoning, repository tool use,
shell execution, and test execution. The runtime did not expose a
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 15:40:39 -05:00
Dotta 7014e46e5b
fix(recovery): wake ambiguous successful runs on normal model (#10184)
## Thinking Path

> - Paperclip is the open source control plane people use to coordinate
AI agents and their work.
> - The heartbeat recovery subsystem detects successful runs that leave
assigned issues `in_progress` without a durable disposition or
continuation path.
> - The existing corrective wake used a cheap, status-only model
profile, so the assignee could not perform missing verification or
deliverable work before choosing the issue disposition.
> - The existing wake prompt also omitted the original issue context and
the agent's own final report, making an honest finish/blocked/continue
decision harder.
> - This pull request keeps the structural handoff guards and
one-attempt loop bound, but wakes the assignee on its normal model lane
with context-rich instructions.
> - The benefit is that Paperclip asks the responsible agent to inspect
its own evidence, perform the smallest missing verification when needed,
and then record a real disposition without server-side prose
classification.

## Linked Issues or Issue Description

Related prior approach: #10154 (closed; this PR intentionally does not
reuse its regex classifier or route-level gate).

**Problem**

A succeeded agent run can leave its issue `in_progress` with no valid
disposition. Paperclip already detects this structurally and queues a
corrective handoff, but that wake currently runs as cheap/status-only
recovery and receives little context. The assignee may be unable to
create deliverables or verify the work, and the prompt does not quote
the report that caused the ambiguity.

**Expected behavior**

The corrective wake should use the assignee's normal model and adapter
settings, include the issue identifier/title/description, quote the
agent's own final report, include any recorded next action, preserve the
four disposition options, and explicitly require concrete verification
before marking the issue done.

**Scope**

This change does not classify run prose, add a route-level disposition
gate, alter run-liveness classification, or change the one-attempt
handoff loop bound.

## What Changed

- Switched successful-run corrective handoff payloads and context
snapshots from `status_only` to `normal_model`, removing cheap-model and
status-only guard hints.
- Added issue description, final-report, next-action, and
detected-progress fallback context to the handoff decision and
instruction builder.
- Reworked the instruction into clear "supposed to do / what happened /
options / what to do" sections with bounded description/report excerpts
and verbatim blockquotes.
- Added unit and heartbeat integration coverage for normal-lane
payloads, context plumbing, evidence quoting, fallback behavior, and
truncation while preserving structural skip tests.

## Verification

- `cd server && pnpm exec vitest run
src/services/recovery/successful-run-handoff.test.ts` — 24 tests passed.
- `cd server && pnpm exec vitest run
src/__tests__/heartbeat-process-recovery.test.ts -t "queues one
finish-handoff wake when a successful run leaves in-progress work
without a next action"` — 1 passed, 90 skipped.
- `pnpm --dir server typecheck` — passed.
- `git diff --check` — passed.

## Risks

- Low-to-moderate behavioral risk: an ambiguous successful run now
consumes the assignee's normal model rather than a cheap profile and may
perform verification or finish work before disposition.
- Prompt excerpts are bounded to approximately 1,200 description
characters and 2,000 report characters; very long context is
intentionally ellipsized.
- The existing structural skip guards, idempotency key, and single
corrective attempt remain unchanged to prevent loops.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using `gpt-5.6-sol`, high reasoning effort, with
repository/tool execution. Context-window size was not exposed by the
runtime configuration.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 14:32:26 -05:00
Dotta 3a16b91217
feat(status-cards): single-message setup drives query and update prompt (#10202)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies and their ongoing work.
> - Status cards turn a standing question into recurring,
agent-generated summaries on the board.
> - The existing setup split intent across a watch prompt and separate
update instructions, which made creation and later behavior harder to
understand.
> - A status card should have one durable source of truth for both
deciding what to watch and telling the summarizer what each update must
contain.
> - This pull request makes the card prompt that source of truth,
simplifies creation to one step, and lets operators choose the running
agent immediately.
> - The benefit is a smaller mental model, fewer configuration modes,
and consistent update instructions throughout the card lifecycle.

## Linked Issues or Issue Description

Status cards currently require operators to express the same intent in
two places: the watch prompt and optional update instructions with
append/replace/none modes. This feature simplifies the experimental
status-card workflow so a single prompt defines both the watch query and
every generated update. The create flow must also support selecting the
responsible agent without a second setup step.

Related prior status-card work: #10101.

## What Changed

- Use the status card's single prompt to compile the watch query and
directly instruct every summary update.
- Add migration `0190_status_card_single_prompt` to remove
`status_cards.instructions_mode` and `status_cards.instructions`.
- Add `agentId` to `createStatusCardSchema`, validate company
membership, and default new cards to the built-in Summarizer.
- Replace the two-step create flow with one prompt-and-agent dialog and
extract a shared `SummarizerAgentSelect` for create/settings surfaces.
- Remove the extra-instructions settings section, reset incremental
history when the prompt changes, and rename the board page to "Status".
- Update the bundled `status-card-query` skill and board-operator
documentation, then regenerate the skills catalog manifest.

## Verification

- Server status-card suites: 29/29 passing.
- UI `StatusCards` suites: 22/22 passing.
- Skills catalog suite: 20/20 passing.
- `tsc -b` passes for server, UI, shared, and database packages.
- `pnpm check:migrations` passes.
- Light and dark mode screenshots cover the new create dialog and
settings tab.

## Risks

- Migration `0190` intentionally drops existing separate instruction
text. Existing card prompts remain and become the update instructions
under the new model; status cards are experimental and feature-flagged.
- Prompt edits now reset the incremental summary chain and trigger a
full rebuild, which is intentional because the prompt is also the update
contract.
- Agent selection is company-scoped; invalid agent ids return a
validation error rather than creating a misrouted card.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Implementation: Anthropic Claude via the `claude_local` adapter, agent
label "Claude Fable 5"; extended reasoning, tool use, and code
execution. The exact provider model id and context-window value were not
retained in the task metadata.
- PR preparation: OpenAI GPT-5.4 through Codex CLI, with reasoning,
repository inspection, GitHub CLI, and Paperclip API tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 14:06:14 -05:00
Dotta 7e40ed8c43
feat(status-cards): add experimental status card update view (#10101)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies.
> - Operators need a board-level way to monitor a changing slice of
company work without repeatedly rebuilding filters or reading raw task
threads.
> - Existing summaries are useful snapshots, but they do not provide a
dedicated query-backed card with refresh policy, change tracking, update
history, and per-update cost visibility.
> - The capability needs to be safe to evaluate before it becomes part
of the default product surface.
> - This pull request adds end-to-end experimental Status Cards, from
schema and query compilation through update orchestration and operator
UI.
> - The entire feature is gated behind the `enableStatusCards`
experimental toggle, including its route and sidebar entry.
> - The benefit is a governed, inspectable way to keep focused
operational rollups current while preserving explicit controls over
refresh frequency and spend.

## Linked Issues or Issue Description

### Subsystem affected

Cross-cutting (`packages/db`, `packages/shared`, `server/`, `ui/`, and
bundled skills/docs).

### Problem or motivation

Operators cannot currently define a reusable natural-language view of
company work, compile it into an inspectable query, and keep its summary
current as matching issues change. Rebuilding filters and rereading task
threads makes board-level monitoring repetitive and hides the
relationship between source changes, refresh cost, and the resulting
summary.

### Proposed solution

Add experimental Status Cards that compile operator intent into a query,
summarize matched work, record each update, expose
manual/interval/reactive refresh policies and costs, and preserve the
last good result across stale, updating, paused, and error states. The
capability is off by default and fully gated behind `enableStatusCards`,
including its route and navigation entry.

### Alternatives considered

- Extend existing one-off summaries: rejected because status cards
require persistent query provenance, refresh policy, update history, and
card-specific cost controls.
- Add a dashboard-only filter widget: rejected because it would not
provide governed background refresh, an update ledger, or an inspectable
compile pipeline.
- Ship the surface by default: rejected in favor of an experimental
toggle while behavior and operator value are evaluated.

### Roadmap alignment

This advances Paperclip’s board-level execution visibility and
output-first product goals. `ROADMAP.md` was checked and no duplicate
status-card initiative was found.

### Additional context

No related open PR was found in the public GitHub search for status
cards. The PR-only design wireframes were removed from the repository
after review; the published prototype remains external to the production
source tree.
## What Changed

- Added company-scoped status-card schema, CRUD APIs, compile
provenance, update ledger, shared contracts, validators, and OpenAPI
coverage.
- Added the text-to-query compile pipeline, bundled `status-card-query`
agent skill, query versioning, and authorized write-back flow.
- Added the experimental board, create flow, lifecycle tiles,
detail/settings/debug drawers, archived view, routing, navigation, and
instance setting.
- Added a change-gated update engine with manual, interval, and reactive
refresh policies, trigger selection, active hours, and daily token caps.
- Added per-update token/cost recording, today and lifetime rollups, and
policy-derived cost previews.
- Added operator documentation and agent-authoring hardening for compile
and update behavior.
- Added PR-prep integration coverage for settings/startup wiring and
replaced raw UI values with design-system tokens.
- Removed the PR-only `design/pap-15023-status-cards` wireframe
artifacts so the repository contains only production feature assets.

## Verification

- `pnpm -r typecheck` — passes on the PR head; includes `ui` `tsc -b`
passing. The UI compile gate was also independently recorded as passing
at `6d7f3cf96b` on July 23, 2026.
- `pnpm build` — passes.
- `pnpm check:token-gates` — passes with all three gates clean.
- `pnpm test:run` — 2,880 tests passed and 1 skipped; the sole failure
was an unrelated 10-second `afterAll` database-cleanup timeout in
`execution-workspaces-service.test.ts`.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/execution-workspaces-service.test.ts` — passes on
immediate focused rerun (25/25).
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/instance-settings-service.test.ts
src/__tests__/server-startup-feedback-export.test.ts` — passes (31/31).
- `pnpm --filter @paperclipai/ui exec vitest run
src/pages/StatusCards/StatusCardSettingsForm.test.tsx
src/pages/StatusCards/StatusCardTile.test.tsx
src/pages/StatusCards/format.test.ts src/lib/status-card-state.test.ts`
— passes (26/26).
- Recorded pre-PR QA: compile-pipeline e2e PASS; full lifecycle and cost
QA PASS; security re-review PASS after write-back hardening; UX
approved.
- `pnpm exec vitest run packages/db/src/status-card-migrations.test.ts`
— passes; reapplies migrations `0185`–`0189` against an already-migrated
embedded Postgres database.
- `pnpm --filter /db check:migrations` — passes migration numbering and
safety checks.
- `pnpm --filter /db typecheck` — passes.
- Merged current `origin/master` on July 24, 2026 with no conflicts;
migrations `0185`–`0189` remain unclaimed on master.

## Risks

- The feature introduces five database migrations and a new background
update path; all new DDL is repeat-safe after partial application,
migration numbering/safety checks pass, and update execution is
company-scoped and change-gated.
- Natural-language compilation can produce invalid or overly broad
queries; compile provenance, query validation, debug visibility, and
version history make failures inspectable and recoverable.
- Reactive or interval refresh could increase spend; active hours, max
refresh frequency, daily token caps, per-update cost records, and
budget-paused states bound and expose that risk.
- The branch name contains an internal execution identifier because it
is a fixed handoff branch; it was intentionally not renamed or rebased
per the release handoff instructions.
- Overall rollout risk is limited because the route, navigation,
services, and UI are disabled by default behind `enableStatusCards`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using GPT-5.5 with reasoning, repository tool use, shell
execution, GitHub CLI, and test/build execution. The runtime did not
expose a context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change; the fixed execution-workspace
identifier is documented as an authorized handoff exception
- [x] I have run tests locally and they pass, with the one cleanup
timeout passing on focused rerun
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 12:26:43 -05:00
Devin Foley 965a827ee7
feat(docker): publish a cloud image variant with built bundled plugins (#10157)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Managed (cloud-hosted) deployments configure instances through
`PAPERCLIP_MANAGED_CONFIG`, including a `plugins.autoInstall` key list
that the boot-time installer resolves against the bundled plugin catalog
> - The installer requires each bundled plugin's `dist/manifest.js`
(`server/src/services/bundled-plugins.ts`), but the published image only
ships the sandbox providers' *source* — they are intentionally excluded
from the pnpm workspace, and the Dockerfile never builds them
> - Every managed auto-install therefore logs `bundled plugin bundle not
present; skipping auto-install` and no sandbox provider can be
provisioned through managed config
> - Baking built plugins into the single published image would fix it
but makes every self-hosted pull carry the providers' `node_modules` for
a managed-only mechanism
> - This pull request adds a `cloud` Dockerfile target extending
`production` with built bundled plugins — parameterized by build arg and
currently just `daytona` — published alongside the default image with a
`-cloud` tag suffix
> - The benefit is working plugin auto-provisioning for managed
deployments while the self-hosted image stays byte-identical and the
cloud variant only carries what is actually deployed

## Linked Issues or Issue Description

Fixes #10158 (filed for this problem; no prior issue existed — searched
for duplicate/related PRs and issues around bundled plugins, docker
image variants, and auto-install). Summary: **What happened:** on a
managed instance with `plugins.autoInstall: ["daytona"]` delivered via
`PAPERCLIP_MANAGED_CONFIG`, boot logs `bundled plugin bundle not
present; skipping auto-install` with `pluginPath:
/app/packages/plugins/sandbox-providers/daytona`, and the plugin is
never installed. **Expected:** the advertised bundled-catalog keys are
installable from the published image. **Why:** the image ships plugin
source without `dist/` — nothing in the Dockerfile builds the
workspace-excluded sandbox providers.

## What Changed

- `Dockerfile`: new `cloud-plugins` stage (based on `build`, so
devDependencies are available for `tsc`) that installs and builds each
provider named in the `CLOUD_BUNDLED_PLUGINS` build arg standalone
(`pnpm install --ignore-workspace --no-lockfile && pnpm build`, exactly
as the providers' READMEs prescribe), asserting `dist/manifest.js`
exists per plugin and failing loudly on unknown names; new `cloud` stage
= `production` + the built plugin tree. The arg defaults to `daytona` —
the only provider managed deployments auto-install today; every entry
adds its `node_modules` to the image, so the list grows only with actual
need (a one-line workflow change).
- `.github/workflows/docker.yml`: the existing build step is pinned to
`target: production` (without this, the new trailing stage would
silently become the default build target — this pin is what keeps the
self-hosted image identical); new metadata + build-push steps publish
the `cloud` target (with `CLOUD_BUNDLED_PLUGINS=daytona`) under the same
tag set with a `-cloud` suffix (`sha-<short>-cloud`, `latest-cloud`,
`<version>-cloud`), same schema labels, reusing the GHA layer cache

## Verification

- All seven sandbox providers build standalone from a clean checkout
with the exact commands the new stage runs, each producing
`dist/manifest.js` — so the current `daytona` default works and future
list additions are known-good
- The stage's shell loop was dry-run against the checkout (directory
existence + per-plugin assertion logic)
- Workflow YAML lints clean
- **Not run:** a full multi-arch `docker build` (no local docker
daemon). The `cloud` stage is additive and the default target is pinned,
so the risk is contained to the new build step; the first master build
after merge proves it end-to-end

## Risks

- Self-hosted behavior: unchanged. The default image build is pinned to
the `production` target, which produces the same layers as before this
change; the `cloud` stages run only for the new build step.
- The plugin installs in the `cloud-plugins` stage use `--no-lockfile`
(the providers are workspace-excluded and lockfile-less by design), so
plugin dependency resolution is not pinned at image-build time. This
mirrors the existing Plugins-page install path, which resolves from npm
at install time.
- CI cost: one additional build-push per master push. It reuses the
layer cache from the production build, so the marginal work is the
single plugin's build layers.
- An unknown name in `CLOUD_BUNDLED_PLUGINS`, or a provider that stops
producing `dist/manifest.js`, fails the cloud build loudly rather than
publishing a broken variant.

## Model Used

Claude (Anthropic), model ID `claude-fable-5[1m]` via Claude Code CLI —
extended thinking and tool use (code edits, standalone plugin build
verification, workflow lint).

## Checklist

- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] Self-hosted behavior unchanged (default build target pinned to
`production`)
- [x] One clear change: publish a cloud image variant with built bundled
plugins
2026-07-24 08:22:34 -07:00
Dotta 7f766526a6
feat(sandbox): add task-scoped egress grants (#10155)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Confinement providers protect agent runs with default-deny network
policies
> - Kubernetes environments currently apply only provider-level,
namespace-wide egress allowances
> - Tasks that legitimately need GitHub or package registries therefore
cannot request narrow access, while network failures do not explain the
governing policy or how to request a grant
> - This pull request adds issue-scoped egress grants that become
workload-owned, run-label-selected policies and carries the effective
grant through lease audit metadata
> - The benefit is that internet-dependent work can run without enabling
broad egress for every concurrent task, and denied requests point
operators to the exact grant path

## Linked Issues or Issue Description

No public issue exists. Related but distinct: Refs #9944, which adds a
provider-wide open-internet posture; this PR keeps provider defaults
narrow and adds per-task grants.

**Problem / motivation**
Kubernetes sandbox egress is configured at the provider/tenant level. A
task that needs to clone from GitHub or install from PyPI cannot request
those destinations without changing the policy for every run in the
tenant namespace. DNS/connectivity failures also surface as generic tool
errors with no policy name or remediation path.

**Proposed solution**
Accept `executionWorkspaceSettings.networkEgress.allowFqdns` and
`allowCidrs`, forward the setting through heartbeat environment
acquisition, and create a workload-owned NetworkPolicy or
CiliumNetworkPolicy selected by `paperclip.io/run-id`. Record the
effective grant in lease activity/metadata, expose policy context
through `PAPERCLIP_NETWORK_EGRESS_*`, and append the grant path to
likely policy-related stderr failures.

**Alternatives considered**
A provider-wide open-internet switch is broader than required and is
already covered by #9944. Mutating the existing namespace policy would
leak each task's destinations to other concurrent runs. Standard
Kubernetes NetworkPolicy cannot enforce FQDNs exactly, so standard mode
uses the existing hardened public-IPv4 TCP 80/443 fallback only for the
selected run; Cilium mode remains exact.

**Roadmap alignment**
This extends the existing cloud/sandbox agent roadmap capability with
task-level control-plane policy and does not duplicate a planned roadmap
item.

## What Changed

- Added validated `networkEgress` grants to issue execution workspace
settings and forwarded them through environment lease acquisition.
- Added workload-owned, run-label-scoped
NetworkPolicy/CiliumNetworkPolicy resources for task FQDN/CIDR grants.
- Added lease audit metadata, sandbox policy environment variables, and
actionable network-denial stderr guidance.
- Added focused parser, manifest, policy creation, and denial-message
tests plus Kubernetes provider documentation.

## Verification

- `pnpm -C packages/shared exec vitest run src/validators/issue.test.ts`
— 27 passed.
- `pnpm -C packages/plugins/sandbox-providers/kubernetes test -- --run
test/unit/network-policy.test.ts test/unit/cilium-network-policy.test.ts
test/unit/scoped-network-egress.test.ts` — 21 passed.
- `pnpm -C server exec vitest run
src/__tests__/execution-workspace-policy.test.ts` — 15 passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-plugin-environment.test.ts
server/src/__tests__/environment-runtime.test.ts` — 26 passed.
- `pnpm --dir packages/db build && pnpm --dir packages/shared build &&
pnpm --dir packages/plugins/sdk build` — passed, including migration
safety checks.
- `pnpm --dir packages/plugins/sandbox-providers/kubernetes typecheck &&
pnpm --dir server typecheck` — passed after refreshing the worktree's
frozen offline dependencies.
- End-to-end cluster validation of the `build-cython-ext` benchmark
remains for CI/maintainer Kubernetes infrastructure; the focused tests
assert `github.com` and `pypi.org` produce a policy selected only by the
granted run.

## Risks

- Standard NetworkPolicy cannot express FQDNs, so an FQDN grant allows
hardened public IPv4 TCP 80/443 for that run; use Cilium mode for exact
hostname enforcement.
- The new field is additive and absent by default, so existing runs keep
the current provider-level policy.
- Workload owner references garbage-collect scoped policies with the
Job/Sandbox; a cluster/controller that ignores owner references could
temporarily strand a policy that still selects no future run ID.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, tool
use and code execution. The runtime did not expose a context-window
size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 09:58:58 -05:00
Nicky Leach 564870020b
Exclude archived projects from the default project list route (#10146) 2026-07-24 07:19:58 -07:00
Michael Nguyen caae2778f0
fix: deliver plugin agent session turns and replies (#10137)
## Thinking Path

> - Paperclip manages agent execution through heartbeat runs and
adapter-specific sessions
> - Plugins can open an agent session and send a conversational message
through the host service
> - The host previously stored that message only in opaque wake payload
metadata, so local adapters never saw it in their CLI prompt
> - The host also forwarded run log chunks but did not expose the
persisted final assistant text as the session reply
> - This pull request defines both sides of the session contract in the
shared wake renderer and terminal run event
> - The benefit is that local adapters receive the actual conversational
turn and plugins receive one canonical final reply

## Linked Issues or Issue Description

Related context: Refs #629 and Refs #2880 describe adjacent
`claude_local` final-text visibility failures. They concern issue
comments rather than plugin agent sessions, but exercise the same need
for a canonical persisted run summary.

Companion consumer change: paperclipai/paperclip-gateway#3.

Bug description:

- **Observed:** calling the plugin host's
`agents.sessions.sendMessage()` with `prompt: "hello"` woke a
`claude_local` agent, but the generated CLI prompt omitted `hello`. On
completion, the session emitted log chunks and a generic `Run completed`
done event, so callers could not reliably recover the assistant reply.
- **Expected:** the prompt becomes the user-supplied conversational turn
for that agent session, and the successful terminal event carries the
run's canonical final user-facing assistant text.
- **Reproduction:** create a plugin agent session for a local adapter,
call `sendMessage()` with a non-empty prompt, inspect the adapter prompt
and terminal session event.
- **Affected baseline:** `b517b887a` on `master`, local trusted
deployment with plugin host services and `claude_local`; `codex_local`
shared the wake-rendering gap because both use the common Paperclip wake
prompt renderer.

## What Changed

- Added a typed `agentMessage` wake payload rendered by the shared
adapter prompt path used by `claude_local`, `codex_local`, and other
local adapters.
- Labeled session content as user-supplied and explicitly
non-authoritative: it cannot expand authorization, permissions, task
scope, or company boundaries.
- Preserved ordinary heartbeat behavior by omitting the section when no
agent-session message exists.
- Added canonical `finalText` to terminal heartbeat status events from
the already-persisted run summary/result/message.
- Defined successful `AgentSessionEvent.message` as the canonical final
user-facing reply (or `null`) and forwarded it on the terminal `done`
event.
- Added host, wake-renderer, normal-heartbeat, and terminal-reply
regression coverage.

## Verification

- `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts
server/src/__tests__/heartbeat-agent-session-message.test.ts
server/src/__tests__/heartbeat-run-status-payload.test.ts
server/src/__tests__/plugin-agent-sessions.test.ts
server/src/__tests__/heartbeat-run-summary.test.ts` — 87 passed.
- `pnpm -r typecheck` — passed across all 31 workspaces.
- `pnpm build` — passed.
- `pnpm test:run` — 2,860 passed, 1 skipped, 3 unrelated failures: two
existing macOS temp-path alias assertions (`/tmp` vs `/private/tmp`) in
workspace branch-containment tests and one reproducible auto-port
runtime-service adoption failure. The same three failures reproduce when
the two files run alone; none touch this change.
- Live Slack verification intentionally remains operator-gated because
it requires rebuilding/restarting the host.

## Risks

- User-controlled chat text now reaches the model prompt, which is an
intentional prompt-injection surface. The renderer labels it as
untrusted conversational content, while the existing plugin/session
company checks and caller authorization remain unchanged.
- `finalText` is added to company-scoped heartbeat status events. It is
derived from the same persisted summary/result/message already used for
run comments; no raw stdout or secrets are added.
- Consumers that ignore the new field remain compatible, and successful
runs without usable final text still emit `message: null`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex (GPT-5), agentic reasoning with repository/tool use and
code execution; context-window size is not surfaced in this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 20:52:42 -07:00
Nicky Leach e41ba306c5
feat(secrets): thread audit actor into skip-user-secret skills routes (#10124)
## Thinking Path

> - Paperclip manages agent work and needs auditable control over secret
resolution
> - The skip-user-secret skills routes still have to attribute access to
the real actor
> - These routes were calling the adapter config resolver without an
access context
> - That dropped actor attribution from the company `secret_ref` audit
trail
> - This pull request threads the existing actor-secret context helper
into both skills routes
> - The benefit is that audit fidelity is restored without changing
`skipUserSecrets` behavior

## Linked Issues or Issue Description

Refs #10115.

This PR fixes a gap in the skills read/sync routes where
`resolveAdapterConfigForRuntime` was being called without an audit
access context, so company secret resolution could not reliably
attribute the request to the acting user or agent. The change keeps
`skipUserSecrets: true` intact and only restores audit fidelity.

## What Changed

- Threaded `buildActorSecretContext(req, { consumerType: "agent",
consumerId })` into `GET /agents/:id/skills`
- Threaded the same actor context into `POST /agents/:id/skills/sync`
- Updated the route tests to assert a non-`undefined` actor context
reaches the resolver while `skipUserSecrets: true` stays unchanged

## Verification

- `tsc --noEmit`
- `agents` and `secrets` Vitest suites: 33 files / 448 tests green
- Route spy assertions confirm both skills routes now pass an
actor-derived context to the resolver

## Risks

- Low risk: the change is limited to audit context propagation on two
skills routes
- If a downstream resolver assumes the third argument can be
`undefined`, this makes the context explicit on these routes
- The user-secret authorization behavior does not change because
`skipUserSecrets` remains true

## Model Used

OpenAI GPT-5 via Codex, tool-using coding agent, 256k context window

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 15:31:55 -07:00
Devin Foley 176a9e8230
fix(built-in-agents): allow first-time setup of a needs_setup built-in under board-approval policy (#10129)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Companies can require **board approval for new agents**; built-in
agents (e.g. the Reflection Coach / Briefs) are provisioned through the
`built-in-agents` service `provision()`
> - Some built-in agents are *auto-provisioned* as a hire that, once
approved, resolves to an idle agent row whose `adapterConfig` is still
empty — status `needs_setup`
> - When the board operator then opens that agent's setup dialog and
submits the adapter config, `provision()` saw
`adapterType`/`adapterConfig` on an already-existing row and classified
it as a **reconfiguration**, throwing a dead-end 409: *"Built-in agent
adapter changes require board approval before they can be applied."*
> - The operator *is* the board, so there was no one left to grant an
approval they already implicitly hold — setup could never be completed
> - This pull request treats first-time adapter setup of a `needs_setup`
built-in as the first-time configuration it actually is, applying it
directly while still gating genuine reconfiguration of a live agent
> - The benefit is the board can finish setting up an auto-provisioned
built-in agent without hitting an unsatisfiable approval wall

## Linked Issues or Issue Description

<!-- No public GitHub issue exists; describing the underlying bug in-PR
following the bug_report template. -->

**What happened?**

With "require board approval for new agents" enabled, completing the
adapter setup of an auto-provisioned but unconfigured built-in agent
(status `needs_setup`, e.g. the Reflection Coach) failed with a 409 —
*"Built-in agent adapter changes require board approval before they can
be applied."* — even for the board user. Because the operator *is* the
board, no additional approver existed, so setup was permanently blocked.
Root cause: in `builtInAgentService.provision()`, any request carrying
`adapterType`/`adapterConfig` against an existing row was treated as a
reconfiguration and gated, regardless of whether that row had ever
completed its initial adapter setup. An auto-provisioned hire resolves
to an idle row with an empty `adapterConfig` (`needs_setup`), so its
very first configuration was misclassified.

**Expected behavior**

The board can complete first-time setup of an already-sanctioned
built-in agent without a fresh approval, matching the behavior when
board approval is not required. Genuine reconfiguration of an
already-configured (`ready`/`paused`) agent should still require
approval.

**Steps to reproduce**

1. In a company with `requireBoardApprovalForNewAgents` enabled, have a
built-in agent auto-provisioned so its row exists but its adapter is
unconfigured (status `needs_setup`).
2. As the board user, open that agent's setup dialog and submit an
adapter type + config.
3. Observe the 409 "Built-in agent adapter changes require board
approval before they can be applied." with no way for the board to grant
the approval.

**Deployment mode**

Local single-instance / self-hosted (server `built-in-agents` service).

## What Changed

- `server/src/services/built-in-agents.ts`: In `provision()`, when the
existing built-in row has **not** yet completed adapter setup
(`!hasCompleteAdapterConfig(...)`, i.e. `needs_setup`), first-time
adapter configuration now applies directly via `ensure()` — the same
path used when board approval is not required. The hire that created the
row was already sanctioned, so no fresh approval is required.
- Reconfiguration of an already-configured (`ready`/`paused`) built-in
agent stays gated behind board approval exactly as before, and
`pending_approval` rows are handled before the new branch.
- `server/src/__tests__/built-in-agents.test.ts`: Added a regression
test — under `requireApproval: true`, completing first-time setup of a
`needs_setup` built-in returns `approval: null`, transitions the agent
to `ready`, and creates **no** approval row.

## Verification

```bash
cd server
npx vitest run src/__tests__/built-in-agents.test.ts
# Test Files  1 passed (1)
#       Tests  31 passed (31)
```

- New test `completes first-time setup of a needs_setup built-in without
a fresh board approval` passes.
- Full `built-in-agents.test.ts` suite (31 tests) passes, including
existing tests that assert genuine reconfiguration of a configured agent
**remains** gated.

## Risks

Low risk. The change narrows an over-broad approval gate: it only opens
the direct-apply path for rows that have never completed adapter setup
(`needs_setup`), determined by the existing `hasCompleteAdapterConfig`
predicate that already drives `deriveBuiltInAgentStatus`.
Already-configured (`ready`/`paused`) agents, and `pending_approval`
rows, are unaffected and still gated. No schema or migration changes.

## Model Used

Claude Opus 4.8 (`claude-opus-4-8`), 1M context, extended thinking, with
tool use / code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (searched my open PRs and compared patch-ids — no duplicate
exists)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-23 14:53:15 -07:00
Michael Nguyen e2068319e7
fix(interactions): tolerate legacy stored result outcomes so listInteractions can't fail the whole list (#10119)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents and humans coordinate on issues through interaction requests
(confirmations, decisions, task suggestions and more) that are stored
per issue and listed by both the web UI and plugin workers such as chat
gateways
> - `listForIssue` hydrates every stored interaction row by hard-parsing
its persisted `result` blob against the current Zod schema
> - Stored rows outlive code: one live row written by an older build
carried `result.outcome: "withdrawn_by_creator"`, a value no longer in
the enum, and that single row made hydration throw
> - Because the throw happened inside the list mapping, it failed the
entire issue's interaction list — the web thread errored, and every
plugin consumer of `issues.listInteractions` (notification drain, digest
confirmation sweep, pending-ledger reads) failed continuously, so
interaction cards never reached chat surfaces
> - This pull request parses stored `result` blobs tolerantly — a
`parseStoredInteractionResult` helper wrapping `safeParse`, applied to
all five interaction kinds — so an unparseable result degrades to `null`
with a warning instead of failing the whole list
> - The benefit is durable robustness at the storage→hydrate boundary:
legacy or future schema drift in a single row can no longer take down an
issue's entire interaction surface

## Linked Issues or Issue Description

No pre-existing public issue; the underlying problem is described here
following the bug-report template. Related (not a duplicate): Refs #6709
— the creator-withdraw flow it explores matches the legacy outcome value
observed in the wild; whether or not that lineage wrote the row, this PR
is defensive against any such stored-schema drift.

**What happened**

Listing interactions for an issue (`GET /api/issues/:id/interactions` on
the web, or the `issues.listInteractions` plugin RPC) fails for the
entire issue when any single stored interaction row carries a
`result.outcome` written by an older build (observed live:
`"withdrawn_by_creator"`). Downstream plugin consumers that poll this
RPC fail continuously — notification drain, digest confirmation sweep,
and pending-ledger reads.

**Expected behavior**

One legacy/unreadable stored `result` should degrade gracefully — the
interaction still lists with its result treated as absent — rather than
failing the whole issue's interaction list.

**Steps to reproduce**

1. Persist a resolved `request_confirmation` interaction whose
`result.outcome` is not in the current enum (e.g.
`"withdrawn_by_creator"`, as written by an older build).
2. Call `issues.listInteractions` (or `GET
/api/issues/:id/interactions`) for that issue.
3. The call throws `invalid_enum_value` and returns nothing, instead of
returning the remaining rows.

**Version or commit**

master @ 3093c5e69 (also reproduces on a live deployment carrying
pre-enum-change rows).

**Deployment mode**

Self-hosted host with plugin workers (chat gateway).

## What Changed

- Added `parseStoredInteractionResult`, a small generic helper in
`server/src/services/issue-thread-interactions.ts` that wraps Zod
`safeParse` for stored `result` blobs: on parse failure it logs a
warning and returns `null` instead of throwing.
- Replaced all five hard `.parse()` calls in `hydrateInteraction` (one
per interaction kind) with the tolerant helper, so a single unreadable
row degrades to `result: null` rather than failing the entire
`listForIssue` mapping.
- Left payload parsing strict on purpose — payloads are written at
creation time by current code; only `result` has demonstrated legacy
drift, and keeping payloads strict preserves detection of genuine
write-path bugs.
- Added a regression test in
`server/src/__tests__/issue-thread-interactions-service.test.ts` that
seeds a resolved `request_confirmation` with `result.outcome:
"withdrawn_by_creator"` and asserts `listForIssue` returns the row with
`result: null` instead of throwing.

## Verification

- `tsc --noEmit` (server) — clean.
- `issue-thread-interactions-service.test.ts` — 39/39 pass, including
the new regression test reproducing the exact live failure value.
- Full CI on this PR is green: typecheck, serialized server suites,
general tests, e2e shards, build, canary dry run.

## Risks

- Low: server-only change at the read/hydrate boundary; no schema or
write-path changes, no SDK dist rebuild.
- Behavioral shift: a resolved interaction with an unreadable stored
`result` now lists with `result: null`. Consumers already handle
`result: null` (it is the shape of every unresolved interaction);
anything assuming "resolved ⇒ non-null result" sees the legacy row
differently than before — though previously the same row produced a hard
failure of the whole list, so this is strictly an improvement.
- The degrade path logs a warning, so stored-schema drift stays visible
rather than silent.

## Model Used

- Claude (Anthropic) — via the Claude Code CLI agent.
- Exact model ID: `claude-fable-5` (Claude Fable 5).
- Extended thinking (chain-of-thought reasoning) enabled; agentic tool
use including file editing and local test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [ ] I have not referenced internal/instance-local Paperclip issues or
links — *the PR title, description, and comments are clean, but the
branch commit message carries an internal ticket id from the originating
workspace; this repo squash-merges, so the final master commit takes the
clean PR title and the interim message never lands*
- [ ] My branch name describes the change and contains no internal
Paperclip ticket id — *the branch was pushed before this check; renaming
now would close this PR and discard its green CI, and the branch name is
likewise dropped at squash-merge*
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
documentation is affected by this server-internal fix)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 14:47:27 -07:00
Dotta e3f8380e70
feat(skills): make summarize-status actions-first (#10117)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Its built-in Summarizer keeps status slots useful for people
overseeing issue trees
> - Those summaries need to tell the reader what they must do now to
unblock progress
> - The existing skill instead imposed rigid Decide:/Review:/Recent
work: sections, cost commentary, and restrictive issue-fetch guidance
> - This pull request rewrites the summarize-status instructions to lead
with 1–3 specific, concrete unblock actions while letting the model use
its judgment for the remaining context
> - The benefit is a shorter, clearer summary that is immediately
actionable without changing slot writes or the streaming status protocol

## Linked Issues or Issue Description

Refs #9713

The built-in summarizer currently prioritizes a fixed reporting template
over the reader's immediate unblock actions. Summaries should instead
open with the 1–3 specific actions the reader needs to take right now,
then provide only the context needed to act. This prompt-only update
preserves all summary-slot mechanics and protocols.

## What Changed

- Rewrote the bundled `summarize-status` skill to open with 1–3
specific, concrete, actionable items needed right now to unblock the
work.
- Removed the rigid Decide:/Review:/Recent work: template, the Cost
discipline section, and the restrictions against fetching issue detail.
- Kept slot-write mechanics and the streaming `STATUS`/sentinel protocol
unchanged.
- Updated all materialized copies and tests for the same skill text: the
`SKILL.md` source, regenerated catalog manifest hashes, compiled
fallback string, summarizer built-in `AGENTS.md` and routine, summary
generation-issue instructions, and the two tests pinning those strings.
- Although the diff touches eight files, every file is either the same
skill text in another materialized form or a test asserting it. No
behavior outside the summarizer's prompt text changes.

## Verification

- `pnpm --filter @paperclipai/skills-catalog test` — 20/20 tests pass.
- `pnpm exec vitest run server/src/__tests__/summary-slots.test.ts
server/src/__tests__/built-in-agents.test.ts` — 46/46 tests pass.
- `git diff --check origin/master...HEAD` — clean.
- `pnpm exec vitest run server/src/__tests__/summary-slots.test.ts` —
16/16 tests pass after the Greptile consistency fix.
- Latest-head GitHub checks — 25 terminal checks, all successful,
neutral, or skipped.

## Risks

- Low risk: this intentionally changes generated summary wording and
prioritization, but does not change APIs, persistence, slot-write
behavior, or the streaming protocol.
- The branch name contains an internal task identifier because it was
pre-created and pre-pushed for this assigned change; the PR title and
body do not expose the internal ticket.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using `gpt-5.6-sol`, high reasoning mode, with
repository, terminal, GitHub CLI, and code-execution tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 15:59:44 -05:00
Dotta 148a5b11f5
Route blocked transitions to explicit unblock owners (#10112)
## Thinking Path

> - Paperclip is the open source control plane people use to coordinate
AI-agent companies.
> - Issue status transitions determine whether work keeps moving or
silently stalls.
> - A blocked issue previously could rely on prose alone, leaving the
intended unblock owner unstructured and unnotified.
> - Existing blocker-attention classification could identify stalled
chains, but the signal was not delivered to the board attention feed.
> - Blocked transitions also need rollout-safe deduplication so upgrades
do not notify for historical issues and repeated processing does not
create notification storms.
> - This pull request adds structured unblock descriptors, prospective
transition timestamps, owner delivery, and board attention routing with
focused authorization controls.
> - The benefit is that newly blocked work has an explicit, routable
next action without weakening company boundaries or allowing agents to
inject arbitrary human attention items.

## Linked Issues or Issue Description

Related documentation PR: #10094.

### Subsystem affected

Cross-cutting: `server/`, `packages/db`, and `packages/shared`.

### Problem or motivation

An issue can enter `blocked` without a machine-readable unblock path.
Prose-only ownership does not reliably wake the responsible agent or
surface human-owned work, while the existing `blockerAttention`
classifier is not delivered to an operator-facing attention feed.

### Proposed solution

Require new transitions into `blocked` to have unresolved blockers, a
pending interaction/approval, or a structured `{ owner, action }`
descriptor. Notify an allowed owner once per prospective transition,
route human-owned cases to board attention, and leave pre-rollout
blocked issues untouched.

### Alternatives considered

- Keep prose-only blockers: rejected because ownership remains
unroutable.
- Backfill all historical blocked issues: rejected because upgrades
would create notification storms.
- Let agents target arbitrary users or the board: rejected after
security review because it creates an attention-injection channel.

### Roadmap alignment

Aligns with `ROADMAP.md` → “Enforced Outcomes (watchdogs, recovery
actions, review gates)” by making blocked work carry an explicit
continuation path.

### Additional context

The implementation is prospective-only and deduplicated per blocked
transition. Agent-authored descriptors are limited to the acting agent;
board actors retain human-owner routing.

## What Changed

- Added persisted unblock descriptors and prospective blocked-transition
delivery timestamps with an idempotent migration.
- Added shared types and validation for board, user, and agent unblock
owners.
- Enforced valid blocked transitions and same-company owner validation
in the issue update route.
- Restricted agent-authored descriptors to the acting agent itself,
preventing board/user attention injection by compromised agents.
- Added one-per-transition agent wake delivery and prospective-only
rollout gating.
- Routed human-owned blocker attention into the board attention feed.
- Added focused tests for validation, prospective delivery, flap
deduplication, attention routing, route authorization, and stop-relay
compatibility.

## Verification

- `pnpm -r typecheck`
- `pnpm exec vitest run
server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts
server/src/__tests__/routable-blocked.test.ts
server/src/__tests__/attention-service.test.ts
packages/shared/src/validators/issue.test.ts`
- `AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= pnpm test:run`
- `pnpm build`
- `pnpm --filter @paperclipai/db check:migrations`

## Risks

- Behavioral shift: new `blocked` transitions without a real blocker,
pending governed action, or structured descriptor now return `422`.
- Notification abuse is constrained by same-company validation, agent
self-only routing, prospective rollout gating, and transition-scoped
deduplication.
- Migration risk is low: columns are additive, nullable, and use `IF NOT
EXISTS`; historical blocked issues are not backfilled or notified.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI with GPT-5.4, reasoning-enabled tool use and code
execution. The runtime did not expose a context-window value.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 15:49:28 -05:00
Nicky Leach 81f47e70a6
feat(secrets): thread the acting user into user-scoped secret resolution (#10115)
## Thinking Path

> - Paperclip is the control plane for autonomous AI companies
> - Its agents and adapters need to resolve secrets through the same
governed runtime path that checks ownership and company boundaries
> - This change fixes a gap where user-scoped secret resolution could
lose the acting-user context before adapter runtime startup
> - Without that context, a required user secret could fail closed with
responsible_user_missing even though an authenticated user was in scope
> - This PR threads the acting user into the user-scoped secret
resolution path and keeps the owner boundary explicit
> - The benefit is adapter runtime setup can resolve the right
credential without broadening access

## Linked Issues or Issue Description

Refs #8309 (related: agent secret_ref env drift and binding context)

No exact public GitHub issue for this specific behavior.

### Bug report

- Problem: two agent-management routes resolved user-scoped secrets
without an acting-user binding, so a required `user_secret_ref` could
not be resolved before runtime.
- Expected behavior: the authenticated acting user should be threaded
into user-scoped secret resolution so the owning user secret can be
selected safely.
- Actual behavior: adapter startup paths failed closed with
`responsible_user_missing` even though a user was already in scope.
- Steps to reproduce: configure an adapter test-environment or login
flow that depends on a user-scoped secret, then invoke it with an
authenticated user context that does not carry the acting-user binding
into runtime secret resolution.
- Impact: the adapter test-environment probe and login path cannot
start, so the runtime never reaches the work it was supposed to do.

## What Changed

- Added an actor secret-context helper so the server can derive
responsible-user context without inventing config-path or binding
allowlists.
- Added an explicit user-secret mediation mode for runtime config
resolution, with an owner-scoped path that resolves by definition plus
owner boundary and fails closed when an allowlist is present.
- Wired the adapter test-environment route to owner-scoped mediation
with an audit-only consumer and kept claude-login on the declared path
with its persisted agent identity.
- Added and updated tests for the factory, owner-scoped resolver mode,
and adapter route coverage.

## Verification

- `tsc --noEmit` clean
- Factory tests: `authz-secret-context` 5/5
- Service tests: `secrets-service-user-secret-owner-scoped` 5/5,
including fail-closed allowlist coverage and company-secret
non-regression
- Route tests: `agents-adapter-config-user-secret` 5/5, including
`responsible_user_missing` and `binding_missing` coverage
- Regression suites: `agents` + `secrets` 194/194

## Risks

- A regression in the owner-scoped mediation path could accidentally
loosen secret access if the audit consumer or allowlist guard changes.
- The change depends on the server-derived responsible user; if auth
context regresses, the system should fail closed with
responsible_user_missing.
- The new mediation mode adds a branch in runtime config resolution, so
future changes need to keep declared-mode behavior intact.

## Model Used

- OpenAI GPT-5 (Codex tool-use session)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 13:30:26 -07:00
Michael Nguyen f2f168f6a1
fix(plugins): seed proactive company scopes before worker setup() + events.subscribe resolver parity (LOOA-695) (#10113)
- [x] I searched the GitHub PR list for similar PRs (dedup search). No
open PR touches the proactive `events.subscribe` ordering path; #10103
(merged) is the predecessor whose ordering bug this fixes.

## Thinking Path

The gateway worker's outbound push path is permanently dead
(`eventSubscriptions: 0`, `notifier.received: 0`, `decisions.delivered:
0`). The plugin loader authorizes the worker's **proactive company
scopes only AFTER `startWorker` resolves**, but a proactive plugin
issues its one-shot `events.subscribe` calls from `setup()` — which runs
*while `startWorker` is still awaiting the worker's initialize
response*.

So at subscribe time `proactiveCompanyScopes` is still empty →
`contextForWorkerMessage` resolves no scope → the governed-access gate
rejects every subscribe with `company context is required`. The gateway
subscribes once and never retries, so `eventSubscriptions` stays 0 for
the worker's life. This is an **ordering bug in the #10103 fix**, not a
new method — same #9557 governed-access class as `config.get` (#10092)
and `state.get` (#10103).

Confirmed live at the 18:21:21Z worker respawn on `3093c5e` (host log),
and again at the 19:01:04Z restart (still `events.subscribe: company
context is required`, `eventSubscriptions:0`).

## What Changed

1. **Loader ordering** (`plugin-loader.ts`): load
`registry.listConfigs(pluginId)` in a new step 4b **before**
`startWorker`, and thread the configured company set into
`WorkerStartOptions.proactiveCompanyScopes` so the worker handle is
authorized *before the child process issues any host call*. The same
rows are reused for startup config delivery (step 5b) — no second
`listConfigs` round-trip. The runtime config-change path
(`routes/plugins.ts`) still refreshes scopes via
`setProactiveCompanyScopes` (unchanged).
2. **Handle seed** (`plugin-worker-manager.ts`):
`createPluginWorkerHandle` seeds its `proactiveCompanyScopes` set from
options at creation, before spawn.
3. **Resolver/gate parity** (`plugin-worker-manager.ts`):
`referencedCompanyId(method, params)` now mirrors the SDK gate
`requestedCompanyScope` exactly in the functional direction — adds
`events.subscribe → params.filter.companyId` (how `ctx.events.on(name, {
companyId }, fn)` issues its subscribe), and declines the gate's
wildcard cases (`companies.list`, `scopeKind:"company"` without
`scopeId`) so proactive access only ever grants a **single explicit
configured company, never "all"**. Answers LOOA-693 AC#4 (host/gate
extraction parity) in the functional direction.

## Tests

New `plugin-worker-manager.test.ts` cases (drive a real worker):
- a `setup()`-time `events.subscribe({ filter: { companyId } })` for an
options-seeded company is **admitted** (fails on prior code — no options
seed, no filter parity);
- an unconfigured company stays **denied**;
- an unseeded worker stays **denied**.

Full `plugin-worker-manager.test.ts` suite: **21 passed**. Server `tsc
--noEmit`: clean. All PR CI green (typecheck, server/workspace suites,
e2e, build, security scans).

## Risks

- **Scope-widening risk (primary).** The change grants proactive host
access keyed off configured company rows. Mitigated by: the authorized
set is exactly `registry.listConfigs(pluginId).map(companyId)`; wildcard
cases (`companies.list`, company-scoped key without `scopeId`) resolve
to `null`, never `{ kind: all }`; empty/whitespace ids dropped; an empty
config set grants zero proactive access. This is the surface
SecurityEngineer must sign off (see Security gate).
- **In-invocation path unchanged.** Calls carrying a host-issued
`paperclipInvocationId` keep the existing strict single-company match;
the proactive branch only applies when there is no invocation id — so no
regression to the enforced request path.
- **Blast radius.** Loader step 4b is best-effort: a `listConfigs`
failure logs and proceeds with an empty seed (fails closed — no push,
not a crash), matching today's behavior.

## Model Used

Claude Opus 4.8 (`claude-opus-4-8`) via Claude Code (agent: CTO).

## Security gate

Touches the company-scope resolution path (same surface as #10103).
Routed through **SecurityEngineer review before merge** (tracked on
LOOA-696) — must not widen beyond configured companies; in-invocation
strict single-company match untouched; wildcard cases deliberately
declined in the proactive direction.

## Verification once live

- Host log clean of `events.subscribe: company context is required` at
worker start
- loader logs `eventSubscriptions: N>0`
- beat `notifier.received` / `decisions.delivered` move on real
issue/approval activity

Parent: LOOA-629 (outbound push half of "gateway active"). LOOA-695.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 12:32:10 -07:00
Michael Nguyen 3093c5e694
fix(plugin-worker): resolve a company scope for proactive worker→host calls (#10103)
Host authorizes the plugin's configured companies as the worker's proactive scopes, set by the loader right after the #10092 config-delivery step and refreshed on operator config-save. At the single worker→host chokepoint, a no-invocation call (notifier drain, decision reconcile, mirror drain, digest, aging, liveness beat) that references a configured company resolves to that company's scope, so the #9557 governed-access gate admits it. One change covers the full proactive surface (state.*, issues.*, approvals.*, config.get, secrets.resolve, etc.).

Safety: never widens beyond configured companies (any other company stays denied); in-invocation calls keep #9557's strict single-company match untouched.

Fixes the Slack gateway DM round-trip for LOOA-629. Security review PASS (LOOA-693); non-blocking LOW follow-up tracked in LOOA-694.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-07-23 11:06:02 -07:00
Dotta a17bee98f2
Allow trust-gated direct-parent issue reports (#10098)
## Thinking Path

> - Paperclip is the control plane used to coordinate and govern
AI-agent companies.
> - Agent issue access must preserve company boundaries and trust-policy
containment without preventing legitimate task coordination.
> - Checked-out standard-trust child runs need a narrow way to report
progress directly to their parent issue, but existing authorization
treated that report like an arbitrary cross-boundary write.
> - Low-trust review runs must remain contained, and stop propagation
must not copy potentially untrusted child prose into a higher-trust
parent context.
> - This pull request adds an audited, one-hop direct-parent comment
grant only for standard checked-out runs and a sanitized, idempotent
relay for blocked or cancelled child stops.
> - The benefit is restored parent/child liveness while retaining least
privilege, complete mediation, and low-trust output quarantine.

## Linked Issues or Issue Description

### What happened?

A standard-trust agent running a checked-out child issue could not post
a progress comment to the direct parent issue because the authorization
boundary treated it as an arbitrary cross-issue write. This could stall
parent/child coordination. Low-trust review runs also need stop
propagation without exposing quarantined child-authored prose.

### Expected behavior

A standard checked-out child run may add a comment only to its direct
parent issue. The grant must not allow grandparent or sibling access,
issue mutation, document writes, reopening, or resuming. Low-trust runs
remain denied unless separately mentioned, while blocked/cancelled stops
relay only sanitized system metadata once.

### Steps to reproduce

1. Create a parent issue and a child issue assigned to different
standard-trust agents.
2. Check out the child issue in a heartbeat run and authenticate as that
run.
3. Post a comment to the parent issue and observe the authorization
denial before this change.
4. Mark a low-trust child blocked or cancelled and observe that no
bounded sanitized parent notification preserves liveness before this
change.

### Paperclip version or commit

Reproduces on `master` before this PR, including base commit
`d36ea13e08`.

### Deployment mode

Local dev (`pnpm dev`).

### Installation method

Built from source (`pnpm dev` / `pnpm build`).

### Agent adapter(s) involved

Not adapter-specific (core authorization and issue-routing behavior).

### Database mode

External Postgres in the focused route regression suite; behavior is
database-mode independent.

### Access context

Agent (bearer API key associated with a checked-out heartbeat run).

### Additional context

The implementation deliberately distinguishes a direct-parent report
decision from general issue mutation permission and records successful
grants in the activity log.

### Privacy checklist

- [x] I have reviewed all pasted output for PII, API keys, tokens,
company names, and private instance references.

## What Changed

- Adds a distinct authorization decision for standard checked-out runs
commenting on their direct parent issue.
- Keeps low-trust direct-parent reports denied unless an existing
explicit mention grant applies.
- Forces direct-parent grants to remain comment-only even when a closed
parent is unassigned or assigned to the reporting agent.
- Audits successful direct-parent report grants in issue activity
details.
- Adds sanitized, parent-scoped, idempotent system comments and parent
wakeups for blocked or cancelled child stops.
- Extends the low-trust red-team route suite for allowed parent reports,
forbidden upward/sibling writes, closed-parent mutation suppression, and
non-laundering stop relays.

## Verification

- `pnpm exec vitest run
server/src/__tests__/low-trust-red-team-routes.test.ts` — 11 tests
passed after the review fix.
- `pnpm --filter @paperclipai/server typecheck` — passed after the
review fix.
- Confirmed the PR changes four files and excludes `pnpm-lock.yaml`,
workflow changes, migrations, and unrelated branch commits.

## Risks

- This is an authorization behavior change. An overly broad grant could
enable cross-boundary writes, while an overly narrow grant could
preserve the liveness failure.
- The implementation constrains the grant to a standard-trust
checked-out run, a direct parent target, and comments only; activity
auditing and red-team coverage make regressions observable.
- Stop relays intentionally contain only system-generated child
identity/status metadata and are deduplicated; child-authored prose is
not copied.
- SecurityEngineer approval is mandatory before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using GPT-5.5 with reasoning, repository tool use, shell
execution, and test execution. The runtime does not expose the
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 13:03:04 -05:00
Devin Foley 429792f1f3
fix(interactions): stop wedging confirmation accept on a terminal workspace_finalize (#10099)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - When an agent finishes work in an execution workspace, the board can
confirm the result through an issue-thread interaction (e.g. the
"Merged" / mark-done confirmation button on a `request_confirmation`).
> - That accept action is gated: it must not race a worktree sync-back
(`workspace_finalize`) that is still copying the agent's commits out of
the sandbox, or the board could act on a base that hasn't received them
yet.
> - The gate (`runWorkspaceIsFinalized`) treated the sync-back as
"settled" only when the latest `workspace_finalize` op was `succeeded` —
so a run whose finalize reached a terminal `failed` state, or died
leaving a stale `running` op, was treated as "still syncing" forever.
> - Users hit a permanent, misleading `... has not finished syncing its
workspace` error and could never click "Merged", even though nothing was
syncing and the run had long since ended.
> - This PR fixes the settle semantics so the gate blocks only while a
sync-back is genuinely pending or in flight, and treats any terminal (or
stale-orphaned) finalize as done.
> - The benefit is that a failed or abandoned sync-back no longer wedges
the human confirmation, while a genuinely in-flight sync-back on a live
run still blocks correctly.

## Linked Issues or Issue Description

No public GitHub issue exists for this. Describing the bug in-PR (bug
report):

**What happened**

Clicking the "Merged" / mark-done confirmation at the bottom of an issue
thread returns an error that the workspace "has not finished syncing its
workspace" — but nothing is actually syncing, and the run that created
the interaction has already ended. The confirmation is permanently
stuck; the only workaround is to merge and mark the task done manually.

**Expected behavior**

Once the source run's worktree sync-back has finished — whether it
succeeded, failed, or was skipped — the confirmation should be
acceptable. The gate should block only while a sync-back is genuinely
still running on a live run.

**Steps to reproduce**

Have an agent run reach `workspace_finalize` and end without a
`succeeded` finalize (e.g. the sync-back fails, or the run process dies
mid-finalize leaving a `running` op). Then attempt to accept the
`request_confirmation` interaction it created → 409 "... has not
finished syncing its workspace" with no way to proceed.

**Paperclip version or commit**

Reproduced on the current `master` line (server service); root cause is
in `runWorkspaceIsFinalized` in `server/src/services/issues.ts`.

**Deployment mode**

Local / self-hosted instance (server service).

**Root cause**

`runWorkspaceIsFinalized` returned `true` only when the latest
`workspace_finalize` operation was `succeeded`. A terminal `failed`
finalize (the sync-back ran and failed; it will not retry within that
run) and a `running` finalize left behind by a dead run both left the
gate closed forever.

## What Changed

- `runWorkspaceIsFinalized` (server/src/services/issues.ts) now treats a
sync-back as **settled** when the latest `workspace_finalize` op reached
any terminal status (`succeeded`, `failed`, or `skipped`), instead of
only `succeeded`.
- A `workspace_finalize` still marked `running` blocks only while its
owning run is alive; a `running` record left behind by a
terminal/missing run is treated as stale (settled), so a dead run can no
longer wedge the gate.
- Preserved existing behavior for the other cases: no operations
recorded at all → settled; earlier phases recorded but no
`workspace_finalize` yet → still blocks (the sync-back hasn't been
attempted).
- Extracted the run-liveness check into a shared exported helper
`heartbeatRunIsTerminalOrMissing` and reused it from the existing
`isTerminalOrMissingHeartbeatRun` closure (no behavior change there).
- Added a short comment at the confirmation-accept gate
(server/src/services/issue-thread-interactions.ts) documenting the
relaxed settle semantics.
- The dependency-readiness / blocker barrier
(`listPendingFinalizeBlockerIssueIds`) is deliberately left unchanged:
an automated dependent must not proceed onto a base that never received
a blocker's synced-back commits, so a failed finalize keeps that gate
closed. Only the human-driven confirmation accept is relaxed.
- Added regression tests for: failed finalize, stale `running` finalize
on a dead run, and a genuinely `running` finalize on a live run (must
still block).

## Verification

- `cd server && node_modules/.bin/vitest run
src/__tests__/issue-thread-interactions-service.test.ts -t "accept"` →
17 passed (includes the 3 new regression tests), 21 unrelated tests
skipped by the name filter.
- Manual reasoning walkthrough of `runWorkspaceIsFinalized` for each
op-history shape (no ops / earlier-phase-only / terminal finalize /
running-on-dead-run / running-on-live-run) confirms the intended
block-vs-settle outcome.

## Risks

- Low risk and narrowly scoped to the human confirmation-accept gate.
The only behavioral change is that a terminal (`failed`/`skipped`) or
stale-orphaned `running` finalize now settles the gate instead of
blocking forever.
- A genuinely in-flight sync-back on a live run still blocks (covered by
a regression test), so the accept cannot race commits that are actively
being synced back.
- The blocker/dependency barrier for automated dependents is unchanged,
so no dependent will be advanced onto a base missing a failed blocker's
commits.

## Model Used

- Provider/model: Claude (Anthropic), **Opus 4.8**, model ID
`claude-opus-4-8`, 1M context window.
- Capabilities used: extended thinking, tool use (repo inspection, local
test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-23 10:34:47 -07:00
Michael Nguyen a7186dce4b
fix(plugin-sdk): thread companyId through configChanged + fail-closed cross-tenant guard (LOOA-687) (#10096)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - First-party **plugins** run as isolated workers spawned by the host
`plugin-loader`, reading company-scoped config through a governed
`ctx.config.get(companyId)` channel.
> - The host→worker `configChanged` RPC carries `{ config, companyId }`,
but the SDK dispatch dropped the scope — `onConfigChanged(newConfig)`
was companyId-blind by design — so a **proactive** worker kept a single
worker-global config.
> - #10092 added a startup replay that fans out **every** stored
company's config through `configChanged`. With no deterministic
ordering, a plugin configured for more than one distinct company ends up
running as whichever DB row was delivered last.
> - That is a latent cross-tenant identity/secret confusion bug: one
company's bot token could be applied to another company's traffic.
> - This pull request threads `companyId` through `onConfigChanged` and
adds a fail-closed cross-tenant guard at the SDK layer, so a
single-tenant worker can never silently collapse to a second company's
config.
> - The benefit is that the config-delivery class is fixed at the SDK
boundary — before any genuinely multi-company proactive plugin ships —
without changing today's single-tenant behavior.

## Linked Issues or Issue Description

No public GitHub issue — describing in-PR (hardening / latent security):

**Latent cross-tenant config collapse.** The worker-side `configChanged`
dispatch forwarded only `config` and dropped `companyId`, so a proactive
plugin kept a single worker-global config. #10092's startup replay
delivers every configured company's config sequentially with no `ORDER
BY`, so a plugin with configs for more than one distinct company would
apply a nondeterministic last-write-wins global config (one tenant's
credential applied to another's traffic).

- Builds on and must merge after #10092.
- Not exploitable today: the only proactive consumer (the chat gateway)
has single-tenant config rows, so last-write-wins is a no-op. This is a
hardening pre-condition before any multi-company proactive plugin ships.

## What Changed

- **Thread scope through:** `onConfigChanged(newConfig, context)` with a
new exported `PluginConfigChangeContext { companyId }`. Backward
compatible — the second arg is optional; existing single-arg
implementations are unaffected.
- **Fail-closed cross-tenant guard** (`worker-rpc-host.ts`): a
single-tenant plugin that receives `configChanged` for a second,
distinct company with a *different* config is rejected with the new
`PLUGIN_RPC_ERROR_CODES.CROSS_TENANT_CONFIG` instead of silently
overwriting the applied tenant's config. Idempotent replays of the
*same* config under a different scope row remain allowed.
- **Opt-in `multiCompanyConfig: true`** on the plugin definition for
plugins that genuinely serve multiple companies from one worker (keying
per-company state on `context.companyId`); the guard is bypassed for
those.
- **Deterministic `ORDER BY companyId`** on `registry.listConfigs`, so
the startup replay binds a single-tenant worker to a stable company
across restarts.
- **Loader visibility:** a `CROSS_TENANT_CONFIG` rejection is logged at
`warn` (was best-effort `debug`) so the misconfiguration is surfaced.
- **Regression test**
(`packages/plugins/sdk/tests/worker-rpc-host.test.ts`): two distinct
companies delivered via the startup-replay path fail closed and stay
bound to the first company; an idempotent same-config replay under a
different scope row is allowed; a `multiCompanyConfig` plugin receives
per-company config with the correct `context.companyId`.

## Verification

- SDK `tsc --noEmit`: clean.
- SDK vitest `worker-rpc-host.test.ts`: 7/7 pass (incl. 3 new). The
two-distinct-company case **fails against pre-fix code** and passes
after the fix.
- #10092 embedded-postgres `plugin-config-startup-delivery.test.ts`: 3/3
pass (unaffected by the new `ORDER BY`).
- Full server `tsc --noEmit` against this SDK: clean.

## Risks

- **Low functional risk.** The second `onConfigChanged` arg is optional
and existing implementations are unchanged. Today's single-tenant
gateway keeps working — idempotent same-config replays are explicitly
allowed, so the go-live is preserved.
- **Behavioral shift on misconfig:** a genuinely multi-company plugin
that has NOT opted into `multiCompanyConfig` now fails closed
(`CROSS_TENANT_CONFIG`) rather than silently collapsing to one tenant.
This is the intended safer default; opt in with `multiCompanyConfig:
true` to serve multiple companies from one worker.
- **Not in scope (residual).** Per-company workers/connections for a
genuinely multi-company gateway increase resource use and are tracked
separately (ties into the #10092 fan-out/timeout follow-up). This PR
fixes the class and fails closed; it does not build multi-tenant
connection management.

## Model Used

Claude — Anthropic `claude-opus-4-8` (Opus 4.8), extended thinking, with
tool use / code execution via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes`
/ `Refs` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change and contains no internal
Paperclip ticket id — branch predates this rule; not renaming an open PR
mid-review
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes — no
doc surface; internal SDK/host behavior only
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: anicca <annica@Michaels-Mac-Studio.local>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-23 09:50:48 -07:00
Michael Nguyen 1ebf5254b6
fix(plugin-loader): deliver stored config to freshly-started plugin workers (#10092)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for work.
> - One capability is first-party **plugins** that run as isolated workers spawned by the host `plugin-loader`, reading company-scoped config through a governed `ctx.config.get(companyId)` channel.
> - A **proactive** plugin (e.g. a chat gateway that opens a Slack Socket Mode connection at startup) does its company work from `setup()`, where there is **no company-scoped invocation** — so `ctx.config.get()` is rejected with `company context is required`.
> - The worker swallows that error and falls back to its default (feature-off) config, so the plugin comes up **inert** even though correct config exists in the database.
> - This is a regression from #9557 ("governed access contracts"), which changed `plugin-loader.ts` `activatePlugin` from loading stored config into the worker bootstrap to `const config = {}`.
> - This pull request replays each configured company's stored config to the freshly-started worker over the **same `configChanged` host→worker path an operator config-save already uses**.
> - The benefit is that proactive plugins receive their config on worker start (both server boot and operator enable) without weakening the governed-access surface.

## Linked Issues or Issue Description

No public GitHub issue — describing in-PR (bug):

**Bug.** After a proactive plugin's worker spawns, it never receives its stored config. Governed access (`packages/plugins/sdk/src/host-client-factory.ts`) only resolves `config.get` inside a company-scoped invocation (event/action/tool, or explicit `params.companyId`). Proactive plugins operate from `setup()` where no such scope exists, so `config.get()` fails with `company context is required`, the worker falls back to defaults, and the feature stays disabled despite valid DB config.

- Regression introduced by #9557.
- Related follow-up (latent multi-company hardening): #10096.

## What Changed

- `plugin-registry.ts`: add read-only `listConfigs(pluginId)` returning all stored company config rows for a plugin (scoped `where eq(pluginConfig.pluginId, pluginId)`).
- `plugin-loader.ts`: after the worker starts in `activatePlugin`, replay each company's stored config through the existing `configChanged` host→worker RPC — one `{ config, companyId }` per row, the same payload shape as the operator config-save path in `routes/plugins.ts`. Best-effort and idempotent; covers both server-boot `loadAll` and operator enable.
- test: DB-backed `plugin-config-startup-delivery.test.ts` covering `registry.listConfigs` completeness and cross-plugin isolation.

## Verification

- `tsc --noEmit` on `@paperclipai/server` — clean.
- New `plugin-config-startup-delivery.test.ts` (embedded-postgres, 3 cases) — pass.
- Full PR CI green: typecheck, all server/e2e/serialized test shards, build, canary dry-run, verify, and the security scanners (Snyk, Socket, Superagent, Greptile).

## Risks

- **Low functional risk.** Adds an outbound host→worker push that mirrors the already-shipped operator-save path. A worker without an `onConfigChanged` handler (or momentarily unavailable) simply keeps the runtime `ctx.config.get(companyId)` model.
- **Startup fan-out.** One `configChanged` per configured company at activation (sequential, default RPC timeout). `plugin_config` rows are writable only by instance-admins, so fan-out size is operator-controlled — not a remote surface.
- **No secret-handling change.** `configJson` is delivered as-is, exactly as `config.get`/operator-save already deliver it. No new secret sink; catch-blocks log only ids + `err.message` at debug, never `configJson`.
- **Latent multi-company behavior (pre-existing, not introduced here).** The worker-side `configChanged` dispatch forwards only `config` (drops `companyId`), and `listConfigs` has no `ORDER BY`, so a plugin configured for **more than one** company would apply a nondeterministic last-write-wins global config. This is existing SDK behavior — operator-save already pushes into the same handler — and is **not reachable by the single-company consumer this fix targets**. Greptile flagged this shape (4/5). It is tracked and fixed as a separate, non-blocking hardening PR (#10096): thread `companyId` through `onConfigChanged`, deterministic ordering, bounded fan-out.

## Model Used

Claude — Anthropic `claude-opus-4-8` (Opus 4.8), extended thinking, with tool use / code execution via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context to this change
- [x] I have specified the model used (with version and capability details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked them above
- [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs)
- [ ] My branch name describes the change and contains no internal Paperclip ticket id — branch predates this rule; not renaming an open PR mid-review
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes — no doc surface; internal SDK/host behavior only
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — 4/5; two latent multi-company items triaged as non-blocking and fixed in follow-up #10096 (see Risks)
- [x] I will address all Greptile and reviewer comments before requesting merge — addressed: triaged as non-blocking follow-up in #10096

🤖 Generated with [Claude Code](https://claude.com/claude-code)


Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-07-23 09:34:42 -07:00
Michael Nguyen 5a00040c4c
feat(plugin-sdk): interactions/approvals respond + attachment read capabilities for the chat gateway (#10066)
Adds the remaining 5 plugin capabilities + 7 worker→host RPC methods (interactions read/respond, approvals read/respond, attachment read) needed by the Slack chat gateway plugin (v0.5.0) to pass manifest capability validation and load.

- Security review: PASS (LOOA-642) after the viewer-role privilege-escalation blocker (LOOA-648) was fixed on this branch (requireActiveHumanMember now rejects viewer on impersonation write-paths, matching assertCompanyAccess).
- CI: Build, Typecheck, all server suites (3/3 + serialized 4/4), workspaces, e2e shard 2/2, and all security scanners (Snyk/Socket/Superagent/Greptile/security-review) green.
- One e2e flake (signoff-policy 'non-participant cannot advance stage') is unrelated: it exercises execution-policy stage advancement (routes/issues.ts, untouched by this PR) and failed on a heartbeat_run_events FK race + 409 checkout conflict.

Unblocks LOOA-629 (Slack gateway go-live) and the interview-ask feature.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-07-22 21:30:34 -07:00
Devin Foley 0ef3b320c7
Harden `POST /plugins/install`: canonicalize `localPath` for all instances; bundled-only floor for managed instances (#10067)
**Builds on** #10058 — managed detection keys off the *presence* of the
`PAPERCLIP_MANAGED_CONFIG` env var that PR introduces, deliberately
never its parsed body.

**Summary.** Two layered hardenings of the plugin install route. (1) For
**all** instances: `localPath` installs previously skipped the
package-name validation entirely; the path is now null-byte-checked,
resolved absolute, `realpath`'d (collapsing `..` traversal and
symlinks), and required to be an existing directory before the loader
ever sees it. (2) For instances running under a managed hosting control
plane (detected by the *presence* of `PAPERCLIP_MANAGED_CONFIG` —
deliberately never its body, so a corrupted document cannot widen the
surface): registry/npm installs return 403, and `localPath` installs
must canonicalize to inside the bundled plugin catalog root
(`packages/plugins`) — a positive allowlist enforced in code at the
route, independent of any flag value. Self-hosted behavior is otherwise
unchanged.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The plugin system lets instance admins install plugins from a
registry or from a local filesystem path, and plugin installation is
code execution on the host
> - The `localPath` branch of `POST /plugins/install` skips the
validation applied to registry installs; the raw path reaches the plugin
loader without canonicalization
> - Separately, instances operated by a managed hosting control plane
must constrain installs to the bundled plugin catalog, because there the
host belongs to the operator, not the tenant
> - This pull request canonicalizes and validates `localPath` for all
instances, and adds a bundled-only install floor for managed instances
> - The benefit is a smaller install-route attack surface everywhere,
and a positive code-enforced allowlist where the operator owns the
machine

## Linked Issues or Issue Description

No public issue exists; `bug_report` template fields for the validation
gap this PR fixes:

- **What happened:** `POST /plugins/install` with `localPath` set
bypasses the package-name validation entirely; the un-canonicalized path
(relative segments, symlinks, no existence check) is handed straight to
the plugin loader.
- **Expected behavior:** path installs are validated like registry
installs — null-byte-checked, resolved absolute, `realpath`'d, and
required to be an existing directory before the loader sees them.
- **Steps to reproduce:** as an instance admin, call `POST
/plugins/install` with a `localPath` containing `..` traversal or a
symlink pointing outside any plugin directory; observe the loader
receives the raw path. Exploitability is bounded (the route already
requires instance admin), so this is hardening of an admin-only surface
rather than an open exploit.
- **Version:** current `master`.

The managed-instance bundled-only floor layered on top is new behavior
(motivation: on managed hosting, arbitrary plugin install is arbitrary
code execution on operator infrastructure), aligned with the in-progress
"Cloud deployments" milestone in `ROADMAP.md`.

## What Changed

- New `server/src/services/plugin-install-guard.ts` — three pure
primitives: managed detection (presence-based), path canonicalization
(null-byte check → absolute resolve → `realpath` → must be an existing
directory), and segment-based containment in the bundled plugin catalog
root.
- Route enforcement in `server/src/routes/plugins.ts`: npm/registry
installs return 403 on managed instances; `localPath` installs are
canonicalized on every instance and, on managed instances, must land
inside the bundled catalog root.
- The plugin loader now receives the canonical path instead of the raw
request string.

## Verification

- 15 guard unit tests
(`server/src/__tests__/plugin-install-guard.test.ts`): traversal,
symlink escape, null byte, file-vs-directory, string-prefix sibling
root.
- 13 route security tests
(`server/src/__tests__/plugin-install-route-security.test.ts`): 403
matrix on managed instances + self-hosted happy paths.
- 36 existing plugin route authz tests green
(`server/src/__tests__/plugin-routes-authz.test.ts`).
- Server `tsc --noEmit` clean.

```bash
cd server
pnpm vitest run src/__tests__/plugin-install-guard.test.ts src/__tests__/plugin-install-route-security.test.ts src/__tests__/plugin-routes-authz.test.ts
pnpm exec tsc --noEmit
```

## Risks

- Managed instances: npm/registry installs and out-of-catalog
`localPath` installs now return 403 — intended new behavior, enforced in
code rather than configuration.
- All instances: `localPath` installs that previously pointed at
nonexistent paths or non-directories now fail with 400 before reaching
the loader (previously the loader failed later, less safely). Symlinked
deployment layouts are handled by canonicalizing both sides of the
containment check.
- Self-hosted npm install path is unchanged. Low residual risk.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use;
independently peer-reviewed by a second AI agent before push.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 20:48:41 -07:00
Devin Foley c4cdcc4826
Generalize bundled plugin provisioning: `ensureBundledKubernetesPlugin` → `ensureBundledPlugins` (#10063)
**Builds on** #10058 — reads `plugins.autoInstall` from the parsed
managed-config contract #10058 introduces (the interim
`readManagedPluginAutoInstall` shim is retired at rebase).

**Summary.** Boot-time bundled-plugin provisioning becomes
catalog-driven. A new bundled-plugin catalog lists the sandbox providers
shipped in-tree (keys like `kubernetes`, `daytona` → plugin key + path
under the catalog root). Managed instances read `plugins.autoInstall`
from `PAPERCLIP_MANAGED_CONFIG`; unknown keys or paths escaping the
catalog root (symlinks resolved) **throw before listen** — a managed
instance refuses to start rather than boot half-provisioned.
Installation keeps today's mechanism: an in-process, fail-safe
`loader.installPlugin({ localPath })` under a system actor — no HTTP
route, no user, no role widening. Self-hosted boot is unchanged
(kubernetes bundle only, existing env override honored, install failures
still log-and-continue).

**Semantics.** A plugin already present in any non-uninstalled state is
skipped, so an operator-disabled plugin is never silently re-enabled;
managed mode reinstalls soft-uninstalled bundles (the control plane owns
provisioning); removal from the autoInstall list never auto-uninstalls.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox-provider plugins ship in-tree, but boot-time provisioning is
hard-coded to exactly one of them (Kubernetes) via a bespoke function
> - On managed hosting, tenant users have no install privileges, so any
bundled plugin that is not provisioned at boot is unusable
> - Widening install routes or granting roles to fix that would trade a
provisioning gap for a security regression
> - This pull request generalizes the existing boot installer into a
catalog-driven `ensureBundledPlugins`, fed by `plugins.autoInstall` from
`PAPERCLIP_MANAGED_CONFIG`
> - The benefit is that managed tenants get working bundled plugins out
of the box, through the same in-process, role-free mechanism the
codebase already trusts, while self-hosted boot is unchanged

## Linked Issues or Issue Description

No public issue exists; `feature_request` template fields:

- **Problem or motivation:** on managed instances tenant users cannot
install plugins (by design they never hold instance admin), so even
plugins shipped with the product are unusable; boot provisioning
currently knows only the Kubernetes bundle.
- **Proposed solution:** a bundled-plugin catalog plus
`ensureBundledPlugins(keys)` driven by the managed config; same
in-process `loader.installPlugin({ localPath })` under a system actor;
unknown keys or catalog-escaping paths fail startup; already-present
plugins are skipped so operator-disabled plugins are never silently
re-enabled.
- **Alternatives considered:** granting tenant users install privileges
(widens secrets/adapters/settings access to solve a one-button problem);
a separate non-admin install route for bundled plugins (new authz
surface; provisioning removes the need for any install action at all).
- **Roadmap alignment:** supports the in-progress "Cloud deployments"
milestone and builds on the shipped sandbox-provider milestone in
`ROADMAP.md`.

Refs #10058.

## What Changed

- New `server/src/services/bundled-plugins.ts`: the bundled-plugin
catalog, the fail-to-start resolver (`resolveBundledPluginInstalls`,
positive allowlist + catalog-root containment with symlinks resolved),
and the fail-safe installer (`ensureBundledPlugins`).
- `server/src/app.ts`: replaces the hard-coded
`ensureBundledKubernetesPlugin` boot hook with resolver + installer
wiring, with test hooks (`managedPluginAutoInstall`,
`bundledPluginCatalogRoot` options).
- `server/src/index.ts`: passes `plugins.autoInstall` from the single
fail-closed `PAPERCLIP_MANAGED_CONFIG` startup parse (#10058) into
`createApp`; absent env means self-hosted and changes nothing.

## Verification

- 24 new tests in `server/src/__tests__/bundled-plugins.test.ts`
(catalog resolution, containment incl. symlink and `..` escapes,
skip/reinstall matrix, self-hosted invariants, installer error paths) —
all green.
- 85 adjacent startup/plugin-route/auto-build/managed-config tests green
(`managed-config`, `instance-settings-managed-overlay`,
`plugin-install-autobuild`, `plugin-routes-authz`,
`server-startup-feedback-export`).
- Server `tsc --noEmit` clean.

```bash
cd server
npx vitest run src/__tests__/bundled-plugins.test.ts
npx vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts src/__tests__/plugin-install-autobuild.test.ts src/__tests__/plugin-routes-authz.test.ts src/__tests__/server-startup-feedback-export.test.ts
npx tsc --noEmit
```

## Risks

- Managed instances with a malformed or unknown `plugins.autoInstall`
entry now **refuse to start** (fail closed, by design) instead of
booting half-provisioned; harness misconfiguration surfaces as a precise
startup error.
- Self-hosted behavior is unchanged (kubernetes bundle only,
`PAPERCLIP_KUBERNETES_PLUGIN_PATH` honored without containment, install
failures log-and-continue), so the default deployment path carries low
risk.
- No uninstall path exists in this module; removal from the autoInstall
list can leave a previously provisioned plugin installed (intentional v1
semantics, documented in code).

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use;
independently peer-reviewed by a second AI agent before push.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 20:35:51 -07:00
Devin Foley 216d3d2680
Managed-instance config: fail-closed PAPERCLIP_MANAGED_CONFIG parsing and read-time settings overlay (#10058)
**Builds on.** #10055 — the `catalogVersion` this config document pins
is the feature-catalog artifact #10055 emits.

**Summary.** Instances operated by a managed hosting control plane can
now receive instance configuration through a single environment
variable, `PAPERCLIP_MANAGED_CONFIG` (versioned JSON: `mode`,
`catalogVersion`, `features`, `plugins.autoInstall`). When the variable
is absent the instance is self-hosted and nothing changes. When present,
parsing is strict and **fail-closed**: blank value, malformed JSON,
unknown feature key, a feature key this build's feature catalog does not
mark tier `managed`, missing required section, or unsupported version
refuses startup with a precise error — a typo that silently does nothing
is how a security control quietly fails. Managed feature values are
overlaid **at read time** inside the instance settings service (never
persisted), so a DB restore or manual row edit cannot resurrect a
disabled capability; responses expose per-key `managedKeys` metadata
(`managed: true`, `managedBy`) so clients can render locked state.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip runs both self-hosted and under managed hosting, where an
operator's control plane owns instance configuration
> - Today instance feature settings live only in the tenant database; a
hosting control plane has no way to enforce a configuration that
tenant-side writes or restores cannot undo
> - Managed configuration will carry security posture, so delivery must
be atomic and parsing must fail closed — a typo that silently does
nothing is how a security control quietly fails
> - This pull request adds strict parsing of one
`PAPERCLIP_MANAGED_CONFIG` env var and overlays its feature values at
read time inside the settings service, never persisting them
> - The benefit is a minimal, auditable managed-hosting contract: absent
var ⇒ self-hosted instances are byte-for-byte unchanged; present ⇒
deterministic, locked configuration surfaced to clients via per-key
managed metadata

## Linked Issues or Issue Description

Refs #966 — this PR delivers that issue's "managed config injection"
hook, via a strict env-var contract rather than the config-file path it
sketches; the issue's other hooks (identity header, health, usage
webhook, lifecycle, external secrets, IAM auth) are out of scope, so the
PR refs rather than closes it.

*Mechanism differs from #966's proposal, so the `feature_request` fields
are also filled in:*

- **Problem or motivation:** managed hosting deployments need to
centrally enable/disable instance features; DB-stored settings can be
edited, restored, or migrated back to permissive values, and nothing
marks a value as operator-enforced.
- **Proposed solution:** one versioned JSON env var; fail-closed parse
at startup; read-time overlay in the settings service (precedence:
managed value over stored value over schema default); `managedKeys`
metadata in settings responses so clients can render locked state.
- **Alternatives considered:** per-feature env vars (non-atomic across a
half-updated env set, unbounded env surface); seeding the DB at boot
(persisted values can be edited or restored over, and cannot express
"forced"); lenient warn-and-drop parsing (fails open — unacceptable for
a security-bearing control).
- **Roadmap alignment:** supports the in-progress "Cloud deployments"
milestone in `ROADMAP.md`.

## What Changed

- New `server/src/services/managed-config.ts` (pure parser over the env
record)
- Startup parse ordered before the first `instanceSettingsService`
construction in `server/src/index.ts`
- Read-time merge + `managedKeys` in the settings service
- Shared validator updates

## Verification

- 29 parser/overlay tests (fail-closed matrix incl. blank/whitespace
env, missing sections, catalog-tier mismatch, empty-section happy path):
`pnpm vitest run src/__tests__/managed-config.test.ts
src/__tests__/instance-settings-managed-overlay.test.ts` (from
`server/`)
- 40 existing settings route/service tests green: `pnpm vitest run
src/__tests__/instance-settings-routes.test.ts
src/__tests__/instance-settings-service.test.ts` (from `server/`)
- 15 shared validator tests: `pnpm vitest run
src/validators/instance.test.ts` (from `packages/shared/`)
- Server `tsc --noEmit` clean: `pnpm typecheck` (from `server/`)

## Risks

- Self-hosted instances (no `PAPERCLIP_MANAGED_CONFIG` set) are
byte-for-byte unchanged — the parser only runs when the variable is
present.
- For managed instances, a malformed document now refuses startup by
design (fail-closed). This is an intentional behavioral guarantee, not a
regression: the control plane owns the variable and a precise startup
error is the contract.
- Overlay values are never persisted, so no migration or data-shape
risk.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use;
independently peer-reviewed by a second AI agent before push.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 19:48:43 -07:00
Nicky Leach 9edde68373
feat(kubernetes): native file-sync lifecycle hooks over pod exec (#10053)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - AI agents run in sandboxed execution environments (Kubernetes pods,
Daytona workspaces, etc.) and need to sync files between the host and
those environments — for workspace setup, asset delivery, and output
retrieval
> - The existing sync path for Kubernetes uses a base64-over-exec chunk
loop: each ~4 MB chunk requires its own `execInPod` round-trip, so large
syncs balloon into many exec calls with corresponding overhead
> - `execInPod` supports piped stdin/stdout, meaning the full transfer
can be done as a single exec that streams a raw `tar` archive over the
data channel — one round-trip regardless of file size, with nothing
base64-encoded and nothing buffered whole in memory on either side
> - PR-1 (#10013, merged) added the
`onEnvironmentSyncIn`/`onEnvironmentSyncOut` opt-in hook API to the
sandbox provider interface and documented the protocol; PR-2 (#10028,
merged) implemented these hooks for the Daytona provider
> - This pull request implements the same two lifecycle hooks in the
Kubernetes sandbox provider, so workspace/asset file sync streams
through one `execInPod` per operation instead of the chunk loop
> - The benefit is significantly fewer exec round-trips for large syncs
and flat memory use on both host and pod, with security properties
preserved: atomic replace, secret-mode enforcement, path confinement,
TOCTOU-safe snapshot, and member-confinement on host-assembled archives
from sandbox-authored tar output

## Linked Issues or Issue Description

This is the third and final PR in a sequential series:
- Refs #10013 — PR-1: opt-in sync hook API + provider docs (merged)
- Refs #10028 — PR-2: native file-sync lifecycle hooks for Daytona
provider (merged)

**Feature:** Native single-exec file-sync lifecycle hooks for the
Kubernetes sandbox provider.

*Motivation:* The existing Kubernetes sync path encodes files as base64
and loops over `execInPod` one chunk at a time (~4 MB per exec). For
large workspaces or asset sets this is slow and resource-intensive. The
Kubernetes `execInPod` API supports piped stdin/stdout, enabling a
raw-`tar` streaming transfer that needs only one exec regardless of file
count or size and never buffers the whole payload in memory.

*Proposed solution:* Implement `onEnvironmentSyncIn` and
`onEnvironmentSyncOut` in the Kubernetes provider using a streaming
`execInPod` with a tar pipeline — for syncIn the host builds the archive
on disk and streams its raw bytes into the pod's stdin (`head -c
<exact-size> | tar -x`, no base64); for syncOut in-pod `tar` writes to
the exec's stdout and the host streams those bytes straight to a file.
Path confinement, atomic replace, secret-mode enforcement, TOCTOU
protection, and a streamed-bytes fail-closed guard are all enforced.

## What Changed

- **New `src/file-sync.ts`** in
`packages/plugins/sandbox-providers/kubernetes/` — `performSyncIn` and
`performSyncOut` over an injected pod-exec closure, keeping transfer
logic hermetically unit-testable
- **New `execInPodStreaming` in `src/pod-exec.ts`** — a streaming exec
primitive that binds a caller-supplied stdin readable and a stdout
writable to the exec WebSocket data channel, added alongside the
existing `execInPod` (which is unchanged). This lets a transfer stream
raw bytes to/from disk instead of buffering the payload as a single
string
- **Updated `src/plugin.ts`** — registers
`onEnvironmentSyncIn`/`onEnvironmentSyncOut`; resolves the `sandbox-cr`
pod exactly like `onEnvironmentExecute` and delegates; `job` backend
rejects file-sync calls explicitly (out of scope)
- **syncIn path:** host builds the tarball to a temp file → streams its
raw bytes over exec stdin, bounded in-pod by `head -c
<exact-archive-size> | tar -x` (no base64 anywhere) → extract into a
`/proc/self/fd`-pinned reserved `0700` staging dir → `chmod`-before-`mv
-f` atomic replace per file (directory mappings use
`followSymlinks`→`-h`)
- **syncOut path:** in-pod validate + realpath-snapshot each source
(closes the validation→copy TOCTOU window) → single-exec `tar -c`
streamed over exec stdout → host streams that stdout straight to a temp
file through a byte-counting transform → member-confined extraction of
the sandbox-authored archive
- **Security properties:** secret files land at requested mode with no
widened window; every interpolated path is shell-quoted and confined
lexically plus via in-pod `realpath`; the outbound stream is bounded by
a **streamed-bytes disk guard** (`MAX_SYNC_OUTPUT_BYTES`, 8 GiB default,
per-call overridable) that fails the transfer closed — writing no target
file — if an untrusted pod emits more bytes than allowed. Neither host
nor pod buffers the whole payload, so there is no in-memory size cap on
the transfer
- **No changes** to `execInPod`, `wrapCommandWithEnv`, or
`FastUploadInterceptor` (the `environmentExecute` path is untouched)
- **No dependency or lockfile changes**
- **New tests** in `test/unit/file-sync.test.ts` (atomic-replace, `0600`
secret mode, symlink preserve/deref, dir-mapping, exclude,
path-confinement rejection, streamed-output guard fail-closed) and
`test/unit/pod-exec.test.ts` (streaming stdin/stdout, caller-sink error
fail-closed), plus extended `test/unit/plugin.test.ts`

## Follow-up: Legacy Job-Lease Base64 Fallback Fix

Addresses the Greptile 4/5 blocking finding ("Handle existing job
leases", `server/src/services/environment-runtime.ts`).

Job leases provisioned before the `nativeFileSyncUnsupported` metadata
flag existed carry `backend: "job"` but no flag, so `supportsSync()`
treated them as native-capable and routed their sync to the pod-exec
hook — which the job backend rejects (it has no exec channel) instead of
using the byte-identical base64 fallback. The fix adds a
belt-and-suspenders gate on the persisted `backend === "job"` field
alongside the existing `nativeFileSyncUnsupported` flag check, so
pre-existing job leases continue syncing via the base64 fallback after
deployment. No behaviour change for `sandbox-cr` leases.

## Verification

- `pnpm --filter @paperclipai/sandbox-provider-kubernetes test` — 19
files / 182 tests green, including the existing `upload-interceptor` and
`pod-exec` suites
- `tsc --noEmit` in the kubernetes package — 0 errors
- The sync hooks are opt-in; existing `environmentExecute` behaviour is
unaffected and tested by the unchanged existing suites

## Risks

- **Opt-in only:** `onEnvironmentSyncIn`/`onEnvironmentSyncOut` are
registered conditionally; providers that do not register them fall back
to the existing chunk loop. No regression risk on the existing path.
- **Shell-injection surface:** all path interpolation uses
shell-quoting; paths are additionally confined lexically and via in-pod
`realpath` before use.
- **TOCTOU on syncOut:** the in-pod snapshot validates and records file
metadata before the tar call, closing the window between validation and
copy.
- **Archive member confinement:** host-side reassembly rejects any tar
member whose resolved path escapes the target directory, preventing a
malicious in-pod tar from writing outside the intended destination.
- **Untrusted-output volume:** an over-large outbound stream trips the
streamed-bytes disk guard and fails closed (no target written and the
temp sink is swept) rather than filling host disk or memory; the guard
bounds disk unconditionally and bounds memory insofar as WebSocket
write-backpressure holds.

## Model Used

Anthropic Claude Sonnet 4.6 (`claude-sonnet-4-6`) — produced by a
Claude-based AI agent using agentic tool use and multi-step code
generation. 200K context window, extended reasoning, code execution and
verification capabilities.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 18:52:47 -07:00
Michael Nguyen e55d702916
feat(plugin-sdk): human-attributed issue comments for chat gateway plugins (#10050)
Adds the `issue.comments.create_human_attributed` capability and `ctx.issues.createComment` `actorUserId` option, with host-side active-human-member verification. LOOA-627.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-07-22 15:54:46 -07:00
Nicky Leach 39b94d0e7a
fix(test): gate interaction-continuation retry barrier on terminal write (#10049) 2026-07-22 14:00:36 -07:00
Nicky Leach 8af70b9fae
fix(test): drain in-flight heartbeat runs before liveness teardown (#10040)
## Thinking Path

> - Paperclip runs AI agent heartbeats to manage work; each heartbeat
dispatches `executeRun` fire-and-forget, which is intentional for
concurrency
> - The server escalation test suite
(`heartbeat-issue-liveness-escalation.test.ts`) exercises
`reconcileIssueGraphLiveness`, which heals a resolved-dependency wake by
enqueuing an on-demand heartbeat run
> - `enqueueWakeup` → `startNextQueuedRunForAgent` dispatches the run
fire-and-forget (`void executeRun(...)`), so the background run outlives
the awaited reconcile call
> - The test's `afterEach` polled `heartbeat_runs.status` to wait for
idle, but that flips to `completed` while `executeRun`'s finally block
is still flushing events — the escaping `heartbeat_run_events` insert
could land between the events delete and the runs delete, tripping the
FK constraint
> - This PR fixes the race deterministically by tracking in-flight
`executeRun` promises and exposing
`heartbeatService.drainActiveRunExecutions()`, which the suite awaits
before clearing tables
> - The benefit is a permanently reliable escalation test suite with no
sleeps, no retry bumps, and no production behavior change

## Linked Issues or Issue Description

**What happened?**

The `heartbeat-issue-liveness-escalation.test.ts` suite intermittently
failed in CI with:
```
delete on table "heartbeat_runs" violates foreign key constraint
"heartbeat_run_events_run_id_heartbeat_runs_id_fk"
```

**Expected behavior**

`afterEach` cleanup should complete without FK violations.

**Steps to reproduce**

The race is timing-dependent but surfaces reliably when the teardown
window is artificially widened. `reconcileIssueGraphLiveness()` heals
resolved-dependency wakes by dispatching a heartbeat run fire-and-forget
(`void executeRun(...)`). The old `afterEach` polled
`heartbeat_runs.status` — but that flips to `completed` while
`executeRun`'s finally block still has pending `heartbeat_run_events`
row writes. The escaping insert can land between the events delete and
the runs delete.

**Paperclip version or commit**

Reproducible on current `master` (commit
`b57aa9950c707a024156c34b79326a82b2dcca31`)

## What Changed

- **`server/src/services/heartbeat.ts`** — tracks all in-flight
`executeRun` promises in a module-level `Set`; exposes
`heartbeatService(db).drainActiveRunExecutions()`, which loops until the
set drains (a completing run can enqueue the next queued run in its
finally, so a single `await` is not enough)
-
**`server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts`**
— replaces the poll-on-`heartbeat_runs.status` teardown with `await
heartbeatService(db).drainActiveRunExecutions()` before clearing tables;
removes the now-unnecessary `waitForHeartbeatRunToComplete` helper

## Verification

```bash
# Full file (22 tests)
npx vitest run server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts

# 12x stress loop (264 test-runs, 0 failures)
for i in $(seq 1 12); do
  npx vitest run server/src/server-suites/heartbeat-issue-liveness-escalation.test.ts || break
done

# Type check the changed files
npx tsc --noEmit
```

- 22/22 tests green locally
- 12/12 full-file loop iterations: 264 test-runs / 264 afterEach cycles,
0 failures
- Widened-teardown stress variant (failed deterministically before the
fix) now passes with the drain

## Risks

Low risk. The drain mechanism is additive — it only affects test
teardown and could also be wired into graceful shutdown. The
fire-and-forget dispatch in production is unchanged. The `Set`-based
tracking adds negligible overhead per run dispatch (insert on dispatch,
delete on completion).

## Model Used

- **Provider:** Anthropic
- **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`)
- **Context window:** 200K tokens
- **Mode:** Tool use, code execution, extended reasoning

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 12:37:09 -07:00
Nicky Leach b57aa9950c
fix(test): stop flaky server-suite afterAll hook timeouts (#10024)
## Thinking Path

> - Paperclip is an open-source AI agent management platform; its test
suite spans a `server` package that mounts real embedded Postgres
databases in `beforeAll`/`afterAll` hooks
> - The `server` package CI shard runs all ~93 suites serially
(`maxWorkers=1`) on a loaded CI host; each suite boots and tears down
its own embedded Postgres in hook callbacks
> - vitest's default `hookTimeout` is 10 seconds; under load, graceful
embedded-Postgres shutdown occasionally crosses that threshold
> - This produces intermittent `Error: Hook timed out in 10000ms`
failures in `afterAll` hooks — not test assertion failures — and the
suites pass on re-run, making them textbook flaky tests
> - Inspecting `embedded-postgres@18.1.0-beta.16` shows that `stop()`
takes no argument (no fast-shutdown mode), SIGINTs postgres (already
PostgreSQL "fast shutdown"), and resolves only on the child's `exit`
event with no internal time bound
> - Two targeted fixes: (1) raise `hookTimeout` and `teardownTimeout` to
30 s in `server/vitest.config.ts` — one config change that eliminates
the flake for all ~93 suites at once; (2) wrap `stop()` in a 5 s bounded
`Promise.race` in the test helper so a slow shutdown can never hang the
hook regardless of OS scheduling variance
> - This PR changes only test-infra and test-config; no production-code
behavior changes

## Linked Issues or Issue Description

No public GitHub issue exists for this flake. Inline bug description
(bug report template):

**What happened?**

The `General tests (server (N/3))` CI shards intermittently fail with
`Error: Hook timed out in 10000ms` in `afterAll` hooks and pass on
re-run. Every test assertion passes; only the teardown hook exceeds
vitest's default timeout.

**Expected behavior**

CI passes reliably. Teardown timeouts should not be a source of flake.

**Steps to reproduce**

Run the server test suite repeatedly on a loaded host or in CI with
`maxWorkers=1` — the shard occasionally crosses 10 s in `afterAll`
during embedded-Postgres shutdown.

**Paperclip version**

`master`, any build that includes `server/vitest.config.ts` without an
explicit `hookTimeout`.

**Deployment mode**

Self-hosted (CI).

## What Changed

- **`server/vitest.config.ts`** — added `hookTimeout: 30000` and
`teardownTimeout: 30000`. Removes flake across all ~93 server suites at
once. 30 s gives generous headroom over observed worst-case teardown
while still catching a genuinely hung hook.
- **`packages/db/src/test-embedded-postgres.ts`** — added
`stopEmbeddedPostgresBounded()`, a 5 s `Promise.race` wrapper around
`stop()`. Applied at all three call sites inside `cleanup()`. Data dir
is still removed unconditionally; errors are still swallowed; the
null-instance guard is preserved. Existing behavior unchanged except the
shutdown can no longer block indefinitely.

## Verification

- `tsc --noEmit` clean on `packages/db` (built against worktree-local
`shared`)
- `packages/db` `client.test.ts` passes 14/14 — boots embedded Postgres
and exercises the bounded teardown via `cleanup()` in `afterEach`
- Standalone bounded-race semantics verified: hang resolves at the 5 s
bound; late or immediate `stop()` rejection swallowed; no unhandled
rejection; null-instance path safe
- CI: all 3 server shards + split-verify lane (Async-Verification Gate)
expected green after this PR

```bash
# Reproduce the teardown test locally:
cd packages/db && npx vitest run src/client.test.ts

# Type-check packages/db:
npx tsc --noEmit -p packages/db/tsconfig.json
```

## Risks

Low risk. No product-code changes — test-infra and test-config only. The
vitest timeout increase is additive (raises the ceiling; never lowers
it). The bounded race wrapper preserves prior teardown behavior exactly:
data dir always removed, errors always swallowed, stop is still
attempted. A worst-case outcome is that a genuinely hung `stop()` now
surfaces as a test timeout at 30 s instead of 10 s — still caught, just
later.

## Model Used

- **Provider:** Anthropic
- **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`)
- **Context window:** 200 k tokens
- **Capabilities:** tool use, code execution, extended reasoning
- **Mode:** Paperclip agent heartbeat (autonomous execution with human
board oversight)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 11:51:30 -07:00
legacykeeperops e1e35881af
feat(cost-events): propagate issue.billing_code at heartbeat record time (#6821)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs report progress through the heartbeat service, which
writes the cost ledger (`cost_events`) as usage accrues
> - `cost_events` already has a `billing_code` column, but nothing
populates it — the heartbeat writes `issueId`/`projectId` and leaves
`billing_code` NULL
> - Issues carry a `billing_code`, so the attribution data sits one join
away but never reaches the ledger rows
> - Reporting therefore has to reconstruct attribution by joining back
to `issues` at query time, which reflects the issue's *current* billing
code rather than the one in effect when the cost was incurred
> - This pull request threads `billingCode` through
`resolveLedgerScopeForRun` so the heartbeat stamps it onto each
`cost_events` row at record time
> - The benefit is that attribution is captured at write time and stays
correct if an issue's billing code later changes

## Linked Issues or Issue Description

No existing public GitHub issue. Describing the problem in-PR:

**Problem.** `cost_events` has a `billing_code` column that is never
written. The heartbeat's cost-ledger insert records `issueId` and
`projectId` but not the billing code of the issue the run belongs to, so
every row lands with `billing_code` NULL.

**Impact.** Cost-per-billing-code reporting has to derive attribution by
joining `cost_events` back to `issues` at query time. That join returns
the issue's billing code *as of the query*, not as of when the cost was
incurred, so historical cost reports shift retroactively whenever an
issue is re-coded.

**Desired behaviour.** The billing code in effect at record time is
stored on the `cost_events` row itself.

**Related PRs.** #6820 — same change to the same file by the same
author, opened separately. These are duplicates; only one should land.

## What Changed

- `resolveLedgerScopeForRun` now selects `issues.billingCode` alongside
`id` and `projectId`.
- The scope object it returns gained a `billingCode` field, populated
with `issue?.billingCode ?? null`.
- The early-return path for runs with no issue in context returns
`billingCode: null`.
- The `costs.createEvent` call in `heartbeatService` passes
`billingCode: ledgerScope.billingCode` alongside `issueId`/`projectId`.

No schema migration: `cost_events.billing_code` already exists.

## Verification

**No automated test accompanies this change.** There is currently no
test asserting that a `cost_events` row carries the issue's billing code
when an issue is in scope, or `null` when there is not. A reviewer
should treat the checks below as manual verification only.

Manual verification against a running instance:

```sql
-- Non-NULL billing_code for recent runs on billed issues
SELECT billing_code, COUNT(*)
FROM cost_events
WHERE created_at > NOW() - INTERVAL '1 hour'
GROUP BY billing_code;

-- Cost attribution query this change is intended to enable
SELECT billing_code, SUM(cost_cents)
FROM cost_events
GROUP BY billing_code;
```

Expected: rows for runs attached to an issue with a billing code now
carry that code; runs with no issue in context remain NULL.

## Risks

Low risk in blast radius, with two things worth a reviewer's attention:

- **Behavioural shift for consumers.** `cost_events.billing_code` was
uniformly NULL and now starts arriving populated. Anything downstream
that groups, filters, or dedupes on that column will see new values and
new cardinality. Existing rows are not backfilled, so the column is
mixed NULL/non-NULL across the historical boundary.
- **No test coverage.** The null-fallback behaviour on both paths is
asserted only by reading the code, not by a test.
- **Migration safety:** not applicable — no schema change; the column
already exists.
- **Failure mode:** if `billingCode` were absent from the `issues`
selection the value would silently be `undefined` rather than erroring,
so the field is worth confirming in review.

## Model Used

**TODO (author):** this section is required and cannot be completed on
your behalf. Please state the provider and model name, the exact model
ID/version, and the reasoning/thinking mode used — or "None —
human-authored" if no AI model was involved. Per the template, the
"Generated with Claude Code" footer is not a substitute for this
section.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [ ] I have specified the model used (with version and capability
details) — **pending author input, see above**
- [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above — #6820 is a duplicate of this PR
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable — **no test added
for the new field**
- [x] I have updated relevant documentation to reflect my changes — not
applicable, no user-facing or documented behaviour changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — **`e2e` did not complete on
`5ca5fde` (Playwright install timed out at 30m and the run was
cancelled); all other checks pass**
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
**currently 4/5, sole finding being this description**
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---

<sub>This description was reformatted to
`.github/PULL_REQUEST_TEMPLATE.md` by the Paperclip PR triage bot. The
code was not modified. Checklist boxes reflect the PR's verifiable state
at commit `5ca5fde`; unchecked items are genuinely outstanding, not
oversights. The **Model Used** section requires input from the author.
The previous description's `LEG-` reference was removed as an internal,
instance-local identifier that the template prohibits.</sub>

---------

Co-authored-by: Lead Backend Engineer Agent <backend1@legacykeeper.io>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>
2026-07-22 13:05:28 -05:00
Nicky Leach b247bf7150
feat(runtime): opt-in sandbox file-sync lifecycle hooks (API + provider docs) (#10013)
## Thinking Path

> - Paperclip is an open-source AI-agent management platform; agents run
tasks inside sandboxed environments (Daytona, Kubernetes, E2B, etc.)
> - The control-plane ↔ sandbox file-transfer path flows through the
`environmentExecute` seam in `protocol.ts` — the only verb available to
plugins — which forces a base64-over-exec chunked loop for every file
move: workspace files, assets, Codex home sync
> - This transport is correct and safe, but it bypasses provider-native
bulk/streaming APIs (Daytona `uploadFiles`, K8s `FastUploadInterceptor`
/ volume mounts), leaving significant throughput on the table for large
workspaces
> - The right fix is an opt-in seam extension: providers with faster
native transfer declare two optional verbs; providers that do not opt in
stay on the existing fallback with zero code or behavior change required
> - This PR adds the first layer of that extension — two optional verbs
(`environmentSyncIn` / `environmentSyncOut`) in the plugin SDK, the
runtime plumbing to prefer the native path for the two clean
destroy-then-replace cases, and a doc for the contract
> - The core correctness invariant is byte-identical fallback: if no
provider opts in, execution is exactly what ships today;
`assertSyncOperationsConfined` enforces host-side path confinement for
providers that do opt in
> - No provider advertises the verbs yet → zero production behavior
change; future PRs wire up Daytona and K8s providers against this
contract

## Linked Issues or Issue Description

No public GitHub issue exists for this feature. Description follows the
`feature_request` issue template:

**Subsystem affected:**
packages/plugins — plugin system; packages/adapter-utils — adapter
runtime; server/ — EnvironmentRuntimeService

**Problem or motivation:**
Sandbox file transfers currently always use a base64-over-exec chunked
loop regardless of what the underlying provider supports. For workspaces
larger than a few MB this becomes the dominant wall-clock cost of every
sandbox run, and it bypasses bulk/stream APIs that providers like
Daytona already expose natively.

**Proposed solution:**
Add two optional, opt-in plugin hooks — `onEnvironmentSyncIn` /
`onEnvironmentSyncOut` — to the plugin SDK. When a provider defines both
hooks and both are advertised via the existing `supportedMethods`
negotiation, the runtime prefers the native path for the two clean
destroy-then-replace transfer cases; all other cases fall back to the
existing byte-identical base64 transport.

**Alternatives considered:**
An unconditional verb would require every provider to implement or stub
the verb. The opt-in / `METHOD_NOT_IMPLEMENTED` pattern (already used by
`environmentExecute`) preserves backward compatibility with zero
provider changes required.

**Roadmap alignment:**
Consistent with the  "Cloud / Sandbox agents" and  "Plugin system"
milestones; extends the plugin seam rather than adding
control-plane-level logic.

**Additional context:**
Searched open pull requests and issues for duplicate sandbox file-sync /
native-transfer work; none found.

## What Changed

- **`packages/plugins/sdk`**
- `protocol.ts`: two new optional `HostToWorkerMethods` —
`environmentSyncIn` / `environmentSyncOut` — plus generic
`SyncOperation`, `SyncFileMapping`, and `SyncOutcome` types
- `define-plugin.ts`: optional `onEnvironmentSyncIn` /
`onEnvironmentSyncOut` fields on `PluginDefinition`; worker advertises
each verb only when its hook is defined (else `METHOD_NOT_IMPLEMENTED`,
mirroring `environmentExecute`)
  - `worker-rpc-host.ts`: route new verbs to plugin hooks
  - `index.ts`: re-export new public types
- **`packages/adapter-utils`**
- `command-managed-runtime.ts`: expose optional `syncIn` / `syncOut` on
`CommandManagedRuntimeRunner` (available only when both verbs are
advertised); add `assertSyncOperationsConfined` host-side
path-confinement guard
- `sandbox-managed-runtime.ts`: `SandboxManagedRuntimeClient` gains
optional `syncIn` / `syncOut`; orchestrator prefers native path for
default-provision asset inbound and workspace-download-into-fresh-dir
outbound; all other paths keep the existing base64 fallback
- `sandbox-file-sync.test.ts` (new): 234-line characterization suite —
native-opt-in branch, fallback branch, `assertSyncOperationsConfined`
escape-path rejection, `followSymlinks` → tar `-h`
- `command-managed-runtime.test.ts`: negotiation + native-sync +
confinement tests
- **`server/src/services/environment-runtime.ts`**:
`EnvironmentRuntimeService` delegates to `syncIn` / `syncOut`, gated on
advertised support
- **`server/src/services/environment-execution-target.ts`**: minor
typing fix alongside the new verbs
- **`doc/plugins/SANDBOX_FILE_SYNC_HOOKS.md`** (new): documents the full
contract — opt-in / no-op guarantee, operation ordering,
provider-may-tar, atomicity, `followSymlinks`, secret modes (0600, no
window), path confinement, `operationId` opacity, resource bounds,
shell-quoting

## Verification

```bash
# SDK suite
pnpm --filter packages/plugins/sdk test

# Adapter-utils suite (includes new sandbox-file-sync characterization tests)
pnpm --filter packages/adapter-utils test
# Expected: 255 pass / 4 skip

# Type-check across affected packages
pnpm --filter packages/plugins/sdk typecheck
pnpm --filter packages/adapter-utils typecheck
# Server changed-file spot check:
cd server && npx tsc --noEmit --skipLibCheck 2>&1 | grep -E "environment-(runtime|execution-target)" | head -20
```

Key behavioral invariant to spot-check: with no provider opting in (the
current state), run any sandbox task and confirm file-transfer behavior
is byte-for-byte identical to what the pre-PR code produces. The
characterization tests assert this at the unit level.

## Risks

- **Zero production risk today**: no provider advertises
`environmentSyncIn` / `environmentSyncOut`, so the new code paths are
unreachable in production; all real traffic stays on the existing base64
fallback
- **Path confinement**: `assertSyncOperationsConfined` rejects any
`targetPath` that escapes the declared root — this is the primary
security boundary for future providers. The test suite covers
escape-path rejection
- **Atomicity**: the contract delegates atomicity to providers; the doc
explicitly calls out that directory-level ops are not guaranteed atomic
- **Secret transport**: credential assets (e.g., Codex `auth.json`,
directory mappings) continue to use the existing tar path — they do not
go through the new verbs in any current provider

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Provider: Anthropic  
Model: `claude-sonnet-4-6` (Claude Sonnet 4.6)  
Context window: 200 K tokens  
Capabilities: extended tool use, multi-file code generation, agentic
reasoning via the Paperclip agent framework

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 10:08:03 -07:00
Dotta d54ff52fc3
test(heartbeat): await execution drain before cleanup (#10023)
## Thinking Path

> - Paperclip is the open source control plane people use to manage AI
agents and their work
> - Heartbeat scheduling tests protect the orchestration rules that
serialize an agent's runs
> - The dependency scheduling suite waits for run rows to become
terminal before deleting shared database fixtures
> - A terminal row is persisted before asynchronous execution
finalization and successful-run handoff work fully drain
> - The test then clears process tracking and deletes heartbeat events
while finalization can still append another event
> - This pull request waits for each tracked run's execution promise to
drain before resetting mocks or deleting fixtures
> - The benefit is deterministic cleanup that preserves the production
lifecycle ordering and prevents release CI flakes

## Linked Issues or Issue Description

### What happened?

Release run `29936031931` failed in
`heartbeat-dependency-scheduling.test.ts` while deleting
`heartbeat_runs`. Asynchronous heartbeat finalization inserted a new
`heartbeat_run_events` row after the test had already deleted existing
events, causing the run-row delete to violate the event foreign key.

### Expected behavior

The serialized heartbeat test suite should finish all asynchronous run
execution work before destructive fixture cleanup.

### Steps to reproduce

1. Check out commit `2aef4641b48e88f5ce7e75ce69fbe3bf6bbfc60d`.
2. Run `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/heartbeat-dependency-scheduling.test.ts
--pool=forks --isolate` repeatedly with PostgreSQL test support enabled.
3. Observe that teardown can delete heartbeat events while execution
finalization is still able to append another event, causing a
foreign-key failure when heartbeat runs are deleted.

### Paperclip version or commit

`2aef4641b48e88f5ce7e75ce69fbe3bf6bbfc60d`

### Deployment mode

Other — GitHub Actions release verification.

### Installation method

Built from source with pnpm.

### Agent adapter(s) involved

Not adapter-specific (core heartbeat test lifecycle).

### Database mode

External PostgreSQL test database.

### Relevant logs or output

`delete from "heartbeat_runs"` failed because the run remained
referenced by `heartbeat_run_events_run_id_heartbeat_runs_id_fk`.

## What Changed

- Collect heartbeat run IDs after queued/running rows settle and await
`heartbeat.waitForRunExecutionDrain()` for each run.
- Reset the adapter mock and clear process tracking only after
asynchronous heartbeat finalization has completed.

## Verification

- Ran `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/heartbeat-dependency-scheduling.test.ts
--pool=forks --isolate` 10 consecutive times; all 10 runs passed with
6/6 tests.

## Risks

- Low risk: test-only cleanup ordering change using an existing
heartbeat service drain API. Production behavior is unchanged.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with exact model IDs `gpt-5.5` for this heartbeat and
`gpt-5.6-sol` for the recovered initial implementation run; tool-enabled
code inspection, GitHub diagnostics, and shell test execution. Runtime
context-window sizes were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 11:53:29 -05:00
Dotta 0b496c9c03
feat(secrets): add run-bound agent secret access (#9921)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies.
> - Agents already receive selected company secrets through `env.*`
bindings at run launch, but environment injection is ambient,
long-lived, and not suitable for every secret consumer.
> - The existing binding and secret-access-event models already provide
company-scoped authorization and per-resolution audit seams.
> - Agents need an explicit way to discover only the secrets granted to
them and fetch a value on demand without exposing the wider company
catalog.
> - That capability must remain run-bound, preserve low-trust token
carve-outs, and make every value read visible in both security and
operator audit trails.
> - This pull request adds an `access.*` delivery namespace, two
run-bound agent routes, dual audit logging, documentation, and an
operator grants editor.
> - The benefit is least-privilege, revocable, auditable secret access
while preserving existing env injection behavior.

## Linked Issues or Issue Description

No pre-existing public issue. Related work:

- Refs #9797 — existing in-sheet agent access UI that this PR extends to
distinguish env and API delivery.
- Refs #9918 — complementary searchable-agent picker improvement for the
same secrets sheet.
- Refs #9530 — related company-wide metadata catalog proposal; this PR
intentionally exposes only the authenticated run's granted aliases and
values.

**Problem / motivation:** Agents can currently consume secrets only
through process environment injection. This keeps values resident for
the run, does not support on-demand consumers, and cannot provide a
discrete operator-visible activity event for each agent-initiated read.

**Proposed solution:** Treat `company_secret_bindings` as the source of
truth for agent secret grants. Keep `env.KEY` as env delivery and add
`access.ALIAS` for API-only delivery; an env binding also implies read
access because the value is already present in the agent process. Add
run-bound list/fetch endpoints that derive scope from the authenticated
heartbeat run and never accept caller-selected overlays.

**Alternatives considered:** A company-wide agent-readable catalog was
rejected for this value path because it increases reconnaissance and
does not prove a per-secret grant. Reusing the ephemeral
environment-probe resolver was rejected because it lacks binding
enforcement. Approval-gated reads and user-scoped secrets remain
deferred beyond v1.

**Roadmap alignment:** This extends the completed **Secrets Manager with
per-agent access** roadmap capability from launch-time env injection to
explicit run-bound API delivery without duplicating a separate planned
initiative.

## What Changed

- Added `access.*` agent binding validation and a dedicated run-bound
resolver that combines `secrets:read` authorization with binding-context
enforcement.
- Added `GET /api/agents/me/secrets` for minimal granted metadata and
`POST /api/agents/me/secrets/:key/value` for on-demand value fetches
with `Cache-Control: no-store`.
- Preserved the existing denials for low-trust review agents,
task-bridge credentials, and skill-test tokens; standard long-lived
agent API keys cannot call the run-bound routes.
- Added dual audit behavior: value attempts write `secret_access_events`
and `activity_log` (`secret.value.read`), while metadata listing writes
the lighter `secret.access.listed` activity event.
- Kept env compatibility: `env.*` remains injected at launch and also
implies API read for the same bound agent; `access.*` never becomes an
environment variable.
- Added the agent-settings **Secret access** editor plus
delivery-mode/alias surfacing on the Secrets page, with focused UI tests
and tokenized layout styles.
- Updated OpenAPI, shared types, agent-facing skill documentation, and
API reference documentation.

### UI Screenshots

P3 produced and reviewed three screenshots using mock data; images are
intentionally not committed to the repository:

- `secret-access-editor.png` — agent settings grant editor.
- `secret-access-light.png` — Secrets-page delivery surfacing in light
mode.
- `secret-access-dark.png` — Secrets-page delivery surfacing in dark
mode.

The source attachments are retained with the implementation task and
linked in the internal handoff; the public page publisher was
unavailable in the PR-prep runtime.

## Verification

- `pnpm exec vitest run
server/src/__tests__/agent-secrets-routes.test.ts
server/src/__tests__/secrets-service.test.ts
server/src/__tests__/secrets-routes.test.ts
ui/src/lib/secret-delivery.test.ts
ui/src/components/AgentSecretAccessEditor.test.tsx` — 5 files, 122 tests
passed.
- Security follow-up: `pnpm exec vitest run
server/src/__tests__/agent-secrets-routes.test.ts
server/src/__tests__/secrets-service.test.ts` — 2 files, 73 tests passed
after active-run and version-consistency fixes.
- Final-head CI: all feature, typecheck, build, e2e, security, and
review gates pass; `General tests (server (1/3))` remains red after one
rerun because unrelated `heartbeat-retry-scheduling.test.ts` cleanup
deletes `heartbeat_runs` before referenced `activity_log` rows.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm --filter @paperclipai/ui typecheck` — passed.
- `pnpm check:token-gates` — feature-local arbitrary-value violations
fixed; command still reports five unchanged `#9627` literals outside
this PR.
- End-to-end QA passed all eight acceptance criteria: grant/list, fetch,
dual audit, env-implies-read, denial matrix, revocation, UI rendering,
and env-injection regression. Evidence:
https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455492
- Security review returned PASS-with-required-changes; the
implementation uses the required dedicated binding-enforcing resolver,
run-bound JWT restriction, run-derived overlays, minimal metadata, and a
resolver redaction-registration hook. Evidence:
https://github.com/paperclipai/paperclip/pull/9921#issuecomment-5027455382

## Risks

- A compromised agent can exfiltrate any secret explicitly granted to
it; explicit company-scoped/run-scoped grants, revocation, and audit
reduce but cannot remove that inherent capability risk.
- The resolver invokes a redaction-registration hook before returning
values, but the current route has no persistent cross-request per-run
redaction registry. Paperclip-owned later comments/events therefore
cannot yet guarantee automatic scrubbing of a deliberately copied
fetched value; QA classified this as non-blocking residual hardening.
- Audit-event insertion currently fails open if the security-event
insert itself fails; the operator activity event provides partial
redundancy, but a future hardening change should define fail-closed
behavior for value delivery.
- This PR overlaps `ui/src/pages/Secrets.tsx` with #9918 and may require
a straightforward rebase after that PR moves.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, `gpt-5.3-codex`, with reasoning, repository tool use,
terminal execution, Paperclip API access, and GitHub CLI capabilities.
Context-window size is not exposed by the runtime.
- Anthropic Claude Opus 4.8 with 1M context and tool use assisted with
the UI implementation commit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 08:04:39 -05:00
Dotta 5ed0b74b34
fix(runtime): scope PAPERCLIP_ env-binding strip to reserved keys (#9974)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs get their environment from user/adapter/project/routine
env bindings resolved by the server heartbeat, plus `PAPERCLIP_*`
runtime vars (identity, wake, workspace, API access) injected by the
harness
> - The heartbeat stripped **every** `PAPERCLIP_`-prefixed binding
before resolution, so legitimately user-named keys (e.g. cloud provider
token bindings like `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_*`) were
silently dropped and never reached the run env
> - At the same time, several adapters honored an explicitly configured
`PAPERCLIP_API_KEY` over the harness-minted run token, which is exactly
the one key config must never control
> - This pull request replaces the blanket prefix strip with a precise
three-rule policy: never accept `PAPERCLIP_API_KEY` from config, always
let harness-assigned runtime vars win, and let every other
`PAPERCLIP_*`-named user binding flow through
> - The benefit is that user secrets with a `PAPERCLIP_`-style name work
like any other binding, while runtime identity and API credentials stay
fully harness-controlled

## Linked Issues or Issue Description

**Bug description** (no public issue exists):

- **What happened:** Env bindings whose key starts with `PAPERCLIP_`
(e.g. a cloud provider token a user deliberately named
`PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_TOKEN`) were silently stripped by
the server before secret resolution, so the spawned agent never received
them. No error, no access event — the variable just never appeared.
- **Expected behavior:** A user-named `PAPERCLIP_*` binding should reach
the run env unless the harness itself uses that key. Only
`PAPERCLIP_API_KEY` should be categorically rejected, and
harness-assigned runtime vars (`PAPERCLIP_RUN_ID`, `PAPERCLIP_AGENT_ID`,
wake/workspace vars, …) should always win over config.
- **Steps to reproduce:** Configure an agent/project env binding named
`PAPERCLIP_<ANYTHING>` (plain or secret_ref), run a heartbeat, and
inspect the spawned process env — the key is absent.
- **Deployment mode:** local server, any local adapter.

Related prior PRs (different, save-time/API-layer blanket-ban approach;
this PR supersedes that direction with a runtime allow-except-reserved
policy): Refs #8239, Refs #8439.

## What Changed

- `server/src/services/heartbeat.ts`: the pre-resolution strip now
removes only `PAPERCLIP_API_KEY` (hard denylist) instead of every
`PAPERCLIP_`-prefixed binding; other `PAPERCLIP_*` keys flow into
binding resolution. Low-trust inline-sensitive-env checks now also cover
those keys.
- `packages/adapter-utils/src/server-utils.ts`: new
`isForbiddenConfigEnvKey()` helper; the shared
`refreshPaperclipWorkspaceEnvForExecution` merge drops
`PAPERCLIP_API_KEY` from config and keeps harness-assigned `PAPERCLIP_*`
keys authoritative.
- `packages/adapter-utils/src/acpx-engine/execute.ts`: removed the
explicit-`PAPERCLIP_API_KEY`-from-config allowance; the run token
(`authToken`) is now always applied; config `PAPERCLIP_API_KEY` is
ignored.
- All local adapters (`claude-local`, `codex-local`, `cursor-local`,
`gemini-local`, `grok-local`, `opencode-local`, `pi-local`) plus
`cursor-cloud`, `hermes`, and the server `process` adapter: removed
`hasExplicitApiKey`-style allowances so the harness token always wins,
and guarded the remaining unguarded env-merge loops (claude-local inline
loop, process adapter) with the same policy.
- Tests updated/added: heartbeat binding-strip test now asserts the
three-rule policy; adapter-utils merge tests assert the
`PAPERCLIP_API_KEY` ban and `PAPERCLIP_*` pass-through; acpx engine
tests moved credential fixtures to `authToken` and assert config
`PAPERCLIP_API_KEY` is ignored while other `PAPERCLIP_*` config keys
forward and still bust the session fingerprint on rotation.

## Verification

- `pnpm vitest run packages/adapter-utils/src/server-utils.test.ts
packages/adapter-utils/src/acpx-engine/execute.test.ts` — 127 passed
- `pnpm vitest run server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/heartbeat-local-environment.test.ts
server/src/__tests__/claude-local-execute.test.ts
server/src/__tests__/codex-local-execute.test.ts
server/src/__tests__/cursor-local-execute.test.ts
server/src/__tests__/gemini-local-execute.test.ts` — 68 passed
- Adapter package execute suites and the server tests touching API-key
fixtures (`heartbeat-run-log`, `redaction`,
`effective-run-config-fingerprints`, `agent-permissions-routes`) —
green. Three pre-existing sandbox/SSH fixture failures reproduce
identically on clean `master` on this host and are unrelated.
- `pnpm --filter <pkg> typecheck` for server, adapter-utils, and all
nine touched adapter packages — all pass.

## Risks

- Behavioral change: a deployment that relied on configuring a static
`PAPERCLIP_API_KEY` in adapter config env loses that override — by
design; the harness-minted run token is now the only source. When no run
token exists, no API key is injected at all.
- `PAPERCLIP_*`-named user bindings now reach binding resolution and run
envs; a key that collides with a harness runtime var is still discarded
at merge time, so runtime identity/wake/workspace vars cannot be
spoofed.
- Low risk otherwise: no migrations, no API surface changes.

## Model Used

- Claude Fable 5 (`claude-fable-5`, Anthropic Claude 5 family,
Mythos-class tier), extended thinking enabled, agentic tool use (file
edits, shell, test runner) via Claude Agent SDK.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 07:01:57 -05:00
Dotta cac3c0fa1a
feat(connections): add runtime subjects and grants (#9982)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Connections is the subsystem that lets operators connect external
apps and govern which subjects may use those credentials
> - #9958 established the v3 schema foundation and #9981 adds the
AppDefinition catalog layer
> - The runtime still needs subject-aware authorization state, scoped
key handling, and API/OpenAPI routes so connected apps can actually be
granted and used safely
> - This pull request adds the runtime grants/authorization behavior on
top of the catalog branch, while keeping unrelated dependency and
workflow sync commits out of the stack
> - The benefit is a reviewable runtime layer that can land after the
catalog PR, then unblock the wizard and orchestrator cutover work

## Linked Issues or Issue Description

Refs #9958 and #9981.
Refs #9981.

No public GitHub issue exists for this branch. This is the runtime layer
for the Connections v3 stack and is rebased onto `master` after #9981
landed.

## What Changed

- Adds the connection user authorization state migration and schema
wiring.
- Adds shared runtime subject/grant types and validators.
- Adds runtime grant and scoped key behavior in the tool-access service.
- Adds runtime route coverage and registers the routes in OpenAPI.
- Replays only the Connections runtime commits on top of the catalog
branch, dropping unrelated sync/dependency history from the prior closed
runtime PR.

## Verification

- `pnpm run preflight:workspace-links`
- `pnpm exec vitest run
packages/shared/src/validators/tool-access.test.ts
server/src/__tests__/tool-access-service.test.ts`

## Risks

- Medium: runtime grant enforcement is security-sensitive and must fail
closed for unknown key scopes.
- Migration ordering depends on the schema and catalog layers already
merged through #9958 and #9981.
- This PR is rebased and retargeted to `master` with runtime-only
commits.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 Codex coding agent with repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-21 16:17:02 -05:00
Dotta d23fbf8ae4
feat(connections): add AppDefinition Wave 1 catalog (#9981)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Connections is the subsystem that defines which external apps and
MCP-style integrations operators can browse, configure, and run
> - The v3 schema core in #9958 added stable connection identities, auth
metadata, and grant-aware contracts, but the app catalog still used the
older gallery shape
> - The product needs a richer, typed AppDefinition catalog so browsing
and setup can render provider-specific auth and configuration
requirements consistently
> - This pull request moves the Wave 1 app catalog onto generated
AppDefinition data and carries that shape through shared types, server
lookup paths, and app connection UI
> - The benefit is that follow-up runtime and wizard work can build
against one catalog contract instead of local-only mock/gallery data

## Linked Issues or Issue Description

Refs #9958.

No public GitHub issue exists for this branch. This is the catalog layer
for the Connections v3 stack after the schema-core foundation in #9958.

## What Changed

- Adds generated AppDefinition data for the Wave 1 catalog and ingestion
reporting.
- Replaces the legacy tool app gallery exports with
AppDefinition-centered shared contracts, validators, and tests.
- Updates server tool-access lookup behavior to use the AppDefinition
catalog.
- Updates app connection UI surfaces and tests to consume
AppDefinition-backed catalog data.
- Documents the catalog ingestion workflow in the connector playbook.

## Verification

- `pnpm run preflight:workspace-links`
- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts
packages/shared/src/app-definitions-url.test.ts
ui/src/pages/apps/AppsConnect.test.tsx
server/src/__tests__/tool-access-service.test.ts`

## Risks

- Medium: this changes the catalog contract used by shared, server, and
UI app connection surfaces.
- Catalog data quality matters because generated definitions now drive
browse/setup display.
- Follow-up runtime and wizard PRs must rebase on this branch or on
master after this lands.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 Codex coding agent with repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-21 15:57:12 -05:00
Dotta 7e00f67138
feat(connections): add v3 schema core (#9958)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies and their governed access to external systems.
> - Connected Apps build on the existing Apps and MCP gateway substrate
so companies can configure reusable, auditable integrations.
> - The current connection record does not yet have a stable public
address, explicit ownership/auth method fields, or subject-specific
credential grants.
> - Without that schema core, later OAuth, per-user authorization, token
brokering, triggers, and connector-service phases cannot enforce tenant
and subject boundaries consistently.
> - This pull request adds the forward-compatible Connections v3 schema
core while preserving the existing connection lifecycle and directly
migrating the remote MCP transport name.
> - The benefit is a company-scoped, least-privilege foundation for
one-click integrations without bypassing Paperclip secrets, profiles,
rules, or audit controls.

## Linked Issues or Issue Description

No matching public issue was found.

**Problem**

Paperclip's current app connections need a durable identity and
authorization substrate before Connected Apps can safely support
multiple setup methods, per-user credentials, provider tenants, and
managed connector services. The existing schema only models a single
connection-level credential set and uses legacy transport terminology.

**Proposed solution**

Add a stable company-scoped connection UID, explicit
ownership/auth/transport fields, a subject-aware `connection_grants`
table, and multi-key credential annotations. Backfill existing
connections and workspace grants in a reversible migration, then update
shared/server/UI contracts to the new `mcp_remote` transport name.

**Related work**

- Related foundation: #9534
- Roadmap: Connected Apps (one-click integrations)

## What Changed

- Added company-scoped connection `uid`, `ownership`, `authKind`, and
canonical transport fields across database, shared contracts,
validators, services, and UI fixtures.
- Added `connection_grants` with workspace/user subject rules, provider
tenant metadata, credential secret refs, revocation state, company
scoping, and uniqueness constraints.
- Added migration `0182_connections_v3_schema_core` to backfill stable
UIDs, rename `remote_http` to `mcp_remote`, infer auth kinds, create
default workspace grants, and support rollback coverage.
- Added multi-key credential annotations and updated gateway/access
services without changing the existing lifecycle behavior.
- Updated the connection glossary, connector playbook, and security
threat model for the new identity, grant, and relay boundaries.
- Added explicit test UIDs to direct database fixtures so the new
non-null invariant is exercised across affected server suites.

## Verification

- `pnpm --filter @paperclipai/shared typecheck`
- `pnpm --filter @paperclipai/db typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/ui typecheck`
- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
server/src/__tests__/tool-gateway-service.test.ts
server/src/__tests__/tool-gateway.test.ts
server/src/__tests__/heartbeat-runtime-skills.test.ts
server/src/__tests__/tool-oauth-legacy-backfill.test.ts
server/src/__tests__/tool-access-policy-service.test.ts
server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts
packages/db/src/connections-v3-schema-core-migration.test.ts
packages/shared/src/validators/tool-access.test.ts --config
vitest.config.ts` — 9 files, 218 tests passed.
- Latest-head GitHub Actions: build, typecheck, general/serialized
suites, backup/worktree restore coverage, both e2e shards, canary,
policy, and security scans pass.
- Greptile: 5/5 with zero unresolved threads.
- `pnpm check:token-gates` remains red only on five pre-existing `#9627`
color literals outside this change.

## Risks

- **Migration risk:** UID backfill and default-grant creation touch
every existing connection. The migration uses company-scoped uniqueness,
deterministic legacy UIDs with ID suffixes, and seeded up/rollback
coverage.
- **Authorization risk:** Grant rows carry credential references.
Constraints enforce workspace-vs-user subject shape, company/connection
lookup indexes, one default grant per connection, and one user grant per
connection/subject. Security review is requested specifically for this
design.
- **Compatibility risk:** `remote_http` is renamed directly to
`mcp_remote`; all repository call sites and fixtures are updated in the
same change.
- **Future-phase risk:** Subject-bound token issuance, triggers, and
connector-service relay verification remain fail-closed requirements
documented for later phases; this PR does not expose those capabilities.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex CLI coding agent. The runtime did not expose an exact
underlying model ID or context-window size; capabilities used include
repository inspection, code editing, shell execution, test execution,
Git/GitHub CLI operations, and structured reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-21 15:16:26 -05:00
Sam b565603a86
fix(server): accept Office issue attachments (#8562)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents and board users can attach files to issues so context and
deliverables stay with the task
> - Some clients upload Microsoft Office files with generic binary MIME
types such as `application/octet-stream`
> - Current `master` now accepts arbitrary issue attachment MIME types,
so the upload should keep working for unknown binary files
> - Office files still benefit from being stored with a specific Office
MIME type when the filename makes that inference safe
> - Shared attachment allow-list defaults should also include common
Office MIME types for routes that still use that allow-list
> - This pull request keeps the current arbitrary-MIME issue upload
behavior and only narrows generic binary uploads to Office MIME types
for known Office filename extensions

## Linked Issues or Issue Description

Fixes #8243

Duplicate search performed before implementation:

- No matching open or closed PR found for `8243`, `Office document`,
`attachment MIME`, or `openxmlformats`.

## What Changed

- Added common Office MIME types to the default shared attachment
allow-list.
- Added upload content-type normalization that maps generic binary
uploads to a specific Office MIME type only for known Office filename
extensions.
- Added an optional helper-level allow-list gate so callers that still
validate against an effective allow-list can keep generic binary uploads
generic when the inferred Office MIME type is not allowed.
- Reused the shared generic attachment content-type list for response
handling.
- Preserved current `master` behavior for issue uploads that use unknown
or arbitrary MIME types.
- Added regression coverage for default Office allow-list matching,
filename inference, optional allow-list fallback, official Office MIME
uploads, inferred generic Office uploads, and preservation of unknown
generic binary uploads.

## Verification

- `env CI=true corepack pnpm install --frozen-lockfile --force`
- `env CI=true corepack pnpm --filter @paperclipai/server exec vitest
run src/__tests__/attachment-types.test.ts
src/__tests__/issue-attachment-routes.test.ts`
- `env CI=true corepack pnpm --filter @paperclipai/plugin-sdk
ensure-build-deps`
- `env CI=true corepack pnpm --filter @paperclipai/server exec tsc
--noEmit`
- `git diff --check origin/master...HEAD`

GitHub CI, security checks, and Greptile pass on rebased head
`acc364cfbe3440a59db6570bb907818046649eb4`.

## Risks

Low risk. The issue attachment route continues to accept arbitrary MIME
types as current `master` does; this change only stores a more specific
Office MIME type for generic binary uploads when the filename has a
known Office extension. Unknown generic binary uploads remain generic.

For callers that use an allow-list before storing uploads,
`normalizeUploadAttachmentContentType` supports an optional gate so
inference can be limited to MIME types that are already allowed.

No docs change included because this is a default upload compatibility
fix covered by server tests.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

This is a narrow bug fix, not roadmap-level core feature work.
`ROADMAP.md` was checked.

## Model Used

OpenAI Codex using GPT-5, tool-enabled coding agent. Context window
details are not exposed in this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Sami Rusani <sr@samirusani>
2026-07-21 10:47:33 -07:00
Harshit Khemani 3e1dc90bf2
fix(execution-policy): final-stage approval terminates the policy instead of rewinding to stage 1 (#7893) (#7936)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Issues can carry an embedded multi-stage `executionPolicy` (e.g. QA
→ CodeReviewer → CodePusher) driven by
`applyIssueExecutionStageTransition` in
`server/src/services/issue-execution-policy.ts`
> - On approval, the next stage was picked with `nextPendingStage()`,
which scans the **whole** stage list from index 0 for the first id not
in `completedStageIds`
> - Stage ids are regenerated whenever the embedded policy is re-sent or
edited mid-flow (a supported operation — the existing "reassigns the
active stage when the current participant is removed" test depends on
it), so earlier `completedStageIds` can stop matching the current
policy; a final-stage approve then "finds" stage 1 pending again and
rebuilds a first-stage review (#7893) — an endless re-review loop that
can recycle indefinitely against a moving main tip
> - This pull request makes approvals advance with a forward-only scan
(only stages *after* the one being approved), so approving the last
stage always terminates the policy, and adds a guard so an
already-completed execution state is terminal for `status=done`
> - The benefit is final-stage approvals close the issue as the policy
intends, with no behavior change for non-final advancement or
reject/changes_requested verdicts

## Linked Issues or Issue Description

Fixes #7893

## What Changed

- `server/src/services/issue-execution-policy.ts`:
- New `nextPendingStageAfter(policy, completedStage, state)` helper —
forward-only scan from the approved stage's index; the approval path
uses it instead of `nextPendingStage()`. Approving the final stage
therefore always yields `nextStage === null` → completed state → the
caller's `done` flows through.
- New guard: `requestedStatus === "done"` with an already-`completed`
execution state returns without restarting the chain at stage 1 (closes
the same loop when a stale completed state lingers).
- Reject/`changes_requested` verdicts and intact-state forward
advancement are untouched.
- `server/src/__tests__/issue-execution-policy.test.ts`: 4 regression
tests, including one that reproduces the exact rewind (regenerated stage
ids + final-stage approve → previously reassigned QA at
`currentStageIndex 0`; now terminal completed) and an explicit
final-stage rejection test pinning the unchanged path.

## Verification

- `npx vitest run server/src/__tests__/issue-execution-policy.test.ts` →
54 passed (50 pre-existing + 4 new).
- `pnpm --filter @paperclipai/server typecheck` → clean.
- The rewind was confirmed empirically against unmodified code first (a
test asserting the buggy output passed pre-fix and flips post-fix), plus
brute-forced realistic operation sequences (checkout dances, status
round-trips, interim comments per the agent flow documented around
#4889) to verify intact-state flows are unaffected.
- Related suites (`issue-execution-policy-routes`,
`issue-comment-reopen-routes`, `issues-service`,
`issue-thread-interaction-routes`,
`issue-agent-mutation-ownership-routes`) also pass locally.

## Risks

- Behavior deliberately preserved: non-final approvals (forward scan is
identical when state is intact), rejections at any stage,
reopen-from-done (state cleared on reopen, fresh chain still starts at
stage 1), and explicit `in_review` restarts.
- The policy schema has no terminal-state field, so per the issue's Ask
the policy simply terminates and the requested `done` status flows
through.

## Model Used

- Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code, agentic
mode with tool use (subagent implementation + independent adversarial
review subagent), extended thinking enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (none found for #7893)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots (N/A — server-only change)
- [x] I have updated relevant documentation to reflect my changes (N/A —
internal stage-advance semantics)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (will confirm once CI runs on
this PR)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending first review)
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 10:36:26 -07:00
Stefano Maffeis 68ba7ccae6
Fail loudly on invalid config files (#9041)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server config loader reads `.paperclip/config.json` and feeds it
into the shared Paperclip config schema.
> - When a config file exists but cannot be parsed or fails schema
validation, Paperclip should not silently ignore it.
> - The current `readConfigFile()` catch block treats invalid files the
same as missing files, so startup falls back to defaults while the
banner can still point at the ignored config path.
> - This pull request keeps the missing-file fallback, but makes present
invalid config files fail with a path-specific error.
> - The benefit is safer startup behavior and a clear diagnostic that
points at the invalid config field.

## Linked Issues or Issue Description

Fixes #8908

## What Changed

- Changed `readConfigFile()` to return `null` only when the config file
is absent.
- Added explicit errors for unreadable/invalid JSON config files.
- Added explicit Zod validation errors that include the config path and
invalid field path without printing config contents.
- Added server tests for missing config, invalid JSON, schema validation
failure, and valid config parsing.

## Verification

- `pnpm exec vitest run server/src/__tests__/config-file.test.ts`
- `pnpm --filter @paperclipai/server typecheck`

## Risks

Low risk for valid configs and missing configs. This intentionally
changes behavior for present invalid config files from silent fallback
to startup failure, which is the issue being fixed.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-5, with repository file inspection, GitHub
CLI, and local command execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-21 10:18:45 -07:00
Harsh Kotak 1ba79d82a5
fix(server): preserve terminal status on issue release (#7524)
Fixes #4206

## Thinking Path

> - Paperclip orchestrates AI agents on issues with checkout/release
semantics for execution locks
> - `POST /api/issues/:id/release` clears checkout and execution locks
when a heartbeat ends without finishing the issue
> - `issues.release()` unconditionally set `status: "todo"`, undoing
terminal and waiting states (`done`, `cancelled`, `in_review`,
`blocked`) set during the session
> - Agents reported status drift after release (e.g. `in_review` →
`todo`, `done` → `todo`), forcing manual PATCH recovery and risking
silent stalls
> - This pull request gates the `todo` re-queue to `in_progress` issues
only and preserves all other statuses on release
> - The benefit is lock cleanup without destroying workflow state agents
already recorded

## Linked Issues or Issue Description

- Fixes #4206 — `issues.release()` must not downgrade terminal/waiting
statuses
- Related internal incident: AIT-114 status drift on terminal issue
release (AI Trading Council)

## What Changed

- `server/src/services/issues.ts` — `releaseStatus` is `todo` only when
`existing.status === "in_progress"`; otherwise preserves
`existing.status`
- `server/src/__tests__/issue-stale-execution-lock-routes.test.ts` —
regression tests: release preserves done, cancelled, in_review, blocked
keeps `done` and clears lock fields
- `server/package.json` — patch bump `0.3.1` → `0.3.2`
- `server/CHANGELOG.md` — documents the fix

## Verification

```sh
pnpm --filter @paperclipai/server test issue-stale-execution-lock-routes
```

- 7/7 tests pass (parametrized done, cancelled, in_review, blocked)
(includes new `preserves terminal status when releasing a done issue`
and existing `in_progress` → `todo` on release)
- CI: Build, Typecheck, serialized server suites, e2e, Canary Dry Run
green on latest head `f31b55f`

## Risks

Low risk. Behaviour change is intentional: non-`in_progress` releases no
longer force `todo`. Agents that relied on release to re-queue
`in_review`/`blocked` work must PATCH status explicitly (documented in
agent lifecycle guidance). Rollback: revert this commit and redeploy
`@paperclipai/server` 0.3.1.

## Model Used

Anthropic Claude Opus 4.6 (extended thinking mode) — 200K context
window, tool use enabled. Assisted implementation and PR packaging for
AI Trading Council upstream port from local hotfix.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots (N/A)
- [x] I have updated relevant documentation to reflect my changes
(CHANGELOG)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(re-review requested on head `f31b55f`)
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: brandon <brandonburr@gmail.com>
2026-07-21 10:07:59 -07:00
dependabot[bot] dc7f09be0d
build(deps-dev): bump vitest from 4.1.8 to 4.1.10 (#9886)
Bumps
[vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest)
from 4.1.8 to 4.1.10.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/vitest-dev/vitest/releases">vitest's
releases</a>.</em></p>
<blockquote>
<h2>v4.1.10</h2>
<h3>   🐞 Bug Fixes</h3>
<ul>
<li><strong>browser</strong>: Check fs access in builtin commands
[backport to v4]  -  by <a
href="https://github.com/hi-ogawa"><code>@​hi-ogawa</code></a>,
<strong>Hiroshi Ogawa</strong> and <strong>OpenCode
(claude-opus-4-8)</strong> in <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10680">vitest-dev/vitest#10680</a>
<a href="https://github.com/vitest-dev/vitest/commit/5c18dd267"><!-- raw
HTML omitted -->(5c18d)<!-- raw HTML omitted --></a></li>
<li><strong>vm</strong>: Fix external module resolve error with deps
optimizer query for encoded URI [backport to v4]  -  by <a
href="https://github.com/SveLil"><code>@​SveLil</code></a> and <a
href="https://github.com/hi-ogawa"><code>@​hi-ogawa</code></a> in <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10661">vitest-dev/vitest#10661</a>
<a href="https://github.com/vitest-dev/vitest/commit/bae52b511"><!-- raw
HTML omitted -->(bae52)<!-- raw HTML omitted --></a></li>
</ul>
<h5>    <a
href="https://github.com/vitest-dev/vitest/compare/v4.1.9...v4.1.10">View
changes on GitHub</a></h5>
<h2>v4.1.9</h2>
<h3>🐞 Bug Fixes</h3>
<ul>
<li>Fix <code>importOriginal</code> with optimizer and query import
[backport to v4] - by <strong>Hiroshi Ogawa</strong>, <strong>David
Harris</strong>, <strong>Codex</strong>and <strong>Vladimir</strong> in
<a
href="https://redirect.github.com/vitest-dev/vitest/issues/10546">vitest-dev/vitest#10546</a>
<a href="https://github.com/vitest-dev/vitest/commit/a5180190c"><!-- raw
HTML omitted -->(a5180)<!-- raw HTML omitted --></a></li>
<li><strong>browser</strong>:
<ul>
<li>Wait for orchestrator readiness before resolving browser sessions
[backport to v4] - by <strong>Vladimir</strong> and <strong>Séamus
O'Connor</strong> in <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10555">vitest-dev/vitest#10555</a>
<a href="https://github.com/vitest-dev/vitest/commit/7fb29651a"><!-- raw
HTML omitted -->(7fb29)<!-- raw HTML omitted --></a></li>
<li>Wait for iframe tester readiness before preparing [backport to v4] -
by <strong>Vladimir</strong> and <strong>Séamus O'Connor</strong> in <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10497">vitest-dev/vitest#10497</a>
and <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10556">vitest-dev/vitest#10556</a>
<a href="https://github.com/vitest-dev/vitest/commit/fbc626c40"><!-- raw
HTML omitted -->(fbc62)<!-- raw HTML omitted --></a></li>
</ul>
</li>
<li><strong>mocker</strong>:
<ul>
<li>Hoist vi.mock() for vite-plus/test imports [backport to v4] - by
<strong>Hiroshi Ogawa</strong>, <strong>LongYinan</strong>,
<strong>Claude Opus 4.8</strong> and <strong>Vladimir</strong> in <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10548">vitest-dev/vitest#10548</a>
<a href="https://github.com/vitest-dev/vitest/commit/2c9559c02"><!-- raw
HTML omitted -->(2c955)<!-- raw HTML omitted --></a></li>
</ul>
</li>
<li><strong>pool</strong>:
<ul>
<li>Prevent test run hang on worker crash [backport to v4] - by
<strong>Ari Perkkiö</strong> and <strong>Jattioui Ismail</strong> in <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10543">vitest-dev/vitest#10543</a>
and <a
href="https://redirect.github.com/vitest-dev/vitest/issues/10564">vitest-dev/vitest#10564</a>
<a href="https://github.com/vitest-dev/vitest/commit/934b0f587"><!-- raw
HTML omitted -->(934b0)<!-- raw HTML omitted --></a></li>
</ul>
</li>
</ul>
<h5><a
href="https://github.com/vitest-dev/vitest/compare/v4.1.8...v4.1.9">View
changes on GitHub</a></h5>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="db616d227b"><code>db616d2</code></a>
chore: release v4.1.10 (<a
href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/10718">#10718</a>)</li>
<li><a
href="bae52b5112"><code>bae52b5</code></a>
fix(vm): fix external module resolve error with deps optimizer query for
enco...</li>
<li><a
href="a7a61e78c7"><code>a7a61e7</code></a>
chore: release v4.1.9 (<a
href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/10598">#10598</a>)</li>
<li><a
href="934b0f587c"><code>934b0f5</code></a>
fix(pool): prevent test run hang on worker crash (<a
href="https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest/issues/10543">#10543</a>)
[backport to v4] (#...</li>
<li><a
href="7fb29651af"><code>7fb2965</code></a>
fix(browser): wait for orchestrator readiness before resolving browser
sessio...</li>
<li><a
href="a5180190c1"><code>a518019</code></a>
fix: fix <code>importOriginal</code> with optimizer and query import
[backport to v4] (#...</li>
<li>See full diff in <a
href="https://github.com/vitest-dev/vitest/commits/v4.1.10/packages/vitest">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=vitest&package-manager=npm_and_yarn&previous-version=4.1.8&new-version=4.1.10)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-21 12:04:11 -05:00