Commit Graph

116 Commits

Author SHA1 Message Date
Dotta b2acc674be
fix: enable isolated subscription login on authenticated self-hosted instances (#13344)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - AI Connections store reusable provider credentials with company and
owner boundaries.
> - Self-hosted instances can require board authentication while running
agents on the local server.
> - The login UI treated those instances as unsupported and displayed
instructions without a command.
> - Removing that restriction must not expose the server operator's
existing CLI account.
> - This pull request enables owner-scoped login attempts and reuses the
existing login UI.
> - Users can connect Codex and Claude subscriptions on an authenticated
self-hosted instance.

## Linked Issues or Issue Description

**What happened?**

On an authenticated self-hosted instance, OpenAI subscription setup
displayed terminal instructions with no command and a disabled Connect
button. Claude also could not complete the local connection flow.

**Expected behavior**

An authorized board user can prepare an isolated sign-in attempt, sign
in on the server, and save the verified account as a Connection. This
must not import another user's or the operator's ambient credentials.

**Steps to reproduce**

Run Paperclip in authenticated mode with a local environment. Open
Connections, choose OpenAI or Anthropic, and select Subscription. The
previous UI never enabled local login preparation.

Related: #13247, #13248, and #10751. This fix preserves the restriction
on remote access to the operator's ambient Claude login.

## What Changed

- Allow company-authorized users to create, check, cancel, and complete
their own isolated local login attempts.
- Keep ambient Claude credential import restricted to the local
operator.
- Support isolated Claude credential files without falling back to the
host account or mutating process-wide environment variables.
- Use Codex device authorization so sign-in does not depend on a browser
callback to the remote server's localhost.
- Gate server-host login on authenticated public deployments unless a
trusted runtime host is configured. Publish the capability through
health so setup shows supported alternatives.
- Read isolated Claude credential files through bounded,
descriptor-bound opens with ownership, permission, and symlink checks.
Try the alternate filename after malformed JSON.
- Reuse shared login instructions and lifecycle hooks in onboarding,
agent setup, and Connections. Show health-query failures explicitly.
- Document authenticated self-hosted behavior and add authorization,
isolation, lifecycle, and UI regression tests.

## Verification

- Passed 71 focused tests across connection routes, credential
isolation, legacy compatibility, the shared login hook, and agent setup.
- Passed 89 onboarding regression tests.
- Passed `pnpm -r typecheck`, `pnpm build`, Storybook build, and `pnpm
check:token-gates`.
- Completed real Codex device authorization and Claude browser
authorization on an authenticated Linux self-hosted instance. Both
accounts were detected automatically and saved as Connected. Both
completed attempt directories were removed.
- These live checks cover login, credential validation, and connection
creation. They do not establish a new model execution or long-running
refresh result.
- Review follow-up: 68 focused checks passed after rerunning one route
socket error; the full route/health rerun passed all 49 tests. The
70-test onboarding suite also passed. Final workspace typecheck,
production build, and Storybook build passed again.
- The broad local run exposed an instance-name assumption in two new
assertions. The fixture now uses an explicit non-default instance, and
all 32 connection tests passed with a different inherited instance name.
The superseded broad run was stopped; this is not a claim that the full
local suite completed. Full CI results will be recorded before merge.

## Risks

- The server must have the provider CLI installed. Users still run the
displayed command on the server that hosts Paperclip.
- Authorization checks must keep login attempts scoped to the company,
owner, provider, and reconnect target. Regression tests cover cross-user
and cross-company access.
- Existing local-trusted Claude behavior stays available. Authenticated
remote users cannot use its ambient import path.
- No database migration, dependency change, agent binding change, or
provider routing change is included.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code execution, and browser
tools. The runtime does not expose a more specific model version or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 23:06:55 +00:00
Dotta 47ded8bf97
feat: manage AI runtime credentials through Connections (#13247)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs need credentials for a specific provider and sign-in
method.
> - Connections already owns accounts, grants, and access permissions.
> - AI authentication should use those same boundaries.
> - This pull request adds the storage, API, adoption, and runtime
foundation.
> - Legacy agents keep their authentication until they explicitly adopt
a managed connection.

## Linked Issues or Issue Description

**Problem or motivation**
AI credentials are configured separately from Connections. Agents cannot
consistently reuse a responsible user's account or a permitted shared
account.

**Proposed solution**
Manage AI accounts with the existing Connections grants and permissions.
Keep model and harness selection independent from credential selection.
Preserve legacy authentication until validated adoption.

**Alternatives considered**
A separate credential registry would duplicate ownership and access
policy. Automatic fallback would risk using the wrong account.

**Roadmap alignment**
This extends the shipped Apps, multi-user, secrets, and agent-runtime
capabilities. The maintainer requested the feature and reviewed the UI.
Related groundwork: #11899 (connection permissions), #10910 (connection
wizard), #11692 (Claude subscription profiles), and #11854 (Codex
account rotation).

## What Changed

- Add AI-purpose/runtime-auth contracts and an additive, idempotent
migration.
- Add Claude, OpenAI, OpenRouter, and Grok provider capabilities and
catalog entries.
- Store credentials on grants. Resolve responsible-user defaults or
explicit permitted grants.
- Isolate managed credentials and provider sessions across accounts.
Block missing credentials without ambient fallback.
- Keep imported legacy secrets unchanged during reconnect. Use
independent local Codex/Grok sign-in attempts for rotating credentials.
- Add authorization, migration, concurrent refresh, retry, cancellation,
and legacy-compatibility tests.

This is part 1 of a two-PR stack. The app UI follows in #13248. Merge
the foundation first.

## Verification

- Updated against master `04e364236`, preserving upstream provider login
and connector workflows.
- Full workspace typecheck, production build, Storybook build, and token
gates passed on the integrated branch. Final local-login changes passed
59 focused tests; new-agent and inbox regression suites passed 63 tests.
- Browser checks verified automatic local Claude account detection,
resumable Codex login commands, retry, focus restoration, and
desktop/phone layouts. Commands create their isolated directory before
invoking the CLI.
- All current-head CI checks passed on `2a996560a`, including all
server/workspace tests, browser shards, runner verification, typecheck,
build, and canary dry run. Greptile reviewed that commit at 5/5 with no
unresolved threads. Earlier local full-suite attempts hit the Mac
PostgreSQL shared-memory limit; the complete suites passed in CI.
- Renumbered the additive AI migration to `0276` after upstream
migrations and regenerated its snapshot. Existing legacy agents retain
their configuration.
- Added local login status checks, owner-scoped retry, managed OpenCode
remote homes, credential-aware model discovery, and task
connection-repair delivery.

## Risks

- Managed credential failures intentionally block execution. They do not
restore legacy fallback.
- Preview-era copied Codex/Grok subscriptions require independent
reconnect.
- The integrated branch has live provider acceptance coverage. This
update verifies local Claude detection and Codex API-key task repair; it
does not add a new subscription authorization/refresh or Daytona stress
pass.
- Runtime-auth connections must stay excluded from tool and channel
handling.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, code
execution, and browser testing. The exact runtime model identifier and
context-window size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 16:30:10 -05:00
Dotta ab15aff390
feat: add experimental persistent agent chat (#13284)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Conversations must use the same tasks, controls, and execution
history.
> - Users need an ongoing chat with an agent without managing task
properties.
> - Agents should clarify and plan work, then hand execution to assigned
project tasks.
> - This pull request combines the reviewed Agent Chat stack for one
squash merge.
> - The benefit is persistent conversation with normal task governance
and shared UI.

## Linked Issues or Issue Description

**Subsystem affected**

Task lifecycle, agent runtime tools, shared task UI, and browser/paid
runner tests.

**Problem or motivation**

Users need one persistent conversation with each agent. A separate chat
store or renderer would duplicate task behavior and bypass existing
controls.

**Proposed solution**

Use a task-backed chat per company, user, and agent. Reuse the task
composer and transcript. Clarify and plan in chat, then create assigned
project tasks with the relevant plan. Keep Agent Chat behind its own
disabled-by-default experimental setting.

**Roadmap alignment**

This implements the task-backed direction in [CEO
Chat](https://github.com/paperclipai/paperclip/blob/master/ROADMAP.md#-ceo-chat).
Related proposals: #2504 and #9693. Related request: #7981. The
maintainer requested one squash merge of the complete stack.

Consolidates the reviewed runtime
[#13281](https://github.com/paperclipai/paperclip/pull/13281), backend
[#13282](https://github.com/paperclipai/paperclip/pull/13282), and UI
[#13283](https://github.com/paperclipai/paperclip/pull/13283) layers
with this PR's E2E coverage. All four layers passed CI and received
Greptile 5/5 before consolidation. This PR targets master and includes
the complete feature.

## What Changed

- Add personal canonical chat tasks with ordinary company visibility,
immutable identity, idempotent first sends, and an idle waiting state.
- Process `/new` in queue order. Preserve history, release a chat pause,
and fence old provider context and delayed writes.
- Keep chat lifecycle rules across recovery, finalization, assignment,
task lists, and rollups.
- Support research and plan revision in chat. Hand plans to ordinary
assigned project tasks before execution starts. Reject new chat
subtasks.
- Add repository-aware project creation and discovery tools, including
multiple repository IDs and GitHub URLs, authorization, idempotency, and
durable project-created cards.
- Reuse task UI components for chat, with starred/recent agent
navigation and a separate `enableAgentChat` experimental flag.
- Add deterministic browser tests and 24 paid chat cells across four
Codex/Claude profiles, with validated reports and screenshots.
- Integrate current master recovery, controller lease, queued-message,
and task UI changes. Gate chat interruption and deferred promotion on
ownership/feature policy. Guarantee lease renewal and active controls
are stopped even if teardown fails.
- Preserve master's migration 0273 and generate chat migration 0274 with
idempotent replay for development databases.

## Verification

- Prior exact heads of all four PRs passed Linux CI, including build,
typecheck, general/serialized tests, and browser E2E. Each had Greptile
5/5 and no unresolved findings.
- Integrated local verification passed: full repository typecheck and
production build, Storybook build, token gates, 340 focused UI tests,
all 20 deterministic chat browser tests, two migration replay tests, 88
focused chat/queue/native/controller tests, and provider/session
regressions including real lease expiry. These include the three
lifecycle regressions for the final admission/teardown fixes; server
typecheck also passes. Current head
`1268eda16cc2af892055917e7292f068820be135` has Greptile 5/5 with no
unresolved findings and passing security scans. All final-head CI gates
passed: build, full Runner verification, typecheck/release registry,
canary, all general/serialized test shards, and all browser E2E shards
([CI
run](https://github.com/paperclipai/paperclip/actions/runs/34696739927)).
Local PostgreSQL startup contention required serialized retries; skipped
fixtures do not count as passing coverage.
- The earlier paid campaign passed all 24 chat cells and retained 32
screenshots:
[report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-34648511170-1/index.html?report=agent-chat#suite-agent-chat).
It tested `abacbdfd2f660709ec37312cdb758284c8399d04`; it is prior
evidence, not a paid run of this integrated head.
- Manual check: enable Agent Chat in Experimental settings, open an
agent, clarify and revise a plan, then hand off to an assigned project
task. Stop a reply, send `/new`, and verify fresh context with retained
history. Disable the setting and verify agent shortcuts/new chat turns
are blocked.

## Risks

- Queue/session integration can affect retries and delayed writes. Tests
cover ownership, cancellation, reset boundaries, idle recovery, and
ordinary task behavior.
- Migration 0274 adds conversation fields and constraints. Replay is
idempotent and preserves existing development chat history.
- This combines the previously reviewed stack at the maintainer's
request. Agent Chat remains off by default and is separate from
Conference Room.

## Model Used

OpenAI Codex, GPT-6 Astra (`gpt-6-astra`), with reasoning, code
execution, browser tools, and parallel review. The exact context-window
size is not exposed in this session. Codex and Claude also ran as test
subjects in the linked paid campaign.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 08:56:04 -05:00
Dotta 2083bf6f9a
feat(connections): add AgentMail inboxes and email tasks (#13256)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connections give agents controlled access to external services.
> - Experimental channels already map conversations to tasks and durable
work queues.
> - Email needs inbox ownership, recipient envelopes, delivery records,
and explicit sends.
> - This pull request adds AgentMail to that infrastructure and keeps
the provider key in the server vault.
> - Agents can receive and send email from local or sandbox execution
while the board follows each conversation in its task.

## Linked Issues or Issue Description

**Problem or motivation**

Agents need dedicated email addresses. Incoming email should become
assigned work. Internal task comments and progress must never become
outgoing email by accident.

**Proposed solution**

Add experimental AgentMail connections, an inbox assignment wizard,
durable email intake and publication, task email cards, and
authenticated API, CLI, and native runtime actions. Agents use Paperclip
credentials to request sends. Paperclip owns the provider key and
enforces access and task authority.

**Alternatives considered**

A general mailbox MCP connector does not provide durable task binding or
publication boundaries. A separate mailbox application duplicates task
collaboration. The board instead directs the agent through the normal
task conversation.

**Roadmap alignment**

This extends the existing experimental connections and task
infrastructure. Product scope and interaction design were reviewed with
the maintainer. Related connection authority work: #11831 and #11818.
The duplicate search found no competing task-based AgentMail
integration.

## What Changed

- Add AgentMail catalog data, shared contracts, company-scoped email
records, and an additive migration.
- Add vaulted setup, inbox assignment, access grants, trust guidance,
and provider-side allowlist guidance.
- Support WebSocket and signed-webhook intake through a shared durable
pipeline, deduplication, catch-up, and task wakeups.
- Queue explicit new conversations and replies with immutable send
intents, idempotency, delivery state, and uncertain-send resolution.
- Show inbound and outbound email cards in normal task conversations.
Keep internal messages internal.
- Add task-scoped CLI actions and the sandbox callback routes required
for Daytona execution.
- Provide a dedicated AgentMail skill automatically only to agents with
active authorized inbox assignments. Keep email instructions out of the
universal Paperclip skill.
- Advertise connector-owned `agentmail_inboxes`,
`agentmail_read_thread`, `agentmail_send`, and `agentmail_delivery`
tools only in eligible native sessions. Recheck live authority on
execution.
- Isolate Codex CLI connector skills by agent and skill revision.
Deliver the assigned skill in the run prompt for adapters that use
shared skill directories, including resumed turns. Keep automatic skills
out of manual persistent sync. Show them as read-only and document the
pattern in the connector playbook.
- Fix AgentMail health checks that entered local-stdio validation and
optional missing Codex credential cleanup in sandboxes.
- Add API, pipeline, authorization, sandbox, browser, and Storybook
coverage.

## Verification

- Live AgentMail testing covered WebSocket intake, signed webhooks,
restart catch-up, and a full receive → task → Daytona Codex CLI →
explicit reply → Delivered round trip. The reply was verified in the
other inbox. The normal task composer also initiated an outgoing email
child task.
- The connector-skill change was verified in the browser: AgentMail
appears once as an automatic, read-only skill with its assigned address.
Disabling experimental chat connections removes it; re-enabling restores
it. A regression test covers assignment data arriving after library
data.
- Connector regression coverage passed 178 runtime utility, email
integration, skill-route, and heartbeat tests. All 17 Codex execution
tests passed, including per-agent skill isolation, model identity,
revision changes, removal, and prompt delivery without shared skill
files.
- After rebasing onto master, all 44 focused email, heartbeat, and
native-authority tests passed. All 313 native-session executor tests
passed. The UI regression suite passed all 3 tests. These test sets
overlap earlier focused runs.
- Full workspace typecheck and build passed after the rebase. Token
gates passed. Earlier focused Playwright task/setup coverage and the
Storybook build also passed.
- Native connector tool execution uses deterministic integration tests.
Live Daytona qualification used the Codex CLI adapter; the new
shared-home prompt fallback has deterministic coverage.
- The full repository suite is run by CI. The earlier unsharded local
full-suite attempt was stopped after the equivalent CI suites passed and
is not reported as a completed local run. Greptile reviewed
`7e57dc267a8446d3c906e3cc5b8abc94fb8860eb` at 5/5 with no unresolved
threads. All server, workspace, serialized server, and browser suites
passed in CI. The build job hit a five-second timeout in a runner
transport test; both variants and the full 80-test file passed locally
with unchanged timeouts. The build passed on retry on the same commit
without code or timeout changes. All required CI gates, including the
final `ci / verify` and `ci / e2e` summaries, are green on
`7e57dc267a8446d3c906e3cc5b8abc94fb8860eb`.

## Risks

- Email from external senders can start normal agent work. Setup
recommends a low-trust agent and AgentMail sender controls. Sender
addresses never grant board membership.
- Provider timeouts can leave uncertain sends. Retries retain their
idempotency key; expired windows require reconciliation or operator
resolution.
- Connector skills and native tools are assignment-dependent and require
current access. Revocation denies retained calls; assignment changes
select a new runtime context.
- Activation remains behind the experimental-channel setting. The native
runner path has deterministic coverage; live Daytona qualification used
the Codex CLI adapter.
- Schema changes are additive. Inbox ownership is unique across
companies. Disconnect preserves provider inboxes and task history.

## Model Used

OpenAI GPT-6 (Codex). Used reasoning, repository tools, code execution,
and browser testing. The exact deployment model ID and context-window
size were not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-11 16:56:38 -05:00
Michael Nguyen 60ee13a0f7
feat: allow operator UI snippets on Cloud instances (#13168)
Adds an optional Cloud-only HTML snippet so operators can load Plain’s
standard chat bubble. **6 files, 16 implementation lines added; 102
additions including tests and docs.**

## Thinking Path

> - Paperclip serves Cloud and self-hosted users.
> - Closed beta users need a way to report problems.
> - Plain provides a ready-made chat widget.
> - Cloud operators can load it through a generic deployment setting.
> - Self-hosted instances ignore that setting.

## Linked Issues or Issue Description

**Subsystem affected**

Server-served UI HTML.

**Problem or motivation**

Enable a chat bubble in Cloud without adding a support feature to the
React app.

**Proposed solution**

Insert trusted `PAPERCLIP_CLOUD_UI_SNIPPET` HTML before `</body>` when
the existing Cloud-managed predicate is true. The setting is off by
default. Related Cloud-gated integration: #12190.

## What Changed

Review the [final
diff](https://github.com/paperclipai/paperclip/pull/13168/files) in this
order:

1. `server/src/cloud-ui-snippet.ts`: the eight-line Cloud gate and HTML
insertion.
2. `server/src/static-index-html.ts` and `server/src/app.ts`: apply it
to static root/index, SPA routes, and Vite HTML.
3. Two test files and `doc/cloud-ui-snippet.md`: boundary checks and
setup instructions.

React UI, customer identity, and database behavior are unchanged. The
existing feedback flag remains. Plain chat is anonymous; no Paperclip
name, email, or organization is supplied.

## Verification

- **Greptile: 5/5**, no actionable findings, reviewed commit
`04bb44515`.
- **[CI
passed](https://github.com/paperclipai/paperclip/actions/runs/34535763243)**,
including build, typecheck, server tests, and end-to-end tests.
- Local: six focused tests, full typecheck, and build passed. The full
local suite has not produced a final result; CI is the completed full
verification.
- Staging deployment and live chat testing remain to be done.

### Staging setup

Set **one server environment variable**, `PAPERCLIP_CLOUD_UI_SNIPPET`,
to:

```html
<script>
(function(d) {
  var script = d.createElement('script');
  script.src = 'https://chat.cdn-plain.com/index.js';
  script.onload = function() {
    Plain.init({ appId: 'liveChatApp_01M26J213F6RR53YRARZVAFCZZ' });
  };
  d.head.appendChild(script);
})(document);
</script>
```

This is the public staging app ID. **No API key or signing secret is
needed.** Deploy to staging and restart the app with this setting. Test
`/`, `/index.html`, and an organization dashboard; send a message and
confirm a support reply returns. Production rollout is separate.

[Plain embed docs](https://www.plain.com/docs/product/channels/chat) ·
[Configuration and
rollback](04bb445151/doc/cloud-ui-snippet.md)

## Risks

Only trusted operators should set this value. The HTML is public and
scripts execute in the app origin; do not include secrets or
user-provided HTML. Plain owns the anonymous browser session, with no
Paperclip account-switch integration. To roll back, unset the variable,
restart, and refresh open tabs.

## Model Used

OpenAI Codex (GPT-6), with repository inspection and code execution.
Exact runtime model identifier and context size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-10 15:30:36 -07:00
Devin Foley daea92b647
feat(server): accept a Cloud control assertion on the task-drain endpoint (#13125)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server has a task-drain admission hold so operators can stop new
agent work and wait for quiescence before maintenance
> - Cloud deploys restart tenant containers, but the Cloud control plane
has no sanctioned credential for the drain routes, so agent runs are
killed mid-restart
> - The only Cloud credential this server trusts is the runtime identity
assertion, deliberately scoped to the one-time bootstrap health call
> - This pull request adds a disjoint, action-bound Cloud control
assertion accepted only on the task-drain endpoint
> - The benefit is that Cloud can hold new work and drain a stack before
it restarts the container, through the same authorization and audit
paths a human operator uses

## Linked Issues or Issue Description

Refs #12485 (the task-drain admission hold this makes reachable for the
Cloud control plane).

**Problem or motivation**

Cloud deploys restart the container without stopping agent work first.
The task-drain hold from #12485 exists for exactly this, but its routes
require instance-admin board authority. The Cloud control plane holds no
such credential: the runtime identity assertion is accepted only on `GET
/api/health`, by design. So in-flight runs die at every deploy.

**Proposed solution**

A second, deliberately disjoint use of the same Cloud signing key
(`PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS`): a control assertion with its
own JWS type (`paperclip-cloud-control+jwt`), its own audience, an
`action` claim, a request id, and a short maximum lifetime. A new
middleware accepts the `x-paperclip-cloud-control` header only on
`/api/instance/task-drain`, binds each method to one exact action
(`task-drain:read` / `task-drain:start` / `task-drain:stop`), verifies
the assertion against the configured JWKS and
`PAPERCLIP_CLOUD_STACK_ID`, and installs a synthetic instance-admin
board actor so the existing route authorization, validation,
transactional audit, and activity publishing run unchanged (audit rows
record actor id `paperclip-cloud`). The header is rejected with 400
anywhere else, so it can never become an ambient credential. The board
mutation guard exempts the new `cloud_control` source exactly like the
other non-browser lanes.

**Alternatives considered**

Widening the existing runtime identity middleware would conflate a
one-time bootstrap claim with a repeatable management credential and
weaken both. A per-stack minted instance-admin API key would work with
no auth change but adds a long-lived privileged credential per tenant to
store and rotate. The action-bound short-lived assertion keeps
authorization per-call and stateless.

**Additional context**

Self-hosted instances have no `PAPERCLIP_CLOUD_STACK_ID` and reject
every assertion — the feature is inert off Cloud. A runtime identity
token cannot replay as a control token or vice versa (disjoint `typ` and
`aud`, covered by tests). The Cloud-side caller (drain before deploy,
bounded quiescence wait) lands separately in the Cloud control plane.

## What Changed

- `server/src/services/cloud-runtime-identity.ts`:
`verifyCloudControlAssertion` plus the control
header/audience/type/action constants, reusing the existing JWKS
resolution, JWS parsing, and lifetime discipline.
- `server/src/middleware/cloud-control.ts` (new): accepts the header
only on the task-drain endpoint, per-method action binding, installs the
synthetic instance-admin actor on success, 401 on invalid assertions,
400 anywhere else.
- `server/src/app.ts`: mounts the middleware directly after the actor
middleware, so a valid assertion replaces whatever actor the request
otherwise resolved to.
- `server/src/middleware/board-mutation-guard.ts`: `cloud_control` joins
the non-browser exemptions.
- `server/src/types/express.d.ts`,
`server/src/services/authorization.ts`: `"cloud_control"` added to the
actor source unions.

## Verification

- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/cloud-control-task-drain.test.ts
server/src/__tests__/instance-settings-routes.test.ts
server/src/__tests__/heartbeat-task-drain.test.ts
server/src/__tests__/heartbeat-scheduling-suppression.test.ts
server/src/__tests__/cloud-runtime-identity.test.ts` — 87 tests, all
passing.
- `pnpm --filter @paperclipai/server exec tsc --noEmit` reports no new
errors against the base commit's known pre-existing set.
- The new suite covers: acceptance per method, cross-action rejection,
unknown-action rejection, runtime-identity-token replay rejection,
wrong-audience rejection, wrong-stack and self-hosted rejection, expiry
and oversized-lifetime rejection, unknown-key rejection, request id
validation, endpoint containment (400 elsewhere, 400 on unbound
methods), pass-through without the header, and the mutation-guard
exemption.

## Risks

Low risk, additive. No behavior changes without the header; the header
grants nothing outside the one endpoint; each assertion authorizes one
action for at most five minutes; the existing route-level validation,
queued transitions, and audit writes are unchanged. The browser-facing
Cloud proxy strips Cloud headers, and possession of the shared
tenant-session token cannot mint an assertion (signing key never leaves
Cloud).

## Model Used

Claude (Anthropic) — Fable 5 (`claude-fable-5`), extended thinking,
agentic tool use via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
(module doc comments carry the contract)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-10 11:56:28 -07:00
Dotta 889947c238
feat: add experimental native chat connectors (#13038)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - People also ask agents for work in their existing chat tools.
> - Each external conversation needs one task and a current authorized
source.
> - Retries, Stop, and provider failures must not duplicate work or
expose private data.
> - The first chat PR establishes the opt-in provider and data
contracts.
> - This PR adds experimental channel integration and its durable
control plane.
> - Users can request work from connected channels and inspect delivery
in Paperclip.

## Linked Issues or Issue Description

Refs #13100 and #13092. This is the second of exactly two chat PRs.
Foundation #13100 is merged and changed 143 files. Runner prerequisite
#13092 is also merged. This PR changes 400 files against master, below
the 500-file review limit. It contains no wireframe images or HTML
galleries.

## What Changed

- Add native Slack, GitHub, Microsoft Teams, Telegram, and Discord chat
connections. Keep chat disabled unless the operator enables experimental
chat connectors. Preserve the production GitHub tool connection and its
normal setup path.
- Bind each provider bot identity to one immutable Paperclip agent. Bind
each admitted external conversation to one task. Paperclip owns tasks,
runs, permissions, and audit records.
- Add durable admission, per-conversation queues, questions, task
controls, progress, final replies, images, files, and delivery receipts.
Board comments remain internal unless explicitly sent to the channel.
- Check current identity, provider reach, resource access, credentials,
runtime generation, and exact source before provider effects. Keep
private responses private. Never send raw reasoning, private logs,
credentials, or tool arguments.
- Hold uncertain sends for explicit audited resolution. Make Board
Send-to-channel atomic and idempotent. Keep reconnect and setup
credentials in Paperclip secret storage.
- Preserve current native-runner authority across retries, lost
acknowledgements, and recovery. Keep immutable input and completion
contracts separate from newer user input. Receipt reconciliation cannot
launch a provider.
- Reconcile chat close/new ordering and provider-effect lock order.
Audit resource access changes in the same transaction. Submit only the
selected resource from each UI toggle so stale pages cannot undo
unrelated access changes.
- Drain Codex stdout before certifying process exit. Bound the drain
with the existing shutdown grace. Preserve observed terminal authority
without treating an undrained process as successful or reusable.
- Incorporate master `018ca5da` with its ACP Stop, mobile task layout,
runner packaging, and official lock changes. Preserve dedicated
chat-answer continuations in both directions when ordinary queued
comments are adopted after Stop.
- Fence late adapter readiness behind an earlier Stop for the same run.
Preserve verified cleanup for registered adapters. Handle single Stop,
agent pause, duplicate Stops, and failure release without creating a
false cancellation receipt.
- Incorporate master's `6dd48cad4` wake-queue extraction. Preserve exact
failed-chat retry authorization and lineage, retired question-source
suppression, and the block on generic recovery that would discard the
admitted source. Fresh deferred input retains its separate promotion
path.
- Incorporate master `2a05b5ed3` and its queue-admission extraction,
simplified transaction ports, and separate runner CI job. Preserve exact
durable receipts, actor separation, and dedicated-answer isolation
through the new module. A failed receipt insert rolls back the
accompanying deferred-wake merge.

## Verification

Current head: `afe19299d06253cb628eb398e91d1200ea9f412a`, incorporating
master `2a05b5ed3457ea33efd6895520447d1d97fe98d8`. The conflicts are
resolved. This successor fixes two test-harness boundaries exposed by
CI: per-case route-module preparation and actual durable-save completion
before intentional runner termination. Production code and all existing
test/turn deadlines are unchanged. [Exact-head Greptile
review](https://github.com/paperclipai/paperclip/pull/13038#issuecomment-5587250594)
is **5/5**, completed September 10 at 13:20:55 UTC, with no actionable
findings or open review threads. [Fresh exact-head
CI](https://github.com/paperclipai/paperclip/actions/runs/34481724341)
passes **all 24 jobs**, including Build and both required aggregates.
Normal exact-head guarded merge was attempted and rejected by the
remaining branch approval policy: CODEOWNER review is required and no
human approval is present. Normal **squash auto-merge is enabled** as of
September 10 at 13:36:26 UTC. Requested CODEOWNERS have been notified;
no approval bypass or self-approval was used. Earlier-head results below
remain historical evidence, not qualification of this successor.

- Final exact-head Linux evidence: 995/995 chat integration cases; 36/36
agent-skills routes; 35/35 runner live-session cases, including real
process kill/resume; 1948 runner Vitest cases with three existing
benchmark/platform guards; 870/870 API-authority cases; and 104 browser
cases with four existing optional skips. Rust, conformance/replay, full
repository build, typecheck, canary, all server/workspace shards, and
both required aggregates pass with normal CI concurrency. Earlier failed
attempts remain recorded below.

- Latest test-only qualification: 141/141
route/permissions/authentication cases pass in separate cold forks, with
plain server types and independent review clear. The real-runner suite
passes 35/35, with plain runner types and independent review clear. A
controlled premature-save acknowledgement fails as expected; matching
ownership/effect/process evidence, rejected saves, real turn outcome,
test abort, and pre-kill liveness are covered. No local reproduction of
the original CI scheduling failure is claimed. The preceding [CI
run](https://github.com/paperclipai/paperclip/actions/runs/34479680858)
passes 21/24 jobs, including all 995 Linux chat cases and browser
aggregate (104 passed, four existing optional skips); only Build, the
skills serialized shard, and the required verification aggregate fail.
Its exact-head Greptile review was 5/5. Both failed job logs are
retained.

- Final fixture qualification: all eight focused Discord cases and all
995 chat integration cases pass. The exact modal statement/PID is
observed before taking the real connection lock; the test then proves
its actual blocking relationship before mutation. Original SQL
execution, provider behavior, negative assertions, and 1s/15s timeouts
remain unchanged. Independent review is clear and test/production hashes
remain frozen. The preceding [CI
attempt](https://github.com/paperclipai/paperclip/actions/runs/34477184777)
passed 22 jobs, including Build/runner, typecheck, canary, all other
test shards, and browser aggregate (104 passed, four existing optional
skips); the two fixture failures and failed verification aggregate
remain recorded, not relabeled as a pass.

- Current queue-module composition: 308/308 recovery/batching/queue/Stop
tests; 995/995 full chat integration; 89/89 module tests, including real
PostgreSQL receipt-insert rollback; 24/24 workflow/module-boundary
tests; plain server and UI types. All four actual local process/ACP
browser paths pass in 1.4 minutes. Fresh databases, no skips or retries,
stable reviewed source hashes. The initial boundary failure is retained;
its no-op service wrapper was removed without changing recovery context
or weakening the check. An exploratory standalone test-directory
typecheck fails because its new upstream transformation config is not a
standalone typechecking project; standard CI/build does not invoke it,
and no configuration was weakened to suppress those diagnostics.

- The preceding head `e02a63d462ce5d47433b0aeb632bb6fd20aab1ba` passed
[all 24 CI
jobs](https://github.com/paperclipai/paperclip/actions/runs/34436462958)
and exact-head Greptile review at 5/5. Required CODEOWNER review
prevented its normal merge before master advanced again.

- Final extracted-module composition: 307/307 recovery, batching, queue
and Stop-control tests; 995/995 full chat integration; 49/49 module
tests including eight PostgreSQL adapter cases; and 19/19 issue-update
tests. Plain server types pass. All four actual local process/ACP
browser paths pass in 1.3 minutes. Fresh databases, no skips or retries
in these cohorts, frozen source hashes, and independent review clear.

- The preceding head `3e4e1c1c` passes [all PR CI
jobs](https://github.com/paperclipai/paperclip/actions/runs/34415826820),
including Build and required `ci / verify` and `ci / e2e`. Both the
original Rust failure and the previously load-sensitive lineage fixture
pass with unchanged Linux concurrency. Master advanced afterward and
required this reconciliation.
- Final master composition: 448/448 focused UI tests, 186/186 adapter
tests, 24/24 queue/control tests, and 11/11 packaging tests. Plain UI,
server, shared, and adapter types pass. Token gates and diff checks
pass. Independent server and UI reviews are clear.
- Stop-registration regression: both real-service cases fail against
exact `a95` source and pass with the fix. The full corrected
recovery/control suite passes 265/265. Duplicate-owner and failed-Stop
controls also pass. Plain server types pass. The readiness barrier
prevents provider startup without adding an acknowledgment to an already
terminal run.
- Final qualification strengthens terminal-field equality and repeats
both affected cases successfully on a fresh database. All four actual
local process/ACP browser paths pass again in 1.3 minutes, without skips
or retries. The final screenshot shows Cancelled, a paused subtree,
retained input, and no error toast.
- Two new actual-service regressions fail before the merge fix. They
prove that queued-comment adoption could consume a dedicated chat answer
or add unrelated input to that answer. The fixed four-case cohort
passes, including ordinary upstream continuation and adapter Stop
controls. Full recovery passes 257/257. All four actual local
process/ACP Stop browser flows pass in 1.4 minutes, without skips or
retries, on a fresh database.
- The unchanged runner artifact was qualified with 171/171 transport
tests, 870/870 API-authority tests, conformance 1/1, and replay 11/11.
Six controlled reader tests prove the exit/drain repair. Its local
serial Rust workspace passed 546 top-level cases plus two invoked
helpers; the later passing Linux CI supplies default-concurrency
evidence.
- Prior exact-source full chat integration passes 995/995. Settings
regressions cover concurrent stale pages, 501 destinations, pending
state, rejected updates, and explicit retry. These deterministic tests
do not prove live provider behavior.
- Retained failed attempts and their causes are in the [qualification
log](afe19299d0/doc/plans/chat-adapters/2026-09-08-chat-queue-and-webhook-repair.md).
The first merge adapter run timed out while macOS slept for 290 seconds.
Its unchanged repeat passed with a temporary sleep guard. No assertion,
deadline, or CI gate was weakened.

Review commands include `pnpm --filter @paperclipai/server exec vitest
run src/__tests__/heartbeat-process-recovery.test.ts
src/__tests__/issue-queued-comments-routes.test.ts` and `pnpm exec
playwright test --config tests/e2e/playwright.config.ts
tests/e2e/acp-stop-continuation.spec.ts`. Database suites require fresh
disposable databases. See the [browser
runbook](afe19299d0/doc/plans/chat-adapters/2026-09-04-chat-adapters-browser-e2e-runbook.md)
for provider setup and separate live acceptance steps.

## Risks

- This remains experimental. Deterministic tests and bounded live
evidence do not establish every provider feature, tenant, permission
layout, or media shape. Teams work-tenant qualification is still open.
- Failed and uncertain provider effects remain visible and can require
operator action. A transport receipt does not prove recipient
visibility.
- Native controller and runner artifacts must remain compatible.
Preserve lease ownership, terminal authority, source binding, and
quarantine during future changes.
- Access and audit rows commit together, but activity notifications
remain best-effort. This is not a new durable event outbox.
- The PR operation does not deploy a live server, replace its runner, or
change provider permissions. Remaining live qualification is documented
in the [temporary
handoff](afe19299d0/doc/plans/chat-adapters/2026-09-08-open-qualification-followups.md).

## Model Used

OpenAI Codex assisted with implementation, tool execution, testing, and
review. The work records `gpt-6-astra` assistance. The environment does
not report a context-window size. No private reasoning traces are
included.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-10 10:06:45 -05:00
Dotta e200104727
feat: review connection actions from tasks (#13063)
Bring governed connection reviews into task history and composer approvals. Share resolution with Connections, add scoped remembered permissions, and resume agents through durable outcome receipts.

Keep cards compact, collapse raw results, isolate untrusted provider output, bound continuation payloads, and reconcile missed live events. Add Storybook coverage, browser journeys, and service regression tests.

Verification: all PR CI gates passed, Greptile 5/5, security scans passed, five connection-review browser journeys passed, and real native Codex approval/continuation was verified against the local MCP fixture.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-08 19:37:13 -05:00
Jannes Stubbemann 023e640a7e
fix(db): reap idle pool connections, name the pool, and end it on shutdown (#12956)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server keeps one postgres.js pool (`packages/db/src/client.ts`,
`createDb`) for every query it runs. #10795 made the pool tunable from
the environment, but the defaults stayed at the driver defaults: an idle
connection never closes, the pool reports itself as `postgres.js`, and
no code path ever calls `sql.end()`.
> - On a hosted Paperclip deployment the server entered a restart loop
(a bundled plugin failure that #12953 describes made every run fail, and
the pool saturated). Each generation opened its ten connections, died,
and left the backends open on the PostgreSQL side until TCP keepalive
reaped them hours later. After about 20 generations the backends
exceeded `max_connections`, and every later boot died on its first
bootstrap query with `sorry, too many clients already`, before
`server.listen()`. The loop could not heal itself. #9555 describes the
same shape on a launchd-supervised self-hosted install.
> - Three properties of the pool combine to make this possible: idle
connections are never reaped, the pool is never ended on any exit path,
and an operator cannot even find the leaked backends in
`pg_stat_activity` because they carry the generic driver name.
> - This pull request gives the pool a 60 second idle timeout and the
`paperclip` application name by default, exposes `max_lifetime` and
`application_name` through the same `DATABASE_*` environment contract
that #10795 introduced, and ends the pool on the orderly SIGINT/SIGTERM
path and on the fail-loud startup path.
> - The benefit is that a restarting or crash-looping server releases
its backends instead of accumulating them, and an operator can see and
count Paperclip's connections.

## Linked Issues or Issue Description

- Refs #9555 — database connection pool leak causes an infinite restart
loop under load. This PR closes the "pool never ends, idle connections
never close" part of that report.
- Refs #12953 — hosted outage report. The pool exhaustion is the second
half of that incident; the first half (a stuck sandbox provider plugin)
has its own PR.
- Related prior PRs: #9597 and #8780 both propose hard-coded
`idle_timeout` / `max_lifetime` values in `createDb`. Both predate
#10795 (merged), which made these options environment-driven; this PR
builds on the merged shape and adds the shutdown `end()` that neither
covers. #4006 and #7481 are closed earlier attempts in the same area.

## What Changed

- `packages/db/src/client.ts`
- New `resolveDatabaseClientOptions()` applies Paperclip defaults on top
of the environment: `idleTimeoutSeconds` defaults to 60
(`DEFAULT_DATABASE_IDLE_TIMEOUT_SECONDS`) and `applicationName` to
`paperclip` (`DEFAULT_DATABASE_APPLICATION_NAME`). `createDb` uses it
for both the environment path and explicit options.
- `DATABASE_IDLE_TIMEOUT_SECONDS` now accepts `0` to restore the driver
default (keep idle connections open). Negative or non-integer values
still throw.
- New environment variables: `DATABASE_MAX_LIFETIME_SECONDS` (positive
integer, maps to `max_lifetime`) and `DATABASE_APPLICATION_NAME`
(non-empty string, maps to `connection.application_name`).
  - `postgresJsOptions()` maps the two new options.
- `server/src/shutdown.ts`
- `finalizeServerShutdown` gains two optional ordered steps:
`closeHttpListener` runs first, before the application services stop;
`closeDatabase` runs after the application services and before the
embedded PostgreSQL stop. A failure in either is logged and does not
stop the teardown. Final order: listener → application services →
database pool → embedded PostgreSQL → instrumentation → Sentry.
- New `closeHttpListenerForShutdown()`: stops accepting requests, closes
idle keep-alive sockets, waits up to 5 s for open connections, then
closes whatever is left. Requests still in flight are drained while
every service is available, and none can reach a route after
`sql.end()`, on the signal path and the programmatic path alike (the
programmatic path's later `server.close` finds the listener closed and
skips).
- `server/src/app.ts`: the app shutdown hook (`shutdownAppServices`) now
stops the plugin job scheduler, whose tick queries the database, so a
programmatic `shutdown()` leaves no timer running against the ended
pool.
- `server/src/index.ts`
- `startServer()` is now a thin wrapper around the boot sequence. When
the boot sequence throws after the pool exists, the wrapper ends the
pool (and the separate migration pool, when configured) before it
rethrows. This covers the `process.exit(1)` path in the main module and
the CLI `paperclip run` path alike.
- The orderly shutdown passes the same `closeDatabaseClients` to
`finalizeServerShutdown`.
- `endDatabaseClient` tolerates a client without `$client` (test
doubles) and uses a 5 second end timeout.
- Docs: `docs/deploy/database.md` gets a "Connection Pool Settings"
table with every `DATABASE_*` pool variable, its default and its effect;
`doc/DATABASE.md` lists the two new variables.
- Tests
- `packages/db/src/client-options.test.ts`: parsing of the new
variables, `0` for the idle timeout, rejection of malformed values,
driver option mapping, and the `resolveDatabaseClientOptions` defaults.
- `packages/db/src/client.test.ts` (embedded PostgreSQL):
`createDb(url)` reports `application_name = paperclip` for its own
backend, and a pool with `idleTimeoutSeconds: 1` has zero backends in
`pg_stat_activity` after the timeout.
- `server/src/shutdown.test.ts`: the listener closes before the
application services, and the database close runs between the
application services and the embedded PostgreSQL stop; a failing
database close is logged while the teardown still finishes;
`closeHttpListenerForShutdown` closes idle sockets and resolves on
close, force-closes after the grace period, and is a no-op when the
listener was never bound.

## Verification

- `pnpm --filter @paperclipai/db typecheck` — passes (`check:migrations`
+ `tsc --noEmit`).
- `cd server && pnpm typecheck` — passes.
- `cd packages/db && pnpm exec vitest run src/client-options.test.ts
src/client.test.ts src/client-teardown-registry.test.ts` — 9 + 18 + 3
tests pass (the `client.test.ts` cases need embedded PostgreSQL; the new
one waits up to 10 s for the idle reap and passed in about 3 s).
- `cd server && pnpm exec vitest run src/shutdown.test.ts
src/__tests__/server-startup-feedback-export.test.ts
src/__tests__/bootstrap-claim-routes.test.ts` — 34 + 11 tests pass. The
startup-feedback suite exercises `startServer()` with a mocked
`createDb`, which is why `endDatabaseClient` tolerates a client without
`$client`.
- Manual check for a reviewer: start the server against any PostgreSQL,
then run `SELECT application_name, state, count(*) FROM pg_stat_activity
GROUP BY 1, 2;`. Paperclip's backends now show `paperclip`. Leave the
server idle for more than 60 s and the idle backends disappear. Send
SIGTERM and the backends close before the process exits.

## Risks

- Behavior change with no environment set: idle pooled connections now
close after 60 s. The next query after an idle period pays a reconnect
(single-digit milliseconds on a local socket). postgres.js reconnects
transparently. Set `DATABASE_IDLE_TIMEOUT_SECONDS=0` to keep the
previous behavior.
- `application_name` changes from `postgres.js` to `paperclip`. Anything
that filtered `pg_stat_activity` on the old name would need an update;
nothing in this repo does.
- The HTTP listener now closes at the start of the final teardown (after
the heartbeat run drain, which still needs the API for running agents).
The pool close runs after the application services. A late query from a
timer that survived the service shutdown would fail with a driver
"connection ended" error instead of running; the known database-backed
timer (the plugin job scheduler) is now stopped in the service shutdown.
- The listener drain adds at most 5 s to a shutdown while long-lived
connections (for example WebSocket clients) are open; after that they
are closed forcibly.
- `startServer()` is split into a wrapper and the boot sequence. The
exported signature and return type are unchanged.
- No migration, no schema change.

## Model Used

- Claude Fable 5.1 (`claude-fable-5-1`) via Claude Code, extended
thinking, tool use (file edits, shell, test runs). The change was
produced with the model and reviewed by the submitting human.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_014t3bi2beVNVVHAxK36dmXm

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-08 08:56:39 -07:00
Dotta 5bddff0920
feat(runner): add guarded API search and call fallback (#13003)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The new runner gives agents dedicated tools for common tasks.
> - Some API operations and parameters have no dedicated tool.
> - Agents need a controlled way to find and use those operations.
> - This pull request adds API search and calls through the real server
routes.
> - Existing tools remain the preferred path. The new tools are disabled
by default.
> - Paired tests measure correctness, tool choice, cost and time.

## Linked Issues or Issue Description

**Subsystem affected**

Paperclip Runner contracts, production tool authority and the server API
catalog.

**Problem or motivation**

The runner cannot use much of the API described by the old Paperclip
skill. A generic HTTP client would also let agents bypass runner control
rules.

**Proposed solution**

Add `search_api` and `call_api`. Resolve calls from the mounted API
catalog. Use server-held, run-bound credentials. Preserve route checks
and runner lifecycle rules. Keep the tools disabled until an operator
enables selected companies.

**Alternatives considered**

A dedicated tool for every endpoint would add a large initial prompt. An
unrestricted HTTP tool would weaken authorization and replay controls.

**Roadmap alignment**

This extends the native runner tooling. The repository owner requested
this design and implementation. The roadmap and related open PRs were
checked. No duplicate API escape-hatch PR was found.

## What Changed

- Register two compact fallback tools in canonical contracts and
provider projections.
- Build deterministic API discovery from OpenAPI, mounted experimental
routes and the old skill reference.
- Execute bounded JSON, text, file and download requests through
authenticated HTTP routes.
- Recheck active runs, company access and work modes. Block runner
lifecycle, scheduling, credential and approval bypasses. Keep routine
annotation collaboration available.
- Retain mutation receipts. Report uncertain outcomes without blindly
repeating writes.
- Add a company rollout gate and a durable eval worker with complete
cost accounting checks.
- Record child-task creation in the activity log with the agent and run.
- Add contract, authorization, file, replay and real runnerd/PRP/HTTP
tests.
- Document rollout gates and paid coverage limits. The companion eval
repository retains immutable attempts and reports.

## Verification

- Final app commit `da58370524c3626a744eec20164397c5fb6ba9ef`: all 32
checks passed; the unrelated Storybook visual check was skipped.
Greptile 5/5; no unresolved review threads.

- Full Linux build and recursive typecheck passed. Repository tests were
run by project and serialized shard; all 143 serialized server suites
passed.
- Runner TypeScript: 1,599 passed, two skipped. Rust release: 451
passing test reports. Conformance and replay parity passed. The required
API check passed 837 tests, including runnerd → PRP → authority → real
HTTP.
- Bindings cannot enable API tools without the explicit deployment flag.
Unit and real-authority tests prove the default-off boundary.
- The standalone API check builds and stages its own binary. It passed
after existing staged and debug binaries were removed from the test
container.
- UI and CLI tests passed. Initial environment failures (missing jq,
Docker overlay file identity, and parallel linker memory pressure) and
focused passing reruns are retained. The macOS full runner suite has
platform-specific failures; Linux is the qualified full-check platform.
- Eval harness: 27 tests passed; existing CI discovery ran 86 tests with
two unrelated skips. Credential export rejection is tested against the
actual report command.
- Luna and OpenRouter Sonnet each passed 60 common-workflow runs: ten
workflows, three repetitions per arm, zero unnecessary API fallback.
- Sonnet passed 11 selected capability/contract cases after fixes.
Gemini passed three smoke cases. DeepSeek exceeded the 120-second limit
and remains unqualified.
- Luna's two cost flags received focused follow-up. The original flags
and a later n=1 latency flag remain visible. Sonnet had no cost or
latency increase above 20%.
- The catalog contains 785 entries; 58 were exercised across all stages.
Most operation probes remain unrun and some need additional fixtures.
Authored probes do not establish successful coverage.
- Total conservative accounted cost: $9.875960. Active paid-campaign
time: 88.16/90 minutes. No missing accounting. Later security and
harness fixes have provider-free verification; no paid validation is
claimed for those revisions.
- Inspect the [qualification
report](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/READINESS.md)
and [verification
record](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/verification.json).

## Risks

- This is a broad authenticated API surface. Keep the default-off gate
until an operator selects initial rollout companies.
- Paid coverage is incomplete. Small regression samples do not prove all
workflows are unchanged.
- A timeout or server failure can follow a committed mutation. The
result reports an unknown outcome and requires state inspection.
- The new definitions add prompt tokens. The report retains cost flags
and cache variation.
- No database migration is required.
- Repository rules require code-owner approval before merge. Technical
CI and automated review are complete.

## Model Used

OpenAI Codex based on GPT-6 assisted with code, tests and review. The
exact serving model ID and context window are not exposed in this
session. It used reasoning, tool calls and code execution.

Eval models: `gpt-5.6-luna` with low reasoning,
`openrouter/anthropic/claude-sonnet-5`,
`openrouter/google/gemini-3.8-flash`, and
`openrouter/deepseek/deepseek-v4-flash-0731`. Attempts retain runtime
versions, model identity, usage and source provenance.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-07 14:14:43 -05:00
Dotta b84964e5a2
fix(runner): stabilize local paid E2E recovery (#12836)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paid runner E2E tests verify the complete runner, control-plane, and
UI path.
> - A server restart could load a fresh task page while Playwright still
waited on an unsettled Vite navigation lifecycle.
> - The current server also ignored the isolated Vite cache path and
skipped Vite's per-request HTML transform from the known-green runner
snapshot.
> - A one-cell paid run then exposed that download-artifact v8 removes
the artifact-name directory for one pattern match.
> - This pull request restores the Vite contract, proves a fresh
document after restart, and accepts only the exact singleton artifact
layout.
> - The benefit is reliable local runner qualification without weaker
UI, source, or artifact checks.

## Linked Issues or Issue Description

Refs #12769
Refs #12828
Refs #12829
Refs #12833

**What happened?**

The structured-question restart test could time out after the
replacement server returned the task route and rendered the durable
pending interaction. A focused one-cell rerun passed the paid test but
failed aggregation because download-artifact v8 flattened its single
artifact.

**Expected behavior**

The test must prove that a new document loaded after the server restart
and that the same pending interaction survived. The aggregate must
accept the exact documented singleton download layout while it continues
to reject ambiguous or foreign artifacts.

**Steps to reproduce**

1. Run the local ACPX-Codex structured-question restart-resume cell.
2. Restart the isolated server while the question waits for an answer.
3. Observe that the route and task UI can reload before Playwright
settles the navigation promise.
4. Run a paid campaign with one selected cell.
5. Observe download-artifact v8 extract the sole campaign directory
directly into the requested path.

**Paperclip version or commit**

The local campaign reproduced the navigation failure at
`3586956a1b794b3cb4a9c5f57ffb7355e2b0c46d`. The one-cell aggregate
reproduced the singleton layout at
`f487660c0a06ba06ca140b57386f21ed39f13120`. This fix is
`de4ccceff453a4b39436bf9a2eb8f03924151af7`.

**Deployment mode**

Local development and paid GitHub Actions.

**Installation method**

Built from source.

**Agent adapter(s) involved**

ACPX-Codex. The Vite and aggregate fixes are provider-neutral.

## What Changed

- Prove a new post-restart browser document with an in-memory sentinel.
- Tolerate only Playwright's navigation timeout before the exact UI and
API checks run.
- Honor `PAPERCLIP_VITE_CACHE_DIR` in the embedded Vite server.
- Limit dependency optimization to the real UI entry.
- Run `vite.transformIndexHtml` for each request while caching only the
branded source template.
- Accept download-artifact v8's flattened layout only for one expected
cell with one unique recognized campaign.
- Keep source SHA, source ref, workflow URL, execution ID, attempt, and
unexpected-entry validation.
- Add focused positive and negative regressions for Vite rendering and
singleton artifact selection.

## Verification

- Exact 45-cell local campaign
https://github.com/paperclipai/paperclip/actions/runs/33888939013 passed
44/45. Its only failure was the post-restart navigation false negative
fixed here.
- Exact focused rerun
https://github.com/paperclipai/paperclip/actions/runs/33891207957 passed
the ACPX-Codex restart cell first attempt with the same session, two
durable runs, the terminal marker once, and cleanup complete.
- The focused Vite renderer suite passed 2/2 tests.
- The focused rerun-artifact selector suite passed 12/12 tests.
- Prettier and `git diff --check` passed.
- An exact-head 45-cell confirmation is pending.

## Risks

Low to medium risk. The Vite change restores known-green per-request
transforms and isolated cache behavior. It can affect all development UI
loads. The paid matrix and ordinary CI will verify that behavior. The
singleton selector remains fail-closed for ambiguous layouts and
validates every result source.

## Model Used

OpenAI Codex, `gpt-5.6-sol`, extended reasoning, tool use, code
execution, and parallel focused agents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 11:16:20 -05:00
Dotta 0798c77fde
Secure Cloud canonical runtime identity (#12766)
Accept and persist Cloud-signed canonical runtime identity before activation, then route absolute self-URLs through the durable runtime identity provider.

Co-Authored-By: Codex <codex@openai.com>
2026-09-03 12:01:47 -05:00
Dotta fdf8c8464d
feat(runner): add managed provider backends (#12699)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner provides durable, provider-neutral agent
execution.
> - The current stack supports qualified local providers but omits the
managed provider paths from the integration branch.
> - Claude Managed Agents and AWS AgentCore need explicit profile
qualification, durable recovery, usage accounting, and cleanup controls.
> - This pull request adds those managed backends as the third part of
the Runner parity stack.
> - The benefit is managed execution without weakening the default-off
Runner rollout gate.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: Runner, server orchestration, database profiles, CLI, and
adapter configuration UI.

**Problem or motivation**

The current Runner stack cannot select or execute the managed Claude
Agents API or AWS Bedrock AgentCore Harness backends. It also lacks
qualified profile storage and recovery checks for those remote
resources.

**Proposed solution**

Add qualified managed and remote profiles, API and CLI management, exact
provider selection, durable lifecycle handling, cumulative usage
accounting, bounded cleanup, and retention acknowledgement. Keep
`enableNativeRunner` default-off.

**Alternatives considered**

A direct copy of the old integration branch was rejected because its
provider contracts, model values, credential flow, and migration history
no longer match the current base. A single large parity pull request was
also rejected because stacked review keeps each subsystem bounded.

**Roadmap alignment**

This continues the existing Runner architecture and rollout work. It
does not introduce a separate execution system.

**Additional context**

This pull request is based on the merged #12691 and #12685 stack. It
also closes the delayed security-review findings reported on #12691 by
binding qualified ACPX and OpenCode launch artifacts to the bytes
actually executed. A GitHub search for managed agent, AgentCore, and
Claude managed work found no duplicate public issue or pull request.

## What Changed

- Add Claude Managed Agents and AWS AgentCore provider executors to
runnerd.
- Add qualified managed and remote profile storage, routes, OpenAPI
contracts, CLI commands, and migration 0237.
- Validate profile ownership, enabled state, exact qualified revision,
model, agent version, and secret binding before persistence and
recovery.
- Persist durable provider session and owned skill state for
restart-safe cleanup.
- Reconcile uncertain create responses and delete remote sessions before
owned skills.
- Track cumulative provider usage and enforce positive session spend
caps.
- Recover interrupted AgentCore usage at the next turn boundary by
charging the prior invocation ceiling exactly once; keep the session
gated until an explicit monotonic budget raise.
- Isolate AgentCore AWS configuration from host profiles and
credential-process/SSO configuration while preserving workload identity.
- Require OpenCode 1.18.17 and fixed build-owned provider-pack artifact
paths; remove the ambient executable override.
- Snapshot and content-verify ACPX and OpenCode commands, scripts, and
provider executables before launch. Linux executes sealed inherited
descriptors; macOS uses authenticated private snapshots with retry-safe
rematerialization at the spawn boundary.
- Persist canonical ACPX and OpenCode launch-profile digests, reject
drift across fresh recovery, and make recovery failures sticky.
- Close and journal unsafe ACPX active-turn recovery before any provider
bootstrap or reconnect.
- Add managed provider fields to the Runner configuration UI and
permission projection.
- Preserve the default-off `enableNativeRunner` experimental flag.

## Verification

- `pnpm -r typecheck`
- `pnpm build`
- Focused managed server, database, CLI, Runner TypeScript, Rust,
Claude, AgentCore, ACPX, OpenCode, process-supervisor, and
durable-recovery tests passed.
- `cargo test -p paperclip-runner-core --lib --locked` (160 tests)
- `cargo check --workspace --all-targets --locked`
- Native Codex integration tests passed (60 tests); native provider
tests passed (7 tests); server native-runtime tests passed (87 tests).
- Verified-launch replacement, nested-spawn retry, exact-version,
profile-drift, sticky-failure, and no-bootstrap active-recovery tests
passed.
- `git diff --check`
- The PR changes 91 files. `pnpm-lock.yaml` is unchanged. The Rust
workspace lockfile adds the approved `rustix` dependency used for safe
descriptor handling while `#![forbid(unsafe_code)]` remains enabled.

## Risks

- The provider APIs can change while they are in beta. Exact
qualification and fail-closed recovery checks limit drift.
- Remote cleanup can fail after a partial create. Durable ownership
inventories and retry-safe deletion preserve recovery state.
- Migration 0237 adds profile tables. The generated migration and
snapshot pass the repository migration checks.
- Managed execution can incur provider cost. Positive default spend caps
and explicit retention acknowledgement limit accidental use.
- An interrupted AgentCore invocation without final metadata is
conservatively charged to its active session ceiling. This can overstate
cost, but cannot undercount it; later work requires an explicit budget
increase.
- Linux qualified launches use sealed memory descriptors. macOS lacks
executable-descriptor APIs, so the runner uses owner-only private
snapshots and minimizes linked-path lifetime; hostile same-UID processes
remain outside the documented local-host trust boundary.
- The global Runner feature remains default-off.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5, with tool use, code execution, and subagent review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-02 00:48:30 -05:00
Dotta d387cc0ff0
feat(connections): add managed external MCP connectors (#12346)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Connection intents need secure provider implementations to complete
setup.
> - Some providers use managed OAuth or external credential brokers.
> - Those tokens must stay out of durable Paperclip state and fail
closed when refresh fails.
> - This pull request adds managed connector backends and the required
storage contract.
> - The benefit is safer provider setup with governed credential
lifecycles.

## Linked Issues or Issue Description

Refs #11965

This is stack 8 of 11. It depends on stack 7 and replaces another
reviewable part of #11965.

## What Changed

- Add managed Google Workspace and external connector backends.
- Add Vercel Connect support without storing provider bearer tokens.
- Add replay-safe migration 0232 and its generated snapshot.
- Fail closed and clear stale token bindings when organization OAuth
refresh needs reauthorization.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- Result: 194 tests passed.
- `pnpm --filter @paperclipai/db check:migrations`
- `pnpm build`
- `pnpm exec vitest run --project @paperclipai/server
server/src/services/remote-url-credentials.test.ts` (5 passed, including
URL userinfo vault extraction)

## Risks

- Broker metadata errors can block provider setup.
- OAuth refresh failure disables the shared organization connection
until reauthorization.
- Migration 0232 is generated, ordered after 0231, and safe to replay.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked the public source pull request with `Refs #`
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-29 12:08:34 -05:00
Dotta b3343dbd64
feat(connections): add self-serve intent runtime (#12345)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents need a governed way to request app connections during issue
work.
> - The catalog now describes the available providers and setup methods.
> - A request must become a durable, company-scoped intent before an
operator acts on it.
> - This pull request adds that intent runtime across server, agent,
CLI, and shared contracts.
> - The benefit is a safe bridge from agent need to operator-approved
setup.

## Linked Issues or Issue Description

Refs #11965

This is stack 7 of 11. It depends on stack 6 and replaces another
reviewable part of #11965.

## What Changed

- Add connection intent types, validation, service logic, and routes.
- Add agent runtime tools and CLI support for connection requests.
- Add issue-thread interaction support for connection intents.
- Add runtime, route, adapter, and contract tests.
- Hold the final resolved-continuation row lock through asynchronous
adapter preparation until an actual process spawn, so parking or
reassignment cannot cross that boundary.
- Report Hermes Gateway's first remote run request through the shared
dispatch hook so the resolved-intent lock is released at the true
dispatch boundary.
- Revalidate the addressed user's live non-viewer membership and
connection-management authority for every intent mutation, including
OAuth completion.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- Result: 176 tests passed.
- `pnpm build`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-stale-queue-invalidation.test.ts` (32 passed;
includes non-process dispatch lock-release coverage)
- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/connection-intents-service.test.ts -t
"addressed-user mutation"` (1 passed)
- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/tool-access-service.test.ts -t "binds OAuth
callback completion to the initiating board session"` (1 passed)
- `pnpm --filter @paperclipai/hermes-paperclip-adapter test --
src/gateway/server/execute.test.ts` (23 passed; includes dispatch-hook
ordering and exactly-once coverage)
- `pnpm --filter @paperclipai/hermes-paperclip-adapter typecheck`

## Risks

- A malformed intent could create an unusable operator request.
- Validators and company checks reject invalid or cross-company
requests.
- The final continuation gate holds the issue row lock through adapter
preparation until process or remote dispatch; later operator changes use
the normal active-run interruption path.
- The change does not add a database migration.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked the public source pull request with `Refs #`
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:34 -05:00
Dotta 6244e4cf32
feat(apps): add Composio and Gmail connectors (#12342)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - App connections need both direct providers and managed provider
hubs.
> - The grant layer now defines safe credential ownership.
> - Composio needs parent and child connection lifecycle rules, and
Gmail needs governed setup.
> - This pull request adds both connector families on the grant
foundation.
> - The benefit is broader app access without weakening credential
isolation.

## Linked Issues or Issue Description

Refs #11965

This is stack 4 of 11. It depends on stack 3 and replaces another
reviewable part of #11965.

## What Changed

- Add Composio parent and child connection support.
- Add Gmail connection setup and governance.
- Preserve credential paths and remove duplicate binding declarations.
- Cascade Composio pause and restore actions to child connections.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- Result: 164 tests passed.
- `pnpm build`

## Risks

- Parent lifecycle changes can affect every Composio child.
- The service restores only children whose provider accounts remain
active.
- Credential binding paths are normalized before secret resolution.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:33 -05:00
Dotta b51112798f
feat(apps): improve gateway and workspace connection UX (#12340)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - App connections must work in both the operator UI and agent tool
gateway.
> - The first stack layer adds secure remote connections.
> - Operators still need clear setup, test, and recovery states.
> - This pull request adds the gateway behavior and the workspace
connection experience.
> - The benefit is a connection flow that is easier to understand and
recover.

## Linked Issues or Issue Description

Refs #11965

This is stack 2 of 11. It depends on stack 1 and replaces another
reviewable part of #11965.

## What Changed

- Improve remote tool gateway connection behavior.
- Add clearer app setup, test, and recovery states.
- Add focused server and UI tests for the new paths.
- Keep the diff isolated from later identity and catalog work.
- Stabilize DNS-pinned remote HTTP protocol fixtures and the
managed-runtime public-origin fixture for this independently tested
layer.

## Verification

- `pnpm -r typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts` (150 passed)
- `pnpm test:run`
- `pnpm check:token-gates`
- `pnpm build`

## Risks

- Gateway errors now surface through new user-facing states.
- A stale connection can require a new setup attempt.
- The change does not add a database migration.
- The injected HTTP transport and public URL are test-only fixtures;
production DNS pinning and runtime behavior are unchanged.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:32 -05:00
Devin Foley ca02d2463a
fix(ui): keep the installed service worker fresh on parked tabs (#12198)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The web UI registers a service worker (`/sw.js`) that caches the app
shell for an offline fallback.
> - Browsers only re-fetch a worker script on navigation or a ~24h
timer, and Paperclip is a parked-tab SPA: a tab can sit open for weeks
without one navigation.
> - An installed worker — and the shell it cached — can therefore keep
serving an old bundle long after a deploy, and the server let `sw.js`
inherit the generic 1h static TTL on top of that.
> - This pull request adds explicit update checks (tab-visible +
hourly), applies a discovered update with one reload while the tab is
hidden, and serves `sw.js` with `Cache-Control: no-cache`.
> - The benefit is that a deploy reaches every open tab within about an
hour, instead of some tabs silently running stale UI indefinitely.

## Linked Issues or Issue Description

Refs #11292 (the network-first `sw.js` fallback fix; this PR closes the
delivery gap that can keep clients pinned on a pre-#11292 worker).

**What happened?**

A browser that had an older cache-first worker installed kept rendering
a stale app shell — old feature set, old naming — while the server was
verified to be running the current release. Nothing on the client checks
for a new worker outside navigations, so a parked tab never picked up
the fixed worker, and `sw.js` was served with a 1h cache TTL that
further delayed update checks.

**Expected behavior**

Every open tab converges to the deployed bundle shortly after a release,
without users unregistering workers in DevTools or hard-reloading.

**Steps to reproduce**

Install a build's service worker, deploy a newer build, and leave the
tab parked (no navigation): the tab keeps running the old bundle
indefinitely; the worker update check only happens if the user
navigates, and even then a cached `sw.js` can answer it.

## What Changed

- New `ui/src/lib/service-worker-updates.ts`: registers `/sw.js`, runs
`registration.update()` when the tab becomes visible and on an hourly
timer, and on `controllerchange` of a previously-controlled page applies
the update with a single reload — only while the tab is hidden, so an
update never yanks the page mid-session; a takeover while visible defers
the reload to the next hidden transition. First-ever installs never
reload.
- `ui/src/main.tsx`: replaces the fire-and-forget `register()` with the
new module.
- New `server/src/static-ui-cache.ts` (`staticUiCacheControl`):
`index.html` and `sw.js` are served `Cache-Control: no-cache`; other
non-hashed statics keep the 1h default. `server/src/app.ts` uses it in
the static middleware.

## Verification

- `npx vitest run ui/src/lib/service-worker-updates.test.ts` — 8 tests:
registration, hidden-takeover reload (once), deferred reload on visible
takeover, no reload on first install, visibility-triggered and
timer-triggered update checks, cleanup, no-container no-op.
- `npx vitest run server/src/__tests__/static-ui-cache.test.ts` — 3
tests incl. the `sw.js.map` lookalike keeping the default TTL.
- `tsc -b` (ui) and `tsc --noEmit` (server) clean; `pnpm check:tokens`
clean.

## Risks

- Behavioral shift: tabs now reload once, while hidden, after a deploy
lands. Unsaved in-page state in a hidden tab is lost at that moment —
the same exposure as a browser discarding a background tab, which SPAs
must already tolerate.
- Self-hosted behavior is otherwise unchanged: same worker script, same
registration URL, one added conditional header.
- Low risk on the server side: the header change only widens
revalidation.

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — agentic
coding session with tool use and extended thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-25 16:12:02 -07:00
Dotta 397de98193
feat(runner): add flagged Codex execution adapter (#12188)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner now has protocol, provider, tool, package,
persistence, and hidden server boundaries.
> - The server still cannot select that path for a real agent heartbeat.
> - A new runtime must not change any existing direct adapter.
> - An experimental runtime must fail closed when its rollout flag is
off.
> - This pull request adds one guarded Codex vertical slice through
runnerd.
> - The benefit is a production-built runner path that users cannot
start by default.

## Linked Issues or Issue Description

Refs #11962

Refs #12111

Refs #12169

Refs #12176

**Subsystem affected**

Cross-cutting. The change affects the runner package, server
orchestration, shared settings, and adapter configuration UI.

**Problem or motivation**

The hidden PRP coordinator cannot execute a real heartbeat. The
application also needs an explicit rollout boundary before it can expose
the experimental runner. Existing direct adapters must keep their
current execution and finalization behavior.

**Proposed solution**

Add `paperclip_runner` as a Codex-only adapter behind the default-off
`enableNativeRunner` instance flag. Select the native runtime only for
that adapter. Persist the run binding before runnerd starts. Wait for
the durable PRP result and terminal event. Resume the real Codex
provider thread on later heartbeats. Keep persisted native runs readable
and recoverable after the flag changes.

**Alternatives considered**

The server could route `codex_local` through runnerd. That option would
change an existing adapter and weaken rollback safety. The server could
expose all providers now. That option would add unreviewed provider
behavior. The build could depend on a prebuilt runner binary. That
option would make source builds architecture-dependent and difficult to
verify.

**Roadmap alignment**

This work supports the shipped enforced-outcomes, governed-tool, and
self-healing-run milestones. It does not add a new roadmap surface. It
is the guarded execution step after the merged hidden runner boundaries.

**Additional context**

This is the next replacement for the closed large runner pull request.
Task-thread presentation remains a separate follow-up so this change can
preserve the current direct-adapter UI.

## What Changed

- Add `paperclip_runner` as an explicit Codex-only adapter.
- Add the default-off `enableNativeRunner` instance flag.
- Reject fresh create, hire, import, switch, and execution requests
while the flag is off.
- Allow edits to persisted runner agents while the flag is off.
- Recover an already persisted native run even after the flag is
disabled.
- Keep every built-in direct adapter on its existing runtime path.
- Persist an immutable native run binding and revisioned completion
contract before runnerd starts.
- Execute server to PRP to runnerd to Codex to server through the hidden
coordinator.
- Validate the durable result against the terminal event and exact
completion criteria before finalization.
- Preserve the Codex provider thread ID and use `thread/resume` on the
next heartbeat.
- Strip unsupported Codex configuration fields from the experimental
adapter.
- Build a target-native release runner binary from source and vendor it
into the server distribution.
- Install Rust only in the Docker build stage. Do not add a workflow or
lockfile change.
- Stop the runner process group on completion, cancellation, and forced
shutdown.

## Verification

- Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All 69
TypeScript tests and 58 Rust tests pass. Protocol, conformance, replay,
formatting, and generated-file checks pass.
- Run the 12 focused adapter, settings, runtime-selection, coordinator,
direct-isolation, and real Codex integration test files. All 186 tests
pass.
- The real integration test uses PostgreSQL, HTTP, WebSocket, runnerd,
and a fake Codex app server. It proves one `thread/start` followed by
one `thread/resume`.
- Run `pnpm -r typecheck`.
- Run `pnpm build`.
- Run `pnpm check:token-gates`.
- Build the Docker `build` target from a clean context. Confirm that the
server distribution contains an executable `paperclip-runnerd` built
with Debian Rust 1.85.
- Start the server through the source-mode tsx entry point with the
package `dist` directory absent. Confirm the vendor shim resolves source
exports and the server boots.
- Run `pnpm test:run` twice. On this macOS host, 405 files pass and 1
file skips. Eight untouched workspace and loopback tests fail because
macOS resolves `/tmp` and `/var` through `/private` and because
PID-derived test ports exceed 65535. Linux CI must pass the full suite.
- Confirm that the diff contains 52 files. Confirm that it contains no
`.github` or `pnpm-lock.yaml` change.

## Risks

- The feature flag is off by default. A fresh native start fails with a
stable error while the flag is off.
- A persisted native run remains recoverable after the flag changes.
This prevents rollout changes from corrupting recorded work.
- Only local Codex execution is accepted. Other providers and remote
work modes fail closed.
- Existing direct adapters do not start runnerd, create native rows, use
native status arbitration, or enter native finalization.
- The runner receives its one-use bootstrap ticket through the child
environment. The server does not put the ticket in command arguments or
logs.
- The server validates the company, task, agent, run, runner, session,
completion contract, result, and terminal binding before it accepts
completion.
- The build compiles a target-native Rust binary. Cross-platform release
packaging remains a later concern. Source builds and Docker builds
compile for their current target.
- Docker needs enough build memory for the existing server TypeScript
compile. The Docker build stage sets a 4 GB V8 heap limit.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and applicable tests pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-25 16:03:41 -05:00
Nicky Leach 63df7ad2b3
feat(login): use the login pseudo-terminal for Codex device login and de-Claude the shared channel (#12020)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters use provider-specific login flows
> - Codex device login needs a live pseudo-terminal (PTY), while the
shared channel still uses Claude-specific names
> - The old streamed-exec path does not provide the prompt transport
that Codex needs
> - This pull request moves Codex device login to the shared login PTY
and removes the dead streamed-exec path
> - The benefit is one controlled login transport with fail-closed
capability checks and safer credential reads

## Linked Issues or Issue Description

**Problem or motivation**

Codex device login used a streamed-exec path that did not provide the
required prompt transport. The shared login channel also exposed
Claude-specific names outside Claude code.

**Expected behavior**

The host selects a fixed login command from trusted adapter data. Codex
login uses the provider login PTY. Providers without that capability
fail closed.

**Proposed solution**

Use a server-controlled session home, create and validate it as a fresh
0700 directory, read credentials from one validated descriptor, and
rename shared channel names to the neutral login PTY family.

**Alternatives considered**

Keep the shared login PTY as the single transport. Do not keep the
removed streamed-exec path because it cannot provide the required prompt
transport.

**Roadmap alignment**

This change supports the planned login transport work. It does not add a
separate roadmap item.

## What Changed

- Route Codex device login through the shared login PTY transport.
- Select the login command from a closed internal command key.
- Carry a server-controlled session home through the launch contract.
- Create and validate the session home as a fresh 0700 directory owned
by the login user.
- Read the credential file with descriptor-relative, no-follow path
walking and final descriptor checks.
- Gate the login route and run lease on the provider login PTY
capability.
- Rename shared channel names to the neutral login PTY family.
- Remove the streamed-exec transport value, selector field, driver
branch, and related tests.
- Hide Codex login in the user interface when the provider lacks the
login PTY capability.

## Verification

- Server unit suites pass: 89/89.
- Adapter-utils suites pass: 262/262.
- Codex-local suites pass: 326/326.
- Credential-read reader suite passes: 20/20.
- Daytona login PTY suite passes: 30/30.
- Device-login suites pass: 56/56.
- TypeScript checks pass for server, adapter-utils, and UI.
- GitHub Actions must pass after pull request creation.
- Greptile review must reach 5/5 with no open P2 findings,
recommendations, or follow-ups.

## Risks

- Providers without a login PTY capability lose Codex login support by
design.
- The credential read rejects invalid ownership, mode, type, path, and
size.
- The launch-time sandbox directory race remains outside the threat
model because the login runs inside the sandbox and a hostile sandbox
already controls its credential.

## Model Used

OpenAI Codex, GPT-5, tool use and code review assistance. The exact
context window and reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-23 09:44:59 -07:00
Nicky Leach e0b64529b3
feat(auth): normalize agent login in the sandbox onto one session table and a capability contract (#11730)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox agents need a safe login path for each supported adapter
> - Codex device login and Claude setup-token login used separate
session stores and route logic
> - Separate stores made session lookup, expiry, and login capability
checks harder to keep consistent
> - This pull request unifies both flows on one session table and one
capability contract
> - The benefit is one company-scoped login model with public session
identifiers and shared lifecycle rules

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting (multiple of the above)

**Problem or motivation**

Codex and Claude sandbox login used separate session stores and
different route paths. This split increased the risk of inconsistent
company scoping, session lookup, and cleanup.

**Proposed solution**

Use `adapter_auth_sessions` for both login flows. Use public session
identifiers for API access. Select login behavior from projected adapter
capability data. Share the route spine, lease arguments, runner
lifecycle, and reaper rules.

**Alternatives considered**

Keep two session tables and add matching fixes to both routes. This
keeps duplicate logic and does not provide one capability contract, so
this pull request uses shared infrastructure.

**Roadmap alignment**

This change supports the shipped Cloud / Sandbox agents milestone in
`ROADMAP.md`.

## What Changed

- Unify Codex device login and Claude setup-token login on
`adapter_auth_sessions`.
- Return and look up sessions with company-scoped public session
identifiers.
- Enforce one active session for each company, owner, and adapter.
- Share the login route spine, sandbox lease arguments, runner
lifecycle, and missing-auth check.
- Add a standalone setup-token reaper with adapter-specific row
selection.
- Add optional login capability projection for adapters and drive route
and UI selection from that data.
- Rename the provider flag to `supportsLoginPty` and validate its
deprecated alias.
- Remove the old Claude setup-token session table and add the required
migrations.

## Verification

- Server typecheck passed with `tsc`.
- Database typecheck passed.
- UI typecheck passed with `tsc -b`.
- Codex login service and route suites passed.
- Setup-token session, route, and reaper suites passed.
- Adapter session schema, plugin validator, capability projection, UI
render, and Daytona suites passed.
- GitHub Actions must confirm the complete CI gate after pull request
creation.

## Risks

- The migrations remove short-lived in-flight login rows during
deployment. A login that spans the migration can continue until its
provider lease expires.
- The Codex credential store remains company-scoped. A cross-owner
credential race remains a documented, board-accepted risk.
- API clients that use internal session row identifiers no longer work.
The API accepts only public session identifiers.

## Model Used

Codex, GPT-5, exact runtime model ID not exposed in this handoff, large
context window, reasoning, and repository tool use. The implementing
engineer produced the code with AI assistance.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-19 11:51:31 -07:00
Devin Foley 962e98b1be
feat(server): operator declaration for platform edge TLS termination on the Claude login guard (#11579)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Claude local adapter supports a setup-token subscription login,
and its confidential routes pass a fail-closed transport guard
> - The guard accepts direct socket TLS, a local_trusted loopback peer,
or an allowlisted proxy peer that forwards https — and deliberately
never reads the global `TRUST_PROXY`
> - On a managed platform the edge terminates TLS, the app socket is
always plain HTTP, and the edge-proxy peer addresses are not stable or
documented, so none of the three cases can hold
> - Every login on such a deployment shows the clear-text transport
warning although the user's connection is HTTPS, and the agent-scoped
confidential routes fail closed entirely
> - This pull request adds a dedicated operator declaration that the
platform edge terminates TLS, as a fourth guard case
> - The benefit is a correct transport decision on managed platforms
with the default posture unchanged everywhere else

## Linked Issues or Issue Description

No public GitHub issue covers this. The problem is described in-PR
following the enhancement template. Related public PRs:
[#11347](https://github.com/paperclipai/paperclip/pull/11347) added the
new-agent login flow and the non-blocking transport advisory, and
[#11286](https://github.com/paperclipai/paperclip/pull/11286) added the
setup-token login and the guard with its `CLAUDE_LOGIN_TRUSTED_PROXIES`
allowlist.

**Subsystem affected**

server/ — the confidential transport guard for the Claude setup-token
login (`services/setup-token-session.ts`, `routes/agents.ts`, `app.ts`).

**Current behavior**

The guard allows a confidential response on direct socket TLS, on a
`local_trusted` loopback peer, or when the immediate peer is on the
dedicated `CLAUDE_LOGIN_TRUSTED_PROXIES` allowlist and forwards `https`.
Behind a managed platform's TLS-terminating edge (Railway, Render, Fly,
and similar), the app socket is plain HTTP and the edge-proxy peer
addresses are not operator-visible or stable, so the allowlist cannot
express them — IPv6 entries match by exact string only. The result: the
login panel shows "This connection is not encrypted" for a connection
that is HTTPS to the user, and the agent-scoped confidential routes
return the fixed no-secret error.

**Proposed behavior**

`CLAUDE_LOGIN_EDGE_TLS_TERMINATED=true` is an explicit, single-purpose
operator declaration that every client request reaches the server
through the platform's TLS-terminating edge. Under the declaration the
guard treats a request as confidential unless the edge itself labels the
client hop as plain `http` in `X-Forwarded-Proto`. The declaration is
never derived from the global `TRUST_PROXY` setting, which the guard
still never reads. Without the declaration, nothing changes.

**Reason and benefit**

The guard's spoofing concern does not apply to this deployment shape: a
client cannot pick its transport, because the platform admits HTTPS
only, and the header the guard consults is set by the platform edge, not
the client. A blanket warning that is always wrong teaches users to
ignore it. The declaration keeps the strict default for every deployment
that does not opt in, and it keeps the allowlist as the precise tool for
operators who do know their proxy addresses.

## What Changed

- `ConfidentialTransportConfig` gains optional `edgeTlsTerminated`
(default false), documented as the operator declaration for platform
edge TLS termination.
- `evaluateConfidentialTransport` adds the declaration as a guard case:
allowed unless the forwarded protocol's first hop is explicitly `http`
(reason `edge_labeled_plain_http` then; `operator_edge_tls_termination`
when allowed).
- `assessConfidentialStartup` reports `edge_tls_termination_declared`,
so the startup log shows why forwarded requests pass.
- `app.ts` parses `CLAUDE_LOGIN_EDGE_TLS_TERMINATED` (truthy:
`1/true/yes/on`) and passes it to the agent routes; the routes build the
guard config from it.
- The SR-7 operator-requirement comment on the setup-token routes
documents the new variable next to the allowlist.
- Tests: five new guard unit cases and a route case asserting the prompt
and code responses carry no `transportAdvisory` under the declaration.

## Verification

```sh
cd server
npx tsc --noEmit    # clean
npx vitest run src/services/setup-token-session.test.ts \
  src/routes/setup-token-route.test.ts \
  src/__tests__/openapi-routes.test.ts   # 3 files, 89 passed
```

The new "keeps failing closed when the declaration is absent" case pins
the unchanged default posture.

## Risks

The declaration is an operator statement the server cannot verify; an
operator who sets it on a deployment whose edge does not terminate TLS
re-labels plain-HTTP requests as confidential. This is the same trust
class as `CLAUDE_LOGIN_TRUSTED_PROXIES` (a wrong allowlist entry has the
same effect) and is opt-in, off by default, and scoped to the login
routes only. The guard still fails closed when the edge explicitly
labels a request `http`. No schema change, no API shape change —
`transportAdvisory` was already nullable.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, with tool use and
code execution — investigation, implementation, and tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-17 17:46:45 -07:00
Nicky Leach c1c46f1e4e
feat: Claude login on the new-agent page before agent creation (#11347)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Claude local adapter supports subscription login through a
sandbox
> - The new-agent page must show login before the user creates an agent
> - Test results must not expose raw sandbox diagnostics or secret
values
> - This pull request adds the login UI to both Test lanes and closes
the diagnostic boundary
> - The branch also adds durable cleanup recovery for failed sandbox
teardown
> - Reusable sandboxes must retain both their recorded teardown
configuration and a valid lifecycle path until destruction succeeds
> - The benefit is a usable login flow with fixed public checks,
redacted server logs, and recoverable sandbox cleanup

## Linked Issues or Issue Description

Related public work:
[#9488](https://github.com/paperclipai/paperclip/pull/9488) adds
first-class recognition for `CLAUDE_CODE_OAUTH_TOKEN` in headless and
remote runs. Related public issue:
[#2681](https://github.com/paperclipai/paperclip/issues/2681) requests
Claude Code subscription support. This pull request adds the login
transport and new-agent UI flow that those changes do not provide.

**Subsystem affected:** Claude local adapter, server login probes,
sandbox provider setup, cleanup recovery, and the new-agent UI.

**Problem or motivation:** The Test lanes did not show the sandbox login
panel in all supported cases. Test results also exposed raw probe
diagnostics, and JSON escapes could end secret redaction early.

**Proposed solution:** Surface the login capability through the bundled
provider manifest. Prepare the same probe runtime in the ACP lane. Send
diagnostics only to redacted server logs. Keep Test checks on fixed
public messages. Normalize login URL hints to allowlisted HTTPS Claude
and Anthropic hosts. Consume JSON escapes during redaction. Preserve
failed sandbox cleanup state across retries and restarts, and prevent
deletion from severing the lifecycle context of a live reusable sandbox.

**Alternatives considered:** Keep raw diagnostics in Test checks or
trust login URL text from the sandbox. Both choices increase information
exposure. Keep separate probe behavior in the ACP lane. That choice
would leave the two Test lanes inconsistent.

## What Changed

- Surface the sandbox login panel on both Test lanes.
- Reconcile the bundled Daytona plugin manifest so
`supportsSetupTokenLogin` reaches the UI capability gate.
- Prepare the ACP Test lane with the same probe runtime as the CLI Test
lane.
- Add the `claude_acp_login_probe_unavailable` warning when the ACP
probe cannot run.
- Send raw sandbox diagnostics only to redacted server logs.
- Keep Test checks on fixed public messages in the ACP, managed-config,
and CLI paths.
- Normalize login URL hints to allowlisted HTTPS Claude and Anthropic
hosts.
- Redact JSON and escaped-JSON secret values, including escaped quotes
and backslashes.
- Preserve orphan cleanup records across provider failures, restarts,
and unavailable plugins.
- Atomically block environment deletion while a live reusable sandbox
lease still depends on it.
- Verify pending cleanup destroys plugin sandboxes with the provider
configuration recorded on the lease, even after the current environment
configuration changes.

## Verification

- Head under review: `506b7fa2d83c36bfa5fd722ee9d95b0c7431c241`.
- Focused environment route/service/runtime coverage passes: 196 tests
across 3 files.
- `pnpm -r typecheck` passes.
- `pnpm build` passes.
- The full Vitest run completed with 4,754 passing and 28 failing tests.
All 23 source-test failures reproduce unchanged on parent head
`58cfe61a33191ce03d965d65085d26064b4888ba`; the other 5 are duplicate
executions from stale `server/dist` output. The failures are unrelated
macOS path/listener and scheduler-fixture failures, so there is no new
bad commit for bisect to localize.
- All required CI checks pass for the current head, including build,
typecheck/release registry, all server and workspace shards, serialized
server suites, canary, and e2e.
- A fresh Greptile review for `506b7fa2d83c36bfa5fd722ee9d95b0c7431c241`
reports 5/5, “safe to merge,” with no blocking failure remaining.

## Risks

- A probe or redaction change could hide useful server diagnostics.
- An allowlist change could reject a valid Claude login URL.
- Cleanup recovery changes could affect provider teardown ordering.
- An environment with a live reusable sandbox can no longer be deleted
until the owning issue or execution workspace completes teardown.
- The implementation keeps public Test messages fixed and sends detail
to redacted server logs.

## Model Used

OpenAI GPT-5 via Codex — exact model ID: GPT-5; tool use and code
execution enabled; extended reasoning enabled. The implementation author
used AI-assisted development.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and documented the result
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation or confirmed no separate
documentation change is needed
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-17 13:42:51 -07:00
Dotta 4c349fe6b7
feat(runtime): managed Tailscale HTTPS lifecycle, durable runtime leases, and bounded control recovery (#11525)
<!-- Simplified Technical English (ASD-STE100). -->

> **Stacked pull request.** This targets #11524. Merge #11524 first.
Review only the second commit, `feat(runtime): managed Tailscale HTTPS
lifecycle...`.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip starts and supervises managed runtime services, so an
agent's branch can be previewed while the agent works
> - The previous pull request added the host broker, the shared
contract, and the database columns, but no code used them
> - A managed runtime can only be exposed over HTTPS if it holds a
stable loopback port pair for the whole life of the service. The current
control path cannot promise this: two controls can race the same
execution workspace, a stranded control can stay `running` forever, and
a start can adopt a port it does not own
> - This pull request adds the HTTPS lifecycle and the control-path
hardening that the lifecycle depends on
> - The benefit is that a managed preview becomes reachable from another
device, and a managed control now always reaches a terminal state

## Linked Issues or Issue Description

No public GitHub issue exists. The change follows the feature request
template.

**Subsystem affected**

Managed workspace runtime services, workspace operations, the execution
workspace routes, and the workspace runtime UI.

**Problem or motivation**

A managed runtime service is reachable only on loopback, so a preview
cannot be opened from a phone or a second computer. Exposing it safely
needs an exclusively held port pair. Three existing gaps block that.
Overlapping controls can race the same workspace. A control whose owner
dies stays `running` and blocks the lane forever. Port allocation does
not confirm that the process holding a port is the process Paperclip
spawned.

**Proposed solution**

Add the exposure lifecycle on top of the broker from #11524: reserve
before spawn, expose after readiness, validate the public URL, and
remove on stop. In the same change, make managed controls mutually
exclusive per workspace, give each control a durable issue-owned lease
and a terminal state, and verify port ownership before use.

**Alternatives considered**

- Add HTTPS exposure without the control hardening. This was rejected
because a raced or stranded control makes exposure point at the wrong
process.
- Guard the lane with an in-memory lock only. This was rejected because
the lock does not survive a server restart, so the lane can be lost or
double-claimed.
- Trust the requested bind address. This was rejected because a checkout
that predates managed HTTPS overwrites `PAPERCLIP_BIND` from its own
`--bind` argument, and then binds the wildcard address.

**Roadmap alignment**

This completes the managed workspace runtime capability that already
exists. It adds no new product surface beyond the HTTPS link.

**Additional context**

This is the second of three pull requests. The third adds central
mediation of leased port pairs.

## What Changed

Exposure lifecycle:

- Add the server-side broker client and the exposure lifecycle manager.
The manager reserves the mapping before spawn, exposes after backend
readiness, validates the public URL, and removes the mapping on stop.
- Default managed worktree runtimes to `tailscale_https`, read exposure
intent from legacy `expose` blocks, and backfill runtimes that are still
HTTP-only.
- Verify listener ownership for the app port and its Vite HMR companion
before the broker is asked to expose anything. An unrelated listener on
either port fails the start closed.
- Force the loopback bind through argv instead of environment hints.
Leave a non-Paperclip service's `--bind` argument alone.
- Probe loopback for readiness instead of the public URL, and give Vite
HMR its own loopback-bound server in middleware mode.
- Preserve operator-declared Serve mappings across the managed
lifecycle, so cleanup never removes a mapping that Paperclip did not
create.
- Name which listener predicate denied an expose, so an operator can act
on the message.

Control-path hardening:

- Make `start`, `stop`, `restart`, and job `run` mutually exclusive per
execution workspace. An overlap gets `409
workspace_runtime_control_in_progress`, and authorization is still
checked first.
- Take a durable exclusivity lease on the execution workspace, owned by
the controlling issue. A different issue gets `409
workspace_runtime_lease_conflict` before any operation is recorded.
Board and operator actions bypass the lease.
- Give every control a terminal state. Each control stamps its owning
process and pid, heartbeats while it runs, and has a wall-clock ceiling.
Recovery of a stranded control uses a compare-and-swap on `updated_at`,
so a live owner is never stolen.
- Bound readiness probes, verify allocated port ownership on POSIX and
Windows, harden sibling port allocation, and reconcile desired runtimes
on server startup.
- Surface exposure state and bounded runtime errors in the workspace
runtime UI.
- Record the new behavior in `doc/DEVELOPING.md`.

## Verification

Focused checks, all run on this branch:

- `npx tsc --noEmit -p server/tsconfig.json` — 139 errors, exactly the
count on `master`. All 139 come from the unbuilt
`@paperclipai/plugin-sdk` package.
- `pnpm --filter @paperclipai/ui typecheck` — clean.
- Server suites, 177 tests pass across 9 files:
`workspace-runtime.test.ts`, `workspace-runtime-leases.test.ts`,
`workspace-runtime-control-recovery.test.ts`,
`execution-workspace-runtime-control-conflict.test.ts`,
`execution-workspace-runtime-lease-route.test.ts`,
`workspace-operations-reconciliation.test.ts`,
`workspace-runtime-start-terminality.test.ts`, `app-hmr-port.test.ts`,
and `workspace-runtime-ready-comment.test.ts`.
- Exposure unit suites, 77 tests pass: `src/services/runtime-exposure/`
and `workspace-runtime-exposure-backfill.test.ts`.
- UI: `WorkspaceRuntimeControls.test.tsx` and
`WorkspaceServiceControlBar.test.tsx` — 34 tests pass.

**One suite is red on the development host and is expected to be green
in CI.** `server/src/services/workspace-runtime-exposure.test.ts` has 10
failures on the machine used to write this branch. The cause is host
contamination, not the code. That machine already runs an HTTPS canary
that holds ports 42000, 42001, 52000, and 52001 on a tailnet address.
The suite allocates from the same range, so the new listener-ownership
check correctly reports:

```
listener_ownership_mismatch — port 42000 is bound to 100.123.243.20, 127.0.0.1,
fd7a:115c:a1e0:0:0:0:dd3a:f314 ... instead of loopback only
```

A CI runner has no listener on those ports, so the check sees loopback
only and the suite passes. Please confirm this from the CI result on
this pull request rather than from a local run on a host that already
exposes a managed runtime. This is a real weakness of the current test
fixture, and the third pull request in the series removes it by
allocating the pair through a central mediator instead of a stubbed
availability check.

`workspace-runtime-https-live-exercise.test.ts` needs a live `tailscale`
host and was not run locally.

## Risks

- This is the behavior-bearing pull request of the three, so it carries
the most risk.
- Two new `409` responses appear on managed control routes. A caller
that assumed a control always starts must handle a conflict. Board and
operator actions are deliberately exempt, so an agent lease cannot lock
an operator out.
- Managed worktree runtimes now default to `tailscale_https`. If the
host has no working broker, the start fails closed and reports the
exposure failure instead of silently serving plain HTTP. This is
intended, and it is the reason the failure message names the denying
predicate.
- Startup reconciliation touches persisted runtime rows. It is scoped to
desired state and does not resurrect a service that never came up.
- The lease has a 30-minute time to live and explicit release paths, so
a crashed owner cannot hold a lane forever.
- No migration runs in this pull request. The tables and columns land in
#11524.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR.

## Model Used

Claude Opus 5 (`claude-opus-5`), 1M context window, extended thinking,
with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass, with the one
host-contaminated suite explained above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-17 06:23:33 -04:00
Tonio f0e6c0f549
feat(server): receive and apply the Paperclip Cloud onboarding seed (#11098)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud provisions a dedicated tenant stack for each
customer. During signup it asks for a mission, a name and role for the
first agent, and a first task.
> - Cloud pushes those answers into the new stack at activation, as
`POST /api/companies/:companyId/onboarding-seed`.
> - No route served that path. The tenant answered 404, so Cloud
recorded the push as unacknowledged and retried on every portfolio
fetch.
> - The failure was soft. The answers stayed durable in Cloud and the
stack still activated. But the stack opened on the empty first-run
wizard, and it asked the customer again for what they had already given.
> - This pull request adds the receiving endpoint. It validates the
seed, applies it, and acknowledges it.
> - The benefit is that a seeded stack opens with the mission, the agent
and the first task already in place.

## Linked Issues or Issue Description

No public GitHub issue covers this. The problem is described in-PR,
following the feature template.

**Subsystem affected**

server/ — Express REST API and orchestration services. Also
`packages/db` (one new table) and `packages/shared` (one new validator).

**Problem or motivation**

Paperclip Cloud collects onboarding answers at signup and pushes them to
the tenant stack at activation. The tenant had no route for that
request. It answered 404. Cloud treats a non-2xx as "not yet applied",
so it kept the answers and retried, but the stack itself stayed
unseeded. A customer who had already named their mission, their first
agent and their first task arrived at an empty first-run wizard that
asked for all three again.

**Proposed solution**

Serve `POST /api/companies/:companyId/onboarding-seed`. Validate the
body, apply it to the company, then acknowledge it.

The seed is customer free text, so it is bounded and validated in
`packages/shared` and read from the JSON body only. It is never read
from an `x-paperclip-cloud-*` header. That header set is the trusted
identity envelope: every member is derived server-side from the host
plus verified domain records, and that is exactly what makes it
trustworthy. Mixing user content into it would remove the property. A
test plants a mission on a cloud header and asserts that the body value
wins.

Application reuses the shapes the first-run wizard already produces, so
a seeded stack and a manually onboarded one look the same afterwards:

- The mission becomes the company-level goal. A multi-line mission
splits into a title and a description, as the wizard does.
- The agent becomes the company's first hire. Its free-text role ("Chief
of Staff") lands on `title`. The structural `role` stays `ceo`, which is
what the org chart and the default-instructions lookup read.
- The first task becomes an issue in the Onboarding project, assigned to
that agent.

Cloud retries until it gets a 2xx, and it reads any 2xx as "the tenant
holds this content". So the endpoint is idempotent per `revision`. A new
`company_onboarding_seeds` table records the applied revision together
with the goal, the agent and the issue it produced. A replay of a
revision that already matches is a successful no-op. A later revision —
the customer edited their answers — updates those three rows in place
instead of creating a second agent and a second task. The record is
written last, after every other write has landed, so a partial
application cannot present itself as acknowledged.

Everything is applied before the 200 is sent. This is an ordering
guarantee, not eventual consistency. The tests read the database
immediately after the response, with no waiting and no polling, so a
lazy receiver fails them on a fast machine as well as a slow one. That
matters because the redirect into the tenant dashboard is gated on this
acknowledgement.

**Alternatives considered**

Store the seed and let the tenant UI apply it on first load. Rejected:
the dashboard redirect is gated on the acknowledgement, so a background
apply would let the dashboard open before the agent and the task exist.
The whole point is that it must not.

Reuse `POST /companies/:companyId/agents` and `POST
/companies/:companyId/issues` over HTTP from Cloud. Rejected: it needs
three round trips with no shared idempotency key, and it moves the "did
all of it land?" decision to the caller.

**Roadmap alignment**

This completes an existing Cloud-to-tenant contract. It does not add a
new user-facing surface.

## What Changed

- Add `POST /api/companies/:companyId/onboarding-seed` in
`server/src/routes/onboarding-seed.ts`. It authenticates exactly as
`POST /api/companies/:companyId/logo` does, through
`assertCompanyAccess`.
- Add `server/src/services/onboarding-seed.ts`. It applies the mission,
the agent and the first task, and records the applied revision last.
- Add the `company_onboarding_seeds` table: schema, migration `0216`,
and journal entry. It holds the applied revision and the ids of the
goal, agent and issue the seed produced.
- Add `applyOnboardingSeedSchema` in `packages/shared`. It bounds
mission to 2000, agent name to 80, agent role to 120, task title to 200,
and task details to 2000 — the same limits Cloud enforces before it
sends.
- Mount the router in `server/src/app.ts` and register the path in the
OpenAPI document.
- Add `server/src/__tests__/onboarding-seed-route.test.ts` with 13
tests.
- The seeded agent is created on `claude_local`. This mirrors the
teams-catalog default for agents created server-side, where no human
runs an environment test first. `PAPERCLIP_ONBOARDING_SEED_ADAPTER_TYPE`
overrides it.

## Verification

```sh
pnpm typecheck                      # whole workspace, passes
npx vitest run \
  server/src/__tests__/onboarding-seed-route.test.ts \
  server/src/__tests__/openapi-routes.test.ts        # 15 passed
```

The suite runs against embedded Postgres with migrations applied, so
migration `0216` is exercised by every test.

The route tests cover:

- the happy path — mission, agent and task all applied, read immediately
after the 200
- replay of the same revision — no second agent, no second task, no
second goal, no second project
- a later revision — the goal, agent and task are updated in place
- a multi-line mission splitting into a goal title and description
- a revision-only seed
- the activity log entry written once, and not again on a replay
- a caller without access to the company — 403, and nothing written
- a body with no revision — 400
- each field bound past its limit — 400
- a mission planted on an `x-paperclip-cloud-*` header — ignored, body
wins
- an existing Onboarding project — reused, not duplicated

Not verified here: the full Cloud-to-tenant walk against a live stack.
That needs a deployed Cloud and a provisioned tenant together, which is
separate staging work.

## Risks

Migration `0216` creates one new table. It adds no column to an existing
table, rewrites nothing, and backfills nothing, so it is safe to apply
online. The migration safety check passes.

The endpoint writes to a company. Access is enforced by
`assertCompanyAccess`, the same gate the company logo write uses, and a
test covers the denial.

Behavioral note for stacks that already hold data. If a company already
has a non-built-in `ceo` agent, a first seed updates that agent's name
and title rather than creating a second lead. Likewise a seed adopts an
existing company-level goal rather than adding a parallel one. This is
deliberate: the seed is the customer's own stated answer from signup,
and two competing missions or two leads would be worse than one updated
in place. In the intended case — a stack that Cloud has just activated —
none of these exist yet.

The seeded agent is created on `claude_local` with an empty adapter
config. It is idle and needs the usual credential setup before it runs.
Seeding it does not start it.

## Update — rebased onto master + review hardening

Master moved on after this PR was cut, so it was **rebased onto
`master`** and
the seed migration was **renumbered from `0212` to `0216`** (the merged
#11101
took `0212_onboarding_first_task_unique`); the drizzle journal was
re-stitched
and `check:migrations` passes.

Two things landed on top of the original receiver:

- **Mission-only walk contract (PAP-67 r17.4).** The tenant now owns the
first
agent and the first task via #11101's server-owned onboarding path,
which
stamps `ONBOARDING_FIRST_TASK_ORIGIN_KIND` and races safely on the
partial
unique index `issues_onboarding_first_task_uq`. A comment in the apply
path
documents why this receiver leaves the first task to that path on the
cloud
walk, and a paperclip-cloud `node:test`
(`src/onboarding/walk-seed.test.ts`)
asserts the walk's seed carries no `agent`/`firstTask`. The receiver
retains
the agent/first-task code for its documented body contract, kept inert
on the
  cloud path by the mission-only seed.
- **Three Greptile P1 fixes** (`95622fa37`): concurrent application is
now
  serialized under a per-company `pg_advisory_xact_lock` (no duplicate
goal/agent/project/task on overlapping pushes); a revised first task
carries
its resolved `assigneeAgentId`/`goalId`; and the
`company.onboarding_seed_applied`
  audit write is best-effort so a logging failure can't leave the entry
  permanently absent. Two new regression tests cover the first two.

## Model Used

Claude Opus 5 (`claude-opus-5`), 1M context window, extended thinking,
with tool use and code execution. Used for the original codebase
investigation, the implementation, and the tests. The rebase, migration
renumber, mission-only contract, and the three P1 fixes were done with
Claude Opus 4.8 (`claude-opus-4-8`), extended thinking, with tool use
and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:54:07 -07:00
dmndbrp-oss a8d118a779
Prefer public base URL for generated invite links (#7619)
Fixes #7623

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Company invites are part of the access subsystem and must produce
URLs that recipients can open from outside the host machine.
> - Paperclip already has public/auth base URL configuration for
deployments behind a public hostname, Tailscale, or a reverse proxy.
> - Invite URL composition was still deriving its origin from the
incoming request host, so loopback-bound servers emitted
`http://127.0.0.1:3100/invite/...`.
> - A loopback invite URL is not shareable with a remote human or agent,
even when the token itself is valid.
> - This pull request makes invite URL builders prefer the configured
public base URL and keep the existing request-host fallback when it is
unset.
> - The benefit is that copied invite links use the reachable deployment
origin without changing local-only behavior.

## Linked Issues or Issue Description

Fixes #7623

No duplicate or related PRs/issues were found in a GitHub search for
invite URL, loopback, public base URL, and `authPublicBaseUrl` terms.

## What Changed

- Added base URL resolution in `server/src/routes/access.ts` that strips
trailing slashes and prefers configured `authPublicBaseUrl` over the
request-derived host.
- Threaded `authPublicBaseUrl` through invite summary, invite onboarding
manifest, onboarding text, access routes, `createApp`, and server
startup wiring.
- Added `server/src/__tests__/invite-url-public-base-url.test.ts`
covering configured public-base precedence, unset fallback behavior, and
trailing-slash normalization.
- Registered the invite public-base URL test in the serialized Vitest
server runner.

## Verification

```bash
pnpm install --frozen-lockfile
pnpm exec vitest run server/src/__tests__/invite-url-public-base-url.test.ts
pnpm run test:run:serialized
```

Local results from the rebased PR branch:

- `pnpm install --frozen-lockfile` exited 0.
- Targeted invite URL test exited 0: 1 file, 3 tests passed.
- Serialized server suite exited 0: 106 serialized suites completed; the
new invite URL test passed inside that runner.

Manual check after deployment: set `PAPERCLIP_AUTH_PUBLIC_BASE_URL` or
equivalent public base URL config, create a company invite, and confirm
the returned/copied invite URL uses that public origin instead of
`127.0.0.1`.

## Risks

Low risk. The new public base URL parameter is optional and falls back
to existing request-derived behavior when unset. The main operational
risk is misconfigured public base URL input; the implementation only
trims trailing slashes and otherwise trusts the configured origin.

## Model Used

- Original implementation: Anthropic `claude-sonnet-4-6`, 200k context,
tool use and test execution.
- Conflict repair and verification: OpenAI Codex GPT-5.5, coding agent
with shell, git, GitHub CLI, and local test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Coder (Claude) <coder-claude@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Paperclip Coder (Claude) <lad-agent@paperclip.ing>
2026-08-12 16:44:28 -07:00
Nicky Leach e31951a17d
feat: Claude agent setup-token login in a sandbox (#11286)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Claude agents that run in a remote sandbox need a safe in-product
login path
> - The existing host login route cannot open a pseudo-terminal inside
that sandbox
> - The login flow must protect the browser code, the login URL, and the
OAuth token at every step
> - This pull request adds the parser, the runner, a Daytona
pseudo-terminal transport, and a guarded, owner-bound session route
behind an injectable transport
> - The route stays inert in the default build and fails closed until a
sandbox provider binds the live transport
> - The benefit is a company-scoped setup-token flow with one-time
secret delivery, redaction, and fail-closed transport checks, ready for
a later staged production rollout

## Linked Issues or Issue Description

**Agent or provider**

Claude Code setup-token login for sandbox agents.

**Why this adapter is useful**

Sandbox agents need a supported way to sign in without host credentials.
An authorized owner completes the browser step and receives the token
one time.

**How the agent is invoked**

When a sandbox provider binds the injectable transport, the server
starts `claude setup-token` through a sandbox pseudo-terminal, sends the
browser code to the matched prompt, and returns the token through the
guarded session route. The default build does not bind the transport. In
that state the start route fails closed with a fixed no-secret `503`. It
does not start a process and it does not hold a sandbox lease.

**Additional context**

The transport is injectable, so each sandbox provider binds its own
pseudo-terminal. This pull request adds the Daytona transport but does
not bind it in the production server. A production wiring needs a lease
manager, a live pseudo-terminal factory, a durable token store, and its
own security review. The route keeps secrets out of logs, activity
details, errors, telemetry, and non-owner responses.

## What Changed

- Add strict parsers for the setup-token URL, the prompt, and the
success token.
- Add a login runner that drives the `claude setup-token` command
through a pseudo-terminal.
- Add the Daytona pseudo-terminal transport and the sandbox plugin
wiring.
- Add a company-scoped, owner-bound login session service with rate
limits, a reaper, cleanup, and one-time token delivery.
- Add the guarded session routes at
`/agents/:id/setup-token-login-sessions/*` behind an injectable
transport. The routes become the live login path only when a provider
binds the transport.
- Keep the start route fail-closed in the default build. It returns a
fixed no-secret `503` and it does not bind `setupTokenLogin`.
- Keep the existing host route `POST /agents/:id/claude-login` in place.
This pull request does not replace it.
- Keep confidential responses behind a fail-closed TLS transport guard
with `Cache-Control: no-store`, and extend redaction for the new fields.
- Export the parser and the runner from the Claude local server entry,
and document the new session routes in the OpenAPI spec.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run setup-token-route
setup-token-session`
- `pnpm --filter @paperclipai/adapter-claude-local exec vitest run`
- `pnpm --filter @paperclipai/server run typecheck`
- Confirm that the pull request checks pass on GitHub.

## Risks

- Low user-facing risk on merge. The default build does not bind the
transport, so the production start route stays fail-closed with a `503`.
The merge does not change the production login behavior.
- When a provider later binds the transport, the flow starts a live
sandbox process and holds a short-lived in-memory secret. Cleanup must
stop the child before it releases the sandbox lease.
- The transport guard fails closed when the deployment does not provide
a trusted TLS path. A wrong proxy allowlist can block a valid request.
- The production wiring is out of scope. It needs a lease manager, a
live pseudo-terminal factory, a durable token store, and its own
security review before the server binds `setupTokenLogin`.

## Model Used

Anthropic Claude Opus 4.8 assisted the implementation. It used extended
reasoning, code execution, repository tool use, and a 200,000-token
context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (the
OpenAPI spec covers the new session routes; no user-facing documentation
needs changes)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-12 13:02:49 -07:00
Devin Foley 23a1b025c2
feat(server): chunked resumable company import transfers (#11223)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Company import moves large packages into an instance, and since the
upload cap rose to 1 GB, the transport is the weak point: one HTTP
request, buffered fully in memory, with no resume
> - A dropped connection at 90% of an 800 MB upload starts the whole
transfer over, and a server restart loses all progress
> - This pull request adds the server side of chunked resumable import
transfers: a durable run ledger and routes that accept the same import
zip as verified ~32 MB parts spooled to disk
> - An interrupted transfer resumes from the parts already uploaded —
across dropped connections, page refreshes, and server restarts — and
peak upload memory drops from the whole package to one part
> - The benefit is that large imports become reliable on real-world
connections instead of all-or-nothing

## Linked Issues or Issue Description

**What happened?**

Large company imports travel as a single HTTP upload. On a slow or flaky
connection, any interruption discards all progress and the upload
restarts from zero. The server buffers the entire compressed package in
memory during upload. A server restart mid-upload loses the transfer
entirely. With the upload cap now at 1 GB, these failure modes govern
exactly the imports the cap was raised for.

**Expected behavior**

A large import upload survives interruptions: already-transferred data
is kept and verified, only the missing remainder is re-sent, and the
server's memory use during upload is bounded by a part, not the package.

**Steps to reproduce**

1. Import a multi-hundred-MB company package over a connection that
drops mid-upload.
2. The upload fails; retrying starts from byte zero.
3. Repeat on an unstable connection and the import may never complete.

## What Changed

- New `company_transfer_runs` table (drizzle schema + migration) and
`companyTransferRunService`: one row per transfer with a content-derived
idempotency key, per-part completion recorded atomically and
idempotently, resume scoped to actor and direction, completed runs
short-circuiting retries of identical content.
- New transfer routes beside the existing import routes, same
authorization: declare a sliced zip (`POST /import/transfers` —
validates cap, 64 MB part ceiling, contiguity, size sums, sha256
format), upload parts (`PUT .../parts/:n` — raw body, hash-and-size
verified before an atomic write to a disk spool under the instance root;
re-uploads are no-op successes), poll resume state (`GET .../:id` —
missing parts recomputed from disk), and apply (`POST .../:id/apply` —
requires all parts, re-verifies the assembled zip against the whole-file
hash fail-closed, then feeds the existing import pipeline through
factored helpers rather than duplicated logic).
- Hourly sweep fails and cleans spools idle for 24 h; a swept transfer
honestly reports all parts missing on resume.
- Strict UUID gating on run ids before any filesystem path construction.
- The existing single-shot upload path is untouched; clients arrive in
the follow-up PR.

## Verification

- Transfer route suite (embedded Postgres): create/upload/status/apply
round-trip with a real imported company, out-of-order parts, wrong-hash
part rejected and unrecorded, re-upload no-op, apply-with-missing-parts
rejection, resume after failure with prior progress intact,
assembled-hash mismatch failing closed with spool deletion, actor
scoping 404s, async-job apply, sweep followed by honest resume.
- Ledger suite (embedded Postgres): part idempotency, actor/direction
scoping, completed-run short-circuit, cancelled runs staying cancelled.
- Existing portability route suite unchanged and green; server + db
typechecks clean. Exact counts in the PR checks.

## Risks

- New routes are additive; the existing import path is untouched. The
transfer routes carry the same board authorization as the import routes
they sit beside.
- Disk spool: bounded by the existing upload cap per transfer, cleaned
on success, failure, hash mismatch, and by the 24 h sweep. Spool paths
are strict-UUID-gated.
- The apply step still materializes the assembled zip in memory once
(same profile as today's single-shot import at apply time); upload-time
memory drops to one part.
- Known limitation, deliberate: transfers are keyed on content alone, so
identical package content cannot be imported twice without re-exporting
(surfaced explicitly to the caller). Acceptable for v1; noted for
review.

## Model Used

- Claude Fable 5 (`claude-fable-5`) via Claude Code CLI, with extended
thinking and tool use (multi-agent implementation with independent
verification).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-11 15:25:07 -07:00
Dotta 5858ccb981
feat: make in-app features cloud-aware (#10850)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Operators use the same board application in self-hosted and
Paperclip Cloud deployments.
> - A Cloud tenant contains one company, so an in-app company switch
does not change the active Cloud stack.
> - Cloud operators need the sidebar and company surfaces to use the
signed-in user's stack portfolio.
> - The server must derive Cloud identity and links from trusted
instance context instead of client input.
> - This pull request adds canonical Cloud context, a trusted stack
portfolio proxy, and Cloud-aware navigation.
> - The benefit is consistent stack switching on Cloud while self-hosted
company behavior stays unchanged.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: server REST routes and the React board UI.

**Problem or motivation**

A Cloud-managed instance contains one company. The existing company
switcher could only switch records inside that tenant. It could not move
the operator to another Cloud stack. The existing header also gave long
organization names too little width.

**Proposed solution**

Expose a canonical public Cloud context in health data. Add a trusted
server proxy for the current user's stack portfolio. Use that data in
the board UI to switch stacks with top-level navigation. Keep the
existing company behavior on self-hosted instances. Move search into the
navigation and keep long organization names inside the sidebar panel.

**Alternatives considered**

An in-app `/stacks` route was rejected because Cloud tenant hosts
reserve that path and stack selection must wake or authenticate another
tenant. Client-supplied user identity was rejected because the server
can derive the trusted Cloud actor.

**Roadmap alignment**

This change advances the Cloud deployments milestone. It keeps the
product local-first and Cloud-ready without changing the self-hosted
mental model.

## What Changed

- Added canonical Cloud instance context and public health metadata.
- Added a Cloud-only stack portfolio proxy with trusted actor forwarding
and per-user caching.
- Prevented normal company creation on Cloud-managed instances.
- Switched the sidebar and Companies page from company actions to stack
actions on Cloud.
- Added full-page stack navigation and Cloud create-stack links.
- Moved search into the sidebar navigation so the organization name
keeps more width.
- Added truncation and hover recovery for long organization and stack
names.
- Added server and UI regression coverage for Cloud and self-hosted
behavior.
- Updated the implementation specification for the Cloud contracts.

## Verification

- `node scripts/check-token-gates.mjs` passed. All three token gates are
clean.
- `pnpm --dir server exec vitest run src/__tests__/health.test.ts
src/__tests__/cloud-instance.test.ts src/__tests__/cloud-routes.test.ts
src/__tests__/company-cloud-floor.test.ts
src/__tests__/company-portability-routes.test.ts` passed: 5 files and 66
tests.
- `pnpm --dir ui exec vitest run
src/components/SidebarCompanyMenu.test.tsx` passed: 1 file and 11 tests.
- Pre-PR QA report `7da87ca7` passed all 8 acceptance criteria with real
HTTP route factories and real Chromium screenshots in Cloud and
self-hosted modes.
- Security reviews passed for the canonical Cloud context and stack
portfolio proxy.

## Risks

- Cloud stack switching depends on the configured Cloud application and
tenant portfolio URLs.
- The new health `cloud` block is public by design, but it contains only
canonical public instance metadata.
- The stack proxy fails closed on self-hosted instances and derives the
user identity from the trusted actor.
- Self-hosted navigation and company creation retain their existing
paths and behavior.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, model `gpt-5`. The run used reasoning, repository tools,
shell execution, and GitHub integration. The deployment did not expose
its context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-04 23:00:14 -05:00
Dotta 30c49c8327
feat(decisions): add queues and prioritized attention feed (#10651)
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100): short sentences, one instruction per sentence, simple
approved vocabulary, and the active voice. -->

## Thinking Path

> - Paperclip is the open source control plane for companies of AI
agents.
> - Operators use the attention feed to find decisions that need action.
> - The feed has eleven source kinds, but it has no durable queue or
triage state.
> - The feed also returns every item and lacks decision deadlines,
snooze state, and decision-focused ordering.
> - This pull request adds secure queue sidecars and enriches the
attention feed with triage data, filters, cursor pagination, and
decide-now ranking.
> - The benefit is a bounded feed that can show the most urgent
decisions first without weakening source visibility rules.

## Linked Issues or Issue Description

This pull request replaces the closed
[#10634](https://github.com/paperclipai/paperclip/pull/10634). It
combines that queue foundation with the dependent attention-feed change
as one review unit.

**Subsystem affected**

Database schema, shared contracts, server authorization and REST APIs,
and the UI attention client library.

**Problem or motivation**

The attention feed can contain hundreds of mixed decision items.
Operators cannot group them into durable queues, set a decision
deadline, snooze an item, or request a bounded page ordered by urgency.
The current client must download the full feed on each refresh.

**Proposed solution**

Store queue membership and triage state by stable attention identity.
Re-authorize each source during queue reads and writes. Enrich attention
items with queue, deadline, snooze, expiry, rule, and origin data. Add
activity and queue filters, opaque cursor pagination, decide-focused
ordering, and a decide-now count.

**Alternatives considered**

Adding queue fields to every source would duplicate schema and
authorization logic across eleven source kinds. Client-only filtering
and sorting would still transfer the full feed and would make pagination
unstable.

**Roadmap alignment**

This change improves the core decision-attention surface and operator
oversight. It does not implement the separate general-purpose work queue
milestone in `ROADMAP.md`.

## What Changed

- Added company-scoped queue, membership, triage, and append-only event
tables with actor and run provenance.
- Added queue CRUD, item membership, starter-rule discovery, and
decide-by and snooze endpoints.
- Kept source authorization on each queue mutation, read, and count.
- Added attention fields for expiry, rule, origin agent, queues,
decide-by attribution, and snooze state.
- Added activity date filters, queue filters, opaque cursor pagination,
and configurable page limits.
- Added decide-now ordering by deadline, expiry, severity, and activity.
- Added `decideNowCount` and excluded actively snoozed items from the
default feed.
- Updated the shared and UI client contracts.
- Added focused server, route, OpenAPI, and UI client tests.

## Verification

- `pnpm exec vitest run server/src/__tests__/attention-service.test.ts
server/src/__tests__/decision-queues-routes.test.ts
server/src/__tests__/openapi-routes.test.ts ui/src/api/attention.test.ts
ui/src/lib/attention.test.ts` (72 tests passed)
- `pnpm --filter @paperclipai/db check:migrations`
- `pnpm -r --filter @paperclipai/db --filter @paperclipai/shared
--filter @paperclipai/server --filter @paperclipai/ui typecheck`
- `git diff --check origin/master...HEAD`

## Risks

- The migration adds four company-scoped tables and provenance foreign
keys. Migration numbering and safety checks pass.
- Attention reads can lazily create starter queues and memberships.
Inserts are idempotent, audited, and transactional.
- Cursor validity depends on the filtered feed. The API returns a clear
validation error when the cursor item no longer exists in that feed.
- Queue reads re-check source visibility. This favors correct
authorization over fewer queries.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, model `gpt-5`. The runtime used agentic reasoning,
repository tools, code execution, and test execution. The runtime did
not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-01 20:37:39 -05:00
Dotta 9c1f8e7887
feat(decisions): add first-class propose mode (#10010)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents can currently perform many mutations directly, while humans
often need a durable review point before cross-issue or destructive
actions occur
> - Existing approvals and issue-thread interactions do not provide a
standalone, reusable object for presenting options, collecting typed
inputs, detecting stale targets, and auditing effect execution
> - The control plane therefore needs a first-class propose mode that
separates an agent's recommendation from the governed mutation it may
cause
> - This pull request adds Decisions v1 across the database, shared
contracts, server execution and telemetry, agent skill guidance, and
operator UI
> - The benefit is that agents can propose multi-option actions safely
while operators get explicit provenance, fail-closed execution,
per-effect results, and a focused attention workflow

## Linked Issues or Issue Description

### Subsystem affected

Cross-cutting: `packages/db`, `packages/shared`, `server`, and `ui`.

### Problem or motivation

Agents need a governed way to propose consequential work without
immediately mutating issues, especially when one choice can affect
several issue trees. Existing approvals and issue-thread interactions do
not provide a standalone object with typed options, target snapshots,
effect-level authorization, expiration, execution outcomes, and reusable
attention-feed presentation.

### Proposed solution

Add first-class Decisions that store options and typed inputs, surface
open proposals in the operator attention feed, validate target freshness
and the origin-agent/operator authorization intersection at decision
time, execute a bounded set of auditable effects, and retain terminal
outcomes. Decisions v1 supports comments, status and assignee changes,
follow-up issue creation, blocker resolution, and issue-tree
cancellation, plus bundle grouping, expiration/dismissal, rule-key
telemetry, and agent-facing API guidance.

### Alternatives considered

- Extend approvals with arbitrary effects: rejected because approvals
represent governed yes/no actions and would become an unsafe generic
mutation envelope.
- Model every proposal as an issue-thread interaction: rejected because
decisions can span several targets and need independent lifecycle,
telemetry, idempotency, and effect results.
- Let agents perform the mutation and ask for retrospective review:
rejected because it removes the pre-execution governance boundary this
feature is meant to provide.

### Roadmap alignment

Aligns with `ROADMAP.md` sections **Agent Reviews and Approvals**,
**Enforced Outcomes**, **MCP Tool Gateway & Apps (governed tool
access)**, and **Activity History** by making explicit decisions,
authorization gates, auditable execution, and terminal outcomes
first-class control-plane objects.

### Additional context

This does not replace existing approvals or issue-thread interactions,
and it does not add an unrestricted generic mutation effect.

## What Changed

- Added company-scoped decision, option, target, and effect-execution
schema plus migration and shared TypeScript/Zod contracts.
- Added decision routes and services for propose, list/get, decide,
dismiss, cancel, target freshness checks, authorization intersection,
idempotency, activity logging, and execution auditing.
- Added rule-key decision telemetry and attention-feed metadata so open
decisions are visible and measurable.
- Added agent skill documentation for proposing and resolving decisions
through the Paperclip API.
- Added the Decisions UI: API client, query keys, inline attention
resolver, bundle grouping, target-issue strip, terminal history,
destructive confirmation, and per-effect result rendering.
- Added server service coverage, DecisionCard state tests, and Storybook
stories for the supported visual states.

## Verification

- `pnpm -r typecheck` — passed.
- `pnpm test:run` — 2,876 passed, 1 skipped, with one unrelated
cross-suite cleanup-order failure in
`heartbeat-responsible-user-invariant.test.ts`; the failing file passes
in isolation (`6/6`).
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-responsible-user-invariant.test.ts` — passed.
- `pnpm --filter @paperclipai/ui exec vitest run
src/components/DecisionCard.test.tsx` — passed (`9/9`).
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/authz-existence-oracle-guard.test.ts
src/__tests__/openapi-routes.test.ts` — passed (`5/5`).
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/decisions-service.test.ts` — passed (`16/16`).
- `pnpm --filter paperclipai exec vitest run
src/__tests__/company-import-export-e2e.test.ts` — passed (`1/1`).
- `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter
paperclipai typecheck` — passed.
- `pnpm build` — passed.
- Rebased-head focused suite — passed (`6` files, `88` tests): shared
decision contracts, Decisions service, OpenAPI routes, startup feedback
export, DecisionCard states, and attention helpers. The follow-up
stale-secondary-target regression passes in the DecisionCard suite
(`10/10`).
- Rebased-head scoped typechecks — passed for `@paperclipai/shared`,
`@paperclipai/db`, `@paperclipai/server`, and `@paperclipai/ui`.
- Rebased-head migration numbering and safety checks — passed after
renumbering the additive migration to `0193` and making it replay-safe
for environments that applied the earlier feature-branch number.
- `pnpm check:token-gates` — passed with all gates clean.
- GitHub PR workflow and Greptile review for
`1f9f7645882d05dfdd9c99377c03a1f53f20e8be` — running after the
stale-secondary-target fix and PR metadata refresh on July 27, 2026.
- `pnpm --filter @paperclipai/ui build-storybook` exposes an existing
Storybook version mismatch (`storybook` 10.4.6 vs
`@storybook/addon-docs` 10.5.0); Decisions stories were validated with
the docs addon temporarily disabled and the tracked config remains
unchanged.

## Risks

- **Migration:** Adds replay-safe migration `0193`; migration numbering
and safety checks pass. The new tables and indexes are additive.
- **Authorization:** Effect execution intersects the proposing agent's
permissions with the responsible user context and fails closed; mistakes
could reject a valid proposal rather than silently over-authorize it.
- **Concurrency:** Target snapshots and idempotency keys protect against
stale or duplicate execution, but reviewers should focus on mixed-effect
partial outcomes and retry behavior.
- **UI:** Decisions are integrated into the existing attention feed
rather than a separate navigation surface, reducing routing risk but
increasing the importance of attention-item metadata compatibility.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI using `gpt-5.6-sol` for final PR preparation, review
fixes, and verification; repository tools and code execution were
enabled, and context-window size is not exposed in this runtime.
- Anthropic Claude Opus 4.8 with 1M context assisted with the Decisions
UI implementation, as recorded in the relevant commits.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-31 19:17:02 -07:00
Devin Foley 78f8c6c3d4
Recover managed bundled plugin workers on demand (#10429)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Managed deployments can auto-provision bundled sandbox-provider
plugins so cloud or remote execution environments appear in the board UI
> - In a multi-service deployment, several server processes can share
one database and boot concurrently
> - A sibling process can create a bundled plugin row while the web
process sees it before it reaches `ready`
> - The web process correctly avoids clobbering the existing row, but
its startup `loadAll()` can miss the plugin and never start that worker
locally
> - The environments capabilities route then filters out the sandbox
provider because the plugin is ready in the database but not running in
the web process
> - This pull request adds a narrow managed-bundle recovery path that
lazily starts the missing worker when the capabilities route sees a
ready managed bundled plugin
> - The benefit is that the sandbox provider becomes visible after the
install finishes, without requiring a web-process restart

## Linked Issues or Issue Description

- No public GitHub issue found for this exact deployment race.
- Related broad plugin runtime context: Refs #432.

Bug description:

- What happened: in a managed multi-service deployment with shared
database state and bundled plugin auto-install enabled, the API-serving
process can skip a plugin row while it is still `installed`, run startup
plugin loading before that row becomes `ready`, and then permanently
omit the sandbox provider from environment capabilities.
- Expected behavior: once the managed bundled plugin row reaches
`ready`, the API-serving process should be able to start the plugin
worker and include its sandbox provider without a restart.
- Steps to reproduce: boot a web process and a sibling worker process
concurrently; have the sibling create the bundled plugin row and
transition it to `ready` after the web process has already skipped
auto-install and run `loadAll()`.
- Deployment mode: managed multi-service deployment with shared database
state and `plugins.autoInstall` configured.

## What Changed

- Added a managed bundled plugin worker recovery helper that
single-flights lazy `loadSingle()` starts and only allows configured
managed bundled plugin keys.
- Passed the managed recovery hook into the environments capabilities
route.
- Updated `listReadyPluginEnvironmentDrivers()` to attempt bounded
recovery for ready managed bundled plugins whose worker is missing in
the current process, and only for plugins that actually declare a
`sandbox_provider` environment driver.
- Made request-time recovery use `loadSingle(id, { markErrorOnFailure:
false })` so a local activation failure in one process never transitions
the shared plugin row to `error` (a sibling process may be running the
plugin successfully).
- When error writes are suppressed and activation fails after the worker
was spawned, the loader now tears down the partially-registered local
runtime (scheduler registration, event subscriptions, agent tools,
worker process) instead of leaving a half-activated worker lingering;
the teardown steps are factored out of `unloadSingle()` into a shared
helper.
- A failed recovery attempt now discards the crashed/stopped handle it
left registered in the worker manager (a worker that dies during
initialize is killed without a scheduled restart), so later capability
requests can retry recovery instead of being blocked by the
handle-presence gate until a process restart. Handles in
starting/running/backoff states are left to the worker manager's own
lifecycle; recovery only ever starts when no handle existed, so no
pre-existing worker can be affected.
- Added a regression test suite covering the installed-to-ready race,
allowlist behavior, the driver-kind gate, existing worker handles,
concurrent single-flight recovery, bounded slow recovery attempts,
suppressed shared error-state writes, partial-runtime teardown on late
activation failure, and retry after a dead handle is discarded.

## Verification

- `pnpm vitest run
src/__tests__/plugin-environment-driver-ready-recovery.test.ts` (in
`server/`) passed: 10 tests.
- `pnpm --filter @paperclipai/server typecheck` passed.

## Risks

- Low risk for self-hosted single-process deployments because lazy
recovery is only wired when managed plugin auto-install config is
present; with no managed config the capabilities route takes the exact
pre-change code path.
- The capabilities route can wait briefly while attempting recovery; the
attempt is bounded and defaults to 2 seconds.
- Failed recovery keeps the prior behavior of omitting the provider
until a later successful worker start, and now also cleans up any
partially-started local worker so retries begin from a clean slate.

## Model Used

- Initial implementation: OpenAI GPT-5 via Codex local coding agent,
with repository tool use and command execution.
- Review-feedback follow-ups (driver-kind gate, partial-runtime
teardown, expanded regression tests): Claude Fable 5 (claude-fable-5)
via Claude Code, with repository tool use and command execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-29 12:50:16 -07:00
Devin Foley 273315a4d0
feat(server): provision managed sandbox environments from the managed config (#10324)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted/managed deployments configure instances entirely from the
control plane: `PAPERCLIP_MANAGED_CONFIG` already delivers feature flags
and `plugins.autoInstall` (bundled sandbox provider plugins), parsed
fail-closed at boot
> - A plugin alone is not usable for execution: runs need an
instance-level `driver: "sandbox"` environment row pointing at the
provider, and today only Kubernetes has a boot path for that
(`PAPERCLIP_EXECUTION_MODE` → `ensureKubernetesEnvironment`); every
other provider requires a manual product-API call the control plane
cannot make on a managed instance
> - Adding one `ensureXxxEnvironment` per provider would multiply
near-identical boot hooks and env-var surfaces
> - This pull request generalizes the existing Kubernetes machinery: the
managed-config document gains an optional `environments` section that
declares a sandbox environment for any bundled provider, ensured
idempotently at boot by a provider-agnostic service function (the
Kubernetes hook becomes a thin wrapper over it)
> - The benefit is that a managed fleet can provision any sandbox
provider (Daytona, Modal, E2B, …) purely from configuration — no
per-provider code, no manual API calls, no secrets in the document —
while self-hosted behavior is untouched

## Linked Issues or Issue Description

No public issue exists. Refs #10157 (the cloud image variant that
bundles sandbox provider plugins — this PR is the configuration half
that makes an installed provider usable).

**Problem (feature-request shape):** on a managed instance the control
plane can auto-install a bundled sandbox provider plugin via
`PAPERCLIP_MANAGED_CONFIG.plugins.autoInstall`, but cannot create the
environment row that makes the provider schedulable. The only boot-time
environment provisioning is Kubernetes-specific
(`PAPERCLIP_EXECUTION_MODE=kubernetes` + `PAPERCLIP_K8S_*`). A generic,
config-driven path is needed so any bundled provider can be provisioned
without per-plugin code or manual API calls.

## What Changed

- `server/src/services/managed-config.ts`: optional `environments`
top-level section — `[{ name, description?, provider, config? }]` —
validated fail-closed: unknown keys, more than one entry (the DB permits
exactly one Paperclip-managed sandbox row,
`environments_managed_sandbox_idx`), a `provider` not present in
`plugins.autoInstall`, `config.provider`, or secret-looking config keys
at any depth (`api_key`/`token`/`secret`/`password`/`credential`) all
refuse startup. Absent section ⇒ `environments: []`, so pre-section
documents keep booting newer builds.
- `server/src/services/environments.ts`: new provider-agnostic
`ensureManagedSandboxEnvironment({ name, description?, provider,
config?, extraMetadata? })` — idempotently owns the single managed
sandbox row: refreshes name/description/config each call, adopts the
slot across provider switches (dropping the stale
`managedKubernetesSandbox` marker), adopts a same-name unmanaged sandbox
row (stamping it managed) instead of colliding on
`environments_name_idx` every boot, and falls back to keeping the
current name if the desired name belongs to a different row.
`ensureKubernetesEnvironment` is now a thin wrapper that pins `provider:
"kubernetes"` and stamps the legacy marker.
- `server/src/services/managed-environments.ts` (new):
`applyManagedEnvironments` boot step — no-op for self-hosted/empty;
throws (fail startup) when `PAPERCLIP_EXECUTION_MODE` is also set, since
both would own the same managed sandbox row; otherwise ensures each
declared environment fail-safe per entry (log + continue boot, matching
bundled-plugin provisioning posture).
- `server/src/index.ts`: runs the new boot step right after the
execution-policy bootstrap, before the heartbeat resumes queued runs.
- `server/src/services/index.ts`: exports `applyManagedEnvironments` and
`ManagedEnvironmentSpec`.
- Secrets stay out of the document by construction: provider credentials
reach managed instances only as process env vars (each provider's
documented fallback, e.g. `DAYTONA_API_KEY` for the Daytona plugin).

## Verification

```sh
cd server
pnpm exec tsc --noEmit -p tsconfig.json
pnpm exec vitest run \
  src/__tests__/managed-config.test.ts \
  src/services/managed-environments.test.ts \
  src/services/execution-policy-bootstrap.test.ts \
  src/__tests__/environment-service.test.ts \
  src/__tests__/environment-instance-routes.test.ts \
  src/__tests__/environment-routes.test.ts \
  src/__tests__/plugin-install-guard.test.ts \
  src/__tests__/environment-execution-target.test.ts \
  src/__tests__/instance-settings-managed-overlay.test.ts \
  src/__tests__/bundled-plugins.test.ts
```

All pass locally (typecheck clean; environment-service suite runs
against embedded Postgres and exercises the refactored Kubernetes
wrapper plus the new generic ensure: create/refresh, provider switch,
unmanaged-row adoption, name-conflict fallback). New tests cover the
parser (12 cases incl. secret-key rejection at depth) and the boot step
(no-op, mutual exclusion, pass-through, fail-safe).

## Risks

- **Self-hosted: none intended.** Without `PAPERCLIP_MANAGED_CONFIG`
nothing new executes; the `PAPERCLIP_EXECUTION_MODE=kubernetes` path is
regression-covered by the existing bootstrap/service/route suites (all
green).
- **Behavioral shift in `ensureKubernetesEnvironment` (deliberate):** it
now also refreshes `name`/`description` to their managed defaults each
boot (desired-state semantics, same as config today) and adopts a
`managedByPaperclip` sandbox row that lacks the Kubernetes marker —
previously that state made the ensure throw every boot.
- **New startup failure modes are all explicit misconfigurations**
(malformed section, provider not auto-installed, secret in config,
execution-mode conflict) and fail with precise errors; DB-side ensure
failures never block boot (fail-safe per entry, logged).
- No migrations; no API surface changes.

## Model Used

Claude Fable 5 (Anthropic, model ID `claude-fable-5`) with extended
thinking and tool use, driving the change end-to-end inside a Claude
Code / agent-harness session (code, tests, and verification runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (the
managed-config module header is the contract doc)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-27 12:00:21 -07:00
Dotta 7e40ed8c43
feat(status-cards): add experimental status card update view (#10101)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies.
> - Operators need a board-level way to monitor a changing slice of
company work without repeatedly rebuilding filters or reading raw task
threads.
> - Existing summaries are useful snapshots, but they do not provide a
dedicated query-backed card with refresh policy, change tracking, update
history, and per-update cost visibility.
> - The capability needs to be safe to evaluate before it becomes part
of the default product surface.
> - This pull request adds end-to-end experimental Status Cards, from
schema and query compilation through update orchestration and operator
UI.
> - The entire feature is gated behind the `enableStatusCards`
experimental toggle, including its route and sidebar entry.
> - The benefit is a governed, inspectable way to keep focused
operational rollups current while preserving explicit controls over
refresh frequency and spend.

## Linked Issues or Issue Description

### Subsystem affected

Cross-cutting (`packages/db`, `packages/shared`, `server/`, `ui/`, and
bundled skills/docs).

### Problem or motivation

Operators cannot currently define a reusable natural-language view of
company work, compile it into an inspectable query, and keep its summary
current as matching issues change. Rebuilding filters and rereading task
threads makes board-level monitoring repetitive and hides the
relationship between source changes, refresh cost, and the resulting
summary.

### Proposed solution

Add experimental Status Cards that compile operator intent into a query,
summarize matched work, record each update, expose
manual/interval/reactive refresh policies and costs, and preserve the
last good result across stale, updating, paused, and error states. The
capability is off by default and fully gated behind `enableStatusCards`,
including its route and navigation entry.

### Alternatives considered

- Extend existing one-off summaries: rejected because status cards
require persistent query provenance, refresh policy, update history, and
card-specific cost controls.
- Add a dashboard-only filter widget: rejected because it would not
provide governed background refresh, an update ledger, or an inspectable
compile pipeline.
- Ship the surface by default: rejected in favor of an experimental
toggle while behavior and operator value are evaluated.

### Roadmap alignment

This advances Paperclip’s board-level execution visibility and
output-first product goals. `ROADMAP.md` was checked and no duplicate
status-card initiative was found.

### Additional context

No related open PR was found in the public GitHub search for status
cards. The PR-only design wireframes were removed from the repository
after review; the published prototype remains external to the production
source tree.
## What Changed

- Added company-scoped status-card schema, CRUD APIs, compile
provenance, update ledger, shared contracts, validators, and OpenAPI
coverage.
- Added the text-to-query compile pipeline, bundled `status-card-query`
agent skill, query versioning, and authorized write-back flow.
- Added the experimental board, create flow, lifecycle tiles,
detail/settings/debug drawers, archived view, routing, navigation, and
instance setting.
- Added a change-gated update engine with manual, interval, and reactive
refresh policies, trigger selection, active hours, and daily token caps.
- Added per-update token/cost recording, today and lifetime rollups, and
policy-derived cost previews.
- Added operator documentation and agent-authoring hardening for compile
and update behavior.
- Added PR-prep integration coverage for settings/startup wiring and
replaced raw UI values with design-system tokens.
- Removed the PR-only `design/pap-15023-status-cards` wireframe
artifacts so the repository contains only production feature assets.

## Verification

- `pnpm -r typecheck` — passes on the PR head; includes `ui` `tsc -b`
passing. The UI compile gate was also independently recorded as passing
at `6d7f3cf96b` on July 23, 2026.
- `pnpm build` — passes.
- `pnpm check:token-gates` — passes with all three gates clean.
- `pnpm test:run` — 2,880 tests passed and 1 skipped; the sole failure
was an unrelated 10-second `afterAll` database-cleanup timeout in
`execution-workspaces-service.test.ts`.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/execution-workspaces-service.test.ts` — passes on
immediate focused rerun (25/25).
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/instance-settings-service.test.ts
src/__tests__/server-startup-feedback-export.test.ts` — passes (31/31).
- `pnpm --filter @paperclipai/ui exec vitest run
src/pages/StatusCards/StatusCardSettingsForm.test.tsx
src/pages/StatusCards/StatusCardTile.test.tsx
src/pages/StatusCards/format.test.ts src/lib/status-card-state.test.ts`
— passes (26/26).
- Recorded pre-PR QA: compile-pipeline e2e PASS; full lifecycle and cost
QA PASS; security re-review PASS after write-back hardening; UX
approved.
- `pnpm exec vitest run packages/db/src/status-card-migrations.test.ts`
— passes; reapplies migrations `0185`–`0189` against an already-migrated
embedded Postgres database.
- `pnpm --filter /db check:migrations` — passes migration numbering and
safety checks.
- `pnpm --filter /db typecheck` — passes.
- Merged current `origin/master` on July 24, 2026 with no conflicts;
migrations `0185`–`0189` remain unclaimed on master.

## Risks

- The feature introduces five database migrations and a new background
update path; all new DDL is repeat-safe after partial application,
migration numbering/safety checks pass, and update execution is
company-scoped and change-gated.
- Natural-language compilation can produce invalid or overly broad
queries; compile provenance, query validation, debug visibility, and
version history make failures inspectable and recoverable.
- Reactive or interval refresh could increase spend; active hours, max
refresh frequency, daily token caps, per-update cost records, and
budget-paused states bound and expose that risk.
- The branch name contains an internal execution identifier because it
is a fixed handoff branch; it was intentionally not renamed or rebased
per the release handoff instructions.
- Overall rollout risk is limited because the route, navigation,
services, and UI are disabled by default behind `enableStatusCards`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using GPT-5.5 with reasoning, repository tool use, shell
execution, GitHub CLI, and test/build execution. The runtime did not
expose a context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change; the fixed execution-workspace
identifier is documented as an authorized handoff exception
- [x] I have run tests locally and they pass, with the one cleanup
timeout passing on focused rerun
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-24 12:26:43 -05:00
Devin Foley c4cdcc4826
Generalize bundled plugin provisioning: `ensureBundledKubernetesPlugin` → `ensureBundledPlugins` (#10063)
**Builds on** #10058 — reads `plugins.autoInstall` from the parsed
managed-config contract #10058 introduces (the interim
`readManagedPluginAutoInstall` shim is retired at rebase).

**Summary.** Boot-time bundled-plugin provisioning becomes
catalog-driven. A new bundled-plugin catalog lists the sandbox providers
shipped in-tree (keys like `kubernetes`, `daytona` → plugin key + path
under the catalog root). Managed instances read `plugins.autoInstall`
from `PAPERCLIP_MANAGED_CONFIG`; unknown keys or paths escaping the
catalog root (symlinks resolved) **throw before listen** — a managed
instance refuses to start rather than boot half-provisioned.
Installation keeps today's mechanism: an in-process, fail-safe
`loader.installPlugin({ localPath })` under a system actor — no HTTP
route, no user, no role widening. Self-hosted boot is unchanged
(kubernetes bundle only, existing env override honored, install failures
still log-and-continue).

**Semantics.** A plugin already present in any non-uninstalled state is
skipped, so an operator-disabled plugin is never silently re-enabled;
managed mode reinstalls soft-uninstalled bundles (the control plane owns
provisioning); removal from the autoInstall list never auto-uninstalls.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox-provider plugins ship in-tree, but boot-time provisioning is
hard-coded to exactly one of them (Kubernetes) via a bespoke function
> - On managed hosting, tenant users have no install privileges, so any
bundled plugin that is not provisioned at boot is unusable
> - Widening install routes or granting roles to fix that would trade a
provisioning gap for a security regression
> - This pull request generalizes the existing boot installer into a
catalog-driven `ensureBundledPlugins`, fed by `plugins.autoInstall` from
`PAPERCLIP_MANAGED_CONFIG`
> - The benefit is that managed tenants get working bundled plugins out
of the box, through the same in-process, role-free mechanism the
codebase already trusts, while self-hosted boot is unchanged

## Linked Issues or Issue Description

No public issue exists; `feature_request` template fields:

- **Problem or motivation:** on managed instances tenant users cannot
install plugins (by design they never hold instance admin), so even
plugins shipped with the product are unusable; boot provisioning
currently knows only the Kubernetes bundle.
- **Proposed solution:** a bundled-plugin catalog plus
`ensureBundledPlugins(keys)` driven by the managed config; same
in-process `loader.installPlugin({ localPath })` under a system actor;
unknown keys or catalog-escaping paths fail startup; already-present
plugins are skipped so operator-disabled plugins are never silently
re-enabled.
- **Alternatives considered:** granting tenant users install privileges
(widens secrets/adapters/settings access to solve a one-button problem);
a separate non-admin install route for bundled plugins (new authz
surface; provisioning removes the need for any install action at all).
- **Roadmap alignment:** supports the in-progress "Cloud deployments"
milestone and builds on the shipped sandbox-provider milestone in
`ROADMAP.md`.

Refs #10058.

## What Changed

- New `server/src/services/bundled-plugins.ts`: the bundled-plugin
catalog, the fail-to-start resolver (`resolveBundledPluginInstalls`,
positive allowlist + catalog-root containment with symlinks resolved),
and the fail-safe installer (`ensureBundledPlugins`).
- `server/src/app.ts`: replaces the hard-coded
`ensureBundledKubernetesPlugin` boot hook with resolver + installer
wiring, with test hooks (`managedPluginAutoInstall`,
`bundledPluginCatalogRoot` options).
- `server/src/index.ts`: passes `plugins.autoInstall` from the single
fail-closed `PAPERCLIP_MANAGED_CONFIG` startup parse (#10058) into
`createApp`; absent env means self-hosted and changes nothing.

## Verification

- 24 new tests in `server/src/__tests__/bundled-plugins.test.ts`
(catalog resolution, containment incl. symlink and `..` escapes,
skip/reinstall matrix, self-hosted invariants, installer error paths) —
all green.
- 85 adjacent startup/plugin-route/auto-build/managed-config tests green
(`managed-config`, `instance-settings-managed-overlay`,
`plugin-install-autobuild`, `plugin-routes-authz`,
`server-startup-feedback-export`).
- Server `tsc --noEmit` clean.

```bash
cd server
npx vitest run src/__tests__/bundled-plugins.test.ts
npx vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts src/__tests__/plugin-install-autobuild.test.ts src/__tests__/plugin-routes-authz.test.ts src/__tests__/server-startup-feedback-export.test.ts
npx tsc --noEmit
```

## Risks

- Managed instances with a malformed or unknown `plugins.autoInstall`
entry now **refuse to start** (fail closed, by design) instead of
booting half-provisioned; harness misconfiguration surfaces as a precise
startup error.
- Self-hosted behavior is unchanged (kubernetes bundle only,
`PAPERCLIP_KUBERNETES_PLUGIN_PATH` honored without containment, install
failures log-and-continue), so the default deployment path carries low
risk.
- No uninstall path exists in this module; removal from the autoInstall
list can leave a previously provisioned plugin installed (intentional v1
semantics, documented in code).

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use;
independently peer-reviewed by a second AI agent before push.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 20:35:51 -07:00
Dotta a090c09ee5
feat: add decision training snapshot foundation (#9702)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies and their work
> - Human approvals, issue interactions, and execution decisions already
capture high-value decision moments
> - Those moments are currently transient and cannot be reused as stable
evaluation or training examples
> - Reusable examples need a server-owned, immutable snapshot so later
comments or runs cannot leak into the recorded state
> - Human notes need to remain editable and auditable without changing
the captured state
> - This pull request adds the database model, snapshot capture service,
API, export format, and attention-feed enrichment for decision training
> - The benefit is a durable, inspectable foundation for evaluating
whether agents can reproduce good human decisions from only the context
available at decision time

## Linked Issues or Issue Description

### Subsystem affected

Cross-cutting (`server/`, `packages/db`, and `packages/shared`).

### Problem or motivation

Paperclip has no durable dataset for converting human decisions into
evaluation-ready examples. Teams need to capture pending or resolved
decisions with the exact issue context, comments, runs, and repository
evidence available at a cutoff, while preventing future context from
leaking into the example.

### Proposed solution

Store immutable, schema-versioned snapshots anchored to durable
interaction, approval, or execution-decision records; keep notes
separately editable with history; expose human-only CRUD, list, and
JSONL export APIs.

### Alternatives considered

Client-generated snapshots were rejected because they duplicate cutoff
logic and cannot reliably enforce no-leakage boundaries. Automatic
outcome backfill was deferred so captured examples remain faithful to
what was known at capture time.

### Roadmap alignment

Supports the roadmap direction of turning completed work and decision
patterns into reusable organizational knowledge.

### Additional context

The implementation records explicit commit-resolution confidence
(`exact`, `nearest_run`, `workspace`, or `none`) so downstream
evaluation can distinguish evidence quality.

## What Changed

- Added the `decision_training_examples` schema and idempotent migration
with company, issue, and source/author indexes.
- Added shared types for decision-training records, notes history, and
versioned snapshots.
- Added a single server-side snapshot capture path with inclusive
comment cutoffs, pre-cutoff run capture, durable decision payloads, and
explicit commit-resolution confidence.
- Added create, list, detail, notes-only update, delete, and JSONL
export routes with human-only write authorization and activity logging
that skips no-op note submissions.
- Added per-user `trainingExampleId` enrichment to attention items.
- Added focused embedded-Postgres tests for cutoff boundaries,
post-cutoff leakage, immutable snapshots, human-only writes, duplicate
prevention, notes history, attention enrichment, and export shape.
- Updated UI test and Storybook attention-item factories for the new
required `trainingExampleId` contract.

## Verification

- `pnpm exec vitest run server/src/__tests__/decision-training.test.ts`
— 10 tests passed.
- `pnpm --filter @paperclipai/db typecheck` — passed, including
migration numbering and safety checks.
- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `pnpm --filter @paperclipai/ui typecheck` — passed.

## Risks

- The migration adds a new table and indexes only; it does not rewrite
existing rows or install resolve-time hooks.
- Snapshot JSON can grow with long comment threads and run histories; v1
intentionally favors complete, inspectable examples over aggressive
truncation.
- Commit SHA resolution is evidence-based and records `exact`,
`nearest_run`, or `none` so downstream consumers can account for
confidence.
- The API is additive, but future UI work must continue to treat the
snapshot as immutable and use notes-only updates.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using `gpt-5.3-codex`, with repository tool use, terminal
execution, and code-editing capabilities; context-window size is not
exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-17 12:17:34 -05:00
Dotta 1f1f545238
feat: add built-in summarizer and summary slots (#9713)
## Thinking Path

> - Paperclip is the open source control plane people use to organize,
govern, and understand AI-agent work
> - Operators need concise, current status views across projects and
execution workspaces without manually reading every issue and run
> - Paperclip already has auditable issues, documents, built-in agents,
routines, and live run events, but no first-class summary-slot workflow
connecting those systems
> - A built-in Summarizer can generate status prose through ordinary
governed tasks while summary slots provide stable, revisioned
destinations for that output
> - The UI needs to show current summaries, generation progress,
failures, revisions, and streaming draft status in the places operators
already work
> - This pull request adds the end-to-end summary-slot data, API, agent,
orchestration, and UI surfaces behind an experimental setting
> - The benefit is decision-oriented status context that remains
company-scoped, auditable, retryable, and inexpensive by default

## Linked Issues or Issue Description

No public GitHub issue exists for this feature.

**Problem**

Operators currently have to reconstruct project and workspace status by
reading many issues, runs, and comments. This makes it hard to identify
decisions, review queues, recent work, and the next event worth
watching.

**Proposed capability**

Add an experimental summary system with revisioned summary slots for
projects and workspaces, a paused-by-default built-in Summarizer agent,
governed generation tasks, live draft status, and reusable UI cards.

**Expected behavior**

- Summary data remains company-scoped and revisions remain auditable.
- Generation runs through normal issue/agent orchestration and
deduplicates active requests.
- Only the linked built-in Summarizer generation task can author a slot
revision.
- Operators can generate, retry, inspect revisions, and follow draft
progress from project and workspace views.
- The feature remains opt-in and background generation remains paused by
default.

## What Changed

- Added summary-slot schema, idempotent migrations, shared contracts,
validators, API paths, and service tests.
- Added company-scoped summary-slot routes for reading revisions,
requesting generation, and guarded Summarizer writes with activity
logging.
- Added terminal generation finalization, failure reasons, assignment
wakeups, and orchestration integration.
- Added the paused-by-default built-in Summarizer bundle, low-cost
runtime defaults, status-summarization skill, and stale-summary routine.
- Added summary cards, revision selection, retry/configuration states,
live draft streaming, transcript chunk handling, and project/workspace
integrations.
- Updated Claude local parsing for streamed status output and expanded
server, adapter, shared, database, catalog, and UI coverage.

## Verification

- `pnpm -r typecheck`
- `pnpm exec vitest run packages/db/src/summary-slots-schema.test.ts
packages/shared/src/summary-slot.test.ts
server/src/__tests__/summary-slot-routes.test.ts
server/src/__tests__/summary-slots.test.ts
server/src/__tests__/built-in-agents.test.ts
ui/src/components/SummarySlotCard.test.tsx
ui/src/components/SummarySlotCard.status.test.tsx
ui/src/components/useSummaryDraftStream.test.tsx
ui/src/lib/summary-draft-stream.test.ts
ui/src/lib/run-log-chunks.test.ts
ui/src/context/LiveUpdatesProvider.hook.test.tsx` — 113 tests passed
- `pnpm test:run` — server and UI suites passed; one CLI AWS doctor test
was affected by inherited `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`,
and passed when those host credentials were removed
- `pnpm exec vitest run cli/src/__tests__/secrets.test.ts` with
inherited AWS credential variables removed — 8 tests passed
- `pnpm build`
- `pnpm check:token-gates` currently reports nine `#9627` comment
references introduced by current `master`; none are in this PR diff

## Risks

- Database risk is limited by incrementally ordered, idempotent
migrations and migration safety checks.
- Summary generation creates normal issues/runs, so misconfiguration can
produce failed slots; the UI exposes retryable failure reasons and agent
configuration entry points.
- Streaming draft parsing depends on the documented `STATUS:` protocol;
final persisted revisions remain the source of truth.
- The feature is experimental, opt-in, and its built-in routine is
paused with no background token spend by default.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5.3 Codex with reasoning, repository tool use, code
execution, GitHub CLI, and Paperclip control-plane integration. Earlier
branch commits also record Claude model co-authorship where applicable.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 11:03:07 -05:00
Dotta 59fb27ff79
feat(inbox): let agents safely tidy user inboxes (#9724)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies and their work
> - The inbox is a per-user attention view, so archiving an item must
not alter the underlying issue, assignment, or status
> - Agents can help responsible users tidy resolved work only when the
action is company-scoped, reversible, policy-controlled, and fully
attributable
> - The database and authorization foundations landed in #9654 and
#9658, but the end-to-end archive routes, audit details, agent workflow
guidance, and operator UI still need to ship together
> - Separate stacked PRs #9659 and #9661 made the complete behavior
harder to review and land as one coherent capability
> - This pull request consolidates the remaining server,
shared-contract, documentation, skill, and UI work on top of current
master
> - The benefit is a single reviewable change that lets agents safely
archive responsible-user inbox items and lets users control or undo that
behavior

## Linked Issues or Issue Description

### Subsystem affected

Cross-cutting inbox management across shared contracts, server
authorization/routes/services, shipped agent skills, and the board UI.

### Problem or motivation

Agents may complete work whose issue remains in the responsible user's
Mine inbox. Existing board-user archive behavior does not provide the
agent-facing policy endpoints, target resolution, heartbeat-run
attribution, typed denials, conservative workflow guidance, or UI needed
for safe agent-managed cleanup.

### Proposed solution

Allow authorized agents to archive or unarchive responsible-user inbox
items under the user's open, allowlist, or disabled policy; preserve
actor/agent/run attribution in issue detail and activity records; expose
policy controls and agent archive attribution in the UI; and document
conservative cleanup rules for agents and PR gardening.

### Alternatives considered

- Reuse generic issue mutation permissions: rejected because inbox state
belongs to a target user and requires user-scoped authorization.
- Automatically archive every completed or closed item: rejected because
completion signals can still require human review or a decision.
- Keep the backend and UI as separate stacked PRs: superseded by this
consolidated PR so the complete user-visible behavior can be reviewed
and verified together.

### Related work

- Builds on merged foundations #9654 and #9658.
- Supersedes the remaining stacked changes in #9659 and #9661.
- `ROADMAP.md` has no overlapping inbox archive or inbox authorization
initiative.

## What Changed

- Added shared inbox-agent policy types and validators plus
company-scoped self-service policy routes and OpenAPI coverage.
- Enabled agent archive/unarchive mutations with responsible-user
targeting, policy enforcement, typed failures, attribution, idempotency,
and detailed activity auditing.
- Returned agent archive attribution in issue detail and documented
reversible inbox cleanup semantics in the implementation spec and
Paperclip skill.
- Added conservative PR-gardening inbox tidy guidance that keeps GitHub
access read-only and avoids archiving work that still needs human
action.
- Added the Profile settings policy control and Issue Properties
attribution/unarchive UI with focused component coverage and narrow-pane
handling.

## Verification

- `pnpm exec vitest run
server/src/__tests__/inbox-archive-routes.test.ts
server/src/__tests__/inbox-agent-policy-routes.test.ts
server/src/__tests__/authorization-service.test.ts
server/src/__tests__/openapi-routes.test.ts
ui/src/components/InboxAgentPolicyControl.test.tsx
ui/src/components/IssueProperties.test.tsx` — 110 passed.
- `pnpm --filter @paperclipai/db exec vitest run
src/inbox-archive-agent-policies-migration.test.ts` — 1 passed.
- `node --test
.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` — 9 passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `pnpm --filter @paperclipai/ui typecheck` — passed.
- `pnpm check:token-gates` — changed files are clean; the
repository-wide command currently reports nine unrelated pre-existing
`#9627` literals outside this PR's diff.

## Risks

- Agent inbox mutations broaden an existing endpoint path, so
authorization and target resolution must remain fail-closed; focused
route and authorization tests cover allowed and denied paths.
- Archive state affects only the responsible user's inbox presentation
and remains reversible; it does not mutate issue status, assignment, or
visibility.
- The UI policy defaults to the existing open behavior, while allowlist
and disabled modes can reduce agent access.
- This PR intentionally builds on #9654 and #9658 and contains no new
migration number or modification to an already-applied migration.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using GPT-5.4, medium reasoning, repository tool use,
shell execution, code review, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change
(`feat/inbox-agent-archive-complete`) and contains no internal Paperclip
ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 16:49:18 -05:00
Dotta 52aea90263
feat: organize skills with nested folders and My Skills (#9633)
## Thinking Path

> - Paperclip is the open source control plane people use to organize
and govern AI-agent companies
> - Company skills are durable resources that users browse, import,
assign, and maintain over time
> - A flat skill list plus tags does not provide a stable location or
hierarchy for personal, company, project-imported, and bundled skills
> - Folder paths need to be canonical, company-scoped, safe to move, and
preserved across re-imports without changing skill IDs
> - The `/skills` UI also needs traversal, breadcrumbs, move/create
flows, and a dedicated My Skills namespace that work on desktop and
mobile
> - This pull request adds the folder data model and APIs, reserved-root
lifecycle, project import behavior, and the folder-first skills
experience
> - The benefit is a predictable filesystem-like organization model
while tags remain available for cross-cutting classification

## Linked Issues or Issue Description

Refs #9619 — the reviewed folder foundation was intentionally closed and
folded into this combined feature PR.
Refs #9026 — earlier flat-folder attempt superseded by this integrated
implementation.
Refs #3281 — related skill organization proposal; this PR uses canonical
persisted folders rather than deriving groups from skill keys, and does
not add hidden-skill behavior.

**Feature request**

- **Problem:** Skills currently lack a canonical hierarchical location,
making personal skills, project imports, bundled skills, and
company-authored skills difficult to traverse and manage at scale.
- **Proposed behavior:** Add nested company-scoped folders with stable
paths, reserved My/Projects/Bundled roots, subtree queries, safe
move/create operations, and a folder-first `/skills` library UI.
- **Import behavior:** New project scans file skills under
`projects/<project-slug>`; later imports update content without
overriding a user-selected folder.
- **Alternatives considered:** Tags alone remain useful for
cross-cutting classification, but they do not provide canonical
location, nesting, reserved namespaces, or stable import placement.
- **Roadmap alignment:** Extends the completed Skills Manager and
Scheduled Routines capabilities without duplicating an active roadmap
item.

## What Changed

- Adds `folders` persistence for routine and skill folders, nested
canonical paths, parent/slug/system-key fields, migration backfills, and
reapply-safe migrations `0174`–`0175` after current master migrations.
- Adds company-scoped folder CRUD, cycle/depth/namespace validation,
reserved My/Projects/Bundled lifecycle, item moves, subtree filtering,
and folder paths on skill results.
- Preserves project-import placement: first import files into the
project folder, while re-import keeps user-owned placement and stable
skill IDs.
- Adds the `/skills` folder tree rail, tags facet, breadcrumbs,
subfolder browser, move/new-folder dialog, canonical detail location,
inline tag editing, and folder-aware Studio creation.
- Keeps bundled skills read-only even when their source metadata is
incomplete by detecting the reserved Bundled folder and hiding
selection/move actions.
- Extends routine folder UI and OpenAPI coverage, and adds regression
tests across migrations, services, routes, tree helpers, pages, and
Studio creation.

## Verification

- `pnpm exec vitest run
packages/db/src/nested-skill-folders-migration.test.ts
server/src/__tests__/folders-routes.test.ts
server/src/__tests__/folders-service.test.ts
server/src/__tests__/company-skills-service.test.ts
server/src/__tests__/routines-service.test.ts
ui/src/components/folders/FolderControls.test.tsx
ui/src/components/folders/SkillFolderTree.test.tsx
ui/src/components/folders/skill-folder-tree.test.ts
ui/src/pages/CompanySkills.test.tsx ui/src/pages/Routines.test.tsx
ui/src/pages/SkillStudio.test.tsx
ui/src/lib/company-skill-routes.test.ts ui/src/lib/skill-create.test.ts`
— 13 files, 192 tests passed.
- `pnpm exec vitest run ui/src/pages/CompanySkills.test.tsx
ui/src/components/folders/SkillFolderTree.test.tsx` — 2 files, 20 tests
passed after preserving the existing PR's bundled-skill fixes.
- `pnpm -r typecheck` — passed for all workspace packages.
- `pnpm test:run` — passed in an isolated CI-like environment with
inherited Paperclip runtime identity and static AWS credential variables
removed.
- `pnpm build` — production build passed for all workspace packages.
- Greptile iteration 2 — 5/5 confidence with zero unresolved threads on
commit `ff2d67aa71`.
- Latest-head GitHub checks — all success, neutral, or skipped; PR is
mergeable with a clean merge state.
- `pnpm check:token-gates` — reports nine existing `#9627` comment false
positives already present on `master`; this PR introduces no new token
violation.

## Risks

- **Migration/backfill:** `0174` creates the foundation and `0175` adds
nested/reserved semantics. Both are ordered after current master
migration `0173`, are covered by numbering/safety checks, and are
designed to be reapply-safe.
- **Reserved namespaces:** My, Projects, and Bundled roots are
service-managed. Regression coverage prevents namespace squatting,
cross-company folder use, bundled writes, cycles, and excessive depth.
- **Behavioral change:** Project scans choose a project folder only on
initial creation; existing skills deliberately retain their current
folder during refresh.
- **UI scope:** The folder rail applies to the Installed library;
Catalog retains the discovery-oriented category sidebar.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI `gpt-5.5` in Codex CLI, medium reasoning mode; runtime did not
expose a context-window value. Used repository/file tools, terminal
execution, Git/GitHub operations, test execution, and code editing.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-16 15:50:45 -05:00
Dotta 89ce36d7af
feat(skills): open-by-default company skill policy and core UX (#9564)
## Thinking Path

> - Paperclip uses company skills to make agent capabilities reusable
across an organization.
> - Skill operations currently mix capability availability with
permission checks, which creates avoidable setup friction and
inconsistent denial handling.
> - The policy contract needs to remain open by default while allowing
company-scoped restrictions for governed deployments.
> - Core owns the canonical policy actions, persistence, evaluation, API
behavior, safe import boundaries, and generic denial/read-only UI.
> - Enterprise policy-editor implementation belongs in the separate
`paperclip-ee` repository and is intentionally excluded from this PR.

### Problem or motivation

Company skill operations can encounter permission dead ends even when no
explicit restriction has been configured, and import-source
classification can drift between policy evaluation and execution.

### Proposed solution

Define eight canonical skill policy actions, default all actions to
allowed, persist company-scoped restrictions, expose policy evaluation
APIs, normalize import sources at the boundary, and update Skill Studio
to present actionable restriction states without embedding Enterprise
Edition implementation in the core repository.

### Alternatives considered

Keeping capability checks distributed across routes and UI surfaces was
rejected because it duplicates policy logic and makes denial behavior
inconsistent. Shipping the Enterprise policy editor in this repository
was rejected because `paperclip-ee` is a separate repository and must
receive its own PR.

### Roadmap alignment

Extends the completed **Skills Manager** roadmap area by adding coherent
governance and removing workflow dead ends.

### Additional context

The core API contract remains suitable for a separate Enterprise Edition
editor, but this PR contains no `paperclip-ee` package or EE-specific UI
integration code.

## What Changed

- Added the company skill policy contract to product and implementation
documentation, including the open-by-default rule, eight canonical
actions, decision shape, and core/EE ownership boundary.
- Added the company-scoped policy schema, migration `0170`, shared
validators, policy service, REST routes, OpenAPI coverage, and focused
server tests.
- Hardened import policy enforcement by normalizing import sources and
keeping source classification consistent between policy evaluation and
execution.
- Updated core Skill Studio behavior to remove generic permission dead
ends and show actionable policy/platform denial states only when an
operation is actually denied.
- Removed the `plugin-paperclip-ee` package, Docker wiring, EE
discovery/deep-link helpers, and EE-specific UI tests/stories from this
PR so that implementation can be submitted separately to the EE
repository.
- Preserved open-by-default behavior when no explicit company
restriction exists.

## Verification

- `pnpm --filter @paperclipai/ui exec vitest run
src/components/skill-studio/SkillPolicySurfaces.test.tsx
src/lib/skill-policy-denial.test.ts` — 20/20 passed.
- `pnpm --filter @paperclipai/ui exec tsc --noEmit` — passed.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/worktree-config.test.ts` — 12/12 passed.
- `pnpm check:token-gates` — passed with all gates clean.
- `git diff --check` — passed.
- `git diff --name-only origin/master | rg
'paperclip-ee|ee-skill-policy'` — no matches.

## Risks

- Migration `0170` introduces company policy persistence; rollout
depends on the migration applying before policy routes are exercised.
- Open-by-default is an intentional behavioral policy: deployments
expecting implicit denials must configure explicit restrictions.
- Import normalization is security-sensitive and should retain focused
review.
- The separate EE editor must stay contract-compatible with the core
policy API as policy actions evolve.

## Model Used

- OpenAI Codex CLI, runtime model identifier and context-window size not
exposed by this execution environment; reasoning, repository tool use,
shell execution, and code review capabilities enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details available to this runtime)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked or
described the result above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green on the latest head
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Evyatar Bluzer <bluzername@users.noreply.github.com>
2026-07-15 11:42:40 -05:00
Dotta 931eec3fbf
feat(mcp) [split 4/8]: wire gateway runtime and Smoke Lab (#9559)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 4/8 and focuses on gateway runtime, Smoke
Lab, plugins, and server wiring
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: The policy core needs runtime execution, endpoint guards,
route registration, heartbeat integration, and adapter MCP injection to
become operational.
- Proposed solution: Adds the remaining server routes/wiring/consumers,
runtime tests, adapter-utils MCP contracts, and Claude/Codex injection
implementations required by the server layer.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `pap10341-split/03-server-tool-access`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: SecurityEngineer for gateway, endpoint guard, token
issuance, and runtime wiring; Greptile on every PR.

## What Changed

- Adds the remaining server routes/wiring/consumers, runtime tests,
adapter-utils MCP contracts, and Claude/Codex injection implementations
required by the server layer.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck`
- Changed server test set — 26 files, 382 tests passed
- Affected server adapter tests — 38 tests passed after concrete adapter
boundary move
- Adapter-utils and Codex focused tests — 76 tests passed

## Risks

- Remote endpoint validation, token handling, and runtime supervision
are security-sensitive and can fail closed or deny legitimate access if
misconfigured.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556#9557#9558#9559#9560#9561#9562#9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 15:07:30 -05:00
Dotta 36ec79c196
feat: add attention queue and Decisions surface (#9380)
## Thinking Path

> - Paperclip is the control plane for autonomous AI companies, where
operators need a reliable way to find and act on work awaiting their
input.
> - The attention and issue-thread interaction subsystems expose those
decision points across server APIs and the board UI.
> - The previous navigation and interaction presentation left these
actions fragmented and did not offer a controlled rollout for the
Decisions surface.
> - This branch adds the attention feed, richer interaction cards,
grouping, dismiss/snooze behavior, and a gated Decisions sidebar entry.
> - It also keeps experimental settings and API contracts synchronized,
with an idempotent migration for the new dismissal state.
> - This pull request delivers the complete, tested attention/Decisions
experience as one reviewable unit.

## Linked Issues or Issue Description

- Adds an operator-focused attention queue and Decisions experience:
grouped decision cards, semantic interaction actions, dismiss/snooze
handling, resilient interaction states, and an experimental flag to
control the Decisions navigation entry.


## Feature Context

### Problem or Motivation

Operators currently have to hunt across approvals, interactions, failed
runs, and budget alerts to find decisions that need their action.

### Proposed Solution

Provide a gated Decisions attention queue that groups actionable items,
supports direct resolution, and preserves operator control through
dismiss and snooze actions.

### Alternatives Considered

Keep separate, source-specific views only; this leaves cross-cutting
operator decisions fragmented and harder to prioritize.

### Roadmap Alignment

This improves the V1 control-plane operator workflow by making pending
governed actions discoverable in one company-scoped surface.

## What Changed

- Added server attention-feed services, routes, interaction handling,
dismiss/snooze support, and an idempotent `0145` inbox-dismissal
migration.
- Added shared attention, inbox-dismissal, and experimental-settings
contracts.
- Added Decisions/attention UI, interaction-card states, sidebar
badge/navigation integration, grouping, keyboard support, and Storybook
coverage.
- Added tests for attention behavior, thread interactions, settings
normalization, dismissals, and API behavior.
- Removed generated screenshots from the final PR diff and rebased the
branch onto current `master`.

## Verification

- `pnpm check:token-gates` — passed.
- `pnpm exec vitest run
packages/shared/src/issue-thread-interactions.test.ts
server/src/__tests__/attention-service.test.ts
server/src/__tests__/inbox-dismissals.test.ts
server/src/__tests__/issue-thread-interactions-service.test.ts
server/src/__tests__/issue-thread-interaction-routes.test.ts
ui/src/lib/attention.test.ts
ui/src/components/AttentionQueueRow.test.tsx
ui/src/components/IssueThreadInteractionCard.test.tsx
ui/src/pages/InstanceExperimentalSettings.test.tsx` — passed: 158 tests
across 9 focused files.
- GitHub Actions for `ad636f560`: build and typecheck/release-registry
have passed; remaining general-server and Greptile checks are in
progress.

## Risks

- Moderate: this is a cross-layer attention/interaction feature with a
new migration and navigation behavior.
- The `enableDecisions` experimental setting defaults to off, limiting
rollout impact.
- Existing dismissal data is backfilled to `dismiss`; the migration is
idempotent and uses guarded constraint creation.

> ROADMAP.md was checked; no duplicate planned core feature was
identified. Related open pull requests were searched before opening this
PR.

## Model Used

- OpenAI GPT-5.5 via Codex CLI, with tool use and local code execution.
Context-window size unavailable in this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My public PR branch name describes the change and contains no
internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally; focused tests pass and the remaining
unrelated AWS test failure is documented above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 17:09:57 -05:00
Dotta 5c85ae64a0
Cases: experimental first-class case object (#9198)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The board currently uses issues for execution, but longer-lived
content work needs a separate object that can survive beyond a single
task thread.
> - The Cases subsystem adds an experimental, company-scoped record for
content artifacts and their supporting metadata.
> - The backend needs durable storage, API routes, revision history,
issue linkage, and company-boundary enforcement before the UI can depend
on Cases.
> - The UI needs an opt-in navigation surface, list/detail views,
reference chips, and issue-page context so operators can inspect Cases
without making them the default workflow.
> - The agent-facing skills need a contract for creating and updating
Cases so automated content workflows can dogfood the feature.
> - This pull request ships that experimental end-to-end path behind the
`enableCases` flag.
> - The benefit is a first-class place to collect content work,
references, attachments, revisions, and related execution threads
without polluting the core issue model.

## Linked Issues or Issue Description

No public GitHub issue exists for this experimental feature.

Feature request fields:

### Problem

Content-oriented work such as release notes, announcements, docs, and
campaigns can span many execution issues, which makes the final artifact
hard to find and reason about after the execution thread moves on.

### Proposed solution

Add an experimental Cases object that is company-scoped, linked to
issues, queryable through the API, inspectable in the board UI, and
writable by agent workflows through documented conventions.

### Alternatives considered

Continue encoding content artifacts directly in issues or documents
only. That keeps the data model smaller, but it does not give operators
a stable artifact-centric view or a clean way to link related execution
history.

### Roadmap alignment

Checked `ROADMAP.md`; this PR does not duplicate an existing planned
core roadmap item.

## What Changed

- Added the `cases` data model, migration, schema exports, and
experimental `enableCases` instance setting.
- Added company-scoped Cases API routes for list/detail/update, issue
links, revisions, children, activity events, annotations, attachments,
and idempotent agent-oriented upserts.
- Scoped case and issue lookup helpers before access checks so
inaccessible cross-company identifiers resolve as not found rather than
leaking existence.
- Fixed case PATCH timestamp handling so non-status updates cannot
overwrite `completedAt` from a stale pre-transaction row snapshot.
- Moved Cases list type/status/project filters into the server request
before the server-side limit is applied, including multi-select filters
and no-project filtering.
- Added backend route coverage for creation, updates, idempotency, issue
linking, attribution, company-boundary enforcement, OpenAPI
registration, list filtering, timestamp patch behavior, and inaccessible
lookup regressions.
- Added the experimental Cases UI surface: sidebar entry, gated routes,
list filters/grouping, detail overview, activity, revisions, children,
attachments, and issue-page case rail.
- Added case reference rendering and company-prefixed case href
generation so case links resolve directly inside the active company
route.
- Added Paperclip skill documentation for agent workflows that create or
update Cases.
- Wired release-content skills to emit Cases for dogfooding.
- Rebased onto current `master` and renumbered the Cases migrations to
`0143`/`0144` after the latest upstream migration sequence.

## Verification

- Current PR head: `ecc13be0d`.
- Rebased on current `master` (`606aa4f266`) and pushed to the existing
PR branch.
- `git diff --check origin/master...HEAD` — passed before the first
update push; subsequent committed diffs were also checked with `git diff
--check` before commit.
- Guardrails checked: no `pnpm-lock.yaml` changes, no
`.github/workflows` changes, and changed-file count is below the
Greptile 100-file limit.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/cases-routes.test.ts
src/__tests__/instance-settings-service.test.ts
src/__tests__/openapi-routes.test.ts` — passed, 3 files / 26 tests
before review-fix commits.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/cases-routes.test.ts` — passed after each server-side
Greptile fix, latest 1 file / 15 tests.
- `pnpm --filter @paperclipai/server typecheck` — passed after the
timestamp and lookup fixes.
- `pnpm --filter @paperclipai/ui exec vitest run
src/pages/Cases.test.tsx src/pages/CaseDetail.test.tsx
src/pages/CompanySkills.test.tsx src/App.cases-routing.test.tsx` —
passed, 4 files / 30 tests before review-fix commits.
- `pnpm --filter @paperclipai/ui exec vitest run
src/pages/Cases.test.tsx` — passed after the list-filter fix, 1 file /
12 tests.
- `pnpm --filter @paperclipai/ui typecheck` — passed after the
list-filter fix.
- `pnpm check:token-gates` — passed after UI changes.
- Remote PR checks on head `ecc13be0d` are green: Paperclip CI, build,
typecheck, test matrix, e2e, Canary Dry Run, policy, commit review,
Superagent Security Scan, Socket, Snyk, and Greptile passed; Storybook
visual regression is skipped and security-review is neutral.
- Greptile Review: 5/5 confidence, zero unresolved Greptile threads.

## Risks

- Medium feature risk because this introduces a new experimental domain
object across database, server, shared contracts, skills, and UI.
- The feature is gated behind `enableCases`, which limits default
operator exposure while the model is exercised.
- Case links now prefer company-prefixed hrefs; the unprefixed redirect
remains for externally entered URLs.
- Cases list filtering now sends multi-select filters to the server
before limiting; the UI still applies the same local filters as a second
pass for ancestor/context rows.
- Migrations were renumbered on top of current master; the SQL uses
guarded `IF NOT EXISTS` / `ADD COLUMN IF NOT EXISTS` patterns where
relevant for safer replay.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 Codex in the Paperclip local coding environment was used
for this PR curation, rebase verification, review-fix implementation,
push, and PR description update. The runtime exposes tool use and shell
execution; context-window size is not exposed by this Paperclip adapter.
Several implementation commits also include AI co-author trailers
recorded in git history.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I searched the GitHub PR list for similar PRs and confirmed this
is not a duplicate
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 22:11:03 -05:00
Dotta 176645187c
Fix request storm polling and issue-list coalescing (#9190)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI keeps issue, agent, activity, and run state fresh
through polling across several pages and sidebar surfaces
> - When multiple components or browser tabs poll the same company data
at the same time, the API can receive bursts of duplicate issue-list
requests
> - Those duplicate requests increase database and server load without
returning meaningfully different data
> - This pull request adds server-side compression/coalescing plus
client-side visibility-aware and cross-tab shared polling
> - The benefit is lower request volume during normal board usage while
preserving fresh UI data for active users

## Linked Issues or Issue Description

No exact public GitHub issue was found for this request.

Problem:
- The board can issue redundant polling requests for the same issue-list
data from multiple UI surfaces and tabs.
- In busy operator sessions, those bursts can trigger request-storm
behavior and unnecessary issue-list load.
- Expected behavior is to reuse identical in-flight work server-side and
reduce hidden-tab or duplicate-tab polling client-side while preserving
normal refresh behavior.

Related public context found during duplicate search:
- #8206 covers a different board UI 404-storm scope.
- #5165 covers separate issue-list behavior around page-size truncation.

## What Changed

- Added API compression middleware foundation and a company/created-at
index for heartbeat run access.
- Added server-side issue-list request storm detection and identical
in-flight request coalescing.
- Added UI fetch metadata, visibility-aware polling, and request
deduplication for issue/activity/client calls.
- Added cross-tab shared polling primitives and wired them into the
sidebar, inbox, dashboard, issue, project, routine, and agent surfaces.
- Resolved the latest `master` migration collision by keeping upstream
`0140_built_in_managed_resources.sql` and renumbering this branch's
heartbeat-run index migration to
`0141_heartbeat_runs_company_created_at_index.sql`; the SQL uses `CREATE
INDEX IF NOT EXISTS` for idempotency.
- Stabilized server heartbeat cleanup tests exposed by the PR check
matrix.
- Fixed the Greptile compression follow-up by weakening strong ETags on
encoded JSON responses and bypassing compression for streamed/download
responses.

## Verification

- `pnpm exec vitest run ui/src/components/IssuesList.test.tsx
ui/src/pages/Inbox.test.tsx` — passed after resolving the latest
`master` conflict in `IssuesList.tsx` and updating the 200-result cap
expectations.
- `pnpm check:token-gates` — passed after the UI conflict resolution.
- `jq -e '.entries | length as $n | (map(.idx) | unique | length == $n)
and (map(.tag) | unique | length == $n)'
packages/db/src/migrations/meta/_journal.json` — passed after
renumbering the migration to `0141`.
- `pnpm exec vitest run server/src/__tests__/api-compression.test.ts` -
passed after the compression follow-up.
- `pnpm exec vitest run
server/src/__tests__/issue-list-assignee-filter-routes.test.ts` - passed
after the compression follow-up.
- `pnpm --filter @paperclipai/server typecheck` - passed after the
compression follow-up.
- Greptile Review for head `8dbddac41ec273fda404100b4981ddb912fad57b` -
passed after the latest conflict/migration fix; all Greptile review
threads are resolved.
- `pnpm exec vitest run server/src/__tests__/api-compression.test.ts
server/src/__tests__/issue-list-assignee-filter-routes.test.ts
ui/src/api/client.test.ts ui/src/api/issues.test.ts
ui/src/lib/polling.test.ts ui/src/lib/cross-tab-poll.test.ts
ui/src/pages/Inbox.test.tsx ui/src/components/SidebarProjects.test.tsx
ui/src/components/SidebarAccountMenu.test.tsx
ui/src/lib/issueDetailCache.test.ts` — passed.
- `pnpm exec vitest run ui/src/api/client.test.ts` — passed.
- `pnpm exec vitest run ui/src/pages/Inbox.test.tsx
ui/src/components/SidebarProjects.test.tsx
ui/src/components/SidebarAccountMenu.test.tsx
ui/src/lib/issueDetailCache.test.ts` — passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-worktree-suppression.test.ts` — passed.
- `pnpm exec vitest run
server/src/__tests__/low-trust-red-team-routes.test.ts` — passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` —
passed.
- `pnpm --filter @paperclipai/ui typecheck` — passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- GitHub PR checks for head `8dbddac41ec273fda404100b4981ddb912fad57b`:
all GitHub Actions/status checks passed; Greptile, Superagent, Socket,
Snyk, build, typecheck/release registry, general tests, serialized
server suites, e2e, canary dry run, policy, and commitperclip review are
green; Storybook visual regression and security-review were
skipped/neutral by policy.
- Confirmed this branch does not include `pnpm-lock.yaml` or
`.github/workflows` changes.

## Risks

- Medium risk: issue-list coalescing changes request timing and cache
semantics for a hot API path.
- Medium risk: cross-tab polling uses browser coordination primitives,
so older or unusual browser environments need fallback behavior to stay
correct.
- Low migration risk: the new index migration is ordered after current
`master` and uses `CREATE INDEX IF NOT EXISTS`.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex coding agent, GPT-5-based model, tool-enabled with
shell/git execution. Exact hosted deployment identifier and
context-window size were not surfaced in the agent runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-09 17:38:34 -05:00
Dotta 8b6a06ee25
[codex] Add built-in agents and Reflection Coach bundle (#9206)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Operators need first-party agent capabilities for repeatable company
work, not just manually created one-off agents.
> - Built-in agents need to behave like normal company-scoped agents
while preserving approval gates, permissions, budgets, and audit trails.
> - Reflection and coaching work also needs bundled instructions, skill
content, and a routine so the feature can be installed and reset
predictably.
> - The API, database, UI, portability, and tests all need to agree on
the built-in lifecycle from not provisioned through setup, approval,
ready, paused, and reset.
> - This pull request adds built-in agent provisioning and the
Reflection Coach bundle end-to-end.
> - The benefit is a safer first-party path for Paperclip-managed agents
without bypassing the same governance model used for operator-created
agents.

## Linked Issues or Issue Description

No public GitHub issue was found for this exact built-in agent and
Reflection Coach bundle work.

Problem/motivation:
- Paperclip did not have a first-party built-in agent lifecycle for
product-owned agents.
- Bundled agent resources such as default instructions, skills, and
routines needed managed ownership and reset semantics.
- Approval-gated companies needed built-in setup to preserve requested
adapter, budget, manager, and permission state through board approval.
- The board UI needed clear built-in badges, setup affordances,
readiness state, and bundle status without exposing secrets.

Proposed solution:
- Add a company-scoped built-in agent registry,
provisioning/reset/reconcile/status APIs, and Reflection Coach bundled
resources.
- Track bundled managed resources in the database with idempotent
migration behavior.
- Reuse existing agent approval, authorization, budget, and activity-log
paths instead of creating a bypass.
- Add UI setup, badges, gates, bundle panels, and route coverage for
built-in agents.

Duplicate search:
- Searched GitHub PRs for `built-in agents Reflection Coach
repo:paperclipai/paperclip`; only this PR was returned.
- Searched GitHub issues for the same query; no public issues were
returned.

## What Changed

- Added built-in agent definitions, lifecycle state derivation,
provisioning, reset, reconcile, status, and routine-control routes.
- Added the `built_in_managed_resources` migration and schema exports
for bundled instructions, skill, and routine ownership.
- Added the Reflection Coach built-in bundle with default instructions,
skill catalog content, routine template, default permissions, and
managed-resource drift handling.
- Added approval-aware provisioning behavior that preserves requested
adapter config, budgets, manager assignment, and built-in permissions
through hire approval.
- Added authorization and mutation gates for built-in agent and skill
changes, including consented Reflection Coach change paths.
- Added UI surfaces for built-in agent setup, roster/detail badges,
readiness gates, bundle status, routine controls, and route filtering.
- Added company import/export and validator coverage for built-in
managed resources and low-trust/red-team presets.
- Addressed Greptile follow-ups for pending approval reconciliation,
consent-gate error propagation, config-read authorization fallback,
approval-path manager preservation, and non-model adapter provisioning.

## Verification

Local verification:
- `git diff --check public/master..HEAD` passed.
- `pnpm check:token-gates` passed with all gates clean.
- `pnpm exec vitest run
ui/src/components/ConfigureBuiltInAgentModal.test.tsx` passed: 1 file, 4
tests.
- `pnpm exec vitest run ui/src/components/EntityRow.test.tsx
ui/src/pages/Agents.test.tsx ui/src/components/BuiltInAgentGate.test.tsx
ui/src/components/ConfigureBuiltInAgentModal.test.tsx
ui/src/components/BuiltInBundlePanel.test.tsx
ui/src/pages/InstanceExperimentalSettings.test.tsx
ui/src/pages/Routines.test.tsx` passed: 7 files, 64 tests.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/built-in-agents.test.ts
src/__tests__/authorization-service.test.ts
src/__tests__/company-skills-routes.test.ts` passed: 3 files, 91 tests.
- `pnpm --filter @paperclipai/db check:migrations` passed.
- `pnpm -r typecheck` passed after the rebase; `pnpm --filter ui
typecheck` passed after the final UI review fix.

Remote verification on latest head
`1c61f693a4ec881d739022b0e75a8ca8bf8c2cd8`:
- Merge state: `CLEAN`.
- Greptile: `5/5`, zero unresolved Greptile threads.
- PR check rollup: all checks successful, neutral, or skipped as
expected.
- Passing gates include Build, Typecheck + Release Registry, all server
shards, all workspace shards, all serialized server suites, e2e, Canary
Dry Run, policy, review, verify, Socket, Superagent, and Snyk.

## Risks

- This adds a new managed-resource table and migration; the migration
uses idempotent create/add/index guards and passed migration safety
checks.
- Built-in agent provisioning touches approval and authorization paths;
tests cover pending approval preservation, stale retry rejection,
consent gates, and config-read fallback behavior.
- Reflection Coach creates managed instructions, skill, and routine
resources; drift/reset behavior is covered by service tests and redacted
API responses.
- Non-model adapter setup now provisions a `needs_setup` built-in row
before command/endpoint fields are complete; this matches the server
lifecycle and is covered by the setup modal regression test.

## Model Used

OpenAI Codex coding agent based on GPT-5. Exact hosted model ID,
context-window size, and reasoning-mode labels are not exposed in this
runtime; tool use, shell execution, GitHub CLI/API access, and local
code editing were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-09 16:29:30 -05:00
Dotta be821a4f7e
Fix DB backup health alerts (#9147)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Operators depend on `/api/health` and OpenAPI status surfaces to
know whether the local control plane is healthy.
> - Database backups are a safety-critical background process, but
backup failures were not represented in health responses.
> - That gap means an instance can look healthy while backup state is
stale, failing, or unavailable.
> - This pull request adds backup-health evaluation and exposes it
through the health route, server startup wiring, and OpenAPI contract.
> - The benefit is earlier operator visibility when automatic backups
stop protecting instance data.

## Linked Issues or Issue Description

No public GitHub issue exists. Inline bug report:

**Pre-submission checklist**

- [x] I have searched existing open and closed issues and this is not a
duplicate.
- [x] I am on the latest released version of Paperclip (or can reproduce
on `master`).
- [x] I have confirmed the error originates in Paperclip itself — not in
my agent adapter, API provider, or local configuration.

**What happened?**

Automatic database backup health was not included in the app health
response, so backup failures or stale backups could be missed while
`/api/health` still looked otherwise usable.

**Expected behavior**

The health endpoint should include backup-health details that let
operators identify disabled, stale, failing, or healthy backup states.

**Steps to reproduce**

1. Configure a Paperclip instance with automatic database backups.
2. Force backup status into a stale or failing state.
3. Call `/api/health` and inspect whether backup state is represented.

**Paperclip version or commit**

`master` at the PR base.

**Deployment mode**

Local dev (`pnpm dev`) and self-hosted server deployments.

**Installation method**

Built from source (`pnpm dev` / `pnpm build`).

**Agent adapter(s) involved**

- [x] Not adapter-specific (core bug)

**Database mode**

Embedded development Postgres and external Postgres backup paths.

**Access context**

Board/operator health checks.

**Relevant logs or output**

Covered by the added `server/src/__tests__/health.test.ts` cases.

**Relevant config (if applicable)**

Not applicable.

**Additional context**

This surfaces backup status only; it does not change backup execution
scheduling.

**Privacy checklist**

- [x] I have reviewed all pasted output for PII (usernames, file paths,
API keys, tokens, company names) and redacted where necessary.

## What Changed

- Added a database backup health service that classifies backup recency,
status, and failure conditions.
- Wired backup health into app/server startup and the health route
response.
- Documented the backup-health behavior in development docs and OpenAPI
output.
- Added focused health route tests for healthy, stale, disabled, and
failing backup states.

## Verification

- `/srv/paperclip/home/paperclipai/paperclip/node_modules/.bin/vitest
run server/src/__tests__/health.test.ts`

## Risks

Low-to-medium risk. This changes health response content and may affect
external health consumers that parse fields strictly. It should not
alter backup execution itself.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5.5 coding agent with repository tool use and local
shell execution. Context window was not surfaced by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-07 05:54:20 -05:00
Dotta ad961227f5
feat(secrets): add user-specific runtime secrets (#8825)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs often need provider credentials, API tokens, and other
environment-bound secrets.
> - Company-level secrets work for shared credentials, but they do not
model values that should differ by human operator.
> - Without a user-scoped model, a run can dispatch without knowing
whether the responsible human has supplied the needed value.
> - Paperclip also needs run attribution to make those user-scoped
runtime checks deterministic and auditable.
> - This pull request adds user-specific secret definitions, per-user
values, environment bindings, responsible-user attribution, and runtime
resolution gates.
> - The benefit is that teams can define the secret once, let each user
provide their own value, and block runs before dispatch when required
user secrets or active definitions are unavailable.

## Linked Issues or Issue Description

Refs #224
Refs #6057

This PR implements user-specific secret support as a core
secret-management capability rather than a one-off adapter setting. It
is related to existing public work on company secrets UI and runtime
secret refs, but is distinct because the value is owned by the
responsible user and resolved at run dispatch time.

Related PR search before opening found existing secrets work such as
#1550, #8256, #8614, #8634, and #8647; none of those add the full
user-secret definition/value/runtime gate covered here.

## What Changed

- Added user-secret definitions and per-user "My secrets" values,
keeping stored values out of access metadata.
- Added `user_secret_ref` environment bindings and UI affordances to
pick them alongside existing secret refs.
- Added responsible-user runtime resolution so user-secret refs resolve
against the human responsible for the run.
- Added pre-dispatch missing-secret gates so runs fail before adapter
dispatch when required user values are absent or definitions are
inactive.
- Added low-trust allowlist hardening for user-secret runtime access.
- Added issue, routine, run, and agent API key responsible-user
attribution and fail-closed dispatch behavior when attribution cannot be
resolved.
- Added denial-copy mapping so responsible-user authorization failures
surface as actionable run outcomes instead of opaque setup failures.
- Added OpenAPI documentation for the user-secret routes.
- Rebases cleanly on current `master`; migrations were renumbered
incrementally as `0128_user_specific_secrets`,
`0129_agent_api_key_responsible_user`, and
`0130_run_responsible_user_invariant` after upstream `0126`/`0127`
migrations.
- Removed previously committed local design screenshots so the PR
contains code/docs/tests only.

## Verification

- PASS: PR head `2527febd106bcf3ca264ca0da7fca491084192d6` is based on
`paperclipai/paperclip:master`.
- PASS: `git diff --check`
- PASS: `git diff --name-only public/master...HEAD | rg
'^(pnpm-lock\\.yaml|\\.github/workflows/|screenshots/)' || true`
produced no files.
- PASS: migration journal audit confirmed unique indexes through `130`
with tail entries `0126_issue_comment_derived_attribution`,
`0127_environment_custom_images_instance_scoped`,
`0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and
`0130_run_responsible_user_invariant`.
- PASS: `pnpm --filter @paperclipai/ui typecheck`
- PASS: `pnpm --filter @paperclipai/server typecheck`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-responsible-user-invariant.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-active-run-output-watchdog.test.ts
src/__tests__/heartbeat-stale-queue-invalidation.test.ts
src/__tests__/heartbeat-workspace-finalize-branch.test.ts
src/__tests__/issue-monitor-scheduler.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-comment-wake-batching.test.ts
src/__tests__/heartbeat-retry-scheduling.test.ts
src/__tests__/heartbeat-accepted-plan-workspace-refresh.test.ts
src/__tests__/heartbeat-plugin-environment.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/low-trust-red-team-routes.test.ts`
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/secrets-service.test.ts` (55 tests)
- PASS: `pnpm vitest run server/src/__tests__/secrets-routes.test.ts
server/src/__tests__/secrets-service.test.ts` (89 tests after final
Greptile cleanup fixes)
- PASS: `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-issue-liveness-escalation.test.ts` (17 tests
after the final rebase CI fix)
- PASS: focused server Vitest batches covering heartbeat recovery,
project env, plugin env, routines, low-trust, pipelines, monitors,
watchdog, and stale queue paths.
- PASS: GitHub checks are green on
`2527febd106bcf3ca264ca0da7fca491084192d6`, including Typecheck +
Release Registry, Build, General tests, serialized server suites, e2e,
Canary Dry Run, verify, security checks, and Greptile Review.
- PASS: Greptile Review completed successfully on
`2527febd106bcf3ca264ca0da7fca491084192d6` with Confidence Score 5/5,
and GraphQL review-thread audit returned zero unresolved non-outdated
threads.

## Risks

- Runtime behavior now depends on a run having a correct responsible
user; missing or incorrect responsibility assignment can block runs
before adapter dispatch.
- `user_secret_ref` bindings intentionally expose metadata without
values, but UI/API callers may need to handle the new binding kind
explicitly.
- External secret providers and IAM policies are not automatically
provisioned by this PR; operators still need to configure provider-side
access for non-local vaults.
- The PR is broad across db/shared/server/UI/runtime paths, so release
validation should include both API and UI secret workflows before merge.
- The migration renumbering is intentionally incremental after upstream
migrations; the branch migrations use guarded
column/table/index/constraint creation so users who tested the older
draft numbering should not hit duplicate DDL for the existing objects.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5-based coding agent (`gpt-5`), Codex local adapter
with shell/tool use and code execution. Context window and internal
reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 05:58:20 -05:00
Dotta 43b005b704
Add pipeline workflow primitives and operator UI (#7903)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The pipeline subsystem models repeatable work as items moving
through stages, with agent automation, review gates, blockers, drift
notices, and linked work.
> - Operators need this to be usable as one coherent workflow surface,
not just as backend primitives or disconnected route experiments.
> - The branch now carries the pipeline data model, service/routes,
CLI/tutorial path, aggregation feeds, operator UI, stage automation
controls, liveness/retry handling, and follow-up polish that make the
primitive reviewable end to end.
> - This pull request is the single review target for that pipeline
workflow primitive stack.
> - The benefit is that reviewers can evaluate the full operator
experience and server contract together against `master`.

## Linked Issues or Issue Description

No public GitHub issue exists for this work. The underlying feature
request is described inline.

### Problem or motivation

Paperclip needs a first-class way to model multi-stage agent/company
workflows where upstream items can spawn downstream work, request
review, carry fields across pipelines, surface drift, retry automation,
and show operators where work is blocked or active. Without a unified
pipeline primitive, these workflows spread across ad hoc issues,
routines, and comments, making the state hard to inspect or operate.

### Proposed solution

Add the pipeline workflow primitive stack: database schema and
migrations, shared validators/types, server services and REST routes,
aggregation and liveness helpers, CLI/tutorial smoke support, and the
React operator UI for pipeline lists, boards, item detail,
review/learnings views, settings, stage automation, secrets, carry-over
fields, and retry/recovery flows.

### Alternatives considered

- Keep workflows as loosely linked issues and routines: rejected because
operators need a single board/detail/settings surface for repeated
workflow patterns.
- Ship backend primitives first and defer UI: rejected for this branch
because the operator experience is the main way to validate the
primitive.
- Add a narrower one-off content workflow: rejected because the same
primitives are useful across future company processes.

## What Changed

- Added and evolved pipeline schema, migrations, shared contracts,
server services, REST routes, route tests, and CLI/tutorial smoke
support.
- Added pipeline aggregation, health/liveness, drift acknowledgment,
blocker/carry-over, automation retry, stage automation environment, and
permission recovery behavior.
- Added the operator UI for pipeline index/board/item
detail/settings/review/learnings flows, including stage secrets,
automation controls, markdown/item descriptions, linked issue assets,
liveness banners, and source automation metadata.
- Refactored issue document frame rendering through the shared
`DocumentFrameHeader` component to keep document controls consistent
with the pipeline document surfaces.
- Kept this PR as the single base-branch review target for the current
pipeline branch.

## Verification

Current branch refresh:

- `pnpm vitest run server/src/__tests__/pipelines-service.test.ts` — 31
passed
- `pnpm vitest run server/src/__tests__/pipelines-routes.test.ts` — 19
passed
- `pnpm --filter ./server typecheck` — passed
- `pnpm --filter ./ui typecheck` — passed
- Verified Pipelines remains gated by `enablePipelines === true`:
sidebar item is hidden unless the flag is enabled, direct pipeline
routes redirect to `/dashboard` when disabled, and the Experimental
settings UI still has no Pipelines toggle.
- GitHub status checks on `df071c710646de625131064c3fb6588b5e97964a` —
all complete with no failing conclusions, including Actions, Socket,
Superagent/Security, and Greptile Review
- Greptile summary on `df071c710646de625131064c3fb6588b5e97964a` —
Confidence Score 5/5
- GitHub review-thread sweep — 0 unresolved Greptile threads

Previously recorded during branch development:

- Server pipeline service/route and aggregation tests
- Shared validator tests
- UI pipeline page/settings/item-detail/learnings/liveness tests
- Pipeline tutorial smoke path

## Risks

- High review surface: this is a large feature branch spanning database,
shared contracts, server behavior, CLI/docs, and UI.
- Migration ordering and schema compatibility need reviewer attention
because this branch has been kept current across multiple `master`
syncs.
- GitHub still reports merge state `BLOCKED` because the PR is awaiting
normal human review/branch-protection completion; all current status
checks are green.
- Branch-name checklist exception: this PR uses the pre-existing
requested branch name, which predates the current public-branch naming
rule. The PR title/body avoid internal issue references.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex coding agent based on GPT-5, with repository tool use,
shell execution, git/GitHub CLI operations, and local verification
commands. Earlier commits in this branch were assisted by Paperclip
agents and other AI coding agents as recorded in commit authorship.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-26 12:02:44 -05:00
Devin Foley cd38c150b0
feat: reuse Daytona sandbox leases (#8513)
Add opt-in reusable Daytona sandbox lease support, including retryable pending cleanup handling.\n\nPR: https://github.com/paperclipai/paperclip/pull/8513
2026-06-22 19:35:53 -07:00
Jannes Stubbemann 937fe62d10
feat(server): TRUST_PROXY supports CIDR list + named subnets (supersedes #3729) (#5872)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - Express looks at incoming `X-Forwarded-For` headers only when
`app.set("trust proxy", …)` says it should, and uses that resolved
client IP downstream for rate-limiting, audit logging, and any
auth/abuse signal that ties back to source IP
> - The original PR #3729 added `TRUST_PROXY` accepting only `"true"` or
a positive integer, which forces operators to pick between two unsafe
defaults: hop-count (brittle if topology changes) or boolean-true (any
client can spoof `X-Forwarded-For` and bypass rate-limits or pollute
audit logs)
> - `trust proxy: true` is one of the most common Express
misconfigurations and trivially exploitable for IP-spoofing-based
rate-limit bypass; the safest config — trust only the LB's actual CIDR
or only loopback — was unreachable with the previous parser
> - This pull request replaces the parser with full Express 5 support —
unset / `false` / `0` (Express default), positive integer hop count,
comma-separated CIDR list, named subnets (`loopback`, `linklocal`,
`uniquelocal`) — and emits a startup error naming the offending token on
invalid input
> - The benefit is that operators can now trust *only* their actual
ingress and close the spoofing window without leaking client-IP
integrity to downstream layers, while preserving every
previously-working config as a strict superset

## Linked Issues or Issue Description

Refs #1690 — login returns 500 behind a reverse proxy because Express
`trust proxy` is not enabled; this PR ships the configuration surface
(`TRUST_PROXY` with CIDR lists and named subnets) that lets operators
enable it safely. It does not change the default, so #1690 still
requires the operator to set `TRUST_PROXY` — hence Refs, not Fixes.

No other existing issue covers this directly — remaining problem
described in-PR:

- The original `TRUST_PROXY` parser (PR #3729, which this PR supersedes)
accepted only `"true"` or a hop count, forcing operators to choose
between brittle hop-counting and the spoofable `trust proxy: true`.
- `trust proxy: true` lets any client spoof `X-Forwarded-For` and bypass
rate limits or pollute audit logs; the safest config — trusting only the
LB's actual CIDR or only loopback — was unreachable with the previous
parser.

Duplicate-PR search: #1854 / #1714 are earlier minimal trust-proxy
enablement PRs; this PR supersedes #3729 and generalizes beyond a
boolean enable (CIDR lists + named subnets).

## What Changed

- **`server/src/middleware/trust-proxy.ts`** — new helper exposing
`parseTrustProxyEnv` (testable) and `applyTrustProxy(app)` (one-call
boot wiring). Surface:
- Unset / `""` / `false` / `0` → no `app.set("trust proxy", …)` (Express
default: trust nothing).
- `true` → `app.set("trust proxy", true)`. Documented as unsafe in
untrusted-LB deployments.
- Positive integer (e.g. `"2"`) → hop count. Strict parse: rejects
`"01"`, leading/trailing whitespace.
- Comma-separated list of CIDRs and/or named subnets (e.g.
`"loopback,uniquelocal,10.0.0.0/8,fd00::/8"`) → array passed to
`app.set("trust proxy", [...])`.
  - Anything else → startup error naming the offending token.
- **`server/src/app.ts`** — one import + one call to
`applyTrustProxy(app)`.
- **`server/src/__tests__/trust-proxy.test.ts`** — 12 cases: unset,
`"true"`, `"0"`, `"2"`, `"01"` rejected, `" 2 "` rejected, `"loopback"`,
`"loopback,uniquelocal"`, `"10.0.0.0/8"`, `"10.0.0.0/8,fd00::/8"`,
`"bogus"` rejected (error names the bad token), mixed-list with one bad
token rejected (error names the offending token specifically).

## Verification

- `pnpm --filter @paperclipai/server run typecheck` — clean.
- `npx vitest run trust-proxy` — 12/12 pass.

## Risks

- **No new required env vars.** Unset means default Express behavior
(trust nothing). Pure superset of #3729's surface — anything that worked
under #3729 still works here.
- **Strict parse.** `"01"` and `" 2 "` are rejected on purpose so
configuration mistakes surface at startup, not as silently-degraded
auth/rate-limit behavior. The error message names the offending token.
- **No runtime cost** — the parse runs once at boot. The downstream
`trust proxy` setting is internal to Express.
- Single-tenant local-first deploys unaffected by default.

## Model Used

Claude Opus 4.7 (1M context), extended thinking mode.

## Checklist

- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] Thinking path traces from project context to this change
- [x] Model used specified
- [x] Checked ROADMAP.md — not in conflict with planned core work
- [x] Tests run locally and pass (`trust-proxy` 12/12)
- [x] Added boundary cases (leading-zero, whitespace, unknown token,
mixed-list-with-bad-token)
- [x] No UI changes
- [x] Documented risks above
- [x] Will address all Greptile and reviewer comments before merge

Closes #3729.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-12 10:37:55 -07:00