Commit Graph

6 Commits

Author SHA1 Message Date
Dotta a560b48d6d
feat(apps): refine Postman and Shopify setup (#12357)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps give those agents governed access to external tools.
> - Provider catalogs must match each provider's current protocol and
credential contract.
> - Postman method labels and API-key placement were outdated.
> - Shopify now offers a UCP commerce endpoint that needs a managed
agent-profile argument.
> - This pull request updates both providers and documents the complete
connection-authoring workflow.
> - The benefit is accurate setup, safer runtime defaults, and a
repeatable provider review process.

## Linked Issues or Issue Description

Refs #11965

This is stack 10 of 11. It depends on stack 9 and preserves the final
catalog work recovered from #11965.

Related: #5904 covers Shopify skill routing. This pull request covers
the Apps connection contract instead.

## What Changed

- Update Postman hosted MCP methods, capability choices, default
selection, and bearer-token placement.
- Add Shopify UCP commerce and Storefront compatibility methods with
public-store prerequisites.
- Inject the reviewed Shopify UCP agent profile at runtime and remove
that managed field from user input schemas.
- Classify Shopify checkout completion and cancellation as destructive
actions.
- Expand the connection authoring runbook from provider research through
verification and pull request handoff.
- Add focused shared, server, and UI coverage.
- Make the approved-execution waiter phase-aware so slow preparation
cannot consume the provider execution timeout and grace period.
- Settle legacy pre-execute-on-approve requests and invocations as
failed, clear their stale idempotency key, and allow a fresh governed
approval instead of leaving work stuck in `executing`.

## Verification

- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts
server/src/__tests__/tool-access-service.test.ts
ui/src/pages/apps/AppsConnect.test.tsx -t
"Postman|Shopify|normalizeConnectionMethodConfig|classifyRisk"` (16
passed)
- `pnpm exec vitest run
server/src/services/approved-execution-wait.test.ts` (4 passed)
- `pnpm exec vitest run server/src/__tests__/tool-gateway.test.ts -t
"enforces policy, approvals, retries, rate limits, and company
boundaries for connected remote MCP calls"` (1 passed)
- `pnpm exec vitest run
server/src/__tests__/tool-gateway-service.test.ts` (21 passed; includes
legacy approval settlement and fresh-approval recovery)
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm check:token-gates`
- `pnpm -r typecheck`
- `pnpm build`

## Risks

- Shopify UCP calls now include a Paperclip-managed agent profile that
overrides caller input at the same path.
- Postman EU credentials now use the hosted MCP server's bearer-token
contract instead of the general REST API header.
- The catalog generator and checked-in definitions change together to
prevent regeneration drift.
- Approved execution preparation has an explicit two-minute bound;
provider execution retains its own 65-second timeout and persistence
grace starting from durable provider start.
- Legacy approvals created before execute-on-approve are intentionally
terminalized and must be requested again under the current signed
contract.

> I checked `ROADMAP.md`. This provider update does not duplicate
planned core work. The related open Shopify PR addresses skill routing,
not Apps connections.

## Model Used

OpenAI Codex, GPT-5. The runtime exact model ID and context window were
not exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-29 12:08:35 -05:00
Dotta fcb2e99e8f
feat(apps): expand the self-serve connection catalog (#12344)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A useful app store needs accurate and selectable provider
definitions.
> - Local brand assets now cover the expanded provider set.
> - Provider methods differ in transport, authentication, ownership, and
required scope.
> - This pull request expands the catalog and encodes those provider
contracts.
> - The benefit is a larger self-serve store with explicit setup
choices.

## Linked Issues or Issue Description

Refs #11965

This is stack 6 of 11. It depends on stack 5 and replaces another
reviewable part of #11965.

## What Changed

- Add and update provider definitions for the self-serve catalog.
- Add Google Workspace connection methods and capability profiles.
- Add catalog generation, ingestion, URL matching, and contract tests.
- Update legacy key tests to use a provider that still uses header
credentials.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- Result: 164 tests passed.
- `pnpm build`

## Risks

- An incorrect provider definition can offer the wrong setup method.
- Contract tests verify transport, authentication, and provider URL
behavior.
- The change does not add a database migration.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Refs #` or (b)
described the issue in this pull request
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:34 -05:00
Dotta 6244e4cf32
feat(apps): add Composio and Gmail connectors (#12342)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - App connections need both direct providers and managed provider
hubs.
> - The grant layer now defines safe credential ownership.
> - Composio needs parent and child connection lifecycle rules, and
Gmail needs governed setup.
> - This pull request adds both connector families on the grant
foundation.
> - The benefit is broader app access without weakening credential
isolation.

## Linked Issues or Issue Description

Refs #11965

This is stack 4 of 11. It depends on stack 3 and replaces another
reviewable part of #11965.

## What Changed

- Add Composio parent and child connection support.
- Add Gmail connection setup and governance.
- Preserve credential paths and remove duplicate binding declarations.
- Cascade Composio pause and restore actions to child connections.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- Result: 164 tests passed.
- `pnpm build`

## Risks

- Parent lifecycle changes can affect every Composio child.
- The service restores only children whose provider accounts remain
active.
- Credential binding paths are normalized before secret resolution.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:33 -05:00
Dotta cabc9146d0
feat(apps): add secure remote MCP and PostHog setup (#12339)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps give those agents governed access to external tools.
> - Remote MCP setup needs secure endpoint validation and durable
credentials.
> - PostHog needs both browser sign-in and personal API key setup paths.
> - This pull request adds the shared remote MCP foundation and the
PostHog definition.
> - The benefit is a secure and reusable base for later app connection
work.

## Linked Issues or Issue Description

Refs #11965

This is stack 1 of 11. It replaces the first reviewable part of #11965.

## What Changed

- Add guarded remote MCP setup and credential handling.
- Add PostHog OAuth and API key connection methods.
- Add focused server, shared contract, and UI coverage.
- Keep the migration replay-safe and idempotent.
- Give the late-close security regression the same 10-second CI headroom
as the adjacent real-timer handshake test.
- Synchronize fake-timer handshake tests at the exact ensure-session
boundary so real filesystem setup cannot race the fake deadline.
- Drive PTY overflow coverage only after listener registration so
scheduling cannot reorder the test fixture.

## Verification

- pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/execute.test.ts
server/src/__tests__/plugin-worker-manager.test.ts (220 passed; affected
cases also passed five focused stress repetitions)
- `pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never leaks a
sandbox-provided value from a late close rejection into logs or the
result"` (1 passed)
- `pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/execute.test.ts -t "never
promotes a late ensureSession resolution|closes a late-resolving real
handle exactly once"` (2 passed)
- `pnpm -r typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- `pnpm --filter @paperclipai/db check:migrations`
- `pnpm build`

## Risks

- Remote endpoint validation can reject configurations that previously
passed without checks.
- OAuth configuration errors can block setup until the operator corrects
the provider settings.
- The migration uses guarded statements so repeated execution is safe.
- The test-only synchronization changes do not affect runtime behavior;
they remove filesystem/fake-clock and listener-registration races
observed under parallel CI load.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:32 -05:00
Dotta 03cfad7ceb
feat(apps): connect Notion through MCP OAuth (#11009)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps give agents governed access to external tools.
> - The Apps gallery lists Notion, but the server required manually
configured OAuth credentials.
> - Notion's hosted MCP server supports OAuth discovery and dynamic
client registration.
> - Notion also requires HTTPS or a loopback HTTP redirect URI.
> - This pull request adds a direct Notion MCP OAuth path with PKCE and
reusable dynamic clients.
> - It also adds the current Apps UI states for connect and
reauthorization.
> - The benefit is a secure Notion connection with no manual client
credential setup.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The Apps gallery, Apps connect route, OAuth token lifecycle, and managed
MCP gateway.

**Subsystem affected**

`server/`, `packages/shared/`, `scripts/`, and `ui/`.

**Current behavior**

The Notion gallery cards are disabled. The server uses the classic
Notion OAuth endpoints and requires operator-supplied client
credentials. It does not register an OAuth client from provider
metadata. Concurrent refreshes can also replay a rotating refresh token.

**Proposed behavior**

Enable the Notion Apps flow. Discover OAuth metadata from
`https://mcp.notion.com/mcp`. Register and reuse a public RFC 7591
client with PKCE. Require HTTPS or loopback HTTP callbacks. Serialize
refreshes, store each rotated refresh token before the new access token
can be used, and show a reconnect state for `invalid_grant`.

**Reason and benefit**

Operators can connect the built-in Notion MCP app without creating or
copying OAuth credentials. Paperclip keeps dynamic clients and rotating
tokens in the company secret store.

**Breaking changes**

None. Explicit environment client credentials still take priority.
Existing Slack and Linear OAuth endpoint hints remain unchanged. Other
OAuth apps remain disabled unless they are allowlisted.

**Additional context**

PR #10910 is a related, broader Connections v3 wizard replacement. This
PR is the focused current Apps flow. The MCP Tool Gateway and Connected
Apps items in `ROADMAP.md` cover this planned capability.

## What Changed

- Classify all 20 reviewed Notion MCP tools with provider-scoped read
and write defaults.
- Require approval for selected Notion mutations, including move,
duplicate, and convert actions that generic verb matching missed.
- Preserve company-scoped connection and catalog resolution for Notion
profiles and policies.
- Add RFC 7591 dynamic client registration with
`token_endpoint_auth_method=none` and mandatory PKCE.
- Store the dynamic client ID on the connection and store any returned
client secret in the company secret store.
- Reuse the registered client for later connects and keep explicit
environment credentials as the first choice.
- Discover protected-resource and authorization-server metadata from the
Notion MCP endpoint.
- Add `redirectConstraints: "https-or-loopback-http"` to the generated
Notion app definition and shared contract.
- Reject non-loopback plain HTTP callbacks before network access with a
TLS setup error.
- Serialize client registration and token refresh operations within the
server process.
- Store a rotated refresh token before publishing the refreshed access
token.
- Treat `invalid_grant` as terminal and move the connection to a clear
reauthorization state.
- Add focused coverage for registration reuse, callback constraints,
refresh rotation, and terminal grants.
- Enable the Notion Apps route and add connect, redirect, success,
error, and reconnect UI states.
- Keep non-allowlisted OAuth apps blocked and cover the UI policy with
regression tests.

## Verification

- The focused Notion policy integration test passed with embedded
PostgreSQL.
- The focused 20-tool classification test passed.
- The server typecheck passed on the governance head.
- `pnpm -r typecheck` passed on the rebased head.
- `pnpm --filter @paperclipai/server typecheck` passed after the
security follow-up.
- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
-t \u0027DCR|refresh tokens|invalid_grant|abandoned lease\u0027` passed
10 focused security tests.
- `pnpm build` passed on the rebased head.
- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
-t 'OAuth|oauth'` passed 14 tests.
- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts`
passed 5 tests.
- The complete server group passed 3,686 tests with 4 skipped.
- The complete UI group passed 3,656 tests.
- The full local runner found one environment-only CLI failure because
this agent runtime injects static AWS credentials into a test that
expects `AWS_PROFILE` only. `env -u AWS_ACCESS_KEY_ID -u
AWS_SECRET_ACCESS_KEY pnpm exec vitest run
cli/src/__tests__/secrets.test.ts` passed all 8 tests.
- The prior UI verification passed 55 focused tests, `pnpm
check:token-gates`, the Storybook build, and review of six 1440 x 1000
screenshots.
- OAuth request sequence: protected-resource metadata `GET
https://mcp.notion.com/.well-known/oauth-protected-resource/mcp`;
authorization metadata `GET
https://mcp.notion.com/.well-known/oauth-authorization-server`; dynamic
registration `POST https://mcp.notion.com/register`; authorization `GET
https://mcp.notion.com/authorize`; token exchange and refresh `POST
https://mcp.notion.com/token`; MCP traffic `POST
https://mcp.notion.com/mcp`.
- The live metadata and registration probe confirmed that Notion accepts
HTTPS and loopback HTTP redirects. It rejects a plain HTTP private
hostname.
- A later QA task owns the full browser consent and managed gateway
tool-list dry run against a configured HTTPS deployment.

## Risks

- Notion can add tools. Unrecognized names use the generic classifier,
and new or changed risky tools stay quarantined after connection
activation.
- A deployment that uses a private non-loopback hostname must configure
HTTPS before it can connect Notion.
- Dynamic registration creates a provider-side client. Paperclip reuses
it because registration does not provide a standard delete operation.
- Refresh coordination uses a database CAS lease across service
instances. An unclean crash leaves an uncertain lease and requires
reconnect instead of risking refresh-token replay.
- The current Apps surface overlaps with PR #10910. Merge order can
require a small conflict resolution if that PR lands first.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex on a GPT-5 runtime. The exact deployment ID and context
window are not exposed. The runtime used reasoning, repository tools,
code execution, and network tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 22:18:08 -05:00
Dotta d23fbf8ae4
feat(connections): add AppDefinition Wave 1 catalog (#9981)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Connections is the subsystem that defines which external apps and
MCP-style integrations operators can browse, configure, and run
> - The v3 schema core in #9958 added stable connection identities, auth
metadata, and grant-aware contracts, but the app catalog still used the
older gallery shape
> - The product needs a richer, typed AppDefinition catalog so browsing
and setup can render provider-specific auth and configuration
requirements consistently
> - This pull request moves the Wave 1 app catalog onto generated
AppDefinition data and carries that shape through shared types, server
lookup paths, and app connection UI
> - The benefit is that follow-up runtime and wizard work can build
against one catalog contract instead of local-only mock/gallery data

## Linked Issues or Issue Description

Refs #9958.

No public GitHub issue exists for this branch. This is the catalog layer
for the Connections v3 stack after the schema-core foundation in #9958.

## What Changed

- Adds generated AppDefinition data for the Wave 1 catalog and ingestion
reporting.
- Replaces the legacy tool app gallery exports with
AppDefinition-centered shared contracts, validators, and tests.
- Updates server tool-access lookup behavior to use the AppDefinition
catalog.
- Updates app connection UI surfaces and tests to consume
AppDefinition-backed catalog data.
- Documents the catalog ingestion workflow in the connector playbook.

## Verification

- `pnpm run preflight:workspace-links`
- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts
packages/shared/src/app-definitions-url.test.ts
ui/src/pages/apps/AppsConnect.test.tsx
server/src/__tests__/tool-access-service.test.ts`

## Risks

- Medium: this changes the catalog contract used by shared, server, and
UI app connection surfaces.
- Catalog data quality matters because generated definitions now drive
browse/setup display.
- Follow-up runtime and wizard PRs must rebase on this branch or on
master after this lands.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 Codex coding agent with repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-21 15:57:12 -05:00