diff --git a/AGENTS.md b/AGENTS.md index 4547b149c4..486b3d8cb0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,6 +20,12 @@ Before making changes, read in this order: `doc/SPEC.md` is long-horizon product context. `doc/SPEC-implementation.md` is the concrete V1 build contract. +When adding or changing an Apps catalog connection, also follow +`doc/connections/CONNECTOR-PLAYBOOK.md`. It is the canonical connection +authoring runbook for provider research, supported transport/auth patterns, +credential handling, branding, implementation, testing, live proof, and PR +submission. + ## 3. Repo Map - `server/`: Express REST API and orchestration services diff --git a/doc/connections/CONNECTOR-PLAYBOOK.md b/doc/connections/CONNECTOR-PLAYBOOK.md index ceef673a5d..fb65d5fb0f 100644 --- a/doc/connections/CONNECTOR-PLAYBOOK.md +++ b/doc/connections/CONNECTOR-PLAYBOOK.md @@ -1,6 +1,15 @@ -# Connector Playbook: Add A Vendor As A Catalog Entry +# Connection Authoring Runbook -This playbook is the repeatable template for adding a vendor to the Apps catalog as data, not as a plugin. It follows the accepted connections framework in [PAP-13211](/PAP/issues/PAP-13211), the first-30 rollout matrix in [PAP-2432](/PAP/issues/PAP-2432), and the production validation scope in [PAP-12373](/PAP/issues/PAP-12373). +Audience: agents and engineers researching, implementing, testing, reviewing, +and shipping Paperclip app connections. + +Status: canonical end-to-end authoring guide for Apps v2 catalog connections. + +This runbook is the repeatable, agent-executable procedure for adding a vendor +to the Apps catalog as data, not as a plugin. It follows the accepted +connections framework in [PAP-13211](/PAP/issues/PAP-13211), the first-30 +rollout matrix in [PAP-2432](/PAP/issues/PAP-2432), and the production +validation scope in [PAP-12373](/PAP/issues/PAP-12373). Use it when Paperclip acts on an external system through a governed connection: a stored credential, a capability catalog, access profiles and policy rules, and audit. Inbound integrations, such as an external client acting on Paperclip, use gateway or webhook guidance instead. @@ -26,6 +35,22 @@ Paperclip resolves short-lived tokens at invocation time. Before writing a connector, read [Identity vs. connections](./README.md#identity-vs-connections) for the P1/P2/P3 boundary and the D7 standing rule. +## Contents + +- [Mental model and support matrix](#mental-model-five-independent-axes) +- [Setup-to-runtime architecture](#architecture-from-setup-to-agent-call) +- [Secret storage and lifecycle](#secret-storage-and-lifecycle) +- [Current access defaults](#current-default-access-policy) +- [Golden-path agent tutorial](#golden-path-agent-tutorial) +- [AppDefinition field reference](#appdefinition-field-reference) +- [Troubleshooting](#troubleshooting-and-failure-classification) +- [Definition of done](#definition-of-done) +- [Detailed design checklist](#detailed-design-checklist) +- [Hosted MCP and OAuth protocol notes](#mcp-direct-connections-hosted-mcp--oauth) +- [Connection proposal template](#template) +- [Linear worked example](#appendix-linear-dry-run) +- [Notion DCR worked example](#appendix-notion-dry-run-mcp-direct-with-dcr) + ## Output A complete connector proposal produces: @@ -35,9 +60,977 @@ A complete connector proposal produces: - Credential secret refs into `company_secrets`; never raw env values. - Action catalog metadata with risk classes, schemas, resource filters, and quarantine defaults. - Default profile and policy behavior for read, write, and destructive actions. -- A smoke checklist aligned to [PAP-12373](/PAP/issues/PAP-12373): connect, discover catalog, allowed read call, ask-first write call, denied/quarantined call, revoke, and audit evidence. +- A smoke checklist aligned to [PAP-12373](/PAP/issues/PAP-12373): connect, + discover catalog, allowed read call, correctly governed write call, + denied/quarantined call when the method declares one, revoke, and audit + evidence. -## Step 1: Confirm It Is A Catalog Entry +## Use This Document As The Checklist + +An agent implementing a connection should be able to begin with only a provider +name and this document. Work in order. Do not jump from finding an MCP URL to +adding a store card; the research, credential, risk, branding, deterministic +test, live proof, and PR steps are all part of the feature. + +The shortest valid implementation usually changes these files: + +```text +scripts/ingest-app-definitions.mjs # human-authored definition source +packages/shared/src/app-definitions/.json # generated definition +packages/shared/src/app-definitions.generated.ts # generated registry +ui/public/brands/apps/.svg # official, sanitized mark +ui/public/brands/apps/manifest.json # branding provenance +packages/shared/src/app-definitions.test.ts # manifest/provider assertions +``` + +Add server or UI code only when the provider cannot be represented by the +existing contract. Prefer extending one generic capability with fixtures over +adding a provider-name branch. Provider branches are justified for behavior +that cannot be inferred safely, such as a provider's reviewed risk exceptions, +permanently blocked actions, or a protocol-required managed argument. + +The rest of this document has three levels: + +1. **Mental model and support matrix** — choose the right connection type. +2. **Golden-path tutorial** — research, implement, test, prove, and submit. +3. **Reference and appendices** — field semantics and worked examples. + +## Mental Model: Five Independent Axes + +Do not describe a connection as merely "an OAuth connection" or "an MCP +connection." OAuth is authentication. MCP is transport. A complete method +chooses all five axes below. + +| Axis | Current values | Question | +| --- | --- | --- | +| Transport | `mcp_remote`, `local_stdio`, `rest_api` | How does Paperclip reach actions? | +| Authentication | `oauth`, `api_key`, `none` | How does the provider authorize requests? | +| OAuth client ownership | `dcr`, `customer`, `platform_shared`, `platform_provisioned` | Who supplies and controls the OAuth client registration? | +| Credential source | `paperclip_vault`, reviewed `vercel_connect` | Where does durable provider credential material live? | +| Grant identity | `organization`, `user` | Does the credential act for the company or one person? | + +These axes produce combinations such as: + +- Remote MCP + DCR OAuth + Paperclip vault + organization identity: Jira. +- Remote MCP + customer OAuth app + Paperclip vault: Asana. +- Remote MCP + DCR or customer OAuth app: Notion and PostHog. +- Remote MCP + API key in an HTTP header: Mem0 and PagerDuty. +- Remote MCP + secret-bearing provider-generated URL: Zapier. +- Remote MCP + no auth + required tenant field: Shopify. +- Remote MCP + Paperclip-managed OAuth client + per-user grant: Google + Workspace MCP previews. +- Local stdio MCP + approved command template: the Google Sheets robot flow and + development fixtures. +- REST API parent + provider-specific child-session bridge: Composio. This is a + specialized implementation, not a generic REST catalog recipe. + +### Transport support and boundaries + +| Transport | Manifest-only? | Runtime status | Authoring rule | +| --- | --- | --- | --- | +| `mcp_remote` | Yes | First-class discovery, health, catalog, gateway, test, OAuth, and credential projection. | Default for official hosted MCP servers. | +| `local_stdio` | Only with an approved template | First-class only through registered templates and a trusted runtime host. Disabled in authenticated/public deployments without that host. | Never put an arbitrary command in an `AppDefinition`. Register and test a template. | +| `rest_api` | No, not generally | Not exposed through the connected MCP gateway. Composio is a provider-specific parent that creates MCP-capable children. | Do not add a generic REST/API card until an execution adapter or wrapper exists. | + +`api_key` in a method means an authentication mode; it does not mean the +transport is a REST API. Most current API-key catalog entries authenticate a +remote MCP server. + +For `mcp_remote`, header credentials and secret-bearing generated URLs have the +complete generic runtime path. The schema also names `query`, `body_json`, and +`env` key placements for specialized transports, but accepting a value in the +schema is not proof that the remote MCP gateway projects it. Do not ship one of +those placements without tracing the invocation path and adding an end-to-end +fixture. `env` belongs primarily to approved local stdio templates. + +### Authentication support matrix + +| Pattern | Definition shape | What the user sees | What Paperclip stores | +| --- | --- | --- | --- | +| Automatic OAuth | `auth: "oauth"`, `ownershipModes: ["dcr"]` | Browser sign-in | DCR/CIMD client binding plus token secret refs. | +| Automatic OAuth with own-app escape hatch | `ownershipModes: ["customer", "dcr"]` | Recommended browser sign-in; own client under **Advanced** | Same as automatic, or supplied client ID plus encrypted client secret. | +| Customer OAuth only | `ownershipModes: ["customer"]` | Required client ID and optional/required client secret, then browser sign-in | Client ID in redacted config; client secret and provider tokens as secret refs. | +| Paperclip-managed OAuth | `oauthStrategy: "paperclip_id_connector"`, `connectorProfile`, `platform_shared` | Browser sign-in through the managed connector | Provider tokens still land in the instance vault on a user grant. Paperclip ID does not retain them. | +| API key/PAT | `auth: "api_key"`, `credentialFields`, `keyPlacement` | Write-only credential field | Encrypted secret version plus placement-only refs. | +| Generated URL | `auth: "none"`, no fixed URL/default template | Paste provider-generated MCP URL | Public URL shape in config; full secret-bearing URL in the vault. | +| No auth | `auth: "none"`, fixed `serverUrl` or validated `serverUrlTemplate` | Zero fields or only required tenant/resource fields | No provider credential. | + +### OAuth client resolution order + +For standard OAuth methods, Paperclip resolves a client in this order: + +1. Deployment-preconfigured provider client. +2. Client ID Metadata Document (CIMD), when advertised and the instance has a + public HTTPS URL. +3. Dynamic client registration (DCR/RFC 7591), when advertised and allowed by + the curated method. +4. A customer-created client ID and secret supplied through setup. + +For a curated method, `ownershipModes` is an allowlist. A method containing +only `customer` must not silently fall through to generic DCR. A method +containing `dcr` permits the automatic CIMD/DCR tiers. Deployment-preconfigured +credentials still take precedence when present. + +### Credential source is not OAuth client ownership + +`ownershipModes` says who owns the OAuth client registration. It does not say +where provider access tokens live. By default, access tokens, refresh tokens, +client secrets, and API keys live in the Paperclip instance vault. + +`credentialSource: "vercel_connect"` is a separately reviewed exception for +specific methods. Such a connection stores a Vercel connector reference and no +Paperclip provider secret refs. Never make a method accept both sources in the +same connection, and never infer Vercel eligibility from a provider name. + +## Architecture From Setup To Agent Call + +```mermaid +flowchart LR + D["AppDefinition"] --> S["Shared validation and gallery"] + S --> W["Connection setup flow"] + W --> C["tool_connections draft"] + W --> V["Instance encrypted vault"] + V --> R["Credential secret refs"] + C --> H["Health and catalog discovery"] + R --> H + H --> A["Risk-classified tool catalog"] + A --> P["Profiles, bindings, policies"] + P --> G["Run-scoped tool gateway"] + G --> X["Provider MCP server"] + X --> G + G --> U["Redacted result and audit"] +``` + +The agent never receives a durable provider credential. A run receives a +Paperclip gateway capability. At invocation time the gateway rechecks company, +connection, grant, catalog, profile, policy, and run state; resolves the needed +secret version; projects only the reviewed headers/arguments; calls the +provider; and writes redacted audit evidence. + +## Secret Storage And Lifecycle + +Connection credentials use the same secrets architecture as agent, project, +and routine environment bindings. They are not files in a provider-named +folder. + +The durable pieces are: + +- `company_secrets`: secret identity, company/user scope, provider metadata, + and ownership. +- `company_secret_versions`: encrypted or externally referenced version + material. +- `company_secret_bindings`: which connection/grant/config path may resolve a + secret. +- `secret_access_events`: audited resolution activity. +- `tool_connections.credentialSecretRefs` and + `connection_grants.credentialSecretRefs`: value-free pointers and config + paths. +- `tool_connections.credentialRefs`: value-free projection shape such as the + HTTP header name and prefix. + +On the default local provider, values are encrypted with the instance master +key under `~/.paperclip/instances//secrets/master.key`. A usable +backup requires both the database and this key. Hosted provider-vault behavior +is configured under Company Settings; the connection contract remains refs, +not raw values. + +Credential handling by pattern: + +- API keys are created through `secretService.create`, then only their refs are + attached to the connection or selected grant. +- OAuth access and refresh tokens use `oauth.access_token` and + `oauth.refresh_token` refs. Refresh rotates versions under a lease so two + servers do not replay a rotating refresh token. +- Customer OAuth client secrets use an encrypted `oauth.client_secret` ref. + Client IDs are identifiers and may remain in redacted connection config. +- A generated URL containing credentials is split. Paperclip stores a safe URL + for display/routing and vaults the complete URL. The gateway verifies that + the secret URL still matches the public URL before use. +- Personal credentials live on a `user` grant and user-scoped secret rows. + Organization credentials live on the connection/default organization grant. + Choosing "Just me" must never first create or silently fall back to a shared + organization credential. +- Failed setup removes newly created orphan secrets. Removal/revocation clears + OAuth state, grant credentials, gateway access, and owned secret material; + shared secrets used by another consumer are retained. + +Never put raw credentials in any of these places: + +- `AppDefinition` JSON or the ingestion script +- connection `config` or `transportConfig` +- application metadata +- agent, project, or routine plain environment values +- test fixtures committed to git +- issue comments, activity details, screenshots, traces, HAR files, or console + output +- PR descriptions or live-proof evidence + +Tests must assert absence, not merely avoid printing a secret during the happy +path. + +## Current Default Access Policy + +The current product behavior is encoded by `recommendedDefaultsForApp` in +`packages/shared/src/app-definitions.ts`: + +- Every discovered action is enabled during successful setup. +- S1-S3 methods default their actions to **Allowed**, including writes. +- S4 methods default `write` and `destructive` actions to **Ask first**. +- Permanently blocked provider actions stay disabled. +- Provider/schema-specific changed-tool quarantine remains a separate catalog + concern; do not turn writes Off as a substitute for correct risk + classification. + +If a destructive provider cannot be safe with those defaults, classify the +method S4 or add a narrowly reviewed provider policy with tests. Do not hide a +dangerous tool by misclassifying it as read, and do not silently change global +defaults in a provider PR. + +## Golden-Path Agent Tutorial + +This is the implementation sequence an autonomous coding agent should follow. + +### Phase 0: Establish scope and preserve the worktree + +1. Read `AGENTS.md`, `doc/GOAL.md`, `doc/PRODUCT.md`, + `doc/SPEC-implementation.md`, `doc/DEVELOPING.md`, and `doc/DATABASE.md`. +2. Read this runbook, [the connections overview](./README.md), and + [the security threat model](./SECURITY-THREAT-MODEL.md). +3. Inspect `git status --short`. Existing changes belong to the user or another + task. Do not reset, rewrite, or format unrelated files. +4. Decide whether the request is research-only, definition-only, or a new + runtime capability. A store card is not evidence of runtime support. +5. Write acceptance criteria before editing. At minimum: actionable store card, + setup completes, credentials are vaulted, tools list, one safe read runs, + refresh/reconnect works, revoke blocks use, and no secret appears in API or + logs. + +### Phase 1: Research the provider from primary sources + +Use current official provider documentation and live protocol metadata. Search +results and Vercel captures are leads, not authority. + +Record this evidence: + +- Official product/docs URL and date verified. +- Exact MCP endpoint, including path and trailing-slash behavior. +- Whether the endpoint uses Streamable HTTP or an older SSE path. +- Unauthenticated response status and `WWW-Authenticate` challenge. +- RFC 9728 protected-resource metadata URL. +- RFC 8414/OIDC authorization-server metadata URL and exact issuer. +- Authorization, token, registration, and revocation endpoints when published. +- PKCE support and token endpoint auth methods. +- Whether CIMD or DCR is actually advertised. +- Required scopes. Separate documented minimum scopes from the full discovery + list. +- Access/refresh lifetime, rotation behavior, and terminal refresh errors. +- Redirect URI constraints: HTTPS, loopback HTTP, exact callback allowlisting, + or reviewed-client requirements. +- Normal prerequisites: account, paid plan, tenant feature flag, administrator + consent, preview enrollment, region, project/site identifier. +- Whether Paperclip itself needs provider approval. Customer-admin approval is + self-serve; provider approval of Paperclip is not. +- Tool/action inventory, provider annotations, known destructive actions, and + resource boundaries. +- Revocation procedure and whether a provider endpoint exists. + +Safe research may fetch public metadata, but it must not perform dynamic client +registration. Paperclip's catalog preflight is intentionally non-registering: + +```text +GET /api/companies/:companyId/tools/apps/:galleryKey/preflight?methodKey= +``` + +Registration and consent happen only after an explicit Connect action. If the +provider requires Paperclip approval or redirect allowlisting that a customer +cannot complete, retain the research entry with an unavailable reason and do +not expose a connect action. + +For researched self-serve MCP providers, update the durable evidence ledger in +`packages/shared/src/self-serve-mcp-research.json` and the dated program plan +when eligibility or endpoints change. + +### Phase 2: Choose the lightest valid product shape + +Use this decision tree: + +```mermaid +flowchart TD + A["Provider connection request"] --> B{"Official remote MCP server?"} + B -->|Yes| C{"Can common auth and fields represent it?"} + C -->|Yes| D["Curated AppDefinition"] + C -->|No| E{"One reusable protocol capability?"} + E -->|Yes| F["Extend common connection runtime plus fixtures"] + E -->|No| G["Provider wrapper or plugin"] + B -->|No| H{"Existing generic execution adapter?"} + H -->|No| I["Do not ship a nonfunctional REST card"] + H -->|Yes| J["Use the adapter through normal connection governance"] + G --> K{"Needs UI, tables, workers, migrations, or webhooks?"} + K -->|Yes| L["Plugin that provisions normal connections"] + K -->|No| F +``` + +Default to an `AppDefinition` for hosted remote MCP. Use a plugin only for +custom product surfaces, tables, workers, migrations, ingestion loops, +webhooks, or other real code ownership. A plugin still provisions normal +connections and cannot bypass secrets, grants, profiles, policy, gateway, or +audit. + +### Phase 3: Design methods and the setup experience + +For every real user choice, create a separate method. Do not create methods for +choices Paperclip can infer. + +Good separate methods: + +- US OAuth versus EU API-key endpoints. +- Read versus write capability profiles when the provider publishes distinct + servers or scope sets. +- Standard browser sign-in versus a materially different API-key path. +- Distinct provider modes such as Postman's minimal, code, and full catalogs. + +Avoid separate methods for: + +- DCR versus CIMD. Paperclip chooses automatically. +- DCR versus a customer-owned OAuth app when both reach the same endpoint. + Keep browser sign-in recommended and fold "use your own OAuth app" under + **Advanced**. +- Optional project filters, read-only switches, or response tuning. These are + advanced fields with safe defaults. + +The default setup screen should ask only for information required to make the +connection work or enforce a real tenant boundary. Follow these rules: + +- Put optional narrowing in fields marked `advanced: true`. +- Give hidden fields a `defaultValue`; never create a hidden required field the + server cannot fill. +- Use `setupPrerequisite` for steps that must happen before credentials or + consent, such as preview enrollment or making a storefront public. +- Put account/plan/admin limitations in `warnings` and `guidanceMd`. +- Give every method a meaningful `label`, `whenToUse`, exact endpoint, risk + tier, and official `consoleLinks`. +- Use `capabilityProfile` for user-facing mode names. Do not infer defaults from + array order when one mode is the useful write-capable choice. +- Use `grantKinds: ["user"]` for providers that only support personal delegated + identity. +- Use `requiredResourceFilters` as reviewed policy metadata, but remember a + label is not enforcement. The provider, gateway, wrapper, or managed header/ + query projection must enforce the boundary. + +### Phase 4: Add official branding before exposing the app + +Every store-visible provider needs an official local mark. A letter tile is +only a runtime image-failure fallback. + +1. Find the provider's official brand kit, product site, or official repository. +2. Prefer an official SVG. Use a high-resolution transparent PNG only when no + official SVG is available. +3. Do not use Google's favicon proxy, scrape a random icon site, or generate an + imitation. +4. Sanitize SVGs. Reject scripts, `foreignObject`, event-handler attributes, + external executable content, or unsafe references. +5. Save assets under `ui/public/brands/apps/`. Add a `-dark` variant only when + the normal mark loses contrast in dark mode. +6. Add the provider to `ui/public/brands/apps/manifest.json` with slug, local + asset, optional dark asset, official source URL, exact upstream asset URL, + asset type, visibility, and dark-variant requirement. +7. Let the ingestion script derive `branding.logoUrl` and `darkLogoUrl` from the + provenance manifest. + +The manifest test decodes PNG headers, requires at least 128 by 128 pixels, +sanity-checks SVG markup, verifies files exist, and requires store-visible +definitions and visible provenance entries to match exactly. + +### Phase 5: Author the definition at the durable source + +The checked-in provider JSON files are generated. Do not edit one and stop. + +1. Add or update the provider in `scripts/ingest-app-definitions.mjs`. +2. Update `packages/shared/src/self-serve-mcp-research.json` when it belongs to + that program. +3. Add branding provenance and assets first; generation fails closed when + branding is missing. +4. Regenerate definitions: + +```sh +pnpm connections:ingest-app-definitions +``` + +The default ingestion corpus is the Vercel research checkout at +`../../paperclip-content/research/connections/vercel/templates`. Override it +when necessary: + +```sh +PAPERCLIP_CONTENT_TEMPLATES=/absolute/path/to/templates \ + pnpm connections:ingest-app-definitions +``` + +Generation currently validates the 99-capture corpus and rewrites provider +JSON, the generated TypeScript registry, and the ingestion report. A PR must +contain the human-authored source and generated output. Inspect the diff after +generation; do not accept unrelated provider churn. + +Minimal automatic OAuth example: + +```json +{ + "key": "mcp-oauth", + "label": "Sign in with Example", + "transport": "mcp_remote", + "auth": "oauth", + "ownershipModes": ["dcr"], + "whenToUse": "Use browser sign-in for the hosted MCP server.", + "defaults": { + "serverUrl": "https://mcp.example.com/mcp", + "scopesHint": ["example.read", "example.write"] + }, + "guidanceMd": "Connect the workspace agents should use.", + "consoleLinks": { + "docs": "https://docs.example.com/mcp" + }, + "riskTier": "S3" +} +``` + +Minimal customer OAuth example: + +```json +{ + "key": "mcp-own-oauth", + "label": "Use your own OAuth app", + "transport": "mcp_remote", + "auth": "oauth", + "ownershipModes": ["customer"], + "whenToUse": "Register an OAuth app, then enter its client ID and secret.", + "defaults": { + "serverUrl": "https://mcp.example.com/mcp" + }, + "guidanceMd": "Register Paperclip's callback URI in the provider console.", + "consoleLinks": { + "register": "https://example.com/developers/apps", + "docs": "https://docs.example.com/mcp/oauth" + }, + "riskTier": "S3" +} +``` + +Minimal API-key example: + +```json +{ + "key": "mcp-api-key", + "label": "Use an API key", + "transport": "mcp_remote", + "auth": "api_key", + "ownershipModes": ["customer"], + "whenToUse": "Use a restricted key from the provider console.", + "defaults": { + "serverUrl": "https://mcp.example.com/mcp" + }, + "credentialFields": [ + { + "key": "authorization", + "label": "Example API key", + "type": "password", + "required": true, + "placeholder": "ex_...", + "secret": true + } + ], + "keyPlacement": { + "location": "header", + "name": "Authorization", + "prefix": "Bearer " + }, + "guidanceMd": "Create a key limited to the resources agents need.", + "riskTier": "S3" +} +``` + +Generated-URL example: + +```json +{ + "key": "generated-url", + "label": "Paste generated MCP URL", + "transport": "mcp_remote", + "auth": "none", + "ownershipModes": ["customer"], + "whenToUse": "Paste the complete server URL generated by the provider.", + "defaults": {}, + "guidanceMd": "Create a server in the provider, then paste its URL.", + "riskTier": "S3" +} +``` + +No-auth tenant-template example: + +```json +{ + "key": "public-mcp", + "label": "Public storefront", + "transport": "mcp_remote", + "auth": "none", + "ownershipModes": ["customer"], + "whenToUse": "Connect a public tenant endpoint.", + "defaults": { + "serverUrlTemplate": "https://{tenantDomain}/api/mcp" + }, + "tenantFields": [ + { + "key": "tenantDomain", + "label": "Tenant domain", + "type": "text", + "required": true, + "placeholder": "store.example.com", + "validation": { + "pattern": "^[A-Za-z0-9.-]+$", + "maxLength": 255 + } + } + ], + "guidanceMd": "Enter the permanent public tenant domain.", + "riskTier": "S2" +} +``` + +Use `defaults.toolArgumentDefaults` only for required, provider-documented +protocol metadata that Paperclip owns, not to force a user's business input. +Managed arguments are deep-merged after caller input and win on collisions; the +same fields are removed from the agent-visible and Test-tab input schema. + +### Phase 6: Add generic runtime support only when needed + +Before adding code, prove the manifest cannot express the provider. + +Common extension points: + +- `packages/shared/src/types/app-definition.ts` and + `packages/shared/src/validators/app-definition.ts` for a reusable manifest + capability. +- `normalizeConnectionMethodConfig` for validated tenant/extension fields and + header/query projection. +- `projectedConnectionHeaders`, `projectedConnectionToolArguments`, and + `projectedConnectionToolInputSchema` for server-managed request material. +- OAuth discovery/client/token/refresh functions in + `server/src/services/tool-access.ts`. +- MCP invocation in `server/src/services/tool-gateway.ts`. +- `classifyRisk` only for reviewed provider exceptions that generic annotations + and name classification cannot represent safely. +- `ConnectionSetupFlow` only when a schema-driven setup capability genuinely + cannot render the flow. + +When adding a reusable field: + +1. Update shared TypeScript types. +2. Update the Zod validator with cross-field invariants. +3. Update server normalization and invocation projection. +4. Update UI rendering and request types. +5. Add fixture tests for valid, invalid, redacted, reconnect, and invocation + behavior. +6. Document the new field here. + +Keep `ConnectToolApp` synchronized across UI and server. Never create a UI-only +request shape that drops `connectionMethodKey`, `oauthClient`, `grantKind`, or +credential source data. + +### Phase 7: Write deterministic tests before using a real account + +At minimum, add or update tests in these layers: + +**Manifest** + +- Schema validates. +- Slug and method keys are unique and stable. +- Endpoint, scopes, ownership modes, risk tier, prerequisite, and field + placement equal the reviewed values. +- Store visibility matches the intended rollout state. +- Official local branding and provenance exist. +- Hidden fields have defaults; required fields have placeholders. +- The default setup path asks only for truly required configuration. + +**Server/service** + +- Method selection is required when multiple methods are real choices. +- Tenant fields normalize, validate, and reach the exact URL/header/query. +- Unknown config fields are rejected. +- API keys/client secrets/tokens become encrypted refs and never appear in the + response. +- Failed setup removes newly created secrets and draft rows when appropriate. +- DCR, CIMD, and manual-client paths use fixture metadata and never require a + real provider. +- Scope widening beyond `scopesHint` is rejected. +- OAuth state, actor/session, issuer, redirect, resource, and company bindings + fail closed. +- Refresh rotates safely; terminal `invalid_grant` requires reauthorization. +- Health, catalog refresh, reconnect, removal, and secret cleanup work. +- SSRF, private/link-local address, redirect, header-name, and header-value + protections remain intact. +- Company A cannot observe or invoke company B's connection. +- Managed arguments/headers cannot be spoofed by caller input. +- Risk exceptions classify every reviewed tool correctly. + +**UI** + +- Browse has an actionable route for an available capability-backed definition. +- Hidden/unavailable providers do not show a dead Connect button. +- Direct `/apps/connect?source=` opens the selected provider, not the + generic gallery. +- Automatic OAuth, customer OAuth, API key, generated URL, no-auth, required + tenant field, prerequisite, warning, and Advanced disclosures render as + declared. +- Finish setup resumes the exact draft using `resumeConnectionId`. +- Optional customer OAuth details stay folded when automatic OAuth exists. +- Setup success leads to the connection's Test page. +- Missing images fall back at runtime, while manifest acceptance still fails + missing branding. + +Useful focused command: + +```sh +pnpm exec vitest run \ + packages/shared/src/app-definitions.test.ts \ + server/src/__tests__/tool-access-service.test.ts \ + server/src/__tests__/generic-mcp-connection.test.ts \ + server/src/__tests__/tool-connection-removal.test.ts \ + ui/src/pages/apps/AppsConnect.test.tsx \ + ui/src/pages/apps/Browse.test.tsx +``` + +Use `-t ''` while iterating, then run each affected file +without a test-name filter before handoff. + +Targeted type checks: + +```sh +pnpm --filter @paperclipai/shared typecheck +pnpm --filter @paperclipai/server typecheck +pnpm --filter @paperclipai/ui typecheck +``` + +If UI code changed, also run: + +```sh +pnpm check:token-gates +``` + +### Phase 8: Start an isolated instance and verify the setup UI + +Use a worktree-local instance; never point two worktrees at the same embedded +database. + +```sh +paperclipai worktree init +pnpm dev +``` + +Confirm the actual port with `pnpm dev:list` and verify health: + +```sh +curl -fsS http://localhost:/api/health +curl -fsS http://localhost:/api/companies +``` + +Walk the user path: + +1. Open `//apps`. +2. Confirm branding, copy, visibility, ordering, and Connect state. +3. Open `//apps/connect?source=` directly. +4. Verify prerequisites and warnings appear before credentials/consent. +5. Exercise every method. Do not test only the default method. +6. Cancel or interrupt once. Confirm the store shows **Finish setup** and that + it returns to + `?source=&resume=` without creating another draft. +7. Complete setup. Confirm the connection is active/healthy and opens + `//apps//test`. + +For OAuth, the instance callback must be browser-reachable and must match the +provider registration. Loopback HTTP is acceptable only when provider and +Paperclip redirect policies permit it. A worktree exposed through HTTPS needs a +unique, correct `PAPERCLIP_PUBLIC_URL`; internal service hostnames are not valid +browser callback origins. + +Use the browser signed-in session only for an explicitly authorized live proof. +Do not inspect cookies, storage, saved passwords, or unrelated account data. + +### Phase 9: Perform the real-provider proof + +Deterministic fixtures prove Paperclip logic. A store-ready provider also needs +one account-bound proof for every method being exposed. + +Run this exact lifecycle: + +1. **Preflight** — public metadata only; no registration or credentials. +2. **Connect** — finish provider consent or enter the credential. +3. **Catalog** — list tools and compare them with reviewed expectations. +4. **Safe read** — execute one narrow, non-mutating action in the Test page. +5. **Write classification** — confirm known writes/destructive actions appear + in the correct risk group and default policy. +6. **Agent path** — run one action through an actual agent/run-scoped gateway, + not only the board Test helper, when the connection changes gateway logic. +7. **Refresh/reconnect** — refresh the catalog, reconnect or force a safe token + refresh, and repeat the safe read. +8. **Revoke/remove** — revoke at the provider or remove in Paperclip. Confirm + tools disappear or calls fail closed immediately. +9. **Reconnect after removal** — when supported, confirm the retained identity + and history are reused rather than duplicated. +10. **Secret inspection** — inspect API responses, application logs, activity, + audit, screenshots, and evidence artifacts for the exact canary credential. + It must be absent. + +Evidence should record only: + +- provider and method key +- date, environment, endpoint origin/path, and connection ID when non-sensitive +- catalog tool names/counts and schema hashes +- policy/risk result +- redacted success/failure codes +- revoke/reconnect outcome + +Do not record token values, secret-bearing URLs, authorization codes, provider +session details, personal email, tenant content, HAR files, or pre-callback +screenshots containing provider data. + +When consent UI has an intentional delay or requires a real pointer/keyboard +interaction, honor that behavior before classifying it as failure. Keep provider +consent problems separate from callback, token exchange, catalog, and gateway +failures. + +### Phase 10: Verify APIs and audit without exposing credentials + +Useful read-only endpoints after setup: + +```text +GET /api/companies/:companyId/tools/connections +GET /api/tool-connections/:connectionId +GET /api/tool-connections/:connectionId/catalog +GET /api/tool-connections/:connectionId/activity?limit=50 +GET /api/tool-connections/:connectionId/grants +``` + +Mutating verification endpoints: + +```text +POST /api/tool-connections/:connectionId/health-check +POST /api/tool-connections/:connectionId/catalog/refresh +POST /api/tool-connections/:connectionId/test-calls +DELETE /api/tool-connections/:connectionId +``` + +Responses may contain secret IDs, version selectors, header names, prefixes, +scope names, expiry timestamps, and redacted provider metadata. They must not +contain secret values. Logs and activity should identify the operation and +outcome without echoing provider-authored credential-bearing errors. + +### Phase 11: Run the PR-ready verification ladder + +Run the smallest relevant suite first, then the full ladder when the change is +ready for review: + +```sh +pnpm check:token-gates +pnpm -r typecheck +pnpm test:run +pnpm build +``` + +Also run an affected Apps browser suite when routing, setup, OAuth popup, +finish-setup, Test page, or branding behavior changed: + +```sh +pnpm test:e2e +``` + +Do not hide failures. Classify each as introduced, pre-existing, environmental, +or live-provider-only, and include the exact command and result in the PR. + +Before staging: + +```sh +git diff --check +git status --short +git diff --stat +git diff -- \ + scripts/ingest-app-definitions.mjs \ + packages/shared/src/app-definitions \ + packages/shared/src/app-definitions.generated.ts \ + ui/public/brands/apps \ + server/src/services/tool-access.ts \ + server/src/services/tool-gateway.ts +``` + +Review every generated change. Verify no credential, provider account data, +unrelated worktree change, or temporary evidence file is staged. + +### Phase 12: Prepare and submit the pull request + +1. Keep commits scoped. A typical split is definition/branding, generic runtime + capability, and tests/docs. Do not split generated output from its source. +2. Do not commit `pnpm-lock.yaml`; GitHub Actions owns it in this repository. +3. Read `.github/PULL_REQUEST_TEMPLATE.md` immediately before writing the PR + body. +4. Fill every required section: + - **Thinking Path** — why this is a catalog entry, chosen transport/auth, + research evidence, and why no lighter path works. + - **What Changed** — manifest, branding, runtime, UI, tests, and docs. + - **Verification** — deterministic commands plus sanitized live proof. + - **Risks** — scopes, provider preview/admin gates, token behavior, catalog + drift, destructive actions, and rollback/de-list plan. + - **Model Used** — provider, exact model ID, context window, and relevant + capabilities, or the template's human-authored value. + - **Checklist** — every item checked truthfully. +5. In the PR, link official provider docs and exact metadata endpoints. Do not + link only to search results or third-party tutorials. +6. Mark live proof that was not run as outstanding; never equate a mocked OAuth + test with provider validation. +7. Do not merge as part of connection authoring unless the task explicitly + authorizes merging. + +Suggested PR verification block: + +```md +## Verification + +- `pnpm exec vitest run ` — passed +- `pnpm check:token-gates` — passed +- `pnpm -r typecheck` — passed +- `pnpm test:run` — passed +- `pnpm build` — passed +- Live `/` proof on ``: + connect ✓, list tools ✓, safe read ✓, refresh/reconnect ✓, revoke ✓, + secret scan ✓ +``` + +## AppDefinition Field Reference + +### App-level fields + +| Field | Meaning and rule | +| --- | --- | +| `schemaVersion` | Must be `1`. Change only with a versioned migration plan. | +| `slug` | Stable lowercase kebab-case identity. Never rename after connections exist without a migration. | +| `name` | Provider/product name shown to users. | +| `description` | Plain-language outcome, not protocol marketing. | +| `categories` | One or more supported catalog categories. | +| `featured` | Optional merchandising signal, not availability. | +| `branding` | Local official `logoUrl`, optional `darkLogoUrl`; ingestion derives this from provenance. | +| `urlPatterns` | HTTPS patterns used to recognize pasted/generated provider URLs. Keep narrow enough to reject lookalikes. | +| `docsUrl` | Current official setup/protocol docs. | +| `setupPrerequisite` | A prerequisite users must understand or complete before credentials/consent. Includes CTA and optional ordered steps. | +| `redirectConstraints` | Currently `https-or-loopback-http`; fail before provider navigation when violated. | +| `methods` | Every genuinely supported connection method. At least one. | +| `availability` | Instance/provider availability and user-facing reason. Unavailable entries must not expose a dead action. | +| `ownershipAvailability` | Deployment override for ownership modes. Defaults currently enable `customer` and `dcr`, disable platform modes. | + +### Method fields + +| Field | Meaning and rule | +| --- | --- | +| `key` | Stable method key stored on the connection as `connectionMethodKey`. | +| `label` | User-facing method label. Required in practice when multiple methods exist. | +| `transport` | `mcp_remote`, `local_stdio`, or specialized `rest_api`. | +| `auth` | `oauth`, `api_key`, or `none`. | +| `ownershipModes` | Allowed OAuth client ownership modes; also present for non-OAuth customer configuration. | +| `oauthStrategy` | Managed broker strategy. Currently `paperclip_id_connector`. Only valid for OAuth. | +| `connectorProfile` | Managed connector capability/scope profile, required with `oauthStrategy`. | +| `capabilityProfile` | User-facing read/write/mode grouping used for method selection. | +| `grantKinds` | Restricts identity to `organization` and/or `user`; omit for flexible methods. | +| `whenToUse` | One sentence distinguishing this method from alternatives. | +| `defaults.serverUrl` | Exact fixed endpoint. For discovery-capable OAuth, omit fixed auth endpoints. | +| `defaults.serverUrlTemplate` | HTTPS endpoint with placeholders supplied by declared tenant/extension fields. Mutually exclusive with `serverUrl`. | +| `defaults.discoveryUrl` | Provider-specific discovery override only when reviewed metadata requires it. | +| `defaults.authorizationEndpoint` / `tokenEndpoint` | Authoritative fixed endpoints. A complete pair bypasses discovery, so ship them only when the provider lacks trustworthy discovery. | +| `defaults.metadataUrl` | Authorization metadata hint. | +| `defaults.scopesHint` | Explicit reviewed allowlist. Omit scope when docs do not require one; never copy every discovered scope. | +| `defaults.oauthAuthorizationParams` | Reviewed `access_type=offline` and/or `prompt=consent` behavior. | +| `defaults.toolArgumentDefaults` | Server-owned provider protocol arguments, hidden from caller schemas and authoritative on collision. | +| `tenantFields` | Account/project/region/resource fields. Keep only required boundaries visible by default. | +| `extensionFields` | Additional method-specific configuration rendered by the same common form. | +| `configRequirements.atLeastOneOf` | Requires one of named tenant/extension fields. Every key must exist. | +| `credentialFields` | Write-only secret/non-secret credential inputs. API-key methods require them in practice. | +| `keyPlacement` | Provider request placement. Remote MCP should use the proven header path unless a new projection is implemented and tested. | +| `guidanceMd` | Setup and scoping guidance. No secrets or internal environment-variable names. | +| `consoleLinks` | Official registration, key, settings, and docs destinations. | +| `warnings` | Plan, preview, admin, financial, production-data, or destructive-action caveats. | +| `variants` | Legacy/simple variant metadata. Prefer explicit methods plus `capabilityProfile` for materially different endpoints/auth. | +| `riskTier` | S1-S4 provider/method sensitivity. Drives recommended policy defaults. | +| `requiredResourceFilters` | Reviewed resource boundaries. Must be backed by enforcement, not only copy. | +| `credentialSources.vercelConnect` | Reviewed services, principal modes, scopes, and header projection for the Vercel exception. | + +### Field definition rules + +`FieldDef` supports `text`, `password`, `textarea`, `datetime`, `select`, and +`checkbox`. + +- A required non-checkbox field needs a placeholder. +- A select needs at least one option. +- A hidden field needs a default value. +- Use `secret: true` only for write-only credential input; tenant/extension + config must never smuggle secrets into connection config. +- `advanced: true` keeps optional expert configuration folded. +- `validation.pattern` is a JavaScript regular expression string; also set a + practical `maxLength`. +- `transport.location: "query"` modifies the normalized server URL. +- `transport.location: "header"` creates a managed, validated non-secret + configuration header. It is separate from `keyPlacement`, which projects a + vaulted credential. +- CSV fields deduplicate comma/newline-separated values. +- `omitFalse` prevents a false checkbox from adding a query/header value. + +## Troubleshooting And Failure Classification + +| Symptom | Likely layer | What to inspect | +| --- | --- | --- | +| Store says Coming soon or Connect route is dead | Definition/availability/routing | `CONNECTABLE_APP_SLUGS`, store hidden set, method capability checks, Browse tests. | +| Direct source link shows generic connection chooser | UI route state | `AppsConnect`, `ConnectionSetupFlow`, source slug lookup, availability. | +| Finish setup opens Edit config and cannot continue | Draft identity/resume | `resumeConnectionId`, stored `sourceTemplateKey`, `connectionMethodKey`, exact draft status. | +| OAuth never redirects | Method capability/client resolution | ownership modes, metadata discovery, callback origin, manual-client requirement. | +| Provider rejects redirect URI | Deployment/provider rule | actual browser origin, `PAPERCLIP_PUBLIC_URL`, `redirectConstraints`, provider app registration. | +| OAuth succeeds then connection needs reconnect | Grant/secret sync or refresh | organization versus user grant, token refs, default grant sync, expiry/refresh lease, `invalid_grant`. | +| Tools list but calls return 401 | Token audience/scope/placement | RFC 8707 resource, `scopesHint`, header prefix/name, provider endpoint path. | +| Health works but Test call fails | Gateway projection/policy | selected grant, managed headers/arguments, effective profile/policy, catalog entry risk/status. | +| Required provider boilerplate appears in Test | Managed schema projection | `toolArgumentDefaults` and `projectedConnectionToolInputSchema`. | +| API key saves but is not sent | Unsupported placement or missing ref | `credentialFieldsFor`, `keyPlacement`, connection/grant refs, gateway header resolution. | +| Config asks for project ID the provider does not require | Manifest UX | make it optional/advanced, add default, or remove it; test the zero-config path. | +| Connected card still says Connect | Identity matching | application/source slug, retained app status, connection-to-definition association. | +| New tool is classified read | Risk inference | annotations, namespaced/camelCase verb normalization, provider exception set, fixture. | +| Connection from another company is visible | Authorization bug | stop; add company-scope negative tests before any further live testing. | +| Raw credential appears anywhere | Security incident | stop, revoke/rotate it, remove evidence, trace every response/log/audit path, add a canary regression test. | + +When a bug appears on one provider, first reproduce it with a fixture or a +second provider of the same auth/transport type. Fix the shared path when the +failure is generic. Keep provider workarounds narrow and documented. + +## Definition Of Done + +A catalog connection is ready only when every applicable item is true: + +- [ ] Official docs and live metadata agree on the endpoint and auth flow. +- [ ] Self-serve/provider-approval classification is documented. +- [ ] Every exposed method has completed the full live lifecycle. +- [ ] The default path asks only for required configuration. +- [ ] Optional own-OAuth and expert fields are folded under Advanced. +- [ ] Scopes are explicit and contained; caller widening is rejected. +- [ ] Official local branding and provenance pass validation in light and dark themes. +- [ ] Ingestion source and generated definitions are synchronized. +- [ ] Credentials and tokens are stored only as encrypted/external refs. +- [ ] Shared and personal grant semantics are correct. +- [ ] Health and catalog discovery succeed. +- [ ] One safe read succeeds in the Test page and, where relevant, an agent run. +- [ ] Writes/destructive tools are correctly classified and use current tier defaults. +- [ ] Refresh, reconnect, revoke/remove, and post-removal reconnect are verified. +- [ ] Company isolation, SSRF, OAuth binding, redaction, and cleanup tests pass. +- [ ] API responses, logs, activity, and evidence contain no credential values. +- [ ] Focused tests, token gates, typecheck, test suite, and build pass or exact blockers are reported. +- [ ] PR uses every section of the repository template and links official evidence. + +## Detailed Design Checklist + +The golden-path tutorial above is the operational sequence. The following +steps are the design-review checklist: use them when the connection proposal +needs a more formal transport, credential, action, and governance analysis. + +### Step 1: Confirm It Is A Catalog Entry Default to a catalog entry when the vendor can be represented as metadata plus a transport: @@ -54,7 +1047,7 @@ Use a plugin only when the integration needs code that cannot fit inside the com A plugin may bundle one or more catalog entries, but it must still create normal applications, connections, credential refs, catalog entries, profiles, policies, and audit events. Plugin code must not bypass the gateway, policy engine, `company_secrets`, changed-action quarantine, or call-event audit log. -## Step 2: Classify The Reuse Path +### Step 2: Classify The Reuse Path Classify the vendor before writing metadata. Use the [PAP-2432](/PAP/issues/PAP-2432) matrix terms so rollout planning, security review, and QA can compare providers consistently. @@ -66,14 +1059,25 @@ Classify the vendor before writing metadata. Use the [PAP-2432](/PAP/issues/PAP- Record the classification in the proposal along with the transport and the reason a lighter path is or is not enough. -## Step 3: Pick Auth And Credential Ownership +### Step 3: Pick Auth And Credential Ownership -Choose one auth mode: +Choose one method auth mode: -- OAuth: user or workspace authorization through Paperclip-owned OAuth app registration. Use for vendors with delegated scopes and revocation APIs. -- API key: operator-supplied token or key. Use only when scopes can be constrained and the key is stored as a `company_secrets` ref. -- App-installation: bot/app token, GitHub App installation, Slack bot token, or similar installation credential. -- None: public/read-only systems or first-party fixtures that do not require vendor credentials. +- OAuth: delegated user or workspace authorization. The OAuth client may come + from DCR/CIMD, a customer-created client, a deployment-preconfigured client, + or a reviewed Paperclip ID connector profile. Do not assume Paperclip owns a + shared client registration. +- API key: operator-supplied token or key. Use only when the provider supports + a suitably restricted key and the value is stored as a `company_secrets` + ref. +- None: public/read-only systems and provider-generated URLs. A generated URL + may still contain a secret and must be split and vaulted. + +Installation credentials such as bot tokens and GitHub App installation +tokens are provider-specific credential shapes. Model them through the generic +secret and grant architecture; do not invent an `AppDefinition.auth` value +named `app-installation` because the current schema accepts only `oauth`, +`api_key`, and `none`. Credentials normally live in `company_secrets` with redacted metadata and versioned material. The catalog entry records the secret binding shape, not the @@ -112,30 +1116,43 @@ The resulting connection has an external connector ref and zero Paperclip credential secret refs; its grants likewise use external metadata or secret refs, never both. -## Step 4: Author The AppDefinition +### Step 4: Author The AppDefinition Author an `AppDefinition` as the canonical data record for the app and every supported connection method. It must explain what the operator gets without exposing protocol details in prosumer surfaces. Developer docs can mention transport, MCP, shim, and gateway terms; the Apps gallery copy should use plain app/action language. Capture: -- `key`: stable lowercase app key, e.g. `linear`. -- `name`, `logoUrl`, `tagline`, `description`: user-facing metadata. -- `methods`: explicit combinations of `transport` (`mcp_remote`, `rest_api`, `local_stdio`), `authKind` (`oauth`, `api_key`, `none`), and `ownership` (`platform_shared`, `platform_provisioned`, `customer`, `dcr`). -- Stable connection UID namespace used to form `{namespace}/{slug}` addresses. -- `credentialFields`: labels, vendor-call placement, header key, prefix, help URL, and required state. User-facing labels should be sanitized by the Apps UI copy layer. The saved value is always a `company_secrets` ref, not an env entry. +- `schemaVersion: 1` and a stable lowercase hyphenated `slug`, such as + `linear`. +- `name`, `description`, `categories`, `branding`, `urlPatterns`, and + `docsUrl`: user-facing and import metadata. Branding points to vetted local + assets under `/brands/apps/`. +- `methods`: explicit combinations of `transport` (`mcp_remote`, `rest_api`, + `local_stdio`), `auth` (`oauth`, `api_key`, `none`), and + `ownershipModes` (`platform_shared`, `platform_provisioned`, `customer`, + `dcr`). +- `credentialFields` plus `keyPlacement`: labels, write-only value fields, + vendor-call placement, header/key name, and prefix. The saved value becomes a + `company_secrets` ref, never a plain env/config value. - `tenantFields` and `extensionFields`: keep unavoidable identity and resource boundaries in the default flow. Mark optional scope reduction, feature/tool filters, response modes, and transport tuning with `advanced: true`, and give advanced fields working defaults that do not require the operator to expand the disclosure. -- `oauth`: provider key, scopes, authorization URL, token URL, metadata URL if applicable. -- `urlPatterns`: URLs that can identify this app during paste/import flows. -- `recommendedDefaults`: access and risk defaults, especially ask-first risk levels. -- `availability`: whether the connector is generally available, gated by deployment config, or needs vendor registration. +- `defaults`: exact server/template URL, optional discovery or OAuth endpoint + hints, a contained `scopesHint`, and only reviewed managed tool arguments. +- `grantKinds`, `oauthStrategy`, `connectorProfile`, `credentialSources`, + `capabilityProfile`, `variants`, `configRequirements`, and + `requiredResourceFilters` only when their documented semantics apply. +- `setupPrerequisite`, `warnings`, `guidanceMd`, and `consoleLinks`: everything + the operator must know before credentials or consent. +- `riskTier`: the method-level S1-S4 tier that drives central access defaults. +- `availability`: whether the connection is usable on this instance and the + precise reason when it is not. Keep `AppDefinition` metadata deterministic and company-scoped at install time. Global catalog data names capabilities; company connection and grant rows hold the configured instance, subject/provider tenant, secret refs, resource filters, status, health, and audit history. -## Step 5: Model Resource Filters +### Step 5: Model Resource Filters Every connector proposal needs resource filters before write actions are enabled. Filters are part of the connection configuration and must be enforced by the gateway or wrapper, not only by UI affordances. @@ -149,7 +1166,7 @@ Common filter dimensions: The connection health and catalog discovery steps should fail or warn when required filters are absent for S3/S4 providers. -## Step 6: Define The Action Catalog +### Step 6: Define The Action Catalog List each initial action before implementation. Do not rely on vendor tool names alone; Paperclip needs normalized metadata for review, policy, and audit. @@ -170,12 +1187,17 @@ Risk classes: | Risk | Examples | Default | | --- | --- | --- | | `read` | Search, list, fetch metadata/content inside allowed resources. | Active when profile includes the app or read risk level. | -| `write` | Create issue, add comment, update status, append block, trigger redeploy. | Ask-first unless the default profile or a reviewed policy narrows it further. New/changed write tools start quarantined when discovered after initial review. | -| `destructive` | Delete, refund, cancel production deployment, send external message, broad tenant mutation. | Quarantined. Requires explicit operator review and usually a `require_approval` policy even after review. | +| `write` | Create issue, add comment, update status, append block, trigger redeploy. | Allowed for S1-S3 under the current product default; ask-first for S4. | +| `destructive` | Delete, refund, cancel production deployment, send external message, broad tenant mutation. | Allowed for S1-S3 and ask-first for S4 under the current default. A provider with meaningful destructive capability should normally be S4 or receive a reviewed explicit policy. | -Changed-action quarantine is mandatory: if catalog refresh finds a new or schema-changed write/destructive action, the entry stays hidden from agents until an operator reviews and re-enables it. Do not mark a changed action active just because a previous action with a similar name was active. +Changed-action quarantine is available when a connection sets +`quarantineNewEntries: true`. Use it for providers whose catalog can change +without a Paperclip release. This is runtime setup behavior, not currently an +`AppDefinition` field, so adding it to a new curated class requires a shared +implementation and tests. Do not claim quarantine in provider copy unless the +connection actually enables it. -## Step 7: Select The Wizard Path +### Step 7: Select The Wizard Path The wizard path comes from auth mode and transport: @@ -183,19 +1205,24 @@ The wizard path comes from auth mode and transport: | --- | --- | --- | | OAuth | Gallery card -> Connect -> vendor consent -> callback -> configure filters -> health/catalog -> access defaults. | OAuth token material in `company_secrets`; connection metadata redacted. | | API key | Gallery card -> paste key -> configure filters -> health/catalog -> access defaults. | Key material in `company_secrets`; no raw key returned after save. | -| App-installation | Gallery card -> install app/bot -> callback or paste installation identifier -> configure filters -> health/catalog -> access defaults. | Installation credential in `company_secrets`; installation account metadata redacted. | | None | Gallery card -> configure allowed resources -> health/catalog -> access defaults. | No vendor secret; connection row still carries config and audit scope. | +Provider-generated URLs also use `auth: "none"`, but the complete URL is +vaulted when it contains credential material. Installation-style providers use +the nearest supported auth path plus provider-specific setup guidance; they do +not add a fourth manifest auth mode. + The operator should see Apps, Connections, and Review language. Keep protocol language behind Developer/Advanced copy. -Default to the broadest vendor permissions and scopes the reviewed connection -can support. Operators should not have to predict every future tool during -setup. Keep the default view to the minimum inputs needed for a working -connection, fold optional expert controls under one collapsed **Advanced** -disclosure, and enforce safe execution afterward through Paperclip's resource -boundaries, catalog review, ask-first policies, quarantine, and audit. +Request only the documented scope set the reviewed connection needs; never +adopt every scope returned by discovery. Operators should not have to predict +every future tool during setup. Keep the default view to the minimum inputs +needed for a working connection, fold optional expert controls under one +collapsed **Advanced** disclosure, and enforce execution afterward through +Paperclip's resource boundaries, risk classification, tier defaults, optional +quarantine, and audit. -## Step 8: Apply Governance Defaults +### Step 8: Apply Governance Defaults Governance is automatic because every catalog entry becomes a normal tool-access object: @@ -207,21 +1234,29 @@ Governance is automatic because every catalog entry becomes a normal tool-access Recommended defaults for a new catalog entry: -- Create a read-friendly default profile only when read actions are low or medium risk and resource filters are present. -- Set `recommendedDefaults.askFirstRiskLevels` to `["write", "destructive"]` unless the connector is read-only. -- Add an explicit block or quarantine for destructive actions until SecurityEngineer review. -- Add rate-limit policy for search/fetch APIs, vendor quota-sensitive APIs, and paid APIs. -- Require approval for any external send, deploy, refund, delete, tenant-wide mutation, or action that can expose private customer data outside Paperclip. +- Use the central `recommendedDefaultsForApp` policy. Do not invent a provider + default in UI code. +- S1-S3 actions default Allowed. S4 writes and destructive actions default Ask + first. +- Classify a method S4 when its normal catalog includes payments, external + sends, refunds, production deployment, deletion, tenant-wide administration, + or comparable high-impact mutations. +- Add an explicit block only for a tool Paperclip must never expose, and prove + it with a provider-specific negative test. +- Enable changed-tool quarantine for catalogs that can drift independently, + and add a rate limit for quota-sensitive or paid APIs. -## Step 9: Align With Production Validation +### Step 9: Align With Production Validation [PAP-12373](/PAP/issues/PAP-12373) owns real-vendor gallery smoke evidence and connector validation. Do not duplicate that issue's screenshot/evidence matrix in this playbook. A connector proposal should instead state exactly how it will be validated there: - Connect succeeds against the real vendor using production-like OAuth/app/key setup. -- Catalog discovery produces the expected actions and quarantines new/changed risky actions. +- Catalog discovery produces the expected actions and the declared changed-tool + behavior. - An allowed read call succeeds through the gateway. -- A write call opens ask-first review and succeeds only after approval. -- A blocked/quarantined action cannot be listed or invoked by an agent. +- A write call matches the method tier: Allowed for S1-S3, Ask first for S4. +- A blocked/quarantined action, when declared, cannot be listed or invoked by + an agent. - Revocation removes tools and blocks execution immediately. - Activity/audit rows prove actor, run/issue context, resource id, decision, reason code, and outcome. @@ -421,15 +1456,29 @@ Copy this section into a connector proposal or implementation issue. ## Manifest -- key: +- schemaVersion: 1 +- slug: - name: -- tagline: -- authKind: -- transportTemplate: +- description: +- categories: +- branding and provenance: +- docsUrl: +- method key and label: +- transport: mcp_remote / local_stdio / rest_api +- auth: oauth / api_key / none +- ownershipModes: dcr / customer / platform_shared / platform_provisioned +- grantKinds: organization / user +- oauthStrategy and connectorProfile, if reviewed: +- capabilityProfile and variants, if needed: +- defaults: endpoint/template/discovery/OAuth hints/scopes/tool defaults +- tenantFields and extensionFields: - credentialFields: -- oauth: +- keyPlacement or credentialSources: +- configRequirements: +- guidanceMd, warnings, and consoleLinks: +- riskTier and requiredResourceFilters: - urlPatterns: -- recommendedDefaults: +- setupPrerequisite and redirectConstraints: - availability: ## Actions @@ -459,7 +1508,7 @@ Copy this section into a connector proposal or implementation issue. - Connect evidence: - Catalog evidence: - Allowed read: -- Ask-first write: +- Governed write (Allowed for S1-S3, Ask first for S4): - Denied/quarantined case: - Revoke: - Audit: @@ -483,7 +1532,9 @@ This dry run applies the template to Linear, one of the [PAP-2432](/PAP/issues/P - Transport: `mcp_remote` - Endpoint: `https://mcp.linear.app/mcp` - Auth mode: OAuth -- OAuth scopes: `read` and `write` initially, with writes governed by profiles and ask-first policies. +- OAuth scopes: the reviewed `read` and `write` set. Linear is S2, so the + current tier default allows reviewed writes; operators may still narrow them + with profiles and policies. - Credential owner: company connection backed by user/workspace consent. - Secret storage: OAuth token material stored as `company_secrets` refs; no token in agent env, project env, comments, logs, or screenshots. - Revocation behavior: disabling or revoking the connection immediately removes Linear tools from agent sessions and denies brokered execution on the next gateway check. @@ -499,26 +1550,31 @@ This dry run applies the template to Linear, one of the [PAP-2432](/PAP/issues/P ```json { - "key": "linear", + "schemaVersion": 1, + "slug": "linear", "name": "Linear", - "tagline": "Create, update and read tickets.", - "authKind": "oauth", - "transportTemplate": { - "transport": "mcp_remote", - "url": "https://mcp.linear.app/mcp" - }, - "credentialFields": [], - "oauth": { - "provider": "linear", - "scopes": ["read", "write"], - "authorizationUrl": "https://linear.app/oauth/authorize", - "tokenUrl": "https://api.linear.app/oauth/token" - }, + "description": "Create, update, and read Linear issues.", + "categories": ["productivity"], + "branding": { "logoUrl": "/brands/apps/linear.svg" }, "urlPatterns": ["https://mcp.linear.app/*"], - "recommendedDefaults": { - "access": "all_agents", - "askFirstRiskLevels": ["write", "destructive"] - } + "methods": [ + { + "key": "mcp-oauth", + "transport": "mcp_remote", + "auth": "oauth", + "ownershipModes": ["customer"], + "whenToUse": "Use the provider-hosted connection for the quickest setup.", + "defaults": { + "serverUrl": "https://mcp.linear.app/mcp", + "authorizationEndpoint": "https://linear.app/oauth/authorize", + "tokenEndpoint": "https://api.linear.app/oauth/token", + "scopesHint": ["read", "write"] + }, + "guidanceMd": "Register a Linear OAuth app and add Paperclip's redirect URI before connecting.", + "riskTier": "S2", + "requiredResourceFilters": ["workspace", "team", "project"] + } + ] } ``` @@ -528,27 +1584,32 @@ This dry run applies the template to Linear, one of the [PAP-2432](/PAP/issues/P | --- | --- | --- | --- | --- | --- | --- | | `linear.search_issues` | read | active after catalog review | workspace, team, project, label, status | allow when profile includes Linear reads | query summary, team/project ids, result count | Granted agent cannot search a disallowed team. | | `linear.get_issue` | read | active after catalog review | workspace, team, issue id | allow when profile includes Linear reads | issue id, team/project ids | Ungranted agent cannot list or invoke the tool. | -| `linear.create_issue` | write | active only after review; changed versions quarantined | workspace, team, project, label | ask-first by default | team/project ids, title hash, created issue id | Missing project/team filter denies. | -| `linear.comment_issue` | write | active only after review; changed versions quarantined | workspace, team, issue id | ask-first by default | issue id, comment body redaction summary | Agent cannot comment on a disallowed issue. | -| `linear.update_issue_status` | write | active only after review; changed versions quarantined | workspace, team, issue id, allowed statuses | ask-first unless a trust rule covers exact shape | issue id, old/new status if returned | Revoked connection blocks retry. | +| `linear.create_issue` | write | active | workspace, team, project, label | allow under S2 default | team/project ids, title hash, created issue id | Missing project/team filter denies. | +| `linear.comment_issue` | write | active | workspace, team, issue id | allow under S2 default | issue id, comment body redaction summary | Agent cannot comment on a disallowed issue. | +| `linear.update_issue_status` | write | active | workspace, team, issue id, allowed statuses | allow under S2 default | issue id, old/new status if returned | Revoked connection blocks retry. | -No destructive Linear action should ship in the first pass. If a future delete/archive/bulk-update action appears during catalog refresh, it starts quarantined and needs explicit SecurityEngineer review before any policy can expose it. +No destructive Linear action should ship in the first pass. If one becomes part +of the normal catalog, re-evaluate the method tier and changed-tool quarantine +before accepting it as an S2 Allowed action. ### Wizard Path 1. Operator opens Apps and selects Linear. 2. Operator clicks Connect and completes Linear OAuth. 3. Paperclip stores OAuth material in `company_secrets` and shows redacted workspace/account metadata. -4. Operator selects workspace/team/project filters and confirms default ask-first writes. +4. Operator selects workspace/team/project filters and reviews the S2 Allowed + action defaults. 5. Paperclip runs health check and catalog refresh. 6. Operator binds the Linear read profile to a company, project, agent, routine, or issue scope. -7. Write actions stay ask-first until the operator approves calls or creates narrow trust rules. +7. Write actions are Allowed by the current S2 default unless the operator + narrows them with profiles or policies. ### Governance Defaults -- Default profile: include Linear read actions for the selected scope; exclude write actions unless the operator opts in. -- Policy defaults: require approval for create, comment, and status updates; block any unreviewed destructive action. -- Quarantine: new or schema-changed write actions receive `quarantineReason: "pending_review"` and are hidden from agent tool lists. +- Default profile: include the reviewed Linear actions for the selected scope. +- Policy defaults: S2 actions are Allowed. Operators may narrow specific writes. +- Quarantine: enable changed-tool quarantine before relying on it; the manifest + declaration alone does not activate it. - Rate limits: apply a per-connection query/write budget to protect vendor quota and avoid noisy issue edits. - Audit: log connect, config/filter changes, grant changes, action requests, allowed/denied calls, revoke, and catalog quarantine events. @@ -556,10 +1617,13 @@ No destructive Linear action should ship in the first pass. If a future delete/a Linear's real-vendor evidence belongs in [PAP-12373](/PAP/issues/PAP-12373). The smoke pass should prove: -- OAuth connect succeeds with Paperclip-owned Linear app registration once [PAP-12372](/PAP/issues/PAP-12372) provides credentials. +- OAuth connect succeeds with a customer-created Linear OAuth app (or an + explicitly reviewed external credential source) and the instance callback + URI. - Catalog discovery returns the expected Linear issue actions. - A read call against an allowed team succeeds. -- `linear.create_issue` opens ask-first review and only executes after approval. +- `linear.create_issue` executes under the S2 Allowed default and remains bound + by resource filters and current policy. - A call against a disallowed team/project is denied. - Revocation removes Linear tools and blocks execution. - Audit rows include company, connection, run/issue, agent/user actor, tool, decision, reason code, and outcome. @@ -724,8 +1788,9 @@ plain-HTTP non-loopback origins. - How to verify: visit `/PAP/apps/connect?source=notion`, complete the Notion consent flow, and land on the wizard's actions step listing `notion-*` tools. Then confirm an agent run sees Notion tools through the runtime MCP - gateway and that a write call (e.g. `notion-create-pages`) opens an - ask-first action request. + gateway and that a write call such as `notion-create-pages` follows the + current S3 Allowed policy. Add a narrower Ask-first rule separately when the + company wants approval for that action. ### Resource Filters @@ -769,18 +1834,20 @@ The shipped `packages/shared/src/app-definitions/notion.json` (regenerate via Notion's hosted server exposes ~20 `notion-*` tools. Representative risk classes below; the full catalog review with per-tool defaults is PAP-16652 -(P4), and changed-action quarantine applies as usual. +(P4). Changed-action quarantine applies only when the connection explicitly +enables `quarantineNewEntries`. | Tool | Risk | Default status | Filters | Approval default | Audit fields | Negative case | | --- | --- | --- | --- | --- | --- | --- | | `notion-search` | read | active after catalog review; plan-gated by Notion (needs Notion AI) — may list but fail at call time | workspace | allow when profile includes Notion reads | query summary, result count | Ungranted agent cannot invoke. | | `notion-fetch` | read | active after catalog review | workspace, page, database | allow when profile includes Notion reads | page/database id | Fetch outside shared pages fails Notion-side and is audited. | -| `notion-create-pages` | write | active only after review; changed versions quarantined | workspace, page, database | ask-first by default | parent id, title hash, created page id | Missing workspace/page filter denies. | -| `notion-update-page` | write | active only after review; changed versions quarantined | workspace, page | ask-first by default | page id, redaction summary | Revoked connection blocks retry. | +| `notion-create-pages` | write | active after catalog review | workspace, page, database | allow under S3 default | parent id, title hash, created page id | Missing workspace/page filter denies. | +| `notion-update-page` | write | active after catalog review | workspace, page | allow under S3 default | page id, redaction summary | Revoked connection blocks retry. | | `notion-query-data-sources` | read | active after catalog review | workspace, database | allow when profile includes Notion reads | data-source id, result count | Granted agent cannot query a disallowed database. | -No destructive Notion action ships in the first pass; any future -delete/archive/bulk action starts quarantined pending SecurityEngineer review. +No destructive Notion action ships in this worked example. A future +delete/archive/bulk action needs explicit risk review; normally classify the +method S4 or add a reviewed narrow policy and tests before enabling it. ### Wizard Path @@ -791,22 +1858,23 @@ delete/archive/bulk action starts quarantined pending SecurityEngineer review. 3. Notion redirects to the instance's own `GET /api/tools/oauth/callback`; Paperclip exchanges the code, stores token material in `company_secrets`, and returns the operator to the wizard (`?oauth=connected`). -4. Operator confirms resource filters and default ask-first writes. +4. Operator confirms resource filters and reviews the S3 Allowed action + defaults. 5. Paperclip runs health check and catalog refresh; `notion-*` tools appear on the actions step. -6. Write actions stay ask-first until the operator approves calls or creates - narrow trust rules. +6. Reviewed write actions are Allowed by the current S3 default unless the + operator narrows them with profiles or an Ask-first policy. Error state: on a plain-HTTP non-loopback instance, step 1 fails fast with the TLS guidance error above — the operator never reaches Notion. ### Governance Defaults -- Default profile: Notion read actions for the selected scope; writes opt-in. -- Policy defaults: ask-first for `notion-create-pages`, `notion-update-page`, - and comment writes; block any unreviewed destructive action. -- Quarantine: new or schema-changed write actions receive - `quarantineReason: "pending_review"` and are hidden from agent tool lists. +- Default profile: reviewed Notion actions for the selected resource scope. +- Policy defaults: S3 actions are Allowed. Operators may narrow page creation, + updates, or comments with profiles or Ask-first rules. +- Quarantine: enable `quarantineNewEntries` before relying on changed-tool + quarantine; the manifest declaration alone does not activate it. - Rate limits: per-connection search/fetch budget to protect vendor quota. - Audit: log connect, DCR registration, config/filter changes, grant changes, action requests, allowed/denied calls, token refresh failures, revoke, and @@ -819,11 +1887,11 @@ End-to-end evidence belongs to PAP-16654 (P6) and the PAP-12373 matrix: - Zero-setup OAuth connect succeeds on `https://paperclip-dev.tail29c1aa.ts.net/PAP/apps/connect?source=notion` with no pre-provisioned OAuth env vars (proves DCR). -- Catalog discovery lists `notion-*` tools; new/changed risky actions are - quarantined. +- Catalog discovery lists the expected `notion-*` tools and applies the + connection's declared changed-tool behavior. - An agent run sees Notion tools through the managed runtime MCP gateway. -- `notion-create-pages` opens ask-first review and executes only after - approval. +- `notion-create-pages` executes under the S3 Allowed default while respecting + resource filters and any narrower company policy. - Revocation removes Notion tools and blocks execution. - Audit rows prove actor, run/issue context, connection, tool, decision, reason code, and outcome. diff --git a/doc/connections/GENERIC-REMOTE-MCP.md b/doc/connections/GENERIC-REMOTE-MCP.md index 98f8e8631d..bbec052655 100644 --- a/doc/connections/GENERIC-REMOTE-MCP.md +++ b/doc/connections/GENERIC-REMOTE-MCP.md @@ -5,7 +5,7 @@ Paperclip code change. A curated `AppDefinition` is a **convenience layer** — branding, tailored fields, scoped defaults, support copy — not a prerequisite. This is the documented baseline for connecting anything. Read -[Connector playbook](./CONNECTOR-PLAYBOOK.md) when you want to add the branded +[Connection authoring runbook](./CONNECTOR-PLAYBOOK.md) when you want to add the branded convenience layer on top for a vendor Paperclip should promote. Accepted in the [generic remote MCP plan](/PAP/issues/PAP-17078#document-plan), diff --git a/doc/connections/README.md b/doc/connections/README.md index 6e99399d3e..9e71188b58 100644 --- a/doc/connections/README.md +++ b/doc/connections/README.md @@ -2,6 +2,12 @@ Audience: internal engineers and product contributors working on integrations. +Start here when adding a provider: +[Connection authoring runbook](./CONNECTOR-PLAYBOOK.md). It is the canonical +agent tutorial from provider research and protocol classification through +manifest generation, branding, secrets, deterministic tests, real-account +proof, and PR submission. + Provider notes: [Google Workspace](./GOOGLE-WORKSPACE.md), [Gmail](./GMAIL.md), [PostHog](./POSTHOG.md). Optional credential custody: [Vercel Connect](./VERCEL-CONNECT.md). @@ -88,11 +94,11 @@ identity-service documentation or re-deriving it. | Plane | Question | Lives where | Token profile | | --- | --- | --- | --- | | **P1. Sign-in methods** | *Who are you?* | `paperclip-id` (id.paperclip.ing → Account) | Minimal-scope provider tokens (`openid email profile`), used once to authenticate, encrypted at rest, never exported | -| **P2. Connections (Apps)** | *What may your agents touch?* | Paperclip App instances (`tool_connections`), acquired via the **connect broker** for hosted + self-hosted | Rich-scope, long-lived resource tokens in the **instance's** encrypted vault; per-agent grants; ask-first on writes | +| **P2. Connections (Apps)** | *What may your agents touch?* | Paperclip App instances (`tool_connections`), acquired via the **connect broker** for hosted + self-hosted | Rich-scope, long-lived resource tokens in the **instance's** encrypted vault; per-agent grants; risk-tier policy defaults | | **P3. Login with Paperclip** | *Who may authenticate against us?* | `paperclip-id` OIDC provider + DB-backed client registry | Our ES256 ID/access tokens issued *by* us to registered RPs (instances, the broker, future third parties) | Everything in `doc/connections/` — the [First-30 matrix](./FIRST-30-MATRIX.md), -the [connector playbook](./CONNECTOR-PLAYBOOK.md), and the connect-broker work — +the [connection authoring runbook](./CONNECTOR-PLAYBOOK.md), and the connect-broker work — lives on **plane P2**. It never acquires, stores, or brokers a P1 sign-in token. ### The standing rule (D7) @@ -163,9 +169,11 @@ not own durable tokens. - [Connecting any remote MCP server](./GENERIC-REMOTE-MCP.md) is the baseline: how an operator connects a standards-compliant remote MCP endpoint with no Paperclip code change, and how sign-in resolves a client. -- [Connector playbook](./CONNECTOR-PLAYBOOK.md) is the repeatable template for - adding a vendor as a catalog entry on Apps v2 — the optional branded - convenience layer over the baseline above. +- [Connection authoring runbook](./CONNECTOR-PLAYBOOK.md) is the one + end-to-end, agent-executable guide for adding a vendor as a catalog entry on + Apps v2: research, connection-type selection, OAuth/API-key/generated-URL + setup, encrypted credential handling, branding, implementation, browser and + live-provider testing, verification, and PR submission. - [Vercel Connect operator guide](./VERCEL-CONNECT.md) documents the optional external credential source, deployment flags, runtime resolution, recovery, and smoke requirements. diff --git a/packages/shared/src/app-definitions.test.ts b/packages/shared/src/app-definitions.test.ts index 6fba90b11e..ec9813ee54 100644 --- a/packages/shared/src/app-definitions.test.ts +++ b/packages/shared/src/app-definitions.test.ts @@ -73,9 +73,14 @@ describe("AppDefinition catalog",()=>{ ]); expect(method("context7")).toMatchObject({auth:"none",defaults:{serverUrl:"https://mcp.context7.com/mcp"}}); expect(APP_DEFINITIONS.find((app)=>app.slug==="planetscale")?.methods.map((candidate)=>candidate.key)).toEqual(["mcp-oauth","mcp-insights-only"]); - expect(APP_DEFINITIONS.find((app)=>app.slug==="postman")?.methods.map((candidate)=>candidate.key)).toEqual([ + const postman=APP_DEFINITIONS.find((app)=>app.slug==="postman"); + expect(postman?.methods.map((candidate)=>candidate.key)).toEqual([ "mcp-oauth-minimal","mcp-oauth-code","mcp-oauth-full","mcp-eu-key-minimal","mcp-eu-key-code","mcp-eu-key-full", ]); + expect(getAvailableConnectionMethod(postman!)?.key).toBe("mcp-oauth-full"); + expect(postman?.methods.filter((candidate)=>candidate.auth==="api_key").every((candidate)=> + candidate.keyPlacement?.name==="Authorization"&&candidate.keyPlacement.prefix==="Bearer " + )).toBe(true); expect(method("supabase")?.tenantFields?.find((field)=>field.key==="readOnly")?.defaultValue).toBe(false); expect(method("asana")?.ownershipModes).toEqual(["customer"]); expect(method("zapier")).toMatchObject({key:"generated-url",auth:"none",defaults:{}}); @@ -188,9 +193,9 @@ describe("AppDefinition catalog",()=>{ ]], ]); }); - it("configures Shopify's official tenant-scoped Storefront MCP without OAuth",()=>{const method=APP_DEFINITIONS.find((app)=>app.slug==="shopify")?.methods[0];expect(method).toMatchObject({key:"storefront-mcp",auth:"none",defaults:{serverUrlTemplate:"https://{storeDomain}/api/mcp"},tenantFields:[expect.objectContaining({key:"storeDomain",required:true})]});expect(resolveConnectionMethodServerUrl(method!,{storeDomain:"paperclip-demo.myshopify.com"})).toBe("https://paperclip-demo.myshopify.com/api/mcp");expect(resolveConnectionMethodServerUrl(method!,{})).toBeNull()}); + it("configures Shopify's current UCP and compatibility MCP methods without OAuth",()=>{const shopify=APP_DEFINITIONS.find((app)=>app.slug==="shopify");expect(shopify?.methods.map((method)=>method.key)).toEqual(["ucp-commerce","storefront-mcp"]);const ucp=shopify?.methods[0];const compatibility=shopify?.methods[1];expect(ucp).toMatchObject({auth:"none",defaults:{serverUrlTemplate:"https://{storeDomain}/api/ucp/mcp",toolArgumentDefaults:{meta:{"ucp-agent":{profile:"https://shopify.dev/ucp/agent-profiles/examples/2026-04-08/valid-with-capabilities.json"}}}},tenantFields:[expect.objectContaining({key:"storeDomain",required:true})]});expect(compatibility).toMatchObject({auth:"none",defaults:{serverUrlTemplate:"https://{storeDomain}/api/mcp"}});expect(resolveConnectionMethodServerUrl(ucp!,{storeDomain:"paperclip-demo.myshopify.com"})).toBe("https://paperclip-demo.myshopify.com/api/ucp/mcp");expect(resolveConnectionMethodServerUrl(compatibility!,{storeDomain:"paperclip-demo.myshopify.com"})).toBe("https://paperclip-demo.myshopify.com/api/mcp");expect(resolveConnectionMethodServerUrl(ucp!,{})).toBeNull();expect(shopify?.setupPrerequisite).toMatchObject({title:"Launch the storefront before connecting",actionUrl:"https://admin.shopify.com/"});expect(shopify?.setupPrerequisite?.steps?.join(" ")).toContain("Storefront visibility to Public")}); it("offers PostHog OAuth and API-key methods with zero-config defaults and advanced narrowing",()=>{const posthog=APP_DEFINITIONS.find((app)=>app.slug==="posthog");expect(posthog?.methods.map((method)=>method.key)).toEqual(["mcp-oauth","mcp-api-key"]);for(const method of posthog?.methods??[]){const projectField=method.tenantFields?.find((field)=>field.key==="projectId");expect(method.riskTier).toBe("S3");expect(method.tenantFields?.find((field)=>field.key==="readOnly")).toMatchObject({defaultValue:false,advanced:true});expect(projectField).toMatchObject({advanced:true,transport:{location:"header",name:"x-posthog-project-id"}});expect(projectField?.required).not.toBe(true);expect(method.tenantFields?.filter((field)=>field.advanced).map((field)=>field.key)).toEqual(["projectId","readOnly","features","tools"]);expect(method.tenantFields?.find((field)=>field.key==="mode")).toMatchObject({hidden:true,defaultValue:"tools",transport:{location:"query",name:"mode"}});expect(method.configRequirements).toBeUndefined();expect(method.requiredResourceFilters).toBeUndefined();expect(method.guidanceMd).toContain("optional advanced controls")}}); - it("requires only reviewed provider or safety-boundary configuration on the default path",()=>{const required=APP_DEFINITIONS.flatMap((app)=>app.methods.flatMap((method)=>[...(method.tenantFields??[]),...(method.extensionFields??[])].filter((field)=>field.required&&field.advanced!==true&&!field.hidden).map((field)=>`${app.slug}:${method.key}:${field.key}`))).sort();expect(required).toEqual(["clickhouse:mcp-oauth:serviceId","shopify:storefront-mcp:storeDomain","supabase:mcp-api-key:projectRef","supabase:mcp-oauth:projectRef"])}); + it("requires only reviewed provider or safety-boundary configuration on the default path",()=>{const required=APP_DEFINITIONS.flatMap((app)=>app.methods.flatMap((method)=>[...(method.tenantFields??[]),...(method.extensionFields??[])].filter((field)=>field.required&&field.advanced!==true&&!field.hidden).map((field)=>`${app.slug}:${method.key}:${field.key}`))).sort();expect(required).toEqual(["clickhouse:mcp-oauth:serviceId","shopify:storefront-mcp:storeDomain","shopify:ucp-commerce:storeDomain","supabase:mcp-api-key:projectRef","supabase:mcp-oauth:projectRef"])}); it("limits Vercel Connect setup to the reviewed pilot methods",()=>{ const reviewed=APP_DEFINITIONS.flatMap((app)=>app.methods.flatMap((method)=>method.credentialSources?.vercelConnect?[{slug:app.slug,key:method.key,review:method.credentialSources.vercelConnect}]:[])); expect(reviewed.map(({slug,key})=>`${slug}:${key}`).sort()).toEqual(["linear:mcp-oauth","notion:mcp-oauth","posthog:mcp-api-key","posthog:mcp-oauth"]); diff --git a/packages/shared/src/app-definitions/postman.json b/packages/shared/src/app-definitions/postman.json index 07e3937646..073c042ffb 100644 --- a/packages/shared/src/app-definitions/postman.json +++ b/packages/shared/src/app-definitions/postman.json @@ -29,13 +29,18 @@ }, "guidanceMd": "Connect Postman in the browser. A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints.", "riskTier": "S3", - "label": "US · Minimal", + "label": "US · Browser sign-in", "consoleLinks": { "docs": "https://learning.postman.com/latest-v-12/docs/reference/postman-api/postman-mcp-server/postman-mcp-remote-server" }, "warnings": [ "A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints." - ] + ], + "capabilityProfile": { + "key": "minimal", + "label": "Minimal", + "description": "Essential workspace, collection, and environment tools with the smallest tool catalog." + } }, { "key": "mcp-oauth-code", @@ -50,13 +55,18 @@ }, "guidanceMd": "Connect Postman in the browser. A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints.", "riskTier": "S3", - "label": "US · Code", + "label": "US · Browser sign-in", "consoleLinks": { "docs": "https://learning.postman.com/latest-v-12/docs/reference/postman-api/postman-mcp-server/postman-mcp-remote-server" }, "warnings": [ "A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints." - ] + ], + "capabilityProfile": { + "key": "code", + "label": "Code", + "description": "Tools for generating client code from API definitions." + } }, { "key": "mcp-oauth-full", @@ -71,13 +81,18 @@ }, "guidanceMd": "Connect Postman in the browser. A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints.", "riskTier": "S3", - "label": "US · Full", + "label": "US · Browser sign-in", "consoleLinks": { "docs": "https://learning.postman.com/latest-v-12/docs/reference/postman-api/postman-mcp-server/postman-mcp-remote-server" }, "warnings": [ "A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints." - ] + ], + "capabilityProfile": { + "key": "write", + "label": "Full", + "description": "All Postman API tools, including write-capable collaboration and advanced features." + } }, { "key": "mcp-eu-key-minimal", @@ -92,7 +107,7 @@ }, "guidanceMd": "Use a customer-created Postman key. A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints.", "riskTier": "S3", - "label": "EU · Minimal", + "label": "EU · API key", "credentialFields": [ { "key": "authorization", @@ -105,8 +120,8 @@ ], "keyPlacement": { "location": "header", - "name": "X-API-Key", - "prefix": null + "name": "Authorization", + "prefix": "Bearer " }, "consoleLinks": { "keys": "https://learning.postman.com/latest-v-12/docs/reference/postman-api/postman-mcp-server/postman-mcp-remote-server", @@ -114,7 +129,12 @@ }, "warnings": [ "A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints." - ] + ], + "capabilityProfile": { + "key": "minimal", + "label": "Minimal", + "description": "Essential workspace, collection, and environment tools with the smallest tool catalog." + } }, { "key": "mcp-eu-key-code", @@ -129,7 +149,7 @@ }, "guidanceMd": "Use a customer-created Postman key. A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints.", "riskTier": "S3", - "label": "EU · Code", + "label": "EU · API key", "credentialFields": [ { "key": "authorization", @@ -142,8 +162,8 @@ ], "keyPlacement": { "location": "header", - "name": "X-API-Key", - "prefix": null + "name": "Authorization", + "prefix": "Bearer " }, "consoleLinks": { "keys": "https://learning.postman.com/latest-v-12/docs/reference/postman-api/postman-mcp-server/postman-mcp-remote-server", @@ -151,7 +171,12 @@ }, "warnings": [ "A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints." - ] + ], + "capabilityProfile": { + "key": "code", + "label": "Code", + "description": "Tools for generating client code from API definitions." + } }, { "key": "mcp-eu-key-full", @@ -166,7 +191,7 @@ }, "guidanceMd": "Use a customer-created Postman key. A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints.", "riskTier": "S3", - "label": "EU · Full", + "label": "EU · API key", "credentialFields": [ { "key": "authorization", @@ -179,8 +204,8 @@ ], "keyPlacement": { "location": "header", - "name": "X-API-Key", - "prefix": null + "name": "Authorization", + "prefix": "Bearer " }, "consoleLinks": { "keys": "https://learning.postman.com/latest-v-12/docs/reference/postman-api/postman-mcp-server/postman-mcp-remote-server", @@ -188,7 +213,12 @@ }, "warnings": [ "A Postman account; OAuth is available for US endpoints and API keys are required for EU endpoints." - ] + ], + "capabilityProfile": { + "key": "write", + "label": "Full", + "description": "All Postman API tools, including write-capable collaboration and advanced features." + } } ] } diff --git a/packages/shared/src/app-definitions/shopify.json b/packages/shared/src/app-definitions/shopify.json index 46b80a9d13..f6bff7db55 100644 --- a/packages/shared/src/app-definitions/shopify.json +++ b/packages/shared/src/app-definitions/shopify.json @@ -11,23 +11,31 @@ "logoUrl": "/brands/apps/shopify.svg" }, "urlPatterns": [ + "https://*.myshopify.com/api/ucp/mcp", "https://*.myshopify.com/api/mcp" ], "methods": [ { - "key": "storefront-mcp", + "key": "ucp-commerce", "transport": "mcp_remote", "auth": "none", "ownershipModes": [ "customer" ], - "whenToUse": "Use a store's public myshopify.com domain. No Shopify app or OAuth registration is required.", + "whenToUse": "Recommended for Shopify's current UCP catalog, cart, and checkout tools.", "defaults": { - "serverUrlTemplate": "https://{storeDomain}/api/mcp" + "serverUrlTemplate": "https://{storeDomain}/api/ucp/mcp", + "toolArgumentDefaults": { + "meta": { + "ucp-agent": { + "profile": "https://shopify.dev/ucp/agent-profiles/examples/2026-04-08/valid-with-capabilities.json" + } + } + } }, - "guidanceMd": "Connect Shopify's official Storefront MCP server for shopper-facing catalog, policy, and cart tools.", + "guidanceMd": "Connect Shopify's current UCP server for shopper-facing catalog and commerce tools. Paperclip supplies the required agent profile automatically.", "riskTier": "S3", - "label": "Connect a Shopify storefront", + "label": "Shopify UCP commerce", "tenantFields": [ { "key": "storeDomain", @@ -35,7 +43,47 @@ "type": "text", "required": true, "placeholder": "your-store.myshopify.com", - "helperMd": "Enter the full myshopify.com domain without https://.", + "helperMd": "Enter the permanent myshopify.com domain without https://. Custom storefront domains are not the MCP endpoint.", + "validation": { + "pattern": "^[A-Za-z0-9][A-Za-z0-9-]*\\.myshopify\\.com$", + "maxLength": 255 + } + } + ], + "consoleLinks": { + "docs": "https://shopify.dev/docs/agents/catalog/storefront-catalog" + }, + "warnings": [ + "This is Shopify's shopper-facing UCP server, not Admin API access. It does not manage merchant products or customers.", + "The storefront must be public. A private or password-protected storefront returns HTTP 401 even when the merchant is signed in to Shopify Admin.", + "Paperclip currently uses Shopify's documented hosted agent-profile fixture while Paperclip's production UCP profile is being established." + ], + "requiredResourceFilters": [ + "store" + ] + }, + { + "key": "storefront-mcp", + "transport": "mcp_remote", + "auth": "none", + "ownershipModes": [ + "customer" + ], + "whenToUse": "Use Shopify's compatibility server when agents need storefront policy and FAQ search.", + "defaults": { + "serverUrlTemplate": "https://{storeDomain}/api/mcp" + }, + "guidanceMd": "Connect Shopify's official Storefront MCP server for shopper-facing catalog, policy, and cart tools.", + "riskTier": "S3", + "label": "Storefront policies and compatibility tools", + "tenantFields": [ + { + "key": "storeDomain", + "label": "Store domain", + "type": "text", + "required": true, + "placeholder": "your-store.myshopify.com", + "helperMd": "Enter the permanent myshopify.com domain without https://. Custom storefront domains are not the MCP endpoint.", "validation": { "pattern": "^[A-Za-z0-9][A-Za-z0-9-]*\\.myshopify\\.com$", "maxLength": 255 @@ -47,11 +95,23 @@ }, "warnings": [ "This is Shopify's Storefront MCP, not Admin API access. It does not manage merchant products, orders, or customers.", - "The storefront must be publicly reachable. Password-protected or restricted trial stores can return HTTP 401." + "The storefront must be public. A private or password-protected storefront returns HTTP 401 even when the merchant is signed in to Shopify Admin." ], "requiredResourceFilters": [ "store" ] } - ] + ], + "docsUrl": "https://shopify.dev/docs/apps/build/storefront-mcp/servers/storefront", + "setupPrerequisite": { + "title": "Launch the storefront before connecting", + "description": "Shopify's Storefront MCP is a public, no-auth endpoint. Paperclip cannot use the merchant's Shopify Admin session to bypass a private storefront.", + "steps": [ + "Select a Shopify plan; Shopify keeps trial storefronts private until a plan is selected.", + "In Shopify Admin, open Online Store → Preferences and set Storefront visibility to Public (remove password protection).", + "Use the permanent .myshopify.com domain in Paperclip, even if the store also has a custom domain." + ], + "actionLabel": "Open Shopify Admin", + "actionUrl": "https://admin.shopify.com/" + } } diff --git a/packages/shared/src/types/app-definition.ts b/packages/shared/src/types/app-definition.ts index 831aabfb13..95f41aa04d 100644 --- a/packages/shared/src/types/app-definition.ts +++ b/packages/shared/src/types/app-definition.ts @@ -2,7 +2,7 @@ import type { ConnectionGrantKind, ToolConnectionOwnership, ToolConnectionTransp export type AppCategory = "ai"|"analytics"|"commerce"|"communication"|"content"|"data"|"developer"|"productivity"|"other"; export type OAuthRedirectConstraints = "https-or-loopback-http"; export interface FieldDef { key:string; label:string; type:"text"|"password"|"textarea"|"datetime"|"select"|"checkbox"; required?:boolean; advanced?:boolean; hidden?:boolean; placeholder?:string; helperMd?:string; secret?:boolean; prefix?:string; defaultValue?:string|boolean; validation?:{pattern?:string;maxLength?:number}; options?:Array<{value:string;label:string}>; transport?:{location:"query"|"header";name:string;format?:"string"|"csv"|"boolean";omitFalse?:boolean} } -export interface ConnectionMethodDef { key:string; label?:string; transport:ToolConnectionTransport; auth:"oauth"|"api_key"|"none"; oauthStrategy?:"paperclip_id_connector"; connectorProfile?:string; capabilityProfile?:{key:string;label:string;description?:string}; grantKinds?:ConnectionGrantKind[]; ownershipModes:ToolConnectionOwnership[]; whenToUse:string; defaults?:{serverUrl?:string;serverUrlTemplate?:string;discoveryUrl?:string|null;serviceHost?:string;templateKey?:string;authorizationEndpoint?:string;tokenEndpoint?:string;metadataUrl?:string;scopesHint?:string[];oauthAuthorizationParams?:{access_type?:"offline";prompt?:"consent"}}; tenantFields?:FieldDef[]; extensionFields?:FieldDef[]; configRequirements?:{atLeastOneOf?:string[]}; credentialFields?:FieldDef[]; keyPlacement?:{location:"header"|"query"|"body_json"|"env";name:string;prefix?:string|null}; credentialSources?:{vercelConnect?:{services:string[];principalModes:VercelConnectPrincipalMode[];scopes:string[];header:{name:string;prefix?:string|null}}}; guidanceMd:string; consoleLinks?:{register?:string;keys?:string;settings?:string;docs?:string}; warnings?:string[]; variants?:Array<{key:string;label:string;whenToUse:string;tenantFields?:FieldDef[]}>; riskTier:"S1"|"S2"|"S3"|"S4"; requiredResourceFilters?:string[] } +export interface ConnectionMethodDef { key:string; label?:string; transport:ToolConnectionTransport; auth:"oauth"|"api_key"|"none"; oauthStrategy?:"paperclip_id_connector"; connectorProfile?:string; capabilityProfile?:{key:string;label:string;description?:string}; grantKinds?:ConnectionGrantKind[]; ownershipModes:ToolConnectionOwnership[]; whenToUse:string; defaults?:{serverUrl?:string;serverUrlTemplate?:string;discoveryUrl?:string|null;serviceHost?:string;templateKey?:string;authorizationEndpoint?:string;tokenEndpoint?:string;metadataUrl?:string;scopesHint?:string[];oauthAuthorizationParams?:{access_type?:"offline";prompt?:"consent"};toolArgumentDefaults?:Record}; tenantFields?:FieldDef[]; extensionFields?:FieldDef[]; configRequirements?:{atLeastOneOf?:string[]}; credentialFields?:FieldDef[]; keyPlacement?:{location:"header"|"query"|"body_json"|"env";name:string;prefix?:string|null}; credentialSources?:{vercelConnect?:{services:string[];principalModes:VercelConnectPrincipalMode[];scopes:string[];header:{name:string;prefix?:string|null}}}; guidanceMd:string; consoleLinks?:{register?:string;keys?:string;settings?:string;docs?:string}; warnings?:string[]; variants?:Array<{key:string;label:string;whenToUse:string;tenantFields?:FieldDef[]}>; riskTier:"S1"|"S2"|"S3"|"S4"; requiredResourceFilters?:string[] } export interface AppDefinition { schemaVersion:1; slug:string; name:string; description:string; categories:AppCategory[]; featured?:boolean; branding:{logoUrl:string;darkLogoUrl?:string;backgroundColor?:string;accentColor?:string}; urlPatterns:string[]; docsUrl?:string; setupPrerequisite?:{title:string;description:string;steps?:string[];actionLabel:string;actionUrl:string}; redirectConstraints?:OAuthRedirectConstraints; methods:ConnectionMethodDef[]; suggestable?:boolean; availability?:{available:boolean;reason?:string;robotEmail?:string}; ownershipAvailability?:Partial> } export type SelfServeMcpAuthMode = diff --git a/packages/shared/src/validators/app-definition.ts b/packages/shared/src/validators/app-definition.ts index c7ae89a165..a2bffe2560 100644 --- a/packages/shared/src/validators/app-definition.ts +++ b/packages/shared/src/validators/app-definition.ts @@ -5,6 +5,6 @@ const appBrandAssetUrlSchema=z.string().refine((value)=>{ try{return new URL(value).protocol==="https:";}catch{return false;} },{message:"Brand assets must be HTTPS URLs or local /brands/apps SVG/PNG paths"}); const field=z.object({key:z.string().min(1),label:z.string().min(1),type:z.enum(["text","password","textarea","datetime","select","checkbox"]),required:z.boolean().optional(),advanced:z.boolean().optional(),hidden:z.boolean().optional(),placeholder:z.string().optional(),helperMd:z.string().optional(),secret:z.boolean().optional(),prefix:z.string().optional(),defaultValue:z.union([z.string(),z.boolean()]).optional(),validation:z.object({pattern:z.string().optional(),maxLength:z.number().int().positive().optional()}).optional(),options:z.array(z.object({value:z.string(),label:z.string()})).optional(),transport:z.object({location:z.enum(["query","header"]),name:z.string().min(1),format:z.enum(["string","csv","boolean"]).optional(),omitFalse:z.boolean().optional()}).optional()}).superRefine((v,c)=>{if(v.required&&v.type!=="checkbox"&&!v.placeholder)c.addIssue({code:"custom",message:"Required fields need placeholders",path:["placeholder"]});if(v.type==="select"&&(!v.options||v.options.length===0))c.addIssue({code:"custom",message:"Select fields need options",path:["options"]});if(v.hidden&&v.defaultValue===undefined)c.addIssue({code:"custom",message:"Hidden fields need defaults",path:["defaultValue"]})}); -export const connectionMethodDefSchema=z.object({key:z.string().min(1),label:z.string().min(1).optional(),transport:toolConnectionTransportSchema,auth:z.enum(["oauth","api_key","none"]),oauthStrategy:z.enum(["paperclip_id_connector"]).optional(),connectorProfile:z.string().regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/).optional(),capabilityProfile:z.object({key:z.string().min(1),label:z.string().min(1),description:z.string().min(1).optional()}).optional(),grantKinds:z.array(connectionGrantKindSchema).min(1).optional(),ownershipModes:z.array(toolConnectionOwnershipSchema).min(1),whenToUse:z.string().min(1),defaults:z.object({serverUrl:z.string().url().optional(),serverUrlTemplate:z.string().regex(/^https:\/\//).optional(),discoveryUrl:z.string().url().nullable().optional(),serviceHost:z.string().optional(),templateKey:z.string().optional(),authorizationEndpoint:z.string().url().optional(),tokenEndpoint:z.string().url().optional(),metadataUrl:z.string().url().optional(),scopesHint:z.array(z.string()).optional(),oauthAuthorizationParams:z.object({access_type:z.literal("offline").optional(),prompt:z.literal("consent").optional()}).optional()}).optional(),tenantFields:z.array(field).optional(),extensionFields:z.array(field).optional(),configRequirements:z.object({atLeastOneOf:z.array(z.string().min(1)).min(1).optional()}).optional(),credentialFields:z.array(field).optional(),keyPlacement:z.object({location:z.enum(["header","query","body_json","env"]),name:z.string().min(1),prefix:z.string().nullable().optional()}).optional(),credentialSources:z.object({vercelConnect:z.object({services:z.array(z.string().min(1)).min(1),principalModes:z.array(z.enum(["app","user"])).min(1),scopes:z.array(z.string().min(1)).min(1),header:z.object({name:z.string().min(1),prefix:z.string().nullable().optional()})}).optional()}).optional(),guidanceMd:z.string().min(1),consoleLinks:z.object({register:z.string().url().optional(),keys:z.string().url().optional(),settings:z.string().url().optional(),docs:z.string().url().optional()}).optional(),warnings:z.array(z.string()).optional(),variants:z.array(z.object({key:z.string(),label:z.string(),whenToUse:z.string(),tenantFields:z.array(field).optional()})).optional(),riskTier:z.enum(["S1","S2","S3","S4"]),requiredResourceFilters:z.array(z.string()).optional()}).superRefine((v,c)=>{if(v.auth==="api_key"&&!v.keyPlacement)c.addIssue({code:"custom",message:"API-key methods require keyPlacement",path:["keyPlacement"]});if(v.oauthStrategy&&v.auth!=="oauth")c.addIssue({code:"custom",message:"OAuth strategies require OAuth auth",path:["oauthStrategy"]});if(v.oauthStrategy==="paperclip_id_connector"&&!v.connectorProfile)c.addIssue({code:"custom",message:"Paperclip ID connector methods require connectorProfile",path:["connectorProfile"]});if(v.connectorProfile&&v.oauthStrategy!=="paperclip_id_connector")c.addIssue({code:"custom",message:"connectorProfile requires the Paperclip ID OAuth strategy",path:["connectorProfile"]});if(v.credentialSources?.vercelConnect&&(v.transport!=="mcp_remote"||v.auth==="none"))c.addIssue({code:"custom",message:"Vercel Connect requires an authenticated remote MCP method",path:["credentialSources","vercelConnect"]});const keys=new Set([...(v.tenantFields??[]),...(v.extensionFields??[])].map((entry)=>entry.key));for(const key of v.configRequirements?.atLeastOneOf??[])if(!keys.has(key))c.addIssue({code:"custom",message:"Config requirement references an unknown field",path:["configRequirements","atLeastOneOf"]});if(v.defaults?.serverUrl&&v.defaults.serverUrlTemplate)c.addIssue({code:"custom",message:"Use either serverUrl or serverUrlTemplate",path:["defaults"]});for(const placeholder of v.defaults?.serverUrlTemplate?.matchAll(/\{([a-zA-Z0-9_-]+)\}/g)??[])if(!keys.has(placeholder[1]))c.addIssue({code:"custom",message:"Server URL template references an unknown field",path:["defaults","serverUrlTemplate"]})}); +export const connectionMethodDefSchema=z.object({key:z.string().min(1),label:z.string().min(1).optional(),transport:toolConnectionTransportSchema,auth:z.enum(["oauth","api_key","none"]),oauthStrategy:z.enum(["paperclip_id_connector"]).optional(),connectorProfile:z.string().regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/).optional(),capabilityProfile:z.object({key:z.string().min(1),label:z.string().min(1),description:z.string().min(1).optional()}).optional(),grantKinds:z.array(connectionGrantKindSchema).min(1).optional(),ownershipModes:z.array(toolConnectionOwnershipSchema).min(1),whenToUse:z.string().min(1),defaults:z.object({serverUrl:z.string().url().optional(),serverUrlTemplate:z.string().regex(/^https:\/\//).optional(),discoveryUrl:z.string().url().nullable().optional(),serviceHost:z.string().optional(),templateKey:z.string().optional(),authorizationEndpoint:z.string().url().optional(),tokenEndpoint:z.string().url().optional(),metadataUrl:z.string().url().optional(),scopesHint:z.array(z.string()).optional(),oauthAuthorizationParams:z.object({access_type:z.literal("offline").optional(),prompt:z.literal("consent").optional()}).optional(),toolArgumentDefaults:z.record(z.string(),z.unknown()).optional()}).optional(),tenantFields:z.array(field).optional(),extensionFields:z.array(field).optional(),configRequirements:z.object({atLeastOneOf:z.array(z.string().min(1)).min(1).optional()}).optional(),credentialFields:z.array(field).optional(),keyPlacement:z.object({location:z.enum(["header","query","body_json","env"]),name:z.string().min(1),prefix:z.string().nullable().optional()}).optional(),credentialSources:z.object({vercelConnect:z.object({services:z.array(z.string().min(1)).min(1),principalModes:z.array(z.enum(["app","user"])).min(1),scopes:z.array(z.string().min(1)).min(1),header:z.object({name:z.string().min(1),prefix:z.string().nullable().optional()})}).optional()}).optional(),guidanceMd:z.string().min(1),consoleLinks:z.object({register:z.string().url().optional(),keys:z.string().url().optional(),settings:z.string().url().optional(),docs:z.string().url().optional()}).optional(),warnings:z.array(z.string()).optional(),variants:z.array(z.object({key:z.string(),label:z.string(),whenToUse:z.string(),tenantFields:z.array(field).optional()})).optional(),riskTier:z.enum(["S1","S2","S3","S4"]),requiredResourceFilters:z.array(z.string()).optional()}).superRefine((v,c)=>{if(v.auth==="api_key"&&!v.keyPlacement)c.addIssue({code:"custom",message:"API-key methods require keyPlacement",path:["keyPlacement"]});if(v.oauthStrategy&&v.auth!=="oauth")c.addIssue({code:"custom",message:"OAuth strategies require OAuth auth",path:["oauthStrategy"]});if(v.oauthStrategy==="paperclip_id_connector"&&!v.connectorProfile)c.addIssue({code:"custom",message:"Paperclip ID connector methods require connectorProfile",path:["connectorProfile"]});if(v.connectorProfile&&v.oauthStrategy!=="paperclip_id_connector")c.addIssue({code:"custom",message:"connectorProfile requires the Paperclip ID OAuth strategy",path:["connectorProfile"]});if(v.credentialSources?.vercelConnect&&(v.transport!=="mcp_remote"||v.auth==="none"))c.addIssue({code:"custom",message:"Vercel Connect requires an authenticated remote MCP method",path:["credentialSources","vercelConnect"]});const keys=new Set([...(v.tenantFields??[]),...(v.extensionFields??[])].map((entry)=>entry.key));for(const key of v.configRequirements?.atLeastOneOf??[])if(!keys.has(key))c.addIssue({code:"custom",message:"Config requirement references an unknown field",path:["configRequirements","atLeastOneOf"]});if(v.defaults?.serverUrl&&v.defaults.serverUrlTemplate)c.addIssue({code:"custom",message:"Use either serverUrl or serverUrlTemplate",path:["defaults"]});for(const placeholder of v.defaults?.serverUrlTemplate?.matchAll(/\{([a-zA-Z0-9_-]+)\}/g)??[])if(!keys.has(placeholder[1]))c.addIssue({code:"custom",message:"Server URL template references an unknown field",path:["defaults","serverUrlTemplate"]})}); export const appDefinitionSchema=z.object({schemaVersion:z.literal(1),slug:z.string().regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/),name:z.string().min(1),description:z.string().min(1),categories:z.array(z.enum(["ai","analytics","commerce","communication","content","data","developer","productivity","other"])).min(1),featured:z.boolean().optional(),branding:z.object({logoUrl:appBrandAssetUrlSchema,darkLogoUrl:appBrandAssetUrlSchema.optional(),backgroundColor:z.string().optional(),accentColor:z.string().optional()}),urlPatterns:z.array(z.string()),docsUrl:z.string().url().optional(),setupPrerequisite:z.object({title:z.string().min(1),description:z.string().min(1),steps:z.array(z.string().min(1)).min(1).optional(),actionLabel:z.string().min(1),actionUrl:z.string().url()} ).optional(),redirectConstraints:z.enum(["https-or-loopback-http"]).optional(),methods:z.array(connectionMethodDefSchema).min(1),suggestable:z.boolean().optional(),availability:z.object({available:z.boolean(),reason:z.string().optional(),robotEmail:z.string().optional()}).optional(),ownershipAvailability:z.object({platform_shared:z.boolean().optional(),platform_provisioned:z.boolean().optional(),customer:z.boolean().optional(),dcr:z.boolean().optional()}).optional()}); export const appDefinitionsSchema=z.array(appDefinitionSchema).superRefine((v,c)=>{const s=new Set();v.forEach((a,i)=>{if(s.has(a.slug))c.addIssue({code:"custom",message:"Duplicate slug",path:[i,"slug"]});s.add(a.slug)})}); diff --git a/scripts/ingest-app-definitions.mjs b/scripts/ingest-app-definitions.mjs index 948ea10627..2e74e2ea8d 100644 --- a/scripts/ingest-app-definitions.mjs +++ b/scripts/ingest-app-definitions.mjs @@ -29,7 +29,7 @@ const apps=[ ["linear","Linear","Create, update, and read Linear issues.","productivity","linear.app",["https://mcp.linear.app/*"],method("mcp-oauth","mcp_remote","oauth",{serverUrl:"https://mcp.linear.app/mcp",authorizationEndpoint:"https://linear.app/oauth/authorize",tokenEndpoint:"https://api.linear.app/oauth/token",scopesHint:["read","write"]},"S2","Register a Linear OAuth app and add Paperclip's redirect URI before connecting.",{ownershipModes:["customer"],requiredResourceFilters:["workspace","team","project"],...vercelConnect("linear","user",["read","write"])})], ["google-sheets","Google Sheets","Read and update selected spreadsheets.","data","sheets.google.com",["https://docs.google.com/spreadsheets/*","https://sheets.google.com/*"],method("local","local_stdio","none",{templateKey:"paperclip.google-sheets"},"S3","Share each spreadsheet with the Paperclip robot email, then paste the sheet links.",{requiredResourceFilters:["spreadsheet"]})], ["context7","Context7","Look up current documentation for software libraries.","developer","context7.com",["https://mcp.context7.com/*"],method("mcp","mcp_remote","none",{serverUrl:"https://mcp.context7.com/mcp"},"S1","Connect Context7 to give agents current library documentation.")], -["shopify","Shopify","Search a store's products and policies, and manage shopping carts.","commerce","shopify.com",["https://*.myshopify.com/api/mcp"],method("storefront-mcp","mcp_remote","none",{serverUrlTemplate:"https://{storeDomain}/api/mcp"},"S3","Connect Shopify's official Storefront MCP server for shopper-facing catalog, policy, and cart tools.",{label:"Connect a Shopify storefront",whenToUse:"Use a store's public myshopify.com domain. No Shopify app or OAuth registration is required.",tenantFields:[{key:"storeDomain",label:"Store domain",type:"text",required:true,placeholder:"your-store.myshopify.com",helperMd:"Enter the full myshopify.com domain without https://.",validation:{pattern:"^[A-Za-z0-9][A-Za-z0-9-]*\\.myshopify\\.com$",maxLength:255}}],consoleLinks:{docs:"https://shopify.dev/docs/apps/build/storefront-mcp/servers/storefront"},warnings:["This is Shopify's Storefront MCP, not Admin API access. It does not manage merchant products, orders, or customers.","The storefront must be publicly reachable. Password-protected or restricted trial stores can return HTTP 401."],requiredResourceFilters:["store"]})], +["shopify","Shopify","Search a store's products and policies, and manage shopping carts.","commerce","shopify.com",["https://*.myshopify.com/api/ucp/mcp","https://*.myshopify.com/api/mcp"],[method("ucp-commerce","mcp_remote","none",{serverUrlTemplate:"https://{storeDomain}/api/ucp/mcp",toolArgumentDefaults:{meta:{"ucp-agent":{profile:"https://shopify.dev/ucp/agent-profiles/examples/2026-04-08/valid-with-capabilities.json"}}}},"S3","Connect Shopify's current UCP server for shopper-facing catalog and commerce tools. Paperclip supplies the required agent profile automatically.",{label:"Shopify UCP commerce",whenToUse:"Recommended for Shopify's current UCP catalog, cart, and checkout tools.",tenantFields:[{key:"storeDomain",label:"Store domain",type:"text",required:true,placeholder:"your-store.myshopify.com",helperMd:"Enter the permanent myshopify.com domain without https://. Custom storefront domains are not the MCP endpoint.",validation:{pattern:"^[A-Za-z0-9][A-Za-z0-9-]*\\.myshopify\\.com$",maxLength:255}}],consoleLinks:{docs:"https://shopify.dev/docs/agents/catalog/storefront-catalog"},warnings:["This is Shopify's shopper-facing UCP server, not Admin API access. It does not manage merchant products or customers.","The storefront must be public. A private or password-protected storefront returns HTTP 401 even when the merchant is signed in to Shopify Admin.","Paperclip currently uses Shopify's documented hosted agent-profile fixture while Paperclip's production UCP profile is being established."],requiredResourceFilters:["store"]}),method("storefront-mcp","mcp_remote","none",{serverUrlTemplate:"https://{storeDomain}/api/mcp"},"S3","Connect Shopify's official Storefront MCP server for shopper-facing catalog, policy, and cart tools.",{label:"Storefront policies and compatibility tools",whenToUse:"Use Shopify's compatibility server when agents need storefront policy and FAQ search.",tenantFields:[{key:"storeDomain",label:"Store domain",type:"text",required:true,placeholder:"your-store.myshopify.com",helperMd:"Enter the permanent myshopify.com domain without https://. Custom storefront domains are not the MCP endpoint.",validation:{pattern:"^[A-Za-z0-9][A-Za-z0-9-]*\\.myshopify\\.com$",maxLength:255}}],consoleLinks:{docs:"https://shopify.dev/docs/apps/build/storefront-mcp/servers/storefront"},warnings:["This is Shopify's Storefront MCP, not Admin API access. It does not manage merchant products, orders, or customers.","The storefront must be public. A private or password-protected storefront returns HTTP 401 even when the merchant is signed in to Shopify Admin."],requiredResourceFilters:["store"]})],{docsUrl:"https://shopify.dev/docs/apps/build/storefront-mcp/servers/storefront",setupPrerequisite:{title:"Launch the storefront before connecting",description:"Shopify's Storefront MCP is a public, no-auth endpoint. Paperclip cannot use the merchant's Shopify Admin session to bypass a private storefront.",steps:["Select a Shopify plan; Shopify keeps trial storefronts private until a plan is selected.","In Shopify Admin, open Online Store → Preferences and set Storefront visibility to Public (remove password protection).","Use the permanent .myshopify.com domain in Paperclip, even if the store also has a custom domain."],actionLabel:"Open Shopify Admin",actionUrl:"https://admin.shopify.com/"}}], ["composio","Composio","Connect Composio so Paperclip can discover and manage the toolkits in your project.","productivity","composio.dev",["https://backend.composio.dev/*"],method("api-key","rest_api","api_key",{serviceHost:"backend.composio.dev"},"S3","Create a scoped project API key in Composio. It needs read access to toolkits and auth configs; later service-connection phases also need connected-account and session access.",{whenToUse:"Use a project API key from the Composio project that owns the toolkits and connected accounts.",credentialFields:[field("apiKey","Composio project API key","Paste the Composio API key")],keyPlacement:{location:"header",name:"x-api-key"},consoleLinks:{keys:"https://app.composio.dev/",settings:"https://app.composio.dev/",docs:"https://docs.composio.dev/reference/authenticating-to-composio/project-api-key-permissions"}}),{featured:true}], ["oauth-generic","OAuth app","Connect a provider using your own OAuth client.","other","oauth.net",[],method("oauth","rest_api","oauth",{},"S3","Register an OAuth client with the provider and add Paperclip's redirect URI.",{credentialFields:[{...field("clientId","Client ID","Paste the client ID"),type:"text",secret:false},field("clientSecret","Client secret","Paste the client secret")]})], ["api-key-generic","API key app","Connect an API using a key from your provider.","other","openapis.org",[],method("api-key","rest_api","api_key",{},"S3","Create a restricted API key and paste it here.",{credentialFields:[field("apiKey","API key","Paste the API key")],keyPlacement:{location:"header",name:"Authorization",prefix:"Bearer "}})], @@ -56,7 +56,9 @@ const apiKeySpec={ mem0:{name:"Authorization",prefix:"Bearer ",placeholder:"m0sk_..."}, oreilly:{name:"Authorization",prefix:"Bearer ",placeholder:"Paste your O'Reilly API token"}, pagerduty:{name:"Authorization",prefix:"Token token=",placeholder:"Paste your PagerDuty user API token"}, - postman:{name:"X-API-Key",prefix:null,placeholder:"PMAK-..."}, + // Postman's general REST API examples use X-API-Key, but its hosted MCP + // server explicitly expects the key as an Authorization bearer token. + postman:{name:"Authorization",prefix:"Bearer ",placeholder:"PMAK-..."}, razorpay:{name:"Authorization",prefix:"Basic ",placeholder:"Paste the base64-encoded key ID and secret"}, sanity:{name:"Authorization",prefix:"Bearer ",placeholder:"sk..."}, similarweb:{name:"api-key",prefix:null,placeholder:"Paste your Similarweb API key"}, @@ -82,12 +84,12 @@ const specialMethodsFor=(entry)=>{ oauthMethodFor(entry,"mcp-insights-only","https://mcp.pscale.dev/mcp/planetscale-insights-only",{label:"Insights only",whenToUse:"Use query insights and schema recommendations without query execution tools.",requiredResourceFilters:["organization","database","branch"]}), ]; if(entry.slug==="postman") return [ - oauthMethodFor(entry,"mcp-oauth-minimal","https://mcp.postman.com/minimal",{label:"US · Minimal"}), - oauthMethodFor(entry,"mcp-oauth-code","https://mcp.postman.com/code",{label:"US · Code"}), - oauthMethodFor(entry,"mcp-oauth-full","https://mcp.postman.com/mcp",{label:"US · Full"}), - apiKeyMethodFor(entry,"mcp-eu-key-minimal","https://mcp.eu.postman.com/minimal",{label:"EU · Minimal"}), - apiKeyMethodFor(entry,"mcp-eu-key-code","https://mcp.eu.postman.com/code",{label:"EU · Code"}), - apiKeyMethodFor(entry,"mcp-eu-key-full","https://mcp.eu.postman.com/mcp",{label:"EU · Full"}), + oauthMethodFor(entry,"mcp-oauth-minimal","https://mcp.postman.com/minimal",{label:"US · Browser sign-in",capabilityProfile:{key:"minimal",label:"Minimal",description:"Essential workspace, collection, and environment tools with the smallest tool catalog."}}), + oauthMethodFor(entry,"mcp-oauth-code","https://mcp.postman.com/code",{label:"US · Browser sign-in",capabilityProfile:{key:"code",label:"Code",description:"Tools for generating client code from API definitions."}}), + oauthMethodFor(entry,"mcp-oauth-full","https://mcp.postman.com/mcp",{label:"US · Browser sign-in",capabilityProfile:{key:"write",label:"Full",description:"All Postman API tools, including write-capable collaboration and advanced features."}}), + apiKeyMethodFor(entry,"mcp-eu-key-minimal","https://mcp.eu.postman.com/minimal",{label:"EU · API key",capabilityProfile:{key:"minimal",label:"Minimal",description:"Essential workspace, collection, and environment tools with the smallest tool catalog."}}), + apiKeyMethodFor(entry,"mcp-eu-key-code","https://mcp.eu.postman.com/code",{label:"EU · API key",capabilityProfile:{key:"code",label:"Code",description:"Tools for generating client code from API definitions."}}), + apiKeyMethodFor(entry,"mcp-eu-key-full","https://mcp.eu.postman.com/mcp",{label:"EU · API key",capabilityProfile:{key:"write",label:"Full",description:"All Postman API tools, including write-capable collaboration and advanced features."}}), ]; if(entry.slug==="pagerduty") return [ apiKeyMethodFor(entry,"mcp-api-key-us","https://mcp.pagerduty.com/mcp",{label:"US service region"}), diff --git a/server/src/__tests__/issue-thread-interaction-routes.test.ts b/server/src/__tests__/issue-thread-interaction-routes.test.ts index c7d66e0926..d8bce7b3e0 100644 --- a/server/src/__tests__/issue-thread-interaction-routes.test.ts +++ b/server/src/__tests__/issue-thread-interaction-routes.test.ts @@ -1126,6 +1126,56 @@ describe.sequential("issue thread interaction routes", () => { ); }); + it("wakes with fresh-approval instructions after an accepted tool action expires", async () => { + const approveToolActionRequest = vi.fn().mockResolvedValue({ + status: "expired", + error: "Managed arguments changed after review", + }); + mockInteractionService.acceptInteraction.mockResolvedValueOnce({ + interaction: { + id: "interaction-tool-action-expired", + companyId: "company-1", + issueId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa", + kind: "request_confirmation", + status: "accepted", + continuationPolicy: "wake_assignee", + payload: { + version: 1, + prompt: "Approve the action?", + toolAction: { + version: 1, + actionRequestId: "action-request-expired", + toolName: "shopify_update_product", + }, + }, + result: { version: 1, outcome: "accepted" }, + }, + createdIssues: [], + }); + const app = await createApp(undefined, { approveToolActionRequest }); + + const res = await request(app) + .post("/api/issues/aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa/interactions/interaction-tool-action-expired/accept") + .send({}); + + expect(res.status).toBe(200); + expect(mockHeartbeatService.wakeup).toHaveBeenCalledWith( + ASSIGNEE_AGENT_ID, + expect.objectContaining({ + payload: expect.objectContaining({ + toolAction: { + toolName: "shopify_update_product", + actionRequestId: "action-request-expired", + decision: "accepted", + executionStatus: "expired", + error: "Managed arguments changed after review", + instructions: "the approved shopify_update_product action expired before execution: Managed arguments changed after review; if the task still requires it, call the tool again to request a fresh approval.", + }, + }), + }), + ); + }); + it("rejects client-supplied tool-action metadata on interaction creation", async () => { const app = await createApp(); diff --git a/server/src/__tests__/tool-access-service.test.ts b/server/src/__tests__/tool-access-service.test.ts index 13512460b3..c784b6d316 100644 --- a/server/src/__tests__/tool-access-service.test.ts +++ b/server/src/__tests__/tool-access-service.test.ts @@ -45,7 +45,13 @@ import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase, } from "./helpers/embedded-postgres.js"; -import { classifyRisk, normalizeConnectionMethodConfig, toolAccessService } from "../services/tool-access.js"; +import { + classifyRisk, + normalizeConnectionMethodConfig, + projectConnectionMethodToolInputSchema, + projectConnectionMethodToolArguments, + toolAccessService, +} from "../services/tool-access.js"; import { accessService } from "../services/access.js"; import { toolAccessPolicyService } from "../services/tool-access-policy.js"; import { secretService } from "../services/secrets.js"; @@ -10972,16 +10978,24 @@ describe("classifyRisk", () => { expect(classifyRisk({ name: "brand_new_tool" }, "posthog")).toBe("write"); expect(classifyRisk({ name: "exec" }, "posthog")).toBe("destructive"); }); + + it("keeps Shopify checkout completion and cancellation behind destructive-action approval", () => { + expect(classifyRisk({ name: "cancel_cart" }, "shopify")).toBe("destructive"); + expect(classifyRisk({ name: "cancel_checkout" }, "shopify")).toBe("destructive"); + expect(classifyRisk({ name: "complete_checkout" }, "shopify")).toBe("destructive"); + expect(classifyRisk({ name: "create_cart" }, "shopify")).toBe("write"); + }); }); describe("normalizeConnectionMethodConfig", () => { const posthog = getConnectableAppDefinition("posthog")!; const apiKeyMethod = posthog.methods.find((method) => method.key === "mcp-api-key")!; const clickhouseMethod = getConnectableAppDefinition("clickhouse")!.methods[0]!; - const shopifyMethod = getConnectableAppDefinition("shopify")!.methods[0]!; + const shopifyMethods = getConnectableAppDefinition("shopify")!.methods; + const shopifyMethod = shopifyMethods.find((method) => method.key === "storefront-mcp")!; + const shopifyUcpMethod = shopifyMethods.find((method) => method.key === "ucp-commerce")!; it("builds a concrete Shopify endpoint from the validated store domain", () => { - const shopifyMethod = getConnectableAppDefinition("shopify")!.methods[0]!; expect(normalizeConnectionMethodConfig(shopifyMethod, { storeDomain: "paperclip-demo.myshopify.com", })).toEqual({ @@ -11045,5 +11059,53 @@ describe("normalizeConnectionMethodConfig", () => { expect(() => normalizeConnectionMethodConfig(shopifyMethod, { storeDomain: "shop.myshopify.com@example.com", })).toThrow("Store domain has an invalid value"); + expect(normalizeConnectionMethodConfig(shopifyUcpMethod, { + storeDomain: "rcvbsa-pz.myshopify.com", + })).toMatchObject({ + url: "https://rcvbsa-pz.myshopify.com/api/ucp/mcp", + }); + expect(projectConnectionMethodToolArguments(shopifyUcpMethod, { + catalog: { query: "shirts" }, + meta: { + caller: "kept", + "ucp-agent": { profile: "https://attacker.example/profile.json" }, + }, + })).toEqual({ + catalog: { query: "shirts" }, + meta: { + caller: "kept", + "ucp-agent": { + profile: "https://shopify.dev/ucp/agent-profiles/examples/2026-04-08/valid-with-capabilities.json", + }, + }, + }); + expect(projectConnectionMethodToolInputSchema(shopifyUcpMethod, { + type: "object", + required: ["meta", "catalog"], + properties: { + meta: { + type: "object", + required: ["ucp-agent", "idempotency-key"], + properties: { + "ucp-agent": { type: "object" }, + "idempotency-key": { type: "string" }, + }, + }, + catalog: { type: "object" }, + }, + })).toEqual({ + type: "object", + required: ["meta", "catalog"], + properties: { + meta: { + type: "object", + required: ["idempotency-key"], + properties: { + "idempotency-key": { type: "string" }, + }, + }, + catalog: { type: "object" }, + }, + }); }); }); diff --git a/server/src/__tests__/tool-gateway-service.test.ts b/server/src/__tests__/tool-gateway-service.test.ts index 0895aabcc5..8cfb065789 100644 --- a/server/src/__tests__/tool-gateway-service.test.ts +++ b/server/src/__tests__/tool-gateway-service.test.ts @@ -523,6 +523,139 @@ describeEmbeddedPostgres("tool gateway service", () => { expect(approved.status).toBe("approved"); const [parkedInvocation] = await db.select().from(toolInvocations).where(eq(toolInvocations.id, invocation.id)); expect(parkedInvocation.status).toBe("awaiting_approval"); + + await expect(gateway.executeTool({ + sessionToken: session.token, + tool: "mcp-remote-fixture:update_note", + parameters, + })).rejects.toMatchObject({ reasonCode: "legacy_approved_action_inert" }); + + const [settledRequest] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.id, actionRequest.id)); + const [settledInvocation] = await db + .select() + .from(toolInvocations) + .where(eq(toolInvocations.id, invocation.id)); + expect(settledRequest.status).toBe("failed"); + expect(settledInvocation.status).toBe("failed"); + expect(settledInvocation.errorCode).toBe("legacy_approved_action_inert"); + expect(settledInvocation.idempotencyKey).toBeNull(); + + await expect(gateway.executeTool({ + sessionToken: session.token, + tool: "mcp-remote-fixture:update_note", + parameters, + })).rejects.toMatchObject({ reasonCode: "approval_required" }); + expect(await db.select().from(toolActionRequests)).toHaveLength(2); + }); + + it("does not let a stale legacy consumer overwrite the winning approved execution", async () => { + const { company, agent, run } = await createRunFixture(db); + await db.insert(toolPolicies).values({ + companyId: company.id, + name: "Review note writes", + policyType: "require_approval", + selectors: { toolName: "mcp-remote-fixture:update_note" }, + }); + + let observeLegacyClaim!: () => void; + const legacyClaimObserved = new Promise((resolve) => { + observeLegacyClaim = resolve; + }); + let releaseLegacyClaim!: () => void; + const legacyClaimBlocked = new Promise((resolve) => { + releaseLegacyClaim = resolve; + }); + const gateway = createTestToolGatewayService(db, { + beforeLegacyApprovedActionClaim: async () => { + observeLegacyClaim(); + await legacyClaimBlocked; + }, + }); + const session = await gateway.createSession({ + companyId: company.id, + agentId: agent.id, + runId: run.id, + }); + const parameters = { noteId: "n1", body: "legacy race" }; + + await expect(gateway.executeTool({ + sessionToken: session.token, + tool: "mcp-remote-fixture:update_note", + parameters, + })).rejects.toMatchObject({ reasonCode: "approval_required" }); + const [actionRequest] = await db.select().from(toolActionRequests); + const [invocation] = await db.select().from(toolInvocations); + const currentSignature = actionRequest.signedArguments!; + const legacySignature = signToolArguments({ + invocationId: invocation.id, + toolName: invocation.toolName, + canonicalArguments: canonicalToolArguments(parameters), + signingSecret: testToolActionSigningSecret, + }); + await db + .update(toolActionRequests) + .set({ signedArguments: legacySignature }) + .where(eq(toolActionRequests.id, actionRequest.id)); + await gateway.approveActionRequest({ + companyId: company.id, + actionRequestId: actionRequest.id, + actor: { userId: "board-user" }, + }); + + const staleAttempt = gateway.executeTool({ + sessionToken: session.token, + tool: "mcp-remote-fixture:update_note", + approvedActionRequestId: actionRequest.id, + parameters, + }).then( + (value) => ({ status: "fulfilled" as const, value }), + (error: unknown) => ({ status: "rejected" as const, error }), + ); + await legacyClaimObserved; + + // Simulate a concurrent repair that restores the current signed envelope + // after the stale consumer has read the legacy envelope but before it owns + // the approved -> executing claim. + await db + .update(toolActionRequests) + .set({ signedArguments: currentSignature, updatedAt: new Date() }) + .where(and( + eq(toolActionRequests.id, actionRequest.id), + eq(toolActionRequests.status, "approved"), + )); + const winner = await gateway.executeTool({ + sessionToken: session.token, + tool: "mcp-remote-fixture:update_note", + approvedActionRequestId: actionRequest.id, + parameters, + }); + expect(winner.status).toBe("completed"); + + releaseLegacyClaim(); + const stale = await staleAttempt; + expect(stale).toMatchObject({ + status: "rejected", + error: { reasonCode: "action_already_consumed" }, + }); + + const [settledRequest] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.id, actionRequest.id)); + const [settledInvocation] = await db + .select() + .from(toolInvocations) + .where(eq(toolInvocations.id, invocation.id)); + expect(settledRequest.status).toBe("executed"); + expect(settledInvocation).toMatchObject({ + status: "succeeded", + errorCode: null, + errorMessage: null, + }); + expect(settledInvocation.idempotencyKey).not.toBeNull(); }); it("does not leave unsigned action requests pending when signing is unavailable", async () => { diff --git a/server/src/__tests__/tool-gateway.test.ts b/server/src/__tests__/tool-gateway.test.ts index bb823b4b03..1ff2c7b2a4 100644 --- a/server/src/__tests__/tool-gateway.test.ts +++ b/server/src/__tests__/tool-gateway.test.ts @@ -46,6 +46,12 @@ import { import type { PluginToolDispatcher } from "../services/plugin-tool-dispatcher.js"; import { mcpGatewayProtocolRoutes, toolGatewayRoutes } from "../routes/tool-gateway.js"; import { toolAccessService } from "../services/tool-access.js"; +import { + canonicalToolArguments, + readSignedToolArgumentsPayload, + signToolArguments, + summarizeToolValue, +} from "../services/tool-content-guards.js"; import { createToolGatewayService, ToolGatewayHttpError } from "../services/tool-gateway.js"; import type { ComposioClient } from "../services/composio.js"; import { secretService } from "../services/secrets.js"; @@ -2657,6 +2663,307 @@ rl.on("line", (line) => { } }); + it("expires legacy managed-connector approvals before provider dispatch", async () => { + const company = await createCompany(db); + const agent = await createAgent(db, company.id); + const { issue, run } = await createIssueAndRun(db, company.id, agent.id); + const fake = await startFakeRemoteMcpServer((fakeRequest) => ({ + body: { + jsonrpc: "2.0", + id: fakeRequest.body?.id, + result: { content: [{ type: "text", text: "should not run" }] }, + }, + })); + + try { + const remoteTool = await createRemoteMcpTool(db, company.id, { + applicationKey: "shopify", + toolName: "kv_set", + url: fake.url, + connectionConfig: { + sourceTemplateKey: "shopify", + connectionMethodKey: "ucp-commerce", + methodConfig: { storeDomain: "paperclip-demo.myshopify.com" }, + }, + }); + const toolName = expectedConnectedToolName({ + applicationKey: remoteTool.application.applicationKey, + connectionId: remoteTool.connection.id, + toolName: remoteTool.catalogEntry.toolName, + }); + await allowToolsForAgent(db, company.id, agent.id, [toolName]); + await db.insert(toolPolicies).values({ + companyId: company.id, + name: "Review managed Shopify writes", + policyType: "require_approval", + selectors: { connectionId: remoteTool.connection.id }, + priority: 10, + }); + + const gateway = createTestToolGatewayService(db); + const session = await gateway.createSession({ companyId: company.id, agentId: agent.id, runId: run.id }); + await gateway.executeTool({ + sessionToken: session.token, + tool: toolName, + parameters: { key: "legacy", value: "reviewed" }, + }).then( + () => { + throw new Error("Expected managed Shopify call to require approval"); + }, + (error) => expectGatewayError(error, 409, "approval_required"), + ); + + const [actionRequest] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.companyId, company.id)); + const signedPayload = readSignedToolArgumentsPayload({ + signedArguments: actionRequest.signedArguments, + invocationId: actionRequest.invocationId, + toolName, + signingSecret: testToolActionSigningSecret, + }); + expect(signedPayload?.executionOnApprove).toBe(true); + + // Model an approval signed before Shopify's UCP agent profile became a + // required managed argument. Its signature and hash are valid for that + // historical payload, but it is no longer compatible with dispatch. + const legacyParameters = { key: "legacy", value: "reviewed" }; + const legacyCanonical = canonicalToolArguments(legacyParameters); + const legacySummary = summarizeToolValue(legacyParameters); + await db + .update(toolActionRequests) + .set({ + signedArguments: signToolArguments({ + invocationId: actionRequest.invocationId, + toolName, + canonicalArguments: legacyCanonical, + approvalSnapshot: signedPayload?.approvalSnapshot, + executionOnApprove: true, + signingSecret: testToolActionSigningSecret, + }), + canonicalArgumentsHash: legacySummary.sha256, + canonicalArgumentsSummary: legacySummary, + updatedAt: new Date(), + }) + .where(eq(toolActionRequests.id, actionRequest.id)); + + await expect(gateway.approveActionRequest({ + companyId: company.id, + issueId: issue.id, + interactionId: actionRequest.interactionId!, + actionRequestId: actionRequest.id, + actor: { agentId: agent.id }, + })).resolves.toMatchObject({ status: "expired" }); + + expect(fake.requests).toHaveLength(0); + const [expiredRequest] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.id, actionRequest.id)); + expect(expiredRequest.status).toBe("expired"); + const [failedInvocation] = await db + .select() + .from(toolInvocations) + .where(eq(toolInvocations.id, actionRequest.invocationId)); + expect(failedInvocation).toMatchObject({ + status: "failed", + approvalState: "expired", + errorCode: "approved_tool_managed_arguments_changed", + }); + } finally { + await fake.close(); + } + }); + + it("does not expire a managed-connector provider execution already in flight", async () => { + const company = await createCompany(db); + const agent = await createAgent(db, company.id); + const { issue, run } = await createIssueAndRun(db, company.id, agent.id); + const fake = await startFakeRemoteMcpServer((fakeRequest) => ({ + body: { + jsonrpc: "2.0", + id: fakeRequest.body?.id, + result: { content: [{ type: "text", text: "concurrent execution won" }] }, + }, + })); + + try { + const remoteTool = await createRemoteMcpTool(db, company.id, { + applicationKey: "shopify", + toolName: "kv_set", + url: fake.url, + connectionConfig: { + sourceTemplateKey: "shopify", + connectionMethodKey: "ucp-commerce", + methodConfig: { storeDomain: "paperclip-demo.myshopify.com" }, + }, + }); + const toolName = expectedConnectedToolName({ + applicationKey: remoteTool.application.applicationKey, + connectionId: remoteTool.connection.id, + toolName: remoteTool.catalogEntry.toolName, + }); + await allowToolsForAgent(db, company.id, agent.id, [toolName]); + await db.insert(toolPolicies).values({ + companyId: company.id, + name: "Review raced managed Shopify writes", + policyType: "require_approval", + selectors: { connectionId: remoteTool.connection.id }, + priority: 10, + }); + + let driftExpiryReached!: () => void; + const driftExpiryStarted = new Promise((resolve) => { + driftExpiryReached = resolve; + }); + let resumeDriftExpiry!: () => void; + const driftExpiryResume = new Promise((resolve) => { + resumeDriftExpiry = resolve; + }); + const gateway = createTestToolGatewayService(db, { + beforeManagedArgumentDriftExpiry: async () => { + driftExpiryReached(); + await driftExpiryResume; + }, + }); + const session = await gateway.createSession({ companyId: company.id, agentId: agent.id, runId: run.id }); + await gateway.executeTool({ + sessionToken: session.token, + tool: toolName, + parameters: { key: "raced", value: "reviewed" }, + }).then( + () => { + throw new Error("Expected managed Shopify call to require approval"); + }, + (error) => expectGatewayError(error, 409, "approval_required"), + ); + + const [actionRequest] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.companyId, company.id)); + const signedPayload = readSignedToolArgumentsPayload({ + signedArguments: actionRequest.signedArguments, + invocationId: actionRequest.invocationId, + toolName, + signingSecret: testToolActionSigningSecret, + }); + const legacyParameters = { key: "raced", value: "reviewed" }; + const legacyCanonical = canonicalToolArguments(legacyParameters); + const legacySummary = summarizeToolValue(legacyParameters); + await db + .update(toolActionRequests) + .set({ + signedArguments: signToolArguments({ + invocationId: actionRequest.invocationId, + toolName, + canonicalArguments: legacyCanonical, + approvalSnapshot: signedPayload?.approvalSnapshot, + executionOnApprove: true, + signingSecret: testToolActionSigningSecret, + }), + canonicalArgumentsHash: legacySummary.sha256, + canonicalArgumentsSummary: legacySummary, + updatedAt: new Date(), + }) + .where(eq(toolActionRequests.id, actionRequest.id)); + + const approvedAt = new Date(); + await db + .update(issueThreadInteractions) + .set({ status: "accepted", resolvedByAgentId: agent.id, resolvedAt: approvedAt, updatedAt: approvedAt }) + .where(eq(issueThreadInteractions.id, actionRequest.interactionId!)); + await db + .update(toolActionRequests) + .set({ status: "approved", resolvedByAgentId: agent.id, resolvedAt: approvedAt, updatedAt: approvedAt }) + .where(eq(toolActionRequests.id, actionRequest.id)); + await db + .update(toolInvocations) + .set({ approvalState: "approved", updatedAt: approvedAt }) + .where(eq(toolInvocations.id, actionRequest.invocationId)); + + const retrying = gateway.executeTool({ + sessionToken: session.token, + tool: toolName, + parameters: legacyParameters, + approvedActionRequestId: actionRequest.id, + }); + await driftExpiryStarted; + + const executingAt = new Date(); + await db + .update(toolActionRequests) + .set({ status: "executing", updatedAt: executingAt }) + .where(eq(toolActionRequests.id, actionRequest.id)); + await db + .update(toolInvocations) + .set({ + status: "executing", + approvalState: "approved", + errorCode: null, + errorMessage: null, + startedAt: executingAt, + completedAt: null, + updatedAt: executingAt, + }) + .where(eq(toolInvocations.id, actionRequest.invocationId)); + resumeDriftExpiry(); + + await retrying.then( + () => { + throw new Error("Expected the stale approved retry to request a new approval"); + }, + (error) => expectGatewayError(error, 409, "approved_tool_managed_arguments_changed"), + ); + const [inFlightRequest] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.id, actionRequest.id)); + const [inFlightInvocation] = await db + .select() + .from(toolInvocations) + .where(eq(toolInvocations.id, actionRequest.invocationId)); + expect(inFlightRequest.status).toBe("executing"); + expect(inFlightInvocation).toMatchObject({ + status: "executing", + approvalState: "approved", + errorCode: null, + errorMessage: null, + }); + + const completedAt = new Date(); + const winnerSummary = summarizeToolValue({ winner: "concurrent execution" }); + await db + .update(toolActionRequests) + .set({ status: "executed", resolvedAt: completedAt, updatedAt: completedAt }) + .where(eq(toolActionRequests.id, actionRequest.id)); + await db + .update(toolInvocations) + .set({ + status: "completed", + resultSummary: winnerSummary, + completedAt, + updatedAt: completedAt, + }) + .where(eq(toolInvocations.id, actionRequest.invocationId)); + const [winnerInvocation] = await db + .select() + .from(toolInvocations) + .where(eq(toolInvocations.id, actionRequest.invocationId)); + expect(winnerInvocation).toMatchObject({ + status: "completed", + approvalState: "approved", + resultSummary: winnerSummary, + errorCode: null, + errorMessage: null, + }); + expect(fake.requests).toHaveLength(0); + } finally { + await fake.close(); + } + }); + it("enforces policy, approvals, retries, rate limits, and company boundaries for connected remote MCP calls", async () => { const company = await createCompany(db); const agent = await createAgent(db, company.id); @@ -2664,19 +2971,41 @@ rl.on("line", (line) => { const otherCompany = await createCompany(db); const otherAgent = await createAgent(db, otherCompany.id); const { run: otherRun } = await createIssueAndRun(db, otherCompany.id, otherAgent.id); - const fake = await startFakeRemoteMcpServer((fakeRequest) => ({ - body: { - jsonrpc: "2.0", - id: fakeRequest.body?.id, - result: { - content: [{ type: "text", text: "connected ok" }], - structuredContent: { - receivedArguments: (fakeRequest.body?.params as Record | undefined)?.arguments, - leakedToken: "sk-connected-mcp-secret-123456", + let pauseApprovedExecution = false; + let approvedExecutionReached!: () => void; + const approvedExecutionStarted = new Promise((resolve) => { + approvedExecutionReached = resolve; + }); + let releaseApprovedExecution = () => {}; + const approvedExecutionRelease = new Promise((resolve) => { + releaseApprovedExecution = resolve; + }); + const fake = await startFakeRemoteMcpServer(async (fakeRequest) => { + const requestArguments = (fakeRequest.body?.params as Record | undefined)?.arguments; + if ( + pauseApprovedExecution + && requestArguments + && typeof requestArguments === "object" + && (requestArguments as Record).key === "approved" + ) { + pauseApprovedExecution = false; + approvedExecutionReached(); + await approvedExecutionRelease; + } + return { + body: { + jsonrpc: "2.0", + id: fakeRequest.body?.id, + result: { + content: [{ type: "text", text: "connected ok" }], + structuredContent: { + receivedArguments: requestArguments, + leakedToken: "sk-connected-mcp-secret-123456", + }, }, }, - }, - })); + }; + }); try { const denyTool = await createRemoteMcpTool(db, company.id, { @@ -2720,9 +3049,14 @@ rl.on("line", (line) => { expect(JSON.stringify(deniedInvocation)).not.toContain("sk-denied-secret-123456"); const approvalTool = await createRemoteMcpTool(db, company.id, { - applicationKey: "approval-app", + applicationKey: "shopify", toolName: "kv_set", url: fake.url, + connectionConfig: { + sourceTemplateKey: "shopify", + connectionMethodKey: "ucp-commerce", + methodConfig: { storeDomain: "paperclip-demo.myshopify.com" }, + }, }); await allowToolsForAgent(db, company.id, agent.id, [ expectedConnectedToolName({ @@ -2760,6 +3094,9 @@ rl.on("line", (line) => { issueId: issue.id, status: "pending", canonicalArgumentsHash: expect.any(String), + canonicalArgumentsSummary: { + summary: expect.stringContaining("valid-with-capabilities.json"), + }, }); const [approvalInteraction] = await db .select() @@ -2802,7 +3139,7 @@ rl.on("line", (line) => { policyDecision: "require_approval", connectionId: approvalTool.connection.id, providerType: "mcp_remote_http", - applicationKey: "approval-app", + applicationKey: "shopify", upstreamToolName: "kv_set", }); @@ -2816,12 +3153,28 @@ rl.on("line", (line) => { }) .where(eq(issueThreadInteractions.id, approvalRequest.interactionId!)); - await expect(gateway.executeTool({ + pauseApprovedExecution = true; + const approvedExecution = gateway.executeTool({ sessionToken: session.token, tool: approvalToolName, parameters: { key: "approved", value: "tampered" }, approvedActionRequestId: approvalRequest.id, - })).resolves.toMatchObject({ + }); + await approvedExecutionStarted; + const [executingApproval] = await db + .select() + .from(toolActionRequests) + .where(eq(toolActionRequests.id, approvalRequest.id)); + expect(executingApproval.status).toBe("executing"); + + const concurrentRetry = gateway.executeTool({ + sessionToken: session.token, + tool: approvalToolName, + parameters: { key: "approved", value: "original" }, + }); + releaseApprovedExecution(); + + await expect(approvedExecution).resolves.toMatchObject({ status: "completed", tool: approvalToolName, result: { @@ -2832,10 +3185,22 @@ rl.on("line", (line) => { }, }, }); + await expect(concurrentRetry).resolves.toMatchObject({ + status: "replayed", + result: expect.anything(), + }); expect(fake.requests.at(-1)!.body).toMatchObject({ params: { name: "kv_set", - arguments: { key: "approved", value: "original" }, + arguments: { + key: "approved", + value: "original", + meta: { + "ucp-agent": { + profile: "https://shopify.dev/ucp/agent-profiles/examples/2026-04-08/valid-with-capabilities.json", + }, + }, + }, }, }); const [executedApproval] = await db @@ -3019,9 +3384,10 @@ rl.on("line", (line) => { expect(persisted).not.toContain("sk-connected-mcp-secret-123456"); expect(persisted).not.toContain("sk-denied-secret-123456"); expect(persisted).toContain("mcp_remote_http"); - expect(persisted).toContain("approval-app"); + expect(persisted).toContain("shopify"); expect(persisted).toContain("kv_set"); } finally { + releaseApprovedExecution(); await fake.close(); } }); diff --git a/server/src/routes/issues.ts b/server/src/routes/issues.ts index 4645e2ffeb..ab07d8c789 100644 --- a/server/src/routes/issues.ts +++ b/server/src/routes/issues.ts @@ -1996,6 +1996,18 @@ function readToolActionContinuationContext(interaction: { }; } + if (executionStatus === "expired") { + const expirationMessage = error ? `: ${error}` : ""; + return { + toolName, + actionRequestId, + decision: "accepted", + executionStatus, + ...(error ? { error } : {}), + instructions: `the approved ${toolName} action expired before execution${expirationMessage}; if the task still requires it, call the tool again to request a fresh approval.`, + }; + } + return { toolName, actionRequestId, diff --git a/server/src/services/approved-execution-wait.test.ts b/server/src/services/approved-execution-wait.test.ts new file mode 100644 index 0000000000..61bc7f6b73 --- /dev/null +++ b/server/src/services/approved-execution-wait.test.ts @@ -0,0 +1,59 @@ +import { describe, expect, it } from "vitest"; +import { extendApprovedExecutionWaitDeadline } from "./approved-execution-wait.js"; + +describe("extendApprovedExecutionWaitDeadline", () => { + it("gives provider execution a full wait budget after approval preparation", () => { + const preparationDeadlineMs = 65_000; + + expect(extendApprovedExecutionWaitDeadline({ + currentDeadlineMs: preparationDeadlineMs, + invocationStatus: "executing", + invocationStartedAt: new Date(60_000), + preparationStartedAt: new Date(0), + preparationWaitMs: 120_000, + executionWaitMs: 65_000, + })).toBe(125_000); + }); + + it("gives asynchronous approval preparation its own bounded wait budget", () => { + expect(extendApprovedExecutionWaitDeadline({ + currentDeadlineMs: 65_000, + invocationStatus: "awaiting_approval", + invocationStartedAt: new Date(60_000), + preparationStartedAt: new Date(10_000), + preparationWaitMs: 120_000, + executionWaitMs: 65_000, + })).toBe(130_000); + }); + + it("still gives the provider its full window after long preparation", () => { + const preparationDeadlineMs = extendApprovedExecutionWaitDeadline({ + currentDeadlineMs: 65_000, + invocationStatus: "authorized", + invocationStartedAt: null, + preparationStartedAt: new Date(10_000), + preparationWaitMs: 120_000, + executionWaitMs: 65_000, + }); + + expect(extendApprovedExecutionWaitDeadline({ + currentDeadlineMs: preparationDeadlineMs, + invocationStatus: "executing", + invocationStartedAt: new Date(125_000), + preparationStartedAt: new Date(10_000), + preparationWaitMs: 120_000, + executionWaitMs: 65_000, + })).toBe(190_000); + }); + + it("never shortens an existing waiter deadline", () => { + expect(extendApprovedExecutionWaitDeadline({ + currentDeadlineMs: 100_000, + invocationStatus: "succeeded", + invocationStartedAt: new Date(10_000), + preparationStartedAt: new Date(0), + preparationWaitMs: 120_000, + executionWaitMs: 65_000, + })).toBe(100_000); + }); +}); diff --git a/server/src/services/approved-execution-wait.ts b/server/src/services/approved-execution-wait.ts new file mode 100644 index 0000000000..0bac30fd6a --- /dev/null +++ b/server/src/services/approved-execution-wait.ts @@ -0,0 +1,32 @@ +import type { ToolInvocationStatus } from "@paperclipai/shared"; + +const PREPARATION_STATUSES = new Set([ + "pending", + "authorized", + "awaiting_approval", +]); + +export function extendApprovedExecutionWaitDeadline(input: { + currentDeadlineMs: number; + invocationStatus: ToolInvocationStatus; + invocationStartedAt: Date | null; + preparationStartedAt: Date | null; + preparationWaitMs: number; + executionWaitMs: number; +}): number { + if (PREPARATION_STATUSES.has(input.invocationStatus)) { + return input.preparationStartedAt + ? Math.max( + input.currentDeadlineMs, + input.preparationStartedAt.getTime() + input.preparationWaitMs, + ) + : input.currentDeadlineMs; + } + if (!input.invocationStartedAt) { + return input.currentDeadlineMs; + } + return Math.max( + input.currentDeadlineMs, + input.invocationStartedAt.getTime() + input.executionWaitMs, + ); +} diff --git a/server/src/services/tool-access.ts b/server/src/services/tool-access.ts index c18965aed7..77b95dd5b3 100644 --- a/server/src/services/tool-access.ts +++ b/server/src/services/tool-access.ts @@ -927,6 +927,88 @@ export function projectedConnectionHeaders(connection: typeof toolConnections.$i return normalizeConnectionMethodConfig(method, asRecord(connection.config.methodConfig)).headers ?? {}; } +function mergeManagedToolArguments( + supplied: Record, + managed: Record, +): Record { + const merged = { ...supplied }; + for (const [key, value] of Object.entries(managed)) { + const suppliedValue = merged[key]; + merged[key] = asRecord(value) === value && asRecord(suppliedValue) === suppliedValue + ? mergeManagedToolArguments(suppliedValue as Record, value as Record) + : value; + } + return merged; +} + +export function projectConnectionMethodToolArguments( + method: ConnectionMethodDef, + parameters: unknown, +): Record { + const supplied = asRecord(parameters); + const managed = method.defaults?.toolArgumentDefaults; + return managed ? mergeManagedToolArguments(supplied, managed) : supplied; +} + +function stripManagedToolArgumentSchema( + schema: Record, + managed: Record, +): Record { + const properties = asRecord(schema.properties); + if (Object.keys(properties).length === 0) return schema; + const nextProperties = { ...properties }; + for (const [key, managedValue] of Object.entries(managed)) { + const propertySchema = asRecord(nextProperties[key]); + const managedRecord = asRecord(managedValue); + if (Object.keys(propertySchema).length === 0 || Object.keys(managedRecord).length === 0) { + delete nextProperties[key]; + continue; + } + const projectedProperty = stripManagedToolArgumentSchema(propertySchema, managedRecord); + if (Object.keys(asRecord(projectedProperty.properties)).length === 0) delete nextProperties[key]; + else nextProperties[key] = projectedProperty; + } + const nextSchema: Record = { ...schema, properties: nextProperties }; + if (Array.isArray(schema.required)) { + const required = schema.required.filter((key): key is string => typeof key === "string" && key in nextProperties); + if (required.length > 0) nextSchema.required = required; + else delete nextSchema.required; + } + return nextSchema; +} + +export function projectConnectionMethodToolInputSchema( + method: ConnectionMethodDef, + inputSchema: Record, +): Record { + const managed = method.defaults?.toolArgumentDefaults; + return managed ? stripManagedToolArgumentSchema(inputSchema, managed) : inputSchema; +} + +export function projectedConnectionToolArguments( + connection: typeof toolConnections.$inferSelect, + parameters: unknown, +): Record { + const sourceTemplateKey = typeof connection.config.sourceTemplateKey === "string" + ? connection.config.sourceTemplateKey + : null; + const app = sourceTemplateKey ? getConnectableAppDefinition(sourceTemplateKey) : null; + if (!app) return asRecord(parameters); + return projectConnectionMethodToolArguments(connectionMethodForConnection(app, connection), parameters); +} + +export function projectedConnectionToolInputSchema( + connection: typeof toolConnections.$inferSelect, + inputSchema: Record, +): Record { + const sourceTemplateKey = typeof connection.config.sourceTemplateKey === "string" + ? connection.config.sourceTemplateKey + : null; + const app = sourceTemplateKey ? getConnectableAppDefinition(sourceTemplateKey) : null; + if (!app) return inputSchema; + return projectConnectionMethodToolInputSchema(connectionMethodForConnection(app, connection), inputSchema); +} + function googleSheetsAllowedSpreadsheetIds(configValues: Record | undefined): string[] { const raw = configValues?.allowedSpreadsheetIds; const values = Array.isArray(raw) ? raw : typeof raw === "string" ? raw.split(/[\n,]/g) : []; @@ -1233,7 +1315,11 @@ function toCatalogEntryForConnection( row: typeof toolCatalogEntries.$inferSelect, connection: typeof toolConnections.$inferSelect, ): ToolCatalogEntry { - const catalogEntry = toCatalogEntry(row); + const rawCatalogEntry = toCatalogEntry(row); + const catalogEntry = { + ...rawCatalogEntry, + inputSchema: projectedConnectionToolInputSchema(connection, rawCatalogEntry.inputSchema ?? {}), + }; if ( connection.transport === "local_stdio" && asRecord(connection.config).templateId === GOOGLE_SHEETS_TEMPLATE_ID @@ -1737,6 +1823,12 @@ const NOTION_WRITE_TOOLS = new Set([ "notion-update-view", ]); +const SHOPIFY_DESTRUCTIVE_TOOLS = new Set([ + "cancel-cart", + "cancel-checkout", + "complete-checkout", +]); + function normalizedProviderToolName(toolName: string): string { return toolName .replace(/([a-z0-9])([A-Z])/g, "$1-$2") @@ -1749,6 +1841,7 @@ export function classifyRisk(tool: McpToolDescriptor, sourceTemplateKey?: string if (annotations.destructiveHint === true || annotations.destructive === true) return "destructive"; const normalizedToolName = normalizedProviderToolName(tool.name); if (sourceTemplateKey === "posthog" && normalizedToolName === "exec") return "destructive"; + if (sourceTemplateKey === "shopify" && SHOPIFY_DESTRUCTIVE_TOOLS.has(normalizedToolName)) return "destructive"; // Notion's hosted MCP catalog contains mutations whose names do not use one // of the generic create/update/delete verbs (move, duplicate, and convert). // Keep all reviewed tools explicit so provider changes are visible in code, diff --git a/server/src/services/tool-gateway.ts b/server/src/services/tool-gateway.ts index 2e4be1eb04..a9caaa433b 100644 --- a/server/src/services/tool-gateway.ts +++ b/server/src/services/tool-gateway.ts @@ -68,7 +68,11 @@ import { mcpHttpRequestHeaders, parseMcpHttpResponseBody, } from "./mcp-http.js"; -import { projectedConnectionHeaders } from "./tool-access.js"; +import { + projectedConnectionHeaders, + projectedConnectionToolArguments, + projectedConnectionToolInputSchema, +} from "./tool-access.js"; import { parseRemoteHttpEndpoint } from "./remote-http-endpoint-guard.js"; import { guardedRemoteHttpFetch, type GuardedRemoteHttpFetchOptions } from "./remote-http-fetch.js"; import { @@ -109,6 +113,7 @@ import { validateToolContent, verifyToolArgumentsSignature, } from "./tool-content-guards.js"; +import { extendApprovedExecutionWaitDeadline } from "./approved-execution-wait.js"; const DEFAULT_SESSION_TTL_MS = 15 * 60 * 1000; const MAX_SESSION_TTL_MS = 60 * 60 * 1000; @@ -139,6 +144,16 @@ export function isConnectionGrantAudienceAllowed( // `tool_timeout` even though the approval succeeded. Give approved executions // the full permitted headroom instead. const APPROVED_EXECUTION_TIMEOUT_MS = 60_000; +const ACTION_REQUEST_EXECUTION_POLL_MS = 25; +// Approval execution performs live target, signature, managed-argument, and +// issue-state checks before provider dispatch. Give that preparation a +// separate bounded window so it cannot consume the provider's execution +// budget for concurrent consumers. +const ACTION_REQUEST_PREPARATION_WAIT_MS = 2 * 60 * 1000; +// Concurrent consumers must wait at least as long as the provider execution +// they are joining. The extra grace lets the owner persist the terminal request +// state after the provider timeout/result settles. +const ACTION_REQUEST_EXECUTION_WAIT_MS = APPROVED_EXECUTION_TIMEOUT_MS + 5_000; // The gateway creates an ask-first request in two steps: it inserts the row // with a null signature, then it signs the row and sets the expiry. A concurrent // matching call can observe the row in this window. A null signature alone does @@ -833,6 +848,10 @@ export function createToolGatewayService( }) => Promise; /** Test seam for resolving Vercel Connect credentials. */ vercelConnectClient?: VercelConnectClient | null; + /** Test seam for reproducing the managed-argument drift expiry race. */ + beforeManagedArgumentDriftExpiry?: () => Promise; + /** Test seam for pausing a legacy approved request before its execution claim. */ + beforeLegacyApprovedActionClaim?: () => Promise; mcpGatewayProtocolLimits?: Partial<{ authFailures: Partial; gatewayRequests: Partial; @@ -1001,7 +1020,7 @@ export function createToolGatewayService( ? `${baseName}-${shortStableId(catalogEntry.id)}` : baseName; const applicationKey = application.applicationKey ?? null; - const inputSchema = catalogEntry.inputSchema ?? {}; + const inputSchema = projectedConnectionToolInputSchema(connection, catalogEntry.inputSchema ?? {}); const outputSchema = catalogEntry.outputSchema ?? null; const annotations = catalogEntry.annotations ?? {}; const risk = riskFromCatalogEntry(catalogEntry); @@ -3253,6 +3272,25 @@ export function createToolGatewayService( return { entry, connection }; } + async function governedToolArguments( + session: ToolGatewaySession, + tool: ToolGatewayDescriptor, + parameters: unknown, + ): Promise { + if (tool.providerType !== "mcp_remote_http") return parameters; + const { connection } = await resolveConnectedRemoteTool(session, tool); + return projectedConnectionToolArguments(connection, parameters); + } + + async function approvedManagedArgumentsRemainCurrent( + session: ToolGatewaySession, + tool: ToolGatewayDescriptor, + reviewedParameters: unknown, + ): Promise { + const currentParameters = await governedToolArguments(session, tool, reviewedParameters); + return stableSerialize(currentParameters) === stableSerialize(reviewedParameters); + } + async function resolveConnectedLocalStdioTool(session: ToolGatewaySession, tool: ToolGatewayDescriptor) { if (tool.providerType !== "mcp_local_stdio" || !tool.connectionId || !tool.catalogEntryId) { throw new ToolGatewayHttpError(404, `Tool "${tool.name}" not found`, "tool_not_found"); @@ -3871,7 +3909,7 @@ export function createToolGatewayService( method: "tools/call", params: { name: entry.toolName, - arguments: parameters ?? {}, + arguments: parameters, }, }), }; @@ -4895,17 +4933,39 @@ export function createToolGatewayService( } async function waitForActionRequestExecution(actionRequestId: string) { - for (let attempt = 0; attempt < 500; attempt += 1) { - const [row] = await db - .select() + let deadline = Date.now() + ACTION_REQUEST_EXECUTION_WAIT_MS; + while (true) { + const [match] = await db + .select({ + actionRequest: toolActionRequests, + invocationStatus: toolInvocations.status, + invocationStartedAt: toolInvocations.startedAt, + }) .from(toolActionRequests) + .innerJoin(toolInvocations, eq(toolInvocations.id, toolActionRequests.invocationId)) .where(eq(toolActionRequests.id, actionRequestId)) .limit(1); + const row = match?.actionRequest; if (!row || row.status !== "executing") return row ?? null; - await new Promise((resolve) => setTimeout(resolve, 25)); + deadline = extendApprovedExecutionWaitDeadline({ + currentDeadlineMs: deadline, + invocationStatus: match.invocationStatus, + invocationStartedAt: match.invocationStartedAt, + preparationStartedAt: row.updatedAt, + preparationWaitMs: ACTION_REQUEST_PREPARATION_WAIT_MS, + executionWaitMs: ACTION_REQUEST_EXECUTION_WAIT_MS, + }); + const remainingMs = deadline - Date.now(); + if (remainingMs <= 0) break; + await new Promise((resolve) => setTimeout( + resolve, + Math.min(ACTION_REQUEST_EXECUTION_POLL_MS, remainingMs), + )); } throw new ToolGatewayHttpError(409, "Approved tool action is still executing", "action_execution_in_progress", { actionRequestId, + preparationWaitMs: ACTION_REQUEST_PREPARATION_WAIT_MS, + executionWaitMs: ACTION_REQUEST_EXECUTION_WAIT_MS, }); } @@ -4936,6 +4996,8 @@ export function createToolGatewayService( async function markApprovedActionFailed(input: { actionRequestId: string; invocationId: string; + claimUpdatedAt: Date; + expectedInvocationStatus: "awaiting_approval" | "executing"; error: unknown; }) { const reasonCode = input.error instanceof ToolGatewayHttpError @@ -4943,25 +5005,109 @@ export function createToolGatewayService( : "tool_execution_failed"; const message = input.error instanceof Error ? input.error.message : String(input.error); const now = new Date(); - await db.update(toolInvocations).set({ - status: "failed", - errorCode: reasonCode, - errorMessage: message, - completedAt: now, - updatedAt: now, - }).where(eq(toolInvocations.id, input.invocationId)); - await db.update(toolActionRequests).set({ - status: "failed", - resolvedAt: now, - updatedAt: now, - }).where(eq(toolActionRequests.id, input.actionRequestId)); + const settled = await db.transaction(async (tx) => { + // Lock in the same invocation -> request order used by the normal + // execution settlement path. The claim timestamp is the ownership token: + // a consumer that merely read an approved row cannot settle another + // consumer's claim, and a pre-dispatch failure cannot overwrite a call + // that has already entered provider execution or completed successfully. + const [invocation] = await tx + .select({ status: toolInvocations.status }) + .from(toolInvocations) + .where(eq(toolInvocations.id, input.invocationId)) + .for("update") + .limit(1); + if (invocation?.status !== input.expectedInvocationStatus) return false; + + const [actionRequest] = await tx + .select({ status: toolActionRequests.status, updatedAt: toolActionRequests.updatedAt }) + .from(toolActionRequests) + .where(eq(toolActionRequests.id, input.actionRequestId)) + .for("update") + .limit(1); + if ( + actionRequest?.status !== "executing" + || actionRequest.updatedAt.getTime() !== input.claimUpdatedAt.getTime() + ) { + return false; + } + + await tx.update(toolInvocations).set({ + status: "failed", + idempotencyKey: null, + errorCode: reasonCode, + errorMessage: message, + completedAt: now, + updatedAt: now, + }).where(and( + eq(toolInvocations.id, input.invocationId), + eq(toolInvocations.status, input.expectedInvocationStatus), + )); + await tx.update(toolActionRequests).set({ + status: "failed", + resolvedAt: now, + updatedAt: now, + }).where(and( + eq(toolActionRequests.id, input.actionRequestId), + eq(toolActionRequests.status, "executing"), + eq(toolActionRequests.updatedAt, input.claimUpdatedAt), + )); + return true; + }); + if (!settled) return { reasonCode, message, settled: false }; await reflectToolActionInteractionLifecycle({ actionRequestId: input.actionRequestId, status: "failed", errorCode: reasonCode, errorMessage: message, }); - return { reasonCode, message }; + return { reasonCode, message, settled: true }; + } + + async function expireApprovedActionForManagedArgumentDrift(input: { + actionRequestId: string; + invocationId: string; + toolName: string; + ownsExecutingClaim?: boolean; + }) { + const error = new ToolGatewayHttpError( + 409, + "Approved tool action managed arguments changed after review; request a new approval", + "approved_tool_managed_arguments_changed", + { actionRequestId: input.actionRequestId, invocationId: input.invocationId, tool: input.toolName }, + ); + const now = new Date(); + await options.beforeManagedArgumentDriftExpiry?.(); + const [expired] = await db + .update(toolActionRequests) + .set({ status: "expired", resolvedAt: now, updatedAt: now }) + .where(and( + eq(toolActionRequests.id, input.actionRequestId), + input.ownsExecutingClaim + ? inArray(toolActionRequests.status, ["approved", "executing"]) + : eq(toolActionRequests.status, "approved"), + )) + .returning({ id: toolActionRequests.id }); + if (!expired) return error; + await db + .update(toolInvocations) + .set({ + status: "failed", + approvalState: "expired", + idempotencyKey: null, + errorCode: error.reasonCode, + errorMessage: error.message, + completedAt: now, + updatedAt: now, + }) + .where(eq(toolInvocations.id, input.invocationId)); + await reflectToolActionInteractionLifecycle({ + actionRequestId: expired.id, + status: "expired", + errorCode: error.reasonCode, + errorMessage: error.message, + }); + return error; } // Guard for approved-action execution: the issue must still be open. Expires @@ -5046,15 +5192,29 @@ export function createToolGatewayService( }); if (!signedPayload) { const error = new ToolGatewayHttpError(409, "Approved tool action arguments signature is invalid", "signed_arguments_invalid"); - await markApprovedActionFailed({ actionRequestId: claimed.id, invocationId: invocation.id, error }); + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); throw error; } if (signedPayload.executionOnApprove !== true) { - throw new ToolGatewayHttpError( + const error = new ToolGatewayHttpError( 409, "This approval predates execute-on-approve and must remain inert", "legacy_approved_action_inert", ); + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); + throw error; } const session: ToolGatewaySession = { @@ -5079,12 +5239,24 @@ export function createToolGatewayService( tool = await findToolForSession(session, invocation.toolName); liveApprovalSnapshot = await connectedRemoteApprovalSnapshot(session, tool); } catch (error) { - await markApprovedActionFailed({ actionRequestId: claimed.id, invocationId: invocation.id, error }); + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); throw error; } if (!approvalSnapshotsMatch(signedPayload.approvalSnapshot, liveApprovalSnapshot)) { const error = new ToolGatewayHttpError(409, "Approved tool action target changed after review", "approved_tool_target_changed"); - await markApprovedActionFailed({ actionRequestId: claimed.id, invocationId: invocation.id, error }); + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); throw error; } const parameters = signedPayload.arguments; @@ -5102,9 +5274,36 @@ export function createToolGatewayService( }) ) { const error = new ToolGatewayHttpError(409, "Approved tool action arguments do not match reviewed hash", "signed_arguments_mismatch"); - await markApprovedActionFailed({ actionRequestId: claimed.id, invocationId: invocation.id, error }); + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); throw error; } + let managedArgumentsRemainCurrent: boolean; + try { + managedArgumentsRemainCurrent = await approvedManagedArgumentsRemainCurrent(session, tool, parameters); + } catch (error) { + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); + throw error; + } + if (!managedArgumentsRemainCurrent) { + throw await expireApprovedActionForManagedArgumentDrift({ + actionRequestId: claimed.id, + invocationId: invocation.id, + toolName: invocation.toolName, + ownsExecutingClaim: true, + }); + } const argumentsSummary = validateToolContent({ value: parameters, @@ -5179,6 +5378,8 @@ export function createToolGatewayService( const { reasonCode } = await markApprovedActionFailed({ actionRequestId: claimed.id, invocationId: invocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "executing", error, }); await writeToolCallEvent({ @@ -5298,13 +5499,6 @@ export function createToolGatewayService( ); } if (actionRequest.status === "approved" && actionRequest.decidedAt) { - const signedPayload = readSignedToolArgumentsPayload({ - signedArguments: actionRequest.signedArguments, - invocationId: invocation.id, - toolName: invocation.toolName, - signingSecret: options.toolActionSigningSecret, - }); - if (signedPayload?.executionOnApprove !== true) return null; const result = await executeApprovedAgentInvocation({ actionRequest, invocation }); return { matched: true as const, result, invocationId: invocation.id }; } @@ -5870,7 +6064,7 @@ export function createToolGatewayService( }); } - const requestedParameters = input.parameters ?? {}; + const requestedParameters = await governedToolArguments(session, tool, input.parameters ?? {}); const argumentValidation = validateToolContent({ value: requestedParameters, direction: "arguments", @@ -6356,6 +6550,15 @@ export function createToolGatewayService( requestedParameters = targetParameters; } + // Managed provider arguments are part of the governed call, not a + // transport decoration. Project them before hashing, policy evaluation, + // approval signing, previews, and audit summaries. Approved retries + // re-project only for a compatibility comparison and dispatch the + // already-reviewed signed payload unchanged. + if (!input.approvedActionRequestId) { + requestedParameters = await governedToolArguments(session, tool, requestedParameters); + } + const argumentValidation = validateToolContent({ value: requestedParameters, direction: "arguments", @@ -6526,6 +6729,43 @@ export function createToolGatewayService( if (!signedPayload) { throw new ToolGatewayHttpError(409, "Approved tool action arguments signature is invalid", "signed_arguments_invalid"); } + if (signedPayload.executionOnApprove !== true) { + const error = new ToolGatewayHttpError( + 409, + "This approval predates execute-on-approve and must remain inert", + "legacy_approved_action_inert", + ); + await options.beforeLegacyApprovedActionClaim?.(); + const claimedAt = new Date(); + const [claimed] = await db + .update(toolActionRequests) + .set({ + status: "executing", + resolvedByAgentId: session.agentId, + updatedAt: claimedAt, + }) + .where(and( + eq(toolActionRequests.id, actionRequest.id), + eq(toolActionRequests.status, "approved"), + )) + .returning(); + if (!claimed) { + throw new ToolGatewayHttpError( + 409, + "Tool action request was already consumed", + "action_already_consumed", + ); + } + await reflectToolActionInteractionLifecycle({ actionRequestId: claimed.id, status: "executing" }); + await markApprovedActionFailed({ + actionRequestId: claimed.id, + invocationId: storedInvocation.id, + claimUpdatedAt: claimed.updatedAt, + expectedInvocationStatus: "awaiting_approval", + error, + }); + throw error; + } const liveApprovalSnapshot = await connectedRemoteApprovalSnapshot(session, tool); if (!approvalSnapshotsMatch(signedPayload.approvalSnapshot, liveApprovalSnapshot)) { throw new ToolGatewayHttpError( @@ -6561,20 +6801,27 @@ export function createToolGatewayService( ) { throw new ToolGatewayHttpError(409, "Approved tool action arguments do not match reviewed hash", "signed_arguments_mismatch"); } - const [consumed] = await db + if (!await approvedManagedArgumentsRemainCurrent(session, tool, storedParameters)) { + throw await expireApprovedActionForManagedArgumentDrift({ + actionRequestId: actionRequest.id, + invocationId: storedInvocation.id, + toolName: storedInvocation.toolName, + }); + } + const claimedAt = new Date(); + const [claimed] = await db .update(toolActionRequests) .set({ - status: "executed", + status: "executing", resolvedByAgentId: session.agentId, - resolvedAt: new Date(), - updatedAt: new Date(), + updatedAt: claimedAt, }) .where(and(eq(toolActionRequests.id, actionRequest.id), eq(toolActionRequests.status, "approved"))) .returning(); - if (!consumed) { + if (!claimed) { throw new ToolGatewayHttpError(409, "Tool action request was already consumed", "action_already_consumed"); } - await reflectToolActionInteractionLifecycle({ actionRequestId: consumed.id, status: "executing" }); + await reflectToolActionInteractionLifecycle({ actionRequestId: claimed.id, status: "executing" }); invocationId = storedInvocation.id as typeof invocationId; effectiveParameters = storedParameters; effectiveArgumentsSummary = storedArgumentValidation.summary; @@ -6724,6 +6971,7 @@ export function createToolGatewayService( sensitiveMode: "redact", promptInjectionMode: "block", }); + const completedAt = new Date(); await db .update(toolInvocations) .set({ @@ -6731,15 +6979,25 @@ export function createToolGatewayService( resultHash: resultValidation.summary.sha256 ?? null, resultSummary: resultValidation.summary, resultSizeBytes: resultValidation.summary.sizeBytes ?? null, - completedAt: new Date(), - updatedAt: new Date(), + completedAt, + updatedAt: completedAt, }) .where(eq(toolInvocations.id, invocationId)); if (input.approvedActionRequestId) { - await reflectToolActionInteractionLifecycle({ - actionRequestId: input.approvedActionRequestId, - status: "executed", - }); + const [executedRequest] = await db + .update(toolActionRequests) + .set({ status: "executed", resolvedAt: completedAt, updatedAt: completedAt }) + .where(and( + eq(toolActionRequests.id, input.approvedActionRequestId), + eq(toolActionRequests.status, "executing"), + )) + .returning({ id: toolActionRequests.id }); + if (executedRequest) { + await reflectToolActionInteractionLifecycle({ + actionRequestId: executedRequest.id, + status: "executed", + }); + } } await writeToolCallEvent({ invocationId, @@ -6813,23 +7071,34 @@ export function createToolGatewayService( if (reasonCode === "elicitation_required") { throw normalizedError; } + const completedAt = new Date(); await db .update(toolInvocations) .set({ status: status === 504 ? "timed_out" : status === 429 ? "rate_limited" : "failed", errorCode: reasonCode, errorMessage: message, - completedAt: new Date(), - updatedAt: new Date(), + completedAt, + updatedAt: completedAt, }) .where(eq(toolInvocations.id, invocationId)); if (input.approvedActionRequestId) { - await reflectToolActionInteractionLifecycle({ - actionRequestId: input.approvedActionRequestId, - status: "failed", - errorCode: reasonCode, - errorMessage: message, - }); + const [failedRequest] = await db + .update(toolActionRequests) + .set({ status: "failed", resolvedAt: completedAt, updatedAt: completedAt }) + .where(and( + eq(toolActionRequests.id, input.approvedActionRequestId), + eq(toolActionRequests.status, "executing"), + )) + .returning({ id: toolActionRequests.id }); + if (failedRequest) { + await reflectToolActionInteractionLifecycle({ + actionRequestId: failedRequest.id, + status: "failed", + errorCode: reasonCode, + errorMessage: message, + }); + } } await writeToolCallEvent({ invocationId, diff --git a/ui/src/pages/apps/AppsConnect.test.tsx b/ui/src/pages/apps/AppsConnect.test.tsx index e61233f1a5..4738eb3025 100644 --- a/ui/src/pages/apps/AppsConnect.test.tsx +++ b/ui/src/pages/apps/AppsConnect.test.tsx @@ -28,6 +28,8 @@ const GITHUB = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "github")! const NOTION = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "notion")!; const ASANA = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "asana")!; const POSTHOG = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "posthog")!; +const POSTMAN = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "postman")!; +const SHOPIFY = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "shopify")!; const GOOGLE_SHEETS = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "google-sheets")!; const GOOGLE_DRIVE = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "google-drive")!; const GMAIL = CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === "gmail")!; @@ -495,6 +497,53 @@ describe("AppsConnect — Connect with a link (M4 frame)", () => { })); }); + it("submits the Postman access mode selected on the setup screen", async () => { + listGalleryMock.mockResolvedValue({ apps: [POSTMAN] }); + mockParams.appKey = "postman"; + connectAppMock.mockResolvedValueOnce({ + connectionId: "conn-postman", + application: { id: "app-postman", name: "Postman" }, + connection: { id: "conn-postman" }, + actions: { readOnly: [], canMakeChanges: [] }, + catalog: [], + suggestedDefaults: {}, + auth: { kind: "oauth", startUrl: "https://oauth.pstmn.io/authorize?state=opaque" }, + }); + + await render(); + await passAccessStep(); + + const full = radioContaining("Full"); + const code = radioContaining("Code"); + expect(full).toBeTruthy(); + expect(code).toBeTruthy(); + expect(full?.getAttribute("aria-checked")).toBe("true"); + await act(async () => { + code?.dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + await flushReact(); + expect(code?.getAttribute("aria-checked")).toBe("true"); + await act(async () => { + full?.dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + await flushReact(); + expect(full?.getAttribute("aria-checked")).toBe("true"); + expect(radioContaining("US · Browser sign-in")?.getAttribute("aria-checked")).toBe("true"); + + await act(async () => { + buttonByText("Continue to sign in")?.dispatchEvent(new MouseEvent("click", { bubbles: true })); + }); + await flushReact(); + + expect(connectAppMock).toHaveBeenCalledWith("company-1", expect.objectContaining({ + galleryKey: "postman", + connectionMethodKey: "mcp-oauth-full", + })); + expect(navigateTopLevelMock).toHaveBeenCalledWith( + "https://oauth.pstmn.io/authorize?state=opaque", + ); + }); + it("opens a manual OAuth app from the same source deep link Browse uses", async () => { listGalleryMock.mockResolvedValue({ apps: [ASANA] }); mockParams.appKey = undefined; @@ -542,6 +591,22 @@ describe("AppsConnect — Connect with a link (M4 frame)", () => { expect(container.textContent).toContain("Your OAuth app"); }); + it("explains Shopify's public-storefront gate before collecting the store domain", async () => { + mockParams.appKey = "shopify"; + listGalleryMock.mockResolvedValue({ apps: [SHOPIFY] }); + + await render(); + + expect(container.textContent).toContain("Launch the storefront before connecting"); + expect(container.textContent).toContain("Storefront visibility to Public"); + expect(container.textContent).toContain("private or password-protected storefront returns HTTP 401"); + expect( + Array.from(container.querySelectorAll("a")).find((link) => + link.textContent?.includes("Open Shopify Admin"), + )?.href, + ).toBe("https://admin.shopify.com/"); + }); + /** * Design §"Question 2": a member who may create a personal grant but cannot * configure a company-wide install sees **Any agent** disabled with the