From d5bb39651861b5b7490d13e347a6ffa9de9a5389 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Tue, 11 Aug 2026 19:10:45 -0700 Subject: [PATCH] fix: pass sandbox provider credential env vars to plugin workers; hide Local default under managed-sandbox-only (#11244) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments give each agent run an execution target, and sandbox providers (Daytona, E2B, Novita, exe.dev) run as plugin workers > - A managed deployment provisions one platform-managed sandbox row with no credential in config; the provider is documented to fall back to its process env var (for example `DAYTONA_API_KEY`) > - Plugin workers spawn with a scrubbed environment, so that fallback never sees the host env var — probe and lease acquisition fail with "require an API key in config or DAYTONA_API_KEY" even when the deployment sets the var > - Separately, the managed-sandbox-only mode hides local rows from every list, but the instance Default picker renders a hardcoded synthetic "Local" option that no filter touches > - This pull request forwards each bundled provider's documented credential env var to its own plugin worker, and gates the synthetic Local option on the flag > - The benefit is that the documented host-env credential fallback works for plugin-backed providers, and managed-sandbox-only instances no longer offer Local anywhere ## Linked Issues or Issue Description **Subsystem affected** Plugin worker environment construction (`server/src/services/plugin-loader.ts`) and the environments UI (instance Default picker, agent form inherited-environment label). **Problem or motivation** Two follow-ups to the managed-sandbox-only mode (#11200), both found on a live managed deployment: 1. The deployment sets `DAYTONA_API_KEY` as a server env var and the managed sandbox row omits `config.apiKey` by contract. "Test Connection" fails with `Sandbox environment probe failed for provider "daytona". Daytona sandbox environments require an API key in config or DAYTONA_API_KEY.` A real agent run fails the same way at lease acquisition. The cause: sandbox providers run as plugin workers, and `buildPluginWorkerEnv` passes only model-provider keys and in-cluster Kubernetes vars. The provider's own documented credential env var never reaches the worker, so the in-plugin `process.env` fallback reads nothing. The self-hosted path has the same gap: the Daytona plugin README documents `DAYTONA_API_KEY` as a host-level fallback, and it does not work today. 2. With `enableManagedSandboxOnly` on, the instance Default environment picker still shows "Local". The server filters local *rows* out of the list, and the client filter mirrors that for cached lists, but this option is a hardcoded `` — not a list row — so no filter removes it. Selecting it writes a null default, which run selection then rejects fail-closed. **Proposed solution** Forward each bundled sandbox provider's documented credential env var into its plugin worker, keyed by the manifest's declared `environmentDrivers[].driverKey` so a worker only receives its own provider's credential (daytona → `DAYTONA_API_KEY`, e2b → `E2B_API_KEY`, exe-dev → `EXE_API_KEY`, novita → `NOVITA_API_KEY`). Keep the existing gate: only plugins that declare `environment.drivers.register` receive any passthrough. In the UI, render the synthetic Local option only when managed-sandbox-only is off; under the flag show a disabled "Select environment" placeholder only while no default is stamped yet, and stop the agent form's inherited label from reading "Local". **Alternatives considered** Adding `DAYTONA_API_KEY` to the existing `ADAPTER_ENV_PASSTHROUGH` list was rejected: that list goes to every environment-driver plugin, so each provider would receive every other provider's credential. A manifest schema field for declared credential env vars was rejected as heavier than needed: the bundled providers are known, and the mapping lives next to the two existing passthrough lists. ## What Changed - `server/src/services/plugin-loader.ts`: new `SANDBOX_PROVIDER_CREDENTIAL_ENV_PASSTHROUGH` map (driverKey → documented credential env vars). `buildPluginWorkerEnv` reads the manifest's `environmentDrivers` and forwards only the matching vars, after the existing `environment.drivers.register` gate. Blank values stay excluded. - `server/src/__tests__/plugin-database.test.ts`: the daytona worker receives `DAYTONA_API_KEY` and not another provider's key; a plugin whose drivers have no mapping (kubernetes) receives no credential var. - `ui/src/pages/CompanyEnvironments.tsx`: the Default picker's synthetic Local option renders only when managed-sandbox-only is off. Under the flag, a disabled "Select environment" placeholder renders only while the default is unset. - `ui/src/pages/CompanyEnvironments.test.tsx`: the Local option is present by default and absent under the flag; saved non-local environments stay selectable. - `ui/src/components/AgentConfigForm.tsx`: the inherited-environment label falls back to "Managed sandbox" instead of "Local" under the flag. ## Verification - `server`: `npx vitest run src/__tests__/plugin-database.test.ts -t buildPluginWorkerEnv` — 5 passed (3 existing, 2 new). - `ui`: `npx vitest run src/pages/CompanyEnvironments.test.tsx` — 22 passed (2 new); `npx vitest run src/components/AgentConfigForm.render.test.tsx` — 10 passed. - `tsc --noEmit` clean in `server` and `ui`. - Live managed deployment: confirmed the tenant service env carries `DAYTONA_API_KEY` while the probe fails with the exact message above, which pins the root cause to the worker env, not delivery. ## Risks - The worker env grows by exactly one var per matching bundled provider, only when the deployment sets it and only for plugins that declare a matching environment driver. Plugins without a mapping see no change. - Self-hosted behavioral shift is the fix itself: a host-level `DAYTONA_API_KEY` (or E2B/EXE/NOVITA equivalent) now reaches the provider as its README documents. Deployments that set the var but expected it to stay inert had no working configuration to preserve — the provider errored on every keyless probe and run. - UI change is inert unless `enableManagedSandboxOnly` is on (default false everywhere). ## Model Used Claude Fable 5 (`claude-fable-5`) via Claude Code — extended thinking, tool use, parallel read-only subagents for the two root-cause traces. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- server/src/__tests__/plugin-database.test.ts | 104 +++++++++++++++++++ server/src/services/plugin-loader.ts | 62 ++++++++++- ui/src/components/AgentConfigForm.tsx | 5 +- ui/src/pages/CompanyEnvironments.test.tsx | 32 ++++++ ui/src/pages/CompanyEnvironments.tsx | 14 ++- 5 files changed, 212 insertions(+), 5 deletions(-) diff --git a/server/src/__tests__/plugin-database.test.ts b/server/src/__tests__/plugin-database.test.ts index 98753b41d1..788c6351aa 100644 --- a/server/src/__tests__/plugin-database.test.ts +++ b/server/src/__tests__/plugin-database.test.ts @@ -202,6 +202,110 @@ describe("buildPluginWorkerEnv", () => { PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", }); }); + + it("passes a first-party sandbox provider's documented credential env var to its own worker", () => { + const env = buildPluginWorkerEnv({ + manifest: { + capabilities: ["environment.drivers.register"], + environmentDrivers: [{ driverKey: "daytona" }], + }, + packageName: "@paperclipai/plugin-daytona", + packagePath: null, + instanceInfo, + processEnv: { + DAYTONA_API_KEY: "daytona-token", + NOVITA_API_KEY: "novita-token", + E2B_API_KEY: " ", + }, + }); + + expect(env).toEqual({ + PAPERCLIP_DEPLOYMENT_MODE: "authenticated", + PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", + DAYTONA_API_KEY: "daytona-token", + }); + }); + + it("passes the credential to a first-party plugin installed from the bundled catalog", () => { + const env = buildPluginWorkerEnv({ + manifest: { + capabilities: ["environment.drivers.register"], + environmentDrivers: [{ driverKey: "daytona" }], + }, + packageName: "@paperclipai/plugin-daytona", + packagePath: "/app/packages/plugins/sandbox-providers/daytona", + trustedLocalPluginRoots: ["/app/packages/plugins"], + instanceInfo, + processEnv: { + DAYTONA_API_KEY: "daytona-token", + }, + }); + + expect(env).toEqual({ + PAPERCLIP_DEPLOYMENT_MODE: "authenticated", + PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", + DAYTONA_API_KEY: "daytona-token", + }); + }); + + it("does not pass the credential to a local plugin that self-declares the first-party name", () => { + const env = buildPluginWorkerEnv({ + manifest: { + capabilities: ["environment.drivers.register"], + environmentDrivers: [{ driverKey: "daytona" }], + }, + packageName: "@paperclipai/plugin-daytona", + packagePath: "/home/operator/.paperclip/plugins/fake-daytona", + trustedLocalPluginRoots: ["/app/packages/plugins"], + instanceInfo, + processEnv: { + DAYTONA_API_KEY: "daytona-token", + }, + }); + + expect(env).toEqual({ + PAPERCLIP_DEPLOYMENT_MODE: "authenticated", + PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", + }); + }); + + it("does not pass a credential to a third-party plugin that claims a first-party driver key", () => { + const env = buildPluginWorkerEnv({ + manifest: { + capabilities: ["environment.drivers.register"], + environmentDrivers: [{ driverKey: "daytona" }], + }, + packageName: "@acme/plugin-fake-daytona", + instanceInfo, + processEnv: { + DAYTONA_API_KEY: "daytona-token", + }, + }); + + expect(env).toEqual({ + PAPERCLIP_DEPLOYMENT_MODE: "authenticated", + PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", + }); + }); + + it("does not pass a credential when the first-party package omits its expected driver key", () => { + const env = buildPluginWorkerEnv({ + manifest: { + capabilities: ["environment.drivers.register"], + environmentDrivers: [{ driverKey: "kubernetes" }], + }, + packageName: "@paperclipai/plugin-daytona", + instanceInfo, + processEnv: { + DAYTONA_API_KEY: "daytona-token", + }, + }); + + expect(env).toEqual({ + PAPERCLIP_DEPLOYMENT_MODE: "authenticated", + PAPERCLIP_DEPLOYMENT_EXPOSURE: "public", + }); + }); }); describeEmbeddedPostgres("plugin database namespaces", () => { diff --git a/server/src/services/plugin-loader.ts b/server/src/services/plugin-loader.ts index b2079a93e4..7e24046e38 100644 --- a/server/src/services/plugin-loader.ts +++ b/server/src/services/plugin-loader.ts @@ -50,6 +50,7 @@ import type { PluginJobStore } from "./plugin-job-store.js"; import type { PluginToolDispatcher } from "./plugin-tool-dispatcher.js"; import type { PluginLifecycleManager } from "./plugin-lifecycle.js"; import { pluginDatabaseService } from "./plugin-database.js"; +import { resolveBundledCatalogRoot } from "./bundled-plugins.js"; const execFileAsync = promisify(execFile); const __dirname = path.dirname(fileURLToPath(import.meta.url)); @@ -118,8 +119,43 @@ const K8S_IN_CLUSTER_ENV_PASSTHROUGH = [ "KUBERNETES_SERVICE_PORT_HTTPS", ]; +/** + * Each first-party sandbox provider's documented credential fallback env + * var. Environment rows may omit `config.apiKey` (managed/platform- + * provisioned rows always do — see `managed-environments.ts`), in which + * case the provider reads its documented process env var. That fallback + * executes inside the plugin worker, whose environment is scrubbed, so + * the deployment-level var must be forwarded explicitly. + * + * Keyed by the installed npm package name and cross-checked against the + * manifest's declared driver key — but name and manifest are both + * plugin-authored, so neither is proof of identity on its own. The gate + * therefore also requires a trusted install origin: a registry install + * (`packagePath` null — the `@paperclipai` scope is project-controlled at + * the registry), or a local path inside the repo/bundled plugin catalog, + * which ships inside the release image and is as trusted as the server + * code itself. An operator-added local plugin directory can claim any + * name and driver key and still receives nothing. + */ +const SANDBOX_PROVIDER_CREDENTIAL_ENV_PASSTHROUGH: Record< + string, + { driverKey: string; envVars: readonly string[] } +> = { + "@paperclipai/plugin-daytona": { driverKey: "daytona", envVars: ["DAYTONA_API_KEY"] }, + "@paperclipai/plugin-e2b": { driverKey: "e2b", envVars: ["E2B_API_KEY"] }, + "@paperclipai/plugin-exe-dev": { driverKey: "exe-dev", envVars: ["EXE_API_KEY"] }, + "@paperclipai/plugin-novita-sandbox": { driverKey: "novita", envVars: ["NOVITA_API_KEY"] }, +}; + export function buildPluginWorkerEnv(input: { - manifest: Pick; + manifest: Pick & { + environmentDrivers?: ReadonlyArray<{ driverKey: string }>; + }; + packageName?: string; + /** Local install path (`PluginRecord.packagePath`); null for registry installs. */ + packagePath?: string | null; + /** Test seam; defaults to the repo plugin tree and the bundled catalog root. */ + trustedLocalPluginRoots?: readonly string[]; instanceInfo: { deploymentMode?: string | null; deploymentExposure?: string | null }; processEnv?: NodeJS.ProcessEnv; }): Record { @@ -132,7 +168,22 @@ export function buildPluginWorkerEnv(input: { && input.manifest.capabilities.includes("environment.drivers.register"); if (!canRegisterEnvironmentDrivers) return env; - for (const key of [...ADAPTER_ENV_PASSTHROUGH, ...K8S_IN_CLUSTER_ENV_PASSTHROUGH]) { + const trustedLocalRoots = input.trustedLocalPluginRoots + ?? [BUNDLED_LOCAL_PLUGIN_ROOT, resolveBundledCatalogRoot(processEnv)]; + const installOriginTrusted = + input.packagePath == null + || trustedLocalRoots.some((root) => isPathWithin(root, path.resolve(input.packagePath as string))); + const credentialEntry = installOriginTrusted && input.packageName + ? SANDBOX_PROVIDER_CREDENTIAL_ENV_PASSTHROUGH[input.packageName] + : undefined; + const credentialKeys = + credentialEntry + && (input.manifest.environmentDrivers ?? []).some( + (driver) => driver.driverKey === credentialEntry.driverKey, + ) + ? credentialEntry.envVars + : []; + for (const key of [...ADAPTER_ENV_PASSTHROUGH, ...K8S_IN_CLUSTER_ENV_PASSTHROUGH, ...credentialKeys]) { const value = processEnv[key]; if (value && value.trim().length > 0) { env[key] = value; @@ -2222,7 +2273,12 @@ export function pluginLoader( databaseNamespace, hostHandlers, autoRestart: true, - env: buildPluginWorkerEnv({ manifest, instanceInfo }), + env: buildPluginWorkerEnv({ + manifest, + packageName: activePlugin.packageName, + packagePath: activePlugin.packagePath, + instanceInfo, + }), // Authorize the worker to act on each configured company from its // proactive loops/timers (LOOA-629). Seeded here so it is in place // before any setup()-time worker→host call (LOOA-695). The authorized diff --git a/ui/src/components/AgentConfigForm.tsx b/ui/src/components/AgentConfigForm.tsx index 44b79ccfeb..8b24f2eea7 100644 --- a/ui/src/components/AgentConfigForm.tsx +++ b/ui/src/components/AgentConfigForm.tsx @@ -490,9 +490,12 @@ export function AgentConfigForm(props: AgentConfigFormProps) { currentDefaultEnvironmentId.length > 0 || runnableEnvironments.length >= 1 ); + const managedSandboxOnly = experimentalSettings?.enableManagedSandboxOnly === true; const inheritedEnvironmentLabel = instanceDefaultEnvironment ? `${instanceDefaultEnvironment.name} (${instanceDefaultEnvironment.driver})` - : "Local"; + : managedSandboxOnly + ? "Managed sandbox" + : "Local"; // Fetch adapter models for the effective adapter type const modelQueryKey = selectedCompanyId diff --git a/ui/src/pages/CompanyEnvironments.test.tsx b/ui/src/pages/CompanyEnvironments.test.tsx index 3d0d8e3294..b0d5c10ff9 100644 --- a/ui/src/pages/CompanyEnvironments.test.tsx +++ b/ui/src/pages/CompanyEnvironments.test.tsx @@ -1387,4 +1387,36 @@ describe("CompanyEnvironments — test provider button", () => { expect(mockEnvironmentsApi.disableCustomImageTemplate).toHaveBeenCalledExactlyOnceWith("env-1", "company-1"); }); }); + + it("offers the implicit Local option in the default picker by default", async () => { + root = createRoot(container); + const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + + await act(async () => { + root!.render(renderCompanyEnvironments(queryClient)); + }); + await flushReact(); + + const options = Array.from(container.querySelectorAll("option")); + expect(options.some((option) => option.textContent?.trim() === "Local")).toBe(true); + }); + + it("hides the implicit Local option in the default picker under managed-sandbox-only", async () => { + mockInstanceSettingsApi.getExperimental.mockResolvedValue({ + enableEnvironments: true, + enableManagedSandboxOnly: true, + }); + root = createRoot(container); + const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + + await act(async () => { + root!.render(renderCompanyEnvironments(queryClient)); + }); + await flushReact(); + + const options = Array.from(container.querySelectorAll("option")); + expect(options.some((option) => option.textContent?.trim() === "Local")).toBe(false); + // Saved non-local environments remain selectable defaults. + expect(options.some((option) => option.textContent?.includes("Alpha"))).toBe(true); + }); }); diff --git a/ui/src/pages/CompanyEnvironments.tsx b/ui/src/pages/CompanyEnvironments.tsx index 59c2a5c692..2c337bf188 100644 --- a/ui/src/pages/CompanyEnvironments.tsx +++ b/ui/src/pages/CompanyEnvironments.tsx @@ -1190,6 +1190,7 @@ export function CompanyEnvironments({ mode = "list" }: CompanyEnvironmentsProps) retry: false, }); const environmentsEnabled = experimentalSettings?.enableEnvironments === true; + const managedSandboxOnly = experimentalSettings?.enableManagedSandboxOnly === true; const { data: environments } = useQuery({ queryKey: selectedCompanyId ? queryKeys.environments.list(selectedCompanyId) : ["environments", "none"], @@ -1660,7 +1661,18 @@ export function CompanyEnvironments({ mode = "list" }: CompanyEnvironmentsProps) defaultEnvironmentMutation.mutate(event.target.value || null)} disabled={defaultEnvironmentMutation.isPending} > - + {managedSandboxOnly ? ( + // Managed-sandbox-only instances never execute locally, so + // the implicit local fallback is not a legal default. The + // placeholder only renders while no default is stamped yet. + instanceDefaultEnvironmentId === "" ? ( + + ) : null + ) : ( + + )} {nonLocalEnvironments.map((environment) => (