fix(server): accept secret_ref binding objects in sandbox provider environment config (#10355)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents execute in environments; sandbox provider plugins (Daytona,
Modal, e2b, …) declare their config via a JSON schema, with credentials
marked `format: "secret-ref"`
> - The environments UI renders those fields with a secret picker that
submits `{ type: "secret_ref", secretId, version }` binding objects,
while the server-side environment config paths only understood raw
string values and bare secret-id strings
> - The binding object reached the plugin worker's
`environmentValidateConfig` untouched; plugins parse non-string config
values as absent, so saving or testing an environment with a
picker-bound secret always failed validation (e.g. "Daytona sandbox
environments require an API key in config or DAYTONA_API_KEY.", "Modal
sandbox environments require tokenId and tokenSecret.")
> - Worse, an environment first saved with raw pasted values becomes
uneditable: the stored value is a secret reference, the edit form
re-submits it as a binding object, and every subsequent save fails the
same way
> - This pull request canonicalizes binding objects to the bare secret
id before plugin validation, and teaches the persistence/runtime/probe
secret-ref resolvers to accept the object shape defensively
> - The benefit is that picker-bound secrets work for every
schema-driven sandbox provider — create, edit, and Test — with no plugin
changes required

## Linked Issues or Issue Description

Fixes #10105

The same failure reproduces with the Daytona provider: Settings →
Instance settings → Environments → New, driver sandbox, provider
daytona, bind Api Key to an existing secret via the picker → Save fails
with "Daytona sandbox environments require an API key in config or
DAYTONA_API_KEY."

## What Changed

- `server/src/services/json-schema-secret-refs.ts`: new
`parseSecretRefBindingObject()` that recognizes the `{ type:
"secret_ref", secretId, version? }` shape the secret picker submits
(version defaults to `"latest"`; malformed objects return null).
- `server/src/services/plugin-environment-driver.ts`:
`validatePluginSandboxProviderConfig()` now canonicalizes binding
objects at the driver schema's `format: "secret-ref"` paths to the bare
secret id (the persisted shape) before invoking the plugin worker's
`environmentValidateConfig`. Pinned numeric versions are rejected with a
clear 422, since sandbox provider references always resolve the latest
version — silently resolving a different version would be worse.
- `server/src/services/environment-config.ts`: the persistence, runtime,
and probe secret-ref resolvers plus `collectEnvironmentSecretRefs()`
accept the binding-object shape defensively, so any previously persisted
object-shaped refs (from providers whose validation tolerated them)
resolve instead of being silently skipped; the missing-companyId runtime
guard also now fails closed for object-shaped refs.

## Verification

- `npx vitest run server/src/__tests__/json-schema-secret-refs.test.ts
server/src/__tests__/plugin-sandbox-provider-config-validation.test.ts
server/src/__tests__/environment-routes.test.ts
server/src/__tests__/environment-config.test.ts` — 82 tests pass,
including new coverage: binding-object canonicalization before plugin
validation, pinned-version rejection, raw-string pass-through, and a
route-level create with a picker-submitted binding object persisting the
bare secret id without minting a duplicate secret.
- `npx vitest run server/src/__tests__/environment-runtime.test.ts` — 24
tests pass against embedded Postgres, including a new test that persists
an object-shaped ref and verifies runtime resolution produces the
plaintext credential for the plugin worker.
- `pnpm typecheck` in `server/` — clean.

## Risks

- Low. The canonical persisted shape (bare secret-id string) is
unchanged, so existing saved environments and lease-resume fingerprints
are unaffected; raw pasted values and bare-id strings take exactly the
same code path as before.
- New behavior only triggers where a save/probe previously failed 422
(binding objects at secret-ref paths) or where an object-shaped ref was
previously skipped silently at runtime (now resolved, or failed closed
without a companyId).
- Pinned binding versions at sandbox-provider paths are now an explicit
422 instead of an accidental validation failure; no UI submits pinned
versions today (`allowVersionSelector={false}`).

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) with extended thinking and
agentic tool use (Claude Code harness): source diagnosis, fix, and
tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
doc surface changed)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Devin Foley 2026-07-28 10:50:13 -07:00 committed by GitHub
parent f9034ab3ca
commit 9f5af4ea5d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 392 additions and 7 deletions

View File

@ -1598,6 +1598,79 @@ describe("environment routes", () => {
);
});
it("persists a picker-submitted secret_ref binding object as the bare secret id", async () => {
const secretId = "11111111-1111-1111-1111-111111111111";
const environment = {
...createEnvironment(),
id: "env-sandbox-secure-plugin",
name: "Secure Sandbox",
driver: "sandbox" as const,
config: {
provider: "secure-plugin",
template: "base",
apiKey: secretId,
timeoutMs: 450000,
reuseLease: true,
},
};
mockEnvironmentService.create.mockResolvedValue(environment);
mockValidatePluginSandboxProviderConfig.mockImplementation(async ({ config }: { config: Record<string, unknown> }) => ({
normalizedConfig: { ...config },
pluginId: "plugin-secure",
pluginKey: "acme.secure-sandbox-provider",
driver: {
driverKey: "secure-plugin",
kind: "sandbox_provider",
displayName: "Secure Sandbox",
configSchema: {
type: "object",
properties: {
template: { type: "string" },
apiKey: { type: "string", format: "secret-ref" },
timeoutMs: { type: "number" },
reuseLease: { type: "boolean" },
},
},
},
}));
const pluginWorkerManager = {};
const app = createApp({
type: "board",
userId: "user-1",
source: "local_implicit",
}, { pluginWorkerManager });
const res = await request(app)
.post("/api/companies/company-1/environments")
.send({
name: "Secure Sandbox",
driver: "sandbox",
config: {
provider: "secure-plugin",
template: "base",
apiKey: { type: "secret_ref", secretId, version: "latest" },
timeoutMs: 450000,
reuseLease: true,
},
});
expect(res.status).toBe(201);
expect(mockEnvironmentService.create).toHaveBeenCalledWith({
name: "Secure Sandbox",
driver: "sandbox",
status: "active",
config: {
provider: "secure-plugin",
template: "base",
apiKey: secretId,
timeoutMs: 450000,
reuseLease: true,
},
envVars: {},
});
expect(mockSecretService.create).not.toHaveBeenCalled();
});
it("uses the configured provider for schema-driven sandbox secret fields", async () => {
process.env.PAPERCLIP_SECRETS_PROVIDER = "aws_secrets_manager";
const environment = {

View File

@ -27,6 +27,7 @@ import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { resolveEnvironmentDriverConfigForRuntime } from "../services/environment-config.ts";
import { environmentRuntimeService, findReusableSandboxLeaseId } from "../services/environment-runtime.ts";
import { environmentService } from "../services/environments.ts";
import { secretService } from "../services/secrets.ts";
@ -849,6 +850,88 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
}), 31234);
});
it("resolves persisted secret_ref binding objects in sandbox provider config at runtime", async () => {
const pluginId = randomUUID();
const { companyId, environment: baseEnvironment } = await seedEnvironment();
const apiSecret = await secretService(db).create(companyId, {
name: `secure-plugin-api-key-${randomUUID()}`,
provider: "local_encrypted",
value: "resolved-provider-key",
});
const providerConfig = {
provider: "secure-plugin",
template: "base",
apiKey: { type: "secret_ref", secretId: apiSecret.id, version: "latest" },
timeoutMs: 1234,
reuseLease: false,
};
const environment = {
...baseEnvironment,
name: "Secure Plugin Sandbox",
driver: "sandbox",
config: providerConfig,
};
await secretService(db).createBinding({
companyId,
secretId: apiSecret.id,
targetType: "environment",
targetId: environment.id,
configPath: "apiKey",
});
await environmentService(db).update(environment.id, {
driver: "sandbox",
name: environment.name,
config: providerConfig,
});
await db.insert(plugins).values({
id: pluginId,
pluginKey: "acme.secure-sandbox-provider",
packageName: "@acme/secure-sandbox-provider",
version: "1.0.0",
apiVersion: 1,
categories: ["automation"],
manifestJson: {
id: "acme.secure-sandbox-provider",
apiVersion: 1,
version: "1.0.0",
displayName: "Secure Sandbox Provider",
description: "Test schema-driven provider",
author: "Paperclip",
categories: ["automation"],
capabilities: ["environment.drivers.register"],
entrypoints: { worker: "dist/worker.js" },
environmentDrivers: [
{
driverKey: "secure-plugin",
kind: "sandbox_provider",
displayName: "Secure Sandbox",
configSchema: {
type: "object",
properties: {
template: { type: "string" },
apiKey: { type: "string", format: "secret-ref" },
timeoutMs: { type: "number" },
reuseLease: { type: "boolean" },
},
},
},
],
},
status: "ready",
installOrder: 1,
updatedAt: new Date(),
} as any);
const resolved = await resolveEnvironmentDriverConfigForRuntime(db, companyId, environment);
expect(resolved.driver).toBe("sandbox");
expect(resolved.config).toMatchObject({
provider: "secure-plugin",
template: "base",
apiKey: "resolved-provider-key",
});
});
it("waits briefly for a ready sandbox provider plugin worker to come online", async () => {
const pluginId = randomUUID();
const { companyId, environment: baseEnvironment, runId } = await seedEnvironment();

View File

@ -1,5 +1,38 @@
import { describe, expect, it } from "vitest";
import { collectSecretRefPaths } from "../services/json-schema-secret-refs.ts";
import { collectSecretRefPaths, parseSecretRefBindingObject } from "../services/json-schema-secret-refs.ts";
describe("parseSecretRefBindingObject", () => {
const secretId = "11111111-1111-1111-1111-111111111111";
it("parses a binding object and defaults the version to latest", () => {
expect(parseSecretRefBindingObject({ type: "secret_ref", secretId })).toEqual({
secretId,
version: "latest",
});
expect(parseSecretRefBindingObject({ type: "secret_ref", secretId, version: "latest" })).toEqual({
secretId,
version: "latest",
});
});
it("parses a pinned numeric version", () => {
expect(parseSecretRefBindingObject({ type: "secret_ref", secretId, version: 3 })).toEqual({
secretId,
version: 3,
});
});
it("rejects non-binding values", () => {
expect(parseSecretRefBindingObject(secretId)).toBeNull();
expect(parseSecretRefBindingObject("raw-api-key")).toBeNull();
expect(parseSecretRefBindingObject(null)).toBeNull();
expect(parseSecretRefBindingObject([{ type: "secret_ref", secretId }])).toBeNull();
expect(parseSecretRefBindingObject({ type: "user_secret_ref", secretId })).toBeNull();
expect(parseSecretRefBindingObject({ type: "secret_ref", secretId: "not-a-uuid" })).toBeNull();
expect(parseSecretRefBindingObject({ type: "secret_ref", secretId, version: 0 })).toBeNull();
expect(parseSecretRefBindingObject({ type: "secret_ref", secretId, version: "2" })).toBeNull();
});
});
describe("collectSecretRefPaths", () => {
it("collects nested secret-ref paths from object properties", () => {

View File

@ -0,0 +1,120 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { validatePluginSandboxProviderConfig } from "../services/plugin-environment-driver.ts";
import type { PluginWorkerManager } from "../services/plugin-worker-manager.ts";
import type { Db } from "@paperclipai/db";
const mockList = vi.fn();
vi.mock("../services/plugin-registry.js", () => ({
pluginRegistryService: () => ({
list: mockList,
}),
}));
const PLUGIN_ID = "22222222-2222-2222-2222-222222222222";
const SECRET_ID = "11111111-1111-1111-1111-111111111111";
function seedProviderPlugin() {
mockList.mockResolvedValue([
{
id: PLUGIN_ID,
pluginKey: "acme.secure-sandbox-provider",
status: "ready",
manifestJson: {
environmentDrivers: [
{
driverKey: "secure-plugin",
kind: "sandbox_provider",
displayName: "Secure Sandbox",
configSchema: {
type: "object",
properties: {
template: { type: "string" },
apiKey: { type: "string", format: "secret-ref" },
timeoutMs: { type: "number" },
},
},
},
],
},
},
]);
}
function createWorkerManager() {
return {
isRunning: vi.fn(() => true),
call: vi.fn(async (_pluginId: string, _method: string, params: { config: Record<string, unknown> }) => ({
ok: true,
normalizedConfig: { ...params.config },
})),
} as unknown as PluginWorkerManager & { call: ReturnType<typeof vi.fn> };
}
describe("validatePluginSandboxProviderConfig secret-ref bindings", () => {
beforeEach(() => {
vi.clearAllMocks();
seedProviderPlugin();
});
it("canonicalizes secret_ref binding objects to bare secret ids before the plugin validates", async () => {
const workerManager = createWorkerManager();
const result = await validatePluginSandboxProviderConfig({
db: {} as Db,
workerManager,
provider: "secure-plugin",
config: {
template: "base",
apiKey: { type: "secret_ref", secretId: SECRET_ID, version: "latest" },
timeoutMs: 1234,
},
});
expect(workerManager.call).toHaveBeenCalledWith(PLUGIN_ID, "environmentValidateConfig", {
driverKey: "secure-plugin",
config: {
template: "base",
apiKey: SECRET_ID,
timeoutMs: 1234,
},
});
expect(result.normalizedConfig.apiKey).toBe(SECRET_ID);
});
it("rejects pinned secret binding versions before calling the plugin", async () => {
const workerManager = createWorkerManager();
await expect(validatePluginSandboxProviderConfig({
db: {} as Db,
workerManager,
provider: "secure-plugin",
config: {
apiKey: { type: "secret_ref", secretId: SECRET_ID, version: 3 },
},
})).rejects.toThrow(/pins version 3/);
expect(workerManager.call).not.toHaveBeenCalled();
});
it("passes raw strings and bare secret ids through untouched", async () => {
const workerManager = createWorkerManager();
await validatePluginSandboxProviderConfig({
db: {} as Db,
workerManager,
provider: "secure-plugin",
config: {
template: "base",
apiKey: "raw-provider-key",
},
});
expect(workerManager.call).toHaveBeenCalledWith(PLUGIN_ID, "environmentValidateConfig", {
driverKey: "secure-plugin",
config: {
template: "base",
apiKey: "raw-provider-key",
},
});
});
});

View File

@ -25,6 +25,7 @@ import type { PluginWorkerManager } from "./plugin-worker-manager.js";
import {
collectSecretRefPaths,
isUuidSecretRef,
parseSecretRefBindingObject,
readConfigValueAtPath,
writeConfigValueAtPath,
} from "./json-schema-secret-refs.js";
@ -200,6 +201,25 @@ async function createEnvironmentSecret(input: {
};
}
/**
* Secret pickers submit `{ type: "secret_ref", secretId, version }` binding
* objects for `format: "secret-ref"` fields, while persisted configs store the
* bare secret id. Collapse binding objects to the secret id so every consumer
* downstream deals with one shape. Sandbox provider references always resolve
* the latest version, so pinned bindings are rejected rather than silently
* resolved to a different version than the caller asked for.
*/
function canonicalizeSecretRefValue(value: unknown, path: string): unknown {
const binding = parseSecretRefBindingObject(value);
if (!binding) return value;
if (binding.version !== "latest") {
throw unprocessable(
`Secret binding at ${path} pins version ${binding.version}; sandbox provider secret references always resolve the latest version.`,
);
}
return binding.secretId;
}
async function persistConfigSecretRefs(input: {
db: Db;
companyId: string;
@ -212,7 +232,7 @@ async function persistConfigSecretRefs(input: {
}): Promise<Record<string, unknown>> {
let nextConfig = { ...input.config };
for (const path of collectSecretRefPaths(input.schema)) {
const rawValue = readConfigValueAtPath(nextConfig, path);
const rawValue = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof rawValue !== "string") continue;
const trimmed = rawValue.trim();
if (trimmed.length === 0) {
@ -252,7 +272,7 @@ async function resolveConfigSecretRefsForRuntime(input: {
const secrets = secretService(input.db);
let nextConfig = { ...input.config };
for (const path of collectSecretRefPaths(input.schema)) {
const current = readConfigValueAtPath(nextConfig, path);
const current = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof current !== "string") continue;
const trimmed = current.trim();
if (!isUuidSecretRef(trimmed)) continue;
@ -291,7 +311,7 @@ async function resolveConfigSecretRefsForProbe(input: {
const secrets = secretService(input.db);
let nextConfig = { ...input.config };
for (const path of collectSecretRefPaths(input.schema)) {
const current = readConfigValueAtPath(nextConfig, path);
const current = canonicalizeSecretRefValue(readConfigValueAtPath(nextConfig, path), path);
if (typeof current !== "string") continue;
const trimmed = current.trim();
if (!isUuidSecretRef(trimmed)) continue;
@ -332,6 +352,11 @@ export async function collectEnvironmentSecretRefs(input: {
const refs: Array<{ secretId: string; configPath: string; versionSelector?: SecretVersionSelector }> = [];
for (const path of collectSecretRefPaths(schema)) {
const current = readConfigValueAtPath(parsed.config as Record<string, unknown>, path);
const binding = parseSecretRefBindingObject(current);
if (binding) {
refs.push({ secretId: binding.secretId, configPath: path, versionSelector: binding.version });
continue;
}
if (typeof current === "string" && isUuidSecretRef(current.trim())) {
refs.push({ secretId: current.trim(), configPath: path, versionSelector: "latest" });
}
@ -623,7 +648,7 @@ export async function resolveEnvironmentDriverConfigForRuntime(
} else {
for (const path of collectSecretRefPaths(schema)) {
const current = readConfigValueAtPath(parsed.config as Record<string, unknown>, path);
if (typeof current === "string" && isUuidSecretRef(current.trim())) {
if (parseSecretRefBindingObject(current) || (typeof current === "string" && isUuidSecretRef(current.trim()))) {
throw unprocessable("Runtime secret resolution requires a companyId context");
}
}

View File

@ -5,6 +5,31 @@ export function isUuidSecretRef(value: string): boolean {
return UUID_RE.test(value);
}
export type SecretRefBindingObject = {
secretId: string;
version: "latest" | number;
};
/**
* Parses the `{ type: "secret_ref", secretId, version? }` binding object that
* secret pickers submit for `format: "secret-ref"` config fields. Returns null
* for anything else (raw values, bare secret-id strings, malformed objects).
*/
export function parseSecretRefBindingObject(value: unknown): SecretRefBindingObject | null {
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
const record = value as Record<string, unknown>;
if (record.type !== "secret_ref") return null;
if (typeof record.secretId !== "string" || !isUuidSecretRef(record.secretId.trim())) return null;
const version = record.version;
if (version === undefined || version === null || version === "latest") {
return { secretId: record.secretId.trim(), version: "latest" };
}
if (typeof version === "number" && Number.isInteger(version) && version > 0) {
return { secretId: record.secretId.trim(), version };
}
return null;
}
export function collectSecretRefPaths(
schema: Record<string, unknown> | null | undefined,
): Set<string> {

View File

@ -21,6 +21,12 @@ import type {
PluginEnvironmentRealizeWorkspaceResult,
} from "@paperclipai/plugin-sdk";
import { unprocessable } from "../errors.js";
import {
collectSecretRefPaths,
parseSecretRefBindingObject,
readConfigValueAtPath,
writeConfigValueAtPath,
} from "./json-schema-secret-refs.js";
import { pluginRegistryService } from "./plugin-registry.js";
import type { PluginWorkerManager } from "./plugin-worker-manager.js";
@ -135,9 +141,29 @@ export async function validatePluginSandboxProviderConfig(input: {
throw unprocessable(`Sandbox provider "${input.provider}" is not installed or its plugin worker is not running.`);
}
// Secret pickers submit `{ type: "secret_ref", secretId, version }` binding
// objects for `format: "secret-ref"` fields. Plugins only understand string
// config values, so canonicalize bindings to the bare secret id (the
// persisted shape) before the plugin validates.
const configSchema =
resolved.driver.configSchema && typeof resolved.driver.configSchema === "object" && !Array.isArray(resolved.driver.configSchema)
? resolved.driver.configSchema as Record<string, unknown>
: null;
let config = input.config;
for (const path of collectSecretRefPaths(configSchema)) {
const binding = parseSecretRefBindingObject(readConfigValueAtPath(config, path));
if (!binding) continue;
if (binding.version !== "latest") {
throw unprocessable(
`Secret binding at ${path} pins version ${binding.version}; sandbox provider secret references always resolve the latest version.`,
);
}
config = writeConfigValueAtPath(config, path, binding.secretId);
}
const result = await input.workerManager.call(resolved.plugin.id, "environmentValidateConfig", {
driverKey: input.provider,
config: input.config,
config,
});
if (!result.ok) {
@ -151,7 +177,7 @@ export async function validatePluginSandboxProviderConfig(input: {
}
return {
normalizedConfig: result.normalizedConfig ?? input.config,
normalizedConfig: result.normalizedConfig ?? config,
pluginId: resolved.plugin.id,
pluginKey: resolved.plugin.pluginKey,
driver: resolved.driver,