From 0c2ec7deb4cdb7d470ea057fef39f232fe3c3a5b Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Tue, 30 Jun 2026 08:41:50 -0700 Subject: [PATCH] Fix sandboxed Claude and Codex probe behavior (#8775) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local adapters are the bridge between Paperclip's control plane and provider CLIs such as Claude Code and Codex. > - Those adapters can run either on the host machine or inside a remote/sandbox execution target. > - Sandbox probes need to validate the same auth/config path that real sandbox execution will use. > - The previous probe paths could surface misleading Claude errors, rely on host-only Codex state, or upload far more Codex home state than the probe needed. > - This pull request fixes the Claude and Codex sandbox probe/runtime behavior together while keeping provider-specific sandbox image work out of scope. > - The benefit is faster, clearer adapter health checks that better match real sandbox execution. ## Linked Issues or Issue Description No public GitHub issue was found for this exact bug during duplicate search. Bug report: **What happened?** Sandboxed Claude/Codex adapter tests could diverge from real runtime auth/config behavior. Claude sandbox probes could show the leading stream init line instead of the real final error, and Codex sandbox probes could upload full managed home state or mask a sandbox-local login with an empty uploaded `CODEX_HOME`. **Expected behavior** Sandbox probes should exercise the remote runtime contract, preserve useful sandbox credentials, avoid relying on unrelated host state, and report actionable probe failures. **Steps to reproduce** 1. Configure a remote/sandbox execution target for `claude_local` or `codex_local`. 2. Run the environment Test/probe path where host credentials differ from the sandbox's runtime credentials or the managed Codex home contains session history. 3. Observe that probe behavior can differ from the actual sandbox runtime path or surface an unhelpful Claude stream initialization line. **Paperclip version or commit** Current `master` before this PR, based on `4a2447da3`. **Deployment mode** Local development/control-plane deployment with remote sandbox execution targets. Related search performed: - Public issues: `Claude sandbox probe`, `Codex CODEX_HOME sandbox` returned no matches. - Public PRs: `Claude Codex sandbox probe`, `codex home sandbox`, `claude auth sandbox` returned no matches. ## What Changed - Made Claude sandbox Test probes materialize the same Paperclip-managed Claude config seed path used by sandbox execution. - Preserved sandbox-local Claude credentials when materializing remote Claude config and expanded auth-required detection for `/login` API-key failures. - Improved Claude hello-probe diagnostics so the final result/error is surfaced instead of the unhelpful stream init event, with transient upstream failures downgraded to warnings. - Changed Codex probe behavior to upload only minimal auth/config files instead of the full managed `CODEX_HOME`. - Let Codex sandbox probes leave `CODEX_HOME` unset when the host has no credentials, so pre-authenticated sandbox images can be tested directly. - Excluded bulky host-local Codex session/shell state from sandbox runtime home uploads. - Switched the Codex local default model away from the ChatGPT-unsupported `gpt-5.3-codex` option. - Added regression coverage for Claude parsing/probe paths, Codex adapter metadata/argument/probe behavior, and server-level Claude sandbox environment behavior. ## Verification Passed locally: - `pnpm install --frozen-lockfile` - `pnpm vitest run packages/adapters/claude-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/test.probe.test.ts server/src/__tests__/claude-local-adapter-environment.test.ts` - `pnpm vitest run packages/adapters/codex-local/src/index.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts packages/adapters/codex-local/src/server/test.remote.test.ts` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` ## Risks - Adapter configuration behavior is sensitive to local vs sandboxed execution mode, so review should focus on environment detection, argument construction, and any state written during probe/test runs. - The Codex default-model change may affect newly created agents that rely on the adapter default instead of an explicit model. - Excluding Codex session/shell state from sandbox uploads should be safe for fresh sandbox runs, but reviewers should confirm no runtime resume path depends on that host-local state. - Provider-specific setup/capture behavior is intentionally left to separate work. ## Model Used OpenAI GPT-5 Codex via Paperclip `codex_local`; tool-enabled local coding session with terminal access. Context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- .../claude-local/src/server/claude-config.ts | 39 +++++ .../claude-local/src/server/execute.ts | 22 +-- .../claude-local/src/server/parse.test.ts | 23 +++ .../adapters/claude-local/src/server/parse.ts | 2 +- .../src/server/test.probe.test.ts | 163 ++++++++++++++++++ .../adapters/claude-local/src/server/test.ts | 149 ++++++++++++++-- .../adapters/codex-local/src/index.test.ts | 9 + packages/adapters/codex-local/src/index.ts | 5 +- .../codex-local/src/server/codex-args.test.ts | 8 +- .../codex-local/src/server/execute.ts | 18 +- .../src/server/test.remote.test.ts | 91 ++++++++-- .../adapters/codex-local/src/server/test.ts | 39 ++++- .../claude-local-adapter-environment.test.ts | 96 +++++++++++ 13 files changed, 618 insertions(+), 46 deletions(-) create mode 100644 packages/adapters/claude-local/src/server/test.probe.test.ts create mode 100644 packages/adapters/codex-local/src/index.test.ts diff --git a/packages/adapters/claude-local/src/server/claude-config.ts b/packages/adapters/claude-local/src/server/claude-config.ts index 058f040c57..f8b0e5d7d5 100644 --- a/packages/adapters/claude-local/src/server/claude-config.ts +++ b/packages/adapters/claude-local/src/server/claude-config.ts @@ -3,7 +3,13 @@ import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import type { AdapterExecutionContext } from "@paperclipai/adapter-utils"; +import { + runAdapterExecutionTargetShellCommand, + type AdapterExecutionTarget, + type AdapterExecutionTargetShellOptions, +} from "@paperclipai/adapter-utils/execution-target"; import { resolvePaperclipInstanceRootForAdapter } from "@paperclipai/adapter-utils/server-utils"; +import { shellQuote } from "@paperclipai/adapter-utils/ssh"; const SEEDED_SHARED_FILES = ["settings.json", "CLAUDE.md"] as const; @@ -161,3 +167,36 @@ export async function prepareClaudeConfigSeed( return targetDir; } + +export function buildRemoteClaudeConfigMaterializationCommand(input: { + remoteClaudeConfigDir: string; + remoteClaudeConfigSeedDir: string; +}): string { + return `mkdir -p ${shellQuote(input.remoteClaudeConfigDir)} && ` + + `if [ -d ${shellQuote(input.remoteClaudeConfigSeedDir)} ]; then ` + + `cp -R ${shellQuote(`${input.remoteClaudeConfigSeedDir}/.`)} ${shellQuote(input.remoteClaudeConfigDir)}/; ` + + `fi; ` + + `for file in .credentials.json credentials.json; do ` + + `if [ -n "\${HOME:-}" ] && [ -f "\${HOME}/.claude/\${file}" ] && [ ! -f ${shellQuote(input.remoteClaudeConfigDir)}/"\${file}" ]; then ` + + `cp "\${HOME}/.claude/\${file}" ${shellQuote(input.remoteClaudeConfigDir)}/"\${file}"; ` + + `fi; ` + + `done`; +} + +export async function materializeRemoteClaudeConfig(input: { + runId: string; + target: AdapterExecutionTarget | null | undefined; + remoteClaudeConfigDir: string; + remoteClaudeConfigSeedDir: string; + options: AdapterExecutionTargetShellOptions; +}): Promise { + await runAdapterExecutionTargetShellCommand( + input.runId, + input.target, + buildRemoteClaudeConfigMaterializationCommand({ + remoteClaudeConfigDir: input.remoteClaudeConfigDir, + remoteClaudeConfigSeedDir: input.remoteClaudeConfigSeedDir, + }), + input.options, + ); +} diff --git a/packages/adapters/claude-local/src/server/execute.ts b/packages/adapters/claude-local/src/server/execute.ts index ad9136d96a..2fd71a6c1b 100644 --- a/packages/adapters/claude-local/src/server/execute.ts +++ b/packages/adapters/claude-local/src/server/execute.ts @@ -19,7 +19,6 @@ import { resolveAdapterExecutionTargetTimeoutSec, resolveAdapterExecutionTargetCommandForLogs, runAdapterExecutionTargetProcess, - runAdapterExecutionTargetShellCommand, startAdapterExecutionTargetPaperclipBridge, } from "@paperclipai/adapter-utils/execution-target"; import { @@ -45,7 +44,6 @@ import { stringifyPaperclipWakePayload, DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE, } from "@paperclipai/adapter-utils/server-utils"; -import { shellQuote } from "@paperclipai/adapter-utils/ssh"; import { parseClaudeStreamJson, describeClaudeFailure, @@ -58,7 +56,11 @@ import { isClaudePoisonedPreviousMessageIdError, isClaudeImageProcessingError, } from "./parse.js"; -import { prepareClaudeConfigSeed, resolveSharedClaudeConfigDir } from "./claude-config.js"; +import { + materializeRemoteClaudeConfig, + prepareClaudeConfigSeed, + resolveSharedClaudeConfigDir, +} from "./claude-config.js"; import { claudeCommandSupportsEffortFlag } from "./cli-capabilities.js"; import { resolveClaudeDesiredSkillNames } from "./skills.js"; import { isBedrockModelId } from "./models.js"; @@ -555,21 +557,19 @@ export async function execute(ctx: AdapterExecutionContext): Promise> = null; if (executionTargetIsRemote && adapterExecutionTargetUsesPaperclipBridge(runtimeExecutionTarget)) { diff --git a/packages/adapters/claude-local/src/server/parse.test.ts b/packages/adapters/claude-local/src/server/parse.test.ts index e197f46c2c..0169e5ac33 100644 --- a/packages/adapters/claude-local/src/server/parse.test.ts +++ b/packages/adapters/claude-local/src/server/parse.test.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from "vitest"; import { + detectClaudeLoginRequired, extractClaudeRetryNotBefore, isClaudeTransientUpstreamError, isClaudePoisonedPreviousMessageIdError, @@ -8,6 +9,28 @@ import { isClaudeImageProcessingError, } from "./parse.js"; +describe("detectClaudeLoginRequired", () => { + it("classifies Claude's invalid API key login prompt as auth required", () => { + expect( + detectClaudeLoginRequired({ + parsed: null, + stdout: "", + stderr: "Invalid API key · Please run /login", + }), + ).toEqual({ requiresLogin: true, loginUrl: null }); + }); + + it("does not classify a bare invalid API key as the Claude login flow", () => { + expect( + detectClaudeLoginRequired({ + parsed: null, + stdout: "", + stderr: "Invalid API key", + }).requiresLogin, + ).toBe(false); + }); +}); + describe("isClaudeTransientUpstreamError", () => { it("classifies the 'out of extra usage' subscription window failure as transient", () => { expect( diff --git a/packages/adapters/claude-local/src/server/parse.ts b/packages/adapters/claude-local/src/server/parse.ts index 14c8553431..657a476a01 100644 --- a/packages/adapters/claude-local/src/server/parse.ts +++ b/packages/adapters/claude-local/src/server/parse.ts @@ -6,7 +6,7 @@ import { parseJson, } from "@paperclipai/adapter-utils/server-utils"; -const CLAUDE_AUTH_REQUIRED_RE = /(?:not\s+logged\s+in|please\s+log\s+in|please\s+run\s+`?claude\s+login`?|login\s+required|requires\s+login|unauthorized|authentication\s+required)/i; +const CLAUDE_AUTH_REQUIRED_RE = /(?:not\s+logged\s+in|please\s+log\s+in|please\s+run\s+(?:`?claude\s+login`?|\/login)|login\s+required|requires\s+login|unauthorized|authentication\s+required|invalid\s+api\s+key[\s\S]{0,120}(?:\/login|claude\s+login|log\s+in))/i; const URL_RE = /(https?:\/\/[^\s'"`<>()[\]{};,!?]+[^\s'"`<>()[\]{};,!.?:]+)/gi; const CLAUDE_TRANSIENT_UPSTREAM_RE = diff --git a/packages/adapters/claude-local/src/server/test.probe.test.ts b/packages/adapters/claude-local/src/server/test.probe.test.ts new file mode 100644 index 0000000000..130fb22e43 --- /dev/null +++ b/packages/adapters/claude-local/src/server/test.probe.test.ts @@ -0,0 +1,163 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; +import type { AdapterExecutionTarget } from "@paperclipai/adapter-utils/execution-target"; + +const { + ensureAdapterExecutionTargetDirectory, + ensureAdapterExecutionTargetCommandResolvable, + maybeRunSandboxInstallCommand, + runAdapterExecutionTargetProcess, + describeAdapterExecutionTarget, + resolveAdapterExecutionTargetCwd, + probeResult, +} = vi.hoisted(() => { + const probeResult: { value: { exitCode: number; stdout: string; stderr: string } } = { + value: { exitCode: 1, stdout: "", stderr: "" }, + }; + return { + probeResult, + ensureAdapterExecutionTargetDirectory: vi.fn(async () => {}), + ensureAdapterExecutionTargetCommandResolvable: vi.fn(async () => {}), + maybeRunSandboxInstallCommand: vi.fn(async () => null), + runAdapterExecutionTargetProcess: vi.fn(async () => ({ + exitCode: probeResult.value.exitCode, + signal: null, + timedOut: false, + stdout: probeResult.value.stdout, + stderr: probeResult.value.stderr, + pid: 123, + startedAt: new Date().toISOString(), + })), + describeAdapterExecutionTarget: vi.fn(() => "Daytona"), + resolveAdapterExecutionTargetCwd: vi.fn(() => "/home/daytona/paperclip-workspace"), + }; +}); + +vi.mock("@paperclipai/adapter-utils/execution-target", async () => { + const actual = await vi.importActual( + "@paperclipai/adapter-utils/execution-target", + ); + return { + ...actual, + ensureAdapterExecutionTargetDirectory, + ensureAdapterExecutionTargetCommandResolvable, + maybeRunSandboxInstallCommand, + runAdapterExecutionTargetProcess, + describeAdapterExecutionTarget, + resolveAdapterExecutionTargetCwd, + }; +}); + +import { testEnvironment } from "./test.js"; + +const sandboxTarget: AdapterExecutionTarget = { + kind: "remote", + transport: "sandbox", + providerKey: "daytona", + remoteCwd: "/home/daytona/paperclip-workspace", + runner: { + execute: async () => ({ + exitCode: 0, + signal: null, + timedOut: false, + stdout: "", + stderr: "", + pid: null, + startedAt: new Date().toISOString(), + }), + }, +}; + +const initLine = + '{"type":"system","subtype":"init","cwd":"/home/daytona/paperclip-workspace","session_id":"abc","tools":["Bash","Read"]}'; + +afterEach(() => { + vi.clearAllMocks(); +}); + +describe("claude sandbox hello probe diagnostics", () => { + it("surfaces the final result error instead of the system/init line on failure", async () => { + probeResult.value = { + exitCode: 1, + stdout: [ + initLine, + '{"type":"result","subtype":"error_during_execution","is_error":true,"result":"API Error: 404 model not found: claude-opus-4-8","session_id":"abc"}', + ].join("\n"), + stderr: "", + }; + + const result = await testEnvironment({ + companyId: "company-1", + adapterType: "claude_local", + config: { command: "claude", model: "claude-opus-4-8" }, + executionTarget: sandboxTarget, + environmentName: "Daytona", + }); + + expect(result.status).toBe("fail"); + const failed = result.checks.find((check) => check.code === "claude_hello_probe_failed"); + expect(failed).toBeTruthy(); + expect(failed?.detail).toContain("404 model not found: claude-opus-4-8"); + // The unhelpful init line must not be what we show the operator. + expect(failed?.detail).not.toContain('"subtype":"init"'); + }); + + it("classifies rate-limit/overload failures as a transient warning, not a hard fail", async () => { + probeResult.value = { + exitCode: 1, + stdout: [ + initLine, + '{"type":"result","subtype":"error_during_execution","is_error":true,"result":"Claude usage limit reached. Please try again later.","session_id":"abc"}', + ].join("\n"), + stderr: "", + }; + + const result = await testEnvironment({ + companyId: "company-1", + adapterType: "claude_local", + config: { command: "claude" }, + executionTarget: sandboxTarget, + environmentName: "Daytona", + }); + + expect(result.checks.some((check) => check.code === "claude_hello_probe_transient_upstream")).toBe(true); + expect(result.checks.some((check) => check.code === "claude_hello_probe_failed")).toBe(false); + }); + + it("falls back to the last stdout line when no result event is emitted", async () => { + probeResult.value = { + exitCode: 1, + stdout: [initLine, "fatal: claude crashed unexpectedly"].join("\n"), + stderr: "", + }; + + const result = await testEnvironment({ + companyId: "company-1", + adapterType: "claude_local", + config: { command: "claude" }, + executionTarget: sandboxTarget, + environmentName: "Daytona", + }); + + const failed = result.checks.find((check) => check.code === "claude_hello_probe_failed"); + expect(failed?.detail).toContain("claude crashed unexpectedly"); + }); + + it("does not show the system/init event when it is the only stdout line", async () => { + probeResult.value = { + exitCode: 1, + stdout: initLine, + stderr: "", + }; + + const result = await testEnvironment({ + companyId: "company-1", + adapterType: "claude_local", + config: { command: "claude" }, + executionTarget: sandboxTarget, + environmentName: "Daytona", + }); + + const failed = result.checks.find((check) => check.code === "claude_hello_probe_failed"); + expect(failed?.detail).toBeUndefined(); + }); +}); diff --git a/packages/adapters/claude-local/src/server/test.ts b/packages/adapters/claude-local/src/server/test.ts index edaaae7c50..b07fc4a38b 100644 --- a/packages/adapters/claude-local/src/server/test.ts +++ b/packages/adapters/claude-local/src/server/test.ts @@ -1,3 +1,6 @@ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; import type { AdapterEnvironmentCheck, AdapterEnvironmentTestContext, @@ -8,6 +11,7 @@ import { asBoolean, asNumber, asStringArray, + parseJson, parseObject, ensurePathInEnv, } from "@paperclipai/adapter-utils/server-utils"; @@ -15,14 +19,22 @@ import { ensureAdapterExecutionTargetCommandResolvable, ensureAdapterExecutionTargetDirectory, maybeRunSandboxInstallCommand, + prepareAdapterExecutionTargetRuntime, runAdapterExecutionTargetProcess, describeAdapterExecutionTarget, resolveAdapterExecutionTargetCwd, + adapterExecutionTargetUsesManagedHome, } from "@paperclipai/adapter-utils/execution-target"; -import { detectClaudeLoginRequired, parseClaudeStreamJson } from "./parse.js"; +import { + describeClaudeFailure, + detectClaudeLoginRequired, + isClaudeTransientUpstreamError, + parseClaudeStreamJson, +} from "./parse.js"; import { claudeCommandLooksLike, claudeCommandSupportsEffortFlag } from "./cli-capabilities.js"; import { isBedrockModelId } from "./models.js"; import { buildClaudeProbePermissionArgs } from "./permissions.js"; +import { materializeRemoteClaudeConfig, prepareClaudeConfigSeed } from "./claude-config.js"; import { SANDBOX_INSTALL_COMMAND } from "../index.js"; function summarizeStatus(checks: AdapterEnvironmentCheck[]): AdapterEnvironmentTestResult["status"] { @@ -44,8 +56,29 @@ function firstNonEmptyLine(text: string): string { ); } +function lastNonInitStdoutLine(text: string): string { + const lines = text + .split(/\r?\n/) + .map((line) => line.trim()) + .filter(Boolean); + for (let index = lines.length - 1; index >= 0; index -= 1) { + const line = lines[index]!; + const parsed = parseJson(line); + if (parsed && asString(parsed.type, "") === "system" && asString(parsed.subtype, "") === "init") { + continue; + } + return line; + } + return ""; +} + +function truncateDetail(value: string, max = 240): string { + const clean = value.replace(/\s+/g, " ").trim(); + return clean.length > max ? `${clean.slice(0, max - 1)}…` : clean; +} + function summarizeProbeDetail(stdout: string, stderr: string): string | null { - const raw = firstNonEmptyLine(stderr) || firstNonEmptyLine(stdout); + const raw = firstNonEmptyLine(stderr) || lastNonInitStdoutLine(stdout); if (!raw) return null; const clean = raw.replace(/\s+/g, " ").trim(); const max = 240; @@ -100,7 +133,6 @@ export async function testEnvironment( for (const [key, value] of Object.entries(envConfig)) { if (typeof value === "string") env[key] = value; } - const runtimeEnv = ensurePathInEnv({ ...process.env, ...env }); const installCheck = await maybeRunSandboxInstallCommand({ runId, target, @@ -110,6 +142,69 @@ export async function testEnvironment( env, }); if (installCheck) checks.push(installCheck); + const hasExplicitClaudeConfigDir = isNonEmpty(env.CLAUDE_CONFIG_DIR); + if (targetIsRemote && adapterExecutionTargetUsesManagedHome(target) && !hasExplicitClaudeConfigDir) { + let tempWorkspaceDir: string | null = null; + let preparedRuntime: Awaited> | null = null; + try { + const seedDir = await prepareClaudeConfigSeed(process.env, async () => {}, ctx.companyId); + const managedRemoteCwd = target?.kind === "remote" ? target.remoteCwd : cwd; + tempWorkspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-claude-envtest-workspace-")); + preparedRuntime = await prepareAdapterExecutionTargetRuntime({ + runId, + target, + adapterKey: "claude", + workspaceLocalDir: tempWorkspaceDir, + workspaceRemoteDir: managedRemoteCwd, + timeoutSec: Math.max(1, asNumber(config.helloProbeTimeoutSec, targetIsSandbox ? 90 : 45)), + assets: [ + { + key: "config-seed", + localDir: seedDir, + followSymlinks: true, + }, + ], + }); + const runtimeRootDir = + preparedRuntime.runtimeRootDir ?? path.posix.join(managedRemoteCwd, ".paperclip-runtime", "claude"); + const remoteClaudeConfigSeedDir = + preparedRuntime.assetDirs["config-seed"] ?? path.posix.join(runtimeRootDir, "config-seed"); + const remoteClaudeConfigDir = path.posix.join(runtimeRootDir, "config"); + env.CLAUDE_CONFIG_DIR = remoteClaudeConfigDir; + await materializeRemoteClaudeConfig({ + runId, + target, + remoteClaudeConfigDir, + remoteClaudeConfigSeedDir, + options: { + cwd, + env, + timeoutSec: Math.max(15, asNumber(config.helloProbeTimeoutSec, targetIsSandbox ? 90 : 45)), + graceSec: 5, + onLog: async () => {}, + }, + }); + checks.push({ + code: "claude_managed_config_dir", + level: "info", + message: "Sandbox probe is using Paperclip-managed Claude config materialization.", + detail: remoteClaudeConfigDir, + }); + } catch (err) { + checks.push({ + code: "claude_managed_config_dir_failed", + level: "error", + message: "Could not materialize Paperclip-managed Claude config for the sandbox probe.", + detail: err instanceof Error ? err.message : String(err), + }); + } finally { + await preparedRuntime?.restoreWorkspace().catch(() => undefined); + if (tempWorkspaceDir) { + await fs.rm(tempWorkspaceDir, { recursive: true, force: true }).catch(() => undefined); + } + } + } + const runtimeEnv = ensurePathInEnv({ ...process.env, ...env }); try { await ensureAdapterExecutionTargetCommandResolvable(command, target, cwd, runtimeEnv); checks.push({ @@ -174,7 +269,12 @@ export async function testEnvironment( } const canRunProbe = - checks.every((check) => check.code !== "claude_cwd_invalid" && check.code !== "claude_command_unresolvable"); + checks.every( + (check) => + check.code !== "claude_cwd_invalid" && + check.code !== "claude_command_unresolvable" && + check.code !== "claude_managed_config_dir_failed", + ); if (canRunProbe) { if (!claudeCommandLooksLike(command, "claude")) { checks.push({ @@ -296,13 +396,42 @@ export async function testEnvironment( }), }); } else { - checks.push({ - code: "claude_hello_probe_failed", - level: "error", - message: "Claude hello probe failed.", - ...(detail ? { detail } : {}), - hint: "Run `claude --print - --output-format stream-json --verbose` manually in this directory and prompt `Respond with hello` to debug.", + // Surface the actual failure instead of the leading stream-json + // `system/init` line: the real error lives in the final `result` + // event (parsed) or, when the CLI dies before emitting one, the last + // non-init stdout line — never the first one `summarizeProbeDetail` + // returns. + const stdoutFallback = lastNonInitStdoutLine(probe.stdout); + const failureDetail = + (parsed ? describeClaudeFailure(parsed) : null) || + (firstNonEmptyLine(probe.stderr) + ? truncateDetail(firstNonEmptyLine(probe.stderr)) + : "") || + (stdoutFallback ? truncateDetail(stdoutFallback) : "") || + detail || + ""; + const transient = isClaudeTransientUpstreamError({ + parsed, + stdout: probe.stdout, + stderr: probe.stderr, }); + checks.push( + transient + ? { + code: "claude_hello_probe_transient_upstream", + level: "warn", + message: "Claude hello probe hit a transient upstream error (rate limit or overload).", + ...(failureDetail ? { detail: failureDetail } : {}), + hint: "This is usually temporary. Wait a moment and re-run Test.", + } + : { + code: "claude_hello_probe_failed", + level: "error", + message: "Claude hello probe failed.", + ...(failureDetail ? { detail: failureDetail } : {}), + hint: `Exit code ${probe.exitCode ?? "unknown"}. Run \`claude --print - --output-format stream-json --verbose\` manually in this directory and prompt \`Respond with hello\` to debug.`, + }, + ); } } } diff --git a/packages/adapters/codex-local/src/index.test.ts b/packages/adapters/codex-local/src/index.test.ts new file mode 100644 index 0000000000..a40331e778 --- /dev/null +++ b/packages/adapters/codex-local/src/index.test.ts @@ -0,0 +1,9 @@ +import { describe, expect, it } from "vitest"; +import { DEFAULT_CODEX_LOCAL_MODEL, models } from "./index.js"; + +describe("codex local adapter metadata", () => { + it("does not advertise the ChatGPT-unsupported gpt-5.3-codex model as a default option", () => { + expect(DEFAULT_CODEX_LOCAL_MODEL).toBe("gpt-5.5"); + expect(models.map((model) => model.id)).not.toContain("gpt-5.3-codex"); + }); +}); diff --git a/packages/adapters/codex-local/src/index.ts b/packages/adapters/codex-local/src/index.ts index b2e9df7a92..dac11bbc76 100644 --- a/packages/adapters/codex-local/src/index.ts +++ b/packages/adapters/codex-local/src/index.ts @@ -5,7 +5,7 @@ export const label = "Codex (local)"; export const SANDBOX_INSTALL_COMMAND = "npm install -g @openai/codex"; -export const DEFAULT_CODEX_LOCAL_MODEL = "gpt-5.3-codex"; +export const DEFAULT_CODEX_LOCAL_MODEL = "gpt-5.5"; export const DEFAULT_CODEX_LOCAL_BYPASS_APPROVALS_AND_SANDBOX = true; export const CODEX_LOCAL_FAST_MODE_SUPPORTED_MODELS = ["gpt-5.5", "gpt-5.4"] as const; @@ -38,9 +38,8 @@ export function isCodexLocalFastModeSupported(model: string | null | undefined): } export const models = [ - { id: "gpt-5.5", label: "gpt-5.5" }, - { id: "gpt-5.4", label: "gpt-5.4" }, { id: DEFAULT_CODEX_LOCAL_MODEL, label: DEFAULT_CODEX_LOCAL_MODEL }, + { id: "gpt-5.4", label: "gpt-5.4" }, { id: "gpt-5.3-codex-spark", label: "gpt-5.3-codex-spark" }, { id: "gpt-5", label: "gpt-5" }, { id: "o3", label: "o3" }, diff --git a/packages/adapters/codex-local/src/server/codex-args.test.ts b/packages/adapters/codex-local/src/server/codex-args.test.ts index 5ef2608d66..46a9890d91 100644 --- a/packages/adapters/codex-local/src/server/codex-args.test.ts +++ b/packages/adapters/codex-local/src/server/codex-args.test.ts @@ -91,7 +91,7 @@ describe("buildCodexExecArgs", () => { it("ignores fast mode for unsupported models", () => { const result = buildCodexExecArgs({ - model: "gpt-5.3-codex", + model: "gpt-5.3-codex-spark", fastMode: true, }); @@ -104,7 +104,7 @@ describe("buildCodexExecArgs", () => { "exec", "--json", "--model", - "gpt-5.3-codex", + "gpt-5.3-codex-spark", "-", ]); }); @@ -112,7 +112,7 @@ describe("buildCodexExecArgs", () => { it("adds --skip-git-repo-check when requested", () => { const result = buildCodexExecArgs( { - model: "gpt-5.3-codex", + model: "gpt-5.5", }, { skipGitRepoCheck: true }, ); @@ -122,7 +122,7 @@ describe("buildCodexExecArgs", () => { "--json", "--skip-git-repo-check", "--model", - "gpt-5.3-codex", + "gpt-5.5", "-", ]); }); diff --git a/packages/adapters/codex-local/src/server/execute.ts b/packages/adapters/codex-local/src/server/execute.ts index 32b211c06c..193febc6de 100644 --- a/packages/adapters/codex-local/src/server/execute.ts +++ b/packages/adapters/codex-local/src/server/execute.ts @@ -470,12 +470,18 @@ export async function execute(ctx: AdapterExecutionContext): Promise { const restoreWorkspace = vi.fn(async () => {}); + // Records the files staged in the uploaded "home" asset at call time, before + // the probe's cleanup deletes the temp dir. Lets tests assert the upload is a + // minimal credentials-only home and not the full managed CODEX_HOME. + const capturedHomeAssetFiles: { value: string[] | null } = { value: null }; return { + capturedHomeAssetFiles, ensureAdapterExecutionTargetDirectory: vi.fn(async () => {}), ensureAdapterExecutionTargetCommandResolvable: vi.fn(async () => {}), maybeRunSandboxInstallCommand: vi.fn(async () => null), @@ -40,16 +46,29 @@ const { } return fallbackCwd; }), - prepareAdapterExecutionTargetRuntime: vi.fn(async () => ({ - target: null, - workspaceRemoteDir: "/remote/workspace/.paperclip-runtime/runs/test/workspace", - runtimeRootDir: "/remote/workspace/.paperclip-runtime/runs/test/workspace/.paperclip-runtime/codex", - assetDirs: { - home: "/remote/workspace/.paperclip-runtime/runs/test/workspace/.paperclip-runtime/codex/home", - }, - restoreWorkspace, - })), - prepareManagedCodexHome: vi.fn(async () => "/tmp/paperclip-managed-codex-home"), + prepareAdapterExecutionTargetRuntime: vi.fn(async (input: { assets?: Array<{ key: string; localDir: string }> }) => { + const homeAsset = input?.assets?.find((asset) => asset.key === "home"); + if (homeAsset) { + capturedHomeAssetFiles.value = (await fs.readdir(homeAsset.localDir)).sort(); + } + return { + target: null, + workspaceRemoteDir: "/remote/workspace/.paperclip-runtime/runs/test/workspace", + runtimeRootDir: "/remote/workspace/.paperclip-runtime/runs/test/workspace/.paperclip-runtime/codex", + assetDirs: { + home: "/remote/workspace/.paperclip-runtime/runs/test/workspace/.paperclip-runtime/codex/home", + }, + restoreWorkspace, + }; + }), + prepareManagedCodexHome: vi.fn(async () => { + // Return a real managed home seeded with credentials so the probe's + // minimal-home copy step (auth.json/config.toml) has something to read. + const dir = await fs.mkdtemp(`${os.tmpdir()}/paperclip-managed-codex-home-`); + await fs.writeFile(`${dir}/auth.json`, JSON.stringify({ OPENAI_API_KEY: "sk-managed" })); + await fs.writeFile(`${dir}/config.toml`, "model = \"gpt-5\"\n"); + return dir; + }), restoreWorkspace, }; }); @@ -122,9 +141,15 @@ describe("codex remote environment diagnostics", () => { workspaceLocalDir: string; target?: { remoteCwd?: string }; workspaceRemoteDir?: string; + assets?: Array<{ key: string; localDir: string }>; }, ]>; const runtimeInput = runtimeCalls[0]?.[0]; + // The probe must upload only a minimal credentials-only home, never the + // full managed CODEX_HOME (which can be hundreds of MB of session history). + const homeAsset = runtimeInput?.assets?.find((asset) => asset.key === "home"); + expect(homeAsset?.localDir).toContain(`${os.tmpdir()}/paperclip-codex-probe-home-`); + expect(capturedHomeAssetFiles.value).toEqual(["auth.json", "config.toml"]); expect(runtimeInput?.workspaceLocalDir).toContain(`${os.tmpdir()}/paperclip-codex-envtest-`); expect(runtimeInput?.workspaceLocalDir).not.toBe("/remote/workspace"); expect(await fs.stat(runtimeInput!.workspaceLocalDir).catch(() => null)).toBeNull(); @@ -191,4 +216,50 @@ describe("codex remote environment diagnostics", () => { expect(probeCall?.[4].env.CODEX_HOME?.startsWith("/tmp/")).toBe(false); expect(probeCall?.[3]).toContain("--skip-git-repo-check"); }); + + it("does not override CODEX_HOME when the host has no credentials to seed", async () => { + // Pre-authenticated sandbox flow: the login lives inside the sandbox image, + // and the host has no Codex auth.json. The probe must not upload an empty + // home or set CODEX_HOME, so Codex falls back to the sandbox's baked-in login. + prepareManagedCodexHome.mockImplementationOnce(async () => { + const dir = await fs.mkdtemp(`${os.tmpdir()}/paperclip-managed-codex-home-noauth-`); + // No auth.json — only a config file. + await fs.writeFile(`${dir}/config.toml`, "model = \"gpt-5\"\n"); + return dir; + }); + + const remoteTarget: AdapterExecutionTarget = { + kind: "remote", + transport: "sandbox", + providerKey: "daytona", + remoteCwd: "/remote/workspace", + runner: { + execute: async () => ({ + exitCode: 0, + signal: null, + timedOut: false, + stdout: "", + stderr: "", + pid: null, + startedAt: new Date().toISOString(), + }), + }, + }; + + const result = await testEnvironment({ + companyId: "company-1", + adapterType: "codex_local", + config: { command: "codex" }, + executionTarget: remoteTarget, + environmentName: "QA Daytona", + }); + + expect(result.status).toBe("pass"); + // No managed-home upload, so the full-runtime staging is skipped entirely. + expect(prepareAdapterExecutionTargetRuntime).not.toHaveBeenCalled(); + const probeCall = runAdapterExecutionTargetProcess.mock.calls[0] as unknown as + | [string, AdapterExecutionTarget, string, string[], { cwd: string; env: Record }] + | undefined; + expect(probeCall?.[4].env.CODEX_HOME).toBeUndefined(); + }); }); diff --git a/packages/adapters/codex-local/src/server/test.ts b/packages/adapters/codex-local/src/server/test.ts index 5c7e77db86..0c9772048c 100644 --- a/packages/adapters/codex-local/src/server/test.ts +++ b/packages/adapters/codex-local/src/server/test.ts @@ -79,18 +79,55 @@ async function prepareCodexHelloProbe(input: { }> { let preparedRuntime: Awaited> | null = null; let preparedRuntimeWorkspaceLocalDir: string | null = null; + let probeHomeLocalDir: string | null = null; const cleanup = async () => { await preparedRuntime?.restoreWorkspace().catch(() => {}); if (preparedRuntimeWorkspaceLocalDir) { await fs.rm(preparedRuntimeWorkspaceLocalDir, { recursive: true, force: true }).catch(() => {}); } + if (probeHomeLocalDir) { + await fs.rm(probeHomeLocalDir, { recursive: true, force: true }).catch(() => {}); + } }; if (input.targetIsRemote && !input.probeApiKey) { const managedHome = await prepareManagedCodexHome(process.env, async () => {}, input.companyId, { apiKey: null, }); + + // Upload only the credential/config files the login probe needs, not the + // entire managed CODEX_HOME. A real managed home accumulates hundreds of MB + // of session/state history (`sessions/`, `state_*.sqlite`, …); tarring and + // streaming all of it into the sandbox made the environment Test probe take + // many minutes and look like it hung. The hello probe only needs auth. + probeHomeLocalDir = await fs.mkdtemp( + path.join(os.tmpdir(), `paperclip-codex-probe-home-${input.runId}-`), + ); + let seededAuth = false; + for (const file of ["auth.json", "config.toml"]) { + // `fs.readFile` follows the managed home's `auth.json` symlink into the + // host's `~/.codex`, so we copy the resolved bytes as a plain file. + const contents = await fs.readFile(path.join(managedHome, file)).catch(() => null); + if (contents) { + await fs.writeFile(path.join(probeHomeLocalDir, file), contents); + if (file === "auth.json") seededAuth = true; + } + } + + // When the host has no Codex credentials to seed, don't override CODEX_HOME. + // Pointing Codex at an empty uploaded home would mask any login already + // baked into a prepared sandbox image; leaving CODEX_HOME unset lets the + // probe exercise that in-sandbox login instead. + if (!seededAuth) { + return { + command: input.command, + args: input.args, + env: { ...input.env }, + cleanup, + }; + } + preparedRuntimeWorkspaceLocalDir = await fs.mkdtemp( path.join(os.tmpdir(), `paperclip-codex-envtest-${input.runId}-`), ); @@ -109,7 +146,7 @@ async function prepareCodexHelloProbe(input: { assets: [ { key: "home", - localDir: managedHome, + localDir: probeHomeLocalDir, followSymlinks: true, }, ], diff --git a/server/src/__tests__/claude-local-adapter-environment.test.ts b/server/src/__tests__/claude-local-adapter-environment.test.ts index 7847d0e1f8..1316c9e9bb 100644 --- a/server/src/__tests__/claude-local-adapter-environment.test.ts +++ b/server/src/__tests__/claude-local-adapter-environment.test.ts @@ -8,6 +8,9 @@ import { resetClaudeCliCapabilitiesCacheForTests, testEnvironment } from "@paper const ORIGINAL_ANTHROPIC = process.env.ANTHROPIC_API_KEY; const ORIGINAL_BEDROCK = process.env.CLAUDE_CODE_USE_BEDROCK; const ORIGINAL_BEDROCK_URL = process.env.ANTHROPIC_BEDROCK_BASE_URL; +const ORIGINAL_CLAUDE_CONFIG_DIR = process.env.CLAUDE_CONFIG_DIR; +const ORIGINAL_PAPERCLIP_HOME = process.env.PAPERCLIP_HOME; +const ORIGINAL_PAPERCLIP_INSTANCE_ID = process.env.PAPERCLIP_INSTANCE_ID; afterEach(() => { resetClaudeCliCapabilitiesCacheForTests(); @@ -26,6 +29,21 @@ afterEach(() => { } else { process.env.ANTHROPIC_BEDROCK_BASE_URL = ORIGINAL_BEDROCK_URL; } + if (ORIGINAL_CLAUDE_CONFIG_DIR === undefined) { + delete process.env.CLAUDE_CONFIG_DIR; + } else { + process.env.CLAUDE_CONFIG_DIR = ORIGINAL_CLAUDE_CONFIG_DIR; + } + if (ORIGINAL_PAPERCLIP_HOME === undefined) { + delete process.env.PAPERCLIP_HOME; + } else { + process.env.PAPERCLIP_HOME = ORIGINAL_PAPERCLIP_HOME; + } + if (ORIGINAL_PAPERCLIP_INSTANCE_ID === undefined) { + delete process.env.PAPERCLIP_INSTANCE_ID; + } else { + process.env.PAPERCLIP_INSTANCE_ID = ORIGINAL_PAPERCLIP_INSTANCE_ID; + } }); async function writeHelpWithoutEffortClaudeCommand(commandPath: string): Promise { @@ -333,6 +351,84 @@ describe("claude_local environment diagnostics", () => { expect(probeCall?.args).toContain("--allowedTools"); }); + it("uses the managed Claude config seed for sandbox hello probes", async () => { + const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-claude-envtest-managed-config-")); + const sourceConfigDir = path.join(root, "host-claude"); + const remoteHome = path.join(root, "remote-home"); + const remoteWorkspace = path.join(root, "remote-workspace"); + const commandPath = path.join(root, "claude"); + + await fs.mkdir(sourceConfigDir, { recursive: true }); + await fs.mkdir(path.join(remoteHome, ".claude"), { recursive: true }); + await fs.mkdir(remoteWorkspace, { recursive: true }); + await fs.writeFile(path.join(sourceConfigDir, "settings.json"), JSON.stringify({ + theme: "dark", + permissions: { defaultMode: "bypassPermissions" }, + hooks: { PreToolUse: [{ matcher: "*" }] }, + mcpServers: { local: { command: "secret-local-server" } }, + permissionMode: "dontAsk", + skipDangerousModePermissionPrompt: true, + }), "utf8"); + await fs.writeFile(path.join(sourceConfigDir, "CLAUDE.md"), "seed instructions", "utf8"); + await fs.writeFile(path.join(sourceConfigDir, "credentials.json"), JSON.stringify({ token: "local" }), "utf8"); + await fs.writeFile(path.join(remoteHome, ".claude", ".credentials.json"), JSON.stringify({ token: "remote" }), "utf8"); + await fs.writeFile(commandPath, `#!/usr/bin/env node +const fs = require("fs"); +const path = require("path"); +const configDir = process.env.CLAUDE_CONFIG_DIR || ""; +function fail(message) { + process.stderr.write(message + "\\n"); + process.exit(2); +} +if (!configDir.includes(".paperclip-runtime/claude/config")) { + fail("missing managed CLAUDE_CONFIG_DIR: " + configDir); +} +const settings = JSON.parse(fs.readFileSync(path.join(configDir, "settings.json"), "utf8")); +if (settings.permissions?.defaultMode !== "default") fail("permissions were not sanitized"); +if (settings.hooks || settings.mcpServers || settings.permissionMode || settings.skipDangerousModePermissionPrompt) { + fail("local-only settings leaked into sandbox config"); +} +if (fs.existsSync(path.join(configDir, "credentials.json"))) fail("host credentials leaked into sandbox config"); +const remoteCredentials = JSON.parse(fs.readFileSync(path.join(configDir, ".credentials.json"), "utf8")); +if (remoteCredentials.token !== "remote") fail("sandbox credentials were not preserved"); +if (fs.readFileSync(path.join(configDir, "CLAUDE.md"), "utf8") !== "seed instructions") { + fail("CLAUDE.md seed was not materialized"); +} +console.log(JSON.stringify({ type: "assistant", message: { content: [{ type: "text", text: "hello" }] } })); +console.log(JSON.stringify({ type: "result", result: "hello", usage: { input_tokens: 1, cache_read_input_tokens: 0, output_tokens: 1 } })); +`, "utf8"); + await fs.chmod(commandPath, 0o755); + + process.env.CLAUDE_CONFIG_DIR = sourceConfigDir; + process.env.PAPERCLIP_HOME = path.join(root, "paperclip-home"); + process.env.PAPERCLIP_INSTANCE_ID = "test-instance"; + + try { + const result = await testEnvironment({ + companyId: "company-1", + adapterType: "claude_local", + config: { + command: commandPath, + env: { HOME: remoteHome }, + }, + executionTarget: { + kind: "remote", + transport: "sandbox", + providerKey: "daytona", + remoteCwd: remoteWorkspace, + runner: createLocalSandboxRunner(), + }, + environmentName: "QA Daytona", + }); + + expect(result.checks.some((check) => check.code === "claude_managed_config_dir")).toBe(true); + expect(result.checks.some((check) => check.code === "claude_hello_probe_passed")).toBe(true); + expect(result.checks.some((check) => check.code === "claude_hello_probe_failed")).toBe(false); + } finally { + await fs.rm(root, { recursive: true, force: true }); + } + }); + it("warns and omits --effort for sandbox probes when the installed Claude CLI does not advertise it", async () => { const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-claude-envtest-sandbox-effort-")); const workspace = path.join(root, "workspace");