refactor(acpx-engine): coordinator-owned ACP run lifecycle with a typed resource ledger (#11576)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent adapters run agent sessions through the ACPX engine > - The ACPX engine handled one run attempt as a long implicit procedure > - That shape made resource ownership, cleanup order, and failure behavior hard to verify > - This pull request gives the attempt a coordinator, a typed resource ledger, separate run sites, and explicit turn and settlement sequences > - The benefit is clear ownership, one cleanup path, safer session reuse, and testable failure behavior ## Linked Issues or Issue Description **What existing behavior does this improve?** The ACPX engine manages startup, turn execution, session reuse, and cleanup inside one large run procedure. **Current behavior** The run procedure owns several resources through implicit control flow. Cleanup and session reuse behavior depend on lane-specific branches and error paths. **Proposed behavior** The coordinator owns the run attempt. A typed ledger records six resources and their states. Host and sandbox run sites own lane-specific acquisition. Turn and settlement sequences expose typed outcomes. The engine emits allowlisted phase telemetry. **Reason and benefit** Explicit ownership makes cleanup and failure behavior easier to inspect. The fault matrix and characterization tests protect the external result while the refactor reduces hidden control flow. **Breaking changes** None to the public adapter contract. The host warm-save path now closes and relaunches the runtime because a transferred runtime could retain a run-scoped credential. A cold session-handshake failure now closes the created runtime. **Additional context** This pull request contains the ACPX engine lifecycle refactor, its tests, and the lifecycle document. ## What Changed - Add a run coordinator for startup, turn execution, settlement, and result reproduction. - Add a typed resource ledger with open, sealed, and consumed states. - Add host and sandbox run sites for lane-specific resource acquisition. - Replace separate runtime maps with a generic session reuse store. - Split session fingerprint identity from the outer session key. - Add typed turn and settlement sequences with one cleanup owner. - Add a closed allowlist for phase telemetry. - Add characterization tests and a 17-case fault matrix. - Add `doc/acp-run-lifecycle.md`. ## Verification - `npx vitest run packages/adapter-utils/src/acpx-engine/` passes 18 files and 286 tests at the submitted commit. - `pnpm --filter @paperclipai/adapter-utils typecheck` reports 0 errors at the submitted commit. - Run the full pull request checks after GitHub starts CI. - Run Greptile review after the pull request opens. ## Risks - The refactor changes internal control flow across the ACPX engine. - Host warm-save behavior now closes and relaunches the runtime. - Settlement changes the handling of a cold session-handshake failure from a leak to a close. - The characterization baselines and fault matrix reduce the risk of an external behavior change. > Paperclip is the open source app people use to manage AI agents for work > The adapter layer runs agent sessions through the ACPX engine > The engine needs explicit lifecycle ownership for reliable cleanup > This pull request adds coordinator-owned phases and a typed resource ledger > The result makes lifecycle behavior easier to test and review ## Model Used OpenAI GPT-5 Codex. Exact model ID: GPT-5. The model used tool execution, repository inspection, and code review support. The implementation author supplied the submitted code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky Leach
GitHub
parent
a1278e6ded
commit
b446ff59bf
|
|
@ -0,0 +1,101 @@
|
|||
# ACP run lifecycle
|
||||
|
||||
This document describes how one ACP agent run starts, runs its turn, and releases
|
||||
its resources. It covers the run coordinator, the six run resources, the
|
||||
settlement order, the server-owned staging lease, and the known limitations.
|
||||
|
||||
## The run coordinator
|
||||
|
||||
One `run()` coordinator owns one attempt. It sequences four steps in a fixed
|
||||
order:
|
||||
|
||||
1. **Startup.** Bring up the runtime and establish the session.
|
||||
2. **Turn.** Run the agent turn against the ready runtime.
|
||||
3. **Settlement.** Release every resource the run acquired.
|
||||
4. **Result reproduction.** Return the external result to the caller.
|
||||
|
||||
The coordinator holds the routing table as data. It reads the startup outcome and
|
||||
routes on it:
|
||||
|
||||
- A build failure or a partial-bridge failure throws from startup. The coordinator
|
||||
replays the startup-rollback entries and rethrows the original error.
|
||||
- A runtime-create, handshake, missing-handle, or configuration failure produces a
|
||||
settle result. The coordinator settles the resources, then reproduces the error
|
||||
result.
|
||||
- Every other startup outcome produces a ready result. The coordinator runs the
|
||||
turn, settles the resources, then reproduces the turn result.
|
||||
|
||||
The coordinator has no catch around the turn: the turn never rejects and returns a
|
||||
typed completion. The coordinator reproduces the external result AFTER settlement,
|
||||
so a caller never observes a result before the run releases its resources.
|
||||
|
||||
## The six run resources
|
||||
|
||||
A run resource ledger holds the resources between acquisition and settlement. The
|
||||
ledger is the single owner. The closed set of run resources is:
|
||||
|
||||
- `acp_runtime` — the composite of the runtime, the session handle, and the child
|
||||
process. `runtime.close()` is the one release boundary for all three.
|
||||
- `staged_runtime` — the workspace and the runtime staged into the sandbox, plus
|
||||
the one-time host-side cleanup of the staged temporary files.
|
||||
- `control_bridge` — the control-plane callback bridge.
|
||||
- `agent_bridge` — the agent process-session bridge.
|
||||
- `managed_home` — the per-run managed-home copy-back hook.
|
||||
- `staging_lease` — the per-session staging lease.
|
||||
|
||||
The host lane acquires only `acp_runtime`. The sandbox lane also acquires the
|
||||
staging and transport resources. Each resource enters the ledger the moment it
|
||||
exists, so the settlement always closes it, even on an early failure.
|
||||
|
||||
## The settlement order
|
||||
|
||||
The settlement sequence is the one live cleanup owner for every settled path. It
|
||||
claims the ledger once, makes the pure reuse decision, then runs the ordered
|
||||
steps:
|
||||
|
||||
1. `end_session` — close every runtime the reuse decision did not transfer, and
|
||||
drop the warm entry when it closes.
|
||||
2. `settle_reuse` — perform the decision. A save transfers the reuse candidate to
|
||||
the site store; every other case discards it.
|
||||
3. `stop_transport` — stop both bridges in one settled batch.
|
||||
4. `sync_back` — run the site sync-back (the managed-home copy-back).
|
||||
5. `release_staging_lease` — release the staging lease last (see below).
|
||||
|
||||
An error policy governs every step: a step records its error, and the later steps
|
||||
still run. Every step is a no-op on an empty resource slot.
|
||||
|
||||
The reuse decision is pure and runs before any close. A save is eligible only when
|
||||
a candidate exists, the settlement cause permits a save, and no run-scoped
|
||||
credential that the candidate would carry into the store remains valid.
|
||||
|
||||
## The server-owned staging lease outer context
|
||||
|
||||
The staging lease is a per-session lease. Only one run of a session may stage into
|
||||
the same remote workspace at a time. A second run of the same session waits on the
|
||||
lease until the first run releases it.
|
||||
|
||||
The lease releases as the run's final act, after the coordinator settles every
|
||||
other resource AND reproduces the result. This ordering keeps a same-session
|
||||
second run blocked on the lease until the first run fully returns, so the second
|
||||
run never re-stages into a workspace the first run still uses. The release runs in
|
||||
a `finally`, so an earlier teardown fault never strands the lease.
|
||||
|
||||
## Per-phase telemetry
|
||||
|
||||
The run emits one telemetry event per named lifecycle phase. Each event carries
|
||||
only the phase name, the wall-time duration, and the outcome (`ok` or `failed`).
|
||||
The phase name is one member of a closed allowlist. An event never carries a
|
||||
command, an argument, a path, an environment value, or a raw identifier. A
|
||||
telemetry failure never fails the run.
|
||||
|
||||
## Known limitations and deferred work
|
||||
|
||||
- **Host-lane runtime reuse is disabled.** A run-minted API key is a stateless
|
||||
token that the control plane never revokes on run end. A warm host runtime would
|
||||
carry a still-valid credential into the next run. The host lane therefore closes
|
||||
and relaunches on every run, rather than keeping the runtime warm, until a
|
||||
run-scoped credential rebind protocol exists.
|
||||
- **The sandbox staged-files reuse stays enabled.** Its reuse payload carries no
|
||||
credential, so a compatible resume reuses the already-staged runtime.
|
||||
- **The per-phase telemetry is observability-only.** It records the duration and
|
||||
the outcome of each phase; it does not change run control flow.
|
||||
|
|
@ -715,7 +715,7 @@ describe("composed ACPX run: host-lane warm handle set", () => {
|
|||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
it("warm-saves a persistent local runtime and reuses it for the next compatible run", async () => {
|
||||
it("closes and relaunches a persistent local runtime, so the next compatible run re-creates it", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
let createCount = 0;
|
||||
|
|
@ -746,8 +746,9 @@ describe("composed ACPX run: host-lane warm handle set", () => {
|
|||
onEvent: async () => {},
|
||||
} as never);
|
||||
expect(first.exitCode).toBe(0);
|
||||
// A persistent local completed turn warm-saves the runtime handle.
|
||||
expect(warmHandles.size).toBe(1);
|
||||
// Amendment B: a persistent local completed turn closes and relaunches the
|
||||
// runtime instead of warm-saving it (the run-minted API key is never revoked).
|
||||
expect(warmHandles.size).toBe(0);
|
||||
expect(createCount).toBe(1);
|
||||
|
||||
const second = await execute({
|
||||
|
|
@ -761,9 +762,8 @@ describe("composed ACPX run: host-lane warm handle set", () => {
|
|||
onEvent: async () => {},
|
||||
} as never);
|
||||
expect(second.exitCode).toBe(0);
|
||||
// The compatible second run reuses the warm runtime, so createRuntime is not
|
||||
// called again.
|
||||
expect(createCount).toBe(1);
|
||||
// No warm runtime survived, so the compatible second run re-creates it.
|
||||
expect(createCount).toBe(2);
|
||||
});
|
||||
|
||||
it("closes (does not warm-save) a completed non-persistent runtime, so the next run re-creates", async () => {
|
||||
|
|
@ -819,7 +819,7 @@ describe("composed ACPX run: host-lane warm handle set", () => {
|
|||
expect(createCount).toBe(2);
|
||||
});
|
||||
|
||||
it("emits a skipped acp.handshake event on a warm-handle hit and does not re-create the runtime", async () => {
|
||||
it("runs a fresh acp.handshake on the second run and re-creates the runtime (no warm reuse)", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
let createCount = 0;
|
||||
|
|
@ -863,15 +863,14 @@ describe("composed ACPX run: host-lane warm handle set", () => {
|
|||
},
|
||||
} as never);
|
||||
|
||||
// The warm-hit skips the handshake work: it emits exactly one acp.handshake
|
||||
// event with outcome = skipped and a zero wall time, and never re-creates the
|
||||
// runtime.
|
||||
// Amendment B: the first run closed and relaunched, so no warm handle survives.
|
||||
// The second run runs a fresh acp.handshake (outcome ok, not skipped) and
|
||||
// re-creates the runtime.
|
||||
const handshakeEvents = secondEvents.filter(
|
||||
(event) => event.eventType === "run.startup.step" && event.payload?.step === "acp.handshake",
|
||||
);
|
||||
expect(handshakeEvents).toHaveLength(1);
|
||||
expect(handshakeEvents[0]!.payload?.outcome).toBe("skipped");
|
||||
expect(handshakeEvents[0]!.payload?.durationMs).toBe(0);
|
||||
expect(createCount).toBe(1);
|
||||
expect(handshakeEvents[0]!.payload?.outcome).not.toBe("skipped");
|
||||
expect(createCount).toBe(2);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -0,0 +1,95 @@
|
|||
// Identity split and launch-environment finalization tests (Phase 17).
|
||||
//
|
||||
// Two tests are compile-time: the real assertion is that this file typechecks.
|
||||
// A `@ts-expect-error` marks an assignment that MUST fail; if it starts to
|
||||
// compile, tsc reports the unused directive and the typecheck fails. Each
|
||||
// compile-time assertion lives in a declared, unexecuted function, so vitest
|
||||
// never runs a synthetic value.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { buildSessionFingerprint, finalizeLaunchEnvironment } from "./execute.js";
|
||||
import type {
|
||||
LaunchEnvironment,
|
||||
RunScopedContribution,
|
||||
RunSiteReuseCandidate,
|
||||
SessionFingerprintIdentity,
|
||||
} from "./run-contracts.js";
|
||||
|
||||
// A complete fingerprint identity with representative values for the 17 fields.
|
||||
const SAMPLE_FINGERPRINT_IDENTITY: SessionFingerprintIdentity = {
|
||||
acpxAgent: "claude",
|
||||
agentCommand: "claude",
|
||||
cwd: "/work",
|
||||
mode: "persistent",
|
||||
permissionMode: "approve-all",
|
||||
nonInteractivePermissions: "deny",
|
||||
requestedModel: "",
|
||||
requestedThinkingEffort: "",
|
||||
fastMode: false,
|
||||
remoteExecutionIdentity: null,
|
||||
additionalSourcesIdentity: {},
|
||||
skillsIdentity: {},
|
||||
skillPromptInstructions: "",
|
||||
paperclipClaudeSettings: null,
|
||||
mcpServers: [],
|
||||
secretManifestHash: "0000",
|
||||
adapterEnvHash: "0000",
|
||||
};
|
||||
|
||||
describe("acpx identity split and launch environment", () => {
|
||||
it("test_fingerprint_builder_accepts_only_session_fingerprint_identity", () => {
|
||||
// The builder accepts a fingerprint identity and returns a stable hash.
|
||||
const first = buildSessionFingerprint(SAMPLE_FINGERPRINT_IDENTITY);
|
||||
const second = buildSessionFingerprint(SAMPLE_FINGERPRINT_IDENTITY);
|
||||
expect(first).toBe(second);
|
||||
expect(first).toMatch(/^[0-9a-f]{16}$/);
|
||||
|
||||
// The compile-time assertions live in an unexecuted function.
|
||||
function _assertOnlyIdentity(): void {
|
||||
// An outer-key field is not a fingerprint field, so the builder rejects it.
|
||||
// @ts-expect-error companyId is not a SessionFingerprintIdentity field.
|
||||
buildSessionFingerprint({ ...SAMPLE_FINGERPRINT_IDENTITY, companyId: "c" });
|
||||
// @ts-expect-error taskKey is not a SessionFingerprintIdentity field.
|
||||
buildSessionFingerprint({ ...SAMPLE_FINGERPRINT_IDENTITY, taskKey: "t" });
|
||||
}
|
||||
void _assertOnlyIdentity;
|
||||
});
|
||||
|
||||
it("test_finalize_launch_environment_is_sole_constructor_of_branded_environment", () => {
|
||||
// The finalizer merges every contribution and freezes the launch env.
|
||||
const baseEnv: Record<string, string> = { LAUNCH_ENV_TEST_BASE: "base" };
|
||||
const launchEnvironment = finalizeLaunchEnvironment(baseEnv, [
|
||||
{ scope: "session", env: { LAUNCH_ENV_TEST_CONTRIB: "contrib" } },
|
||||
]);
|
||||
expect(launchEnvironment.env.LAUNCH_ENV_TEST_BASE).toBe("base");
|
||||
expect(launchEnvironment.env.LAUNCH_ENV_TEST_CONTRIB).toBe("contrib");
|
||||
expect(Object.isFrozen(launchEnvironment.env)).toBe(true);
|
||||
|
||||
// No plain object literal can stand in for the branded environment; it lacks
|
||||
// the private brand that only `finalizeLaunchEnvironment` sets.
|
||||
function _assertBrand(): void {
|
||||
// @ts-expect-error missing the private launch-environment brand.
|
||||
const _fake: LaunchEnvironment = { env: {} };
|
||||
void _fake;
|
||||
}
|
||||
void _assertBrand;
|
||||
});
|
||||
|
||||
it("test_run_scoped_contribution_is_not_assignable_to_a_reuse_payload", () => {
|
||||
// A run-scoped contribution and the branded launch environment must never
|
||||
// enter a reuse payload (Amendment B). The reuse candidate types reject both.
|
||||
function _assertNotReusable(
|
||||
contribution: RunScopedContribution,
|
||||
launchEnvironment: LaunchEnvironment,
|
||||
): void {
|
||||
// @ts-expect-error a run-scoped contribution is not a reuse candidate.
|
||||
const _a: RunSiteReuseCandidate = contribution;
|
||||
// @ts-expect-error the branded launch environment is not a reuse candidate.
|
||||
const _b: RunSiteReuseCandidate = launchEnvironment;
|
||||
void _a;
|
||||
void _b;
|
||||
}
|
||||
void _assertNotReusable;
|
||||
expect(true).toBe(true);
|
||||
});
|
||||
});
|
||||
|
|
@ -336,7 +336,7 @@ const ALLOWED_TURN_SPAN_ATTRIBUTE_KEYS = new Set<string>([
|
|||
]);
|
||||
|
||||
describe("shared ACPX engine runtime behavior", () => {
|
||||
it("persists ACP agent process identity before prompting and reuses it for the next warm heartbeat", async () => {
|
||||
it("persists ACP agent process identity before prompting on each run (host lane re-creates, no warm reuse)", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const startedAt = "2026-07-30T07:00:00.000Z";
|
||||
const processPid = 43_210;
|
||||
|
|
@ -419,7 +419,9 @@ describe("shared ACPX engine runtime behavior", () => {
|
|||
} as never);
|
||||
|
||||
expect(second.exitCode).toBe(0);
|
||||
expect(runtimeCreateCount).toBe(1);
|
||||
// Amendment B: the host lane closes and relaunches (no warm reuse), so the
|
||||
// second run re-creates the runtime and re-persists the process identity.
|
||||
expect(runtimeCreateCount).toBe(2);
|
||||
expect(secondOnSpawn).toHaveBeenCalledOnce();
|
||||
expect(turnStartedBeforeProcessIdentity).toBe(false);
|
||||
});
|
||||
|
|
@ -4382,7 +4384,7 @@ describe("ACPX engine per-step startup timing (run.startup.step events)", () =>
|
|||
}
|
||||
});
|
||||
|
||||
it("emits a skipped acp.handshake event when a warm-handle hit skips the handshake", async () => {
|
||||
it("runs a fresh acp.handshake on a compatible second run (host lane re-creates, no warm hit)", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
const warmHandles = new Map();
|
||||
|
|
@ -4408,9 +4410,9 @@ describe("ACPX engine per-step startup timing (run.startup.step events)", () =>
|
|||
onMeta: async () => {},
|
||||
onEvent: async () => {},
|
||||
} as never);
|
||||
// The second run reuses the warm handle, so the handshake does no work. It
|
||||
// must emit exactly one acp.handshake event with outcome = skipped and a
|
||||
// zero wall time, not a misleading zero-work `ok` event.
|
||||
// Amendment B: the first run closed and relaunched (no warm save), so the
|
||||
// compatible second run re-creates the runtime and runs a fresh handshake — one
|
||||
// acp.handshake event whose outcome is not `skipped`.
|
||||
await execute({
|
||||
runId: "run-warm-2",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
|
|
@ -4428,8 +4430,7 @@ describe("ACPX engine per-step startup timing (run.startup.step events)", () =>
|
|||
(event) => event.eventType === "run.startup.step" && event.payload?.step === "acp.handshake",
|
||||
);
|
||||
expect(handshakeEvents).toHaveLength(1);
|
||||
expect(handshakeEvents[0]!.payload?.outcome).toBe("skipped");
|
||||
expect(handshakeEvents[0]!.payload?.durationMs).toBe(0);
|
||||
expect(handshakeEvents[0]!.payload?.outcome).not.toBe("skipped");
|
||||
});
|
||||
|
||||
it("does not emit startup-step events on a local (non-sandbox) run except workspace.resolve", async () => {
|
||||
|
|
@ -4663,7 +4664,7 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
runtimeSessionName: "runtime-session",
|
||||
};
|
||||
|
||||
it("test_handshake_failure_closes_runtime_and_removes_warm_entry", async () => {
|
||||
it("test_persistent_host_failure_closes_recreated_runtime_no_warm_reuse", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
const closeSpy = vi.fn(async () => {});
|
||||
|
|
@ -4709,10 +4710,13 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
onSpawn: async () => {},
|
||||
} as never);
|
||||
expect(first.exitCode).toBe(0);
|
||||
expect(warmHandles.size).toBe(1);
|
||||
// Amendment B: the first clean persistent turn closes and relaunches instead of
|
||||
// warm-saving, so no warm handle survives.
|
||||
expect(warmHandles.size).toBe(0);
|
||||
expect(created).toBe(1);
|
||||
|
||||
// The warm-hit reuses runtime #1 and fails while persisting process identity.
|
||||
// The second run finds no warm handle, so it re-creates the runtime and fails
|
||||
// while persisting process identity.
|
||||
const second = await execute({
|
||||
runId: "warm-handshake-2",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
|
|
@ -4726,11 +4730,11 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
},
|
||||
} as never);
|
||||
|
||||
expect(created).toBe(1);
|
||||
expect(created).toBe(2);
|
||||
expect(second.exitCode).toBe(1);
|
||||
expect(second.resultJson?.phase).toBe("ensure_session");
|
||||
// The reused runtime is closed and the warm entry removed.
|
||||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
// Each run closed its own runtime (one per run); no warm entry survived.
|
||||
expect(closeSpy).toHaveBeenCalledTimes(2);
|
||||
expect(warmHandles.size).toBe(0);
|
||||
});
|
||||
|
||||
|
|
@ -4768,7 +4772,7 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("test_warm_hit_failure_closes_runtime_and_does_not_rearm_idle_timer", async () => {
|
||||
it("test_host_lane_closes_and_relaunches_every_run_and_leaves_no_warm_entry", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
let created = 0;
|
||||
|
|
@ -4816,9 +4820,13 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
onSpawn: async () => {},
|
||||
} as never);
|
||||
expect(first.exitCode).toBe(0);
|
||||
expect(warmHandles.size).toBe(1);
|
||||
// Amendment B: the first clean persistent turn closes runtime #1 and leaves no
|
||||
// warm entry, so the host lane never keeps a live runtime warm.
|
||||
expect(warmHandles.size).toBe(0);
|
||||
expect(closes).toEqual([1]);
|
||||
|
||||
// A warm-hit whose identity-persist step fails reuses runtime #1 (no new create).
|
||||
// The second run finds no warm entry, so it re-creates runtime #2 and fails
|
||||
// while persisting process identity; runtime #2 closes.
|
||||
const second = await execute({
|
||||
runId: "warm-timer-2",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
|
|
@ -4831,15 +4839,12 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
throw new Error("onSpawn boom");
|
||||
},
|
||||
} as never);
|
||||
expect(created).toBe(1);
|
||||
expect(created).toBe(2);
|
||||
expect(second.exitCode).toBe(1);
|
||||
// Runtime #1 is closed and its warm entry removed. The warm-hit already cleared
|
||||
// the idle timer, and the failure does not re-arm it, so the map is empty.
|
||||
expect(closes).toEqual([1]);
|
||||
expect(closes).toEqual([1, 2]);
|
||||
expect(warmHandles.size).toBe(0);
|
||||
|
||||
// A later heartbeat finds no warm entry and constructs a fresh runtime, proving
|
||||
// the failed warm-hit did not leave a re-armed entry behind.
|
||||
// A later heartbeat also finds no warm entry and constructs a fresh runtime.
|
||||
const third = await execute({
|
||||
runId: "warm-timer-3",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
|
|
@ -4851,7 +4856,7 @@ describe("ACPX engine run lifecycle corrections (F2: close the runtime for every
|
|||
onSpawn: async () => {},
|
||||
} as never);
|
||||
expect(third.exitCode).toBe(0);
|
||||
expect(created).toBe(2);
|
||||
expect(created).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,85 @@
|
|||
// Compile-time type tests for the run contracts.
|
||||
//
|
||||
// The real assertion in each test is that this file compiles. A `@ts-expect-error`
|
||||
// marks an assignment that MUST fail to typecheck; if it starts to compile, tsc
|
||||
// reports the unused directive and the typecheck fails. The runtime body only
|
||||
// keeps vitest happy, so each test names the property it proves.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type {
|
||||
AcquiredRunResources,
|
||||
ConsumedRunResources,
|
||||
PreTurnFailedCause,
|
||||
ReadyRunResources,
|
||||
SessionFingerprintIdentity,
|
||||
TurnCause,
|
||||
} from "./run-contracts.js";
|
||||
|
||||
// A type-level equality check. `Equal<A, B>` is true only when A and B are the
|
||||
// same type.
|
||||
type Equal<A, B> =
|
||||
(<T>() => T extends A ? 1 : 2) extends <T>() => T extends B ? 1 : 2 ? true : false;
|
||||
type Expect<T extends true> = T;
|
||||
|
||||
describe("run contracts", () => {
|
||||
it("test_a_turn_cause_is_not_assignable_where_pre_turn_failed_cause_is_required", () => {
|
||||
function requiresPreTurn(_cause: PreTurnFailedCause): void {}
|
||||
const turnCause: TurnCause = { kind: "turn_failed", error: new Error("boom") };
|
||||
|
||||
// A turn cause has a different `kind` and no `phase`, so it is not
|
||||
// assignable where a `PreTurnFailedCause` is required.
|
||||
// @ts-expect-error a TurnCause is not a PreTurnFailedCause.
|
||||
requiresPreTurn(turnCause);
|
||||
|
||||
expect(turnCause.kind).toBe("turn_failed");
|
||||
});
|
||||
|
||||
it("test_only_seal_produces_ready_run_resources_and_only_take_for_settlement_produces_consumed_run_resources", () => {
|
||||
// The assertions live in an unexecuted function so tsc checks the types
|
||||
// without a runtime call on a synthetic ledger.
|
||||
function _assertBrands(ledger: AcquiredRunResources): void {
|
||||
// `seal()` is the only source of a `ReadyRunResources`;
|
||||
// `takeForSettlement()` is the only source of a `ConsumedRunResources`.
|
||||
const ready: ReadyRunResources = ledger.seal([]);
|
||||
const consumed: ConsumedRunResources = ledger.takeForSettlement();
|
||||
|
||||
// The two brands are distinct: neither result is assignable to the other.
|
||||
// @ts-expect-error a ReadyRunResources is not a ConsumedRunResources.
|
||||
const _crossA: ConsumedRunResources = ready;
|
||||
// @ts-expect-error a ConsumedRunResources is not a ReadyRunResources.
|
||||
const _crossB: ReadyRunResources = consumed;
|
||||
|
||||
// A plain object cannot stand in for a branded result; it lacks the
|
||||
// private brand that only `seal()` / `takeForSettlement()` can set.
|
||||
// @ts-expect-error missing the private ready brand.
|
||||
const _fakeReady: ReadyRunResources = {
|
||||
get: () => undefined,
|
||||
scopeOf: () => undefined,
|
||||
has: () => false,
|
||||
sealed: [],
|
||||
};
|
||||
// @ts-expect-error missing the private consumed brand.
|
||||
const _fakeConsumed: ConsumedRunResources = { entries: () => [] };
|
||||
|
||||
void ready;
|
||||
void consumed;
|
||||
void _crossA;
|
||||
void _crossB;
|
||||
void _fakeReady;
|
||||
void _fakeConsumed;
|
||||
}
|
||||
|
||||
void _assertBrands;
|
||||
expect(true).toBe(true);
|
||||
});
|
||||
|
||||
it("test_session_fingerprint_identity_has_no_company_agent_or_task_member", () => {
|
||||
// None of the outer-key parts appear as a fingerprint field.
|
||||
type ForbiddenKey = "companyId" | "agentId" | "taskKey";
|
||||
type LeakedKey = Extract<keyof SessionFingerprintIdentity, ForbiddenKey>;
|
||||
type _noLeak = Expect<Equal<LeakedKey, never>>;
|
||||
|
||||
const _ok: _noLeak = true;
|
||||
expect(_ok).toBe(true);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,509 @@
|
|||
// Contract types for the ACPX run coordinator refactor.
|
||||
//
|
||||
// This module declares the types only. No production code consumes them yet.
|
||||
// Each placeholder type maps to a field that `AcpxPreparedRuntime` carries in
|
||||
// `execute.ts` today. The later coordinator phases move the ownership of those
|
||||
// fields onto these contracts one seam at a time.
|
||||
//
|
||||
// Two brands (`ReadyRunResources`, `ConsumedRunResources`) mark a resource set
|
||||
// that only the resource ledger can produce. Two more brands mark a run-scoped
|
||||
// launch contribution and the finalized launch environment. A brand is a private
|
||||
// `unique symbol`, so no other module can build a branded value by a plain
|
||||
// object literal. Only the module that owns the symbol returns the value.
|
||||
|
||||
import type { AcpRuntime, AcpRuntimeHandle } from "acpx/runtime";
|
||||
import type {
|
||||
AdapterExecutionTarget,
|
||||
AdapterExecutionTargetPaperclipBridgeHandle,
|
||||
AdapterExecutionTargetProcessSessionBridgeHandle,
|
||||
PreparedAdapterExecutionTargetRuntime,
|
||||
} from "@paperclipai/adapter-utils/execution-target";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Resource identity
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* The closed set of run resources the ledger can hold. The set is fixed; a new
|
||||
* resource needs a new member here and a payload in `RunResourcePayloads`.
|
||||
*
|
||||
* `acp_runtime` is a composite of the runtime, the session handle, and the
|
||||
* child process. `runtime.close()` is the one release boundary for all three.
|
||||
*
|
||||
* These are NOT resources and never enter the ledger: `sessionKey`, the
|
||||
* environment lease, the realized execution target, the cleanup registrations,
|
||||
* the reuse-store entries, the three OTel spans, the turn timer, the child
|
||||
* stderr flush state, and the idle timers.
|
||||
*/
|
||||
export type ResourceId =
|
||||
| "acp_runtime"
|
||||
| "staged_runtime"
|
||||
| "control_bridge"
|
||||
| "agent_bridge"
|
||||
| "managed_home"
|
||||
| "staging_lease";
|
||||
|
||||
/**
|
||||
* The outcome the ledger records for a resource when it leaves the ledger.
|
||||
* `finalized` means the run released the resource here. `transferred` means the
|
||||
* run handed ownership to settlement.
|
||||
*/
|
||||
export type ResourceDisposition = "finalized" | "transferred";
|
||||
|
||||
/**
|
||||
* The lifetime a cleanup belongs to.
|
||||
*
|
||||
* A `startup_rollback` cleanup runs only when startup fails, to undo a partial
|
||||
* bring-up. A `per_run` cleanup runs at settlement, at the end of the run. A
|
||||
* successful `seal()` promotes each promotable `startup_rollback` entry to
|
||||
* `per_run`, because the resource now lives through the whole run.
|
||||
*/
|
||||
export type CleanupScope = "startup_rollback" | "per_run";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Resource payloads
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* The `acp_runtime` composite. `runtime.close()` releases the runtime, the
|
||||
* session handle, and the child process together. Maps to the runtime, the
|
||||
* session handle, and the spawned child that `execute.ts` builds in
|
||||
* `buildRuntime`.
|
||||
*/
|
||||
export interface AcpRuntimeResource {
|
||||
readonly runtime: AcpRuntime;
|
||||
readonly sessionHandle: AcpRuntimeHandle;
|
||||
/** The pid of the spawned agent child, or null before the first spawn. */
|
||||
readonly childProcessPid: number | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The staged in-sandbox runtime and its one-time host-side cleanup. Maps to
|
||||
* `AcpxPreparedRuntime.stagedRuntime` and `remoteStagingDispose`.
|
||||
*/
|
||||
export interface StagedRuntimeResource {
|
||||
readonly stagedRuntime: PreparedAdapterExecutionTargetRuntime;
|
||||
/** Remove the host-side staged temp. Fired only when the stage is dropped. */
|
||||
readonly disposeStaged: () => Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* The per-adapter managed-home seam. Maps to
|
||||
* `AcpxPreparedRuntime.remoteManagedHomeTeardown` and `remoteStagingEnvDelta`.
|
||||
*/
|
||||
export interface ManagedHomeResource {
|
||||
/** Per-run copy-back hook. Runs the auth copy-back on every exit path. */
|
||||
readonly teardown: () => Promise<void>;
|
||||
/** The env keys the seam mutated on this run. */
|
||||
readonly envDelta: Record<string, string>;
|
||||
}
|
||||
|
||||
/**
|
||||
* The per-session staging lease. Maps to
|
||||
* `AcpxPreparedRuntime.sessionStagingLeaseRelease`.
|
||||
*/
|
||||
export interface StagingLeaseResource {
|
||||
/** Release the staging lease so a later overlapping run can re-stage. */
|
||||
readonly release: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Maps each resource id to its payload. The `control_bridge` is the Paperclip
|
||||
* control-plane bridge; the `agent_bridge` is the agent process-session bridge.
|
||||
*/
|
||||
export interface RunResourcePayloads {
|
||||
readonly acp_runtime: AcpRuntimeResource;
|
||||
readonly staged_runtime: StagedRuntimeResource;
|
||||
readonly control_bridge: AdapterExecutionTargetPaperclipBridgeHandle;
|
||||
readonly agent_bridge: AdapterExecutionTargetProcessSessionBridgeHandle;
|
||||
readonly managed_home: ManagedHomeResource;
|
||||
readonly staging_lease: StagingLeaseResource;
|
||||
}
|
||||
|
||||
/** One resource the caller registers into the ledger. */
|
||||
export interface RunResourceRegistration<Id extends ResourceId = ResourceId> {
|
||||
readonly id: Id;
|
||||
readonly payload: RunResourcePayloads[Id];
|
||||
readonly scope: CleanupScope;
|
||||
/**
|
||||
* Marks a `startup_rollback` entry that `seal()` promotes to `per_run`.
|
||||
* Default true. Set false to pin the entry as rollback-only. Ignored for a
|
||||
* `per_run` entry.
|
||||
*/
|
||||
readonly promotable?: boolean;
|
||||
/**
|
||||
* The ledger calls this once when it finalizes or transfers the resource. It
|
||||
* records the disposition. It runs no release itself.
|
||||
*/
|
||||
readonly onDisposition?: (disposition: ResourceDisposition) => void;
|
||||
}
|
||||
|
||||
/** One resource the settlement claim carries out of the ledger. */
|
||||
export interface SettledResourceEntry<Id extends ResourceId = ResourceId> {
|
||||
readonly id: Id;
|
||||
readonly payload: RunResourcePayloads[Id];
|
||||
readonly scope: CleanupScope;
|
||||
readonly disposition: ResourceDisposition;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Ledger surface and its two branded results
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
declare const READY_RUN_RESOURCES: unique symbol;
|
||||
declare const CONSUMED_RUN_RESOURCES: unique symbol;
|
||||
|
||||
/**
|
||||
* The sealed, validated resource set. Only `AcquiredRunResources.seal()`
|
||||
* produces this brand. The turn reads its resources through this handle.
|
||||
*/
|
||||
export interface ReadyRunResources {
|
||||
readonly [READY_RUN_RESOURCES]: true;
|
||||
/** The payload for a resource id, or undefined when the slot is empty. */
|
||||
get<Id extends ResourceId>(id: Id): RunResourcePayloads[Id] | undefined;
|
||||
/** The scope of a resource id after promotion, or undefined when empty. */
|
||||
scopeOf(id: ResourceId): CleanupScope | undefined;
|
||||
/** True when the slot holds a resource. */
|
||||
has(id: ResourceId): boolean;
|
||||
/** The required ids that `seal()` validated. */
|
||||
readonly sealed: readonly ResourceId[];
|
||||
}
|
||||
|
||||
/**
|
||||
* The one-time settlement claim. Only `AcquiredRunResources.takeForSettlement()`
|
||||
* produces this brand. Settlement releases the resources this claim carries.
|
||||
*/
|
||||
export interface ConsumedRunResources {
|
||||
readonly [CONSUMED_RUN_RESOURCES]: true;
|
||||
/** The resources, each with its final scope and disposition. */
|
||||
entries(): readonly SettledResourceEntry[];
|
||||
}
|
||||
|
||||
/**
|
||||
* The write surface the startup path holds while it acquires resources.
|
||||
* `register` is the only writer. `seal` and `takeForSettlement` end the write
|
||||
* phase. The state machine is `open -> sealed -> consumed` or `open ->
|
||||
* consumed`.
|
||||
*/
|
||||
export interface AcquiredRunResources {
|
||||
register<Id extends ResourceId>(registration: RunResourceRegistration<Id>): void;
|
||||
seal(required: readonly ResourceId[]): ReadyRunResources;
|
||||
takeForSettlement(): ConsumedRunResources;
|
||||
}
|
||||
|
||||
/** The error the ledger throws on an out-of-order or duplicate operation. */
|
||||
export class LedgerStateError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = "LedgerStateError";
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Startup, turn, and settlement outcomes
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* The four phases a pre-turn failure can occur in. A pre-turn failure happens
|
||||
* after the ledger acquires resources but before the turn runs.
|
||||
*/
|
||||
export interface PreTurnFailedCause {
|
||||
readonly kind: "pre_turn_failed";
|
||||
readonly phase:
|
||||
| "runtime_create"
|
||||
| "handshake"
|
||||
| "session_handle_missing"
|
||||
| "session_configuration";
|
||||
readonly error: Error;
|
||||
}
|
||||
|
||||
/** The turn ran and failed with an error. */
|
||||
export interface TurnFailedCause {
|
||||
readonly kind: "turn_failed";
|
||||
readonly error: Error;
|
||||
}
|
||||
|
||||
/** The caller cancelled the turn. */
|
||||
export interface TurnCancelledCause {
|
||||
readonly kind: "turn_cancelled";
|
||||
readonly reason: string;
|
||||
}
|
||||
|
||||
/** The turn hit its wall-clock timeout. */
|
||||
export interface TurnTimedOutCause {
|
||||
readonly kind: "turn_timed_out";
|
||||
readonly timeoutSec: number;
|
||||
}
|
||||
|
||||
/** A cause that the turn itself produced. Disjoint from `PreTurnFailedCause`. */
|
||||
export type TurnCause = TurnFailedCause | TurnCancelledCause | TurnTimedOutCause;
|
||||
|
||||
/** Any cause that leads a run into settlement. */
|
||||
export type SettlementCause = PreTurnFailedCause | TurnCause;
|
||||
|
||||
/**
|
||||
* Startup produced a ready runtime. The turn can run against these resources.
|
||||
*/
|
||||
export interface StartupReady {
|
||||
readonly kind: "ready";
|
||||
readonly resources: ReadyRunResources;
|
||||
readonly context: AcpRunContext;
|
||||
}
|
||||
|
||||
/**
|
||||
* Startup failed after it acquired resources, so the run must settle them. A
|
||||
* build failure and a partial-bridge failure do NOT reach here — they throw,
|
||||
* and the coordinator owns that rollback (Phase 20).
|
||||
*/
|
||||
export interface StartupSettle {
|
||||
readonly kind: "settle";
|
||||
readonly cause: PreTurnFailedCause;
|
||||
readonly resources: ConsumedRunResources;
|
||||
}
|
||||
|
||||
export type StartupResult = StartupReady | StartupSettle;
|
||||
|
||||
/**
|
||||
* The turn finished and released everything in place. It carries no resources,
|
||||
* so settlement has nothing to do.
|
||||
*/
|
||||
export interface FinalizedTurn {
|
||||
readonly kind: "finalized";
|
||||
}
|
||||
|
||||
/** The turn failed. Settlement releases the carried resources. */
|
||||
export interface FailedTurn {
|
||||
readonly kind: "failed";
|
||||
readonly cause: TurnFailedCause;
|
||||
readonly resources: ConsumedRunResources;
|
||||
}
|
||||
|
||||
/** The caller cancelled the turn. Settlement releases the carried resources. */
|
||||
export interface CancelledTurn {
|
||||
readonly kind: "cancelled";
|
||||
readonly cause: TurnCancelledCause;
|
||||
readonly resources: ConsumedRunResources;
|
||||
}
|
||||
|
||||
/** The turn timed out. Settlement releases the carried resources. */
|
||||
export interface TimedOutTurn {
|
||||
readonly kind: "timed_out";
|
||||
readonly cause: TurnTimedOutCause;
|
||||
readonly resources: ConsumedRunResources;
|
||||
}
|
||||
|
||||
/** The four ways a turn completes. Only `FinalizedTurn` is resourceless. */
|
||||
export type TurnCompletion = FinalizedTurn | FailedTurn | CancelledTurn | TimedOutTurn;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Run site
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** The host site runs on the host. The sandbox site runs in a remote runner. */
|
||||
export type RunSiteKind = "host" | "sandbox";
|
||||
|
||||
/**
|
||||
* The synchronous plan a site produces before the fingerprint build. `plan()`
|
||||
* must be synchronous, because `sessionCwd` is a fingerprint input.
|
||||
*/
|
||||
export interface SitePlan {
|
||||
/**
|
||||
* The cwd the session binds to and the fingerprint reads. `placeWorkspace` is
|
||||
* not its source; the site derives it before it places the workspace.
|
||||
*/
|
||||
readonly sessionCwd: string;
|
||||
/**
|
||||
* The host-only spawn cwd for the relay proxy on the remote process-session
|
||||
* lane. Undefined on every other lane. Maps to `AcpxPreparedRuntime.
|
||||
* hostSpawnCwd`.
|
||||
*/
|
||||
readonly spawnCwd: string | undefined;
|
||||
/** True only for a sandbox target, where the run emits real OTel spans. */
|
||||
readonly tracingEnabled: boolean;
|
||||
}
|
||||
|
||||
/** One referenced project whose staging into the sandbox failed. */
|
||||
export interface ReferencedProjectStagingFailure {
|
||||
readonly projectId: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The result of placing the workspace. Carries the referenced-project staging
|
||||
* failures the run reports back on its result.
|
||||
*/
|
||||
export interface PlacedWorkspace {
|
||||
readonly referencedProjectStagingFailures: readonly ReferencedProjectStagingFailure[];
|
||||
}
|
||||
|
||||
/** The transport handles the site started for the run. */
|
||||
export interface RunSiteTransport {
|
||||
readonly controlBridge: AdapterExecutionTargetPaperclipBridgeHandle | null;
|
||||
readonly agentBridge: AdapterExecutionTargetProcessSessionBridgeHandle | null;
|
||||
}
|
||||
|
||||
/** What the host site's store can save: the live runtime and session handle. */
|
||||
export interface HostReuseCandidate {
|
||||
readonly runtime: AcpRuntime;
|
||||
readonly sessionHandle: AcpRuntimeHandle;
|
||||
}
|
||||
|
||||
/** What the sandbox site's store can save: the staged files. */
|
||||
export interface SandboxReuseCandidate {
|
||||
readonly stagedRuntime: PreparedAdapterExecutionTargetRuntime;
|
||||
}
|
||||
|
||||
export type RunSiteReuseCandidate = HostReuseCandidate | SandboxReuseCandidate;
|
||||
|
||||
/** Construction options for a `SessionReuseStore`. */
|
||||
export interface SessionReuseStoreOptions {
|
||||
/**
|
||||
* The optional per-entry idle timeout, in milliseconds. When set, `evictIdle`
|
||||
* drops an entry that stayed idle longer than this bound.
|
||||
*/
|
||||
readonly idleMs?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* The per-site store that saves a reuse candidate across runs. The store is
|
||||
* generic in the saved type, so a generic `save` cannot know what a site saves.
|
||||
* `RunSite.reuseCandidate()` names the concrete type per site.
|
||||
*/
|
||||
export interface SessionReuseStore<T> {
|
||||
borrow(sessionKey: string): T | undefined;
|
||||
save(sessionKey: string, entry: T): void;
|
||||
/**
|
||||
* Identity-guarded removal that fires the idempotent release. It removes the
|
||||
* key synchronously, so a later reuse never reads a discarded entry. It
|
||||
* returns a promise the caller can await when the release must finish before
|
||||
* the run releases a downstream resource (e.g. the staging lease).
|
||||
*/
|
||||
discard(sessionKey: string): void | Promise<void>;
|
||||
/**
|
||||
* Run-start idle sweep. It removes each stale key synchronously, so
|
||||
* `buildRuntime` never reuses an expired entry. It returns a promise the
|
||||
* caller can await so the sweep completes before the run reads the cache.
|
||||
*/
|
||||
evictIdle(now: number): void | Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* A run site. The host site and the sandbox site differ in what they place,
|
||||
* what transport they start, and what their store saves.
|
||||
*/
|
||||
export interface RunSite<TReuse extends RunSiteReuseCandidate = RunSiteReuseCandidate> {
|
||||
readonly kind: RunSiteKind;
|
||||
/** Synchronous. Runs before the fingerprint build. */
|
||||
plan(context: AcpRunContext): SitePlan;
|
||||
placeWorkspace(context: AcpRunContext): Promise<PlacedWorkspace>;
|
||||
startTransport(context: AcpRunContext): Promise<RunSiteTransport>;
|
||||
reuse(): SessionReuseStore<TReuse>;
|
||||
/** Names the concrete candidate this site's store can save. */
|
||||
reuseCandidate(resources: ReadyRunResources): TReuse;
|
||||
syncBack(context: AcpRunContext): Promise<void>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Identity, launch environment, and run context
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type AcpxRunMode = "persistent" | "oneshot";
|
||||
export type AcpxPermissionMode = "approve-all" | "approve-reads" | "deny-all";
|
||||
export type AcpxNonInteractivePermissions = "deny" | "fail";
|
||||
|
||||
/** One MCP server, in the shape the fingerprint reads. */
|
||||
export interface McpServerIdentity {
|
||||
readonly name: string;
|
||||
readonly url: string;
|
||||
readonly connectionId: string;
|
||||
}
|
||||
|
||||
/** The Paperclip Claude settings the fingerprint reads. */
|
||||
export interface PaperclipClaudeSettingsIdentity {
|
||||
readonly allow: readonly string[];
|
||||
readonly additionalDirectories: readonly string[];
|
||||
readonly defaultMode: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The 17 fields the session fingerprint hashes. Company, agent, and task
|
||||
* identifiers are NOT here; they scope the outer session key only (see
|
||||
* `SessionKeyIdentity`). A change to any field here invalidates a warm handle
|
||||
* or a resumable session and forces a fresh launch.
|
||||
*/
|
||||
export interface SessionFingerprintIdentity {
|
||||
readonly acpxAgent: string;
|
||||
readonly agentCommand: string;
|
||||
readonly cwd: string;
|
||||
readonly mode: AcpxRunMode;
|
||||
readonly permissionMode: AcpxPermissionMode;
|
||||
readonly nonInteractivePermissions: AcpxNonInteractivePermissions;
|
||||
readonly requestedModel: string;
|
||||
readonly requestedThinkingEffort: string;
|
||||
readonly fastMode: boolean;
|
||||
readonly remoteExecutionIdentity: Record<string, unknown> | null;
|
||||
readonly additionalSourcesIdentity: Record<string, unknown>;
|
||||
readonly skillsIdentity: Record<string, unknown>;
|
||||
readonly skillPromptInstructions: string;
|
||||
readonly paperclipClaudeSettings: PaperclipClaudeSettingsIdentity | null;
|
||||
readonly mcpServers: readonly McpServerIdentity[];
|
||||
readonly secretManifestHash: string;
|
||||
readonly adapterEnvHash: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The company, agent, and task parts of the session key. The key form is
|
||||
* `paperclip:companyId:agentId:taskKey:fingerprint`. These parts stay out of
|
||||
* the fingerprint hash.
|
||||
*/
|
||||
export interface SessionKeyIdentity {
|
||||
readonly companyId: string;
|
||||
readonly agentId: string;
|
||||
readonly taskKey: string;
|
||||
}
|
||||
|
||||
declare const RUN_SCOPED_CONTRIBUTION: unique symbol;
|
||||
declare const LAUNCH_ENVIRONMENT: unique symbol;
|
||||
|
||||
/**
|
||||
* A launch-environment contribution that lives for one run only. The run API
|
||||
* key and every bridge token are run-scoped. The brand makes this type
|
||||
* non-transferable: no reuse payload type can hold it (Amendment B).
|
||||
*/
|
||||
export interface RunScopedContribution {
|
||||
readonly scope: "run";
|
||||
readonly env: Record<string, string>;
|
||||
readonly [RUN_SCOPED_CONTRIBUTION]: true;
|
||||
}
|
||||
|
||||
/**
|
||||
* A launch-environment contribution that lives for the whole session. The Codex
|
||||
* auth copy-back material is session-scoped.
|
||||
*/
|
||||
export interface SessionScopedContribution {
|
||||
readonly scope: "session";
|
||||
readonly env: Record<string, string>;
|
||||
}
|
||||
|
||||
export type LaunchEnvironmentContribution = RunScopedContribution | SessionScopedContribution;
|
||||
|
||||
/**
|
||||
* The finalized launch environment. Only `finalizeLaunchEnvironment()`
|
||||
* (Phase 17) produces this brand. No reuse payload type can hold it.
|
||||
*/
|
||||
export interface LaunchEnvironment {
|
||||
readonly [LAUNCH_ENVIRONMENT]: true;
|
||||
readonly env: Readonly<Record<string, string>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* The context the coordinator threads through the run. The site reads the
|
||||
* execution target for its effective capabilities.
|
||||
*/
|
||||
export interface AcpRunContext {
|
||||
readonly executionTarget: AdapterExecutionTarget;
|
||||
readonly fingerprintIdentity: SessionFingerprintIdentity;
|
||||
readonly keyIdentity: SessionKeyIdentity;
|
||||
readonly fingerprint: string;
|
||||
readonly sessionKey: string;
|
||||
readonly launchEnvironment: LaunchEnvironment;
|
||||
}
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import { runAttempt, type RunPlan, type SettlementReason } from "./run-coordinator.js";
|
||||
import { createRunResourceLedger } from "./run-resource-ledger.js";
|
||||
import type {
|
||||
AcquiredRunResources,
|
||||
PreTurnFailedCause,
|
||||
ReadyRunResources,
|
||||
StartupReady,
|
||||
StartupResult,
|
||||
TurnCompletion,
|
||||
} from "./run-contracts.js";
|
||||
|
||||
// The coordinator owns the run routing table. These tests drive it with fakes
|
||||
// for the four steps (startup, turn, settle, reproduce) and assert the order and
|
||||
// the routing, never the engine's real run state.
|
||||
|
||||
const readyResources = {} as ReadyRunResources;
|
||||
|
||||
function readyStartup(): StartupReady {
|
||||
return { kind: "ready", resources: readyResources, context: {} as StartupReady["context"] };
|
||||
}
|
||||
|
||||
function preTurnCause(): PreTurnFailedCause {
|
||||
return { kind: "pre_turn_failed", phase: "handshake", error: new Error("handshake boom") };
|
||||
}
|
||||
|
||||
function finalizedTurn(): TurnCompletion {
|
||||
return { kind: "finalized" };
|
||||
}
|
||||
|
||||
// A minimal plan whose steps push their names into `order`, so a test reads the
|
||||
// exact sequence the coordinator ran.
|
||||
function makePlan(
|
||||
order: string[],
|
||||
overrides: Partial<RunPlan<string>> & { ledger?: AcquiredRunResources } = {},
|
||||
): RunPlan<string> {
|
||||
const ledger = overrides.ledger ?? createRunResourceLedger();
|
||||
return {
|
||||
ledger,
|
||||
startup: overrides.startup ?? (async (): Promise<StartupResult> => {
|
||||
order.push("startup");
|
||||
return readyStartup();
|
||||
}),
|
||||
rollbackStartup: overrides.rollbackStartup ?? ((consumed) => {
|
||||
order.push(`rollback:${consumed.entries().length}`);
|
||||
}),
|
||||
recordDisposition: overrides.recordDisposition ?? ((report) => {
|
||||
order.push(`disposition:${report.records.map((r) => `${r.id}=${r.disposition}`).join(",")}`);
|
||||
}),
|
||||
runTurn: overrides.runTurn ?? (async (): Promise<TurnCompletion> => {
|
||||
order.push("turn");
|
||||
return finalizedTurn();
|
||||
}),
|
||||
settle: overrides.settle ?? (async (reason: SettlementReason) => {
|
||||
order.push(`settle:${reason.kind}`);
|
||||
}),
|
||||
reproduceResult: overrides.reproduceResult ?? (async (outcome) => {
|
||||
order.push(`reproduce:${outcome.kind}`);
|
||||
return "result";
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
describe("ACPX run coordinator", () => {
|
||||
it("test_startup_throw_replays_startup_rollback_entries_and_rethrows", async () => {
|
||||
const order: string[] = [];
|
||||
// Fill a partial ledger with two startup_rollback entries that record their
|
||||
// release order when the rollback replays them.
|
||||
const ledger = createRunResourceLedger();
|
||||
const released: string[] = [];
|
||||
ledger.register({
|
||||
id: "staged_runtime",
|
||||
payload: {
|
||||
stagedRuntime: {} as never,
|
||||
disposeStaged: async () => {
|
||||
released.push("staged_runtime");
|
||||
},
|
||||
},
|
||||
scope: "startup_rollback",
|
||||
});
|
||||
ledger.register({
|
||||
id: "control_bridge",
|
||||
payload: { stop: async () => { released.push("control_bridge"); } } as never,
|
||||
scope: "startup_rollback",
|
||||
});
|
||||
|
||||
const boom = new Error("build boom");
|
||||
const plan = makePlan(order, {
|
||||
ledger,
|
||||
startup: async (): Promise<StartupResult> => {
|
||||
order.push("startup");
|
||||
throw boom;
|
||||
},
|
||||
// Replay every claimed entry through its own release, newest first.
|
||||
rollbackStartup: async (consumed) => {
|
||||
for (const entry of [...consumed.entries()].reverse()) {
|
||||
if (entry.id === "staged_runtime") {
|
||||
await (entry.payload as { disposeStaged: () => Promise<void> }).disposeStaged();
|
||||
} else if (entry.id === "control_bridge") {
|
||||
await (entry.payload as { stop: () => Promise<void> }).stop();
|
||||
}
|
||||
}
|
||||
order.push("rollback");
|
||||
},
|
||||
});
|
||||
|
||||
// The coordinator rethrows the original startup error.
|
||||
await expect(runAttempt(plan)).rejects.toBe(boom);
|
||||
// It replayed the startup_rollback entries through the partial ledger...
|
||||
expect(released).toEqual(["control_bridge", "staged_runtime"]);
|
||||
// ...recorded a disposition report that marks each claimed entry finalized...
|
||||
expect(order).toEqual([
|
||||
"startup",
|
||||
"rollback",
|
||||
"disposition:staged_runtime=finalized,control_bridge=finalized",
|
||||
]);
|
||||
// ...and never ran the turn or the settlement.
|
||||
expect(order).not.toContain("turn");
|
||||
expect(order.some((step) => step.startsWith("settle"))).toBe(false);
|
||||
});
|
||||
|
||||
it("test_settle_path_returns_error_result_after_settlement", async () => {
|
||||
const order: string[] = [];
|
||||
const cause = preTurnCause();
|
||||
const plan = makePlan(order, {
|
||||
startup: async (): Promise<StartupResult> => {
|
||||
order.push("startup");
|
||||
return { kind: "settle", cause, resources: createRunResourceLedger().takeForSettlement() };
|
||||
},
|
||||
settle: async (reason) => {
|
||||
expect(reason.kind).toBe("pre_turn");
|
||||
order.push("settle");
|
||||
},
|
||||
reproduceResult: async (outcome) => {
|
||||
expect(outcome.kind).toBe("pre_turn");
|
||||
order.push("reproduce");
|
||||
return "error-result";
|
||||
},
|
||||
});
|
||||
|
||||
const result = await runAttempt(plan);
|
||||
expect(result).toBe("error-result");
|
||||
// The settle path never runs the turn; it settles first, then reproduces.
|
||||
expect(order).toEqual(["startup", "settle", "reproduce"]);
|
||||
});
|
||||
|
||||
it("test_ready_path_runs_turn_then_settles_then_reproduces_result", async () => {
|
||||
const order: string[] = [];
|
||||
const plan = makePlan(order);
|
||||
|
||||
const result = await runAttempt(plan);
|
||||
expect(result).toBe("result");
|
||||
// The ready path runs the turn, settles, then reproduces — in that order.
|
||||
expect(order).toEqual(["startup", "turn", "settle:turn", "reproduce:turn"]);
|
||||
});
|
||||
|
||||
it("test_coordinator_has_no_catch_around_the_turn", async () => {
|
||||
// `runTurn` is contracted never to reject. This test proves the coordinator
|
||||
// holds no catch around the turn: an (illegal) turn rejection propagates out
|
||||
// of `runAttempt` unchanged, and neither settlement nor result reproduction
|
||||
// runs. A catch around the turn would swallow the rejection and route it into
|
||||
// settlement, so this rejection-propagation is the structural proof.
|
||||
const order: string[] = [];
|
||||
const boom = new Error("turn must never reject");
|
||||
const settleCalls: number[] = [];
|
||||
const reproduceCalls: number[] = [];
|
||||
const plan = makePlan(order, {
|
||||
runTurn: async (): Promise<TurnCompletion> => {
|
||||
throw boom;
|
||||
},
|
||||
settle: async () => {
|
||||
settleCalls.push(1);
|
||||
},
|
||||
reproduceResult: async () => {
|
||||
reproduceCalls.push(1);
|
||||
return "unreachable";
|
||||
},
|
||||
});
|
||||
|
||||
await expect(runAttempt(plan)).rejects.toBe(boom);
|
||||
expect(settleCalls).toHaveLength(0);
|
||||
expect(reproduceCalls).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,125 @@
|
|||
// The run coordinator.
|
||||
//
|
||||
// One `run()` coordinator owns the attempt. It sequences four steps: startup,
|
||||
// turn, settlement, and result reproduction. It holds the routing table as
|
||||
// data:
|
||||
//
|
||||
// * A build failure or a partial-bridge failure throws from startup. The
|
||||
// coordinator's one catch replays the `startup_rollback` entries through the
|
||||
// partial ledger, records the disposition report, and rethrows.
|
||||
// * A runtime-create, handshake, missing-handle, or configuration failure
|
||||
// produces a `settle` result. The coordinator settles the resources, then
|
||||
// reproduces the error result.
|
||||
// * Every other startup outcome produces a `ready` result. The coordinator
|
||||
// runs the turn, settles the resources, then reproduces the turn result.
|
||||
//
|
||||
// The coordinator has no catch around the turn: the turn never rejects and
|
||||
// returns a typed `TurnCompletion` (Phase 21). The external result is reproduced
|
||||
// AFTER settlement, so a caller never observes a result before the run releases
|
||||
// its resources.
|
||||
|
||||
import type {
|
||||
AcquiredRunResources,
|
||||
ConsumedRunResources,
|
||||
PreTurnFailedCause,
|
||||
ResourceDisposition,
|
||||
ResourceId,
|
||||
StartupReady,
|
||||
StartupResult,
|
||||
TurnCompletion,
|
||||
} from "./run-contracts.js";
|
||||
|
||||
/** One resource's final disposition, as the coordinator or settlement records it. */
|
||||
export interface ResourceDispositionRecord {
|
||||
readonly id: ResourceId;
|
||||
readonly disposition: ResourceDisposition;
|
||||
}
|
||||
|
||||
/** The per-resource disposition report the coordinator records at each end. */
|
||||
export interface SettlementDispositionReport {
|
||||
readonly records: readonly ResourceDispositionRecord[];
|
||||
}
|
||||
|
||||
/** The cause the coordinator hands to settlement. A clean turn carries no cause. */
|
||||
export type SettlementReason =
|
||||
| { readonly kind: "pre_turn"; readonly cause: PreTurnFailedCause }
|
||||
| { readonly kind: "turn"; readonly completion: TurnCompletion };
|
||||
|
||||
/** The outcome the coordinator hands to result reproduction. */
|
||||
export type ReproducibleOutcome = SettlementReason;
|
||||
|
||||
/**
|
||||
* The four steps the coordinator sequences, plus the run ledger. The engine
|
||||
* supplies each step; the coordinator owns only the order and the startup
|
||||
* rollback. This keeps the coordinator free of the engine's private run state.
|
||||
*/
|
||||
export interface RunPlan<TResult> {
|
||||
/** The one run ledger for the attempt. The coordinator creates it (see `runAttempt`). */
|
||||
readonly ledger: AcquiredRunResources;
|
||||
/**
|
||||
* Run the startup sequence. It returns a `ready` or a `settle` result, or it
|
||||
* throws on a build failure or a partial-bridge failure.
|
||||
*/
|
||||
startup(): Promise<StartupResult>;
|
||||
/**
|
||||
* Replay the `startup_rollback` entries the partial ledger carries. The
|
||||
* coordinator calls it only on a startup throw, with the claimed entries.
|
||||
*/
|
||||
rollbackStartup(consumed: ConsumedRunResources): Promise<void> | void;
|
||||
/** Record the per-resource disposition report at the end of the run. */
|
||||
recordDisposition(report: SettlementDispositionReport): void;
|
||||
/** Run the turn against the ready resources. It never rejects. */
|
||||
runTurn(ready: StartupReady): Promise<TurnCompletion>;
|
||||
/**
|
||||
* Settle the run's resources for the given reason. The one cleanup owner. A
|
||||
* synchronous return adds no awaited boundary, so a step that already settled
|
||||
* inline keeps the caller's timing unchanged.
|
||||
*/
|
||||
settle(reason: SettlementReason): void | Promise<void>;
|
||||
/** Reproduce the external result after settlement. */
|
||||
reproduceResult(outcome: ReproducibleOutcome): TResult | Promise<TResult>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the attempt through the routing table. The coordinator is the one owner of
|
||||
* the startup-to-settlement order.
|
||||
*/
|
||||
export async function runAttempt<TResult>(plan: RunPlan<TResult>): Promise<TResult> {
|
||||
let startupResult: StartupResult;
|
||||
try {
|
||||
startupResult = await plan.startup();
|
||||
} catch (startupError) {
|
||||
// The coordinator's only catch wraps startup. A build failure and a
|
||||
// partial-bridge failure reach here. Claim the partial ledger, replay its
|
||||
// `startup_rollback` entries, record the disposition report (each entry the
|
||||
// rollback released is `finalized`), and rethrow the original error.
|
||||
const consumed = plan.ledger.takeForSettlement();
|
||||
await plan.rollbackStartup(consumed);
|
||||
plan.recordDisposition(finalizedReport(consumed));
|
||||
throw startupError;
|
||||
}
|
||||
|
||||
if (startupResult.kind === "settle") {
|
||||
// A pre-turn failure after the ledger acquired resources. Settle the
|
||||
// resources, then reproduce the error result AFTER settlement.
|
||||
const reason: SettlementReason = { kind: "pre_turn", cause: startupResult.cause };
|
||||
const settled = plan.settle(reason);
|
||||
if (settled) await settled;
|
||||
return plan.reproduceResult(reason);
|
||||
}
|
||||
|
||||
// The ready path: run the turn, settle, then reproduce the result. There is no
|
||||
// catch around the turn — `runTurn` never rejects.
|
||||
const completion = await plan.runTurn(startupResult);
|
||||
const reason: SettlementReason = { kind: "turn", completion };
|
||||
const settled = plan.settle(reason);
|
||||
if (settled) await settled;
|
||||
return plan.reproduceResult(reason);
|
||||
}
|
||||
|
||||
/** Build a disposition report that marks every claimed entry `finalized`. */
|
||||
function finalizedReport(consumed: ConsumedRunResources): SettlementDispositionReport {
|
||||
return {
|
||||
records: consumed.entries().map((entry) => ({ id: entry.id, disposition: "finalized" as const })),
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,883 @@
|
|||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
prepareAdapterExecutionTargetRuntime,
|
||||
startAdapterExecutionTargetPaperclipBridge,
|
||||
startAdapterExecutionTargetProcessSessionBridge,
|
||||
} from "@paperclipai/adapter-utils/execution-target";
|
||||
import { runChildProcess } from "../server-utils.js";
|
||||
|
||||
// The composed fault matrix.
|
||||
//
|
||||
// Each case drives the whole ACP run and asserts the run's final settlement
|
||||
// disposition report. The report is the authority on which resources the run
|
||||
// acquired and how each one left the run: `finalized` (the run released it) or
|
||||
// `transferred` (the run saved it for reuse). For each case the file asserts:
|
||||
//
|
||||
// 1. the report keys equal the acquired subset of the six resource ids;
|
||||
// 2. each resource carries exactly one disposition;
|
||||
// 3. the external result equals the composed baseline for that exit path.
|
||||
//
|
||||
// The six resource ids are: acp_runtime, staged_runtime, control_bridge,
|
||||
// agent_bridge, managed_home, staging_lease. The host lane acquires only
|
||||
// acp_runtime; the sandbox lane acquires the staging and transport resources too.
|
||||
|
||||
// Wrap the staging seam and both sandbox bridges so a test can stub them without
|
||||
// changing behavior for the other tests.
|
||||
vi.mock("@paperclipai/adapter-utils/execution-target", async (importActual) => {
|
||||
const actual = await importActual<typeof import("@paperclipai/adapter-utils/execution-target")>();
|
||||
return {
|
||||
...actual,
|
||||
prepareAdapterExecutionTargetRuntime: vi.fn(actual.prepareAdapterExecutionTargetRuntime),
|
||||
startAdapterExecutionTargetPaperclipBridge: vi.fn(actual.startAdapterExecutionTargetPaperclipBridge),
|
||||
startAdapterExecutionTargetProcessSessionBridge: vi.fn(actual.startAdapterExecutionTargetProcessSessionBridge),
|
||||
};
|
||||
});
|
||||
|
||||
import {
|
||||
createAcpxEngineExecutor,
|
||||
type AcpxEngineExecutorOptions,
|
||||
} from "./execute.js";
|
||||
import { settleAcpRun, type SettlementSteps } from "./settlement-sequence.js";
|
||||
import { createRunResourceLedger } from "./run-resource-ledger.js";
|
||||
import { LedgerStateError, type ResourceId } from "./run-contracts.js";
|
||||
import type { SettlementDispositionReport } from "./run-coordinator.js";
|
||||
|
||||
const tempRoots: string[] = [];
|
||||
|
||||
async function makeTempRoot() {
|
||||
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-fault-"));
|
||||
tempRoots.push(root);
|
||||
return root;
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(
|
||||
tempRoots.splice(0).map((root) =>
|
||||
fs.rm(root, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }),
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
function createLocalSandboxRunner() {
|
||||
let counter = 0;
|
||||
return {
|
||||
execute: async (input: {
|
||||
command: string;
|
||||
args?: string[];
|
||||
cwd?: string;
|
||||
env?: Record<string, string>;
|
||||
stdin?: string;
|
||||
timeoutMs?: number;
|
||||
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
|
||||
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
|
||||
}) => {
|
||||
counter += 1;
|
||||
const command = input.command === "bash" ? "/bin/bash" : input.command;
|
||||
return await runChildProcess(`acpx-sandbox-run-${counter}`, command, input.args ?? [], {
|
||||
cwd: input.cwd ?? process.cwd(),
|
||||
env: input.env ?? {},
|
||||
stdin: input.stdin,
|
||||
timeoutSec: Math.max(1, Math.ceil((input.timeoutMs ?? 30_000) / 1000)),
|
||||
graceSec: 5,
|
||||
onLog: input.onLog ?? (async () => {}),
|
||||
onSpawn: input.onSpawn
|
||||
? async (meta) => input.onSpawn?.({ pid: meta.pid, startedAt: meta.startedAt })
|
||||
: undefined,
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const okHandle = {
|
||||
backendSessionId: "backend-session",
|
||||
agentSessionId: "agent-session",
|
||||
runtimeSessionName: "runtime-session",
|
||||
};
|
||||
|
||||
function completedTurn() {
|
||||
return {
|
||||
events: (async function* () {})(),
|
||||
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
|
||||
cancel: async () => {},
|
||||
};
|
||||
}
|
||||
|
||||
function throwingTurn() {
|
||||
return {
|
||||
events: (async function* () {
|
||||
throw new Error("turn upstream boom");
|
||||
})(),
|
||||
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
|
||||
cancel: async () => {},
|
||||
};
|
||||
}
|
||||
|
||||
function cancelledTurn() {
|
||||
return {
|
||||
events: (async function* () {})(),
|
||||
result: Promise.resolve({ status: "cancelled", stopReason: "cancelled" }),
|
||||
cancel: async () => {},
|
||||
};
|
||||
}
|
||||
|
||||
// A runtime whose completed turn keeps the session handshake and turn simple.
|
||||
function completedRuntime() {
|
||||
return {
|
||||
ensureSession: async () => okHandle,
|
||||
startTurn: () => completedTurn(),
|
||||
setConfigOption: async () => {},
|
||||
close: async () => {},
|
||||
} as never;
|
||||
}
|
||||
|
||||
// A context whose only prompt-read field throws when read, so the prompt build
|
||||
// fails after the session handshake and before the turn starts.
|
||||
function throwingHandoffContext(): Record<string, unknown> {
|
||||
const context: Record<string, unknown> = {};
|
||||
Object.defineProperty(context, "paperclipSessionHandoffMarkdown", {
|
||||
enumerable: false,
|
||||
get() {
|
||||
throw new Error("prompt build boom");
|
||||
},
|
||||
});
|
||||
return context;
|
||||
}
|
||||
|
||||
// Stub both sandbox bridges, so a case reaches the post-build window with live
|
||||
// handles without running the real bridge transport. `stopRejects` makes the
|
||||
// bridge stop reject during settlement.
|
||||
function stubBridges(options: { stopRejects?: boolean } = {}) {
|
||||
const stop = options.stopRejects
|
||||
? vi.fn(async () => {
|
||||
throw new Error("bridge stop boom");
|
||||
})
|
||||
: vi.fn(async () => {});
|
||||
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementation(async () => {
|
||||
return { env: {}, stop } as never;
|
||||
});
|
||||
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementation(async () => {
|
||||
return { agentCommand: null, stop } as never;
|
||||
});
|
||||
}
|
||||
|
||||
async function setupRemoteSandbox() {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
const localCwd = path.join(root, "worktree");
|
||||
const remoteCwd = path.join(root, "remote-workspace");
|
||||
await fs.mkdir(localCwd, { recursive: true });
|
||||
await fs.mkdir(remoteCwd, { recursive: true });
|
||||
await fs.writeFile(path.join(localCwd, "hello.txt"), "hi", "utf8");
|
||||
const executionTarget = {
|
||||
kind: "remote",
|
||||
transport: "sandbox",
|
||||
providerKey: "fake-plugin",
|
||||
remoteCwd,
|
||||
runner: createLocalSandboxRunner(),
|
||||
};
|
||||
return { root, stateDir, localCwd, remoteCwd, executionTarget };
|
||||
}
|
||||
|
||||
function remoteArgs(
|
||||
stateDir: string,
|
||||
localCwd: string,
|
||||
executionTarget: unknown,
|
||||
overrides: Record<string, unknown> = {},
|
||||
) {
|
||||
return {
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
runtime: {},
|
||||
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
|
||||
context: {},
|
||||
authToken: "real-run-jwt",
|
||||
executionTarget,
|
||||
onLog: async () => {},
|
||||
onMeta: async () => {},
|
||||
onEvent: async () => {},
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
// Seed the managed home through the staging seam and return a per-run copy-back
|
||||
// so the run registers the managed_home resource. `teardownRejects` makes the
|
||||
// copy-back reject during the settlement sync-back.
|
||||
function managedHomeSeed(options: { teardownRejects?: boolean } = {}): AcpxEngineExecutorOptions["prepareRemoteManagedHome"] {
|
||||
return async (input) => {
|
||||
const stagedRuntime = await input.stage([]);
|
||||
return {
|
||||
stagedRuntime,
|
||||
teardown: options.teardownRejects
|
||||
? async () => {
|
||||
throw new Error("copy-back boom");
|
||||
}
|
||||
: async () => {},
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
// Capture the final settlement disposition report the run records.
|
||||
function captureDisposition() {
|
||||
const reports: SettlementDispositionReport[] = [];
|
||||
return {
|
||||
onSettlementDisposition: (report: SettlementDispositionReport) => {
|
||||
reports.push(report);
|
||||
},
|
||||
last(): SettlementDispositionReport | null {
|
||||
return reports.length > 0 ? reports[reports.length - 1]! : null;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// The shared per-case assertion. It proves the report keys equal the acquired
|
||||
// subset, each resource carries exactly one disposition, and the transferred id
|
||||
// (if any) is the only `transferred` entry.
|
||||
function assertDispositionReport(
|
||||
report: SettlementDispositionReport | null,
|
||||
expected: { acquired: ResourceId[]; transferred?: ResourceId | null },
|
||||
) {
|
||||
expect(report, "the run must record a disposition report").not.toBeNull();
|
||||
const records = report!.records;
|
||||
const ids = records.map((record) => record.id);
|
||||
// Each resource carries exactly one disposition (no duplicate id).
|
||||
expect(ids.length, "no resource is reported twice").toBe(new Set(ids).size);
|
||||
// The report keys equal the acquired subset of the six resource ids.
|
||||
expect(new Set(ids)).toEqual(new Set(expected.acquired));
|
||||
const transferred = expected.transferred ?? null;
|
||||
for (const record of records) {
|
||||
expect(["finalized", "transferred"]).toContain(record.disposition);
|
||||
if (record.id === transferred) {
|
||||
expect(record.disposition, `${record.id} must be transferred`).toBe("transferred");
|
||||
} else {
|
||||
expect(record.disposition, `${record.id} must be finalized`).toBe("finalized");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The sandbox-lane resource set before the runtime is created (staging and
|
||||
// transport), and after (adds the runtime). Without a managed-home seed the
|
||||
// managed_home resource is absent.
|
||||
const SANDBOX_PRE_RUNTIME: ResourceId[] = ["staged_runtime", "staging_lease", "control_bridge", "agent_bridge"];
|
||||
const SANDBOX_WITH_RUNTIME: ResourceId[] = [...SANDBOX_PRE_RUNTIME, "acp_runtime"];
|
||||
const SANDBOX_WITH_MANAGED_HOME: ResourceId[] = [
|
||||
"staged_runtime",
|
||||
"managed_home",
|
||||
"staging_lease",
|
||||
"control_bridge",
|
||||
"agent_bridge",
|
||||
"acp_runtime",
|
||||
];
|
||||
|
||||
describe("composed ACPX run fault matrix", () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
// Case 1 — a build failure throws before the ledger acquires any resource.
|
||||
it("case_01_build_failure_throws_and_acquires_nothing", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
vi.mocked(prepareAdapterExecutionTargetRuntime).mockImplementationOnce(async () => {
|
||||
throw new Error("staging boom");
|
||||
});
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
await expect(
|
||||
execute({ runId: "fault-build", ...remoteArgs(stateDir, localCwd, executionTarget) } as never),
|
||||
).rejects.toThrow("staging boom");
|
||||
|
||||
assertDispositionReport(capture.last(), { acquired: [] });
|
||||
});
|
||||
|
||||
// Case 2 — a partial bridge failure throws; the staged entry and the lease are
|
||||
// finalized (the rollback removed the borrowed staged entry).
|
||||
it("case_02_partial_bridge_failure_finalizes_staged_and_lease", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
const capture = captureDisposition();
|
||||
const stop = vi.fn(async () => {});
|
||||
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementationOnce(async () => {
|
||||
throw new Error("paperclip bridge boom");
|
||||
});
|
||||
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementationOnce(
|
||||
async () => ({ agentCommand: null, stop }) as never,
|
||||
);
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
await expect(
|
||||
execute({ runId: "fault-bridge", ...remoteArgs(stateDir, localCwd, executionTarget) } as never),
|
||||
).rejects.toThrow("paperclip bridge boom");
|
||||
|
||||
assertDispositionReport(capture.last(), { acquired: ["staged_runtime", "staging_lease"] });
|
||||
});
|
||||
|
||||
// Case 3 — a runtime-create failure settles the staging and transport resources.
|
||||
// The runtime never enters the ledger.
|
||||
it("case_03_runtime_create_failure_finalizes_staging_and_transport", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => {
|
||||
throw new Error("createRuntime boom");
|
||||
},
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-create",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("create_runtime");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_PRE_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 4 — a handshake failure closes the runtime; every resource finalizes.
|
||||
it("case_04_handshake_failure_closes_runtime", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => {
|
||||
throw new Error("ensureSession boom");
|
||||
},
|
||||
startTurn: () => completedTurn(),
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-handshake",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("ensure_session");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 5 — a missing session handle closes the runtime; the result carries a
|
||||
// null clearSession and no errorMeta.
|
||||
it("case_05_missing_handle_closes_runtime_null_error", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => undefined,
|
||||
startTurn: () => completedTurn(),
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-missing-handle",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("ensure_session");
|
||||
expect(result.errorCode).toBe("acpx_runtime_error");
|
||||
expect(result.clearSession).toBeUndefined();
|
||||
expect(result.errorMeta).toBeUndefined();
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 6 — a configuration failure closes the runtime.
|
||||
it("case_06_configuration_failure_closes_runtime", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => okHandle,
|
||||
startTurn: () => completedTurn(),
|
||||
setConfigOption: async () => {
|
||||
throw new Error("setConfigOption boom");
|
||||
},
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-configure",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget, {
|
||||
config: {
|
||||
agent: "custom",
|
||||
agentCommand: "node ./fake-acp.js",
|
||||
model: "custom-model-x",
|
||||
stateDir,
|
||||
cwd: localCwd,
|
||||
},
|
||||
}),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("configure_session");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 7 — a prompt-build failure fails the turn in the prepare phase.
|
||||
it("case_07_prompt_build_failure_finalizes_every_resource", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-prepare",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget, { context: throwingHandoffContext() }),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("prepare_turn");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 8 — a resume retry runs ensureSession twice but registers the runtime
|
||||
// once, so the acquired set has no doubled resource.
|
||||
it("case_08_resume_retry_does_not_double_resources", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
const cwd = path.join(root, "worktree");
|
||||
await fs.mkdir(cwd, { recursive: true });
|
||||
const config = { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd };
|
||||
|
||||
// A first clean host run mints the session params the second run resumes from.
|
||||
const firstExecute = createAcpxEngineExecutor({ createRuntime: () => completedRuntime() });
|
||||
const first = await firstExecute({
|
||||
runId: "fault-resume-a",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
runtime: {},
|
||||
config,
|
||||
context: {},
|
||||
onLog: async () => {},
|
||||
onMeta: async () => {},
|
||||
} as never);
|
||||
expect(first.exitCode).toBe(0);
|
||||
|
||||
// The second run resumes: the resume ensureSession fails, the fresh retry
|
||||
// succeeds. The runtime is created once, so acp_runtime registers once.
|
||||
const capture = captureDisposition();
|
||||
let created = 0;
|
||||
const secondExecute = createAcpxEngineExecutor({
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
createRuntime: () => {
|
||||
created += 1;
|
||||
return {
|
||||
ensureSession: async (session: Record<string, unknown>) => {
|
||||
if (session.resumeSessionId) throw new Error("resume session not found");
|
||||
return okHandle;
|
||||
},
|
||||
startTurn: () => completedTurn(),
|
||||
setConfigOption: async () => {},
|
||||
close: async () => {},
|
||||
} as never;
|
||||
},
|
||||
});
|
||||
const second = await secondExecute({
|
||||
runId: "fault-resume-b",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
runtime: { sessionParams: (first as { sessionParams?: unknown }).sessionParams },
|
||||
config,
|
||||
context: {},
|
||||
onLog: async () => {},
|
||||
onMeta: async () => {},
|
||||
} as never);
|
||||
|
||||
expect(second.exitCode).toBe(0);
|
||||
// The runtime was created exactly once across the resume and the fresh retry.
|
||||
expect(created).toBe(1);
|
||||
// The host lane acquires only the runtime, and it appears once.
|
||||
assertDispositionReport(capture.last(), { acquired: ["acp_runtime"] });
|
||||
});
|
||||
|
||||
// Case 9 — a turn exception finalizes every resource (the staged runtime is
|
||||
// discarded, not transferred).
|
||||
it("case_09_turn_exception_discards_staged_runtime", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => okHandle,
|
||||
startTurn: () => throwingTurn(),
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-turn",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("turn");
|
||||
expect(result.errorCode).toBe("acpx_turn_failed");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 10 — a turn timeout finalizes every resource.
|
||||
it("case_10_turn_timeout_finalizes_every_resource", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
let releaseTurn: (() => void) | null = null;
|
||||
const turnCancelled = new Promise<void>((resolve) => {
|
||||
releaseTurn = resolve;
|
||||
});
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => okHandle,
|
||||
startTurn: () => ({
|
||||
events: (async function* () {
|
||||
await turnCancelled;
|
||||
})(),
|
||||
result: turnCancelled.then(() => ({ status: "cancelled", stopReason: "cancelled" })),
|
||||
cancel: async () => {
|
||||
releaseTurn?.();
|
||||
},
|
||||
}),
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-timeout",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget, {
|
||||
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd, timeoutSec: 1 },
|
||||
}),
|
||||
} as never);
|
||||
|
||||
expect(result.timedOut).toBe(true);
|
||||
expect(result.errorCode).toBe("acpx_timeout");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
}, 15_000);
|
||||
|
||||
// Case 11 — a cancelled turn finalizes every resource.
|
||||
it("case_11_turn_cancellation_finalizes_every_resource", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => okHandle,
|
||||
startTurn: () => cancelledTurn(),
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-cancel",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.status).toBe("cancelled");
|
||||
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
|
||||
});
|
||||
|
||||
// Case 12 — a clean success saves the staged runtime (transferred); every other
|
||||
// resource finalizes. A managed-home seed exercises the full six-resource set.
|
||||
it("case_12_clean_success_transfers_staged_runtime", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
prepareRemoteManagedHome: managedHomeSeed(),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-success",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.resultJson?.status).toBe("completed");
|
||||
assertDispositionReport(capture.last(), {
|
||||
acquired: SANDBOX_WITH_MANAGED_HOME,
|
||||
transferred: "staged_runtime",
|
||||
});
|
||||
});
|
||||
|
||||
// Case 13 — a bridge stop that rejects during settlement never changes the
|
||||
// result or the report (allSettled absorbs the fault).
|
||||
it("case_13_bridge_stop_reject_does_not_change_result", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges({ stopRejects: true });
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-bridge-stop",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.resultJson?.status).toBe("completed");
|
||||
assertDispositionReport(capture.last(), {
|
||||
acquired: SANDBOX_WITH_RUNTIME,
|
||||
transferred: "staged_runtime",
|
||||
});
|
||||
});
|
||||
|
||||
// Case 14 — a sync-back copy-back that rejects during settlement never changes
|
||||
// the result or the report.
|
||||
it("case_14_sync_back_failure_does_not_change_result", async () => {
|
||||
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
|
||||
stubBridges();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
stagedRuntimes: new Map(),
|
||||
stagingLocks: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
prepareRemoteManagedHome: managedHomeSeed({ teardownRejects: true }),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-sync-back",
|
||||
...remoteArgs(stateDir, localCwd, executionTarget),
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.resultJson?.status).toBe("completed");
|
||||
assertDispositionReport(capture.last(), {
|
||||
acquired: SANDBOX_WITH_MANAGED_HOME,
|
||||
transferred: "staged_runtime",
|
||||
});
|
||||
});
|
||||
|
||||
// Case 15 — a concurrent double settlement is a structural error: the second
|
||||
// claim of the ledger throws a LedgerStateError.
|
||||
it("case_15_concurrent_double_settlement_throws_ledger_state_error", async () => {
|
||||
const ledger = createRunResourceLedger();
|
||||
ledger.register({
|
||||
id: "acp_runtime",
|
||||
payload: { runtime: {}, sessionHandle: {}, childProcessPid: null } as never,
|
||||
scope: "per_run",
|
||||
});
|
||||
const noopSteps: SettlementSteps = {
|
||||
reuseCandidate: () => null,
|
||||
endSession: () => {},
|
||||
settleReuse: () => {},
|
||||
stopTransport: () => {},
|
||||
syncBack: () => {},
|
||||
releaseStagingLease: () => {},
|
||||
recordError: () => {},
|
||||
};
|
||||
|
||||
// The first settlement claims the ledger and produces the report.
|
||||
const report = await settleAcpRun(ledger, null, noopSteps);
|
||||
assertDispositionReport(report, { acquired: ["acp_runtime"] });
|
||||
|
||||
// A second settlement of the same ledger is a structural error.
|
||||
await expect(settleAcpRun(ledger, null, noopSteps)).rejects.toBeInstanceOf(LedgerStateError);
|
||||
});
|
||||
|
||||
// Case 16 — the host-lane credential gate. Constructed gate-passed: the runtime
|
||||
// transfers and never closes. Live host path (gate failed): the runtime closes
|
||||
// and never transfers.
|
||||
it("case_16_host_credential_gate_transfer_versus_close", async () => {
|
||||
// Constructed gate-passed condition: a host candidate with no live run-scoped
|
||||
// credential. The reuse decision saves, so the runtime transfers and endSession
|
||||
// never closes it.
|
||||
const ledger = createRunResourceLedger();
|
||||
ledger.register({
|
||||
id: "acp_runtime",
|
||||
payload: { runtime: {}, sessionHandle: {}, childProcessPid: null } as never,
|
||||
scope: "per_run",
|
||||
});
|
||||
let closed = false;
|
||||
const gatePassedSteps: SettlementSteps = {
|
||||
reuseCandidate: () => ({ kind: "host", causePermitsSave: true, liveRunScopedCredentials: [] }),
|
||||
endSession: (_slots, decision) => {
|
||||
if (decision.kind !== "save") closed = true;
|
||||
},
|
||||
settleReuse: () => {},
|
||||
stopTransport: () => {},
|
||||
syncBack: () => {},
|
||||
releaseStagingLease: () => {},
|
||||
recordError: () => {},
|
||||
};
|
||||
const gatePassedReport = await settleAcpRun(ledger, null, gatePassedSteps);
|
||||
expect(closed).toBe(false);
|
||||
assertDispositionReport(gatePassedReport, { acquired: ["acp_runtime"], transferred: "acp_runtime" });
|
||||
|
||||
// Live host path: the engine populates the run-scoped credential, so the gate
|
||||
// fails, the runtime closes, and it never transfers.
|
||||
const root = await makeTempRoot();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
const result = await execute({
|
||||
runId: "fault-host-gate",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
runtime: {},
|
||||
config: {
|
||||
agent: "custom",
|
||||
agentCommand: "node ./fake-acp.js",
|
||||
stateDir: path.join(root, "state"),
|
||||
mode: "persistent",
|
||||
warmHandleIdleMs: 60_000,
|
||||
},
|
||||
context: {},
|
||||
onLog: async () => {},
|
||||
onMeta: async () => {},
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
// The live host path closes and never transfers the runtime.
|
||||
assertDispositionReport(capture.last(), { acquired: ["acp_runtime"], transferred: null });
|
||||
});
|
||||
|
||||
// Case 17 — a host-lane turn failure finalizes (closes) the runtime.
|
||||
it("case_17_host_turn_failure_finalizes_runtime", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const capture = captureDisposition();
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
createRuntime: () =>
|
||||
({
|
||||
ensureSession: async () => okHandle,
|
||||
startTurn: () => throwingTurn(),
|
||||
setConfigOption: async () => {},
|
||||
close: async () => {},
|
||||
}) as never,
|
||||
onSettlementDisposition: capture.onSettlementDisposition,
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "fault-host-turn-fail",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
runtime: {},
|
||||
config: {
|
||||
agent: "custom",
|
||||
agentCommand: "node ./fake-acp.js",
|
||||
stateDir: path.join(root, "state"),
|
||||
mode: "persistent",
|
||||
warmHandleIdleMs: 60_000,
|
||||
},
|
||||
context: {},
|
||||
onLog: async () => {},
|
||||
onMeta: async () => {},
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("turn");
|
||||
assertDispositionReport(capture.last(), { acquired: ["acp_runtime"], transferred: null });
|
||||
});
|
||||
});
|
||||
|
||||
describe("composed ACPX run per-phase telemetry", () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
it("emits a run.phase.timing event for each lifecycle phase a clean host run reaches", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const events: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
|
||||
const execute = createAcpxEngineExecutor({
|
||||
warmHandles: new Map(),
|
||||
createRuntime: () => completedRuntime(),
|
||||
});
|
||||
|
||||
const result = await execute({
|
||||
runId: "phase-telemetry",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
runtime: {},
|
||||
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
|
||||
context: {},
|
||||
onLog: async () => {},
|
||||
onMeta: async () => {},
|
||||
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
|
||||
events.push(event);
|
||||
},
|
||||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
const phases = events
|
||||
.filter((event) => event.eventType === "run.phase.timing")
|
||||
.map((event) => String(event.payload?.phase));
|
||||
// A clean host run runs the runtime, session, prepare, turn, and settlement
|
||||
// phases. Each phase emits a phase-timing event.
|
||||
for (const phase of ["create_runtime", "ensure_session", "prepare_turn", "turn", "end_session"]) {
|
||||
expect(phases, `missing phase telemetry for ${phase}`).toContain(phase);
|
||||
}
|
||||
// Every phase-timing event carries exactly the three closed fields.
|
||||
for (const event of events.filter((entry) => entry.eventType === "run.phase.timing")) {
|
||||
expect(Object.keys(event.payload ?? {}).sort()).toEqual(["durationMs", "outcome", "phase"]);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,185 @@
|
|||
import { describe, expect, it, vi } from "vitest";
|
||||
import { createRunResourceLedger, createScopedCleanupRegistry } from "./run-resource-ledger.js";
|
||||
import { LedgerStateError } from "./run-contracts.js";
|
||||
import type { RunResourcePayloads } from "./run-contracts.js";
|
||||
|
||||
// A typed placeholder payload for a resource whose value the test never reads.
|
||||
function stub<T>(): T {
|
||||
return undefined as unknown as T;
|
||||
}
|
||||
|
||||
describe("run resource ledger", () => {
|
||||
it("test_register_throws_on_filled_slot", () => {
|
||||
const ledger = createRunResourceLedger();
|
||||
ledger.register({ id: "staging_lease", scope: "per_run", payload: { release: () => {} } });
|
||||
|
||||
expect(() =>
|
||||
ledger.register({ id: "staging_lease", scope: "per_run", payload: { release: () => {} } }),
|
||||
).toThrow(LedgerStateError);
|
||||
});
|
||||
|
||||
it("test_register_throws_after_seal_and_after_take", () => {
|
||||
const sealed = createRunResourceLedger();
|
||||
sealed.register({ id: "staging_lease", scope: "per_run", payload: { release: () => {} } });
|
||||
sealed.seal(["staging_lease"]);
|
||||
expect(() =>
|
||||
sealed.register({
|
||||
id: "managed_home",
|
||||
scope: "per_run",
|
||||
payload: stub<RunResourcePayloads["managed_home"]>(),
|
||||
}),
|
||||
).toThrow(LedgerStateError);
|
||||
|
||||
const taken = createRunResourceLedger();
|
||||
taken.register({ id: "staging_lease", scope: "per_run", payload: { release: () => {} } });
|
||||
taken.takeForSettlement();
|
||||
expect(() =>
|
||||
taken.register({
|
||||
id: "managed_home",
|
||||
scope: "per_run",
|
||||
payload: stub<RunResourcePayloads["managed_home"]>(),
|
||||
}),
|
||||
).toThrow(LedgerStateError);
|
||||
});
|
||||
|
||||
it("test_seal_validates_site_declared_required_slots", () => {
|
||||
const ledger = createRunResourceLedger();
|
||||
ledger.register({
|
||||
id: "acp_runtime",
|
||||
scope: "per_run",
|
||||
payload: stub<RunResourcePayloads["acp_runtime"]>(),
|
||||
});
|
||||
|
||||
// The site declares control_bridge required, but it is not registered, so
|
||||
// seal throws and leaves the ledger open.
|
||||
expect(() => ledger.seal(["acp_runtime", "control_bridge"])).toThrow(LedgerStateError);
|
||||
|
||||
// The failed seal left the ledger open, so the missing slot can be filled.
|
||||
ledger.register({
|
||||
id: "control_bridge",
|
||||
scope: "per_run",
|
||||
payload: stub<RunResourcePayloads["control_bridge"]>(),
|
||||
});
|
||||
const ready = ledger.seal(["acp_runtime", "control_bridge"]);
|
||||
|
||||
expect(ready.has("acp_runtime")).toBe(true);
|
||||
expect(ready.has("control_bridge")).toBe(true);
|
||||
expect([...ready.sealed]).toEqual(["acp_runtime", "control_bridge"]);
|
||||
});
|
||||
|
||||
it("test_seal_promotes_startup_rollback_entries_to_per_run", () => {
|
||||
const ledger = createRunResourceLedger();
|
||||
// Promotable by default.
|
||||
ledger.register({
|
||||
id: "staged_runtime",
|
||||
scope: "startup_rollback",
|
||||
payload: stub<RunResourcePayloads["staged_runtime"]>(),
|
||||
});
|
||||
// Pinned as rollback-only.
|
||||
ledger.register({
|
||||
id: "control_bridge",
|
||||
scope: "startup_rollback",
|
||||
promotable: false,
|
||||
payload: stub<RunResourcePayloads["control_bridge"]>(),
|
||||
});
|
||||
ledger.register({
|
||||
id: "acp_runtime",
|
||||
scope: "per_run",
|
||||
payload: stub<RunResourcePayloads["acp_runtime"]>(),
|
||||
});
|
||||
|
||||
const ready = ledger.seal(["staged_runtime", "control_bridge", "acp_runtime"]);
|
||||
|
||||
expect(ready.scopeOf("staged_runtime")).toBe("per_run");
|
||||
expect(ready.scopeOf("control_bridge")).toBe("startup_rollback");
|
||||
expect(ready.scopeOf("acp_runtime")).toBe("per_run");
|
||||
});
|
||||
|
||||
it("test_take_for_settlement_is_synchronous_one_time_claim", () => {
|
||||
const ledger = createRunResourceLedger();
|
||||
ledger.register({
|
||||
id: "acp_runtime",
|
||||
scope: "per_run",
|
||||
payload: stub<RunResourcePayloads["acp_runtime"]>(),
|
||||
});
|
||||
|
||||
const consumed = ledger.takeForSettlement();
|
||||
|
||||
// Synchronous: the claim comes back directly, not as a promise.
|
||||
expect(consumed).not.toBeInstanceOf(Promise);
|
||||
expect(consumed.entries().map((entry) => entry.id)).toContain("acp_runtime");
|
||||
|
||||
// One-time: a second claim throws.
|
||||
expect(() => ledger.takeForSettlement()).toThrow(LedgerStateError);
|
||||
});
|
||||
|
||||
it("test_second_take_throws_ledger_state_error_before_any_side_effect", () => {
|
||||
const ledger = createRunResourceLedger();
|
||||
const onDisposition = vi.fn();
|
||||
ledger.register({
|
||||
id: "staging_lease",
|
||||
scope: "per_run",
|
||||
payload: { release: () => {} },
|
||||
onDisposition,
|
||||
});
|
||||
|
||||
const first = ledger.takeForSettlement();
|
||||
expect(first.entries().map((entry) => entry.id)).toEqual(["staging_lease"]);
|
||||
expect(onDisposition).toHaveBeenCalledTimes(1);
|
||||
expect(onDisposition).toHaveBeenCalledWith("transferred");
|
||||
|
||||
let thrown: unknown;
|
||||
try {
|
||||
ledger.takeForSettlement();
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
|
||||
expect(thrown).toBeInstanceOf(LedgerStateError);
|
||||
// The second claim threw before it marked a disposition a second time.
|
||||
expect(onDisposition).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("scoped cleanup registry runs and promotes cleanups by scope", async () => {
|
||||
const registry = createScopedCleanupRegistry();
|
||||
const order: string[] = [];
|
||||
registry.add("startup_rollback", () => {
|
||||
order.push("rollback-a");
|
||||
});
|
||||
registry.add("startup_rollback", async () => {
|
||||
order.push("rollback-b");
|
||||
});
|
||||
registry.add("per_run", () => {
|
||||
order.push("per-run");
|
||||
});
|
||||
expect(registry.size("startup_rollback")).toBe(2);
|
||||
|
||||
registry.promote("startup_rollback", "per_run");
|
||||
expect(registry.size("startup_rollback")).toBe(0);
|
||||
expect(registry.size("per_run")).toBe(3);
|
||||
|
||||
await registry.run("per_run");
|
||||
|
||||
// Reverse registration order (last in, first out).
|
||||
expect(order).toEqual(["rollback-b", "rollback-a", "per-run"]);
|
||||
expect(registry.size("per_run")).toBe(0);
|
||||
});
|
||||
|
||||
it("scoped cleanup registry runs every cleanup even when one fails", async () => {
|
||||
const registry = createScopedCleanupRegistry();
|
||||
const ran: string[] = [];
|
||||
registry.add("per_run", () => {
|
||||
ran.push("first");
|
||||
});
|
||||
registry.add("per_run", () => {
|
||||
throw new Error("boom");
|
||||
});
|
||||
registry.add("per_run", () => {
|
||||
ran.push("third");
|
||||
});
|
||||
|
||||
await expect(registry.run("per_run")).rejects.toBeInstanceOf(AggregateError);
|
||||
// The failing cleanup did not skip the others.
|
||||
expect(ran).toEqual(["third", "first"]);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,201 @@
|
|||
// The run resource ledger and the scoped cleanup registry.
|
||||
//
|
||||
// The ledger is the single owner of the run resources between acquisition and
|
||||
// settlement. `register` is the only writer. `seal` ends acquisition and
|
||||
// validates the required set. `takeForSettlement` makes the one-time claim that
|
||||
// hands the resources to settlement. The state machine is `open -> sealed ->
|
||||
// consumed` or `open -> consumed`.
|
||||
//
|
||||
// The scoped cleanup registry holds cleanup thunks by lifetime. The coordinator
|
||||
// runs the `startup_rollback` scope when startup fails and the `per_run` scope
|
||||
// at settlement. A successful seal promotes the rollback cleanups to per_run.
|
||||
|
||||
import { LedgerStateError } from "./run-contracts.js";
|
||||
import type {
|
||||
AcquiredRunResources,
|
||||
CleanupScope,
|
||||
ConsumedRunResources,
|
||||
ReadyRunResources,
|
||||
ResourceDisposition,
|
||||
ResourceId,
|
||||
RunResourcePayloads,
|
||||
RunResourceRegistration,
|
||||
SettledResourceEntry,
|
||||
} from "./run-contracts.js";
|
||||
|
||||
type LedgerState = "open" | "sealed" | "consumed";
|
||||
|
||||
interface LedgerSlot {
|
||||
readonly id: ResourceId;
|
||||
readonly payload: RunResourcePayloads[ResourceId];
|
||||
scope: CleanupScope;
|
||||
readonly promotable: boolean;
|
||||
readonly onDisposition?: (disposition: ResourceDisposition) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a run resource ledger. The ledger starts `open`. `register` fills at
|
||||
* most one slot per resource id. `seal` validates the required set and promotes
|
||||
* promotable `startup_rollback` entries to `per_run`. `takeForSettlement` makes
|
||||
* the one-time settlement claim.
|
||||
*/
|
||||
export function createRunResourceLedger(): AcquiredRunResources {
|
||||
const slots = new Map<ResourceId, LedgerSlot>();
|
||||
let state: LedgerState = "open";
|
||||
let sealedRequired: readonly ResourceId[] = [];
|
||||
|
||||
function readyView(): ReadyRunResources {
|
||||
const view = {
|
||||
get(id: ResourceId): RunResourcePayloads[ResourceId] | undefined {
|
||||
return slots.get(id)?.payload;
|
||||
},
|
||||
scopeOf(id: ResourceId): CleanupScope | undefined {
|
||||
return slots.get(id)?.scope;
|
||||
},
|
||||
has(id: ResourceId): boolean {
|
||||
return slots.has(id);
|
||||
},
|
||||
sealed: sealedRequired,
|
||||
};
|
||||
return view as unknown as ReadyRunResources;
|
||||
}
|
||||
|
||||
return {
|
||||
register(registration: RunResourceRegistration): void {
|
||||
if (state !== "open") {
|
||||
throw new LedgerStateError(
|
||||
state === "sealed"
|
||||
? "cannot register a resource after the ledger is sealed"
|
||||
: "cannot register a resource after the ledger is taken for settlement",
|
||||
);
|
||||
}
|
||||
if (slots.has(registration.id)) {
|
||||
throw new LedgerStateError(`resource slot "${registration.id}" is already filled`);
|
||||
}
|
||||
slots.set(registration.id, {
|
||||
id: registration.id,
|
||||
payload: registration.payload,
|
||||
scope: registration.scope,
|
||||
promotable: registration.promotable ?? true,
|
||||
onDisposition: registration.onDisposition,
|
||||
});
|
||||
},
|
||||
|
||||
seal(required: readonly ResourceId[]): ReadyRunResources {
|
||||
if (state !== "open") {
|
||||
throw new LedgerStateError(
|
||||
state === "sealed"
|
||||
? "the ledger is already sealed"
|
||||
: "cannot seal the ledger after it is taken for settlement",
|
||||
);
|
||||
}
|
||||
// Validate the required set first. A failed seal must leave the ledger
|
||||
// open and unchanged, so the caller can register the missing slot and
|
||||
// seal again.
|
||||
const missing = required.filter((id) => !slots.has(id));
|
||||
if (missing.length > 0) {
|
||||
throw new LedgerStateError(`seal is missing required resource(s): ${missing.join(", ")}`);
|
||||
}
|
||||
// Promote every promotable startup_rollback entry to per_run in one pass.
|
||||
// The resource now lives through the whole run, so settlement owns it.
|
||||
for (const slot of slots.values()) {
|
||||
if (slot.scope === "startup_rollback" && slot.promotable) {
|
||||
slot.scope = "per_run";
|
||||
}
|
||||
}
|
||||
state = "sealed";
|
||||
sealedRequired = [...required];
|
||||
return readyView();
|
||||
},
|
||||
|
||||
takeForSettlement(): ConsumedRunResources {
|
||||
// The one-time claim. Check the state first, so a second claim throws
|
||||
// before it can mark a disposition or hand the resources out again.
|
||||
if (state === "consumed") {
|
||||
throw new LedgerStateError("run resources are already taken for settlement");
|
||||
}
|
||||
state = "consumed";
|
||||
const entries: SettledResourceEntry[] = [];
|
||||
for (const slot of slots.values()) {
|
||||
slot.onDisposition?.("transferred");
|
||||
entries.push({
|
||||
id: slot.id,
|
||||
payload: slot.payload,
|
||||
scope: slot.scope,
|
||||
disposition: "transferred",
|
||||
});
|
||||
}
|
||||
const consumed = { entries: () => entries };
|
||||
return consumed as unknown as ConsumedRunResources;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A registry of cleanup thunks, grouped by lifetime scope. The coordinator adds
|
||||
* a cleanup under a scope, promotes a scope's cleanups to another scope, and
|
||||
* runs a scope's cleanups.
|
||||
*/
|
||||
export interface ScopedCleanupRegistry {
|
||||
/** Add a cleanup under a scope. */
|
||||
add(scope: CleanupScope, cleanup: () => void | Promise<void>): void;
|
||||
/** Move every cleanup from one scope to another, in registration order. */
|
||||
promote(from: CleanupScope, to: CleanupScope): void;
|
||||
/** The count of cleanups in a scope. */
|
||||
size(scope: CleanupScope): number;
|
||||
/**
|
||||
* Run and clear a scope's cleanups in reverse registration order. Run every
|
||||
* cleanup even when one fails. Throw an `AggregateError` at the end when one
|
||||
* or more failed, so no error is lost and no cleanup is skipped.
|
||||
*/
|
||||
run(scope: CleanupScope): Promise<void>;
|
||||
}
|
||||
|
||||
/** Create an empty scoped cleanup registry. */
|
||||
export function createScopedCleanupRegistry(): ScopedCleanupRegistry {
|
||||
const byScope = new Map<CleanupScope, Array<() => void | Promise<void>>>();
|
||||
|
||||
function bucket(scope: CleanupScope): Array<() => void | Promise<void>> {
|
||||
let list = byScope.get(scope);
|
||||
if (!list) {
|
||||
list = [];
|
||||
byScope.set(scope, list);
|
||||
}
|
||||
return list;
|
||||
}
|
||||
|
||||
return {
|
||||
add(scope: CleanupScope, cleanup: () => void | Promise<void>): void {
|
||||
bucket(scope).push(cleanup);
|
||||
},
|
||||
|
||||
promote(from: CleanupScope, to: CleanupScope): void {
|
||||
if (from === to) return;
|
||||
const source = byScope.get(from);
|
||||
if (!source || source.length === 0) return;
|
||||
const target = bucket(to);
|
||||
target.push(...source);
|
||||
byScope.set(from, []);
|
||||
},
|
||||
|
||||
size(scope: CleanupScope): number {
|
||||
return byScope.get(scope)?.length ?? 0;
|
||||
},
|
||||
|
||||
async run(scope: CleanupScope): Promise<void> {
|
||||
const list = byScope.get(scope) ?? [];
|
||||
byScope.set(scope, []);
|
||||
const errors: unknown[] = [];
|
||||
for (let index = list.length - 1; index >= 0; index -= 1) {
|
||||
try {
|
||||
await list[index]();
|
||||
} catch (err) {
|
||||
errors.push(err);
|
||||
}
|
||||
}
|
||||
if (errors.length > 0) {
|
||||
throw new AggregateError(errors, `cleanup scope "${scope}" had ${errors.length} failure(s)`);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,113 @@
|
|||
// Tests for the host run site (Phase 18).
|
||||
//
|
||||
// The host site is the null path with a warm-runtime store. These tests pin its
|
||||
// warm-hit and warm-save behavior against the baseline warm sets, and confirm
|
||||
// the null-path operations: host cwd plan, no staging failures, no transport,
|
||||
// and no sync-back.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { createHostRunSite, type RuntimeCacheEntry } from "./run-site-host.js";
|
||||
import type { AcpRunContext, ReadyRunResources } from "./run-contracts.js";
|
||||
|
||||
function makeEntry(id: string, lastUsedAt: number): RuntimeCacheEntry {
|
||||
return {
|
||||
runtime: { id, close: async () => {} } as never,
|
||||
handle: { sessionKey: id } as never,
|
||||
childStderrState: { logPath: null, pendingLiveLine: "" },
|
||||
processIdentitySink: { current: undefined, latest: null },
|
||||
fingerprint: `fp-${id}`,
|
||||
lastUsedAt,
|
||||
};
|
||||
}
|
||||
|
||||
describe("host run site", () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("test_host_site_warm_hit_and_warm_save_match_baseline_sets", async () => {
|
||||
const warmHandles = new Map<string, RuntimeCacheEntry>();
|
||||
const closed: RuntimeCacheEntry[] = [];
|
||||
const site = createHostRunSite({
|
||||
warmHandles,
|
||||
now: () => Date.now(),
|
||||
idleMs: 60_000,
|
||||
closeWarmEntry: async (entry) => {
|
||||
closed.push(entry);
|
||||
},
|
||||
});
|
||||
const store = site.reuse();
|
||||
|
||||
// Warm save: the entry lands in the caller's map as the sole warm entry, the
|
||||
// same set the baseline warm-save path produces (`warmHandles.size` is 1).
|
||||
const entry = makeEntry("paperclip:c:a:t:fp", Date.now());
|
||||
store.save(entry.handle.sessionKey, entry);
|
||||
expect(new Set(warmHandles.keys())).toEqual(new Set(["paperclip:c:a:t:fp"]));
|
||||
expect(warmHandles.get("paperclip:c:a:t:fp")).toBe(entry);
|
||||
// The save armed a per-entry idle timer.
|
||||
expect(entry.cleanupTimer).toBeDefined();
|
||||
|
||||
// Warm hit: a borrow returns the same entry and leaves it in the map, so an
|
||||
// overlapping run of the same session still reads it. The borrow cleared the
|
||||
// idle timer, so the reused runtime does not expire while it is in use.
|
||||
expect(store.borrow("paperclip:c:a:t:fp")).toBe(entry);
|
||||
expect(warmHandles.size).toBe(1);
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(closed).toEqual([]);
|
||||
expect(warmHandles.get("paperclip:c:a:t:fp")).toBe(entry);
|
||||
|
||||
// A discard drops the entry and closes its runtime once, matching the
|
||||
// baseline drop set (empty map, one close).
|
||||
await store.discard("paperclip:c:a:t:fp");
|
||||
expect(warmHandles.size).toBe(0);
|
||||
expect(closed).toEqual([entry]);
|
||||
});
|
||||
|
||||
it("test_host_site_plan_and_null_path_operations", async () => {
|
||||
const site = createHostRunSite({
|
||||
warmHandles: new Map(),
|
||||
now: () => 0,
|
||||
idleMs: 0,
|
||||
closeWarmEntry: async () => {},
|
||||
});
|
||||
|
||||
const context = {
|
||||
fingerprintIdentity: { cwd: "/work/dir" },
|
||||
} as unknown as AcpRunContext;
|
||||
|
||||
// The plan binds the session to the host cwd, names no relay-proxy spawn cwd,
|
||||
// and disables sandbox tracing.
|
||||
expect(site.plan(context)).toEqual({
|
||||
sessionCwd: "/work/dir",
|
||||
spawnCwd: undefined,
|
||||
tracingEnabled: false,
|
||||
});
|
||||
|
||||
// The host runs in place, so it places nothing, starts no transport, and
|
||||
// syncs nothing back.
|
||||
expect(await site.placeWorkspace(context)).toEqual({ referencedProjectStagingFailures: [] });
|
||||
expect(await site.startTransport(context)).toEqual({ controlBridge: null, agentBridge: null });
|
||||
await expect(site.syncBack(context)).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("test_host_site_reuse_candidate_names_runtime_and_session_handle", () => {
|
||||
const site = createHostRunSite({
|
||||
warmHandles: new Map(),
|
||||
now: () => 0,
|
||||
idleMs: 0,
|
||||
closeWarmEntry: async () => {},
|
||||
});
|
||||
|
||||
const runtime = { id: "rt" };
|
||||
const sessionHandle = { id: "handle" };
|
||||
const resources = {
|
||||
get: (id: string) =>
|
||||
id === "acp_runtime" ? { runtime, sessionHandle, childProcessPid: 42 } : undefined,
|
||||
} as unknown as ReadyRunResources;
|
||||
|
||||
expect(site.reuseCandidate(resources)).toEqual({ runtime, sessionHandle });
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,146 @@
|
|||
// The host run site.
|
||||
//
|
||||
// The host site is the literal null path: it runs the agent on the host, with
|
||||
// no sandbox, no staging, and no bridges. Its store keeps a live ACP runtime and
|
||||
// session handle warm across runs of the same session, so a compatible resume
|
||||
// reuses the running agent instead of a fresh spawn. This is today's warm-handle
|
||||
// cache, factored behind the generic reuse store.
|
||||
//
|
||||
// The site owns the warm store only. The engine still drives the run body; the
|
||||
// site's `plan`, `placeWorkspace`, `startTransport`, and `syncBack` are the null
|
||||
// operations the host lane always did (host cwd, no staging failures, no
|
||||
// transport, no sync-back). A later phase routes the whole lane through the site.
|
||||
|
||||
import type { AdapterExecutionContext } from "@paperclipai/adapter-utils";
|
||||
import type { AcpRuntime, AcpRuntimeHandle } from "acpx/runtime";
|
||||
import type {
|
||||
AcpRunContext,
|
||||
HostReuseCandidate,
|
||||
PlacedWorkspace,
|
||||
ReadyRunResources,
|
||||
RunSiteTransport,
|
||||
SessionReuseStore,
|
||||
SitePlan,
|
||||
} from "./run-contracts.js";
|
||||
import { createSessionReuseStore } from "./session-reuse-store.js";
|
||||
|
||||
/** The agent child's identity, as the runtime reports it on each spawn. */
|
||||
export type AcpxAgentProcessIdentity = { pid: number; startedAt: string };
|
||||
|
||||
/**
|
||||
* The mutable spawn-callback target and the last spawn identity a warm runtime
|
||||
* carries across runs. A warm handle can outlive the run that created it, so the
|
||||
* `current` callback stays mutable: a later heartbeat repoints it to its own
|
||||
* `onSpawn`, and `latest` lets that heartbeat re-announce the running agent.
|
||||
*/
|
||||
export type AcpxProcessIdentitySink = {
|
||||
current: AdapterExecutionContext["onSpawn"];
|
||||
latest: AcpxAgentProcessIdentity | null;
|
||||
};
|
||||
|
||||
/** The live-line buffer and log path a warm runtime's child stderr carries. */
|
||||
export interface ChildStderrState {
|
||||
logPath: string | null;
|
||||
pendingLiveLine: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* One warm host runtime the store keeps for the next compatible resume. It is
|
||||
* the value the host warm-handle map holds. The entry carries its own last-used
|
||||
* time and its own optional idle timer, which the store reads and writes.
|
||||
*/
|
||||
export interface RuntimeCacheEntry {
|
||||
runtime: AcpRuntime;
|
||||
handle: AcpRuntimeHandle;
|
||||
childStderrState: ChildStderrState;
|
||||
processIdentitySink: AcpxProcessIdentitySink;
|
||||
fingerprint: string;
|
||||
lastUsedAt: number;
|
||||
cleanupTimer?: ReturnType<typeof setTimeout>;
|
||||
}
|
||||
|
||||
/** Construction options for the host site. */
|
||||
export interface HostRunSiteOptions {
|
||||
/**
|
||||
* The warm-handle map the store operates over. The engine owns it and reuses
|
||||
* it across runs, so a saved runtime stays warm for the next resume.
|
||||
*/
|
||||
readonly warmHandles: Map<string, RuntimeCacheEntry>;
|
||||
readonly now: () => number;
|
||||
/** The warm-idle bound, in milliseconds. `0` or less disables the store. */
|
||||
readonly idleMs: number;
|
||||
/**
|
||||
* Close a warm runtime and flush its child stderr. The store fires it when the
|
||||
* per-entry timer expires, when the run-start sweep evicts an idle entry, and
|
||||
* on an explicit discard. The engine injects it, because the close reason and
|
||||
* the stderr flush live in the engine module.
|
||||
*/
|
||||
readonly closeWarmEntry: (entry: RuntimeCacheEntry) => Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* The host run site. Its store keeps warm runtimes; every other operation is the
|
||||
* null path the host lane always ran.
|
||||
*/
|
||||
export interface HostRunSite {
|
||||
readonly kind: "host";
|
||||
plan(context: AcpRunContext): SitePlan;
|
||||
placeWorkspace(context: AcpRunContext): Promise<PlacedWorkspace>;
|
||||
startTransport(context: AcpRunContext): Promise<RunSiteTransport>;
|
||||
reuse(): SessionReuseStore<RuntimeCacheEntry>;
|
||||
reuseCandidate(resources: ReadyRunResources): HostReuseCandidate;
|
||||
syncBack(context: AcpRunContext): Promise<void>;
|
||||
}
|
||||
|
||||
/** Create the host run site over the engine's warm-handle map. */
|
||||
export function createHostRunSite(options: HostRunSiteOptions): HostRunSite {
|
||||
const store = createSessionReuseStore<RuntimeCacheEntry>({
|
||||
entries: options.warmHandles,
|
||||
now: options.now,
|
||||
idleMs: options.idleMs,
|
||||
lastUsedAt: (entry) => entry.lastUsedAt,
|
||||
getTimer: (entry) => entry.cleanupTimer,
|
||||
setTimer: (entry, timer) => {
|
||||
entry.cleanupTimer = timer;
|
||||
},
|
||||
// The host lane arms a per-entry idle timer on every warm save, so an idle
|
||||
// runtime closes on its own without a later run.
|
||||
perEntryTimer: true,
|
||||
release: (entry) => options.closeWarmEntry(entry),
|
||||
});
|
||||
|
||||
return {
|
||||
kind: "host",
|
||||
plan(context: AcpRunContext): SitePlan {
|
||||
// The host binds the session to the run cwd. There is no relay-proxy spawn
|
||||
// cwd off the host, and the host emits no sandbox trace spans.
|
||||
return {
|
||||
sessionCwd: context.fingerprintIdentity.cwd,
|
||||
spawnCwd: undefined,
|
||||
tracingEnabled: false,
|
||||
};
|
||||
},
|
||||
async placeWorkspace(): Promise<PlacedWorkspace> {
|
||||
// The host runs in place, so it stages nothing and reports no referenced-
|
||||
// project staging failure.
|
||||
return { referencedProjectStagingFailures: [] };
|
||||
},
|
||||
async startTransport(): Promise<RunSiteTransport> {
|
||||
// The host lane starts no control bridge and no agent bridge.
|
||||
return { controlBridge: null, agentBridge: null };
|
||||
},
|
||||
reuse(): SessionReuseStore<RuntimeCacheEntry> {
|
||||
return store;
|
||||
},
|
||||
reuseCandidate(resources: ReadyRunResources): HostReuseCandidate {
|
||||
const runtime = resources.get("acp_runtime");
|
||||
if (!runtime) {
|
||||
throw new Error("host run site cannot name a reuse candidate without an acp_runtime resource");
|
||||
}
|
||||
return { runtime: runtime.runtime, sessionHandle: runtime.sessionHandle };
|
||||
},
|
||||
async syncBack(): Promise<void> {
|
||||
// The host runs in place, so there is nothing to sync back.
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,257 @@
|
|||
// Tests for the sandbox run site (Phase 19).
|
||||
//
|
||||
// The sandbox site owns staging, the staging lease, both bridges, the launch-
|
||||
// environment contribution, and sync-back. These tests drive the site in
|
||||
// isolation with fakes: they pin the ledger registrations, the bridge overlap and
|
||||
// the callback-environment sequencing point, the staged-files reuse (which still
|
||||
// creates the runtime), the synchronous plan, and Amendment B — no run-scoped
|
||||
// contribution or bridge token escapes into the ledger or the reuse payload.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
createSandboxRunSite,
|
||||
type ManagedHomeSeamResult,
|
||||
type SandboxRunSiteOptions,
|
||||
type StagedRuntimeStoreEntry,
|
||||
} from "./run-site-sandbox.js";
|
||||
import type {
|
||||
AcpRunContext,
|
||||
AcquiredRunResources,
|
||||
LaunchEnvironmentContribution,
|
||||
ReadyRunResources,
|
||||
ResourceId,
|
||||
RunResourceRegistration,
|
||||
SandboxReuseCandidate,
|
||||
StagedRuntimeResource,
|
||||
} from "./run-contracts.js";
|
||||
import type { PreparedAdapterExecutionTargetRuntime } from "@paperclipai/adapter-utils/execution-target";
|
||||
|
||||
function makeStagedRuntime(id: string): PreparedAdapterExecutionTargetRuntime {
|
||||
return {
|
||||
runtimeRootDir: `/remote/${id}/.paperclip-runtime/acpx`,
|
||||
additionalSourceDirs: {},
|
||||
additionalSourceFailures: [],
|
||||
} as unknown as PreparedAdapterExecutionTargetRuntime;
|
||||
}
|
||||
|
||||
function makeLedger(): { ledger: AcquiredRunResources; registered: RunResourceRegistration[] } {
|
||||
const registered: RunResourceRegistration[] = [];
|
||||
const ledger: AcquiredRunResources = {
|
||||
register(registration) {
|
||||
registered.push(registration as RunResourceRegistration);
|
||||
},
|
||||
seal() {
|
||||
throw new Error("seal is not exercised by these unit tests");
|
||||
},
|
||||
takeForSettlement() {
|
||||
throw new Error("takeForSettlement is not exercised by these unit tests");
|
||||
},
|
||||
};
|
||||
return { ledger, registered };
|
||||
}
|
||||
|
||||
function makeContext(sessionKey: string): AcpRunContext {
|
||||
return { sessionKey } as unknown as AcpRunContext;
|
||||
}
|
||||
|
||||
/** Build a site over injectable fakes. Every option has a sensible default. */
|
||||
function makeSite(overrides: Partial<SandboxRunSiteOptions> = {}) {
|
||||
const { ledger, registered } = makeLedger();
|
||||
const env: Record<string, string> = { BASE: "1" };
|
||||
const stagedRuntimes = new Map<string, StagedRuntimeStoreEntry>();
|
||||
const stagingLocks = new Map<string, Promise<unknown>>();
|
||||
const freshStaged = makeStagedRuntime("fresh");
|
||||
const bridgeCalls: string[] = [];
|
||||
const options: SandboxRunSiteOptions = {
|
||||
ledger,
|
||||
stagedRuntimes,
|
||||
stagingLocks,
|
||||
now: () => 1000,
|
||||
idleMs: 60_000,
|
||||
sessionCwd: "/remote/workspace",
|
||||
spawnCwd: "/host/workspace",
|
||||
target: { kind: "remote" } as unknown as SandboxRunSiteOptions["target"],
|
||||
env,
|
||||
isCompatibleResume: false,
|
||||
stage: async () => freshStaged,
|
||||
seedManagedHome: async (stage) => {
|
||||
const stagedRuntime = await stage([]);
|
||||
// Repoint a managed-home env var, so the site captures the delta.
|
||||
env.CODEX_HOME = "/remote/home";
|
||||
return { stagedRuntime, teardown: async () => {}, dispose: async () => {} } satisfies ManagedHomeSeamResult;
|
||||
},
|
||||
disposeFreshStagedRuntime: async () => {},
|
||||
measureStageStep: (run) => run(),
|
||||
publishStagedProjectHints: () => {},
|
||||
onReuseLog: async () => {},
|
||||
startPaperclipBridge: async () => {
|
||||
bridgeCalls.push("paperclip:start");
|
||||
return { env: { PAPERCLIP_API_KEY: "run-token" }, stop: async () => {} } as never;
|
||||
},
|
||||
startProcessSessionBridge: async ({ launchEnv }) => {
|
||||
bridgeCalls.push("process-session:start");
|
||||
await launchEnv();
|
||||
return { agentCommand: null, stop: async () => {} } as never;
|
||||
},
|
||||
measureBridgeStep: (_step, run) => run(),
|
||||
finalizeLaunchEnv: (contributions) => {
|
||||
const merged = { ...env };
|
||||
for (const contribution of contributions) Object.assign(merged, contribution.env);
|
||||
return merged;
|
||||
},
|
||||
onPaperclipBridgeLog: async () => {},
|
||||
stopBridges: async () => {},
|
||||
...overrides,
|
||||
};
|
||||
const site = createSandboxRunSite(options);
|
||||
return { site, ledger, registered, env, stagedRuntimes, stagingLocks, freshStaged, bridgeCalls, options };
|
||||
}
|
||||
|
||||
/** A minimal `ReadyRunResources` that resolves a single staged runtime. */
|
||||
function makeReady(stagedRuntime: PreparedAdapterExecutionTargetRuntime): ReadyRunResources {
|
||||
const staged: StagedRuntimeResource = { stagedRuntime, disposeStaged: async () => {} };
|
||||
return {
|
||||
get: (id: ResourceId) => (id === "staged_runtime" ? staged : undefined),
|
||||
scopeOf: () => undefined,
|
||||
has: (id: ResourceId) => id === "staged_runtime",
|
||||
sealed: ["staged_runtime"],
|
||||
} as unknown as ReadyRunResources;
|
||||
}
|
||||
|
||||
describe("sandbox run site", () => {
|
||||
it("test_sandbox_site_registers_staging_bridges_and_lease_in_ledger", async () => {
|
||||
const { site, registered } = makeSite();
|
||||
await site.placeWorkspace(makeContext("paperclip:c:a:t:fp"));
|
||||
await site.startTransport(makeContext("paperclip:c:a:t:fp"));
|
||||
|
||||
const byId = new Set(registered.map((entry) => entry.id));
|
||||
// Staging registers the staged runtime, the managed-home copy-back, and the
|
||||
// per-session staging lease. Both bridge starts register their handles.
|
||||
expect(byId).toEqual(
|
||||
new Set<ResourceId>(["staged_runtime", "managed_home", "staging_lease", "control_bridge", "agent_bridge"]),
|
||||
);
|
||||
// The staged runtime is startup-rollback until a clean turn promotes it; the
|
||||
// copy-back, the lease, and the bridges live through the whole run.
|
||||
const scopeOf = (id: ResourceId) => registered.find((entry) => entry.id === id)?.scope;
|
||||
expect(scopeOf("staged_runtime")).toBe("startup_rollback");
|
||||
expect(scopeOf("managed_home")).toBe("per_run");
|
||||
expect(scopeOf("staging_lease")).toBe("per_run");
|
||||
expect(scopeOf("control_bridge")).toBe("per_run");
|
||||
expect(scopeOf("agent_bridge")).toBe("per_run");
|
||||
});
|
||||
|
||||
it("test_sandbox_site_preserves_bridge_overlap_and_callback_sequencing", async () => {
|
||||
const events: string[] = [];
|
||||
let releasePaperclip!: () => void;
|
||||
const paperclipGate = new Promise<void>((resolve) => {
|
||||
releasePaperclip = resolve;
|
||||
});
|
||||
let processLaunchEnv: Record<string, string> | null = null;
|
||||
const { site } = makeSite({
|
||||
startPaperclipBridge: async () => {
|
||||
events.push("paperclip:start");
|
||||
await paperclipGate;
|
||||
events.push("paperclip:env-ready");
|
||||
return { env: { PAPERCLIP_API_KEY: "run-token" }, stop: async () => {} } as never;
|
||||
},
|
||||
startProcessSessionBridge: async ({ launchEnv }) => {
|
||||
events.push("process-session:start");
|
||||
// The launch awaits the sequencing point, so it observes the merged env.
|
||||
processLaunchEnv = await launchEnv();
|
||||
events.push("process-session:launch");
|
||||
return { agentCommand: null, stop: async () => {} } as never;
|
||||
},
|
||||
});
|
||||
|
||||
await site.placeWorkspace(makeContext("s"));
|
||||
const transportPromise = site.startTransport(makeContext("s"));
|
||||
// Both bridges start before the paperclip env resolves: their setup overlaps.
|
||||
await Promise.resolve();
|
||||
await Promise.resolve();
|
||||
expect(events).toContain("paperclip:start");
|
||||
expect(events).toContain("process-session:start");
|
||||
expect(events).not.toContain("process-session:launch");
|
||||
|
||||
// Release the paperclip env; the process-session launch now observes the
|
||||
// merged run-scoped env (the single sequencing point).
|
||||
releasePaperclip();
|
||||
const transport = await transportPromise;
|
||||
expect(events.indexOf("paperclip:env-ready")).toBeLessThan(events.indexOf("process-session:launch"));
|
||||
expect(processLaunchEnv).toEqual({ BASE: "1", CODEX_HOME: "/remote/home", PAPERCLIP_API_KEY: "run-token" });
|
||||
expect(transport.launchEnv).toEqual(processLaunchEnv);
|
||||
});
|
||||
|
||||
it("test_sandbox_site_borrow_yields_files_and_runtime_is_still_created", async () => {
|
||||
const cachedRuntime = makeStagedRuntime("cached");
|
||||
const stagedRuntimes = new Map<string, StagedRuntimeStoreEntry>([
|
||||
[
|
||||
"paperclip:c:a:t:fp",
|
||||
{
|
||||
stagedRuntime: cachedRuntime,
|
||||
envDelta: { CODEX_HOME: "/remote/home" },
|
||||
teardown: async () => {},
|
||||
dispose: async () => {},
|
||||
lastUsedAt: 500,
|
||||
},
|
||||
],
|
||||
]);
|
||||
const { site, env } = makeSite({ stagedRuntimes, isCompatibleResume: true });
|
||||
|
||||
// A borrow reads the staged entry without removal, so an overlapping run of
|
||||
// the same session still reads it.
|
||||
const borrowed = site.reuse().borrow("paperclip:c:a:t:fp");
|
||||
expect(borrowed?.stagedRuntime).toBe(cachedRuntime);
|
||||
expect(stagedRuntimes.has("paperclip:c:a:t:fp")).toBe(true);
|
||||
|
||||
// A reuse hit yields the staged FILES (not a live runtime): `placeWorkspace`
|
||||
// reports `reused` and re-applies the seam env delta, and the reuse candidate
|
||||
// names the staged files only. So the run still creates the runtime and runs
|
||||
// the handshake against the reused files.
|
||||
await site.placeWorkspace(makeContext("paperclip:c:a:t:fp"));
|
||||
expect(site.staged?.reused).toBe(true);
|
||||
expect(site.staged?.stagedRuntime).toBe(cachedRuntime);
|
||||
expect(env.CODEX_HOME).toBe("/remote/home");
|
||||
const candidate: SandboxReuseCandidate = site.reuseCandidate(makeReady(cachedRuntime));
|
||||
expect(candidate).toEqual({ stagedRuntime: cachedRuntime });
|
||||
expect("runtime" in candidate).toBe(false);
|
||||
expect("sessionHandle" in candidate).toBe(false);
|
||||
});
|
||||
|
||||
it("test_site_plan_supplies_session_cwd_spawn_cwd_and_tracing_before_the_fingerprint_build", () => {
|
||||
const { site } = makeSite();
|
||||
// `plan()` is synchronous and reads no staged state, so it runs before the
|
||||
// fingerprint build. It binds the session to the in-sandbox cwd, redirects the
|
||||
// relay-proxy spawn to the host cwd, and enables sandbox tracing.
|
||||
expect(site.plan(makeContext("s"))).toEqual({
|
||||
sessionCwd: "/remote/workspace",
|
||||
spawnCwd: "/host/workspace",
|
||||
tracingEnabled: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("test_bridge_contribution_is_not_retained_by_site_ledger_or_reuse_payload", async () => {
|
||||
const { site, registered } = makeSite();
|
||||
await site.placeWorkspace(makeContext("s"));
|
||||
const transport = await site.startTransport(makeContext("s"));
|
||||
|
||||
// The ledger holds the bridge HANDLES, never the run-scoped token env.
|
||||
const control = registered.find((entry) => entry.id === "control_bridge");
|
||||
expect(control?.payload).toBe(transport.controlBridge);
|
||||
// The staged reuse payload carries the staged files only — no credential
|
||||
// material and no bridge token.
|
||||
const candidate = site.reuseCandidate(makeReady(site.staged!.stagedRuntime));
|
||||
expect(Object.keys(candidate)).toEqual(["stagedRuntime"]);
|
||||
|
||||
// Compile-time (Amendment B): a run-scoped contribution is not assignable to
|
||||
// the staged reuse payload. This function never runs; the assertion holds at
|
||||
// typecheck only. A runtime call on a synthetic value would crash.
|
||||
// eslint-disable-next-line @typescript-eslint/no-unused-vars
|
||||
function _staged_reuse_payload_rejects_a_run_scoped_contribution(
|
||||
contribution: LaunchEnvironmentContribution,
|
||||
): void {
|
||||
// @ts-expect-error a launch-environment contribution is not a staged reuse payload
|
||||
const payload: SandboxReuseCandidate = contribution;
|
||||
void payload;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,459 @@
|
|||
// The sandbox run site.
|
||||
//
|
||||
// The sandbox site runs the agent in a remote runner. It owns the four concerns
|
||||
// the host site never has: it stages the workspace and the managed home into the
|
||||
// sandbox, it holds the per-session staging lease, it starts both host-side
|
||||
// bridges, and it syncs the managed home back on teardown. Its store keeps the
|
||||
// staged files warm across runs of the same session, so a compatible resume
|
||||
// reuses the already-staged in-sandbox runtime instead of re-shipping.
|
||||
//
|
||||
// The site owns the orchestration and the policy: the staging lease, the reuse
|
||||
// and stale decision, the ledger registrations, the launch-environment envelope,
|
||||
// the bridge overlap, and the single callback-environment sequencing point. The
|
||||
// engine injects the leaf input/output primitives — the workspace stage call, the
|
||||
// managed-home seam, the two bridge starts, and the step timer — so this module
|
||||
// stays free of the engine's private staging helpers and the unit tests drive it
|
||||
// with fakes.
|
||||
//
|
||||
// Amendment B (credential scope): `startTransport` builds the run-scoped bridge
|
||||
// contribution and hands it to the injected `finalizeLaunchEnv`, the sole
|
||||
// consumer of a contribution. The site, its ledger entries, and its reuse
|
||||
// candidate never retain a run-scoped contribution or a bridge token. The staged
|
||||
// reuse payload carries the staged files only.
|
||||
|
||||
import type {
|
||||
AdapterExecutionTarget,
|
||||
AdapterExecutionTargetPaperclipBridgeHandle,
|
||||
AdapterExecutionTargetProcessSessionBridgeHandle,
|
||||
AdapterManagedRuntimeAsset,
|
||||
PreparedAdapterExecutionTargetRuntime,
|
||||
} from "@paperclipai/adapter-utils/execution-target";
|
||||
import type {
|
||||
AcpRunContext,
|
||||
AcquiredRunResources,
|
||||
LaunchEnvironmentContribution,
|
||||
PlacedWorkspace,
|
||||
ReadyRunResources,
|
||||
RunScopedContribution,
|
||||
RunSiteTransport,
|
||||
SandboxReuseCandidate,
|
||||
SessionReuseStore,
|
||||
SitePlan,
|
||||
} from "./run-contracts.js";
|
||||
import { createSessionReuseStore } from "./session-reuse-store.js";
|
||||
|
||||
/**
|
||||
* One staged in-sandbox runtime the store keeps for the next compatible resume.
|
||||
* It is the value the staged-runtime map holds. The engine still owns this shape;
|
||||
* the store reads its `lastUsedAt` and its `dispose` through the config
|
||||
* accessors. The staged cache has no per-entry idle timer, so the entry carries
|
||||
* none — the run-start sweep is the only eviction path.
|
||||
*/
|
||||
export interface StagedRuntimeStoreEntry {
|
||||
stagedRuntime: PreparedAdapterExecutionTargetRuntime;
|
||||
envDelta: Record<string, string>;
|
||||
teardown: (() => Promise<void>) | null;
|
||||
dispose: (() => Promise<void>) | null;
|
||||
lastUsedAt: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* The result of one workspace stage into the sandbox. The site records these
|
||||
* fields on the run and, after a clean turn, into the staged store.
|
||||
*/
|
||||
export interface StagedWorkspace {
|
||||
readonly stagedRuntime: PreparedAdapterExecutionTargetRuntime;
|
||||
readonly teardown: (() => Promise<void>) | null;
|
||||
readonly dispose: (() => Promise<void>) | null;
|
||||
readonly envDelta: Record<string, string>;
|
||||
/** True when the run reused an already-staged runtime rather than staging fresh. */
|
||||
readonly reused: boolean;
|
||||
}
|
||||
|
||||
/** The leaf staging primitive the engine injects. It ships one asset set. */
|
||||
export type StageWorkspace = (assets: AdapterManagedRuntimeAsset[]) => Promise<PreparedAdapterExecutionTargetRuntime>;
|
||||
|
||||
/**
|
||||
* The managed-home seam result, in the shape the site consumes. The engine maps
|
||||
* its adapter seam onto this before it passes the seam in.
|
||||
*/
|
||||
export interface ManagedHomeSeamResult {
|
||||
readonly stagedRuntime: PreparedAdapterExecutionTargetRuntime;
|
||||
readonly teardown: (() => Promise<void>) | null;
|
||||
readonly dispose: (() => Promise<void>) | null;
|
||||
}
|
||||
|
||||
/** The construction options for the sandbox site. Every value is per run. */
|
||||
export interface SandboxRunSiteOptions {
|
||||
/** The run ledger. `executeAcpxEngine` creates it and passes it in (one ledger). */
|
||||
readonly ledger: AcquiredRunResources;
|
||||
/** The staged-runtime map the store operates over. The engine owns it across runs. */
|
||||
readonly stagedRuntimes: Map<string, StagedRuntimeStoreEntry>;
|
||||
/** The per-key staging mutex the lease and the sweep share. */
|
||||
readonly stagingLocks: Map<string, Promise<unknown>>;
|
||||
readonly now: () => number;
|
||||
/** The staged-idle bound, in milliseconds. `0` or less disables the sweep. */
|
||||
readonly idleMs: number;
|
||||
/** The in-sandbox workspace dir the session binds to (a fingerprint input). */
|
||||
readonly sessionCwd: string;
|
||||
/** The host spawn cwd for the relay proxy on the remote process-session lane. */
|
||||
readonly spawnCwd: string;
|
||||
/** The remote runner-backed sandbox target. */
|
||||
readonly target: AdapterExecutionTarget;
|
||||
/**
|
||||
* The run env the site mutates in place. Staging repoints the managed-home env
|
||||
* var onto the in-sandbox home here, and the site captures the delta from it.
|
||||
*/
|
||||
readonly env: Record<string, string>;
|
||||
/**
|
||||
* True when the supplied session params resume THIS staged session. Only a
|
||||
* compatible resume may reuse the already-staged runtime.
|
||||
*/
|
||||
readonly isCompatibleResume: boolean;
|
||||
/** Ship the workspace (and, via the seam, the managed home) into the sandbox. */
|
||||
readonly stage: StageWorkspace;
|
||||
/**
|
||||
* Seed the managed home through `stage`, repoint the home env var, and wire the
|
||||
* per-run copy-back. Absent for custom agents and the shared-engine tests, where
|
||||
* the site stages the workspace with no home asset.
|
||||
*/
|
||||
readonly seedManagedHome?: (stage: StageWorkspace) => Promise<ManagedHomeSeamResult>;
|
||||
/**
|
||||
* Remove a freshly staged in-sandbox runtime after the seam threw. The site
|
||||
* fires it when the seam fails after a successful stage.
|
||||
*/
|
||||
readonly disposeFreshStagedRuntime: (stagedRuntime: PreparedAdapterExecutionTargetRuntime) => Promise<void>;
|
||||
/** Wrap the `stage.sync` step in the run's startup step timer. */
|
||||
readonly measureStageStep: <T>(run: () => Promise<T>) => Promise<T>;
|
||||
/** Publish the referenced-project workspace hints after a fresh stage. */
|
||||
readonly publishStagedProjectHints: (stagedProjectDirs: Record<string, string>) => void;
|
||||
readonly onReuseLog: () => Promise<void>;
|
||||
|
||||
/** Start the host-side paperclip callback bridge. */
|
||||
readonly startPaperclipBridge: (
|
||||
runtimeRootDir: string | null,
|
||||
) => Promise<AdapterExecutionTargetPaperclipBridgeHandle | null>;
|
||||
/** Start the host-side process-session bridge with a deferred launch env. */
|
||||
readonly startProcessSessionBridge: (input: {
|
||||
runtimeRootDir: string | null;
|
||||
launchEnv: () => Promise<Record<string, string>>;
|
||||
}) => Promise<AdapterExecutionTargetProcessSessionBridgeHandle | null>;
|
||||
/** Wrap each concurrent bridge start in the run's startup step timer. */
|
||||
readonly measureBridgeStep: <T>(step: "bridge.paperclip" | "bridge.process-session", run: () => Promise<T>) => Promise<T>;
|
||||
/**
|
||||
* Finalize the run's branded launch environment from the bridge contribution.
|
||||
* The engine owns `finalizeLaunchEnvironment`, so it stays the sole consumer of
|
||||
* a contribution; the site calls this capability at the sequencing point.
|
||||
*/
|
||||
readonly finalizeLaunchEnv: (contributions: readonly LaunchEnvironmentContribution[]) => Record<string, string>;
|
||||
readonly onPaperclipBridgeLog: () => Promise<void>;
|
||||
|
||||
/** Stop both bridges and run the managed-home copy-back on teardown. */
|
||||
readonly stopBridges: (input: {
|
||||
controlBridge: AdapterExecutionTargetPaperclipBridgeHandle | null;
|
||||
agentBridge: AdapterExecutionTargetProcessSessionBridgeHandle | null;
|
||||
}) => Promise<void>;
|
||||
}
|
||||
|
||||
/** The transport the sandbox site started, plus the run-scoped launch env. */
|
||||
export interface SandboxRunSiteTransport extends RunSiteTransport {
|
||||
/** The finalized launch env for the process launch. */
|
||||
readonly launchEnv: Record<string, string>;
|
||||
}
|
||||
|
||||
/** The sandbox run site. It owns staging, the lease, both bridges, and sync-back. */
|
||||
export interface SandboxRunSite {
|
||||
readonly kind: "sandbox";
|
||||
plan(context: AcpRunContext): SitePlan;
|
||||
placeWorkspace(context: AcpRunContext): Promise<PlacedWorkspace>;
|
||||
startTransport(context: AcpRunContext): Promise<SandboxRunSiteTransport>;
|
||||
reuse(): SessionReuseStore<StagedRuntimeStoreEntry>;
|
||||
reuseCandidate(resources: ReadyRunResources): SandboxReuseCandidate;
|
||||
syncBack(context: AcpRunContext): Promise<void>;
|
||||
/** The staged workspace the run installed or reused, or null before `placeWorkspace`. */
|
||||
readonly staged: StagedWorkspace | null;
|
||||
/**
|
||||
* The paperclip callback bridge the run started, or null before
|
||||
* `startTransport` or on the host lane. `startTransport` sets it before it
|
||||
* rethrows a partial-bring-up failure, so an abandon path can stop the bridge
|
||||
* that started when its sibling threw.
|
||||
*/
|
||||
readonly controlBridge: AdapterExecutionTargetPaperclipBridgeHandle | null;
|
||||
/** The process-session bridge the run started, or null before `startTransport`. */
|
||||
readonly agentBridge: AdapterExecutionTargetProcessSessionBridgeHandle | null;
|
||||
/**
|
||||
* Release the per-session staging lease `placeWorkspace` acquired, or null
|
||||
* before `placeWorkspace`. An abandon path calls it to free the lease when the
|
||||
* run never reaches sync-back.
|
||||
*/
|
||||
readonly stagingLeaseRelease: (() => void) | null;
|
||||
}
|
||||
|
||||
/** Create the sandbox run site over the engine's staged-runtime map and ledger. */
|
||||
export function createSandboxRunSite(options: SandboxRunSiteOptions): SandboxRunSite {
|
||||
const store = createSessionReuseStore<StagedRuntimeStoreEntry>({
|
||||
entries: options.stagedRuntimes,
|
||||
now: options.now,
|
||||
idleMs: options.idleMs,
|
||||
lastUsedAt: (entry) => entry.lastUsedAt,
|
||||
// The staged cache arms no per-entry timer; the run-start sweep is the only
|
||||
// eviction path, and it runs under the per-key staging lease.
|
||||
release: async (entry) => {
|
||||
if (entry.dispose) await entry.dispose().catch(() => {});
|
||||
},
|
||||
withEvictLease: async (key, critical) => {
|
||||
const lease = await withStagingLease(options.stagingLocks, key, critical);
|
||||
lease.release();
|
||||
},
|
||||
});
|
||||
|
||||
// The lease the run holds from the stage-or-reuse decision through the active
|
||||
// turn until `syncBack` releases it. Held on the run, not on the store, because
|
||||
// a run keeps its lease across its whole lifetime.
|
||||
let leaseRelease: (() => void) | null = null;
|
||||
let staged: StagedWorkspace | null = null;
|
||||
let controlBridge: AdapterExecutionTargetPaperclipBridgeHandle | null = null;
|
||||
let agentBridge: AdapterExecutionTargetProcessSessionBridgeHandle | null = null;
|
||||
|
||||
return {
|
||||
kind: "sandbox",
|
||||
|
||||
plan(): SitePlan {
|
||||
// The sandbox binds the session to the in-sandbox workspace dir, redirects
|
||||
// the relay proxy spawn to the host cwd, and emits real sandbox trace spans.
|
||||
return {
|
||||
sessionCwd: options.sessionCwd,
|
||||
spawnCwd: options.spawnCwd,
|
||||
tracingEnabled: true,
|
||||
};
|
||||
},
|
||||
|
||||
async placeWorkspace(context: AcpRunContext): Promise<PlacedWorkspace> {
|
||||
const sessionKey = context.sessionKey;
|
||||
const lease = await withStagingLease(options.stagingLocks, sessionKey, async (): Promise<StagedWorkspace> => {
|
||||
const cached = options.isCompatibleResume ? store.borrow(sessionKey) : undefined;
|
||||
if (cached) {
|
||||
// Reuse the already-staged in-sandbox workspace and managed home.
|
||||
// Re-apply the env keys the seam repointed and reuse the per-run
|
||||
// copy-back so the copy-back cadence stays exactly per run.
|
||||
Object.assign(options.env, cached.envDelta);
|
||||
cached.lastUsedAt = options.now();
|
||||
await options.onReuseLog();
|
||||
return {
|
||||
stagedRuntime: cached.stagedRuntime,
|
||||
teardown: cached.teardown,
|
||||
dispose: cached.dispose,
|
||||
envDelta: cached.envDelta,
|
||||
reused: true,
|
||||
};
|
||||
}
|
||||
// Not a compatible resume: stage fresh. Drop and dispose a stale entry
|
||||
// that collides on this key first, so a fresh stage neither reuses nor
|
||||
// leaks it.
|
||||
const stale = options.stagedRuntimes.get(sessionKey);
|
||||
if (stale) {
|
||||
options.stagedRuntimes.delete(sessionKey);
|
||||
if (stale.dispose) await stale.dispose().catch(() => {});
|
||||
}
|
||||
const envBeforeStage = { ...options.env };
|
||||
let freshlyStaged: PreparedAdapterExecutionTargetRuntime | null = null;
|
||||
const stage: StageWorkspace = async (assets) => {
|
||||
const result = await options.stage(assets);
|
||||
freshlyStaged = result;
|
||||
return result;
|
||||
};
|
||||
const seam = await options.measureStageStep(async (): Promise<ManagedHomeSeamResult> => {
|
||||
if (options.seedManagedHome) {
|
||||
try {
|
||||
return await options.seedManagedHome(stage);
|
||||
} catch (seamErr) {
|
||||
// The seam threw after a possible successful stage. Dispose the fresh
|
||||
// staged runtime so the abandoned in-sandbox managed home never leaks,
|
||||
// then rethrow the seam error.
|
||||
if (freshlyStaged) await options.disposeFreshStagedRuntime(freshlyStaged);
|
||||
throw seamErr;
|
||||
}
|
||||
}
|
||||
return { stagedRuntime: await stage([]), teardown: null, dispose: null };
|
||||
});
|
||||
const envDelta: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(options.env)) {
|
||||
if (envBeforeStage[key] !== value) envDelta[key] = value;
|
||||
}
|
||||
return {
|
||||
stagedRuntime: seam.stagedRuntime,
|
||||
teardown: seam.teardown,
|
||||
dispose: seam.dispose,
|
||||
envDelta,
|
||||
reused: false,
|
||||
};
|
||||
});
|
||||
leaseRelease = lease.release;
|
||||
staged = lease.value;
|
||||
// Publish the referenced-project workspace hints (known only after staging).
|
||||
const stagedProjectDirs = staged.stagedRuntime.additionalSourceDirs ?? {};
|
||||
if (Object.keys(stagedProjectDirs).length > 0) {
|
||||
options.publishStagedProjectHints(stagedProjectDirs);
|
||||
}
|
||||
// Register the acquired resources into the ledger. A per-run copy-back and a
|
||||
// staging lease live through the whole run; the staged runtime is
|
||||
// startup-rollback until a clean turn promotes it.
|
||||
options.ledger.register({
|
||||
id: "staged_runtime",
|
||||
payload: { stagedRuntime: staged.stagedRuntime, disposeStaged: staged.dispose ?? (async () => {}) },
|
||||
scope: "startup_rollback",
|
||||
});
|
||||
if (staged.teardown) {
|
||||
options.ledger.register({
|
||||
id: "managed_home",
|
||||
payload: { teardown: staged.teardown, envDelta: staged.envDelta },
|
||||
scope: "per_run",
|
||||
});
|
||||
}
|
||||
if (leaseRelease) {
|
||||
const release = leaseRelease;
|
||||
options.ledger.register({
|
||||
id: "staging_lease",
|
||||
payload: { release },
|
||||
scope: "per_run",
|
||||
});
|
||||
}
|
||||
return {
|
||||
referencedProjectStagingFailures: (staged.stagedRuntime.additionalSourceFailures ?? []).map(
|
||||
(failure) => ({ projectId: failure.projectId }),
|
||||
),
|
||||
};
|
||||
},
|
||||
|
||||
async startTransport(): Promise<SandboxRunSiteTransport> {
|
||||
// Bring up both host-side bridges concurrently. Their remote subtrees are
|
||||
// disjoint, so their env-independent setup overlaps. The one real
|
||||
// dependency — the paperclip bridge's returned env must reach the
|
||||
// process-session launch — is sequenced by `launchEnv`, a memoized thunk the
|
||||
// process-session bridge awaits right before its launch.
|
||||
const stagedRootDir = staged?.stagedRuntime.runtimeRootDir ?? null;
|
||||
const paperclipStart = options.measureBridgeStep("bridge.paperclip", () =>
|
||||
options.startPaperclipBridge(stagedRootDir),
|
||||
);
|
||||
// The single sequencing point (paperclip env → process-session launch),
|
||||
// memoized so the merge runs exactly once whether the process-session bridge
|
||||
// consumes it at launch or `startTransport` finalizes it below.
|
||||
let launchEnvPromise: Promise<Record<string, string>> | null = null;
|
||||
let launchEnv: Record<string, string> = {};
|
||||
const finalizeLaunchEnv = (): Promise<Record<string, string>> =>
|
||||
(launchEnvPromise ??= (async () => {
|
||||
const paperclip = await paperclipStart;
|
||||
// The paperclip bridge token is run-scoped: it lives for this run only and
|
||||
// never enters a reuse payload (Amendment B). The site hands it to the
|
||||
// engine's `finalizeLaunchEnv`, the sole consumer of a contribution, and
|
||||
// retains nothing.
|
||||
const contributions: LaunchEnvironmentContribution[] = [];
|
||||
if (paperclip) {
|
||||
contributions.push({ scope: "run", env: paperclip.env } as unknown as RunScopedContribution);
|
||||
await options.onPaperclipBridgeLog();
|
||||
}
|
||||
launchEnv = options.finalizeLaunchEnv(contributions);
|
||||
return launchEnv;
|
||||
})());
|
||||
const processSessionStart = options.measureBridgeStep("bridge.process-session", () =>
|
||||
options.startProcessSessionBridge({ runtimeRootDir: stagedRootDir, launchEnv: finalizeLaunchEnv }),
|
||||
);
|
||||
// Settle BOTH starts, so a partial failure can stop whichever bridge started.
|
||||
const [paperclip, processSession] = await Promise.allSettled([paperclipStart, processSessionStart]);
|
||||
controlBridge = paperclip.status === "fulfilled" ? paperclip.value : null;
|
||||
agentBridge = processSession.status === "fulfilled" ? processSession.value : null;
|
||||
const failure =
|
||||
paperclip.status === "rejected"
|
||||
? paperclip.reason
|
||||
: processSession.status === "rejected"
|
||||
? processSession.reason
|
||||
: null;
|
||||
if (failure) throw failure;
|
||||
// Guarantee the merge ran even if the process-session bridge returned without
|
||||
// consuming the launch env (memoized, so a no-op if it did).
|
||||
await finalizeLaunchEnv();
|
||||
if (controlBridge) {
|
||||
options.ledger.register({ id: "control_bridge", payload: controlBridge, scope: "per_run" });
|
||||
}
|
||||
if (agentBridge) {
|
||||
options.ledger.register({ id: "agent_bridge", payload: agentBridge, scope: "per_run" });
|
||||
}
|
||||
return { controlBridge, agentBridge, launchEnv };
|
||||
},
|
||||
|
||||
reuse(): SessionReuseStore<StagedRuntimeStoreEntry> {
|
||||
return store;
|
||||
},
|
||||
|
||||
reuseCandidate(resources: ReadyRunResources): SandboxReuseCandidate {
|
||||
const stagedResource = resources.get("staged_runtime");
|
||||
if (!stagedResource) {
|
||||
throw new Error("sandbox run site cannot name a reuse candidate without a staged_runtime resource");
|
||||
}
|
||||
// The reuse payload carries the staged files only — no credential material.
|
||||
return { stagedRuntime: stagedResource.stagedRuntime };
|
||||
},
|
||||
|
||||
async syncBack(): Promise<void> {
|
||||
try {
|
||||
// Stop both bridges, then run the managed-home copy-back (mirrors the CLI
|
||||
// finally: stop bridge, then restore workspace).
|
||||
await options.stopBridges({ controlBridge, agentBridge });
|
||||
} finally {
|
||||
// Release the per-session staging lease in a finally, so an earlier
|
||||
// teardown fault never strands it and deadlocks the next same-session run.
|
||||
leaseRelease?.();
|
||||
leaseRelease = null;
|
||||
}
|
||||
},
|
||||
|
||||
get staged(): StagedWorkspace | null {
|
||||
return staged;
|
||||
},
|
||||
|
||||
get controlBridge(): AdapterExecutionTargetPaperclipBridgeHandle | null {
|
||||
return controlBridge;
|
||||
},
|
||||
|
||||
get agentBridge(): AdapterExecutionTargetProcessSessionBridgeHandle | null {
|
||||
return agentBridge;
|
||||
},
|
||||
|
||||
get stagingLeaseRelease(): (() => void) | null {
|
||||
return leaseRelease;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Per-`sessionKey` async lease: chains each caller after the previous one so the
|
||||
// stage-or-reuse decision for a session runs serially, then keeps the lease held
|
||||
// until the run releases it. Two overlapping runs of the same session can never
|
||||
// stage into the same remote workspace at once: the loser waits, then re-checks
|
||||
// the cache before it decides.
|
||||
async function withStagingLease<T>(
|
||||
locks: Map<string, Promise<unknown>>,
|
||||
key: string,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<{ value: T; release: () => void }> {
|
||||
const prev = locks.get(key) ?? Promise.resolve();
|
||||
let releaseGate!: () => void;
|
||||
const gate = new Promise<void>((resolve) => {
|
||||
releaseGate = resolve;
|
||||
});
|
||||
const mine: Promise<unknown> = prev.then(() => gate);
|
||||
locks.set(key, mine);
|
||||
await prev.catch(() => {});
|
||||
let released = false;
|
||||
const release = () => {
|
||||
if (released) return;
|
||||
released = true;
|
||||
releaseGate();
|
||||
if (locks.get(key) === mine) locks.delete(key);
|
||||
};
|
||||
try {
|
||||
return { value: await fn(), release };
|
||||
} catch (error) {
|
||||
if (!released) release();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,160 @@
|
|||
// Tests for the generic per-session reuse store (Phase 18).
|
||||
//
|
||||
// The store backs both site caches over a caller-provided map. These tests pin
|
||||
// the four behaviors the two caches had before the refactor: borrow reads
|
||||
// without removal and clears the per-entry idle timer; discard is
|
||||
// identity-guarded and releases once; save arms a per-entry idle timer that
|
||||
// discards without a run; and the run-start sweep runs each eviction under the
|
||||
// optional per-key lease.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { createSessionReuseStore } from "./session-reuse-store.js";
|
||||
|
||||
interface Entry {
|
||||
readonly id: string;
|
||||
lastUsedAt: number;
|
||||
cleanupTimer?: ReturnType<typeof setTimeout>;
|
||||
}
|
||||
|
||||
function hostConfig(entries: Map<string, Entry>, released: Entry[], idleMs: number) {
|
||||
return {
|
||||
entries,
|
||||
now: () => Date.now(),
|
||||
idleMs,
|
||||
lastUsedAt: (entry: Entry) => entry.lastUsedAt,
|
||||
getTimer: (entry: Entry) => entry.cleanupTimer,
|
||||
setTimer: (entry: Entry, timer: ReturnType<typeof setTimeout> | undefined) => {
|
||||
entry.cleanupTimer = timer;
|
||||
},
|
||||
perEntryTimer: true,
|
||||
release: (entry: Entry) => {
|
||||
released.push(entry);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe("session reuse store", () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("test_borrow_reads_without_removal_and_clears_the_per_entry_idle_timer", async () => {
|
||||
const entries = new Map<string, Entry>();
|
||||
const released: Entry[] = [];
|
||||
const store = createSessionReuseStore<Entry>(hostConfig(entries, released, 1000));
|
||||
|
||||
const entry: Entry = { id: "a", lastUsedAt: Date.now() };
|
||||
store.save("k", entry);
|
||||
expect(entry.cleanupTimer).toBeDefined();
|
||||
|
||||
// A borrow reads the entry and leaves it in the store, so a second borrow
|
||||
// still finds it.
|
||||
expect(store.borrow("k")).toBe(entry);
|
||||
expect(store.borrow("k")).toBe(entry);
|
||||
|
||||
// The borrow cleared the per-entry idle timer, so the entry does not expire
|
||||
// while it is in use. Advance well past the idle bound.
|
||||
await vi.advanceTimersByTimeAsync(5000);
|
||||
expect(released).toEqual([]);
|
||||
expect(store.borrow("k")).toBe(entry);
|
||||
// The entry stays in the caller's map for a later run to inspect.
|
||||
expect(entries.get("k")).toBe(entry);
|
||||
});
|
||||
|
||||
it("test_discard_is_identity_guarded_and_fires_release_once", async () => {
|
||||
const entries = new Map<string, Entry>();
|
||||
const released: Entry[] = [];
|
||||
const store = createSessionReuseStore<Entry>({
|
||||
entries,
|
||||
now: () => Date.now(),
|
||||
idleMs: 0,
|
||||
lastUsedAt: (entry) => entry.lastUsedAt,
|
||||
release: (entry) => {
|
||||
released.push(entry);
|
||||
},
|
||||
});
|
||||
|
||||
const first: Entry = { id: "first", lastUsedAt: Date.now() };
|
||||
store.save("k", first);
|
||||
|
||||
// The first discard removes the entry and fires the release once.
|
||||
await store.discard("k");
|
||||
expect(released).toEqual([first]);
|
||||
expect(store.borrow("k")).toBeUndefined();
|
||||
|
||||
// A second discard of the same key finds no entry, so it releases nothing.
|
||||
await store.discard("k");
|
||||
expect(released).toEqual([first]);
|
||||
|
||||
// A re-saved entry is a new entry. A discard of the key releases the new
|
||||
// entry, never the already-released first entry again.
|
||||
const second: Entry = { id: "second", lastUsedAt: Date.now() };
|
||||
store.save("k", second);
|
||||
await store.discard("k");
|
||||
expect(released).toEqual([first, second]);
|
||||
});
|
||||
|
||||
it("test_save_arms_a_per_entry_idle_timer_that_discards_without_a_run", async () => {
|
||||
const entries = new Map<string, Entry>();
|
||||
const released: Entry[] = [];
|
||||
const store = createSessionReuseStore<Entry>(hostConfig(entries, released, 1000));
|
||||
|
||||
const entry: Entry = { id: "a", lastUsedAt: Date.now() };
|
||||
store.save("k", entry);
|
||||
|
||||
// Before the idle deadline the entry is still present and unreleased.
|
||||
await vi.advanceTimersByTimeAsync(999);
|
||||
expect(released).toEqual([]);
|
||||
|
||||
// At the idle deadline the armed timer discards the entry with no run and
|
||||
// no explicit discard call.
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
expect(released).toEqual([entry]);
|
||||
expect(store.borrow("k")).toBeUndefined();
|
||||
expect(entries.has("k")).toBe(false);
|
||||
});
|
||||
|
||||
it("test_evict_idle_sweep_runs_under_per_key_lease", async () => {
|
||||
const entries = new Map<string, Entry>();
|
||||
const released: Entry[] = [];
|
||||
const leaseOrder: string[] = [];
|
||||
let now = 0;
|
||||
const store = createSessionReuseStore<Entry>({
|
||||
entries,
|
||||
now: () => now,
|
||||
idleMs: 1000,
|
||||
lastUsedAt: (entry) => entry.lastUsedAt,
|
||||
// The sandbox lane leaves the per-entry timer off and relies on the
|
||||
// run-start sweep only.
|
||||
release: (entry) => {
|
||||
released.push(entry);
|
||||
},
|
||||
withEvictLease: async (key, critical) => {
|
||||
leaseOrder.push(`lease:${key}`);
|
||||
await critical();
|
||||
leaseOrder.push(`release:${key}`);
|
||||
},
|
||||
});
|
||||
|
||||
store.save("stale", { id: "stale", lastUsedAt: 0 });
|
||||
store.save("fresh", { id: "fresh", lastUsedAt: 0 });
|
||||
|
||||
// Move the clock so only the "stale" entry crosses the idle bound. Touch
|
||||
// "fresh" so it stays inside the window.
|
||||
now = 1500;
|
||||
store.borrow("fresh");
|
||||
store.save("fresh", { id: "fresh2", lastUsedAt: now });
|
||||
|
||||
await store.evictIdle(now);
|
||||
|
||||
// Only the stale entry evicts, and its critical section ran inside the
|
||||
// per-key lease (lease acquired, critical ran, lease released, in order).
|
||||
expect(released.map((entry) => entry.id)).toEqual(["stale"]);
|
||||
expect(leaseOrder).toEqual(["lease:stale", "release:stale"]);
|
||||
expect(store.borrow("stale")).toBeUndefined();
|
||||
expect(store.borrow("fresh")).toEqual({ id: "fresh2", lastUsedAt: 1500 });
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,189 @@
|
|||
// The generic per-session reuse store.
|
||||
//
|
||||
// One `SessionReuseStore<T>` implementation backs both site caches. The host
|
||||
// site saves a live runtime and session handle. The sandbox site saves the
|
||||
// staged files. The store is generic in the saved type, so a generic `save`
|
||||
// cannot read what a site saves. `RunSite.reuseCandidate()` names the concrete
|
||||
// type per site.
|
||||
//
|
||||
// The store operates OVER a caller-provided map, not a private internal map.
|
||||
// The host lane passes its warm-handle map; the sandbox lane passes its staged-
|
||||
// runtime map. Both maps persist across runs, so a saved entry stays visible to
|
||||
// a later run of the same session. Each entry carries its own last-used time and
|
||||
// its own optional per-entry idle timer, so the store reads and writes those
|
||||
// through the caller-supplied accessors. This keeps the exact shape both caches
|
||||
// had before this refactor, so a caller that inspects the map still reads the
|
||||
// same entries.
|
||||
//
|
||||
// The store keeps the exact behavior the two caches had before this refactor:
|
||||
//
|
||||
// * `borrow` reads without removal. It keeps today's visibility for
|
||||
// overlapping runs of the same session. It clears the entry's per-entry
|
||||
// idle timer, because an in-use entry must not expire under its own timer.
|
||||
// * `save` sets the entry in the map. When the store has a per-entry idle
|
||||
// timer, `save` arms an unref'd timer that discards the entry at the idle
|
||||
// deadline without a run (today's `scheduleIdleHandleCleanup`).
|
||||
// * `discard` is an identity-guarded removal that fires the idempotent
|
||||
// release once. It acts on the entry that is at the key now, so a re-saved
|
||||
// entry is safe and a second discard is a no-op.
|
||||
// * `evictIdle` is the run-start sweep (today's `cleanupIdleHandles` and
|
||||
// `cleanupIdleStagedRuntimes`). The host lane runs the critical section
|
||||
// directly. The sandbox lane runs it under its per-key staging lease and
|
||||
// re-checks the idle window inside the lease.
|
||||
|
||||
import type { SessionReuseStore } from "./run-contracts.js";
|
||||
|
||||
/**
|
||||
* The construction options for a reuse store. The public `SessionReuseStore<T>`
|
||||
* surface stays generic; these options carry the per-lane behavior the store
|
||||
* needs but the interface does not name: the backing map, the clock, the idle
|
||||
* bound, how to read an entry's last-used time and per-entry timer, how to
|
||||
* release an entry, whether to arm a per-entry timer, and the optional eviction
|
||||
* lease.
|
||||
*/
|
||||
export interface SessionReuseStoreConfig<T> {
|
||||
/**
|
||||
* The map the store reads and writes. The caller owns it, so a caller that
|
||||
* inspects the map still reads the same entries. The map persists across runs.
|
||||
*/
|
||||
readonly entries: Map<string, T>;
|
||||
/** The clock. The store reads it for the idle deadline and the sweep. */
|
||||
readonly now: () => number;
|
||||
/**
|
||||
* The idle bound, in milliseconds. When it is `0` or less, the store arms no
|
||||
* per-entry timer and the sweep evicts nothing.
|
||||
*/
|
||||
readonly idleMs: number;
|
||||
/** Read an entry's last-used time. The store compares it to the idle bound. */
|
||||
readonly lastUsedAt: (entry: T) => number;
|
||||
/**
|
||||
* Release an entry's resources. The store calls it once per entry on a
|
||||
* discard, on a per-entry timer fire, or on an idle sweep. The store guards
|
||||
* every release with a map-identity check, so a discarded entry never
|
||||
* releases twice.
|
||||
*/
|
||||
readonly release: (entry: T) => void | Promise<void>;
|
||||
/**
|
||||
* Read an entry's per-entry idle timer, or `undefined` when it holds none.
|
||||
* The host lane supplies this; the sandbox lane omits it and arms no timer.
|
||||
*/
|
||||
readonly getTimer?: (entry: T) => ReturnType<typeof setTimeout> | undefined;
|
||||
/**
|
||||
* Write (or clear with `undefined`) an entry's per-entry idle timer. The host
|
||||
* lane supplies this; the sandbox lane omits it.
|
||||
*/
|
||||
readonly setTimer?: (entry: T, timer: ReturnType<typeof setTimeout> | undefined) => void;
|
||||
/**
|
||||
* Arm a per-entry idle timer on `save`. The host lane sets it true. The
|
||||
* sandbox lane leaves it false and relies on the run-start sweep only.
|
||||
*/
|
||||
readonly perEntryTimer?: boolean;
|
||||
/**
|
||||
* Run each eviction critical section under a per-key lease. The sandbox lane
|
||||
* supplies its per-key staging lease so the sweep re-checks the entry inside
|
||||
* the lease. The host lane omits it and the sweep runs the critical section
|
||||
* directly.
|
||||
*/
|
||||
readonly withEvictLease?: (key: string, critical: () => Promise<void>) => Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a generic per-session reuse store over the caller-provided map. The
|
||||
* store holds at most one entry per session key. Each entry carries its saved
|
||||
* value, its last-used time, and an optional per-entry idle timer, all read
|
||||
* through the config accessors.
|
||||
*/
|
||||
export function createSessionReuseStore<T>(
|
||||
config: SessionReuseStoreConfig<T>,
|
||||
): SessionReuseStore<T> {
|
||||
const { entries } = config;
|
||||
|
||||
function clearEntryTimer(entry: T): void {
|
||||
const timer = config.getTimer?.(entry);
|
||||
if (timer === undefined) return;
|
||||
clearTimeout(timer);
|
||||
config.setTimer?.(entry, undefined);
|
||||
}
|
||||
|
||||
// Release an entry once and drop it from the map when it is still the current
|
||||
// entry at the key. The map-identity guard makes the release idempotent: the
|
||||
// synchronous `delete` runs before the awaited release, so a second release of
|
||||
// the same entry finds a different or absent entry at the key and stops.
|
||||
async function releaseEntry(key: string, entry: T): Promise<void> {
|
||||
clearEntryTimer(entry);
|
||||
if (entries.get(key) !== entry) return;
|
||||
entries.delete(key);
|
||||
await config.release(entry);
|
||||
}
|
||||
|
||||
function armTimer(key: string, entry: T): void {
|
||||
if (!config.perEntryTimer || config.idleMs <= 0) return;
|
||||
clearEntryTimer(entry);
|
||||
const delayMs = Math.max(1, config.lastUsedAt(entry) + config.idleMs - config.now());
|
||||
const timer = setTimeout(() => {
|
||||
void (async () => {
|
||||
if (entries.get(key) !== entry) return;
|
||||
// A later touch can move `lastUsedAt` forward, so re-check the idle
|
||||
// window and re-arm the timer when the entry is still in use.
|
||||
if (config.now() - config.lastUsedAt(entry) < config.idleMs) {
|
||||
armTimer(key, entry);
|
||||
return;
|
||||
}
|
||||
await releaseEntry(key, entry);
|
||||
})();
|
||||
}, delayMs);
|
||||
timer.unref?.();
|
||||
config.setTimer?.(entry, timer);
|
||||
}
|
||||
|
||||
return {
|
||||
borrow(sessionKey: string): T | undefined {
|
||||
const entry = entries.get(sessionKey);
|
||||
if (entry === undefined) return undefined;
|
||||
// The entry is in use now, so clear its per-entry idle timer. The next
|
||||
// `save` re-arms a fresh timer after the run finishes.
|
||||
clearEntryTimer(entry);
|
||||
return entry;
|
||||
},
|
||||
|
||||
save(sessionKey: string, entry: T): void {
|
||||
// The caller sets the entry's last-used time before it saves, so `save`
|
||||
// sets the entry as-is and arms its timer from that time.
|
||||
entries.set(sessionKey, entry);
|
||||
armTimer(sessionKey, entry);
|
||||
},
|
||||
|
||||
discard(sessionKey: string): Promise<void> {
|
||||
const entry = entries.get(sessionKey);
|
||||
// `releaseEntry` deletes the key synchronously and then fires the release,
|
||||
// so the returned promise lets the caller await the release when it must
|
||||
// finish before a downstream resource release.
|
||||
if (entry === undefined) return Promise.resolve();
|
||||
return releaseEntry(sessionKey, entry);
|
||||
},
|
||||
|
||||
async evictIdle(now: number): Promise<void> {
|
||||
if (config.idleMs <= 0) return;
|
||||
const stale: Array<[string, T]> = [];
|
||||
for (const [key, entry] of entries.entries()) {
|
||||
if (now - config.lastUsedAt(entry) >= config.idleMs) stale.push([key, entry]);
|
||||
}
|
||||
for (const [key, entry] of stale) {
|
||||
const critical = async (): Promise<void> => {
|
||||
if (entries.get(key) !== entry) return;
|
||||
// Under a lease the sweep re-checks the idle window, so a run that
|
||||
// re-saved the key while the lease was held keeps its entry.
|
||||
if (config.withEvictLease && config.now() - config.lastUsedAt(entry) < config.idleMs) {
|
||||
return;
|
||||
}
|
||||
await releaseEntry(key, entry);
|
||||
};
|
||||
if (config.withEvictLease) {
|
||||
await config.withEvictLease(key, critical);
|
||||
} else {
|
||||
await critical();
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -354,9 +354,11 @@ describe("ACP settlement — Layer A: engine teardown orchestration", () => {
|
|||
expect(stagingLocks.size).toBe(0);
|
||||
});
|
||||
|
||||
it("test_clean_persistent_local_run_warm_saves_instead_of_closing", async () => {
|
||||
// execute.ts:3886 keeps a clean persistent local turn warm (no runtime.close)
|
||||
// when warmIdleMs>0 and there is no process-session bridge.
|
||||
it("test_clean_persistent_local_run_closes_and_relaunches_amendment_b", async () => {
|
||||
// Amendment B: a clean persistent local turn closes the runtime instead of
|
||||
// warm-saving it. The host-lane reuse candidate carries the run-minted API key,
|
||||
// which is never revoked, so the credential gate fails and the settlement
|
||||
// discards (close-and-relaunch).
|
||||
const root = await makeTempRoot();
|
||||
const closeSpy = vi.fn(async () => {});
|
||||
const warmHandles = new Map();
|
||||
|
|
@ -393,16 +395,9 @@ describe("ACP settlement — Layer A: engine teardown orchestration", () => {
|
|||
} as never);
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
// The runtime stayed warm: no close, one warm entry.
|
||||
expect(closeSpy).not.toHaveBeenCalled();
|
||||
expect(warmHandles.size).toBe(1);
|
||||
|
||||
// Clear the scheduled idle-cleanup timer so no timer outlives the test. The
|
||||
// long idle window kept the timer from racing the assertions above; clearing
|
||||
// it here makes cleanup deterministic instead of a timed drain.
|
||||
for (const entry of warmHandles.values()) {
|
||||
if (entry.cleanupTimer) clearTimeout(entry.cleanupTimer);
|
||||
}
|
||||
// The runtime closed and relaunched: one close, no warm entry.
|
||||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
expect(warmHandles.size).toBe(0);
|
||||
});
|
||||
|
||||
it("test_completed_non_persistent_local_run_closes_runtime_once", async () => {
|
||||
|
|
@ -543,12 +538,12 @@ describe("ACP settlement — Layer A: engine teardown orchestration", () => {
|
|||
}
|
||||
});
|
||||
|
||||
it("test_handshake_failure_after_warm_hit_closes_runtime_and_removes_warm_entry", async () => {
|
||||
// Mirror execute.test.ts F2 test_handshake_failure_closes_runtime_and_removes_warm_entry
|
||||
// (:4666). The first run warms a handle; the second run reuses it and fails
|
||||
// while persisting process identity (onSpawn throws). Because the warm hit
|
||||
// already holds the handle, the handshake failure closes the reused runtime
|
||||
// (execute.ts:3591) and removes the warm entry.
|
||||
it("test_handshake_failure_on_second_run_closes_the_recreated_runtime", async () => {
|
||||
// Amendment B: the first clean persistent turn closes and relaunches instead of
|
||||
// warm-saving, so the second run finds no warm handle. The second run re-creates
|
||||
// the runtime and fails while persisting process identity (onSpawn throws); the
|
||||
// handshake failure closes the re-created runtime. The host-lane warm-hit
|
||||
// teardown path itself is characterized by the composed fault matrix.
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
const startedAt = "2026-01-01T00:00:00.000Z";
|
||||
|
|
@ -591,7 +586,8 @@ describe("ACP settlement — Layer A: engine teardown orchestration", () => {
|
|||
onSpawn: async () => {},
|
||||
} as never);
|
||||
expect(first.exitCode).toBe(0);
|
||||
expect(warmHandles.size).toBe(1);
|
||||
// Amendment B: no warm entry survives the first clean persistent turn.
|
||||
expect(warmHandles.size).toBe(0);
|
||||
expect(created).toBe(1);
|
||||
|
||||
const second = await execute({
|
||||
|
|
@ -607,11 +603,12 @@ describe("ACP settlement — Layer A: engine teardown orchestration", () => {
|
|||
},
|
||||
} as never);
|
||||
|
||||
// The reused runtime (no new create) is closed and the warm entry removed.
|
||||
expect(created).toBe(1);
|
||||
// The second run re-created the runtime (no warm reuse) and closed it on the
|
||||
// failure, reporting the ensure_session phase. Both runtimes closed.
|
||||
expect(created).toBe(2);
|
||||
expect(second.exitCode).toBe(1);
|
||||
expect(second.resultJson?.phase).toBe("ensure_session");
|
||||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
expect(closeSpy).toHaveBeenCalledTimes(2);
|
||||
expect(warmHandles.size).toBe(0);
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,323 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
decideReuse,
|
||||
settleAcpRun,
|
||||
type ReuseCandidateInput,
|
||||
type ReuseDecision,
|
||||
type SettlementSlots,
|
||||
type SettlementSteps,
|
||||
} from "./settlement-sequence.js";
|
||||
import { createRunResourceLedger } from "./run-resource-ledger.js";
|
||||
import type { AcquiredRunResources, ResourceId, SettlementCause } from "./run-contracts.js";
|
||||
|
||||
// The settlement sequence is the one cleanup owner. These tests drive it with
|
||||
// fakes for the steps and assert the claim, the order, the Phase 3 error policy,
|
||||
// the staging-lease finally, and the Amendment B credential gate. They never
|
||||
// touch the engine's real run state.
|
||||
|
||||
// Register a per-run resource for each id, with an optional payload the steps
|
||||
// read. The tests do not seal the ledger; `takeForSettlement` runs from `open`.
|
||||
function ledgerWith(
|
||||
ids: readonly ResourceId[],
|
||||
payloads: Partial<Record<ResourceId, unknown>> = {},
|
||||
): AcquiredRunResources {
|
||||
const ledger = createRunResourceLedger();
|
||||
const register = ledger.register as (registration: {
|
||||
id: ResourceId;
|
||||
payload: unknown;
|
||||
scope: "per_run";
|
||||
}) => void;
|
||||
for (const id of ids) {
|
||||
register({ id, payload: payloads[id] ?? {}, scope: "per_run" });
|
||||
}
|
||||
return ledger;
|
||||
}
|
||||
|
||||
// Count the claims so a test proves the sequence claims the ledger once.
|
||||
function countingLedger(inner: AcquiredRunResources): {
|
||||
ledger: AcquiredRunResources;
|
||||
claims: () => number;
|
||||
} {
|
||||
let claims = 0;
|
||||
const ledger: AcquiredRunResources = {
|
||||
register: inner.register.bind(inner),
|
||||
seal: inner.seal.bind(inner),
|
||||
takeForSettlement: () => {
|
||||
claims += 1;
|
||||
return inner.takeForSettlement();
|
||||
},
|
||||
};
|
||||
return { ledger, claims: () => claims };
|
||||
}
|
||||
|
||||
// A base set of steps whose bodies record their name. Each test overrides the
|
||||
// steps it drives.
|
||||
function baseSteps(order: string[], overrides: Partial<SettlementSteps> = {}): SettlementSteps {
|
||||
return {
|
||||
reuseCandidate: overrides.reuseCandidate ?? (() => null),
|
||||
endSession: overrides.endSession ?? (() => {
|
||||
order.push("end_session");
|
||||
}),
|
||||
settleReuse: overrides.settleReuse ?? (() => {
|
||||
order.push("settle_reuse");
|
||||
}),
|
||||
stopTransport: overrides.stopTransport ?? (() => {
|
||||
order.push("stop_transport");
|
||||
}),
|
||||
syncBack: overrides.syncBack ?? (() => {
|
||||
order.push("sync_back");
|
||||
}),
|
||||
releaseStagingLease: overrides.releaseStagingLease ?? (() => {
|
||||
order.push("release_staging_lease");
|
||||
}),
|
||||
recordError: overrides.recordError ?? (() => {}),
|
||||
};
|
||||
}
|
||||
|
||||
const cleanTurnCause: SettlementCause | null = null;
|
||||
|
||||
describe("ACPX settlement sequence", () => {
|
||||
it("test_settlement_claims_ledger_once_then_runs_the_steps_in_order", async () => {
|
||||
const order: string[] = [];
|
||||
const { ledger, claims } = countingLedger(
|
||||
ledgerWith(["acp_runtime", "control_bridge", "agent_bridge", "staging_lease"]),
|
||||
);
|
||||
|
||||
await settleAcpRun(ledger, cleanTurnCause, baseSteps(order));
|
||||
|
||||
// The sequence claims the ledger exactly once.
|
||||
expect(claims()).toBe(1);
|
||||
// It runs the effect steps in the fixed order; the lease release runs last.
|
||||
expect(order).toEqual([
|
||||
"end_session",
|
||||
"settle_reuse",
|
||||
"stop_transport",
|
||||
"sync_back",
|
||||
"release_staging_lease",
|
||||
]);
|
||||
});
|
||||
|
||||
it("test_every_step_noops_on_empty_slot", async () => {
|
||||
const order: string[] = [];
|
||||
// An empty ledger: no slot holds a resource. Each step reads its slot and
|
||||
// no-ops when the slot is empty.
|
||||
const ledger = ledgerWith([]);
|
||||
const noopFor = (name: string, id: ResourceId) => (slots: SettlementSlots) => {
|
||||
order.push(slots.has(id) ? `${name}:work` : `${name}:noop`);
|
||||
};
|
||||
const steps = baseSteps(order, {
|
||||
endSession: noopFor("end_session", "acp_runtime"),
|
||||
settleReuse: noopFor("settle_reuse", "acp_runtime"),
|
||||
stopTransport: noopFor("stop_transport", "control_bridge"),
|
||||
syncBack: noopFor("sync_back", "staged_runtime"),
|
||||
releaseStagingLease: (slots) => {
|
||||
order.push(slots.has("staging_lease") ? "release_staging_lease:work" : "release_staging_lease:noop");
|
||||
},
|
||||
});
|
||||
|
||||
const report = await settleAcpRun(ledger, cleanTurnCause, steps);
|
||||
|
||||
// Every step saw an empty slot and no-opped.
|
||||
expect(order).toEqual([
|
||||
"end_session:noop",
|
||||
"settle_reuse:noop",
|
||||
"stop_transport:noop",
|
||||
"sync_back:noop",
|
||||
"release_staging_lease:noop",
|
||||
]);
|
||||
// An empty ledger yields an empty disposition report.
|
||||
expect(report.records).toEqual([]);
|
||||
});
|
||||
|
||||
it("test_transferred_runtime_is_never_closed_and_discarded_entry_is_finalized", async () => {
|
||||
const order: string[] = [];
|
||||
const ledger = ledgerWith(["acp_runtime", "staging_lease"], {
|
||||
acp_runtime: { runtime: {}, sessionHandle: {}, childProcessPid: null },
|
||||
});
|
||||
const steps = baseSteps(order, {
|
||||
// A clean host turn with no live run-scoped credential is save-eligible.
|
||||
reuseCandidate: (): ReuseCandidateInput => ({
|
||||
kind: "host",
|
||||
causePermitsSave: true,
|
||||
liveRunScopedCredentials: [],
|
||||
}),
|
||||
endSession: (_slots, decision) => {
|
||||
// The transferred runtime is never closed.
|
||||
order.push(decision.kind === "save" ? "kept" : "closed");
|
||||
},
|
||||
settleReuse: (_slots, decision) => {
|
||||
order.push(decision.kind === "save" ? "transferred" : "discarded");
|
||||
},
|
||||
});
|
||||
|
||||
const report = await settleAcpRun(ledger, cleanTurnCause, steps);
|
||||
|
||||
// The runtime transferred; it was never closed.
|
||||
expect(order).toContain("kept");
|
||||
expect(order).not.toContain("closed");
|
||||
expect(order).toContain("transferred");
|
||||
// The report marks the transferred runtime `transferred` and the discarded
|
||||
// staging lease `finalized`.
|
||||
expect(report.records).toEqual([
|
||||
{ id: "acp_runtime", disposition: "transferred" },
|
||||
{ id: "staging_lease", disposition: "finalized" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("test_step_error_is_recorded_and_later_steps_still_run", async () => {
|
||||
const order: string[] = [];
|
||||
const recorded: string[] = [];
|
||||
const boom = new Error("end_session boom");
|
||||
const steps = baseSteps(order, {
|
||||
endSession: () => {
|
||||
throw boom;
|
||||
},
|
||||
recordError: (step, error) => {
|
||||
recorded.push(step);
|
||||
expect(error).toBe(boom);
|
||||
},
|
||||
});
|
||||
|
||||
await settleAcpRun(ledgerWith(["acp_runtime", "staging_lease"]), cleanTurnCause, steps);
|
||||
|
||||
// The failed step is recorded...
|
||||
expect(recorded).toContain("end_session");
|
||||
// ...and every later step still runs.
|
||||
expect(order).toEqual([
|
||||
"settle_reuse",
|
||||
"stop_transport",
|
||||
"sync_back",
|
||||
"release_staging_lease",
|
||||
]);
|
||||
});
|
||||
|
||||
it("test_staging_lease_release_runs_in_finally", async () => {
|
||||
const order: string[] = [];
|
||||
const recorded: string[] = [];
|
||||
const steps = baseSteps(order, {
|
||||
syncBack: () => {
|
||||
throw new Error("sync_back boom");
|
||||
},
|
||||
recordError: (step) => {
|
||||
recorded.push(step);
|
||||
},
|
||||
});
|
||||
|
||||
await settleAcpRun(ledgerWith(["staging_lease"]), cleanTurnCause, steps);
|
||||
|
||||
// The sync-back failure is recorded, and the lease release still runs.
|
||||
expect(recorded).toContain("sync_back");
|
||||
expect(order).toContain("release_staging_lease");
|
||||
});
|
||||
|
||||
it("test_two_compatible_runs_do_not_share_run_scoped_credentials", async () => {
|
||||
// Two compatible runs each own their ledger, bridge token, and run API key.
|
||||
// Run B must never observe run A's run-scoped values.
|
||||
const observed: Array<{ run: string; token: unknown; credentials: readonly string[] }> = [];
|
||||
const settleRun = async (run: string, token: string, credential: string) => {
|
||||
const ledger = ledgerWith(["acp_runtime", "control_bridge"], {
|
||||
control_bridge: { token },
|
||||
});
|
||||
const steps = baseSteps([], {
|
||||
reuseCandidate: (): ReuseCandidateInput => ({
|
||||
kind: "host",
|
||||
causePermitsSave: true,
|
||||
liveRunScopedCredentials: [credential],
|
||||
}),
|
||||
stopTransport: (slots) => {
|
||||
const bridge = slots.get("control_bridge") as { token: unknown } | undefined;
|
||||
observed.push({ run, token: bridge?.token, credentials: [credential] });
|
||||
},
|
||||
});
|
||||
return settleAcpRun(ledger, cleanTurnCause, steps);
|
||||
};
|
||||
|
||||
await settleRun("A", "token-A", "run-api-key-A");
|
||||
await settleRun("B", "token-B", "run-api-key-B");
|
||||
|
||||
const runA = observed.find((o) => o.run === "A");
|
||||
const runB = observed.find((o) => o.run === "B");
|
||||
// Run B observed only its own bridge token and credential, never run A's.
|
||||
expect(runB?.token).toBe("token-B");
|
||||
expect(runB?.token).not.toBe(runA?.token);
|
||||
expect(runB?.credentials).not.toContain("run-api-key-A");
|
||||
});
|
||||
|
||||
it("test_no_run_scoped_credential_appears_in_reuse_payload_or_telemetry", async () => {
|
||||
const runScopedCredential = "run-api-key-secret";
|
||||
let savedPayload: Record<string, unknown> | null = null;
|
||||
// A save-eligible sandbox candidate: the staged-files payload carries no
|
||||
// credential, and the bridge tokens already died with the bridges, so the
|
||||
// live run-scoped credential list is empty.
|
||||
const ledger = ledgerWith(["staged_runtime", "staging_lease"], {
|
||||
staged_runtime: { stagedRuntime: { files: ["a", "b"] } },
|
||||
});
|
||||
const steps = baseSteps([], {
|
||||
reuseCandidate: (): ReuseCandidateInput => ({
|
||||
kind: "sandbox",
|
||||
causePermitsSave: true,
|
||||
liveRunScopedCredentials: [],
|
||||
}),
|
||||
settleReuse: (slots, decision) => {
|
||||
if (decision.kind === "save") {
|
||||
const staged = slots.get("staged_runtime") as unknown as {
|
||||
stagedRuntime: Record<string, unknown>;
|
||||
};
|
||||
savedPayload = staged.stagedRuntime;
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
const report = await settleAcpRun(ledger, cleanTurnCause, steps);
|
||||
|
||||
// The saved reuse payload carries no run-scoped credential.
|
||||
expect(savedPayload).not.toBeNull();
|
||||
expect(JSON.stringify(savedPayload)).not.toContain(runScopedCredential);
|
||||
// The disposition report (telemetry) carries no run-scoped credential.
|
||||
expect(JSON.stringify(report)).not.toContain(runScopedCredential);
|
||||
// The staged files transferred.
|
||||
expect(report.records).toContainEqual({ id: "staged_runtime", disposition: "transferred" });
|
||||
});
|
||||
|
||||
it("test_warm_save_requires_credential_gate_pass_else_close_and_relaunch", async () => {
|
||||
const runWithCredentials = async (liveRunScopedCredentials: readonly string[]) => {
|
||||
const order: string[] = [];
|
||||
const ledger = ledgerWith(["acp_runtime"]);
|
||||
const steps = baseSteps(order, {
|
||||
reuseCandidate: (): ReuseCandidateInput => ({
|
||||
kind: "host",
|
||||
causePermitsSave: true,
|
||||
liveRunScopedCredentials,
|
||||
}),
|
||||
endSession: (_slots, decision) => {
|
||||
order.push(decision.kind === "save" ? "kept" : "closed");
|
||||
},
|
||||
});
|
||||
const report = await settleAcpRun(ledger, cleanTurnCause, steps);
|
||||
return { order, report };
|
||||
};
|
||||
|
||||
// A live run-scoped credential fails the gate: close-and-relaunch, finalized.
|
||||
const gated = await runWithCredentials(["run-api-key"]);
|
||||
expect(gated.order).toContain("closed");
|
||||
expect(gated.report.records).toContainEqual({ id: "acp_runtime", disposition: "finalized" });
|
||||
|
||||
// No live run-scoped credential passes the gate: the runtime transfers.
|
||||
const cleared = await runWithCredentials([]);
|
||||
expect(cleared.order).toContain("kept");
|
||||
expect(cleared.report.records).toContainEqual({ id: "acp_runtime", disposition: "transferred" });
|
||||
});
|
||||
|
||||
it("decideReuse blocks a save when the Amendment B credential gate fails", () => {
|
||||
// A direct unit check on the pure decision, so the gate is legible on its own.
|
||||
const base: ReuseCandidateInput = { kind: "host", causePermitsSave: true, liveRunScopedCredentials: [] };
|
||||
expect(decideReuse(base)).toEqual({ kind: "save", savedId: "acp_runtime" } satisfies ReuseDecision);
|
||||
expect(decideReuse({ ...base, liveRunScopedCredentials: ["k"] }).kind).toBe("discard");
|
||||
expect(decideReuse({ ...base, causePermitsSave: false }).kind).toBe("discard");
|
||||
expect(decideReuse(null).kind).toBe("discard");
|
||||
expect(decideReuse({ ...base, kind: "sandbox" })).toEqual({
|
||||
kind: "save",
|
||||
savedId: "staged_runtime",
|
||||
} satisfies ReuseDecision);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,204 @@
|
|||
// The settlement sequence.
|
||||
//
|
||||
// `settleAcpRun(ledger, cause, steps)` is the one cleanup owner for every site
|
||||
// and every settled path. The one exception is the startup rollback, which the
|
||||
// coordinator owns (Phase 20).
|
||||
//
|
||||
// The sequence claims the ledger once, makes the pure reuse decision, then runs
|
||||
// the ordered steps:
|
||||
//
|
||||
// 1. endSession — close every runtime the decision did not transfer, and
|
||||
// drop the warm entry when it closes.
|
||||
// 2. settleReuse — perform the decision: a save transfers the candidate to the
|
||||
// site store (which arms the per-entry idle timer); everything
|
||||
// else discards.
|
||||
// 3. stopTransport — stop both bridges in one `allSettled`.
|
||||
// 4. syncBack — the site sync-back.
|
||||
// 5. releaseStagingLease — runs in a `finally`, so an earlier step fault never
|
||||
// strands the lease.
|
||||
//
|
||||
// The Phase 3 error policy governs every step: a step records its error and the
|
||||
// later steps still run. Every step no-ops on an empty slot.
|
||||
//
|
||||
// Amendment B — the credential gate. A runtime is save-eligible only when no
|
||||
// run-scoped credential issued to it remains valid. The decision is pure and
|
||||
// runs before any close. Settlement stops both bridges before a sandbox-lane
|
||||
// transfer, so the bridge tokens die with the bridges. The run API key is not
|
||||
// invalidated on run end (grounded server-side), so the host-lane warm save
|
||||
// discards instead of transfers (close-and-relaunch) until a credential-rebind
|
||||
// protocol exists. The sandbox staged-files reuse is unaffected: its payload
|
||||
// carries no credentials.
|
||||
|
||||
import type {
|
||||
AcquiredRunResources,
|
||||
ResourceDisposition,
|
||||
ResourceId,
|
||||
RunResourcePayloads,
|
||||
RunSiteKind,
|
||||
SettledResourceEntry,
|
||||
SettlementCause,
|
||||
} from "./run-contracts.js";
|
||||
|
||||
/** The save-versus-discard decision the settlement makes before any close. */
|
||||
export type ReuseDecision =
|
||||
| { readonly kind: "save"; readonly savedId: ResourceId }
|
||||
| { readonly kind: "discard"; readonly reason: string };
|
||||
|
||||
/**
|
||||
* The inputs the reuse decision reads. The site derives the candidate from the
|
||||
* settled slots. A null candidate means the run holds nothing to save.
|
||||
*/
|
||||
export interface ReuseCandidateInput {
|
||||
/** The site kind. "host" saves the runtime; "sandbox" saves the staged files. */
|
||||
readonly kind: RunSiteKind;
|
||||
/** True when the settlement cause permits a save (a clean persistent turn). */
|
||||
readonly causePermitsSave: boolean;
|
||||
/**
|
||||
* Amendment B: the run-scoped credentials the candidate would still carry into
|
||||
* the store. A save is eligible only when this list is empty — no run-scoped
|
||||
* credential issued to the runtime remains valid.
|
||||
*/
|
||||
readonly liveRunScopedCredentials: readonly string[];
|
||||
}
|
||||
|
||||
/** One resource's final disposition, as the settlement records it. */
|
||||
export interface ResourceDispositionRecord {
|
||||
readonly id: ResourceId;
|
||||
readonly disposition: ResourceDisposition;
|
||||
}
|
||||
|
||||
/** The per-resource disposition report the settlement produces at the end. */
|
||||
export interface SettlementDispositionReport {
|
||||
readonly records: readonly ResourceDispositionRecord[];
|
||||
}
|
||||
|
||||
/** The settled slots view a step reads. `get` returns undefined for an empty slot. */
|
||||
export interface SettlementSlots {
|
||||
get<Id extends ResourceId>(id: Id): RunResourcePayloads[Id] | undefined;
|
||||
has(id: ResourceId): boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The effectful steps the sequence runs. The engine implements each step over
|
||||
* its own run state. Each step no-ops on an empty slot. A step must not throw
|
||||
* for the sequence to record its error and continue; a throw is caught and
|
||||
* recorded under the Phase 3 error policy.
|
||||
*/
|
||||
export interface SettlementSteps {
|
||||
/**
|
||||
* Derive the reuse candidate from the settled slots and the cause, or null
|
||||
* when the run holds nothing to save.
|
||||
*/
|
||||
reuseCandidate(slots: SettlementSlots, cause: SettlementCause | null): ReuseCandidateInput | null;
|
||||
/** Close every runtime the decision did not transfer; drop the warm entry on close. */
|
||||
endSession(slots: SettlementSlots, decision: ReuseDecision): Promise<void> | void;
|
||||
/** Perform the decision: a save transfers to the site store; everything else discards. */
|
||||
settleReuse(slots: SettlementSlots, decision: ReuseDecision): Promise<void> | void;
|
||||
/** Stop both bridges in one `allSettled`. */
|
||||
stopTransport(slots: SettlementSlots): Promise<void> | void;
|
||||
/** The site sync-back. */
|
||||
syncBack(slots: SettlementSlots): Promise<void> | void;
|
||||
/** Release the staging lease. Runs in a `finally`. */
|
||||
releaseStagingLease(slots: SettlementSlots): void;
|
||||
/** Record a per-step error under the Phase 3 error policy. */
|
||||
recordError(step: string, error: unknown): Promise<void> | void;
|
||||
}
|
||||
|
||||
/**
|
||||
* The pure reuse decision. A save is eligible only when a candidate exists, the
|
||||
* cause permits a save, and Amendment B holds: no run-scoped credential issued
|
||||
* to the candidate remains valid. Everything else discards, with a reason.
|
||||
*/
|
||||
export function decideReuse(candidate: ReuseCandidateInput | null): ReuseDecision {
|
||||
if (!candidate) {
|
||||
return { kind: "discard", reason: "no reuse candidate" };
|
||||
}
|
||||
if (!candidate.causePermitsSave) {
|
||||
return { kind: "discard", reason: "settlement cause forbids save" };
|
||||
}
|
||||
if (candidate.liveRunScopedCredentials.length > 0) {
|
||||
// Amendment B credential gate: a live run-scoped credential blocks the save.
|
||||
return { kind: "discard", reason: "run-scoped credential still valid" };
|
||||
}
|
||||
const savedId: ResourceId = candidate.kind === "host" ? "acp_runtime" : "staged_runtime";
|
||||
return { kind: "save", savedId };
|
||||
}
|
||||
|
||||
/**
|
||||
* Settle the run's resources. Claims the ledger once, makes the reuse decision,
|
||||
* runs the ordered steps under the Phase 3 error policy, releases the staging
|
||||
* lease in a `finally`, and returns the final per-resource disposition report.
|
||||
*/
|
||||
export async function settleAcpRun(
|
||||
ledger: AcquiredRunResources,
|
||||
cause: SettlementCause | null,
|
||||
steps: SettlementSteps,
|
||||
): Promise<SettlementDispositionReport> {
|
||||
// Claim the ledger once. Every later step reads the claimed slots.
|
||||
const consumed = ledger.takeForSettlement();
|
||||
const entries = consumed.entries();
|
||||
const slots = makeSlots(entries);
|
||||
|
||||
// The reuse decision is pure and runs before any close.
|
||||
const candidate = steps.reuseCandidate(slots, cause);
|
||||
const decision = decideReuse(candidate);
|
||||
|
||||
const runStep = async (name: string, run: () => Promise<void> | void): Promise<void> => {
|
||||
try {
|
||||
await run();
|
||||
} catch (error) {
|
||||
// The Phase 3 error policy: record the step error and let later steps run.
|
||||
await steps.recordError(name, error);
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
await runStep("end_session", () => steps.endSession(slots, decision));
|
||||
await runStep("settle_reuse", () => steps.settleReuse(slots, decision));
|
||||
await runStep("stop_transport", () => steps.stopTransport(slots));
|
||||
await runStep("sync_back", () => steps.syncBack(slots));
|
||||
} finally {
|
||||
// The staging lease release runs in a `finally`, so an earlier step fault
|
||||
// never strands the lease.
|
||||
try {
|
||||
steps.releaseStagingLease(slots);
|
||||
} catch (error) {
|
||||
await steps.recordError("release_staging_lease", error);
|
||||
}
|
||||
}
|
||||
|
||||
return buildReport(entries, decision);
|
||||
}
|
||||
|
||||
/** Build the settled slots view from the claimed entries. */
|
||||
function makeSlots(entries: readonly SettledResourceEntry[]): SettlementSlots {
|
||||
const byId = new Map<ResourceId, SettledResourceEntry>();
|
||||
for (const entry of entries) byId.set(entry.id, entry);
|
||||
return {
|
||||
get<Id extends ResourceId>(id: Id): RunResourcePayloads[Id] | undefined {
|
||||
return byId.get(id)?.payload as RunResourcePayloads[Id] | undefined;
|
||||
},
|
||||
has(id: ResourceId): boolean {
|
||||
return byId.has(id);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the final per-resource disposition report. The claim-time value the
|
||||
* ledger stamps ("transferred") is not the final report; the settlement decides
|
||||
* it here. A saved resource is `transferred`; every other resource is
|
||||
* `finalized`.
|
||||
*/
|
||||
function buildReport(
|
||||
entries: readonly SettledResourceEntry[],
|
||||
decision: ReuseDecision,
|
||||
): SettlementDispositionReport {
|
||||
const savedId = decision.kind === "save" ? decision.savedId : null;
|
||||
return {
|
||||
records: entries.map((entry) => ({
|
||||
id: entry.id,
|
||||
disposition: entry.id === savedId ? ("transferred" as const) : ("finalized" as const),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
|
@ -629,7 +629,7 @@ describe("ACPX engine startup characterization", () => {
|
|||
expect(stop).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("cold ensure_session throw: ensure_session error, and the handshake-catch close does NOT fire (no handle yet)", async () => {
|
||||
it("cold ensure_session throw: ensure_session error, and the runtime closes (the cold-handshake leak is fixed)", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const closeSpy = vi.fn(async () => {});
|
||||
const execute = createAcpxEngineExecutor({
|
||||
|
|
@ -655,11 +655,10 @@ describe("ACPX engine startup characterization", () => {
|
|||
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.resultJson?.phase).toBe("ensure_session");
|
||||
// Current behavior (execute.ts ~:3590): the handshake catch closes the runtime
|
||||
// only `if (handle)`. A cold `ensureSession` throw never assigns a handle, so
|
||||
// this close never fires. The warm-hit path below has a cached handle and does
|
||||
// close. This pins the real cold-path behavior, not an aspiration.
|
||||
expect(closeSpy).not.toHaveBeenCalled();
|
||||
// The runtime enters the ledger the moment it is created, so a cold
|
||||
// `ensureSession` throw (before any handle exists) still closes it through the
|
||||
// settlement `endSession` step. This closes the former cold-handshake leak.
|
||||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("missing session handle: ensure_session error and the minimal runtime closes", async () => {
|
||||
|
|
@ -728,7 +727,7 @@ describe("ACPX engine startup characterization", () => {
|
|||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("warm-hit failure: reuses the runtime, fails with ensure_session, closes it, drops the warm entry", async () => {
|
||||
it("persistent host failure: the first run closes and relaunches, so a failing second run re-creates and closes", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
const startedAt = "2026-01-01T00:00:00.000Z";
|
||||
|
|
@ -771,10 +770,13 @@ describe("ACPX engine startup characterization", () => {
|
|||
onSpawn: async () => {},
|
||||
} as never);
|
||||
expect(first.exitCode).toBe(0);
|
||||
expect(warmHandles.size).toBe(1);
|
||||
// Amendment B: the first clean persistent turn closes and relaunches instead
|
||||
// of warm-saving the runtime, so no warm handle survives.
|
||||
expect(warmHandles.size).toBe(0);
|
||||
expect(created).toBe(1);
|
||||
|
||||
// The warm-hit reuses runtime #1 and fails while persisting process identity.
|
||||
// The second run finds no warm handle, so it re-creates the runtime and fails
|
||||
// while persisting process identity during ensure_session.
|
||||
const second = await execute({
|
||||
runId: "warm-2",
|
||||
agent: { id: "agent-1", companyId: "company-1" },
|
||||
|
|
@ -788,12 +790,12 @@ describe("ACPX engine startup characterization", () => {
|
|||
},
|
||||
} as never);
|
||||
|
||||
// No new runtime was created; the reused runtime is closed and the warm entry
|
||||
// removed, with the failure reported on the ensure_session phase.
|
||||
expect(created).toBe(1);
|
||||
// The second run built runtime #2 (no warm reuse), closed it on the failure,
|
||||
// and reported the ensure_session phase. Both runtimes closed (one per run).
|
||||
expect(created).toBe(2);
|
||||
expect(second.exitCode).toBe(1);
|
||||
expect(second.resultJson?.phase).toBe("ensure_session");
|
||||
expect(closeSpy).toHaveBeenCalledTimes(1);
|
||||
expect(closeSpy).toHaveBeenCalledTimes(2);
|
||||
expect(warmHandles.size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
|
@ -820,7 +822,7 @@ describe("ACPX engine startup characterization", () => {
|
|||
expect(runtimeOptions[0]?.cwd).toBe(localCwd);
|
||||
});
|
||||
|
||||
it("persistent host lane: warm-saves the handle so a second run reuses the runtime", async () => {
|
||||
it("persistent host lane: closes and relaunches the handle, so a second run re-creates the runtime", async () => {
|
||||
const root = await makeTempRoot();
|
||||
const stateDir = path.join(root, "state");
|
||||
let created = 0;
|
||||
|
|
@ -857,9 +859,11 @@ describe("ACPX engine startup characterization", () => {
|
|||
|
||||
expect(first.exitCode).toBe(0);
|
||||
expect(second.exitCode).toBe(0);
|
||||
// The warm handle survives, so the second run reuses runtime #1.
|
||||
expect(created).toBe(1);
|
||||
expect(warmHandles.size).toBe(1);
|
||||
// Amendment B: the host lane never warm-saves a live runtime (the run-minted
|
||||
// API key is never revoked), so the first run closes and relaunches. No warm
|
||||
// handle survives, so the second run re-creates runtime #2.
|
||||
expect(created).toBe(2);
|
||||
expect(warmHandles.size).toBe(0);
|
||||
});
|
||||
|
||||
it("remote process-session lane: does NOT warm-save the handle, so a second run re-creates the runtime", async () => {
|
||||
|
|
|
|||
|
|
@ -4,12 +4,15 @@ import type { StartupSpan, StartupTraceContext, StartupTracer } from "./startup-
|
|||
import {
|
||||
clampSpanLabel,
|
||||
createRuntimeSpanRunner,
|
||||
emitRunPhaseTiming,
|
||||
emitSkippedStartupStep,
|
||||
getActiveStepContext,
|
||||
measureStartupStep,
|
||||
NOOP_STARTUP_SPAN,
|
||||
NOOP_STARTUP_TRACE_CONTEXT,
|
||||
normalizeProviderFamily,
|
||||
RUN_PHASE_NAMES,
|
||||
RUN_PHASE_TIMING_EVENT_TYPE,
|
||||
runWithoutActiveStep,
|
||||
runWithRuntimeParent,
|
||||
SANDBOX_STARTUP_SPAN_ATTR_PREFIX,
|
||||
|
|
@ -712,3 +715,59 @@ describe("createRuntimeSpanRunner", () => {
|
|||
expect(childStep).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("run phase timing telemetry", () => {
|
||||
it("test_phase_telemetry_fields_are_exactly_phase_duration_outcome", async () => {
|
||||
const events: AdapterRuntimeEvent[] = [];
|
||||
const ctx = { onEvent: async (event: AdapterRuntimeEvent) => void events.push(event) };
|
||||
|
||||
await emitRunPhaseTiming(ctx, "create_runtime", 12, "ok");
|
||||
|
||||
expect(events).toHaveLength(1);
|
||||
const event = events[0]!;
|
||||
expect(event.eventType).toBe(RUN_PHASE_TIMING_EVENT_TYPE);
|
||||
// The payload carries exactly the three closed fields and nothing else.
|
||||
expect(Object.keys(event.payload ?? {}).sort()).toEqual(["durationMs", "outcome", "phase"]);
|
||||
expect(event.payload).toEqual({ phase: "create_runtime", durationMs: 12, outcome: "ok" });
|
||||
});
|
||||
|
||||
it("test_phase_telemetry_emits_no_command_path_environment_or_identifier_value", async () => {
|
||||
const events: AdapterRuntimeEvent[] = [];
|
||||
const ctx = { onEvent: async (event: AdapterRuntimeEvent) => void events.push(event) };
|
||||
|
||||
// Every allowlisted phase emits exactly one closed-shape event.
|
||||
for (const phase of RUN_PHASE_NAMES) {
|
||||
await emitRunPhaseTiming(ctx, phase, 5, "ok");
|
||||
}
|
||||
// A phase name outside the closed allowlist emits nothing, so a free-form
|
||||
// label (a command, a path, an environment value, or a raw identifier) can
|
||||
// never reach the stream.
|
||||
await emitRunPhaseTiming(ctx, "/usr/bin/node --flag /secret/path", 5, "ok");
|
||||
await emitRunPhaseTiming(ctx, "PAPERCLIP_API_KEY=abc123", 5, "ok");
|
||||
await emitRunPhaseTiming(ctx, "run-7f3a-agent-42", 5, "ok");
|
||||
|
||||
expect(events).toHaveLength(RUN_PHASE_NAMES.length);
|
||||
for (const event of events) {
|
||||
// The phase field is only ever an allowlisted name.
|
||||
expect(RUN_PHASE_NAMES).toContain(event.payload?.phase);
|
||||
// No field carries a command, a path, an environment value, or an identifier.
|
||||
const serialized = JSON.stringify(event);
|
||||
expect(serialized).not.toContain("/usr/bin/node");
|
||||
expect(serialized).not.toContain("/secret/path");
|
||||
expect(serialized).not.toContain("PAPERCLIP_API_KEY");
|
||||
expect(serialized).not.toContain("abc123");
|
||||
expect(serialized).not.toContain("run-7f3a-agent-42");
|
||||
}
|
||||
});
|
||||
|
||||
it("test_telemetry_failure_does_not_fail_the_run", async () => {
|
||||
const ctx = {
|
||||
onEvent: async () => {
|
||||
throw new Error("telemetry sink boom");
|
||||
},
|
||||
};
|
||||
|
||||
// A throwing telemetry sink never propagates, so the run continues.
|
||||
await expect(emitRunPhaseTiming(ctx, "turn", 9, "failed")).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -786,3 +786,72 @@ export async function emitSkippedStartupStep(
|
|||
// Observability must not change startup control flow.
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Structured event emitted once per named run-lifecycle phase, so the duration
|
||||
* and the outcome of each phase land in the run-events stream. It is
|
||||
* observability-only and rides the existing `ctx.onEvent` bridge. The payload is
|
||||
* a closed shape: exactly `phase`, `durationMs`, and `outcome`. The phase name is
|
||||
* from a closed allowlist, so the event never carries a command, an argument, a
|
||||
* path, an environment value, or a raw identifier.
|
||||
*/
|
||||
export const RUN_PHASE_TIMING_EVENT_TYPE = "run.phase.timing";
|
||||
|
||||
/**
|
||||
* The closed set of run-lifecycle phase names. A phase-timing event may name only
|
||||
* one of these. The list is fixed and low-cardinality; it never derives from run
|
||||
* or user data.
|
||||
*/
|
||||
export const RUN_PHASE_NAMES = [
|
||||
"place_workspace",
|
||||
"start_transport",
|
||||
"create_runtime",
|
||||
"ensure_session",
|
||||
"configure_session",
|
||||
"prepare_turn",
|
||||
"turn",
|
||||
"end_session",
|
||||
"settle_reuse",
|
||||
"stop_transport",
|
||||
"sync_back",
|
||||
"release_staging_lease",
|
||||
] as const;
|
||||
|
||||
/** One run-lifecycle phase name from the closed allowlist. */
|
||||
export type RunPhaseName = (typeof RUN_PHASE_NAMES)[number];
|
||||
|
||||
const RUN_PHASE_NAME_SET: ReadonlySet<string> = new Set(RUN_PHASE_NAMES);
|
||||
|
||||
/** The closed outcome set for a phase-timing event. */
|
||||
export type RunPhaseOutcome = "ok" | "failed";
|
||||
|
||||
/**
|
||||
* Emit exactly one `run.phase.timing` event for a run-lifecycle phase. The
|
||||
* payload carries only `phase`, `durationMs`, and `outcome`. It never carries a
|
||||
* command, an argument, a path, an environment value, or a raw identifier. The
|
||||
* phase name must be one member of the closed allowlist; a name outside the
|
||||
* allowlist emits nothing, so a free-form label can never reach the stream. A
|
||||
* negative or a non-finite duration clamps to 0. Every sink call sits inside an
|
||||
* error swallow, so a throwing telemetry sink never fails the run.
|
||||
*/
|
||||
export async function emitRunPhaseTiming(
|
||||
ctx: Pick<AdapterExecutionContext, "onEvent">,
|
||||
phase: string,
|
||||
durationMs: number,
|
||||
outcome: RunPhaseOutcome,
|
||||
): Promise<void> {
|
||||
if (!RUN_PHASE_NAME_SET.has(phase)) return;
|
||||
const safeDuration = Number.isFinite(durationMs) && durationMs > 0 ? durationMs : 0;
|
||||
const event: AdapterRuntimeEvent = {
|
||||
eventType: RUN_PHASE_TIMING_EVENT_TYPE,
|
||||
stream: "system",
|
||||
level: "info",
|
||||
message: `run phase: ${phase} (${safeDuration}ms)`,
|
||||
payload: { phase, durationMs: safeDuration, outcome },
|
||||
};
|
||||
try {
|
||||
await ctx.onEvent?.(event);
|
||||
} catch {
|
||||
// Telemetry never fails the run.
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,190 @@
|
|||
import { describe, expect, it, vi } from "vitest";
|
||||
import { runTurn, type StartedTurn, type TurnFinalizeInput, type TurnSteps } from "./turn-sequence.js";
|
||||
import { createRunResourceLedger } from "./run-resource-ledger.js";
|
||||
import type { TurnCompletion } from "./run-contracts.js";
|
||||
|
||||
// The turn sequence owns the run order, the wall-clock timer, and the abort
|
||||
// controller. These tests drive it with fakes for the five steps and assert the
|
||||
// order, the never-reject guarantee, and the timer ownership. They never touch
|
||||
// the engine's real run state.
|
||||
|
||||
function finalizedTurn(): TurnCompletion {
|
||||
return { kind: "finalized" };
|
||||
}
|
||||
|
||||
function failedTurn(error: unknown): TurnCompletion {
|
||||
return {
|
||||
kind: "failed",
|
||||
cause: { kind: "turn_failed", error: error instanceof Error ? error : new Error(String(error)) },
|
||||
resources: createRunResourceLedger().takeForSettlement(),
|
||||
};
|
||||
}
|
||||
|
||||
function noopStarted(): StartedTurn {
|
||||
return { cancel: async () => {} };
|
||||
}
|
||||
|
||||
// A base set of steps whose bodies do nothing. Each test overrides the steps it
|
||||
// drives and records the calls it cares about.
|
||||
function baseSteps(order: string[], overrides: Partial<TurnSteps<string>> = {}): TurnSteps<string> {
|
||||
return {
|
||||
timeoutMs: overrides.timeoutMs,
|
||||
timeoutMessage: overrides.timeoutMessage ?? "timed out",
|
||||
promptBuild: overrides.promptBuild ?? (async () => {
|
||||
order.push("promptBuild");
|
||||
}),
|
||||
preTurnUsage: overrides.preTurnUsage ?? (async () => {
|
||||
order.push("preTurnUsage");
|
||||
}),
|
||||
turnStart: overrides.turnStart ?? (() => {
|
||||
order.push("turnStart");
|
||||
return noopStarted();
|
||||
}),
|
||||
eventRelay: overrides.eventRelay ?? (async () => {
|
||||
order.push("eventRelay");
|
||||
return "terminal";
|
||||
}),
|
||||
turnFinalize: overrides.turnFinalize ?? (async (input) => {
|
||||
order.push(`turnFinalize:${input.kind}`);
|
||||
return input.kind === "terminal" ? finalizedTurn() : failedTurn(input.error);
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
describe("ACPX turn sequence", () => {
|
||||
it("test_run_turn_never_rejects_every_error_is_a_completion", async () => {
|
||||
// A step error must become a `TurnCompletion`, never a rejection. The event
|
||||
// relay throws after the turn started, so the sequence reports a `turn`
|
||||
// failure and returns a completion.
|
||||
const finalizeInputs: TurnFinalizeInput<string>[] = [];
|
||||
const boom = new Error("relay boom");
|
||||
const steps = baseSteps([], {
|
||||
eventRelay: async () => {
|
||||
throw boom;
|
||||
},
|
||||
turnFinalize: async (input) => {
|
||||
finalizeInputs.push(input);
|
||||
return input.kind === "terminal" ? finalizedTurn() : failedTurn(input.error);
|
||||
},
|
||||
});
|
||||
|
||||
// The returned promise resolves; it never rejects.
|
||||
const completion = await runTurn(steps);
|
||||
expect(completion.kind).toBe("failed");
|
||||
// The error routed through the one finalize step with the `turn` phase.
|
||||
expect(finalizeInputs).toHaveLength(1);
|
||||
expect(finalizeInputs[0]).toMatchObject({ kind: "error", error: boom, phase: "turn" });
|
||||
});
|
||||
|
||||
it("test_run_turn_owns_timer_and_abort_controller", async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const captured: { signal: AbortSignal | null; cancelReason: string | null } = {
|
||||
signal: null,
|
||||
cancelReason: null,
|
||||
};
|
||||
let releaseRelay: (value: string) => void = () => {};
|
||||
const relayPromise = new Promise<string>((resolve) => {
|
||||
releaseRelay = resolve;
|
||||
});
|
||||
const finalizeInputs: TurnFinalizeInput<string>[] = [];
|
||||
const steps = baseSteps([], {
|
||||
timeoutMs: 1000,
|
||||
timeoutMessage: "wall-clock timeout",
|
||||
turnStart: (signal) => {
|
||||
captured.signal = signal;
|
||||
return {
|
||||
cancel: async (reason) => {
|
||||
captured.cancelReason = reason;
|
||||
},
|
||||
};
|
||||
},
|
||||
eventRelay: async () => relayPromise,
|
||||
turnFinalize: async (input) => {
|
||||
finalizeInputs.push(input);
|
||||
return input.kind === "terminal" ? finalizedTurn() : failedTurn(input.error);
|
||||
},
|
||||
});
|
||||
|
||||
const runPromise = runTurn(steps);
|
||||
// Let promptBuild, preTurnUsage, and turnStart run.
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
expect(captured.signal?.aborted).toBe(false);
|
||||
// Fire the wall-clock timeout the sequence owns.
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
// The sequence owns the abort controller: the shared signal is aborted...
|
||||
expect(captured.signal?.aborted).toBe(true);
|
||||
// ...and it cancelled the started turn with the timeout message.
|
||||
expect(captured.cancelReason).toBe("wall-clock timeout");
|
||||
// Complete the relay so the sequence finalizes the timed-out turn.
|
||||
releaseRelay("terminal");
|
||||
const completion = await runPromise;
|
||||
expect(completion).toEqual({ kind: "finalized" });
|
||||
// The finalize step saw the timeout flag the sequence tracked.
|
||||
expect(finalizeInputs).toHaveLength(1);
|
||||
expect(finalizeInputs[0]).toMatchObject({ kind: "terminal", timedOut: true });
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("test_prompt_build_failure_is_failed_completion_with_prepare_turn", async () => {
|
||||
// A prompt-build failure happens before the turn starts, so the sequence
|
||||
// reports a `prepare_turn` failure and never runs the later steps.
|
||||
const order: string[] = [];
|
||||
const finalizeInputs: TurnFinalizeInput<string>[] = [];
|
||||
const boom = new Error("prompt build boom");
|
||||
const steps = baseSteps(order, {
|
||||
promptBuild: async () => {
|
||||
throw boom;
|
||||
},
|
||||
turnFinalize: async (input) => {
|
||||
finalizeInputs.push(input);
|
||||
return input.kind === "terminal" ? finalizedTurn() : failedTurn(input.error);
|
||||
},
|
||||
});
|
||||
|
||||
const completion = await runTurn(steps);
|
||||
expect(completion.kind).toBe("failed");
|
||||
expect(finalizeInputs).toHaveLength(1);
|
||||
expect(finalizeInputs[0]).toMatchObject({ kind: "error", error: boom, phase: "prepare_turn" });
|
||||
// The later steps never ran.
|
||||
expect(order).not.toContain("preTurnUsage");
|
||||
expect(order).not.toContain("turnStart");
|
||||
expect(order).not.toContain("eventRelay");
|
||||
});
|
||||
|
||||
it("test_run_turn_finalizes_no_resource", async () => {
|
||||
// Structural proof: the sequence borrows the resources and finalizes none.
|
||||
// The `TurnSteps` contract has no resource-release method; the sequence runs
|
||||
// only the five ordered steps. A test-side resource whose release the
|
||||
// sequence could never call stays untouched, and the recorded call order is
|
||||
// exactly the five steps. The coordinator settles the resources after the
|
||||
// turn, so a resource release inside the sequence would be a double release.
|
||||
const order: string[] = [];
|
||||
let released = false;
|
||||
// A resource the sequence has no hook to release. If the sequence ever
|
||||
// released a resource, it would have to reach one, and it cannot.
|
||||
const borrowedResource = {
|
||||
release: () => {
|
||||
released = true;
|
||||
},
|
||||
};
|
||||
void borrowedResource;
|
||||
|
||||
const steps = baseSteps(order);
|
||||
const completion = await runTurn(steps);
|
||||
|
||||
expect(completion).toEqual({ kind: "finalized" });
|
||||
// The sequence ran exactly the five ordered steps, then finalized.
|
||||
expect(order).toEqual([
|
||||
"promptBuild",
|
||||
"preTurnUsage",
|
||||
"turnStart",
|
||||
"eventRelay",
|
||||
"turnFinalize:terminal",
|
||||
]);
|
||||
// It never released the borrowed resource.
|
||||
expect(released).toBe(false);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,100 @@
|
|||
// The turn sequence.
|
||||
//
|
||||
// `runTurn(steps)` is a plain sequence. It runs five steps in a fixed order:
|
||||
// promptBuild, preTurnUsage, turnStart, eventRelay, turnFinalize. It never
|
||||
// rejects: every step error becomes a typed `TurnCompletion` through the
|
||||
// `turnFinalize` step. The sequence owns the wall-clock timer and the abort
|
||||
// controller. It borrows the run resources; it finalizes no resource. The
|
||||
// coordinator (Phase 20) settles the resources after the turn.
|
||||
//
|
||||
// The sequence is data, not a graph and not a state machine. The engine supplies
|
||||
// each step; the sequence owns only the order, the timer, and the abort signal.
|
||||
// This keeps the sequence free of the engine's private run state.
|
||||
|
||||
import type { TurnCompletion } from "./run-contracts.js";
|
||||
|
||||
/**
|
||||
* A turn the site started. The sequence borrows it to cancel it on a timeout; it
|
||||
* releases nothing. The engine's `eventRelay` step reads the events and the
|
||||
* terminal result.
|
||||
*/
|
||||
export interface StartedTurn {
|
||||
cancel(reason: string): Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* The input the sequence hands to the one `turnFinalize` step. A `terminal`
|
||||
* input carries the turn's terminal result; an `error` input carries the step
|
||||
* error and the phase it failed in. Both carry the wall-clock timeout flag.
|
||||
*/
|
||||
export type TurnFinalizeInput<TTerminal> =
|
||||
| { readonly kind: "terminal"; readonly terminal: TTerminal; readonly timedOut: boolean }
|
||||
| {
|
||||
readonly kind: "error";
|
||||
readonly error: unknown;
|
||||
readonly phase: "prepare_turn" | "turn";
|
||||
readonly timedOut: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* The five steps the sequence runs, plus the timer inputs. The engine implements
|
||||
* each step over its own run state. `turnFinalize` maps a terminal result or a
|
||||
* step error to a `TurnCompletion`; it must not reject, so the sequence never
|
||||
* rejects.
|
||||
*/
|
||||
export interface TurnSteps<TTerminal> {
|
||||
/** The wall-clock timeout, in milliseconds, or undefined for no timeout. */
|
||||
readonly timeoutMs: number | undefined;
|
||||
/** The message the timeout cancel carries. */
|
||||
readonly timeoutMessage: string;
|
||||
/** Build the prompt and emit the run metadata. A failure here is `prepare_turn`. */
|
||||
promptBuild(signal: AbortSignal): Promise<void>;
|
||||
/** Snapshot the pre-turn usage, so this run's cost excludes earlier runs. */
|
||||
preTurnUsage(): Promise<void>;
|
||||
/** Start the turn with the run's six inputs. The sequence adds the signal and the timeout. */
|
||||
turnStart(signal: AbortSignal, timeoutMs: number | undefined): StartedTurn;
|
||||
/** Relay the turn events and return the terminal result. */
|
||||
eventRelay(turn: StartedTurn): Promise<TTerminal>;
|
||||
/** Map a terminal result or a step error to a `TurnCompletion`. Must not reject. */
|
||||
turnFinalize(input: TurnFinalizeInput<TTerminal>): Promise<TurnCompletion>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the turn sequence. It returns a `TurnCompletion` on every path and never
|
||||
* rejects. A failure before `turnStart` returns is a `prepare_turn` failure; a
|
||||
* failure after it is a `turn` failure. Only the reported phase differs.
|
||||
*/
|
||||
export async function runTurn<TTerminal>(steps: TurnSteps<TTerminal>): Promise<TurnCompletion> {
|
||||
const controller = new AbortController();
|
||||
let timeout: ReturnType<typeof setTimeout> | null = null;
|
||||
let timedOut = false;
|
||||
let started: StartedTurn | null = null;
|
||||
// `turnStarted` separates a pre-turn preparation failure from a running-turn
|
||||
// failure, so `turnFinalize` reports the right phase.
|
||||
let turnStarted = false;
|
||||
try {
|
||||
// Build the prompt and snapshot the pre-turn usage inside the failure
|
||||
// boundary. A failure here is a `prepare_turn` failure.
|
||||
await steps.promptBuild(controller.signal);
|
||||
await steps.preTurnUsage();
|
||||
// The sequence owns the wall-clock timer. On a timeout it marks the run timed
|
||||
// out, aborts the shared signal, and cancels the started turn. The cancel
|
||||
// no-ops before `turnStart` returns, because `started` is still null.
|
||||
if (steps.timeoutMs !== undefined) {
|
||||
timeout = setTimeout(() => {
|
||||
timedOut = true;
|
||||
controller.abort();
|
||||
void started?.cancel(steps.timeoutMessage).catch(() => {});
|
||||
}, steps.timeoutMs);
|
||||
}
|
||||
started = steps.turnStart(controller.signal, steps.timeoutMs);
|
||||
turnStarted = true;
|
||||
const terminal = await steps.eventRelay(started);
|
||||
if (timeout) clearTimeout(timeout);
|
||||
return await steps.turnFinalize({ kind: "terminal", terminal, timedOut });
|
||||
} catch (error) {
|
||||
if (timeout) clearTimeout(timeout);
|
||||
const phase: "prepare_turn" | "turn" = turnStarted ? "turn" : "prepare_turn";
|
||||
return await steps.turnFinalize({ kind: "error", error, phase, timedOut });
|
||||
}
|
||||
}
|
||||
|
|
@ -306,6 +306,45 @@ ensure-session sub-times.
|
|||
To read the detailed timing, use the startup spans. The spans need an OTLP
|
||||
endpoint. A run with no endpoint keeps only the three run-log fields above.
|
||||
|
||||
## Run Phase Timing Run-Log Event
|
||||
|
||||
Paperclip writes one `run.phase.timing` event to the run log for each
|
||||
run-lifecycle phase. This event is a run-log record, not a first-party telemetry
|
||||
event. The generated telemetry contract does not cover it, so this section is its
|
||||
canonical contract. The producer is `emitRunPhaseTiming` in
|
||||
`packages/adapter-utils/src/acpx-engine/startup-timing.ts`.
|
||||
|
||||
The event payload carries only three fields.
|
||||
|
||||
| Field | Type | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `phase` | string | The run-lifecycle phase name from the closed allowlist below. |
|
||||
| `durationMs` | number | The wall time of the phase. A negative or a non-finite value clamps to `0`. |
|
||||
| `outcome` | string | The phase outcome (`ok` or `failed`). |
|
||||
|
||||
The `phase` field is one member of a closed, low-cardinality allowlist. The
|
||||
producer drops any event whose phase name is outside this allowlist, so a
|
||||
free-form label never reaches the run log. The allowlist has twelve phase names.
|
||||
|
||||
| Phase | Meaning |
|
||||
| --- | --- |
|
||||
| `place_workspace` | Place the run workspace. |
|
||||
| `start_transport` | Start the agent transport. |
|
||||
| `create_runtime` | Create the agent runtime. |
|
||||
| `ensure_session` | Ensure the agent session exists. |
|
||||
| `configure_session` | Configure the agent session. |
|
||||
| `prepare_turn` | Prepare the turn. |
|
||||
| `turn` | Run the turn. |
|
||||
| `end_session` | End the agent session. |
|
||||
| `settle_reuse` | Settle the session for reuse. |
|
||||
| `stop_transport` | Stop the agent transport. |
|
||||
| `sync_back` | Sync the workspace back. |
|
||||
| `release_staging_lease` | Release the staging lease. |
|
||||
|
||||
The payload never carries a command, an argument, a path, an environment value,
|
||||
or a raw identifier. The event rides the `ctx.onEvent` run-event bridge and is
|
||||
observability-only. It needs no OTLP endpoint.
|
||||
|
||||
## Dimension Values
|
||||
|
||||
Telemetry dimension values must be primitives. Use only the value types allowed
|
||||
|
|
|
|||
Loading…
Reference in New Issue