feat(runner): add hidden server PRP coordinator (#12176)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner needs a narrow server trust boundary before an adapter can start it. > - The package has durable runner transport, but the server does not host or authorize that transport. > - Native persistence exists, but no writer connects PRP events to those records. > - A direct adapter must not enter this path by accident. > - This pull request adds a hidden, run-bound PRP server coordinator. > - The benefit is a recoverable server boundary that remains unavailable to normal execution. ## Linked Issues or Issue Description Refs #11962 Refs #12129 Refs #12169 **Subsystem affected** Cross-cutting. The change affects the runner package and server orchestration. **Problem or motivation** The server cannot authenticate runnerd, commit PRP events before ACK, authorize semantic tools, or enter native finalization from a durable runner result. The application must have this hidden boundary before a guarded adapter can use the runner. **Proposed solution** Add an authenticated PRP WebSocket authority and register it only for one exact persisted native Codex run. Bind each connection and event to the company, issue, agent, run, runner, session, turn, item, and verified runner identity. Commit each event before its cumulative ACK. Project only authorized same-task read tools. Rebuild the accepted result and finalization record from durable result and terminal events. **Alternatives considered** The server could expose a broad runner API key or route semantic calls through existing adapter endpoints. Those options grant too much authority and weaken replay recovery. The server could also add the user-facing adapter in this pull request. That option would mix rollout selection with the transport trust boundary and make legacy compatibility harder to review. **Roadmap alignment** This work supports the shipped enforced-outcomes, governed-tool, and self-healing-run milestones. It does not add a new roadmap surface. ## What Changed - Add the durable PRP server authority with one-use bootstrap tickets, reconnect leases, encrypted frames, bounded state, cumulative ACKs, and idempotent commands. - Add `/api/runner/v1/connect/:runId`. Derive its `ws://` or `wss://` URL from the configured Paperclip API URL. - Register one authority only after the coordinator verifies the complete native Codex run binding. - Commit validated PRP events to `heartbeat_run_events` before ACK. Reject source gaps and conflicting replays. - Rebuild accepted results and finalization records from durable result and terminal events. Enforce finalization owner leases and retry times. - Project five same-task read operations. Recheck run, agent, task, and company authority for each call. - Keep the route hidden. No adapter selects this coordinator, and no code starts runnerd. - Vendor the compiled runner TypeScript runtime into the server package while keeping the workspace package development-only for the server. - Document the package, database writer, run-log payload, and credential exclusions. ## Verification - Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All TypeScript protocol checks and 69 Vitest tests pass, including commit-before-ACK crash recovery. All 43 Rust unit tests and 13 Rust integration tests pass. Conformance and replay parity pass. - Run the focused server WebSocket, coordinator, package-build, and startup-wiring suites. All 26 tests pass, including a clean-checkout reproduction with the runner `dist` directory absent. - Run `pnpm -r typecheck`. - Run `pnpm test:run`. - Run `pnpm build`. - Confirm that the diff contains 19 files. Confirm that it contains no workflow or `pnpm-lock.yaml` change. ## Risks - The server installs the WebSocket route at startup. An unregistered or malformed run path fails closed and creates no native record. - Bootstrap tickets are one use. The private state directory uses mode `0700`, and the state file uses mode `0600`. The file stores derived authentication verifiers and never stores raw tickets or lease tokens. - The journal has explicit frame, command, event-window, and file-size bounds. A bound violation closes the runner connection or rejects the command. - A runner event reaches the database before its ACK. A crash between event commit and ACK causes a byte-equivalent replay, not a second logical effect. - The coordinator accepts only an existing queued or running native Codex row with exact company, task, agent, runner, session, and completion-contract ownership. - Existing direct adapters do not call this service. They keep their current execution, transcript, result, and finalization paths. - The server has no production dependency on the private runner package. Its build copies the compiled runtime into `server/dist`; the workspace link is development-only. This adds no external package and does not change the lockfile. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
This commit is contained in:
Dotta
GitHub
parent
86fe9339e1
commit
9964b034bb
|
|
@ -216,8 +216,14 @@ finalization to one company, issue, and run. The database rejects mixed-owner
|
|||
evidence even when every referenced ID exists. Native source identities on
|
||||
`heartbeat_run_events` are nullable so legacy events remain readable without
|
||||
rewriting historical rows. Per-run native source identifiers are unique, while
|
||||
the existing legacy sequence behavior remains unchanged until an atomic event
|
||||
allocator is introduced with the native writer.
|
||||
the existing legacy sequence behavior remains unchanged. The hidden native
|
||||
coordinator serializes on its bound `heartbeat_runs` row, allocates
|
||||
`next_event_seq`, and commits a validated PRP event before the transport sends
|
||||
its cumulative ACK. Byte-equivalent source retries return the existing cursor;
|
||||
gaps and conflicting replays fail closed. Accepted structured results enter the
|
||||
finalization ledger, whose retry time and owner lease are checked under a row
|
||||
lock. None of these writes selects a runtime or changes a legacy run's execution
|
||||
path.
|
||||
|
||||
Issue `status_version` advances only when `status` changes. The JavaScript backup
|
||||
path includes user-defined functions and triggers so a restored database keeps
|
||||
|
|
|
|||
|
|
@ -5,6 +5,24 @@ Run-log events write to the `heartbeat_run_events` table
|
|||
Paperclip Telemetry events, and they are not OpenTelemetry exports. A run-log
|
||||
event needs no operator endpoint.
|
||||
|
||||
## Native PRP Run-Log Events
|
||||
|
||||
The hidden native coordinator writes each validated PRP event to the bound
|
||||
run's existing event stream before it acknowledges the runner. The row keeps
|
||||
the PRP `eventType`, source instance, source event ID, source sequence, protocol
|
||||
schema version, and a SHA-256 digest of the canonical source envelope. Its
|
||||
payload is `{ "prpEvent": <canonical PRP event> }`.
|
||||
|
||||
The writer locks the native `heartbeat_runs` row and allocates the existing
|
||||
per-run `seq` cursor. A byte-equivalent retry reuses the first row; a changed
|
||||
retry or source-sequence gap is rejected. Company, issue, agent, run, session,
|
||||
and runner-source bindings must match the persisted native run. Bootstrap
|
||||
tickets, reconnect leases, authentication proofs, encryption keys, and raw
|
||||
credential material are never written to the run log.
|
||||
|
||||
These records remain run-log events. They do not create an OpenTelemetry or
|
||||
Paperclip Telemetry export, and legacy adapters do not use this writer.
|
||||
|
||||
## Sandbox Startup Run-Log Event
|
||||
|
||||
Paperclip writes one `run.startup.step` event to the run log for each bring-up
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
This private workspace package contains the staged Paperclip Runner work.
|
||||
|
||||
The package currently exposes only the language-neutral PRP v1 TypeScript
|
||||
The package currently exposes the language-neutral PRP v1 TypeScript
|
||||
contract, provider-neutral structured questions and responses, deterministic
|
||||
fixture validation/replay, structured-result normalization, and the session
|
||||
reducer oracle. It also contains a package-local Rust runner, scripted fake
|
||||
|
|
@ -11,27 +11,35 @@ PRP transport. The transport authenticates and encrypts loopback WebSocket
|
|||
sessions, persists an ACK-driven outbox and command journal, and reconnects with
|
||||
a short-lived lease. The Rust runner now includes a Codex-only app-server
|
||||
provider bridge with durable thread resume, cancellation, structured questions,
|
||||
and provider-neutral event normalization. No server code starts or invokes it.
|
||||
and provider-neutral event normalization. The root surface now also exposes an
|
||||
authenticated durable PRP authority for server-side use. It stores only
|
||||
bootstrap and reconnect credential digests, validates immutable run identity on
|
||||
every connection and event, and persists commands and cumulative event ACK
|
||||
state across server restarts.
|
||||
The package also publishes the canonical semantic action declarations and
|
||||
their input and output schemas. Its package-local dispatcher projects only
|
||||
bound, run-authorized actions and emits redacted semantic receipts. It does not
|
||||
add application bindings, a server adapter, or production Paperclip behavior.
|
||||
|
||||
The first and only installed provider is Codex. Dynamic semantic tools remain
|
||||
undiscoverable because no production application binding or server authority
|
||||
has landed. Catalog membership alone does not grant authority. See
|
||||
undiscoverable unless the hidden server coordinator projects one of the five
|
||||
same-task read bindings for an already persisted native Codex run. Catalog
|
||||
membership alone does not grant authority, and no production adapter can create
|
||||
or start such a run yet. See
|
||||
[`SEMANTIC_ACTIONS.md`](SEMANTIC_ACTIONS.md) for the catalog boundary.
|
||||
|
||||
The package has two initial public surfaces:
|
||||
|
||||
- `@paperclipai/paperclip-runner` contains runtime contracts, validation,
|
||||
replay/reducer logic, the semantic catalog, and the authorization dispatcher.
|
||||
replay/reducer logic, the semantic catalog, the authorization dispatcher, and
|
||||
the Node-only durable server authority.
|
||||
- `@paperclipai/paperclip-runner/testing` adds Node-only fixture loading and a
|
||||
provider-neutral semantic conformance kit for deterministic test adapters.
|
||||
|
||||
No SDK, browser, React, eval, live-console, lab, or provider-experiment entry
|
||||
point is exported. The package remains private in this wave, and no production
|
||||
adapter starts it yet.
|
||||
point is exported. The package remains private in this wave. The server route
|
||||
at `/api/runner/v1/connect/:runId` has no authority until the hidden coordinator
|
||||
registers an exact existing run binding, and no production adapter starts it.
|
||||
|
||||
Run the complete contract gate with:
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,618 @@
|
|||
import {
|
||||
createCipheriv,
|
||||
createDecipheriv,
|
||||
createHash,
|
||||
createHmac,
|
||||
} from "node:crypto";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { connect, type Socket } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { validatePrpEvent } from "../protocol/replay-contract.js";
|
||||
import { digestPaperclipSemanticContent } from "../semantic-tools/receipts.js";
|
||||
import { DurablePrpControlPlane } from "./durable-prp-control-plane.js";
|
||||
import type { DurableRecoveryIdentity } from "./prp-transport-types.js";
|
||||
|
||||
const identity: DurableRecoveryIdentity = {
|
||||
runnerInstanceId: "runner-test-1",
|
||||
environmentLeaseId: "environment-test-1",
|
||||
runId: "00000000-0000-4000-8000-000000000001",
|
||||
normalizedSessionId: "session-test-1",
|
||||
turnId: "turn-test-1",
|
||||
itemId: "item-test-1",
|
||||
};
|
||||
const expectedRunnerVersion = "0.3.0";
|
||||
const expectedRunnerDigest = `sha256:${"a".repeat(64)}`;
|
||||
|
||||
function domainDigest(domain: string, parts: readonly Buffer[]): Buffer {
|
||||
const digest = createHash("sha256")
|
||||
.update(domain)
|
||||
.update(Buffer.from([0]));
|
||||
for (const part of parts) {
|
||||
const length = Buffer.alloc(8);
|
||||
length.writeBigUInt64BE(BigInt(part.length));
|
||||
digest.update(length).update(part);
|
||||
}
|
||||
return digest.digest();
|
||||
}
|
||||
|
||||
function domainHmac(
|
||||
key: Buffer,
|
||||
domain: string,
|
||||
parts: readonly Buffer[],
|
||||
): Buffer {
|
||||
const digest = createHmac("sha256", key)
|
||||
.update(domain)
|
||||
.update(Buffer.from([0]));
|
||||
for (const part of parts) {
|
||||
const length = Buffer.alloc(8);
|
||||
length.writeBigUInt64BE(BigInt(part.length));
|
||||
digest.update(length).update(part);
|
||||
}
|
||||
return digest.digest();
|
||||
}
|
||||
|
||||
function canonicalJson(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonicalJson).join(",")}]`;
|
||||
if (typeof value === "object" && value !== null) {
|
||||
const record = value as Record<string, unknown>;
|
||||
return `{${Object.keys(record)
|
||||
.sort()
|
||||
.map((key) => `${JSON.stringify(key)}:${canonicalJson(record[key])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function credentialMaterial(token: string): {
|
||||
credentialId: string;
|
||||
authKey: Buffer;
|
||||
} {
|
||||
const bytes = Buffer.from(token);
|
||||
const authKey = domainDigest("paperclip-runner-auth-key-v1", [bytes]);
|
||||
return {
|
||||
credentialId: `sha256:${domainDigest("paperclip-runner-credential-id-v1", [bytes]).toString("hex")}`,
|
||||
authKey,
|
||||
};
|
||||
}
|
||||
|
||||
class ServerFrameReader {
|
||||
#buffer = Buffer.alloc(0);
|
||||
#frames: Array<Record<string, unknown> | null> = [];
|
||||
#waiters: Array<(value: Record<string, unknown> | null) => void> = [];
|
||||
|
||||
constructor(socket: Socket) {
|
||||
socket.on("data", (chunk: Buffer) => {
|
||||
this.#buffer = Buffer.concat([this.#buffer, chunk]);
|
||||
this.#drain();
|
||||
});
|
||||
socket.once("close", () => this.#publish(null));
|
||||
}
|
||||
|
||||
next(): Promise<Record<string, unknown> | null> {
|
||||
const queued = this.#frames.shift();
|
||||
if (queued !== undefined) return Promise.resolve(queued);
|
||||
return new Promise((resolveFrame, rejectFrame) => {
|
||||
const timer = setTimeout(
|
||||
() => rejectFrame(new Error("Server WebSocket frame timed out.")),
|
||||
2_000,
|
||||
);
|
||||
this.#waiters.push((value) => {
|
||||
clearTimeout(timer);
|
||||
resolveFrame(value);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
#publish(value: Record<string, unknown> | null): void {
|
||||
const waiter = this.#waiters.shift();
|
||||
if (waiter) waiter(value);
|
||||
else this.#frames.push(value);
|
||||
}
|
||||
|
||||
#drain(): void {
|
||||
while (this.#buffer.length >= 2) {
|
||||
let length = this.#buffer[1]! & 0x7f;
|
||||
let cursor = 2;
|
||||
if (length === 126) {
|
||||
if (this.#buffer.length < 4) return;
|
||||
length = this.#buffer.readUInt16BE(2);
|
||||
cursor = 4;
|
||||
} else if (length === 127) {
|
||||
if (this.#buffer.length < 10) return;
|
||||
length = Number(this.#buffer.readBigUInt64BE(2));
|
||||
cursor = 10;
|
||||
}
|
||||
if (this.#buffer.length < cursor + length) return;
|
||||
const payload = this.#buffer.subarray(cursor, cursor + length);
|
||||
this.#buffer = this.#buffer.subarray(cursor + length);
|
||||
this.#publish(
|
||||
JSON.parse(payload.toString("utf8")) as Record<string, unknown>,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function upgradeSocket(url: string): Promise<{
|
||||
socket: Socket;
|
||||
reader: ServerFrameReader;
|
||||
}> {
|
||||
const parsed = new URL(url);
|
||||
const socket = await new Promise<Socket>((resolveSocket, rejectSocket) => {
|
||||
const candidate = connect(Number(parsed.port), parsed.hostname);
|
||||
let response = Buffer.alloc(0);
|
||||
const onData = (chunk: Buffer): void => {
|
||||
response = Buffer.concat([response, chunk]);
|
||||
const boundary = response.indexOf("\r\n\r\n");
|
||||
if (boundary < 0) return;
|
||||
candidate.off("data", onData);
|
||||
const status = Number(
|
||||
response.toString("utf8").match(/^HTTP\/1\.1 (\d{3})/)?.[1],
|
||||
);
|
||||
if (status !== 101) {
|
||||
candidate.destroy();
|
||||
rejectSocket(new Error(`WebSocket upgrade returned ${String(status)}`));
|
||||
} else {
|
||||
resolveSocket(candidate);
|
||||
}
|
||||
};
|
||||
candidate.once("error", rejectSocket);
|
||||
candidate.once("connect", () => {
|
||||
candidate.write(
|
||||
[
|
||||
`GET ${parsed.pathname} HTTP/1.1`,
|
||||
`Host: ${parsed.host}`,
|
||||
"Upgrade: websocket",
|
||||
"Connection: Upgrade",
|
||||
"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==",
|
||||
"Sec-WebSocket-Version: 13",
|
||||
"\r\n",
|
||||
].join("\r\n"),
|
||||
);
|
||||
});
|
||||
candidate.on("data", onData);
|
||||
});
|
||||
return { socket, reader: new ServerFrameReader(socket) };
|
||||
}
|
||||
|
||||
function sendMaskedJson(socket: Socket, value: unknown): void {
|
||||
const payload = Buffer.from(JSON.stringify(value));
|
||||
const mask = Buffer.from([0x11, 0x22, 0x33, 0x44]);
|
||||
const header: number[] = [0x81];
|
||||
if (payload.length <= 125) {
|
||||
header.push(0x80 | payload.length);
|
||||
} else if (payload.length <= 0xffff) {
|
||||
header.push(0x80 | 126, payload.length >>> 8, payload.length & 0xff);
|
||||
} else {
|
||||
throw new Error("Test client frame exceeds the supported size.");
|
||||
}
|
||||
const masked = Buffer.from(payload);
|
||||
for (let index = 0; index < masked.length; index += 1) {
|
||||
masked[index] = masked[index]! ^ mask[index % mask.length]!;
|
||||
}
|
||||
socket.write(Buffer.concat([Buffer.from(header), mask, masked]));
|
||||
}
|
||||
|
||||
interface AuthenticatedClient {
|
||||
socket: Socket;
|
||||
reader: ServerFrameReader;
|
||||
authKey: Buffer;
|
||||
sessionId: string;
|
||||
sendCounter: bigint;
|
||||
receiveCounter: bigint;
|
||||
leaseToken: string | null;
|
||||
welcome: Record<string, unknown>;
|
||||
}
|
||||
|
||||
function authHello(
|
||||
credentialId: string,
|
||||
selectedIdentity: DurableRecoveryIdentity = identity,
|
||||
): Record<string, unknown> {
|
||||
return {
|
||||
protocol: "paperclip.runner",
|
||||
version: 1,
|
||||
kind: "auth_hello",
|
||||
payload: {
|
||||
credentialId,
|
||||
clientNonce: "client-nonce-test",
|
||||
protocolMin: 1,
|
||||
protocolMax: 1,
|
||||
...selectedIdentity,
|
||||
runnerVersion: expectedRunnerVersion,
|
||||
runnerDigest: expectedRunnerDigest,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function authenticate(
|
||||
controlPlane: DurablePrpControlPlane,
|
||||
token: string,
|
||||
selectedIdentity: DurableRecoveryIdentity = identity,
|
||||
): Promise<AuthenticatedClient | null> {
|
||||
const { socket, reader } = await upgradeSocket(controlPlane.connectUrl);
|
||||
const material = credentialMaterial(token);
|
||||
sendMaskedJson(socket, authHello(material.credentialId, selectedIdentity));
|
||||
const challenge = await reader.next();
|
||||
if (challenge === null) return null;
|
||||
const challengePayload = challenge.payload as Record<string, unknown>;
|
||||
const serverProof = challengePayload.serverProof;
|
||||
if (typeof serverProof !== "string") throw new Error("Missing server proof.");
|
||||
const canonicalPayload = { ...challengePayload };
|
||||
delete canonicalPayload.serverProof;
|
||||
const canonicalChallenge = canonicalJson(canonicalPayload);
|
||||
const expectedServerProof = domainHmac(
|
||||
material.authKey,
|
||||
"paperclip-runner-server-proof-v1",
|
||||
[Buffer.from(canonicalChallenge)],
|
||||
).toString("hex");
|
||||
expect(serverProof).toBe(expectedServerProof);
|
||||
const clientProof = domainHmac(
|
||||
material.authKey,
|
||||
"paperclip-runner-client-proof-v1",
|
||||
[Buffer.from(canonicalChallenge), Buffer.from(serverProof)],
|
||||
).toString("hex");
|
||||
sendMaskedJson(socket, {
|
||||
protocol: "paperclip.runner",
|
||||
version: 1,
|
||||
kind: "auth_response",
|
||||
payload: {
|
||||
credentialId: material.credentialId,
|
||||
clientNonce: challengePayload.clientNonce,
|
||||
serverNonce: challengePayload.serverNonce,
|
||||
clientProof,
|
||||
},
|
||||
});
|
||||
const binding = domainDigest("paperclip-runner-session-binding-v1", [
|
||||
Buffer.from(canonicalChallenge),
|
||||
Buffer.from(serverProof),
|
||||
Buffer.from(clientProof),
|
||||
]);
|
||||
const client: AuthenticatedClient = {
|
||||
socket,
|
||||
reader,
|
||||
authKey: material.authKey,
|
||||
sessionId: `sha256:${binding.toString("hex")}`,
|
||||
sendCounter: 0n,
|
||||
receiveCounter: 0n,
|
||||
leaseToken: null,
|
||||
welcome: {},
|
||||
};
|
||||
const welcome = await receiveSecure(client);
|
||||
if (welcome === null) return null;
|
||||
expect(welcome.kind).toBe("welcome");
|
||||
client.welcome = welcome;
|
||||
const leaseToken = (welcome.payload as Record<string, unknown>)
|
||||
.connectionLeaseToken;
|
||||
client.leaseToken = typeof leaseToken === "string" ? leaseToken : null;
|
||||
return client;
|
||||
}
|
||||
|
||||
function secureNonce(prefix: "P3C1" | "P3S1", counter: bigint): Buffer {
|
||||
const nonce = Buffer.alloc(12);
|
||||
nonce.write(prefix, 0, "ascii");
|
||||
nonce.writeBigUInt64BE(counter, 4);
|
||||
return nonce;
|
||||
}
|
||||
|
||||
function secureAad(
|
||||
client: AuthenticatedClient,
|
||||
direction: "client_to_core" | "core_to_client",
|
||||
counter: bigint,
|
||||
): Buffer {
|
||||
return Buffer.from(
|
||||
`paperclip.runner.secure-frame.v1\0${client.sessionId}\0${direction}\0${counter}`,
|
||||
);
|
||||
}
|
||||
|
||||
function sendSecure(
|
||||
client: AuthenticatedClient,
|
||||
value: Record<string, unknown>,
|
||||
): void {
|
||||
const binding = Buffer.from(client.sessionId.slice("sha256:".length), "hex");
|
||||
const key = domainHmac(
|
||||
client.authKey,
|
||||
"paperclip-runner-client-to-core-key-v1",
|
||||
[binding],
|
||||
);
|
||||
const counter = client.sendCounter;
|
||||
const cipher = createCipheriv(
|
||||
"aes-256-gcm",
|
||||
key,
|
||||
secureNonce("P3C1", counter),
|
||||
);
|
||||
cipher.setAAD(secureAad(client, "client_to_core", counter));
|
||||
const encrypted = Buffer.concat([
|
||||
cipher.update(Buffer.from(JSON.stringify(value))),
|
||||
cipher.final(),
|
||||
cipher.getAuthTag(),
|
||||
]);
|
||||
sendMaskedJson(client.socket, {
|
||||
schema: "paperclip.runner.secure-frame.v1",
|
||||
counter: Number(counter),
|
||||
ciphertext: encrypted.toString("hex"),
|
||||
});
|
||||
client.sendCounter += 1n;
|
||||
}
|
||||
|
||||
async function receiveSecure(
|
||||
client: AuthenticatedClient,
|
||||
): Promise<Record<string, unknown> | null> {
|
||||
const frame = await client.reader.next();
|
||||
if (frame === null) return null;
|
||||
const counter = BigInt(frame.counter as number);
|
||||
expect(counter).toBe(client.receiveCounter);
|
||||
const binding = Buffer.from(client.sessionId.slice("sha256:".length), "hex");
|
||||
const key = domainHmac(
|
||||
client.authKey,
|
||||
"paperclip-runner-core-to-client-key-v1",
|
||||
[binding],
|
||||
);
|
||||
const sealed = Buffer.from(String(frame.ciphertext), "hex");
|
||||
const decipher = createDecipheriv(
|
||||
"aes-256-gcm",
|
||||
key,
|
||||
secureNonce("P3S1", counter),
|
||||
);
|
||||
decipher.setAAD(secureAad(client, "core_to_client", counter));
|
||||
decipher.setAuthTag(sealed.subarray(-16));
|
||||
const value = JSON.parse(
|
||||
Buffer.concat([
|
||||
decipher.update(sealed.subarray(0, -16)),
|
||||
decipher.final(),
|
||||
]).toString("utf8"),
|
||||
) as Record<string, unknown>;
|
||||
client.receiveCounter += 1n;
|
||||
return value;
|
||||
}
|
||||
|
||||
function semanticInputEvent(sourceSeq = 1): Record<string, unknown> {
|
||||
return {
|
||||
protocol: "paperclip.runner",
|
||||
version: 1,
|
||||
kind: "event",
|
||||
runnerInstanceId: identity.runnerInstanceId,
|
||||
environmentLeaseId: identity.environmentLeaseId,
|
||||
runId: identity.runId,
|
||||
normalizedSessionId: identity.normalizedSessionId,
|
||||
turnId: identity.turnId,
|
||||
itemId: identity.itemId,
|
||||
payload: {
|
||||
sourceSeq,
|
||||
sourceEventId: `semantic-event-${sourceSeq}`,
|
||||
sourceInstanceId: identity.runnerInstanceId,
|
||||
sourceKind: "runner",
|
||||
runId: identity.runId,
|
||||
normalizedSessionId: identity.normalizedSessionId,
|
||||
turnId: identity.turnId,
|
||||
itemId: identity.itemId,
|
||||
eventType: "semantic_tool.input",
|
||||
schema: "paperclip.prp.event.v1",
|
||||
schemaVersion: 1,
|
||||
priority: 0,
|
||||
emittedAt: "2026-08-25T18:00:00.000Z",
|
||||
payload: {
|
||||
semantic_tool: {
|
||||
schema: "paperclip.prp.semantic_tool.v1",
|
||||
schemaVersion: 1,
|
||||
phase: "input",
|
||||
callId: "call-1",
|
||||
operationId: "get_task_context",
|
||||
correlation: {
|
||||
runId: identity.runId,
|
||||
normalizedSessionId: identity.normalizedSessionId,
|
||||
turnId: identity.turnId,
|
||||
itemId: identity.itemId,
|
||||
},
|
||||
idempotencyKey: null,
|
||||
content: {
|
||||
digest: digestPaperclipSemanticContent({}),
|
||||
redactionDisposition: "digest_only",
|
||||
references: [],
|
||||
},
|
||||
input: {},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe.sequential("DurablePrpControlPlane", () => {
|
||||
it("exchanges a one-use bootstrap for a run-bound reconnect lease", async () => {
|
||||
const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-auth-"));
|
||||
const controlPlane = new DurablePrpControlPlane({
|
||||
stateDirectory: root,
|
||||
identity,
|
||||
expectedRunnerVersion,
|
||||
expectedRunnerDigest,
|
||||
});
|
||||
try {
|
||||
await controlPlane.start();
|
||||
const cancel = controlPlane.queueCommand(
|
||||
"run.cancel",
|
||||
{ reason: "test" },
|
||||
"command-cancel-1",
|
||||
);
|
||||
expect(
|
||||
controlPlane.queueCommand(
|
||||
"run.cancel",
|
||||
{ reason: "test" },
|
||||
"command-cancel-1",
|
||||
),
|
||||
).toEqual(cancel);
|
||||
expect(() =>
|
||||
controlPlane.queueCommand(
|
||||
"run.cancel",
|
||||
{ reason: "different" },
|
||||
"command-cancel-1",
|
||||
),
|
||||
).toThrow("command replay conflicts");
|
||||
expect(() => controlPlane.queueCommand("unknown.command")).toThrow(
|
||||
"command is invalid",
|
||||
);
|
||||
const ticket = controlPlane.issueBootstrapTicket();
|
||||
const first = await authenticate(controlPlane, ticket);
|
||||
expect(first?.leaseToken).toEqual(expect.any(String));
|
||||
first?.socket.destroy();
|
||||
|
||||
const reused = await authenticate(controlPlane, ticket);
|
||||
expect(reused).toBeNull();
|
||||
|
||||
const lease = await authenticate(controlPlane, first!.leaseToken!);
|
||||
expect(lease).not.toBeNull();
|
||||
expect(lease?.leaseToken).toBeNull();
|
||||
lease?.socket.destroy();
|
||||
|
||||
const wrongRun = await authenticate(
|
||||
controlPlane,
|
||||
controlPlane.issueBootstrapTicket(),
|
||||
{
|
||||
...identity,
|
||||
runId: "00000000-0000-4000-8000-000000000999",
|
||||
},
|
||||
);
|
||||
expect(wrongRun).toBeNull();
|
||||
} finally {
|
||||
await controlPlane.stop();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("recovers one semantic call from the durable event after a coordinator restart", async () => {
|
||||
const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-recovery-"));
|
||||
let firstCalls = 0;
|
||||
const first = new DurablePrpControlPlane({
|
||||
stateDirectory: root,
|
||||
identity,
|
||||
expectedRunnerVersion,
|
||||
expectedRunnerDigest,
|
||||
onSemanticToolInput: async () => {
|
||||
firstCalls += 1;
|
||||
return new Promise(() => undefined);
|
||||
},
|
||||
});
|
||||
let leaseToken: string;
|
||||
try {
|
||||
await first.start();
|
||||
const client = await authenticate(first, first.issueBootstrapTicket());
|
||||
leaseToken = client!.leaseToken!;
|
||||
const validation = validatePrpEvent(semanticInputEvent().payload);
|
||||
expect(validation, JSON.stringify(validation)).toMatchObject({
|
||||
ok: true,
|
||||
});
|
||||
sendSecure(client!, semanticInputEvent());
|
||||
const ack = await receiveSecure(client!);
|
||||
expect(ack).toMatchObject({ kind: "ack" });
|
||||
expect(firstCalls).toBe(1);
|
||||
client?.socket.destroy();
|
||||
} finally {
|
||||
await first.stop();
|
||||
}
|
||||
|
||||
let recoveredCalls = 0;
|
||||
const recovered = new DurablePrpControlPlane({
|
||||
stateDirectory: root,
|
||||
identity,
|
||||
expectedRunnerVersion,
|
||||
expectedRunnerDigest,
|
||||
onSemanticToolInput: async (call) => {
|
||||
recoveredCalls += 1;
|
||||
return {
|
||||
result: { ok: true, operationId: call.operationId },
|
||||
};
|
||||
},
|
||||
});
|
||||
try {
|
||||
await recovered.start();
|
||||
const client = await authenticate(recovered, leaseToken!);
|
||||
sendSecure(client!, semanticInputEvent());
|
||||
const outcomes = [
|
||||
await receiveSecure(client!),
|
||||
await receiveSecure(client!),
|
||||
];
|
||||
const command = outcomes.find((outcome) => outcome?.kind === "command");
|
||||
expect(outcomes.some((outcome) => outcome?.kind === "ack")).toBe(true);
|
||||
expect(command?.payload).toMatchObject({
|
||||
type: "semantic_tool.result",
|
||||
payload: {
|
||||
callId: "call-1",
|
||||
operationId: "get_task_context",
|
||||
result: { ok: true, operationId: "get_task_context" },
|
||||
isError: false,
|
||||
},
|
||||
});
|
||||
expect(recoveredCalls).toBe(1);
|
||||
|
||||
const tampered = semanticInputEvent(2);
|
||||
const tamperedEvent = tampered.payload as Record<string, unknown>;
|
||||
const tamperedPayload = tamperedEvent.payload as Record<string, unknown>;
|
||||
const tamperedSemantic = tamperedPayload.semantic_tool as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
tamperedSemantic.content = {
|
||||
...(tamperedSemantic.content as Record<string, unknown>),
|
||||
digest: `sha256:${"0".repeat(64)}`,
|
||||
};
|
||||
sendSecure(client!, tampered);
|
||||
await expect(receiveSecure(client!)).resolves.toBeNull();
|
||||
client?.socket.destroy();
|
||||
} finally {
|
||||
await recovered.stop();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("does not acknowledge an event before the caller commits it", async () => {
|
||||
const root = mkdtempSync(resolve(tmpdir(), "paperclip-prp-commit-order-"));
|
||||
const first = new DurablePrpControlPlane({
|
||||
stateDirectory: root,
|
||||
identity,
|
||||
expectedRunnerVersion,
|
||||
expectedRunnerDigest,
|
||||
onSemanticToolInput: async () => ({ result: { ok: true } }),
|
||||
onCommittedEvent: async () => {
|
||||
throw new Error("database commit failed");
|
||||
},
|
||||
});
|
||||
let leaseToken: string;
|
||||
try {
|
||||
await first.start();
|
||||
const client = await authenticate(first, first.issueBootstrapTicket());
|
||||
leaseToken = client!.leaseToken!;
|
||||
sendSecure(client!, semanticInputEvent());
|
||||
await expect(receiveSecure(client!)).resolves.toBeNull();
|
||||
} finally {
|
||||
await first.stop();
|
||||
}
|
||||
|
||||
let committed = 0;
|
||||
const recovered = new DurablePrpControlPlane({
|
||||
stateDirectory: root,
|
||||
identity,
|
||||
expectedRunnerVersion,
|
||||
expectedRunnerDigest,
|
||||
onSemanticToolInput: async () => ({ result: { ok: true } }),
|
||||
onCommittedEvent: async () => {
|
||||
committed += 1;
|
||||
},
|
||||
});
|
||||
try {
|
||||
await recovered.start();
|
||||
const client = await authenticate(recovered, leaseToken!);
|
||||
expect(client?.welcome.payload).toMatchObject({ ackedSourceSeq: 0 });
|
||||
sendSecure(client!, semanticInputEvent());
|
||||
await expect(receiveSecure(client!)).resolves.toMatchObject({
|
||||
kind: "ack",
|
||||
payload: { ackedSourceSeq: 1 },
|
||||
});
|
||||
expect(committed).toBe(1);
|
||||
client?.socket.destroy();
|
||||
} finally {
|
||||
await recovered.stop();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,29 @@
|
|||
export interface DurableRecoveryIdentity {
|
||||
runnerInstanceId: string;
|
||||
environmentLeaseId: string;
|
||||
runId: string;
|
||||
normalizedSessionId: string;
|
||||
turnId: string;
|
||||
itemId: string;
|
||||
}
|
||||
|
||||
export interface DurableRecoveryCoreCommand {
|
||||
schema: "paperclip.prp.command.v1";
|
||||
commandId: string;
|
||||
controllerSeq: number;
|
||||
type: string;
|
||||
issuedAt: string;
|
||||
payload: Record<string, unknown>;
|
||||
status: "pending" | "completed" | "failed" | "rejected";
|
||||
result: Record<string, unknown> | null;
|
||||
}
|
||||
|
||||
export interface DurableRecoveryCommittedEvent {
|
||||
sourceSeq: number;
|
||||
sourceEventId: string;
|
||||
eventType: string;
|
||||
priority: 0 | 1 | 2;
|
||||
envelope: Record<string, unknown>;
|
||||
deliveryCount: number;
|
||||
logicalEffectCount: number;
|
||||
}
|
||||
|
|
@ -1,6 +1,11 @@
|
|||
export * from "./catalog/index.js";
|
||||
export * from "./contracts/completion-result.js";
|
||||
export * from "./contracts/question-set.js";
|
||||
export {
|
||||
DurablePrpControlPlane,
|
||||
type DurablePrpControlPlaneOptions,
|
||||
} from "./control-plane/durable-prp-control-plane.js";
|
||||
export type { DurableRecoveryIdentity } from "./control-plane/prp-transport-types.js";
|
||||
export * from "./protocol/replay-contract.js";
|
||||
export * from "./protocol/result-normalization.js";
|
||||
export * from "./reducer/session-reducer.js";
|
||||
|
|
|
|||
|
|
@ -35,12 +35,13 @@
|
|||
"dev": "tsx src/index.ts",
|
||||
"dev:watch": "cross-env PAPERCLIP_MIGRATION_PROMPT=never PAPERCLIP_MIGRATION_AUTO_APPLY=true tsx ./scripts/dev-watch.ts",
|
||||
"prepare:ui-dist": "bash ../scripts/prepare-server-ui-dist.sh",
|
||||
"build": "tsc && mkdir -p dist/onboarding-assets dist/built-ins dist/services/scripts && cp -R src/onboarding-assets/. dist/onboarding-assets/ && cp -R src/built-ins/. dist/built-ins/ && cp -R src/services/scripts/. dist/services/scripts/ && node scripts/write-build-stamp.mjs",
|
||||
"build": "pnpm run prepare:runner-vendor && tsc && mkdir -p dist/onboarding-assets dist/built-ins dist/services/scripts dist/vendor/paperclip-runner && cp -R src/onboarding-assets/. dist/onboarding-assets/ && cp -R src/built-ins/. dist/built-ins/ && cp -R src/services/scripts/. dist/services/scripts/ && cp -R ../packages/paperclip-runner/dist/. dist/vendor/paperclip-runner/ && node scripts/write-build-stamp.mjs",
|
||||
"prepack": "pnpm run prepare:ui-dist && pnpm run build",
|
||||
"postpack": "rm -rf ui-dist",
|
||||
"clean": "rm -rf dist",
|
||||
"start": "node dist/index.js",
|
||||
"typecheck": "pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && tsc --noEmit"
|
||||
"prepare:runner-vendor": "pnpm --filter @paperclipai/paperclip-runner build",
|
||||
"typecheck": "pnpm run prepare:runner-vendor && pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.1115.0",
|
||||
|
|
@ -83,6 +84,7 @@
|
|||
"zod": "^4.4.3"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@paperclipai/paperclip-runner": "workspace:*",
|
||||
"@types/express": "^5.0.0",
|
||||
"@types/express-serve-static-core": "^5.1.3",
|
||||
"@types/jsdom": "^30.0.0",
|
||||
|
|
|
|||
|
|
@ -2,7 +2,9 @@ import { readFileSync } from "node:fs";
|
|||
import { fileURLToPath } from "node:url";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
const packageJsonPath = fileURLToPath(new URL("../../package.json", import.meta.url));
|
||||
const packageJsonPath = fileURLToPath(
|
||||
new URL("../../package.json", import.meta.url),
|
||||
);
|
||||
|
||||
describe("server package build script", () => {
|
||||
it("builds the compiled package entry during prepack", () => {
|
||||
|
|
@ -10,7 +12,9 @@ describe("server package build script", () => {
|
|||
scripts?: Record<string, string>;
|
||||
};
|
||||
|
||||
expect(packageJson.scripts?.prepack).toBe("pnpm run prepare:ui-dist && pnpm run build");
|
||||
expect(packageJson.scripts?.prepack).toBe(
|
||||
"pnpm run prepare:ui-dist && pnpm run build",
|
||||
);
|
||||
});
|
||||
|
||||
it("copies static runtime asset directories into dist", () => {
|
||||
|
|
@ -19,8 +23,33 @@ describe("server package build script", () => {
|
|||
};
|
||||
const buildScript = packageJson.scripts?.build ?? "";
|
||||
|
||||
expect(buildScript).toContain("mkdir -p dist/onboarding-assets dist/built-ins");
|
||||
expect(buildScript).toContain("cp -R src/onboarding-assets/. dist/onboarding-assets/");
|
||||
expect(buildScript).toContain(
|
||||
"mkdir -p dist/onboarding-assets dist/built-ins",
|
||||
);
|
||||
expect(buildScript).toContain(
|
||||
"cp -R src/onboarding-assets/. dist/onboarding-assets/",
|
||||
);
|
||||
expect(buildScript).toContain("cp -R src/built-ins/. dist/built-ins/");
|
||||
});
|
||||
|
||||
it("vendors the private runner runtime without a production workspace dependency", () => {
|
||||
const packageJson = JSON.parse(readFileSync(packageJsonPath, "utf8")) as {
|
||||
scripts?: Record<string, string>;
|
||||
dependencies?: Record<string, string>;
|
||||
devDependencies?: Record<string, string>;
|
||||
};
|
||||
|
||||
expect(
|
||||
packageJson.dependencies?.["@paperclipai/paperclip-runner"],
|
||||
).toBeUndefined();
|
||||
expect(packageJson.devDependencies?.["@paperclipai/paperclip-runner"]).toBe(
|
||||
"workspace:*",
|
||||
);
|
||||
expect(packageJson.scripts?.["prepare:runner-vendor"]).toBe(
|
||||
"pnpm --filter @paperclipai/paperclip-runner build",
|
||||
);
|
||||
expect(packageJson.scripts?.build).toContain(
|
||||
"cp -R ../packages/paperclip-runner/dist/. dist/vendor/paperclip-runner/",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -114,6 +114,7 @@ const {
|
|||
};
|
||||
const feedbackServiceFactoryMock = vi.fn(() => feedbackExportServiceMock);
|
||||
const fakeServer = {
|
||||
on: vi.fn().mockReturnThis(),
|
||||
once: vi.fn().mockReturnThis(),
|
||||
off: vi.fn().mockReturnThis(),
|
||||
listen: vi.fn((_port: number, _host: string, callback?: () => void) => {
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ import {
|
|||
} from "./services/managed-config.js";
|
||||
import { setupEnvironmentCustomImageTerminalWebSocketServer } from "./realtime/environment-custom-image-terminal-ws.js";
|
||||
import { setupLiveEventsWebSocketServer } from "./realtime/live-events-ws.js";
|
||||
import { setupRunnerPrpWebSocketServer } from "./realtime/runner-prp-ws.js";
|
||||
import { cloudActorHeaderSourceFromHeaders, resolveCloudTenantActor } from "./middleware/auth.js";
|
||||
import {
|
||||
feedbackService,
|
||||
|
|
@ -890,6 +891,7 @@ export async function startServer(): Promise<StartedServer> {
|
|||
process.env.PAPERCLIP_RUNTIME_API_CANDIDATES_JSON = JSON.stringify(runtimeApiCandidates);
|
||||
process.env.PAPERCLIP_API_URL = configuredApiUrl;
|
||||
|
||||
setupRunnerPrpWebSocketServer(server, { apiUrl: configuredApiUrl });
|
||||
setupEnvironmentCustomImageTerminalWebSocketServer(server, db as any, {
|
||||
pluginWorkerManager,
|
||||
});
|
||||
|
|
|
|||
|
|
@ -0,0 +1,93 @@
|
|||
import { createServer } from "node:http";
|
||||
import { PassThrough } from "node:stream";
|
||||
|
||||
import type { DurablePrpControlPlane } from "@paperclipai/paperclip-runner";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import {
|
||||
registerRunnerPrpAuthority,
|
||||
runnerPrpWebSocketInternals,
|
||||
setupRunnerPrpWebSocketServer,
|
||||
} from "./runner-prp-ws.js";
|
||||
|
||||
describe("runner PRP websocket route", () => {
|
||||
afterEach(() => runnerPrpWebSocketInternals.resetForTests());
|
||||
|
||||
it("routes only the registered run and releases it by generation", async () => {
|
||||
const server = createServer();
|
||||
setupRunnerPrpWebSocketServer(server, { apiUrl: "http://127.0.0.1:3210" });
|
||||
const handleUpgrade = vi.fn();
|
||||
const runId = "00000000-0000-4000-8000-000000000777";
|
||||
const registration = await registerRunnerPrpAuthority({
|
||||
companyId: "company-1",
|
||||
runId,
|
||||
authority: { handleUpgrade } as unknown as DurablePrpControlPlane,
|
||||
});
|
||||
|
||||
expect(registration.connectUrl).toBe(
|
||||
`ws://127.0.0.1:3210/api/runner/v1/connect/${runId}`,
|
||||
);
|
||||
expect(
|
||||
runnerPrpWebSocketInternals.activeRegistration({
|
||||
companyId: "company-1",
|
||||
runId,
|
||||
}),
|
||||
).toBe(true);
|
||||
|
||||
const socket = new PassThrough();
|
||||
const request = { url: `/api/runner/v1/connect/${runId}`, headers: {} };
|
||||
server.emit("upgrade", request, socket, Buffer.alloc(0));
|
||||
expect(request).toMatchObject({ paperclipWebSocketHandled: true });
|
||||
expect(handleUpgrade).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ url: `/api/runner/v1/connect/${runId}` }),
|
||||
socket,
|
||||
`/api/runner/v1/connect/${runId}`,
|
||||
expect.any(Buffer),
|
||||
);
|
||||
|
||||
await registration.release();
|
||||
expect(
|
||||
runnerPrpWebSocketInternals.activeRegistration({
|
||||
companyId: "company-1",
|
||||
runId,
|
||||
}),
|
||||
).toBe(false);
|
||||
server.close();
|
||||
});
|
||||
|
||||
it.each([
|
||||
["/api/runner/v1/connect/not-a-run", "400 Bad Request"],
|
||||
[
|
||||
"/api/runner/v1/connect/00000000-0000-4000-8000-000000000778",
|
||||
"404 Not Found",
|
||||
],
|
||||
])("fails closed for %s", (path, expectedStatus) => {
|
||||
const server = createServer();
|
||||
setupRunnerPrpWebSocketServer(server, { apiUrl: "http://127.0.0.1:3211" });
|
||||
const socket = new PassThrough();
|
||||
const writes: Buffer[] = [];
|
||||
socket.on("data", (chunk) => writes.push(Buffer.from(chunk)));
|
||||
server.emit("upgrade", { url: path, headers: {} }, socket, Buffer.alloc(0));
|
||||
expect(Buffer.concat(writes).toString("utf8")).toContain(expectedStatus);
|
||||
server.close();
|
||||
});
|
||||
|
||||
it("does not replace an active registration", async () => {
|
||||
const server = createServer();
|
||||
setupRunnerPrpWebSocketServer(server, { apiUrl: "http://127.0.0.1:3212" });
|
||||
const runId = "00000000-0000-4000-8000-000000000779";
|
||||
const authority = {
|
||||
handleUpgrade: vi.fn(),
|
||||
} as unknown as DurablePrpControlPlane;
|
||||
const first = await registerRunnerPrpAuthority({
|
||||
companyId: "company-1",
|
||||
runId,
|
||||
authority,
|
||||
});
|
||||
await expect(
|
||||
registerRunnerPrpAuthority({ companyId: "company-2", runId, authority }),
|
||||
).rejects.toThrow("runner_prp_authority_already_registered");
|
||||
await first.release();
|
||||
server.close();
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,131 @@
|
|||
import type { IncomingMessage, Server } from "node:http";
|
||||
import type { Duplex } from "node:stream";
|
||||
|
||||
import type { DurablePrpControlPlane } from "../vendor/paperclip-runner/index.js";
|
||||
|
||||
import { logger } from "../middleware/logger.js";
|
||||
|
||||
const CONNECT_PATH_PREFIX = "/api/runner/v1/connect/";
|
||||
const UUID_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
|
||||
|
||||
interface RegisteredAuthority {
|
||||
readonly companyId: string;
|
||||
readonly authority: DurablePrpControlPlane;
|
||||
readonly generation: symbol;
|
||||
}
|
||||
|
||||
interface RunnerPrpUpgradeRequest extends IncomingMessage {
|
||||
paperclipWebSocketHandled?: boolean;
|
||||
}
|
||||
|
||||
const registrations = new Map<string, RegisteredAuthority>();
|
||||
let loopbackOrigin: string | null = null;
|
||||
|
||||
function rejectUpgrade(
|
||||
socket: Duplex,
|
||||
status: "400 Bad Request" | "404 Not Found",
|
||||
): void {
|
||||
if (socket.destroyed) return;
|
||||
try {
|
||||
socket.end(
|
||||
`HTTP/1.1 ${status}\r\nConnection: close\r\nContent-Length: 0\r\n\r\n`,
|
||||
);
|
||||
} catch (error) {
|
||||
logger.warn(
|
||||
{ errorName: error instanceof Error ? error.name : typeof error },
|
||||
"failed to reject runner PRP websocket upgrade",
|
||||
);
|
||||
socket.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
export function setupRunnerPrpWebSocketServer(
|
||||
server: Server,
|
||||
options: { readonly apiUrl: string },
|
||||
): void {
|
||||
const apiUrl = new URL(options.apiUrl);
|
||||
if (!["http:", "https:"].includes(apiUrl.protocol)) {
|
||||
throw new Error("runner_prp_websocket_api_url_invalid");
|
||||
}
|
||||
apiUrl.protocol = apiUrl.protocol === "https:" ? "wss:" : "ws:";
|
||||
apiUrl.username = "";
|
||||
apiUrl.password = "";
|
||||
apiUrl.pathname = "";
|
||||
apiUrl.search = "";
|
||||
apiUrl.hash = "";
|
||||
loopbackOrigin = apiUrl.toString().replace(/\/$/, "");
|
||||
server.on(
|
||||
"upgrade",
|
||||
(request: IncomingMessage, socket: Duplex, head: Buffer) => {
|
||||
const url = new URL(request.url ?? "/", "http://paperclip.invalid");
|
||||
if (!url.pathname.startsWith(CONNECT_PATH_PREFIX)) return;
|
||||
|
||||
const ownedRequest = request as RunnerPrpUpgradeRequest;
|
||||
if (ownedRequest.paperclipWebSocketHandled) return;
|
||||
ownedRequest.paperclipWebSocketHandled = true;
|
||||
socket.on("error", (error) => {
|
||||
logger.warn(
|
||||
{ errorName: error.name },
|
||||
"runner PRP websocket upgrade socket failed",
|
||||
);
|
||||
});
|
||||
|
||||
const runId = url.pathname.slice(CONNECT_PATH_PREFIX.length);
|
||||
if (!UUID_PATTERN.test(runId)) {
|
||||
rejectUpgrade(socket, "400 Bad Request");
|
||||
return;
|
||||
}
|
||||
const registration = registrations.get(runId);
|
||||
if (!registration) {
|
||||
rejectUpgrade(socket, "404 Not Found");
|
||||
return;
|
||||
}
|
||||
registration.authority.handleUpgrade(request, socket, url.pathname, head);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
export async function registerRunnerPrpAuthority(input: {
|
||||
readonly companyId: string;
|
||||
readonly runId: string;
|
||||
readonly authority: DurablePrpControlPlane;
|
||||
}): Promise<{ readonly connectUrl: string; release(): Promise<void> }> {
|
||||
if (loopbackOrigin === null) {
|
||||
throw new Error("runner_prp_websocket_server_not_configured");
|
||||
}
|
||||
if (!UUID_PATTERN.test(input.runId) || input.companyId.length === 0) {
|
||||
throw new Error("runner_prp_authority_binding_invalid");
|
||||
}
|
||||
if (registrations.has(input.runId)) {
|
||||
throw new Error("runner_prp_authority_already_registered");
|
||||
}
|
||||
const generation = Symbol(input.runId);
|
||||
registrations.set(input.runId, {
|
||||
companyId: input.companyId,
|
||||
authority: input.authority,
|
||||
generation,
|
||||
});
|
||||
return {
|
||||
connectUrl: `${loopbackOrigin}${CONNECT_PATH_PREFIX}${input.runId}`,
|
||||
release: async () => {
|
||||
if (registrations.get(input.runId)?.generation === generation) {
|
||||
registrations.delete(input.runId);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export const runnerPrpWebSocketInternals = {
|
||||
connectPathPrefix: CONNECT_PATH_PREFIX,
|
||||
activeRegistration(input: {
|
||||
readonly companyId: string;
|
||||
readonly runId: string;
|
||||
}): boolean {
|
||||
return registrations.get(input.runId)?.companyId === input.companyId;
|
||||
},
|
||||
resetForTests(): void {
|
||||
registrations.clear();
|
||||
loopbackOrigin = null;
|
||||
},
|
||||
};
|
||||
|
|
@ -0,0 +1,460 @@
|
|||
import { createHash } from "node:crypto";
|
||||
|
||||
import { and, desc, eq, lt } from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
heartbeatRunEvents,
|
||||
heartbeatRuns,
|
||||
nativeRunFinalizations,
|
||||
nativeRunResults,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
type PrpEvent,
|
||||
type PrpStructuredRunResult,
|
||||
type PrpTerminalState,
|
||||
validatePrpEvent,
|
||||
validatePrpStructuredRunResult,
|
||||
} from "../../vendor/paperclip-runner/index.js";
|
||||
|
||||
export interface NativeRunStoreBinding {
|
||||
readonly companyId: string;
|
||||
readonly issueId: string;
|
||||
readonly runId: string;
|
||||
readonly agentId: string;
|
||||
readonly normalizedSessionId: string;
|
||||
readonly runnerSourceInstanceId: string;
|
||||
readonly completionContractId: string;
|
||||
readonly completionContractSha256: string;
|
||||
}
|
||||
|
||||
export interface CompleteNativeRunInput {
|
||||
readonly result: PrpStructuredRunResult;
|
||||
readonly terminal: PrpTerminalState;
|
||||
readonly turnId?: string;
|
||||
readonly callerResultId?: string;
|
||||
readonly callerDedupeKey?: string;
|
||||
}
|
||||
|
||||
function canonicalJson(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonicalJson).join(",")}]`;
|
||||
if (typeof value === "object" && value !== null) {
|
||||
const record = value as Record<string, unknown>;
|
||||
return `{${Object.keys(record)
|
||||
.sort()
|
||||
.map((key) => `${JSON.stringify(key)}:${canonicalJson(record[key])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
return JSON.stringify(value) ?? "null";
|
||||
}
|
||||
|
||||
function sha256(value: unknown): string {
|
||||
return `sha256:${createHash("sha256").update(canonicalJson(value)).digest("hex")}`;
|
||||
}
|
||||
|
||||
function assertTerminal(value: PrpTerminalState): void {
|
||||
if (
|
||||
value.schema !== "paperclip.prp.terminal.v1" ||
|
||||
!["completed", "failed", "interrupted", "cancelled"].includes(
|
||||
value.turnTerminalState,
|
||||
) ||
|
||||
!["succeeded", "failed", "cancelled"].includes(value.runTerminalState) ||
|
||||
!["done", "blocked", "needs_review", "yielded"].includes(
|
||||
value.reportedWorkDisposition,
|
||||
)
|
||||
) {
|
||||
throw new Error("native_terminal_schema_invalid");
|
||||
}
|
||||
}
|
||||
|
||||
/** Durable DB boundary used by the hidden PRP coordinator. */
|
||||
export class NativeRunCoordinatorStore {
|
||||
readonly #db: Db;
|
||||
readonly #binding: NativeRunStoreBinding;
|
||||
|
||||
constructor(db: Db, binding: NativeRunStoreBinding) {
|
||||
this.#db = db;
|
||||
this.#binding = structuredClone(binding);
|
||||
}
|
||||
|
||||
async appendEvent(value: PrpEvent): Promise<{
|
||||
readonly disposition: "committed" | "duplicate";
|
||||
readonly cursor: number;
|
||||
readonly highestContiguousSourceSeq: number;
|
||||
}> {
|
||||
const validated = validatePrpEvent(value);
|
||||
if (!validated.ok) throw new Error("native_event_schema_invalid");
|
||||
const event = validated.event;
|
||||
if (
|
||||
event.runId !== this.#binding.runId ||
|
||||
event.normalizedSessionId !== this.#binding.normalizedSessionId ||
|
||||
event.sourceKind !== "runner" ||
|
||||
event.sourceInstanceId !== this.#binding.runnerSourceInstanceId
|
||||
) {
|
||||
throw new Error("native_event_binding_mismatch");
|
||||
}
|
||||
const canonicalPayload = event as unknown as Record<string, unknown>;
|
||||
const payloadDigest = sha256(canonicalPayload);
|
||||
|
||||
return this.#db.transaction(async (tx) => {
|
||||
const [run] = await tx
|
||||
.select()
|
||||
.from(heartbeatRuns)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRuns.id, this.#binding.runId),
|
||||
eq(heartbeatRuns.companyId, this.#binding.companyId),
|
||||
eq(heartbeatRuns.agentId, this.#binding.agentId),
|
||||
eq(heartbeatRuns.nativeIssueId, this.#binding.issueId),
|
||||
eq(
|
||||
heartbeatRuns.nativeSessionId,
|
||||
this.#binding.normalizedSessionId,
|
||||
),
|
||||
eq(heartbeatRuns.runtimeMode, "native"),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.limit(1);
|
||||
if (!run) throw new Error("native_event_run_not_authorized");
|
||||
|
||||
const [existing] = await tx
|
||||
.select()
|
||||
.from(heartbeatRunEvents)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRunEvents.runId, this.#binding.runId),
|
||||
eq(heartbeatRunEvents.sourceEventId, event.sourceEventId),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (existing) {
|
||||
if (
|
||||
existing.sourcePayloadSha256 !== payloadDigest ||
|
||||
existing.sourceInstanceId !== event.sourceInstanceId ||
|
||||
existing.sourceSeq !== event.sourceSeq
|
||||
) {
|
||||
throw new Error("native_event_replay_conflict");
|
||||
}
|
||||
const [latest] = await tx
|
||||
.select({ sourceSeq: heartbeatRunEvents.sourceSeq })
|
||||
.from(heartbeatRunEvents)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRunEvents.runId, this.#binding.runId),
|
||||
eq(heartbeatRunEvents.sourceInstanceId, event.sourceInstanceId),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(heartbeatRunEvents.sourceSeq))
|
||||
.limit(1);
|
||||
return {
|
||||
disposition: "duplicate" as const,
|
||||
cursor: existing.seq,
|
||||
highestContiguousSourceSeq: latest?.sourceSeq ?? event.sourceSeq,
|
||||
};
|
||||
}
|
||||
|
||||
const [previous] = await tx
|
||||
.select({ sourceSeq: heartbeatRunEvents.sourceSeq })
|
||||
.from(heartbeatRunEvents)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRunEvents.runId, this.#binding.runId),
|
||||
eq(heartbeatRunEvents.sourceInstanceId, event.sourceInstanceId),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(heartbeatRunEvents.sourceSeq))
|
||||
.limit(1);
|
||||
const expectedSourceSeq = (previous?.sourceSeq ?? 0) + 1;
|
||||
if (event.sourceSeq !== expectedSourceSeq) {
|
||||
throw new Error("native_event_source_gap");
|
||||
}
|
||||
|
||||
const cursor = run.nextEventSeq;
|
||||
const [inserted] = await tx
|
||||
.insert(heartbeatRunEvents)
|
||||
.values({
|
||||
companyId: this.#binding.companyId,
|
||||
runId: this.#binding.runId,
|
||||
agentId: this.#binding.agentId,
|
||||
seq: cursor,
|
||||
eventType: event.eventType,
|
||||
stream: "system",
|
||||
level: event.eventType.includes("failed") ? "error" : "info",
|
||||
payload: { prpEvent: canonicalPayload },
|
||||
sourceInstanceId: event.sourceInstanceId,
|
||||
sourceEventId: event.sourceEventId,
|
||||
sourceSeq: event.sourceSeq,
|
||||
sourcePayloadSha256: payloadDigest,
|
||||
protocolSchemaVersion: event.schemaVersion,
|
||||
})
|
||||
.returning({ seq: heartbeatRunEvents.seq });
|
||||
if (!inserted) throw new Error("native_event_not_persisted");
|
||||
await tx
|
||||
.update(heartbeatRuns)
|
||||
.set({ nextEventSeq: cursor + 1, updatedAt: new Date() })
|
||||
.where(eq(heartbeatRuns.id, this.#binding.runId));
|
||||
return {
|
||||
disposition: "committed" as const,
|
||||
cursor: inserted.seq,
|
||||
highestContiguousSourceSeq: event.sourceSeq,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async completeRun(input: CompleteNativeRunInput): Promise<{
|
||||
readonly disposition: "committed" | "duplicate";
|
||||
readonly resultId: string;
|
||||
}> {
|
||||
const validated = validatePrpStructuredRunResult(input.result);
|
||||
if (!validated.ok) throw new Error("native_result_schema_invalid");
|
||||
assertTerminal(input.terminal);
|
||||
if (
|
||||
validated.result.reportedWorkDisposition !==
|
||||
input.terminal.reportedWorkDisposition
|
||||
) {
|
||||
throw new Error("native_result_terminal_disposition_mismatch");
|
||||
}
|
||||
const canonical = {
|
||||
result: validated.result,
|
||||
terminal: input.terminal,
|
||||
turnId: input.turnId ?? null,
|
||||
};
|
||||
const canonicalSha256 = sha256(canonical);
|
||||
const serverFingerprint = sha256({
|
||||
runId: this.#binding.runId,
|
||||
completionContractSha256: this.#binding.completionContractSha256,
|
||||
canonicalSha256,
|
||||
});
|
||||
|
||||
return this.#db.transaction(async (tx) => {
|
||||
const [run] = await tx
|
||||
.select()
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, this.#binding.runId))
|
||||
.for("update")
|
||||
.limit(1);
|
||||
if (
|
||||
!run ||
|
||||
run.companyId !== this.#binding.companyId ||
|
||||
run.agentId !== this.#binding.agentId ||
|
||||
run.nativeIssueId !== this.#binding.issueId ||
|
||||
run.runtimeMode !== "native" ||
|
||||
run.completionContractId !== this.#binding.completionContractId ||
|
||||
run.completionContractSha256 !== this.#binding.completionContractSha256
|
||||
) {
|
||||
throw new Error("native_result_run_not_authorized");
|
||||
}
|
||||
|
||||
const [existing] = await tx
|
||||
.select()
|
||||
.from(nativeRunResults)
|
||||
.where(
|
||||
and(
|
||||
eq(nativeRunResults.runId, this.#binding.runId),
|
||||
eq(nativeRunResults.schemaStatus, "accepted"),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (existing) {
|
||||
if (existing.canonicalSha256 !== canonicalSha256) {
|
||||
throw new Error("native_result_replay_conflict");
|
||||
}
|
||||
return { disposition: "duplicate" as const, resultId: existing.id };
|
||||
}
|
||||
|
||||
const [inserted] = await tx
|
||||
.insert(nativeRunResults)
|
||||
.values({
|
||||
companyId: this.#binding.companyId,
|
||||
issueId: this.#binding.issueId,
|
||||
runId: this.#binding.runId,
|
||||
turnId: input.turnId ?? null,
|
||||
completionContractId: this.#binding.completionContractId,
|
||||
callerResultId: input.callerResultId ?? null,
|
||||
callerDedupeKey: input.callerDedupeKey ?? null,
|
||||
serverFingerprint,
|
||||
schemaStatus: "accepted",
|
||||
resultJson: canonical as unknown as Record<string, unknown>,
|
||||
canonicalSha256,
|
||||
})
|
||||
.returning({ id: nativeRunResults.id });
|
||||
if (!inserted) throw new Error("native_result_not_persisted");
|
||||
await tx
|
||||
.insert(nativeRunFinalizations)
|
||||
.values({
|
||||
runId: this.#binding.runId,
|
||||
companyId: this.#binding.companyId,
|
||||
issueId: this.#binding.issueId,
|
||||
phase: "workspace_finalizing",
|
||||
resultId: inserted.id,
|
||||
})
|
||||
.onConflictDoUpdate({
|
||||
target: nativeRunFinalizations.runId,
|
||||
set: {
|
||||
phase: "workspace_finalizing",
|
||||
resultId: inserted.id,
|
||||
failureCode: null,
|
||||
failureDetail: null,
|
||||
nextAttemptAt: null,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
await tx
|
||||
.update(heartbeatRuns)
|
||||
.set({
|
||||
nativePhase: "workspace_finalizing",
|
||||
nativePhaseUpdatedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(heartbeatRuns.id, this.#binding.runId));
|
||||
return { disposition: "committed" as const, resultId: inserted.id };
|
||||
});
|
||||
}
|
||||
|
||||
/** Rebuild the accepted result/finalization record from durable PRP events. */
|
||||
async reconcileTerminalEvent(event: PrpEvent): Promise<{
|
||||
readonly disposition: "committed" | "duplicate";
|
||||
readonly resultId: string;
|
||||
} | null> {
|
||||
if (event.eventType !== "run.terminal") return null;
|
||||
if (
|
||||
event.runId !== this.#binding.runId ||
|
||||
event.normalizedSessionId !== this.#binding.normalizedSessionId ||
|
||||
event.sourceInstanceId !== this.#binding.runnerSourceInstanceId
|
||||
) {
|
||||
throw new Error("native_terminal_binding_mismatch");
|
||||
}
|
||||
const terminal = event.payload as PrpTerminalState;
|
||||
assertTerminal(terminal);
|
||||
const [proposed] = await this.#db
|
||||
.select({ payload: heartbeatRunEvents.payload })
|
||||
.from(heartbeatRunEvents)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRunEvents.runId, this.#binding.runId),
|
||||
eq(
|
||||
heartbeatRunEvents.sourceInstanceId,
|
||||
this.#binding.runnerSourceInstanceId,
|
||||
),
|
||||
eq(heartbeatRunEvents.eventType, "run.result.proposed"),
|
||||
lt(heartbeatRunEvents.sourceSeq, event.sourceSeq),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(heartbeatRunEvents.sourceSeq))
|
||||
.limit(1);
|
||||
const proposedEnvelope = (
|
||||
proposed?.payload as Record<string, unknown> | undefined
|
||||
)?.prpEvent as Record<string, unknown> | undefined;
|
||||
if (proposedEnvelope?.payload === undefined) {
|
||||
throw new Error("native_terminal_result_missing");
|
||||
}
|
||||
return this.completeRun({
|
||||
result: proposedEnvelope.payload as PrpStructuredRunResult,
|
||||
terminal,
|
||||
turnId: event.turnId,
|
||||
callerDedupeKey: `prp-terminal:${event.sourceEventId}`,
|
||||
});
|
||||
}
|
||||
|
||||
async claimFinalization(input: {
|
||||
readonly leaseOwner: string;
|
||||
readonly leaseTtlMs?: number;
|
||||
}): Promise<{ readonly attempt: number; readonly resultId: string }> {
|
||||
const leaseTtlMs = input.leaseTtlMs ?? 30_000;
|
||||
if (
|
||||
!/^[A-Za-z0-9][A-Za-z0-9._:-]{0,239}$/.test(input.leaseOwner) ||
|
||||
!Number.isInteger(leaseTtlMs) ||
|
||||
leaseTtlMs < 1_000 ||
|
||||
leaseTtlMs > 300_000
|
||||
) {
|
||||
throw new Error("native_finalization_lease_invalid");
|
||||
}
|
||||
return this.#db.transaction(async (tx) => {
|
||||
const [row] = await tx
|
||||
.select()
|
||||
.from(nativeRunFinalizations)
|
||||
.where(
|
||||
and(
|
||||
eq(nativeRunFinalizations.runId, this.#binding.runId),
|
||||
eq(nativeRunFinalizations.companyId, this.#binding.companyId),
|
||||
eq(nativeRunFinalizations.issueId, this.#binding.issueId),
|
||||
),
|
||||
)
|
||||
.for("update")
|
||||
.limit(1);
|
||||
if (!row?.resultId || ["completed", "failed"].includes(row.phase)) {
|
||||
throw new Error("native_finalization_not_claimable");
|
||||
}
|
||||
if (row.nextAttemptAt && row.nextAttemptAt.getTime() > Date.now()) {
|
||||
throw new Error("native_finalization_retry_not_due");
|
||||
}
|
||||
if (
|
||||
row.leaseOwner &&
|
||||
row.leaseOwner !== input.leaseOwner &&
|
||||
row.leaseExpiresAt &&
|
||||
row.leaseExpiresAt.getTime() > Date.now()
|
||||
) {
|
||||
throw new Error("native_finalization_lease_conflict");
|
||||
}
|
||||
if (
|
||||
row.leaseOwner === input.leaseOwner &&
|
||||
row.leaseExpiresAt &&
|
||||
row.leaseExpiresAt.getTime() > Date.now()
|
||||
) {
|
||||
await tx
|
||||
.update(nativeRunFinalizations)
|
||||
.set({
|
||||
leaseExpiresAt: new Date(Date.now() + leaseTtlMs),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(nativeRunFinalizations.runId, this.#binding.runId));
|
||||
return { attempt: row.attempt, resultId: row.resultId };
|
||||
}
|
||||
const attempt = row.attempt + 1;
|
||||
await tx
|
||||
.update(nativeRunFinalizations)
|
||||
.set({
|
||||
attempt,
|
||||
leaseOwner: input.leaseOwner,
|
||||
leaseExpiresAt: new Date(Date.now() + leaseTtlMs),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(nativeRunFinalizations.runId, this.#binding.runId));
|
||||
return { attempt, resultId: row.resultId };
|
||||
});
|
||||
}
|
||||
|
||||
async markFinalizationRetry(input: {
|
||||
readonly leaseOwner: string;
|
||||
readonly failureCode: string;
|
||||
readonly retryAfterMs: number;
|
||||
}): Promise<void> {
|
||||
if (
|
||||
!/^[A-Za-z0-9][A-Za-z0-9._:-]{0,239}$/.test(input.failureCode) ||
|
||||
!Number.isInteger(input.retryAfterMs) ||
|
||||
input.retryAfterMs < 1_000 ||
|
||||
input.retryAfterMs > 24 * 60 * 60 * 1_000
|
||||
) {
|
||||
throw new Error("native_finalization_retry_invalid");
|
||||
}
|
||||
const [updated] = await this.#db
|
||||
.update(nativeRunFinalizations)
|
||||
.set({
|
||||
leaseOwner: null,
|
||||
leaseExpiresAt: null,
|
||||
failureCode: input.failureCode,
|
||||
nextAttemptAt: new Date(Date.now() + input.retryAfterMs),
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(
|
||||
and(
|
||||
eq(nativeRunFinalizations.runId, this.#binding.runId),
|
||||
eq(nativeRunFinalizations.companyId, this.#binding.companyId),
|
||||
eq(nativeRunFinalizations.issueId, this.#binding.issueId),
|
||||
eq(nativeRunFinalizations.leaseOwner, input.leaseOwner),
|
||||
),
|
||||
)
|
||||
.returning({ runId: nativeRunFinalizations.runId });
|
||||
if (!updated) throw new Error("native_finalization_lease_lost");
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,421 @@
|
|||
import { randomUUID } from "node:crypto";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { createServer } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import { eq } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
agents,
|
||||
companies,
|
||||
completionContracts,
|
||||
createDb,
|
||||
heartbeatRunEvents,
|
||||
heartbeatRuns,
|
||||
issues,
|
||||
nativeRunFinalizations,
|
||||
nativeRunResults,
|
||||
} from "@paperclipai/db";
|
||||
import type {
|
||||
PrpEvent,
|
||||
PrpStructuredRunResult,
|
||||
PrpTerminalState,
|
||||
} from "@paperclipai/paperclip-runner";
|
||||
|
||||
import {
|
||||
getEmbeddedPostgresTestSupport,
|
||||
startEmbeddedPostgresTestDatabase,
|
||||
} from "../../__tests__/helpers/embedded-postgres.js";
|
||||
import {
|
||||
runnerPrpWebSocketInternals,
|
||||
setupRunnerPrpWebSocketServer,
|
||||
} from "../../realtime/runner-prp-ws.js";
|
||||
import { NativeRunCoordinatorStore } from "./native-run-coordinator-store.js";
|
||||
import { runnerPrpCoordinator } from "./runner-prp-coordinator.js";
|
||||
import { PaperclipRunnerSemanticAuthority } from "./runner-semantic-authority.js";
|
||||
|
||||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported
|
||||
? describe
|
||||
: describe.skip;
|
||||
|
||||
if (!embeddedPostgresSupport.supported) {
|
||||
console.warn(
|
||||
`Skipping runner coordinator tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
||||
);
|
||||
}
|
||||
|
||||
interface SeededNativeRun {
|
||||
companyId: string;
|
||||
issueId: string;
|
||||
agentId: string;
|
||||
runId: string;
|
||||
runnerInstanceId: string;
|
||||
sessionId: string;
|
||||
completionContractId: string;
|
||||
completionContractSha256: string;
|
||||
}
|
||||
|
||||
const result: PrpStructuredRunResult = {
|
||||
schema: "paperclip.run_result.v1",
|
||||
reportedWorkDisposition: "done",
|
||||
summary: "The hidden runner completed the bounded task.",
|
||||
completionClaim: {
|
||||
contractRevision: "contract-v1",
|
||||
objectiveSatisfied: true,
|
||||
criteria: [],
|
||||
remainingWork: [],
|
||||
},
|
||||
evidence: [],
|
||||
verification: [{ commandOrCheck: "coordinator test", status: "passed" }],
|
||||
attentionRequests: [],
|
||||
artifacts: [],
|
||||
};
|
||||
|
||||
const terminal: PrpTerminalState = {
|
||||
schema: "paperclip.prp.terminal.v1",
|
||||
turnTerminalState: "completed",
|
||||
runTerminalState: "succeeded",
|
||||
reportedWorkDisposition: "done",
|
||||
};
|
||||
|
||||
function runnerEvent(seed: SeededNativeRun, sourceSeq = 1): PrpEvent {
|
||||
return {
|
||||
schema: "paperclip.prp.event.v1",
|
||||
sourceEventId: `event-${sourceSeq}`,
|
||||
sourceSeq,
|
||||
sourceInstanceId: seed.runnerInstanceId,
|
||||
sourceKind: "runner",
|
||||
runId: seed.runId,
|
||||
normalizedSessionId: seed.sessionId,
|
||||
turnId: "turn-1",
|
||||
itemId: "item-1",
|
||||
eventType: "turn.started",
|
||||
schemaVersion: 1,
|
||||
priority: 1,
|
||||
emittedAt: "2026-08-25T18:00:00.000Z",
|
||||
payload: {},
|
||||
};
|
||||
}
|
||||
|
||||
describeEmbeddedPostgres("hidden runner PRP coordinator", () => {
|
||||
let db!: ReturnType<typeof createDb>;
|
||||
let tempDb: Awaited<
|
||||
ReturnType<typeof startEmbeddedPostgresTestDatabase>
|
||||
> | null = null;
|
||||
const scratchDirectories: string[] = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
tempDb = await startEmbeddedPostgresTestDatabase(
|
||||
"paperclip-runner-coordinator-",
|
||||
);
|
||||
db = createDb(tempDb.connectionString);
|
||||
}, 20_000);
|
||||
|
||||
afterEach(async () => {
|
||||
runnerPrpWebSocketInternals.resetForTests();
|
||||
await db.delete(nativeRunFinalizations);
|
||||
await db.delete(nativeRunResults);
|
||||
await db.delete(heartbeatRunEvents);
|
||||
await db.update(issues).set({ executionRunId: null });
|
||||
await db.delete(heartbeatRuns);
|
||||
await db.delete(completionContracts);
|
||||
await db.delete(issues);
|
||||
await db.delete(agents);
|
||||
await db.delete(companies);
|
||||
for (const directory of scratchDirectories.splice(0)) {
|
||||
rmSync(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await tempDb?.cleanup();
|
||||
});
|
||||
|
||||
async function seedNativeRun(): Promise<SeededNativeRun> {
|
||||
const companyId = randomUUID();
|
||||
const issueId = randomUUID();
|
||||
const agentId = randomUUID();
|
||||
const runId = randomUUID();
|
||||
const runnerInstanceId = randomUUID();
|
||||
const sessionId = randomUUID();
|
||||
const completionContractId = randomUUID();
|
||||
const completionContractSha256 = `sha256:${"c".repeat(64)}`;
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
name: "Runner Test Company",
|
||||
issuePrefix: `R${companyId.replaceAll("-", "").slice(0, 6).toUpperCase()}`,
|
||||
requireBoardApprovalForNewAgents: false,
|
||||
});
|
||||
await db.insert(agents).values({
|
||||
id: agentId,
|
||||
companyId,
|
||||
name: "Codex runner",
|
||||
role: "engineer",
|
||||
status: "running",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
runtimeConfig: {},
|
||||
});
|
||||
await db.insert(issues).values({
|
||||
id: issueId,
|
||||
companyId,
|
||||
identifier: `RUN-${runId.slice(0, 8)}`,
|
||||
title: "Exercise the hidden native coordinator",
|
||||
description: "Verify transport and durable server boundaries.",
|
||||
status: "in_progress",
|
||||
priority: "medium",
|
||||
workMode: "standard",
|
||||
assigneeAgentId: agentId,
|
||||
});
|
||||
await db.insert(completionContracts).values({
|
||||
id: completionContractId,
|
||||
companyId,
|
||||
issueId,
|
||||
revision: 1,
|
||||
schemaVersion: "paperclip.completion-contract.v1",
|
||||
policyVersion: "policy-v1",
|
||||
risk: "low",
|
||||
completionAuthority: "runner",
|
||||
incompleteCriteriaPolicy: "fail_closed",
|
||||
contractJson: { criteria: [] },
|
||||
canonicalSha256: completionContractSha256,
|
||||
createdByActorType: "system",
|
||||
createdByActorId: "test",
|
||||
});
|
||||
await db.insert(heartbeatRuns).values({
|
||||
id: runId,
|
||||
companyId,
|
||||
agentId,
|
||||
status: "running",
|
||||
runtimeMode: "native",
|
||||
nativeIssueId: issueId,
|
||||
runnerInstanceId,
|
||||
nativeSessionId: sessionId,
|
||||
driverKind: "codex",
|
||||
driverVersion: "0.3.0",
|
||||
completionContractId,
|
||||
completionContractSha256,
|
||||
nativePhase: "observed",
|
||||
});
|
||||
await db
|
||||
.update(issues)
|
||||
.set({ executionRunId: runId })
|
||||
.where(eq(issues.id, issueId));
|
||||
return {
|
||||
companyId,
|
||||
issueId,
|
||||
agentId,
|
||||
runId,
|
||||
runnerInstanceId,
|
||||
sessionId,
|
||||
completionContractId,
|
||||
completionContractSha256,
|
||||
};
|
||||
}
|
||||
|
||||
function store(seed: SeededNativeRun): NativeRunCoordinatorStore {
|
||||
return new NativeRunCoordinatorStore(db, {
|
||||
companyId: seed.companyId,
|
||||
issueId: seed.issueId,
|
||||
runId: seed.runId,
|
||||
agentId: seed.agentId,
|
||||
normalizedSessionId: seed.sessionId,
|
||||
runnerSourceInstanceId: seed.runnerInstanceId,
|
||||
completionContractId: seed.completionContractId,
|
||||
completionContractSha256: seed.completionContractSha256,
|
||||
});
|
||||
}
|
||||
|
||||
it("registers only an exact Codex native binding and exposes read-only tools", async () => {
|
||||
const seed = await seedNativeRun();
|
||||
const server = createServer();
|
||||
setupRunnerPrpWebSocketServer(server, { apiUrl: "http://127.0.0.1:3213" });
|
||||
const stateRoot = mkdtempSync(resolve(tmpdir(), "paperclip-runner-state-"));
|
||||
scratchDirectories.push(stateRoot);
|
||||
const coordinator = runnerPrpCoordinator(db, { stateRoot });
|
||||
await expect(
|
||||
coordinator.prepare({
|
||||
...seed,
|
||||
companyId: randomUUID(),
|
||||
normalizedSessionId: seed.sessionId,
|
||||
environmentLeaseId: "environment-lease-1",
|
||||
turnId: "turn-1",
|
||||
itemId: "item-1",
|
||||
runnerVersion: "0.3.0",
|
||||
runnerDigest: `sha256:${"a".repeat(64)}`,
|
||||
}),
|
||||
).rejects.toThrow("runner_prp_run_not_authorized");
|
||||
const prepared = await coordinator.prepare({
|
||||
...seed,
|
||||
normalizedSessionId: seed.sessionId,
|
||||
environmentLeaseId: "environment-lease-1",
|
||||
turnId: "turn-1",
|
||||
itemId: "item-1",
|
||||
runnerVersion: "0.3.0",
|
||||
runnerDigest: `sha256:${"a".repeat(64)}`,
|
||||
});
|
||||
|
||||
expect(prepared.connectUrl).toBe(
|
||||
`ws://127.0.0.1:3213/api/runner/v1/connect/${seed.runId}`,
|
||||
);
|
||||
expect(prepared.bootstrapTicket).toMatch(/^bootstrap_/);
|
||||
expect(prepared.semanticTools.map((tool) => tool.name)).toEqual([
|
||||
"get_task_context",
|
||||
"get_task_history",
|
||||
"list_documents",
|
||||
"read_document",
|
||||
"list_document_revisions",
|
||||
]);
|
||||
expect(
|
||||
runnerPrpWebSocketInternals.activeRegistration({
|
||||
companyId: seed.companyId,
|
||||
runId: seed.runId,
|
||||
}),
|
||||
).toBe(true);
|
||||
await expect(
|
||||
coordinator.prepare({
|
||||
...seed,
|
||||
normalizedSessionId: seed.sessionId,
|
||||
environmentLeaseId: "environment-lease-1",
|
||||
turnId: "turn-1",
|
||||
itemId: "item-1",
|
||||
runnerVersion: "0.3.0",
|
||||
runnerDigest: `sha256:${"a".repeat(64)}`,
|
||||
}),
|
||||
).rejects.toThrow("runner_prp_authority_already_registered");
|
||||
await prepared.release();
|
||||
expect(
|
||||
runnerPrpWebSocketInternals.activeRegistration({
|
||||
companyId: seed.companyId,
|
||||
runId: seed.runId,
|
||||
}),
|
||||
).toBe(false);
|
||||
server.close();
|
||||
});
|
||||
|
||||
it("rechecks task ownership and returns semantic receipts", async () => {
|
||||
const seed = await seedNativeRun();
|
||||
const authority = new PaperclipRunnerSemanticAuthority(db, {
|
||||
companyId: seed.companyId,
|
||||
issueId: seed.issueId,
|
||||
runId: seed.runId,
|
||||
agentId: seed.agentId,
|
||||
});
|
||||
const call = {
|
||||
callId: "call-1",
|
||||
operationId: "get_task_context",
|
||||
correlation: {
|
||||
runId: seed.runId,
|
||||
normalizedSessionId: seed.sessionId,
|
||||
turnId: "turn-1",
|
||||
itemId: "item-1",
|
||||
},
|
||||
input: {},
|
||||
};
|
||||
const allowed = await authority.dispatch(call);
|
||||
expect(allowed).toMatchObject({
|
||||
ok: true,
|
||||
operationId: "get_task_context",
|
||||
value: { activeTask: { id: seed.issueId }, run: { id: seed.runId } },
|
||||
inputReceipt: { phase: "input" },
|
||||
resultReceipt: { phase: "result" },
|
||||
});
|
||||
|
||||
await db
|
||||
.update(issues)
|
||||
.set({ assigneeAgentId: null })
|
||||
.where(eq(issues.id, seed.issueId));
|
||||
const denied = await authority.dispatch({ ...call, callId: "call-2" });
|
||||
expect(denied).toMatchObject({
|
||||
ok: false,
|
||||
error: { code: "task_ownership_denied", retryable: false },
|
||||
resultReceipt: { phase: "result" },
|
||||
});
|
||||
});
|
||||
|
||||
it("persists events and results idempotently and leases finalization", async () => {
|
||||
const seed = await seedNativeRun();
|
||||
const nativeStore = store(seed);
|
||||
const event = runnerEvent(seed);
|
||||
await expect(nativeStore.appendEvent(event)).resolves.toMatchObject({
|
||||
disposition: "committed",
|
||||
cursor: 1,
|
||||
highestContiguousSourceSeq: 1,
|
||||
});
|
||||
await expect(nativeStore.appendEvent(event)).resolves.toMatchObject({
|
||||
disposition: "duplicate",
|
||||
cursor: 1,
|
||||
});
|
||||
await expect(
|
||||
nativeStore.appendEvent({ ...event, priority: 2 }),
|
||||
).rejects.toThrow("native_event_replay_conflict");
|
||||
await expect(nativeStore.appendEvent(runnerEvent(seed, 3))).rejects.toThrow(
|
||||
"native_event_source_gap",
|
||||
);
|
||||
|
||||
const resultEvent = {
|
||||
...runnerEvent(seed, 2),
|
||||
eventType: "run.result.proposed",
|
||||
payload: result,
|
||||
} as PrpEvent;
|
||||
const terminalEvent = {
|
||||
...runnerEvent(seed, 3),
|
||||
eventType: "run.terminal",
|
||||
payload: terminal,
|
||||
} as PrpEvent;
|
||||
await nativeStore.appendEvent(resultEvent);
|
||||
await nativeStore.appendEvent(terminalEvent);
|
||||
const firstResult = await nativeStore.reconcileTerminalEvent(terminalEvent);
|
||||
if (!firstResult)
|
||||
throw new Error("terminal reconciliation returned no result");
|
||||
expect(firstResult.disposition).toBe("committed");
|
||||
await expect(nativeStore.appendEvent(event)).resolves.toMatchObject({
|
||||
disposition: "duplicate",
|
||||
highestContiguousSourceSeq: 3,
|
||||
});
|
||||
await expect(
|
||||
nativeStore.completeRun({
|
||||
result,
|
||||
terminal,
|
||||
turnId: "turn-1",
|
||||
callerDedupeKey: "result-1",
|
||||
}),
|
||||
).resolves.toEqual({ ...firstResult, disposition: "duplicate" });
|
||||
await expect(
|
||||
nativeStore.completeRun({
|
||||
result: { ...result, summary: "Conflicting retry" },
|
||||
terminal,
|
||||
turnId: "turn-1",
|
||||
callerDedupeKey: "result-1",
|
||||
}),
|
||||
).rejects.toThrow("native_result_replay_conflict");
|
||||
|
||||
await expect(
|
||||
nativeStore.claimFinalization({ leaseOwner: "server-1" }),
|
||||
).resolves.toMatchObject({ attempt: 1, resultId: firstResult.resultId });
|
||||
await expect(
|
||||
nativeStore.claimFinalization({ leaseOwner: "server-1" }),
|
||||
).resolves.toMatchObject({ attempt: 1, resultId: firstResult.resultId });
|
||||
await expect(
|
||||
nativeStore.claimFinalization({ leaseOwner: "server-2" }),
|
||||
).rejects.toThrow("native_finalization_lease_conflict");
|
||||
await nativeStore.markFinalizationRetry({
|
||||
leaseOwner: "server-1",
|
||||
failureCode: "workspace_busy",
|
||||
retryAfterMs: 1_000,
|
||||
});
|
||||
await expect(
|
||||
nativeStore.claimFinalization({ leaseOwner: "server-2" }),
|
||||
).rejects.toThrow("native_finalization_retry_not_due");
|
||||
await db
|
||||
.update(nativeRunFinalizations)
|
||||
.set({ nextAttemptAt: new Date(0) })
|
||||
.where(eq(nativeRunFinalizations.runId, seed.runId));
|
||||
await expect(
|
||||
nativeStore.claimFinalization({ leaseOwner: "server-2" }),
|
||||
).resolves.toMatchObject({ attempt: 2, resultId: firstResult.resultId });
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,261 @@
|
|||
import { resolve } from "node:path";
|
||||
|
||||
import { and, eq } from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { agents, heartbeatRuns, issues } from "@paperclipai/db";
|
||||
import {
|
||||
DurablePrpControlPlane,
|
||||
type PaperclipSemanticToolDefinition,
|
||||
type PrpStructuredRunResult,
|
||||
type PrpTerminalState,
|
||||
} from "../../vendor/paperclip-runner/index.js";
|
||||
|
||||
import { registerRunnerPrpAuthority } from "../../realtime/runner-prp-ws.js";
|
||||
import { NativeRunCoordinatorStore } from "./native-run-coordinator-store.js";
|
||||
import { PaperclipRunnerSemanticAuthority } from "./runner-semantic-authority.js";
|
||||
|
||||
const UUID_PATTERN =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
|
||||
const STABLE_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,239}$/;
|
||||
const RUNNER_DIGEST_PATTERN = /^sha256:[0-9a-f]{64}$/;
|
||||
|
||||
export interface PrepareRunnerPrpSessionInput {
|
||||
readonly companyId: string;
|
||||
readonly issueId: string;
|
||||
readonly runId: string;
|
||||
readonly agentId: string;
|
||||
readonly runnerInstanceId: string;
|
||||
readonly environmentLeaseId: string;
|
||||
readonly normalizedSessionId: string;
|
||||
readonly turnId: string;
|
||||
readonly itemId: string;
|
||||
readonly runnerVersion: string;
|
||||
readonly runnerDigest: string;
|
||||
readonly bootstrapTtlMs?: number;
|
||||
readonly connectionLeaseTtlMs?: number;
|
||||
}
|
||||
|
||||
export interface PreparedRunnerPrpSession {
|
||||
readonly connectUrl: string;
|
||||
/** One-use secret. Pass it only through the runner's protected bootstrap channel. */
|
||||
readonly bootstrapTicket: string;
|
||||
readonly semanticTools: readonly PaperclipSemanticToolDefinition[];
|
||||
queueCommand(
|
||||
type: string,
|
||||
payload?: Record<string, unknown>,
|
||||
commandId?: string,
|
||||
): { readonly commandId: string; readonly controllerSeq: number };
|
||||
completeRun(input: {
|
||||
readonly result: PrpStructuredRunResult;
|
||||
readonly terminal: PrpTerminalState;
|
||||
readonly turnId?: string;
|
||||
readonly callerResultId?: string;
|
||||
readonly callerDedupeKey?: string;
|
||||
}): Promise<{
|
||||
readonly disposition: "committed" | "duplicate";
|
||||
readonly resultId: string;
|
||||
}>;
|
||||
release(): Promise<void>;
|
||||
}
|
||||
|
||||
function clampDuration(
|
||||
value: number | undefined,
|
||||
fallback: number,
|
||||
minimum: number,
|
||||
maximum: number,
|
||||
): number {
|
||||
if (value === undefined) return fallback;
|
||||
if (!Number.isInteger(value) || value < minimum || value > maximum) {
|
||||
throw new Error("runner_prp_credential_ttl_invalid");
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function validateInput(input: PrepareRunnerPrpSessionInput): void {
|
||||
for (const id of [
|
||||
input.companyId,
|
||||
input.issueId,
|
||||
input.runId,
|
||||
input.agentId,
|
||||
input.runnerInstanceId,
|
||||
input.normalizedSessionId,
|
||||
]) {
|
||||
if (!UUID_PATTERN.test(id)) throw new Error("runner_prp_binding_invalid");
|
||||
}
|
||||
for (const id of [input.environmentLeaseId, input.turnId, input.itemId]) {
|
||||
if (!STABLE_ID_PATTERN.test(id))
|
||||
throw new Error("runner_prp_binding_invalid");
|
||||
}
|
||||
if (
|
||||
!STABLE_ID_PATTERN.test(input.runnerVersion) ||
|
||||
!RUNNER_DIGEST_PATTERN.test(input.runnerDigest)
|
||||
) {
|
||||
throw new Error("runner_prp_runner_identity_invalid");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the hidden, run-bound PRP authority. This module does not select a
|
||||
* runtime or start runnerd. The flagged adapter owns those actions later.
|
||||
*/
|
||||
export function runnerPrpCoordinator(
|
||||
db: Db,
|
||||
options: {
|
||||
readonly stateRoot: string;
|
||||
},
|
||||
) {
|
||||
const stateRoot = resolve(options.stateRoot);
|
||||
|
||||
return {
|
||||
prepare: async (
|
||||
input: PrepareRunnerPrpSessionInput,
|
||||
): Promise<PreparedRunnerPrpSession> => {
|
||||
validateInput(input);
|
||||
const bootstrapTtlMs = clampDuration(
|
||||
input.bootstrapTtlMs,
|
||||
30_000,
|
||||
1_000,
|
||||
60_000,
|
||||
);
|
||||
const connectionLeaseTtlMs = clampDuration(
|
||||
input.connectionLeaseTtlMs,
|
||||
60 * 60 * 1_000,
|
||||
60_000,
|
||||
24 * 60 * 60 * 1_000,
|
||||
);
|
||||
|
||||
const [binding] = await db
|
||||
.select({ run: heartbeatRuns, issue: issues, agent: agents })
|
||||
.from(heartbeatRuns)
|
||||
.innerJoin(
|
||||
issues,
|
||||
and(
|
||||
eq(issues.id, heartbeatRuns.nativeIssueId),
|
||||
eq(issues.companyId, heartbeatRuns.companyId),
|
||||
),
|
||||
)
|
||||
.innerJoin(
|
||||
agents,
|
||||
and(
|
||||
eq(agents.id, heartbeatRuns.agentId),
|
||||
eq(agents.companyId, heartbeatRuns.companyId),
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRuns.id, input.runId),
|
||||
eq(heartbeatRuns.companyId, input.companyId),
|
||||
eq(heartbeatRuns.agentId, input.agentId),
|
||||
eq(heartbeatRuns.nativeIssueId, input.issueId),
|
||||
eq(heartbeatRuns.runnerInstanceId, input.runnerInstanceId),
|
||||
eq(heartbeatRuns.nativeSessionId, input.normalizedSessionId),
|
||||
eq(heartbeatRuns.runtimeMode, "native"),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (
|
||||
!binding ||
|
||||
!["queued", "running"].includes(binding.run.status) ||
|
||||
binding.run.driverKind !== "codex" ||
|
||||
!binding.run.completionContractId ||
|
||||
!binding.run.completionContractSha256 ||
|
||||
binding.issue.assigneeAgentId !== input.agentId ||
|
||||
binding.issue.executionRunId !== input.runId ||
|
||||
["paused", "terminated", "pending_approval", "error"].includes(
|
||||
binding.agent.status,
|
||||
)
|
||||
) {
|
||||
throw new Error("runner_prp_run_not_authorized");
|
||||
}
|
||||
|
||||
const semanticAuthority = new PaperclipRunnerSemanticAuthority(db, {
|
||||
companyId: input.companyId,
|
||||
issueId: input.issueId,
|
||||
runId: input.runId,
|
||||
agentId: input.agentId,
|
||||
});
|
||||
const semanticTools = await semanticAuthority.listAlwaysAvailableTools();
|
||||
const nativeStore = new NativeRunCoordinatorStore(db, {
|
||||
companyId: input.companyId,
|
||||
issueId: input.issueId,
|
||||
runId: input.runId,
|
||||
agentId: input.agentId,
|
||||
normalizedSessionId: input.normalizedSessionId,
|
||||
runnerSourceInstanceId: input.runnerInstanceId,
|
||||
completionContractId: binding.run.completionContractId,
|
||||
completionContractSha256: binding.run.completionContractSha256,
|
||||
});
|
||||
const authority = new DurablePrpControlPlane({
|
||||
stateDirectory: resolve(stateRoot, input.runId),
|
||||
identity: {
|
||||
runnerInstanceId: input.runnerInstanceId,
|
||||
environmentLeaseId: input.environmentLeaseId,
|
||||
runId: input.runId,
|
||||
normalizedSessionId: input.normalizedSessionId,
|
||||
turnId: input.turnId,
|
||||
itemId: input.itemId,
|
||||
},
|
||||
expectedRunnerVersion: input.runnerVersion,
|
||||
expectedRunnerDigest: input.runnerDigest,
|
||||
connectionLeaseTtlMs,
|
||||
onCommittedEvent: async (event) => {
|
||||
await nativeStore.appendEvent(event);
|
||||
await nativeStore.reconcileTerminalEvent(event);
|
||||
},
|
||||
onSemanticToolInput: async (call) => {
|
||||
const result = await semanticAuthority.dispatch({
|
||||
callId: call.callId,
|
||||
operationId: call.operationId,
|
||||
correlation: call.correlation,
|
||||
input: call.input,
|
||||
});
|
||||
return { result, isError: !result.ok };
|
||||
},
|
||||
});
|
||||
|
||||
const registration = await registerRunnerPrpAuthority({
|
||||
companyId: input.companyId,
|
||||
runId: input.runId,
|
||||
authority,
|
||||
});
|
||||
let bootstrapTicket: string;
|
||||
try {
|
||||
bootstrapTicket = authority.issueBootstrapTicket(bootstrapTtlMs);
|
||||
} catch (error) {
|
||||
authority.disconnectActiveRunner();
|
||||
await registration.release();
|
||||
throw error;
|
||||
}
|
||||
let released = false;
|
||||
return {
|
||||
connectUrl: registration.connectUrl,
|
||||
bootstrapTicket,
|
||||
semanticTools,
|
||||
queueCommand: (type, payload = {}, commandId) => {
|
||||
if (released) throw new Error("runner_prp_session_released");
|
||||
const command = authority.queueCommand(
|
||||
type,
|
||||
payload,
|
||||
commandId,
|
||||
true,
|
||||
);
|
||||
return {
|
||||
commandId: command.commandId,
|
||||
controllerSeq: command.controllerSeq,
|
||||
};
|
||||
},
|
||||
completeRun: (completeInput) => {
|
||||
if (released) throw new Error("runner_prp_session_released");
|
||||
return nativeStore.completeRun(completeInput);
|
||||
},
|
||||
release: async () => {
|
||||
if (released) return;
|
||||
released = true;
|
||||
authority.disconnectActiveRunner();
|
||||
await registration.release();
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,351 @@
|
|||
import { and, desc, eq, isNull } from "drizzle-orm";
|
||||
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
agents,
|
||||
documentRevisions,
|
||||
documents,
|
||||
heartbeatRuns,
|
||||
issueComments,
|
||||
issueDocuments,
|
||||
issues,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
PaperclipSemanticDispatcher,
|
||||
type PaperclipJsonValue,
|
||||
type PaperclipSemanticActionBinding,
|
||||
type PaperclipSemanticActionId,
|
||||
type PaperclipSemanticAuthorizationRecord,
|
||||
type PaperclipSemanticRunContext,
|
||||
type PaperclipSemanticToolCall,
|
||||
type PaperclipSemanticToolDefinition,
|
||||
type PaperclipSemanticToolResult,
|
||||
} from "../../vendor/paperclip-runner/index.js";
|
||||
|
||||
export interface PaperclipRunnerSemanticBinding {
|
||||
readonly companyId: string;
|
||||
readonly issueId: string;
|
||||
readonly runId: string;
|
||||
readonly agentId: string;
|
||||
}
|
||||
|
||||
const READ_OPERATION_IDS = [
|
||||
"get_task_context",
|
||||
"get_task_history",
|
||||
"list_documents",
|
||||
"read_document",
|
||||
"list_document_revisions",
|
||||
] as const satisfies readonly PaperclipSemanticActionId[];
|
||||
|
||||
type BoundContext = {
|
||||
readonly run: typeof heartbeatRuns.$inferSelect;
|
||||
readonly issue: typeof issues.$inferSelect;
|
||||
readonly agent: typeof agents.$inferSelect;
|
||||
};
|
||||
|
||||
function boundedLimit(value: unknown): number {
|
||||
return typeof value === "number" && Number.isInteger(value)
|
||||
? Math.max(1, Math.min(value, 200))
|
||||
: 50;
|
||||
}
|
||||
|
||||
function requiredString(value: unknown): string {
|
||||
if (typeof value !== "string" || value.length === 0 || value.length > 240) {
|
||||
throw new Error("paperclip_runner_semantic_input_invalid");
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function jsonValue(value: unknown): PaperclipJsonValue {
|
||||
return JSON.parse(JSON.stringify(value)) as PaperclipJsonValue;
|
||||
}
|
||||
|
||||
function activeAgentStatus(status: string): "active" | "inactive" {
|
||||
return ["paused", "terminated", "pending_approval", "error"].includes(status)
|
||||
? "inactive"
|
||||
: "active";
|
||||
}
|
||||
|
||||
/**
|
||||
* Run-scoped semantic authority for the hidden native coordinator.
|
||||
* This first server slice binds only same-task read operations. A catalog
|
||||
* entry remains undiscoverable until a later PR adds its guarded binding.
|
||||
*/
|
||||
export class PaperclipRunnerSemanticAuthority {
|
||||
readonly #db: Db;
|
||||
readonly #binding: PaperclipRunnerSemanticBinding;
|
||||
readonly #dispatcher: PaperclipSemanticDispatcher;
|
||||
|
||||
constructor(db: Db, binding: PaperclipRunnerSemanticBinding) {
|
||||
this.#db = db;
|
||||
this.#binding = structuredClone(binding);
|
||||
this.#dispatcher = new PaperclipSemanticDispatcher({
|
||||
contextProvider: (runId) => this.#context(runId),
|
||||
bindings: READ_OPERATION_IDS.map((operationId) =>
|
||||
this.#readBinding(operationId),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
listAlwaysAvailableTools(): Promise<
|
||||
readonly PaperclipSemanticToolDefinition[]
|
||||
> {
|
||||
return this.#dispatcher.listAlwaysAvailableTools(this.#binding.runId);
|
||||
}
|
||||
|
||||
dispatch(
|
||||
call: Omit<PaperclipSemanticToolCall, "runId">,
|
||||
): Promise<PaperclipSemanticToolResult> {
|
||||
return this.#dispatcher.dispatch({ ...call, runId: this.#binding.runId });
|
||||
}
|
||||
|
||||
authorizationRecords(): readonly PaperclipSemanticAuthorizationRecord[] {
|
||||
return this.#dispatcher.authorizationRecords();
|
||||
}
|
||||
|
||||
#readBinding(
|
||||
operationId: (typeof READ_OPERATION_IDS)[number],
|
||||
): PaperclipSemanticActionBinding {
|
||||
return {
|
||||
operationId,
|
||||
execute: async (invocation) => {
|
||||
const context = await this.#loadBoundContext();
|
||||
this.#assertActiveContext(context, true);
|
||||
const input = invocation.input;
|
||||
switch (operationId) {
|
||||
case "get_task_context":
|
||||
return {
|
||||
value: jsonValue({
|
||||
company: { id: this.#binding.companyId },
|
||||
actor: {
|
||||
id: context.agent.id,
|
||||
name: context.agent.name,
|
||||
role: context.agent.role,
|
||||
title: context.agent.title,
|
||||
capabilities: context.agent.capabilities,
|
||||
},
|
||||
activeTask: {
|
||||
id: context.issue.id,
|
||||
identifier: context.issue.identifier,
|
||||
title: context.issue.title,
|
||||
description: context.issue.description,
|
||||
status: context.issue.status,
|
||||
statusVersion: context.issue.statusVersion,
|
||||
priority: context.issue.priority,
|
||||
workMode: context.issue.workMode,
|
||||
parentId: context.issue.parentId,
|
||||
projectId: context.issue.projectId,
|
||||
goalId: context.issue.goalId,
|
||||
},
|
||||
run: {
|
||||
id: context.run.id,
|
||||
status: context.run.status,
|
||||
invocationSource: context.run.invocationSource,
|
||||
},
|
||||
}),
|
||||
references: [{ kind: "task", id: context.issue.id }],
|
||||
};
|
||||
case "get_task_history": {
|
||||
const rows = await this.#db
|
||||
.select({
|
||||
id: issueComments.id,
|
||||
body: issueComments.body,
|
||||
authorAgentId: issueComments.authorAgentId,
|
||||
authorUserId: issueComments.authorUserId,
|
||||
createdAt: issueComments.createdAt,
|
||||
})
|
||||
.from(issueComments)
|
||||
.where(
|
||||
and(
|
||||
eq(issueComments.companyId, this.#binding.companyId),
|
||||
eq(issueComments.issueId, this.#binding.issueId),
|
||||
isNull(issueComments.deletedAt),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(issueComments.createdAt))
|
||||
.limit(boundedLimit(input.limit));
|
||||
return {
|
||||
value: jsonValue({ comments: rows.reverse() }),
|
||||
references: [{ kind: "task", id: context.issue.id }],
|
||||
};
|
||||
}
|
||||
case "list_documents": {
|
||||
const rows = await this.#db
|
||||
.select({
|
||||
key: issueDocuments.key,
|
||||
id: documents.id,
|
||||
title: documents.title,
|
||||
format: documents.format,
|
||||
latestRevisionId: documents.latestRevisionId,
|
||||
latestRevisionNumber: documents.latestRevisionNumber,
|
||||
lockedAt: documents.lockedAt,
|
||||
updatedAt: documents.updatedAt,
|
||||
})
|
||||
.from(issueDocuments)
|
||||
.innerJoin(documents, eq(documents.id, issueDocuments.documentId))
|
||||
.where(
|
||||
and(
|
||||
eq(issueDocuments.companyId, this.#binding.companyId),
|
||||
eq(issueDocuments.issueId, this.#binding.issueId),
|
||||
eq(documents.companyId, this.#binding.companyId),
|
||||
),
|
||||
);
|
||||
return {
|
||||
value: jsonValue({ documents: rows }),
|
||||
references: rows.map((row) => ({
|
||||
kind: "document_revision" as const,
|
||||
id: row.latestRevisionId ?? row.id,
|
||||
})),
|
||||
};
|
||||
}
|
||||
case "read_document": {
|
||||
const key = requiredString(input.key);
|
||||
const [row] = await this.#db
|
||||
.select({
|
||||
key: issueDocuments.key,
|
||||
id: documents.id,
|
||||
title: documents.title,
|
||||
format: documents.format,
|
||||
body: documents.latestBody,
|
||||
latestRevisionId: documents.latestRevisionId,
|
||||
latestRevisionNumber: documents.latestRevisionNumber,
|
||||
lockedAt: documents.lockedAt,
|
||||
updatedAt: documents.updatedAt,
|
||||
})
|
||||
.from(issueDocuments)
|
||||
.innerJoin(documents, eq(documents.id, issueDocuments.documentId))
|
||||
.where(
|
||||
and(
|
||||
eq(issueDocuments.companyId, this.#binding.companyId),
|
||||
eq(issueDocuments.issueId, this.#binding.issueId),
|
||||
eq(issueDocuments.key, key),
|
||||
eq(documents.companyId, this.#binding.companyId),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (!row) throw new Error("paperclip_runner_document_not_found");
|
||||
return {
|
||||
value: jsonValue({ document: row }),
|
||||
references: [
|
||||
{
|
||||
kind: "document_revision",
|
||||
id: row.latestRevisionId ?? row.id,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
case "list_document_revisions": {
|
||||
const key = requiredString(input.key);
|
||||
const rows = await this.#db
|
||||
.select({
|
||||
id: documentRevisions.id,
|
||||
revisionNumber: documentRevisions.revisionNumber,
|
||||
title: documentRevisions.title,
|
||||
format: documentRevisions.format,
|
||||
body: documentRevisions.body,
|
||||
changeSummary: documentRevisions.changeSummary,
|
||||
createdByAgentId: documentRevisions.createdByAgentId,
|
||||
createdByUserId: documentRevisions.createdByUserId,
|
||||
createdAt: documentRevisions.createdAt,
|
||||
})
|
||||
.from(issueDocuments)
|
||||
.innerJoin(
|
||||
documentRevisions,
|
||||
eq(documentRevisions.documentId, issueDocuments.documentId),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(issueDocuments.companyId, this.#binding.companyId),
|
||||
eq(issueDocuments.issueId, this.#binding.issueId),
|
||||
eq(issueDocuments.key, key),
|
||||
eq(documentRevisions.companyId, this.#binding.companyId),
|
||||
),
|
||||
)
|
||||
.orderBy(desc(documentRevisions.revisionNumber))
|
||||
.limit(boundedLimit(input.limit));
|
||||
return {
|
||||
value: jsonValue({ revisions: rows }),
|
||||
references: rows.map((row) => ({
|
||||
kind: "document_revision" as const,
|
||||
id: row.id,
|
||||
})),
|
||||
};
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async #context(requestedRunId: string): Promise<PaperclipSemanticRunContext> {
|
||||
if (requestedRunId !== this.#binding.runId) {
|
||||
throw new Error("paperclip_runner_semantic_run_mismatch");
|
||||
}
|
||||
const context = await this.#loadBoundContext();
|
||||
this.#assertActiveContext(context, false);
|
||||
return {
|
||||
runId: context.run.id,
|
||||
companyId: context.run.companyId,
|
||||
actor: {
|
||||
id: context.agent.id,
|
||||
companyId: context.agent.companyId,
|
||||
status: activeAgentStatus(context.agent.status),
|
||||
role: context.agent.role,
|
||||
claims: [],
|
||||
},
|
||||
activeTask: {
|
||||
id: context.issue.id,
|
||||
companyId: context.issue.companyId,
|
||||
assigneeActorId: context.issue.assigneeAgentId,
|
||||
executionRunId: context.issue.executionRunId,
|
||||
status: context.issue.status,
|
||||
workMode: context.issue
|
||||
.workMode as PaperclipSemanticRunContext["activeTask"]["workMode"],
|
||||
},
|
||||
delegatedClaims: [],
|
||||
};
|
||||
}
|
||||
|
||||
async #loadBoundContext(): Promise<BoundContext> {
|
||||
const [row] = await this.#db
|
||||
.select({ run: heartbeatRuns, issue: issues, agent: agents })
|
||||
.from(heartbeatRuns)
|
||||
.innerJoin(
|
||||
issues,
|
||||
and(
|
||||
eq(issues.id, heartbeatRuns.nativeIssueId),
|
||||
eq(issues.companyId, heartbeatRuns.companyId),
|
||||
),
|
||||
)
|
||||
.innerJoin(
|
||||
agents,
|
||||
and(
|
||||
eq(agents.id, heartbeatRuns.agentId),
|
||||
eq(agents.companyId, heartbeatRuns.companyId),
|
||||
),
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(heartbeatRuns.id, this.#binding.runId),
|
||||
eq(heartbeatRuns.companyId, this.#binding.companyId),
|
||||
eq(heartbeatRuns.agentId, this.#binding.agentId),
|
||||
eq(heartbeatRuns.nativeIssueId, this.#binding.issueId),
|
||||
eq(heartbeatRuns.runtimeMode, "native"),
|
||||
),
|
||||
)
|
||||
.limit(1);
|
||||
if (!row) throw new Error("paperclip_runner_semantic_binding_not_found");
|
||||
return row;
|
||||
}
|
||||
|
||||
#assertActiveContext(context: BoundContext, requireOwnership: boolean): void {
|
||||
if (
|
||||
!["queued", "running"].includes(context.run.status) ||
|
||||
activeAgentStatus(context.agent.status) !== "active" ||
|
||||
(requireOwnership &&
|
||||
(context.issue.assigneeAgentId !== this.#binding.agentId ||
|
||||
context.issue.executionRunId !== this.#binding.runId))
|
||||
) {
|
||||
throw new Error("paperclip_runner_semantic_binding_inactive");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
/**
|
||||
* Development shim for the package-local runner runtime.
|
||||
*
|
||||
* The server build replaces this emitted module with the runner package's
|
||||
* compiled `dist` tree so published server packages have no workspace runtime
|
||||
* dependency. Keep server imports pointed at this relative boundary.
|
||||
*/
|
||||
export * from "@paperclipai/paperclip-runner";
|
||||
|
|
@ -1,6 +1,18 @@
|
|||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { defineConfig } from "vitest/config";
|
||||
|
||||
export default defineConfig({
|
||||
resolve: {
|
||||
alias: [
|
||||
{
|
||||
find: /^@paperclipai\/paperclip-runner$/,
|
||||
replacement: fileURLToPath(
|
||||
new URL("../packages/paperclip-runner/src/index.ts", import.meta.url),
|
||||
),
|
||||
},
|
||||
],
|
||||
},
|
||||
test: {
|
||||
environment: "node",
|
||||
include: ["src/**/*.test.ts"],
|
||||
|
|
|
|||
Loading…
Reference in New Issue