feat(deploy): Cloudflare Workers + Sandbox container deployment path
Adds deploy/cloudflare: a Worker that serves a full Paperclip instance from a Cloudflare Sandbox container on the Worker's own origin (HTTP + WebSockets via containerFetch/wsConnect — no custom domain required), optional credential-less R2 FUSE mount for durable attachment storage, docs, unit tests, and cross-file config invariants. Claude-Session: https://claude.ai/code/session_01NWpwXBqxrPZdv75ud8xszr
This commit is contained in:
Daniel Bodnar
parent
8a3cc86531
commit
406d3a720f
|
|
@ -0,0 +1,5 @@
|
|||
# Wrangler local state (miniflare D1/KV/DO/R2 sqlite files) — never commit.
|
||||
.wrangler/
|
||||
# Local secrets for `wrangler dev`.
|
||||
.dev.vars
|
||||
node_modules/
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
# Paperclip on Cloudflare
|
||||
|
||||
Deploys a full Paperclip instance to Cloudflare Workers: the Worker proxies
|
||||
your `*.workers.dev` origin (HTTP + WebSockets) into a
|
||||
[Cloudflare Sandbox](https://developers.cloudflare.com/sandbox/) container
|
||||
running `paperclipai` with embedded Postgres and the local-adapter agent CLIs
|
||||
preinstalled. No custom domain required.
|
||||
|
||||
```sh
|
||||
pnpm install --ignore-workspace
|
||||
npx wrangler login
|
||||
npx wrangler deploy
|
||||
```
|
||||
|
||||
> ⚠️ Container disk is **ephemeral** — for any data you want to keep, set an
|
||||
> external database first (`npx wrangler secret put DATABASE_URL`) and
|
||||
> optionally enable the R2 attachments mount (see `wrangler.jsonc`).
|
||||
|
||||
Full operator guide (prerequisites, configuration, durability, costs,
|
||||
troubleshooting): **[docs/deploy/cloudflare.md](../../docs/deploy/cloudflare.md)**.
|
||||
|
||||
## Layout
|
||||
|
||||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `src/index.ts` | Worker: boots Paperclip in the sandbox, proxies HTTP + WS |
|
||||
| `src/lib.ts` | Pure helpers (unit-tested) |
|
||||
| `container/Dockerfile` | Sandbox image: paperclipai + agent CLIs |
|
||||
| `container/start-paperclip.sh` | Onboard-once boot script (non-root) |
|
||||
| `wrangler.jsonc` | Worker + container + Durable Object config |
|
||||
| `test/` | Unit tests + cross-file config consistency checks |
|
||||
|
||||
This package is intentionally **not** part of the pnpm workspace (same
|
||||
pattern as `packages/plugins/sandbox-providers/*`) so its Cloudflare
|
||||
toolchain never churns the root lockfile. Install with
|
||||
`pnpm install --ignore-workspace`.
|
||||
|
||||
```sh
|
||||
pnpm test # vitest: lib + config invariants (Dockerfile↔SDK version pin)
|
||||
pnpm typecheck
|
||||
```
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# Paperclip on Cloudflare Sandbox containers.
|
||||
#
|
||||
# The base image tag MUST match the @cloudflare/sandbox version pinned in
|
||||
# ../package.json — the SDK and the in-container runtime are versioned
|
||||
# together. test/config.test.ts enforces this.
|
||||
FROM docker.io/cloudflare/sandbox:0.12.4
|
||||
|
||||
# Quality-of-life tools for agent workloads (subset of the tools the root
|
||||
# Dockerfile's production stage installs; the sandbox base already ships
|
||||
# git, curl, python3 and node).
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends jq ripgrep openssh-client \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Paperclip (npm release) plus the local-adapter agent CLIs, mirroring the
|
||||
# root Dockerfile's production stage.
|
||||
RUN npm install -g \
|
||||
paperclipai@latest \
|
||||
@anthropic-ai/claude-code@latest \
|
||||
@openai/codex@latest \
|
||||
opencode-ai \
|
||||
@google/gemini-cli@latest
|
||||
|
||||
# Paperclip runtime defaults (mirrors the root Dockerfile / quadlet unit).
|
||||
ENV HOST=0.0.0.0 \
|
||||
PORT=3100 \
|
||||
PAPERCLIP_HOME=/paperclip \
|
||||
PAPERCLIP_DEPLOYMENT_MODE=authenticated \
|
||||
PAPERCLIP_DEPLOYMENT_EXPOSURE=private
|
||||
|
||||
# Embedded Postgres refuses to run as root, so Paperclip gets its own user.
|
||||
# The uid is pinned (must match PAPERCLIP_UID in src/lib.ts — enforced by
|
||||
# test/config.test.ts) so the optional R2 attachments mount can be exposed
|
||||
# as owned by this user. embedded-postgres also creates symlinks inside the
|
||||
# package directory on first boot, hence the chown of the installed package.
|
||||
RUN useradd -m -u 4100 -s /bin/bash paperclip \
|
||||
&& mkdir -p /paperclip \
|
||||
&& chown -R paperclip:paperclip /paperclip \
|
||||
&& chown -R paperclip:paperclip /usr/local/lib/node_modules/paperclipai
|
||||
|
||||
COPY start-paperclip.sh /opt/start-paperclip.sh
|
||||
RUN chmod +x /opt/start-paperclip.sh
|
||||
|
||||
# Local dev (wrangler dev) requires exposed ports to be declared.
|
||||
EXPOSE 3100
|
||||
|
||||
# No USER directive on purpose (trivy DS-0002): the Cloudflare Sandbox
|
||||
# runtime daemon in the base image must run as root; the Paperclip workload
|
||||
# itself drops to the non-root `paperclip` user in start-paperclip.sh.
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
#!/bin/bash
|
||||
# Boots Paperclip as the non-root `paperclip` user (embedded Postgres refuses
|
||||
# to run as root). Onboards non-interactively on first boot, then runs the
|
||||
# server. Started by the Worker via sandbox.startProcess() with the runtime
|
||||
# environment (PAPERCLIP_*, optional ANTHROPIC_API_KEY / DATABASE_URL) —
|
||||
# runuser without --login preserves that environment for the child shell.
|
||||
set -euo pipefail
|
||||
|
||||
# Own the data dir, but skip the (optional) R2-mounted storage directory:
|
||||
# s3fs rejects chown, which would abort the boot under `set -e`. The mount
|
||||
# is already exposed with the paperclip uid via s3fs options (src/lib.ts).
|
||||
find /paperclip -path /paperclip/instances/default/data/storage -prune \
|
||||
-o -exec chown paperclip:paperclip {} +
|
||||
|
||||
exec runuser -u paperclip -- bash -c '
|
||||
set -euo pipefail
|
||||
export HOME=/home/paperclip
|
||||
if [ ! -f /paperclip/instances/default/config.json ]; then
|
||||
paperclipai onboard --yes --bind lan --data-dir /paperclip
|
||||
fi
|
||||
exec paperclipai run --data-dir /paperclip
|
||||
'
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
{
|
||||
"name": "@paperclipai/deploy-cloudflare",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"description": "Deploy Paperclip to Cloudflare Workers with a Sandbox container",
|
||||
"license": "MIT",
|
||||
"homepage": "https://github.com/paperclipai/paperclip",
|
||||
"bugs": {
|
||||
"url": "https://github.com/paperclipai/paperclip/issues"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/paperclipai/paperclip",
|
||||
"directory": "deploy/cloudflare"
|
||||
},
|
||||
"scripts": {
|
||||
"dev": "wrangler dev",
|
||||
"deploy": "wrangler deploy",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"dependencies": {
|
||||
"@cloudflare/sandbox": "0.12.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cloudflare/workers-types": "^4.20260701.0",
|
||||
"@types/node": "^22.10.0",
|
||||
"typescript": "^5.7.3",
|
||||
"vitest": "^4.1.10",
|
||||
"wrangler": "^4.113.0"
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,11 @@
|
|||
# Marks deploy/cloudflare as its own single-package workspace so pnpm does
|
||||
# not attach it to the repo root workspace (same isolation rationale as
|
||||
# packages/plugins/sandbox-providers/*): the Cloudflare toolchain
|
||||
# (wrangler/workerd) never churns the root pnpm-lock.yaml.
|
||||
# sharp stays false: it is an optional wrangler dependency (static-asset
|
||||
# image processing) with no prebuilt binary on some platforms, and its
|
||||
# source build fails without node-addon-api. Wrangler works without it.
|
||||
allowBuilds:
|
||||
esbuild: true
|
||||
sharp: false
|
||||
workerd: true
|
||||
|
|
@ -0,0 +1,130 @@
|
|||
/**
|
||||
* Cloudflare Worker that serves a full Paperclip instance from a Cloudflare
|
||||
* Sandbox container on the Worker's own origin (works on *.workers.dev — no
|
||||
* custom domain required).
|
||||
*
|
||||
* Request flow:
|
||||
* 1. Ensure the Paperclip boot process is running in the sandbox
|
||||
* (idempotent; memoized per isolate, re-checked after any failure).
|
||||
* 2. WebSocket upgrades -> sandbox.wsConnect(request, 3100)
|
||||
* Everything else -> sandbox.containerFetch(request, 3100)
|
||||
* 3. While the container provisions / Paperclip onboards, serve a
|
||||
* self-refreshing 503 status page instead of a raw error.
|
||||
*
|
||||
* See docs/deploy/cloudflare.md for the operator guide.
|
||||
*/
|
||||
import { getSandbox, type Sandbox as SandboxType } from "@cloudflare/sandbox";
|
||||
import {
|
||||
ARTIFACTS_BINDING,
|
||||
PAPERCLIP_PORT,
|
||||
SANDBOX_ID,
|
||||
START_COMMAND,
|
||||
STORAGE_MOUNT_PATH,
|
||||
bootingResponse,
|
||||
buildPaperclipEnv,
|
||||
isMountAlreadyInUse,
|
||||
isPaperclipRunning,
|
||||
isTransientBootError,
|
||||
isTransientBootMessage,
|
||||
isWebSocketUpgrade,
|
||||
storageMountOptions,
|
||||
} from "./lib";
|
||||
|
||||
// Required by the Sandbox SDK: the Durable Object class backing the container,
|
||||
// and the ContainerProxy entrypoint used for credential-less R2 bucket mounts
|
||||
// (harmless when no bucket is configured).
|
||||
export { ContainerProxy, Sandbox } from "@cloudflare/sandbox";
|
||||
|
||||
interface Env {
|
||||
Sandbox: DurableObjectNamespace<SandboxType>;
|
||||
/**
|
||||
* Optional R2 bucket for durable attachment storage — uncomment the
|
||||
* r2_buckets block in wrangler.jsonc to enable.
|
||||
*/
|
||||
ARTIFACTS?: R2Bucket;
|
||||
/** Optional override; defaults to the request origin (e.g. *.workers.dev). */
|
||||
PAPERCLIP_PUBLIC_URL?: string;
|
||||
PAPERCLIP_DEPLOYMENT_MODE?: string;
|
||||
PAPERCLIP_DEPLOYMENT_EXPOSURE?: string;
|
||||
/** Secrets (wrangler secret put …); forwarded to the container when set. */
|
||||
ANTHROPIC_API_KEY?: string;
|
||||
DATABASE_URL?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-isolate memo so steady-state requests skip the listProcesses round
|
||||
* trip. Reset whenever proxying fails, which also heals container restarts
|
||||
* (the boot process does not survive a sandbox sleep/wake cycle).
|
||||
*/
|
||||
let paperclipEnsured = false;
|
||||
|
||||
async function ensurePaperclip(sandbox: SandboxType, env: Env, requestUrl: URL): Promise<void> {
|
||||
const processes = await sandbox.listProcesses();
|
||||
if (isPaperclipRunning(processes)) return;
|
||||
|
||||
// Mount durable attachment storage before Paperclip boots so the very
|
||||
// first upload already lands in R2. Credential-less: the SDK routes s3fs
|
||||
// traffic through the Worker's R2 binding (requires the ContainerProxy
|
||||
// export above).
|
||||
if (env[ARTIFACTS_BINDING]) {
|
||||
try {
|
||||
await sandbox.mountBucket(ARTIFACTS_BINDING, STORAGE_MOUNT_PATH, storageMountOptions());
|
||||
} catch (error) {
|
||||
if (!isMountAlreadyInUse(error)) throw error;
|
||||
}
|
||||
}
|
||||
|
||||
await sandbox.startProcess(START_COMMAND, {
|
||||
env: buildPaperclipEnv({
|
||||
publicUrl: env.PAPERCLIP_PUBLIC_URL || `https://${requestUrl.host}`,
|
||||
deploymentMode: env.PAPERCLIP_DEPLOYMENT_MODE,
|
||||
deploymentExposure: env.PAPERCLIP_DEPLOYMENT_EXPOSURE,
|
||||
anthropicApiKey: env.ANTHROPIC_API_KEY,
|
||||
databaseUrl: env.DATABASE_URL,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Distinguish the Sandbox SDK's "still starting" 5xx responses from genuine
|
||||
* Paperclip errors so operators see the status page, not a JSON stack trace.
|
||||
*/
|
||||
async function isProvisioningResponse(response: Response): Promise<boolean> {
|
||||
if (response.status < 500) return false;
|
||||
if (!response.headers.get("content-type")?.includes("json")) return false;
|
||||
const text = await response.clone().text().catch(() => "");
|
||||
return isTransientBootMessage(text);
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
const sandbox = getSandbox(env.Sandbox, SANDBOX_ID);
|
||||
const url = new URL(request.url);
|
||||
|
||||
try {
|
||||
if (!paperclipEnsured) {
|
||||
await ensurePaperclip(sandbox, env, url);
|
||||
paperclipEnsured = true;
|
||||
}
|
||||
|
||||
if (isWebSocketUpgrade(request.headers)) {
|
||||
return await sandbox.wsConnect(request, PAPERCLIP_PORT);
|
||||
}
|
||||
|
||||
const response = await sandbox.containerFetch(request, PAPERCLIP_PORT);
|
||||
if (await isProvisioningResponse(response)) {
|
||||
paperclipEnsured = false;
|
||||
return bootingResponse();
|
||||
}
|
||||
return response;
|
||||
} catch (error) {
|
||||
paperclipEnsured = false;
|
||||
if (isTransientBootError(error)) {
|
||||
return isWebSocketUpgrade(request.headers)
|
||||
? new Response("Paperclip is starting; retry shortly", { status: 503 })
|
||||
: bootingResponse();
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
} satisfies ExportedHandler<Env>;
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
/**
|
||||
* Pure helpers for the Cloudflare Sandbox deployment Worker.
|
||||
*
|
||||
* Everything here is side-effect free so it can be unit tested without a
|
||||
* Workers runtime (see ../test/lib.test.ts).
|
||||
*/
|
||||
|
||||
/** Port the Paperclip server listens on inside the sandbox container. */
|
||||
export const PAPERCLIP_PORT = 3100;
|
||||
|
||||
/**
|
||||
* Stable sandbox id. One deployment == one Paperclip control plane, so a
|
||||
* fixed id always routes to the same Durable Object / container.
|
||||
*/
|
||||
export const SANDBOX_ID = "paperclip";
|
||||
|
||||
/** Boot script baked into the container image (container/Dockerfile). */
|
||||
export const START_COMMAND = "/opt/start-paperclip.sh";
|
||||
|
||||
/**
|
||||
* Optional R2 binding name for durable attachment storage. When the binding
|
||||
* exists, the Worker FUSE-mounts the bucket (credential-less, via the SDK's
|
||||
* egress interception) at Paperclip's local-disk storage directory before
|
||||
* boot, so uploaded files survive container recycling.
|
||||
*/
|
||||
export const ARTIFACTS_BINDING = "ARTIFACTS";
|
||||
|
||||
/**
|
||||
* Paperclip's local_disk storage provider path (docs/deploy/storage.md)
|
||||
* under PAPERCLIP_HOME=/paperclip. Only file uploads live here — never the
|
||||
* Postgres data directory, which must not sit on a FUSE mount.
|
||||
*/
|
||||
export const STORAGE_MOUNT_PATH = "/paperclip/instances/default/data/storage";
|
||||
|
||||
/**
|
||||
* Fixed uid/gid of the non-root `paperclip` user created in
|
||||
* container/Dockerfile (useradd -u). Pinned so the s3fs mount can be owned
|
||||
* by that user; test/config.test.ts enforces the pin matches the Dockerfile.
|
||||
*/
|
||||
export const PAPERCLIP_UID = 4100;
|
||||
|
||||
/**
|
||||
* s3fs options for the attachments mount: expose it as owned by the
|
||||
* `paperclip` user (s3fs mounts as root and rejects chown) and allow other
|
||||
* users to traverse it. The SDK's R2 defaults are applied on top.
|
||||
*/
|
||||
export function storageMountOptions(): { s3fsOptions: string[] } {
|
||||
return {
|
||||
s3fsOptions: [
|
||||
"allow_other",
|
||||
`uid=${PAPERCLIP_UID}`,
|
||||
`gid=${PAPERCLIP_UID}`,
|
||||
"umask=0022",
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
/** Benign when two isolates race to mount the same path — first one wins. */
|
||||
export function isMountAlreadyInUse(error: unknown): boolean {
|
||||
return error instanceof Error && /already in use/i.test(error.message);
|
||||
}
|
||||
|
||||
/** Process states that mean "no longer serving" (safe to start a new one). */
|
||||
const DEAD_STATUSES = new Set(["completed", "failed", "killed", "stopped"]);
|
||||
|
||||
export interface ProcessLike {
|
||||
command?: string;
|
||||
status?: string;
|
||||
}
|
||||
|
||||
/** True when the request is a WebSocket upgrade that must not be buffered. */
|
||||
export function isWebSocketUpgrade(headers: Headers): boolean {
|
||||
return headers.get("Upgrade")?.toLowerCase() === "websocket";
|
||||
}
|
||||
|
||||
/** True when a live Paperclip boot process already exists in the sandbox. */
|
||||
export function isPaperclipRunning(processes: ProcessLike[]): boolean {
|
||||
return processes.some(
|
||||
(p) => (p.command ?? "").includes(START_COMMAND) && !DEAD_STATUSES.has(p.status ?? "")
|
||||
);
|
||||
}
|
||||
|
||||
export interface PaperclipEnvOptions {
|
||||
/** Public origin the instance is reachable at, e.g. https://x.workers.dev */
|
||||
publicUrl: string;
|
||||
deploymentMode?: string;
|
||||
deploymentExposure?: string;
|
||||
anthropicApiKey?: string;
|
||||
databaseUrl?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Environment passed to the Paperclip boot process. Secrets are only
|
||||
* forwarded when actually configured so the container env stays minimal.
|
||||
*/
|
||||
export function buildPaperclipEnv(options: PaperclipEnvOptions): Record<string, string> {
|
||||
const env: Record<string, string> = {
|
||||
HOST: "0.0.0.0",
|
||||
PORT: String(PAPERCLIP_PORT),
|
||||
PAPERCLIP_HOME: "/paperclip",
|
||||
PAPERCLIP_DEPLOYMENT_MODE: options.deploymentMode ?? "authenticated",
|
||||
PAPERCLIP_DEPLOYMENT_EXPOSURE: options.deploymentExposure ?? "private",
|
||||
PAPERCLIP_PUBLIC_URL: options.publicUrl,
|
||||
};
|
||||
if (options.anthropicApiKey) env.ANTHROPIC_API_KEY = options.anthropicApiKey;
|
||||
if (options.databaseUrl) env.DATABASE_URL = options.databaseUrl;
|
||||
return env;
|
||||
}
|
||||
|
||||
/**
|
||||
* Matches the Sandbox SDK's own transient startup errors: the container VM is
|
||||
* still provisioning, or the port is not accepting connections yet (Paperclip
|
||||
* onboards its database on first boot, which takes a minute or two).
|
||||
* Deliberately specific to SDK wording so genuine Paperclip 5xx responses are
|
||||
* never mistaken for boot noise.
|
||||
*/
|
||||
const TRANSIENT_BOOT_PATTERNS = [
|
||||
/currently provisioning/i,
|
||||
/no container instance/i,
|
||||
/container.*(?:not running|starting|is starting)/i,
|
||||
/connection refused/i,
|
||||
/port.*not (?:ready|mapped|found)/i,
|
||||
/network connection lost/i,
|
||||
/timed out.*(?:port|start|container)/i,
|
||||
];
|
||||
|
||||
export function isTransientBootMessage(message: string): boolean {
|
||||
return TRANSIENT_BOOT_PATTERNS.some((pattern) => pattern.test(message));
|
||||
}
|
||||
|
||||
export function isTransientBootError(error: unknown): boolean {
|
||||
return error instanceof Error && isTransientBootMessage(error.message);
|
||||
}
|
||||
|
||||
/**
|
||||
* Self-refreshing status page served while the container provisions and
|
||||
* Paperclip onboards. Inline styles only — nothing else is reachable yet.
|
||||
*/
|
||||
export function bootingPage(): string {
|
||||
return `<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1"/>
|
||||
<meta http-equiv="refresh" content="10"/>
|
||||
<title>Paperclip is starting…</title>
|
||||
<style>
|
||||
:root{color-scheme:light dark}
|
||||
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;
|
||||
font-family:ui-sans-serif,system-ui,sans-serif;background:#0d1017;color:#e6e6e6}
|
||||
main{text-align:center;padding:2rem}
|
||||
.spinner{width:28px;height:28px;margin:0 auto 1.25rem;border-radius:50%;
|
||||
border:3px solid #232838;border-top-color:#7dd3fc;animation:spin 1s linear infinite}
|
||||
h1{font-size:1.1rem;font-weight:600;margin:0 0 .5rem}
|
||||
p{margin:.25rem 0;color:#9aa4bf;font-size:.9rem}
|
||||
@keyframes spin{to{transform:rotate(360deg)}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<div class="spinner" role="status" aria-label="loading"></div>
|
||||
<h1>Paperclip is starting</h1>
|
||||
<p>The sandbox container is provisioning and Paperclip is onboarding its database.</p>
|
||||
<p>First boot takes a minute or two. This page refreshes automatically.</p>
|
||||
</main>
|
||||
</body>
|
||||
</html>`;
|
||||
}
|
||||
|
||||
/** 503 + Retry-After so health checkers and browsers both behave. */
|
||||
export function bootingResponse(): Response {
|
||||
return new Response(bootingPage(), {
|
||||
status: 503,
|
||||
headers: {
|
||||
"content-type": "text/html; charset=utf-8",
|
||||
"retry-after": "15",
|
||||
"cache-control": "no-store",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
/**
|
||||
* Consistency checks across wrangler.jsonc, the container Dockerfile and
|
||||
* package.json. These encode the deployment's cross-file invariants — most
|
||||
* importantly the Sandbox SDK requirement that the base image tag match the
|
||||
* @cloudflare/sandbox package version exactly.
|
||||
*/
|
||||
import { readFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { PAPERCLIP_PORT, PAPERCLIP_UID, STORAGE_MOUNT_PATH } from "../src/lib";
|
||||
|
||||
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
|
||||
function read(path: string): string {
|
||||
return readFileSync(join(root, path), "utf8");
|
||||
}
|
||||
|
||||
/** Minimal JSONC parser: strips // and /* *\/ comments outside strings. */
|
||||
function parseJsonc(text: string): any {
|
||||
const stripped = text.replace(
|
||||
/"(?:[^"\\]|\\.)*"|\/\/[^\n]*|\/\*[\s\S]*?\*\//g,
|
||||
(match) => (match.startsWith('"') ? match : "")
|
||||
);
|
||||
return JSON.parse(stripped);
|
||||
}
|
||||
|
||||
const wrangler = parseJsonc(read("wrangler.jsonc"));
|
||||
const pkg = JSON.parse(read("package.json"));
|
||||
const dockerfile = read("container/Dockerfile");
|
||||
|
||||
describe("wrangler.jsonc", () => {
|
||||
it("wires the Sandbox container, DO binding and migration to one class", () => {
|
||||
const containerClass = wrangler.containers[0].class_name;
|
||||
expect(containerClass).toBe("Sandbox");
|
||||
expect(wrangler.durable_objects.bindings[0].class_name).toBe(containerClass);
|
||||
expect(wrangler.migrations[0].new_sqlite_classes).toContain(containerClass);
|
||||
});
|
||||
|
||||
it("uses the container Dockerfile as the image", () => {
|
||||
expect(wrangler.containers[0].image).toBe("./container/Dockerfile");
|
||||
});
|
||||
|
||||
it("enables nodejs_compat (required by the Sandbox SDK)", () => {
|
||||
expect(wrangler.compatibility_flags).toContain("nodejs_compat");
|
||||
});
|
||||
|
||||
it("carries no account- or zone-specific configuration", () => {
|
||||
const raw = read("wrangler.jsonc");
|
||||
expect(wrangler.account_id).toBeUndefined();
|
||||
expect(wrangler.routes).toBeUndefined();
|
||||
expect(raw).not.toMatch(/account_id/);
|
||||
});
|
||||
|
||||
it("defaults to private exposure (embedded Postgres requirement)", () => {
|
||||
expect(wrangler.vars.PAPERCLIP_DEPLOYMENT_EXPOSURE).toBe("private");
|
||||
});
|
||||
});
|
||||
|
||||
describe("container image", () => {
|
||||
it("pins the base image to the exact @cloudflare/sandbox version", () => {
|
||||
const sdkVersion = pkg.dependencies["@cloudflare/sandbox"];
|
||||
// Must be an exact pin — the SDK and in-container runtime version together.
|
||||
expect(sdkVersion).toMatch(/^\d+\.\d+\.\d+$/);
|
||||
expect(dockerfile).toContain(`FROM docker.io/cloudflare/sandbox:${sdkVersion}`);
|
||||
});
|
||||
|
||||
it("exposes the Paperclip port for local dev", () => {
|
||||
expect(dockerfile).toContain(`EXPOSE ${PAPERCLIP_PORT}`);
|
||||
});
|
||||
|
||||
it("installs Paperclip and the agent CLIs", () => {
|
||||
expect(dockerfile).toContain("paperclipai@latest");
|
||||
expect(dockerfile).toContain("@anthropic-ai/claude-code");
|
||||
});
|
||||
|
||||
it("pins the paperclip uid the R2 mount options rely on", () => {
|
||||
expect(dockerfile).toContain(`useradd -m -u ${PAPERCLIP_UID} `);
|
||||
});
|
||||
});
|
||||
|
||||
describe("boot script", () => {
|
||||
const script = read("container/start-paperclip.sh");
|
||||
|
||||
it("excludes the R2 storage mount from the ownership pass", () => {
|
||||
// s3fs rejects chown; a bare `chown -R /paperclip` would abort the boot.
|
||||
expect(script).toContain(`-path ${STORAGE_MOUNT_PATH} -prune`);
|
||||
expect(script).not.toMatch(/chown -R paperclip:paperclip \/paperclip\s*$/m);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,154 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
PAPERCLIP_PORT,
|
||||
PAPERCLIP_UID,
|
||||
START_COMMAND,
|
||||
STORAGE_MOUNT_PATH,
|
||||
bootingPage,
|
||||
bootingResponse,
|
||||
buildPaperclipEnv,
|
||||
isMountAlreadyInUse,
|
||||
isPaperclipRunning,
|
||||
isTransientBootError,
|
||||
isTransientBootMessage,
|
||||
isWebSocketUpgrade,
|
||||
storageMountOptions,
|
||||
} from "../src/lib";
|
||||
|
||||
describe("isWebSocketUpgrade", () => {
|
||||
it("detects a standard upgrade request", () => {
|
||||
const headers = new Headers({ Upgrade: "websocket", Connection: "Upgrade" });
|
||||
expect(isWebSocketUpgrade(headers)).toBe(true);
|
||||
});
|
||||
|
||||
it("is case-insensitive", () => {
|
||||
expect(isWebSocketUpgrade(new Headers({ Upgrade: "WebSocket" }))).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects plain requests and non-websocket upgrades", () => {
|
||||
expect(isWebSocketUpgrade(new Headers())).toBe(false);
|
||||
expect(isWebSocketUpgrade(new Headers({ Upgrade: "h2c" }))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("isPaperclipRunning", () => {
|
||||
it("finds a live boot process", () => {
|
||||
expect(
|
||||
isPaperclipRunning([{ command: START_COMMAND, status: "running" }])
|
||||
).toBe(true);
|
||||
expect(
|
||||
isPaperclipRunning([{ command: `bash ${START_COMMAND}`, status: "starting" }])
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("ignores dead processes so a restart can happen", () => {
|
||||
for (const status of ["completed", "failed", "killed", "stopped"]) {
|
||||
expect(isPaperclipRunning([{ command: START_COMMAND, status }])).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("ignores unrelated processes and empty lists", () => {
|
||||
expect(isPaperclipRunning([])).toBe(false);
|
||||
expect(isPaperclipRunning([{ command: "sleep 1", status: "running" }])).toBe(false);
|
||||
expect(isPaperclipRunning([{}])).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildPaperclipEnv", () => {
|
||||
it("produces the baseline environment", () => {
|
||||
const env = buildPaperclipEnv({ publicUrl: "https://example.workers.dev" });
|
||||
expect(env).toEqual({
|
||||
HOST: "0.0.0.0",
|
||||
PORT: String(PAPERCLIP_PORT),
|
||||
PAPERCLIP_HOME: "/paperclip",
|
||||
PAPERCLIP_DEPLOYMENT_MODE: "authenticated",
|
||||
PAPERCLIP_DEPLOYMENT_EXPOSURE: "private",
|
||||
PAPERCLIP_PUBLIC_URL: "https://example.workers.dev",
|
||||
});
|
||||
});
|
||||
|
||||
it("only forwards secrets that are actually set", () => {
|
||||
const bare = buildPaperclipEnv({ publicUrl: "https://x.dev" });
|
||||
expect(bare).not.toHaveProperty("ANTHROPIC_API_KEY");
|
||||
expect(bare).not.toHaveProperty("DATABASE_URL");
|
||||
|
||||
const withSecrets = buildPaperclipEnv({
|
||||
publicUrl: "https://x.dev",
|
||||
anthropicApiKey: "test-key",
|
||||
databaseUrl: "postgres://example",
|
||||
});
|
||||
expect(withSecrets.ANTHROPIC_API_KEY).toBe("test-key");
|
||||
expect(withSecrets.DATABASE_URL).toBe("postgres://example");
|
||||
});
|
||||
|
||||
it("honors mode/exposure overrides", () => {
|
||||
const env = buildPaperclipEnv({
|
||||
publicUrl: "https://x.dev",
|
||||
deploymentMode: "authenticated",
|
||||
deploymentExposure: "public",
|
||||
});
|
||||
expect(env.PAPERCLIP_DEPLOYMENT_EXPOSURE).toBe("public");
|
||||
});
|
||||
});
|
||||
|
||||
describe("transient boot detection", () => {
|
||||
it("matches the Sandbox SDK's startup errors", () => {
|
||||
for (const message of [
|
||||
"Container is currently provisioning. This can take several minutes on first deployment.",
|
||||
"no container instance available",
|
||||
"connection refused",
|
||||
"Port 3100 not ready",
|
||||
"Network connection lost",
|
||||
]) {
|
||||
expect(isTransientBootMessage(message), message).toBe(true);
|
||||
expect(isTransientBootError(new Error(message)), message).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("does not swallow genuine application errors", () => {
|
||||
for (const message of [
|
||||
"Internal Server Error",
|
||||
"database migration failed",
|
||||
"TypeError: cannot read properties of undefined",
|
||||
]) {
|
||||
expect(isTransientBootMessage(message), message).toBe(false);
|
||||
}
|
||||
expect(isTransientBootError("connection refused")).toBe(false); // non-Error
|
||||
});
|
||||
});
|
||||
|
||||
describe("storage mount", () => {
|
||||
it("targets Paperclip's local_disk storage path, never the DB dir", () => {
|
||||
expect(STORAGE_MOUNT_PATH).toBe("/paperclip/instances/default/data/storage");
|
||||
expect(STORAGE_MOUNT_PATH).not.toContain("postgres");
|
||||
});
|
||||
|
||||
it("exposes the mount as the paperclip user", () => {
|
||||
const { s3fsOptions } = storageMountOptions();
|
||||
expect(s3fsOptions).toContain(`uid=${PAPERCLIP_UID}`);
|
||||
expect(s3fsOptions).toContain(`gid=${PAPERCLIP_UID}`);
|
||||
expect(s3fsOptions).toContain("allow_other");
|
||||
});
|
||||
|
||||
it("recognizes the benign already-mounted race", () => {
|
||||
expect(isMountAlreadyInUse(new Error("Mount path already in use: /x"))).toBe(true);
|
||||
expect(isMountAlreadyInUse(new Error("S3FS mount command failed"))).toBe(false);
|
||||
expect(isMountAlreadyInUse("already in use")).toBe(false); // non-Error
|
||||
});
|
||||
});
|
||||
|
||||
describe("booting page", () => {
|
||||
it("self-refreshes and explains what is happening", () => {
|
||||
const html = bootingPage();
|
||||
expect(html).toContain('http-equiv="refresh"');
|
||||
expect(html).toContain("Paperclip is starting");
|
||||
});
|
||||
|
||||
it("responds 503 with Retry-After and no caching", () => {
|
||||
const response = bootingResponse();
|
||||
expect(response.status).toBe(503);
|
||||
expect(response.headers.get("retry-after")).toBe("15");
|
||||
expect(response.headers.get("cache-control")).toBe("no-store");
|
||||
expect(response.headers.get("content-type")).toContain("text/html");
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
{
|
||||
"compilerOptions": {
|
||||
"target": "es2022",
|
||||
"lib": ["es2022"],
|
||||
"module": "es2022",
|
||||
"moduleResolution": "bundler",
|
||||
"types": ["@cloudflare/workers-types", "node"],
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"isolatedModules": true,
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true
|
||||
},
|
||||
"include": ["src", "test", "vitest.config.ts"]
|
||||
}
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
import { defineConfig } from "vitest/config";
|
||||
|
||||
export default defineConfig({
|
||||
test: {
|
||||
environment: "node",
|
||||
include: ["test/**/*.test.ts"],
|
||||
},
|
||||
});
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
{
|
||||
"$schema": "node_modules/wrangler/config-schema.json",
|
||||
"name": "paperclip-sandbox",
|
||||
"main": "src/index.ts",
|
||||
"compatibility_date": "2026-07-01",
|
||||
"compatibility_flags": ["nodejs_compat"],
|
||||
"containers": [
|
||||
{
|
||||
"class_name": "Sandbox",
|
||||
"image": "./container/Dockerfile",
|
||||
// Paperclip control plane + embedded Postgres + agent CLI processes.
|
||||
// Shrink at your own risk; embedded Postgres alone wants real memory.
|
||||
"instance_type": { "vcpu": 2, "memory_mib": 8192, "disk_mb": 10240 },
|
||||
// One control plane per deployment — a fixed sandbox id routes every
|
||||
// request to the same instance, so extra instances would sit idle.
|
||||
"max_instances": 1,
|
||||
"name": "paperclip-sandbox"
|
||||
}
|
||||
],
|
||||
"durable_objects": {
|
||||
"bindings": [{ "class_name": "Sandbox", "name": "Sandbox" }]
|
||||
},
|
||||
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["Sandbox"] }],
|
||||
// Optional: durable attachment storage. Paperclip's upload directory is
|
||||
// FUSE-mounted from this bucket (credential-less, via the Worker binding),
|
||||
// so attachments survive container recycling. To enable:
|
||||
// 1. npx wrangler r2 bucket create paperclip-attachments
|
||||
// 2. Uncomment the block below and redeploy.
|
||||
// See docs/deploy/cloudflare.md#durable-attachments-via-r2-optional.
|
||||
// "r2_buckets": [
|
||||
// { "binding": "ARTIFACTS", "bucket_name": "paperclip-attachments" }
|
||||
// ],
|
||||
"vars": {
|
||||
// Defaults follow docs/deploy/deployment-modes.md. Embedded Postgres
|
||||
// currently requires "private" exposure; switch to "public" only with an
|
||||
// external DATABASE_URL (see docs/deploy/cloudflare.md#data-durability).
|
||||
"PAPERCLIP_DEPLOYMENT_MODE": "authenticated",
|
||||
"PAPERCLIP_DEPLOYMENT_EXPOSURE": "private",
|
||||
// Leave empty to default to the request origin (your workers.dev URL).
|
||||
"PAPERCLIP_PUBLIC_URL": ""
|
||||
},
|
||||
"observability": { "enabled": true }
|
||||
}
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
---
|
||||
title: Cloudflare
|
||||
summary: Run Paperclip on Cloudflare Workers with a Sandbox container
|
||||
---
|
||||
|
||||
Deploy a full Paperclip instance to Cloudflare: a Worker proxies your
|
||||
`*.workers.dev` origin into a [Cloudflare Sandbox](https://developers.cloudflare.com/sandbox/)
|
||||
container that runs `paperclipai` (npm release) with embedded Postgres and the
|
||||
local-adapter agent CLIs (Claude Code, Codex, OpenCode, Gemini) preinstalled.
|
||||
|
||||
No custom domain is required — HTTP and WebSocket traffic are proxied on the
|
||||
Worker's own origin.
|
||||
|
||||
## ⚠️ Data Durability
|
||||
|
||||
**Sandbox container disk is ephemeral.** When the container sleeps after
|
||||
inactivity or is replaced by a deploy, the embedded Postgres data directory is
|
||||
wiped — companies, agents, issues, everything.
|
||||
|
||||
Treat the default configuration as an **evaluation deployment**. For anything
|
||||
you want to keep, point Paperclip at an external Postgres before onboarding:
|
||||
|
||||
```sh
|
||||
npx wrangler secret put DATABASE_URL
|
||||
# e.g. postgres://USER:PASSWORD@HOST:5432/paperclip (Neon, Supabase, RDS, …)
|
||||
```
|
||||
|
||||
The Worker forwards `DATABASE_URL` into the container and Paperclip uses it
|
||||
instead of embedded Postgres.
|
||||
|
||||
## Durable Attachments via R2 (Optional)
|
||||
|
||||
Uploaded files (issue attachments, images) can survive container recycling
|
||||
without any Paperclip configuration: the Worker FUSE-mounts an R2 bucket at
|
||||
Paperclip's [local-disk storage directory](/deploy/storage) before boot,
|
||||
credential-less, through the Worker's own R2 binding.
|
||||
|
||||
```sh
|
||||
npx wrangler r2 bucket create paperclip-attachments
|
||||
# then uncomment the r2_buckets block in wrangler.jsonc and redeploy
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Only the attachments directory is mounted. The Postgres data directory
|
||||
stays on container disk **by design** — databases must not run on FUSE
|
||||
mounts. Durable attachments complement, not replace, `DATABASE_URL`.
|
||||
- In `wrangler dev` the SDK syncs the directory through the R2 binding
|
||||
instead of s3fs; behavior is equivalent for testing.
|
||||
- Alternatively, Paperclip's own `s3` storage provider can talk to R2
|
||||
directly via its S3-compatible API (`paperclipai configure --section
|
||||
storage`) — the mount is just the zero-config path.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A Cloudflare account on the **Workers Paid** plan (containers are not
|
||||
available on the free tier)
|
||||
- [Docker](https://docs.docker.com/get-docker/) running locally (wrangler
|
||||
builds the container image and, for `wrangler dev`, runs it)
|
||||
- Node.js 20+ and pnpm
|
||||
|
||||
## Deploy
|
||||
|
||||
```sh
|
||||
cd deploy/cloudflare
|
||||
pnpm install --ignore-workspace
|
||||
npx wrangler login
|
||||
npx wrangler deploy
|
||||
|
||||
# optional: give in-container agents an API key
|
||||
npx wrangler secret put ANTHROPIC_API_KEY
|
||||
```
|
||||
|
||||
Then open the printed `https://paperclip-sandbox.<your-subdomain>.workers.dev`
|
||||
URL. The **first request** provisions the container and onboards Paperclip
|
||||
(a minute or two) — you'll see a self-refreshing status page until the app is
|
||||
up. Paperclip boots in `authenticated` mode with a pending bootstrap invite;
|
||||
the first login claims the operator account, so open the URL yourself promptly
|
||||
after deploying.
|
||||
|
||||
## Configuration
|
||||
|
||||
Set via `vars` in `wrangler.jsonc` or `wrangler secret put`:
|
||||
|
||||
| Name | Kind | Default | Purpose |
|
||||
| --- | --- | --- | --- |
|
||||
| `PAPERCLIP_PUBLIC_URL` | var | request origin | Public URL Paperclip advertises |
|
||||
| `PAPERCLIP_DEPLOYMENT_MODE` | var | `authenticated` | See [Deployment Modes](/deploy/deployment-modes) |
|
||||
| `PAPERCLIP_DEPLOYMENT_EXPOSURE` | var | `private` | Embedded Postgres currently requires `private`; use `public` only with an external `DATABASE_URL` |
|
||||
| `ANTHROPIC_API_KEY` | secret | — | Forwarded to in-container agent CLIs |
|
||||
| `DATABASE_URL` | secret | — | External Postgres (strongly recommended, see above) |
|
||||
| `ARTIFACTS` | R2 binding | — | Durable attachment storage (see above) |
|
||||
|
||||
Container sizing lives in `wrangler.jsonc` (`instance_type`, default
|
||||
2 vCPU / 8 GiB / 10 GB). Embedded Postgres plus concurrent agent processes
|
||||
want real memory; shrink with care.
|
||||
|
||||
## How It Works
|
||||
|
||||
- `src/index.ts` — Worker entry. On each request it idempotently ensures the
|
||||
Paperclip boot process is running (`sandbox.startProcess`), then proxies:
|
||||
WebSocket upgrades via `sandbox.wsConnect(request, 3100)`, everything else
|
||||
via `sandbox.containerFetch(request, 3100)`. Same-origin proxying keeps
|
||||
Paperclip's cookies and live-update WebSockets on one host.
|
||||
- `container/Dockerfile` — extends `cloudflare/sandbox` (tag pinned to the
|
||||
`@cloudflare/sandbox` package version; enforced by `test/config.test.ts`),
|
||||
installs `paperclipai` and the agent CLIs.
|
||||
- `container/start-paperclip.sh` — onboards once
|
||||
(`paperclipai onboard --yes --bind lan`), then `paperclipai run`, as a
|
||||
non-root user (embedded Postgres refuses root).
|
||||
|
||||
## Local Development
|
||||
|
||||
```sh
|
||||
cd deploy/cloudflare
|
||||
pnpm install --ignore-workspace
|
||||
pnpm dev # wrangler dev — builds and runs the container via Docker
|
||||
```
|
||||
|
||||
Run the unit and config-consistency tests:
|
||||
|
||||
```sh
|
||||
pnpm test
|
||||
pnpm typecheck
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Status page loops for more than ~5 minutes** — check `npx wrangler tail`
|
||||
for container/start errors. First-ever deploys can also spend a few minutes
|
||||
provisioning container capacity.
|
||||
- **`Container is currently provisioning`** in logs is normal on first boot.
|
||||
- **Everything reset after idling** — that's the ephemerality caveat above;
|
||||
configure `DATABASE_URL`.
|
||||
- **Inspect the container directly** — `npx wrangler dev` locally, then
|
||||
`docker exec` into the running container; or add temporary debug output to
|
||||
`start-paperclip.sh`.
|
||||
|
||||
## Cost Notes
|
||||
|
||||
You pay for Worker requests plus container runtime (vCPU-seconds, memory,
|
||||
disk) while the sandbox is awake. The container sleeps after inactivity;
|
||||
with embedded Postgres that also means data loss (see above), which is the
|
||||
other reason to use an external database.
|
||||
|
|
@ -82,6 +82,7 @@
|
|||
"deploy/local-development",
|
||||
"deploy/tailscale-private-access",
|
||||
"deploy/docker",
|
||||
"deploy/cloudflare",
|
||||
"deploy/deployment-modes",
|
||||
"deploy/database",
|
||||
"deploy/secrets",
|
||||
|
|
|
|||
Loading…
Reference in New Issue