From 216d3d2680d9be66eab5d98e1607bd217a062c1f Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Wed, 22 Jul 2026 19:48:43 -0700 Subject: [PATCH] Managed-instance config: fail-closed PAPERCLIP_MANAGED_CONFIG parsing and read-time settings overlay (#10058) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **Builds on.** #10055 — the `catalogVersion` this config document pins is the feature-catalog artifact #10055 emits. **Summary.** Instances operated by a managed hosting control plane can now receive instance configuration through a single environment variable, `PAPERCLIP_MANAGED_CONFIG` (versioned JSON: `mode`, `catalogVersion`, `features`, `plugins.autoInstall`). When the variable is absent the instance is self-hosted and nothing changes. When present, parsing is strict and **fail-closed**: blank value, malformed JSON, unknown feature key, a feature key this build's feature catalog does not mark tier `managed`, missing required section, or unsupported version refuses startup with a precise error — a typo that silently does nothing is how a security control quietly fails. Managed feature values are overlaid **at read time** inside the instance settings service (never persisted), so a DB restore or manual row edit cannot resurrect a disabled capability; responses expose per-key `managedKeys` metadata (`managed: true`, `managedBy`) so clients can render locked state. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs both self-hosted and under managed hosting, where an operator's control plane owns instance configuration > - Today instance feature settings live only in the tenant database; a hosting control plane has no way to enforce a configuration that tenant-side writes or restores cannot undo > - Managed configuration will carry security posture, so delivery must be atomic and parsing must fail closed — a typo that silently does nothing is how a security control quietly fails > - This pull request adds strict parsing of one `PAPERCLIP_MANAGED_CONFIG` env var and overlays its feature values at read time inside the settings service, never persisting them > - The benefit is a minimal, auditable managed-hosting contract: absent var ⇒ self-hosted instances are byte-for-byte unchanged; present ⇒ deterministic, locked configuration surfaced to clients via per-key managed metadata ## Linked Issues or Issue Description Refs #966 — this PR delivers that issue's "managed config injection" hook, via a strict env-var contract rather than the config-file path it sketches; the issue's other hooks (identity header, health, usage webhook, lifecycle, external secrets, IAM auth) are out of scope, so the PR refs rather than closes it. *Mechanism differs from #966's proposal, so the `feature_request` fields are also filled in:* - **Problem or motivation:** managed hosting deployments need to centrally enable/disable instance features; DB-stored settings can be edited, restored, or migrated back to permissive values, and nothing marks a value as operator-enforced. - **Proposed solution:** one versioned JSON env var; fail-closed parse at startup; read-time overlay in the settings service (precedence: managed value over stored value over schema default); `managedKeys` metadata in settings responses so clients can render locked state. - **Alternatives considered:** per-feature env vars (non-atomic across a half-updated env set, unbounded env surface); seeding the DB at boot (persisted values can be edited or restored over, and cannot express "forced"); lenient warn-and-drop parsing (fails open — unacceptable for a security-bearing control). - **Roadmap alignment:** supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`. ## What Changed - New `server/src/services/managed-config.ts` (pure parser over the env record) - Startup parse ordered before the first `instanceSettingsService` construction in `server/src/index.ts` - Read-time merge + `managedKeys` in the settings service - Shared validator updates ## Verification - 29 parser/overlay tests (fail-closed matrix incl. blank/whitespace env, missing sections, catalog-tier mismatch, empty-section happy path): `pnpm vitest run src/__tests__/managed-config.test.ts src/__tests__/instance-settings-managed-overlay.test.ts` (from `server/`) - 40 existing settings route/service tests green: `pnpm vitest run src/__tests__/instance-settings-routes.test.ts src/__tests__/instance-settings-service.test.ts` (from `server/`) - 15 shared validator tests: `pnpm vitest run src/validators/instance.test.ts` (from `packages/shared/`) - Server `tsc --noEmit` clean: `pnpm typecheck` (from `server/`) ## Risks - Self-hosted instances (no `PAPERCLIP_MANAGED_CONFIG` set) are byte-for-byte unchanged — the parser only runs when the variable is present. - For managed instances, a malformed document now refuses startup by design (fail-closed). This is an intentional behavioral guarantee, not a regression: the control plane owns the variable and a precise startup error is the contract. - Overlay values are never persisted, so no migration or data-shape risk. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use; independently peer-reviewed by a second AI agent before push. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- packages/shared/src/index.ts | 6 + packages/shared/src/types/index.ts | 4 + packages/shared/src/types/instance.ts | 30 ++- packages/shared/src/validators/index.ts | 2 + packages/shared/src/validators/instance.ts | 14 +- .../instance-settings-managed-overlay.test.ts | 166 ++++++++++++ server/src/__tests__/managed-config.test.ts | 246 ++++++++++++++++++ server/src/index.ts | 26 ++ server/src/services/index.ts | 9 +- server/src/services/instance-settings.ts | 69 ++++- server/src/services/managed-config.ts | 222 ++++++++++++++++ 11 files changed, 780 insertions(+), 14 deletions(-) create mode 100644 server/src/__tests__/instance-settings-managed-overlay.test.ts create mode 100644 server/src/__tests__/managed-config.test.ts create mode 100644 server/src/services/managed-config.ts diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index e6f05efb3f..a9aad0542a 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -669,8 +669,11 @@ export type { AgentSkillSyncRequest, InstanceExecutionMode, InstanceExperimentalSettings, + InstanceExperimentalSettingsWithManaged, InstanceGeneralSettings, InstanceSettings, + ManagedExperimentalFeatureKey, + ManagedSettingMetadata, IssueGraphLivenessAutoRecoveryPreview, IssueGraphLivenessAutoRecoveryPreviewItem, BackupRetentionPolicy, @@ -1361,6 +1364,7 @@ export { DEFAULT_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, MIN_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, MAX_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, + PAPERCLIP_CLOUD_MANAGED_BY, } from "./types/instance.js"; export type { @@ -1409,6 +1413,8 @@ export { patchInstanceGeneralSettingsSchema, type PatchInstanceGeneralSettings, instanceExperimentalSettingsSchema, + instanceExperimentalSettingsWithManagedSchema, + managedSettingMetadataSchema, patchInstanceExperimentalSettingsSchema, patchInstanceSettingsSchema, issueGraphLivenessAutoRecoveryRequestSchema, diff --git a/packages/shared/src/types/index.ts b/packages/shared/src/types/index.ts index 33f40ee74a..ee8c9eae23 100644 --- a/packages/shared/src/types/index.ts +++ b/packages/shared/src/types/index.ts @@ -64,8 +64,11 @@ export type { export type { InstanceExecutionMode, InstanceExperimentalSettings, + InstanceExperimentalSettingsWithManaged, InstanceGeneralSettings, InstanceSettings, + ManagedExperimentalFeatureKey, + ManagedSettingMetadata, BackupRetentionPolicy, IssueGraphLivenessAutoRecoveryPreview, IssueGraphLivenessAutoRecoveryPreviewItem, @@ -93,6 +96,7 @@ export { DEFAULT_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, MIN_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, MAX_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, + PAPERCLIP_CLOUD_MANAGED_BY, } from "./instance.js"; export { TRUST_PRESETS, diff --git a/packages/shared/src/types/instance.ts b/packages/shared/src/types/instance.ts index 29aaedb9ce..9da1ac6056 100644 --- a/packages/shared/src/types/instance.ts +++ b/packages/shared/src/types/instance.ts @@ -87,11 +87,39 @@ export interface InstanceExperimentalSettings { issueGraphLivenessAutoRecoveryLookbackHours: number; } +/** + * Boolean feature-flag keys of the experimental settings — the only keys a + * cloud managed-config overlay may target. Server-managed bookkeeping fields + * (activation cutoffs, lookback hours) are excluded by construction. + */ +export type ManagedExperimentalFeatureKey = { + [K in keyof InstanceExperimentalSettings]-?: InstanceExperimentalSettings[K] extends boolean + ? K + : never; +}[keyof InstanceExperimentalSettings]; + +export const PAPERCLIP_CLOUD_MANAGED_BY = "paperclip-cloud" as const; + +/** Per-key metadata attached to settings responses for cloud-overlaid keys. */ +export interface ManagedSettingMetadata { + managed: true; + managedBy: typeof PAPERCLIP_CLOUD_MANAGED_BY; +} + +/** + * Experimental settings as returned by the settings API. On cloud-managed + * instances (`PAPERCLIP_MANAGED_CONFIG` present) `managedKeys` lists every key + * whose value is overlaid by the harness; self-hosted responses omit it. + */ +export interface InstanceExperimentalSettingsWithManaged extends InstanceExperimentalSettings { + managedKeys?: Partial>; +} + export interface InstanceSettings { id: string; defaultEnvironmentId: string | null; general: InstanceGeneralSettings; - experimental: InstanceExperimentalSettings; + experimental: InstanceExperimentalSettingsWithManaged; createdAt: Date; updatedAt: Date; } diff --git a/packages/shared/src/validators/index.ts b/packages/shared/src/validators/index.ts index 38bb67c9a7..2ea2bd0a27 100644 --- a/packages/shared/src/validators/index.ts +++ b/packages/shared/src/validators/index.ts @@ -5,6 +5,8 @@ export { type InstanceGeneralSettings, type PatchInstanceGeneralSettings, instanceExperimentalSettingsSchema, + instanceExperimentalSettingsWithManagedSchema, + managedSettingMetadataSchema, patchInstanceExperimentalSettingsSchema, patchInstanceSettingsSchema, issueGraphLivenessAutoRecoveryRequestSchema, diff --git a/packages/shared/src/validators/instance.ts b/packages/shared/src/validators/instance.ts index 4ba5cb6e7f..e333bb380c 100644 --- a/packages/shared/src/validators/instance.ts +++ b/packages/shared/src/validators/instance.ts @@ -80,6 +80,18 @@ export const patchInstanceExperimentalSettingsSchema = instanceExperimentalSetti .partial() .strip(); +export const managedSettingMetadataSchema = z.object({ + managed: z.literal(true), + managedBy: z.literal("paperclip-cloud"), +}).strict(); + +// Response shape of the experimental settings endpoints: on cloud-managed +// instances every overlaid key is listed in `managedKeys`; self-hosted +// responses omit the field entirely. +export const instanceExperimentalSettingsWithManagedSchema = instanceExperimentalSettingsSchema.extend({ + managedKeys: z.record(managedSettingMetadataSchema).optional(), +}).strict(); + export const patchInstanceSettingsSchema = z.object({ defaultEnvironmentId: z.string().uuid().nullable().optional(), }).strict(); @@ -106,7 +118,7 @@ export const instanceSettingsSchema = z.object({ id: z.string().uuid(), defaultEnvironmentId: z.string().uuid().nullable(), general: instanceGeneralSettingsSchema, - experimental: instanceExperimentalSettingsSchema, + experimental: instanceExperimentalSettingsWithManagedSchema, createdAt: z.union([z.date(), z.string().datetime()]), updatedAt: z.union([z.date(), z.string().datetime()]), }).strict(); diff --git a/server/src/__tests__/instance-settings-managed-overlay.test.ts b/server/src/__tests__/instance-settings-managed-overlay.test.ts new file mode 100644 index 0000000000..7fcfd61c38 --- /dev/null +++ b/server/src/__tests__/instance-settings-managed-overlay.test.ts @@ -0,0 +1,166 @@ +import { describe, expect, it } from "vitest"; +import type { Db } from "@paperclipai/db"; +import { + applyManagedExperimentalOverlay, + instanceSettingsService, + normalizeExperimentalSettings, +} from "../services/instance-settings.js"; +import { parseManagedConfigEnv } from "../services/managed-config.js"; + +const MANAGED_RAW = JSON.stringify({ + v: 1, + mode: "cloud", + catalogVersion: "2026.720.0", + // enableApps stored true in the DB gets forced off; enablePipelines has no + // stored value, so the overlay wins over the schema default (false). + features: { enableApps: false, enablePipelines: true }, + plugins: { autoInstall: [] }, +}); + +function managedEnv(raw: string | undefined = MANAGED_RAW) { + return { PAPERCLIP_MANAGED_CONFIG: raw }; +} + +/** + * Minimal stand-in for the drizzle query chains instanceSettingsService uses. + * Captures every `update().set()` payload so tests can assert what would be + * persisted. + */ +function stubDb(row: Record) { + const persistedSets: Array> = []; + const db = { + select: () => ({ from: () => ({ where: () => Promise.resolve([row]) }) }), + insert: () => { + throw new Error("unexpected insert in test"); + }, + update: () => ({ + set: (values: Record) => { + persistedSets.push(values); + return { where: () => ({ returning: () => Promise.resolve([{ ...row, ...values }]) }) }; + }, + }), + } as unknown as Db; + return { db, persistedSets }; +} + +function settingsRow(experimental: Record) { + return { + id: "row-1", + singletonKey: "default", + defaultEnvironmentId: null, + general: {}, + experimental, + createdAt: new Date("2026-06-20T00:00:00.000Z"), + updatedAt: new Date("2026-06-20T00:00:00.000Z"), + }; +} + +describe("applyManagedExperimentalOverlay", () => { + it("is the identity with no managed config (self-hosted)", () => { + const experimental = normalizeExperimentalSettings({ enableApps: true }); + const result = applyManagedExperimentalOverlay(experimental, null); + expect(result.experimental).toEqual(experimental); + expect(result.managedKeys).toEqual({}); + }); + + it("overlays managed values over stored values and records metadata", () => { + const config = parseManagedConfigEnv(managedEnv())!; + const stored = normalizeExperimentalSettings({ enableApps: true }); + const { experimental, managedKeys } = applyManagedExperimentalOverlay(stored, config); + + // managed overlay > tenant DB value + expect(experimental.enableApps).toBe(false); + // managed overlay > schema default + expect(experimental.enablePipelines).toBe(true); + // unmanaged keys keep their stored/default values + expect(experimental.enableCases).toBe(false); + expect(managedKeys).toEqual({ + enableApps: { managed: true, managedBy: "paperclip-cloud" }, + enablePipelines: { managed: true, managedBy: "paperclip-cloud" }, + }); + // input is not mutated + expect(stored.enableApps).toBe(true); + }); +}); + +describe("instanceSettingsService managed overlay", () => { + it("fails closed at construction on a malformed managed config", () => { + const { db } = stubDb(settingsRow({})); + expect(() => instanceSettingsService(db, { runtimeEnv: managedEnv("{bad") })).toThrow( + /PAPERCLIP_MANAGED_CONFIG is not valid JSON/, + ); + }); + + it("overlays managed values on getExperimental and exposes managedKeys", async () => { + const { db } = stubDb(settingsRow({ enableApps: true })); + const svc = instanceSettingsService(db, { runtimeEnv: managedEnv() }); + + const experimental = await svc.getExperimental(); + expect(experimental.enableApps).toBe(false); + expect(experimental.enablePipelines).toBe(true); + expect(experimental.managedKeys).toEqual({ + enableApps: { managed: true, managedBy: "paperclip-cloud" }, + enablePipelines: { managed: true, managedBy: "paperclip-cloud" }, + }); + }); + + it("overlays managed values on get()", async () => { + const { db } = stubDb(settingsRow({ enableApps: true })); + const svc = instanceSettingsService(db, { runtimeEnv: managedEnv() }); + + const settings = await svc.get(); + expect(settings.experimental.enableApps).toBe(false); + expect(settings.experimental.managedKeys?.enableApps).toEqual({ + managed: true, + managedBy: "paperclip-cloud", + }); + }); + + it("leaves the self-hosted read path unchanged (no managedKeys field)", async () => { + const { db } = stubDb(settingsRow({ enableApps: true })); + const svc = instanceSettingsService(db, { runtimeEnv: {} }); + + const experimental = await svc.getExperimental(); + expect(experimental.enableApps).toBe(true); + expect(Object.prototype.hasOwnProperty.call(experimental, "managedKeys")).toBe(false); + expect(experimental).toEqual(normalizeExperimentalSettings({ enableApps: true })); + + const settings = await svc.get(); + expect(Object.prototype.hasOwnProperty.call(settings.experimental, "managedKeys")).toBe(false); + }); + + it("never persists the overlay: updates write stored values, responses show managed ones", async () => { + const { db, persistedSets } = stubDb(settingsRow({ enableApps: true })); + const svc = instanceSettingsService(db, { runtimeEnv: managedEnv() }); + + const updated = await svc.updateExperimental({ enableCases: true }); + + expect(persistedSets).toHaveLength(1); + const persisted = persistedSets[0]!.experimental as Record; + // The tenant's stored value survives in the DB even though the overlay + // masks it at read time — a later un-managing restores tenant intent. + expect(persisted.enableApps).toBe(true); + // The overlay-added value is not written. + expect(persisted.enablePipelines).toBe(false); + expect(persisted.enableCases).toBe(true); + expect(persisted).not.toHaveProperty("managedKeys"); + + // The response still reflects the overlay. + expect(updated.experimental.enableApps).toBe(false); + expect(updated.experimental.enablePipelines).toBe(true); + expect(updated.experimental.managedKeys?.enableApps).toEqual({ + managed: true, + managedBy: "paperclip-cloud", + }); + }); + + it("does not let managed metadata leak into self-hosted writes", async () => { + const { db, persistedSets } = stubDb(settingsRow({})); + const svc = instanceSettingsService(db, { runtimeEnv: {} }); + + const updated = await svc.updateExperimental({ enableCases: true }); + expect(persistedSets).toHaveLength(1); + expect(persistedSets[0]!.experimental).not.toHaveProperty("managedKeys"); + expect(Object.prototype.hasOwnProperty.call(updated.experimental, "managedKeys")).toBe(false); + }); +}); diff --git a/server/src/__tests__/managed-config.test.ts b/server/src/__tests__/managed-config.test.ts new file mode 100644 index 0000000000..1717b33288 --- /dev/null +++ b/server/src/__tests__/managed-config.test.ts @@ -0,0 +1,246 @@ +import { describe, expect, it } from "vitest"; +import { + MANAGED_CONFIG_ENV_KEY, + getManagedInstanceConfig, + managedFeatureKeySet, + parseManagedConfigEnv, +} from "../services/managed-config.js"; + +function envWith(raw: string | undefined) { + return { [MANAGED_CONFIG_ENV_KEY]: raw }; +} + +function validDoc(overrides: Record = {}) { + return JSON.stringify({ + v: 1, + mode: "cloud", + catalogVersion: "2026.720.0", + features: { enableApps: false, enablePipelines: true }, + plugins: { autoInstall: ["daytona", "kubernetes"] }, + ...overrides, + }); +} + +describe("managedFeatureKeySet", () => { + it("contains exactly the boolean flags of the experimental schema", () => { + const keys = managedFeatureKeySet(); + expect(keys.has("enableApps")).toBe(true); + expect(keys.has("enableWorktreeRunExecution")).toBe(true); + // Server-managed bookkeeping fields are not overlayable features. + expect(keys.has("worktreeRunExecutionActivatedAt")).toBe(false); + expect(keys.has("worktreeRunExecutionActivationInstanceId")).toBe(false); + expect(keys.has("issueGraphLivenessAutoRecoveryLookbackHours")).toBe(false); + }); +}); + +describe("parseManagedConfigEnv", () => { + it("returns null when the env var is absent (self-hosted)", () => { + expect(parseManagedConfigEnv({})).toBeNull(); + expect(parseManagedConfigEnv(envWith(undefined))).toBeNull(); + }); + + it("throws when the env var is present but blank (fail closed)", () => { + expect(() => parseManagedConfigEnv(envWith(""))).toThrow(/is set but blank/); + expect(() => parseManagedConfigEnv(envWith(" "))).toThrow(/is set but blank/); + expect(() => parseManagedConfigEnv(envWith("\n\t"))).toThrow(/is set but blank/); + }); + + it("parses a complete valid document", () => { + const config = parseManagedConfigEnv(envWith(validDoc())); + expect(config).toEqual({ + v: 1, + mode: "cloud", + catalogVersion: "2026.720.0", + features: { enableApps: false, enablePipelines: true }, + plugins: { autoInstall: ["daytona", "kubernetes"] }, + }); + }); + + it("accepts empty features {} and autoInstall [] sections", () => { + const config = parseManagedConfigEnv( + envWith(validDoc({ features: {}, plugins: { autoInstall: [] } })), + ); + expect(config).toEqual({ + v: 1, + mode: "cloud", + catalogVersion: "2026.720.0", + features: {}, + plugins: { autoInstall: [] }, + }); + }); + + it("throws when the features section is missing (fail closed)", () => { + const doc = { v: 1, mode: "cloud", catalogVersion: "2026.720.0", plugins: { autoInstall: [] } }; + expect(() => parseManagedConfigEnv(envWith(JSON.stringify(doc)))).toThrow( + /requires a "features" object/, + ); + }); + + it("throws when the plugins section or autoInstall is missing (fail closed)", () => { + const noPlugins = { + v: 1, + mode: "cloud", + catalogVersion: "2026.720.0", + features: {}, + }; + expect(() => parseManagedConfigEnv(envWith(JSON.stringify(noPlugins)))).toThrow( + /requires a "plugins" object/, + ); + expect(() => parseManagedConfigEnv(envWith(validDoc({ plugins: {} })))).toThrow( + /requires a "plugins.autoInstall" array/, + ); + }); + + it("returns a frozen document", () => { + const config = parseManagedConfigEnv(envWith(validDoc())); + expect(Object.isFrozen(config)).toBe(true); + expect(Object.isFrozen(config?.features)).toBe(true); + expect(Object.isFrozen(config?.plugins.autoInstall)).toBe(true); + }); + + it("throws on invalid JSON", () => { + expect(() => parseManagedConfigEnv(envWith("{not json"))).toThrow( + /PAPERCLIP_MANAGED_CONFIG is not valid JSON/, + ); + }); + + it("throws on non-object documents", () => { + expect(() => parseManagedConfigEnv(envWith("[]"))).toThrow(/must be a JSON object/); + expect(() => parseManagedConfigEnv(envWith("42"))).toThrow(/must be a JSON object/); + expect(() => parseManagedConfigEnv(envWith("null"))).toThrow(/must be a JSON object/); + expect(() => parseManagedConfigEnv(envWith('"cloud"'))).toThrow(/must be a JSON object/); + }); + + it("throws on an unknown top-level key", () => { + expect(() => parseManagedConfigEnv(envWith(validDoc({ extra: true })))).toThrow( + /unknown top-level key "extra"/, + ); + }); + + it("throws on an unsupported v", () => { + expect(() => parseManagedConfigEnv(envWith(validDoc({ v: 2 })))).toThrow( + /unsupported "v" 2; this build supports v=1/, + ); + expect(() => parseManagedConfigEnv(envWith(validDoc({ v: "1" })))).toThrow(/unsupported "v"/); + expect(() => + parseManagedConfigEnv( + envWith(JSON.stringify({ mode: "cloud", catalogVersion: "x" })), + ), + ).toThrow(/unsupported "v"/); + }); + + it("throws on a non-cloud mode", () => { + expect(() => parseManagedConfigEnv(envWith(validDoc({ mode: "self-hosted" })))).toThrow( + /invalid "mode" "self-hosted"; expected "cloud"/, + ); + expect(() => + parseManagedConfigEnv(envWith(JSON.stringify({ v: 1, catalogVersion: "x" }))), + ).toThrow(/invalid "mode"/); + }); + + it("throws on a missing or empty catalogVersion", () => { + expect(() => + parseManagedConfigEnv(envWith(JSON.stringify({ v: 1, mode: "cloud" }))), + ).toThrow(/non-empty string "catalogVersion"/); + expect(() => parseManagedConfigEnv(envWith(validDoc({ catalogVersion: "" })))).toThrow( + /non-empty string "catalogVersion"/, + ); + expect(() => parseManagedConfigEnv(envWith(validDoc({ catalogVersion: 7 })))).toThrow( + /non-empty string "catalogVersion"/, + ); + }); + + it("throws on a non-object features section", () => { + expect(() => parseManagedConfigEnv(envWith(validDoc({ features: ["enableApps"] })))).toThrow( + /"features" must be an object/, + ); + }); + + it("throws on an unknown feature key", () => { + expect(() => + parseManagedConfigEnv(envWith(validDoc({ features: { enableTimeTravel: true } }))), + ).toThrow(/unknown feature key "enableTimeTravel"/); + // A server-managed bookkeeping field is not an overlayable feature. + expect(() => + parseManagedConfigEnv( + envWith(validDoc({ features: { worktreeRunExecutionActivatedAt: true } })), + ), + ).toThrow(/unknown feature key "worktreeRunExecutionActivatedAt"/); + }); + + it("throws on a feature key the catalog does not mark tier \"managed\"", () => { + // `enableStreamlinedLeftNavigation` is a real schema flag, but its catalog + // tier is `preference` (tenant-controllable) — a managed-config document + // targeting it has incompatible catalog semantics and must fail closed. + expect(() => + parseManagedConfigEnv( + envWith(validDoc({ features: { enableStreamlinedLeftNavigation: true } })), + ), + ).toThrow( + /"features" key "enableStreamlinedLeftNavigation" has tier "preference".*only tier "managed" keys/, + ); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ features: { enableDecisions: false } }))), + ).toThrow(/has tier "preference"/); + }); + + it("throws on non-boolean feature values", () => { + expect(() => + parseManagedConfigEnv(envWith(validDoc({ features: { enableApps: "true" } }))), + ).toThrow(/"features.enableApps" must be a boolean/); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ features: { enableApps: 1 } }))), + ).toThrow(/"features.enableApps" must be a boolean/); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ features: { enableApps: null } }))), + ).toThrow(/"features.enableApps" must be a boolean/); + }); + + it("throws on malformed plugins sections", () => { + expect(() => parseManagedConfigEnv(envWith(validDoc({ plugins: [] })))).toThrow( + /"plugins" must be an object/, + ); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ plugins: { install: [] } }))), + ).toThrow(/"plugins" has unknown key "install"/); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ plugins: { autoInstall: "daytona" } }))), + ).toThrow(/"plugins.autoInstall" must be an array/); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ plugins: { autoInstall: [""] } }))), + ).toThrow(/non-empty strings/); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ plugins: { autoInstall: [" daytona"] } }))), + ).toThrow(/non-empty strings/); + expect(() => + parseManagedConfigEnv(envWith(validDoc({ plugins: { autoInstall: [42] } }))), + ).toThrow(/non-empty strings/); + expect(() => + parseManagedConfigEnv( + envWith(validDoc({ plugins: { autoInstall: ["daytona", "daytona"] } })), + ), + ).toThrow(/duplicate entry "daytona"/); + }); +}); + +describe("getManagedInstanceConfig", () => { + it("caches by raw env value and reparses when it changes", () => { + const raw = validDoc(); + const first = getManagedInstanceConfig(envWith(raw)); + const second = getManagedInstanceConfig(envWith(raw)); + expect(second).toBe(first); + + const changed = getManagedInstanceConfig( + envWith(validDoc({ catalogVersion: "2026.721.0" })), + ); + expect(changed?.catalogVersion).toBe("2026.721.0"); + expect(changed).not.toBe(first); + + expect(getManagedInstanceConfig(envWith(undefined))).toBeNull(); + }); + + it("rethrows parse failures on every call instead of caching them", () => { + expect(() => getManagedInstanceConfig(envWith("{bad"))).toThrow(/not valid JSON/); + expect(() => getManagedInstanceConfig(envWith("{bad"))).toThrow(/not valid JSON/); + }); +}); diff --git a/server/src/index.ts b/server/src/index.ts index 2755cc9177..03f6104a3f 100644 --- a/server/src/index.ts +++ b/server/src/index.ts @@ -33,6 +33,7 @@ import detectPort from "detect-port"; import { createApp } from "./app.js"; import { loadConfig } from "./config.js"; import { logger } from "./middleware/logger.js"; +import { getManagedInstanceConfig } from "./services/managed-config.js"; import { setupEnvironmentCustomImageTerminalWebSocketServer } from "./realtime/environment-custom-image-terminal-ws.js"; import { setupLiveEventsWebSocketServer } from "./realtime/live-events-ws.js"; import { @@ -586,6 +587,31 @@ export async function startServer(): Promise { serverPort: listenPort, databasePort: resolvedEmbeddedPostgresPort, }); + // Cloud managed-config contract (harness → app). Parse PAPERCLIP_MANAGED_CONFIG + // once so a malformed document (blank value, bad JSON, unknown feature key, + // unsupported v, missing section) refuses startup with a precise error instead + // of silently running without the feature overlay. Absent env = self-hosted: + // nothing changes. The parsed document is never persisted; instanceSettingsService + // overlays it per read. This MUST run before any instanceSettingsService(db) + // construction — that constructor parses the same env, and it would otherwise + // throw first, bypassing this fail-closed log path. + try { + const managedConfig = getManagedInstanceConfig(); + if (managedConfig) { + logger.warn( + { + catalogVersion: managedConfig.catalogVersion, + managedFeatureKeys: Object.keys(managedConfig.features).sort(), + autoInstallPlugins: [...managedConfig.plugins.autoInstall], + }, + "cloud managed configuration active", + ); + } + } catch (err) { + logger.error({ err }, "invalid PAPERCLIP_MANAGED_CONFIG; refusing to start (fail closed)"); + throw err; + } + const uiMode = config.uiDevMiddleware ? "vite-dev" : config.serveUi ? "static" : "none"; const storageService = createStorageServiceFromConfig(config); const feedback = feedbackService(db as any, { diff --git a/server/src/services/index.ts b/server/src/services/index.ts index 2fc0666740..52186f7cbe 100644 --- a/server/src/services/index.ts +++ b/server/src/services/index.ts @@ -108,7 +108,14 @@ export type { AuthorizationResource, } from "./authorization.js"; export { boardAuthService } from "./board-auth.js"; -export { instanceSettingsService } from "./instance-settings.js"; +export { instanceSettingsService, applyManagedExperimentalOverlay } from "./instance-settings.js"; +export { + getManagedInstanceConfig, + managedFeatureKeySet, + parseManagedConfigEnv, + MANAGED_CONFIG_ENV_KEY, + type ManagedInstanceConfig, +} from "./managed-config.js"; export { bootstrapExecutionPolicyFromEnv } from "./execution-policy-bootstrap.js"; export { cloudUpstreamService, reconcileCloudUpstreamRunsOnStartup } from "./cloud-upstreams.js"; export { companyPortabilityService } from "./company-portability.js"; diff --git a/server/src/services/instance-settings.ts b/server/src/services/instance-settings.ts index b8656d8d37..4b36d298dc 100644 --- a/server/src/services/instance-settings.ts +++ b/server/src/services/instance-settings.ts @@ -4,16 +4,21 @@ import { DEFAULT_FEEDBACK_DATA_SHARING_PREFERENCE, DEFAULT_BACKUP_RETENTION, DEFAULT_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS, + PAPERCLIP_CLOUD_MANAGED_BY, instanceGeneralSettingsSchema, type InstanceGeneralSettings, instanceExperimentalSettingsSchema, type InstanceExperimentalSettings, + type InstanceExperimentalSettingsWithManaged, + type ManagedExperimentalFeatureKey, + type ManagedSettingMetadata, type PatchInstanceGeneralSettings, type InstanceSettings, type PatchInstanceSettings, type PatchInstanceExperimentalSettings, } from "@paperclipai/shared"; import { eq } from "drizzle-orm"; +import { getManagedInstanceConfig, type ManagedInstanceConfig } from "./managed-config.js"; const DEFAULT_SINGLETON_KEY = "default"; const instanceGeneralSettingsStorageSchema = instanceGeneralSettingsSchema.strip(); @@ -264,18 +269,60 @@ export function normalizeExperimentalSettings(raw: unknown): InstanceExperimenta }; } -function toInstanceSettings(row: typeof instanceSettings.$inferSelect): InstanceSettings { - return { - id: row.id, - defaultEnvironmentId: row.defaultEnvironmentId ?? null, - general: normalizeGeneralSettings(row.general), - experimental: normalizeExperimentalSettings(row.experimental), - createdAt: row.createdAt, - updatedAt: row.updatedAt, - } as InstanceSettings; +export type ManagedExperimentalKeyMetadata = Partial< + Record +>; + +/** + * Overlay the cloud managed-config feature values over normalized settings. + * + * Read-time precedence: code floor (cloud) > managed overlay > tenant DB + * value > schema default. (No code floors are expressed as flags today — + * floors are enforced in code at the guarded routes, independent of any + * flag value.) The overlay is deliberately never persisted: it re-asserts on + * every read, so a DB restore or manual row edit cannot resurrect a + * capability the harness has disabled. + */ +export function applyManagedExperimentalOverlay( + experimental: InstanceExperimentalSettings, + managedConfig: ManagedInstanceConfig | null, +): { experimental: InstanceExperimentalSettings; managedKeys: ManagedExperimentalKeyMetadata } { + if (!managedConfig) return { experimental, managedKeys: {} }; + const next: InstanceExperimentalSettings = { ...experimental }; + const managedKeys: ManagedExperimentalKeyMetadata = {}; + for (const [key, value] of Object.entries(managedConfig.features) as Array< + [ManagedExperimentalFeatureKey, boolean] + >) { + next[key] = value; + managedKeys[key] = { managed: true, managedBy: PAPERCLIP_CLOUD_MANAGED_BY }; + } + return { experimental: next, managedKeys }; } export function instanceSettingsService(db: Db, options: InstanceSettingsServiceOptions = {}) { + // Fail closed: a malformed PAPERCLIP_MANAGED_CONFIG throws here (and at + // boot in index.ts) rather than silently running without the overlay. + const managedConfig = getManagedInstanceConfig(options.runtimeEnv ?? process.env); + + function toExperimentalView(raw: unknown): InstanceExperimentalSettingsWithManaged { + const { experimental, managedKeys } = applyManagedExperimentalOverlay( + normalizeExperimentalSettings(raw), + managedConfig, + ); + // Self-hosted responses stay byte-identical: no managedKeys field at all. + return managedConfig ? { ...experimental, managedKeys } : experimental; + } + + function toInstanceSettings(row: typeof instanceSettings.$inferSelect): InstanceSettings { + return { + id: row.id, + defaultEnvironmentId: row.defaultEnvironmentId ?? null, + general: normalizeGeneralSettings(row.general), + experimental: toExperimentalView(row.experimental), + createdAt: row.createdAt, + updatedAt: row.updatedAt, + } as InstanceSettings; + } async function getOrCreateRow() { const existing = await db .select() @@ -338,9 +385,9 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService return normalizeGeneralSettings(row.general); }, - getExperimental: async (): Promise => { + getExperimental: async (): Promise => { const row = await getOrCreateRow(); - return normalizeExperimentalSettings(row.experimental); + return toExperimentalView(row.experimental); }, updateGeneral: async (patch: PatchInstanceGeneralSettings): Promise => { diff --git a/server/src/services/managed-config.ts b/server/src/services/managed-config.ts new file mode 100644 index 0000000000..87c37b5332 --- /dev/null +++ b/server/src/services/managed-config.ts @@ -0,0 +1,222 @@ +/** + * Cloud managed-config bootstrap (harness → app contract). + * + * Instances managed by the Paperclip Cloud harness receive one environment + * variable, `PAPERCLIP_MANAGED_CONFIG`, holding a single JSON document: + * + * { + * "v": 1, + * "mode": "cloud", + * "catalogVersion": "2026.720.0", + * "features": { "": true | false, ... }, + * "plugins": { "autoInstall": ["daytona", "kubernetes"] } + * } + * + * Parsing follows the `execution-policy-bootstrap.ts` doctrine: a pure + * function over `Record`, strict, and fail + * closed — bad JSON, an unsupported `v`, an unknown feature key, a feature + * key this build's feature catalog does not mark tier "managed", or any + * malformed section throws with a precise error so a managed instance + * refuses to start instead of silently dropping a security control. Only an + * ABSENT env var means self-hosted (no overlay, zero behavior change); a + * present-but-blank value or a document missing the `features` or + * `plugins.autoInstall` sections is malformed and fails startup, so a + * harness misconfiguration can never silently drop the managed overlay. + * + * Unlike the execution-policy bootstrap, the parsed document is NEVER + * persisted. `instanceSettingsService` overlays it at read time, so a DB + * restore or manual row edit cannot resurrect a capability the harness has + * disabled (see `applyManagedExperimentalOverlay` in instance-settings.ts). + */ + +import { + INSTANCE_FEATURE_CATALOG, + instanceExperimentalSettingsSchema, + type ManagedExperimentalFeatureKey, +} from "@paperclipai/shared"; + +export type ManagedConfigEnv = Record; + +export const MANAGED_CONFIG_ENV_KEY = "PAPERCLIP_MANAGED_CONFIG"; +export const SUPPORTED_MANAGED_CONFIG_VERSION = 1; + +export interface ManagedInstanceConfig { + v: typeof SUPPORTED_MANAGED_CONFIG_VERSION; + mode: "cloud"; + /** App feature-catalog version the document was validated against. */ + catalogVersion: string; + features: Readonly>>; + plugins: { readonly autoInstall: readonly string[] }; +} + +let cachedFeatureKeys: ReadonlySet | null = null; + +/** + * The set of feature keys a managed-config document may target: the boolean + * flag keys of `instanceExperimentalSettingsSchema`. The schema is the + * manifest — server-managed bookkeeping fields (activation cutoffs, lookback + * hours) are not booleans and are excluded by construction. + */ +export function managedFeatureKeySet(): ReadonlySet { + if (!cachedFeatureKeys) { + const defaults = instanceExperimentalSettingsSchema.parse({}) as Record; + cachedFeatureKeys = new Set( + Object.keys(defaults).filter((key) => typeof defaults[key] === "boolean"), + ); + } + return cachedFeatureKeys; +} + +function fail(detail: string): never { + throw new Error(`${MANAGED_CONFIG_ENV_KEY} ${detail}`); +} + +function isPlainObject(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function describeJsonValue(value: unknown): string { + if (value === null) return "null"; + if (Array.isArray(value)) return "an array"; + return `${JSON.stringify(value)}`; +} + +/** + * Parse `PAPERCLIP_MANAGED_CONFIG` from a raw env map. Returns null only when + * the variable is absent (self-hosted). Throws with a precise error on a + * present-but-blank value or any malformed document so a cloud instance fails + * to start (fail closed). + */ +export function parseManagedConfigEnv(env: ManagedConfigEnv): ManagedInstanceConfig | null { + const raw = env[MANAGED_CONFIG_ENV_KEY]; + if (raw === undefined) return null; + if (raw.trim().length === 0) { + fail( + "is set but blank; a managed instance requires the full JSON document (unset the variable entirely for self-hosted mode)", + ); + } + + let doc: unknown; + try { + doc = JSON.parse(raw); + } catch (err) { + fail(`is not valid JSON: ${err instanceof Error ? err.message : String(err)}`); + } + if (!isPlainObject(doc)) { + fail(`must be a JSON object (got ${describeJsonValue(doc)})`); + } + + const allowedTopLevelKeys = new Set(["v", "mode", "catalogVersion", "features", "plugins"]); + for (const key of Object.keys(doc)) { + if (!allowedTopLevelKeys.has(key)) { + fail(`has unknown top-level key "${key}" (allowed: v, mode, catalogVersion, features, plugins)`); + } + } + + if (doc.v !== SUPPORTED_MANAGED_CONFIG_VERSION) { + fail( + `has unsupported "v" ${describeJsonValue(doc.v)}; this build supports v=${SUPPORTED_MANAGED_CONFIG_VERSION}`, + ); + } + if (doc.mode !== "cloud") { + fail(`has invalid "mode" ${describeJsonValue(doc.mode)}; expected "cloud"`); + } + if (typeof doc.catalogVersion !== "string" || doc.catalogVersion.trim().length === 0) { + fail(`requires a non-empty string "catalogVersion" (got ${describeJsonValue(doc.catalogVersion)})`); + } + + // The cloud contract is the FULL document: `features` and + // `plugins.autoInstall` are required (empty {} / [] are fine). A missing + // section means a truncated or mis-built document, and defaulting it to + // empty would silently drop the managed overlay or auto-install list. + // `plugins.autoInstall` is validated here as part of the atomic v1 + // document; the bundled-plugin provisioning path that consumes it lands in + // the follow-up PR that generalizes `ensureBundledKubernetesPlugin`. + const features: Partial> = {}; + if (doc.features === undefined) { + fail(`requires a "features" object mapping feature key → boolean (use {} for none)`); + } + if (!isPlainObject(doc.features)) { + fail(`"features" must be an object mapping feature key → boolean (got ${describeJsonValue(doc.features)})`); + } + const knownKeys = managedFeatureKeySet(); + for (const [key, value] of Object.entries(doc.features)) { + if (!knownKeys.has(key)) { + fail( + `"features" has unknown feature key "${key}"; known keys are the boolean flags of instanceExperimentalSettingsSchema`, + ); + } + // Catalog-compatibility enforcement: the key exists in this build, but the + // control plane may only manage keys this build's feature catalog marks + // tier "managed". A key whose tier differs (a tenant `preference`, a + // code-pinned `floor`, or a tier demoted since the document's + // `catalogVersion` was published) is version skew — refuse startup rather + // than apply a control with mismatched catalog semantics. + const tier = INSTANCE_FEATURE_CATALOG[key as ManagedExperimentalFeatureKey].tier; + if (tier !== "managed") { + fail( + `"features" key "${key}" has tier "${tier}" in this build's feature catalog; only tier "managed" keys may be set by a managed-config document (catalogVersion ${JSON.stringify(doc.catalogVersion)} is incompatible with this build)`, + ); + } + if (typeof value !== "boolean") { + fail(`"features.${key}" must be a boolean (got ${describeJsonValue(value)})`); + } + features[key as ManagedExperimentalFeatureKey] = value; + } + + const autoInstall: string[] = []; + if (doc.plugins === undefined) { + fail(`requires a "plugins" object with an "autoInstall" array (use { "autoInstall": [] } for none)`); + } + if (!isPlainObject(doc.plugins)) { + fail(`"plugins" must be an object (got ${describeJsonValue(doc.plugins)})`); + } + for (const key of Object.keys(doc.plugins)) { + if (key !== "autoInstall") { + fail(`"plugins" has unknown key "${key}" (allowed: autoInstall)`); + } + } + const rawAutoInstall = doc.plugins.autoInstall; + if (rawAutoInstall === undefined) { + fail(`requires a "plugins.autoInstall" array of plugin keys (use [] for none)`); + } + if (!Array.isArray(rawAutoInstall)) { + fail(`"plugins.autoInstall" must be an array of plugin keys (got ${describeJsonValue(rawAutoInstall)})`); + } + for (const entry of rawAutoInstall) { + if (typeof entry !== "string" || entry.length === 0 || entry.trim() !== entry) { + fail( + `"plugins.autoInstall" entries must be non-empty strings without surrounding whitespace (got ${describeJsonValue(entry)})`, + ); + } + if (autoInstall.includes(entry)) { + fail(`"plugins.autoInstall" has duplicate entry "${entry}"`); + } + autoInstall.push(entry); + } + + return Object.freeze({ + v: SUPPORTED_MANAGED_CONFIG_VERSION, + mode: "cloud", + catalogVersion: doc.catalogVersion, + features: Object.freeze(features), + plugins: Object.freeze({ autoInstall: Object.freeze(autoInstall) }), + }) as ManagedInstanceConfig; +} + +let cache: { raw: string | undefined; config: ManagedInstanceConfig | null } | null = null; + +/** + * Parse-once accessor keyed on the raw env value. Callers that pass a custom + * env (tests) get a fresh parse whenever the raw value differs; process.env + * callers share one parsed document for the process lifetime. + */ +export function getManagedInstanceConfig( + env: ManagedConfigEnv = process.env, +): ManagedInstanceConfig | null { + const raw = env[MANAGED_CONFIG_ENV_KEY]; + if (cache && cache.raw === raw) return cache.config; + const config = parseManagedConfigEnv(env); + cache = { raw, config }; + return config; +}