fix(server): raise company import zip upload limit to 1 GB and make it operator-configurable (#11184)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Company import/export lets an operator move a full company package
between instances, with the Import page uploading the package as one
compressed `.zip`
> - The server caps that upload at 128 MB, and real company packages
with attachments now exceed it — imports fail at the preview step
> - The failure message tells the user to use the CLI folder import, but
that path posts inline JSON capped at 64 MB, so the advice is a dead end
for exactly these packages
> - This pull request raises the zip upload cap to a 1 GB default, makes
it operator-configurable through an environment variable, scales the
decompression-bomb guards from the cap in effect, and replaces the
misleading hint
> - The benefit is that large real-world company packages import
successfully, and operators with unusual needs can tune the cap without
a code change

## Linked Issues or Issue Description

**What happened?**

A company import fails at the preview step with `Preview failed: Import
package exceeds 134217728 bytes`. The package is a valid Paperclip
export. Its compressed size is larger than the 128 MB server cap (one
reported package is 257 MB). The error panel suggests the CLI folder
import, but that path sends the package as one inline JSON body capped
at 64 MB, so it also fails.

**Expected behavior**

A valid company package of realistic size imports successfully through
the Import page. If a package is too large, the error must state the
limit clearly and suggest a step that can work.

**Steps to reproduce**

1. Export a company with enough attachments to make the compressed
package larger than 128 MB.
2. Open the Import page and upload the `.zip`.
3. Click "Preview import".
4. The preview fails with `Import package exceeds 134217728 bytes`.

**Deployment mode**

Reported from a managed deployment; the limit applies to all deployment
modes.

## What Changed

- Raise `PORTABLE_ZIP_UPLOAD_LIMIT_BYTES` from 128 MB to a 1 GB default
(`server/src/http/body-limits.ts`).
- Add the `PAPERCLIP_IMPORT_ZIP_MAX_BYTES` environment override. Invalid
or non-positive values fall back to the default.
- Scale the zip decompression-bomb guard from the configured cap at the
import route: the aggregate inflated ceiling is 4x the cap. The
per-entry ceiling stays at 512 MB because V8's string length limit
applies to an entry regardless (`server/src/routes/companies.ts`,
`packages/shared/src/portability-zip.ts`).
- Report the 422 limit error in MB instead of raw bytes.
- Replace the "use the CLI folder import for very large packages" hint
on preview failure with advice that works: re-export the package without
large attachments (`ui/src/pages/CompanyImport.tsx`).
- Update the stale comment in `ui/src/lib/import-preflight.ts` that made
the same CLI claim.
- Add tests for the new default, the env override, and the
invalid-override fallback.

## Verification

- `pnpm vitest run server/src/__tests__/body-limits.test.ts
packages/shared/src/portability-zip.test.ts
server/src/__tests__/company-portability-routes.test.ts
server/src/__tests__/company-portability.test.ts
server/src/__tests__/company-portability-import-batching.test.ts` — all
pass.
- `pnpm vitest run ui/src/pages/CompanyImport.test.tsx` — passes,
including the updated failure-panel copy assertion.
- `pnpm typecheck` — clean across the workspace.
- Manual: upload a `.zip` larger than the configured cap; the preview
fails with `Import package exceeds the 1024 MB upload limit` and the new
hint. A package between 128 MB and 1 GB now previews and imports.

## Risks

- Peak per-import memory rises with the cap: the upload is buffered in
memory and unzipped in one pass. A 1 GB compressed package can use
several GB transiently. Imports are instance-admin actions, so the
exposure is a deliberate operator action, not anonymous traffic.
Operators on small hosts can lower the cap with
`PAPERCLIP_IMPORT_ZIP_MAX_BYTES`.
- The aggregate bomb guard moves from a fixed 512 MB to 4x the
configured cap. It still bounds expansion far below what a decompression
bomb needs.
- No migration and no API shape change. The 422 message text changes; no
code matches on the old text.

## Model Used

- Claude Fable 5 (`claude-fable-5`) via Claude Code CLI, with extended
thinking and tool use (file edits, local test runs, live-instance
inspection).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Devin Foley 2026-08-10 12:47:03 -07:00 committed by GitHub
parent f9173782cd
commit 5ca752dc81
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
8 changed files with 82 additions and 13 deletions

View File

@ -270,7 +270,7 @@ describe("readZipArchive", () => {
it("bounds a highly compressible DEFLATE entry at the per-entry decompressed limit", async () => {
// Compresses tiny but expands to 8 KiB; a 1 KiB cap must reject it before it
// materializes. Real packages sit far under the 256 MB production default.
// materializes. Real packages sit far under the 512 MB production default.
const bomb = new TextEncoder().encode("a".repeat(8 * 1024));
const archive = buildZip([{ path: "bomb.txt", bytes: bomb, method: 8 }], "paperclip-demo");
await expect(

View File

@ -20,10 +20,13 @@ const strictTextDecoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: tru
// malicious highly-compressible package cannot exhaust server memory: a
// per-entry ceiling (passed to zlib as maxOutputLength, so it fails before
// over-allocating) plus an aggregate ceiling across all entries. Both sit far
// above any real company package (inline JSON was historically capped at 64MB)
// yet far below what a bomb would need.
export const MAX_ZIP_ENTRY_DECOMPRESSED_BYTES = 256 * 1024 * 1024;
export const MAX_ZIP_TOTAL_DECOMPRESSED_BYTES = 512 * 1024 * 1024;
// above any real company package yet far below what a bomb would need. These
// are only fallback defaults: the server import route passes explicit limits
// scaled from its configured upload cap. The per-entry ceiling stays at 512 MB
// even for large caps because a text or base64 entry beyond that cannot
// materialize as a JS string (V8's string length limit) regardless.
export const MAX_ZIP_ENTRY_DECOMPRESSED_BYTES = 512 * 1024 * 1024;
export const MAX_ZIP_TOTAL_DECOMPRESSED_BYTES = 1024 * 1024 * 1024;
export const binaryContentTypeByExtension: Record<string, string> = {
".gif": "image/gif",

View File

@ -1,12 +1,18 @@
import { describe, expect, it } from "vitest";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
DEFAULT_JSON_BODY_LIMIT,
PORTABLE_JSON_BODY_LIMIT,
PORTABLE_JSON_BODY_LIMIT_BYTES,
PORTABLE_ZIP_UPLOAD_LIMIT_BYTES,
} from "../http/body-limits.js";
describe("HTTP body limits", () => {
afterEach(() => {
vi.unstubAllEnvs();
vi.resetModules();
});
it("keeps the global JSON parser at the established ceiling", () => {
expect(DEFAULT_JSON_BODY_LIMIT).toBe("10mb");
});
@ -16,4 +22,34 @@ describe("HTTP body limits", () => {
expect(PORTABLE_JSON_BODY_LIMIT_BYTES).toBe(64 * 1024 * 1024);
expect(PORTABLE_JSON_BODY_LIMIT_BYTES).toBeGreaterThan(10 * 1024 * 1024);
});
it("accepts large real-world compressed company packages by default", () => {
expect(PORTABLE_ZIP_UPLOAD_LIMIT_BYTES).toBe(1024 * 1024 * 1024);
});
it("lets operators override the zip upload cap via PAPERCLIP_IMPORT_ZIP_MAX_BYTES", async () => {
vi.stubEnv("PAPERCLIP_IMPORT_ZIP_MAX_BYTES", String(2 * 1024 * 1024 * 1024));
vi.resetModules();
const reloaded = await import("../http/body-limits.js");
expect(reloaded.PORTABLE_ZIP_UPLOAD_LIMIT_BYTES).toBe(2 * 1024 * 1024 * 1024);
});
it("falls back to the default when the env override is not a usable number", async () => {
for (const raw of ["unlimited", "0", "-5", "0.5"]) {
vi.stubEnv("PAPERCLIP_IMPORT_ZIP_MAX_BYTES", raw);
vi.resetModules();
const reloaded = await import("../http/body-limits.js");
expect(reloaded.PORTABLE_ZIP_UPLOAD_LIMIT_BYTES, `override ${JSON.stringify(raw)}`).toBe(
1024 * 1024 * 1024,
);
}
});
it("clamps an oversized override so derived guard math cannot overflow", async () => {
vi.stubEnv("PAPERCLIP_IMPORT_ZIP_MAX_BYTES", String(Number.MAX_VALUE));
vi.resetModules();
const reloaded = await import("../http/body-limits.js");
expect(reloaded.PORTABLE_ZIP_UPLOAD_LIMIT_BYTES).toBe(64 * 1024 * 1024 * 1024);
expect(Number.isSafeInteger(reloaded.PORTABLE_ZIP_UPLOAD_LIMIT_BYTES * 4)).toBe(true);
});
});

View File

@ -6,4 +6,18 @@ export const PORTABLE_JSON_BODY_LIMIT_BYTES = 64 * 1024 * 1024;
// application/zip) instead of an inflated inline JSON body. The compressed zip
// is roughly a third of the inline size, but the limit is kept generous so a
// large company package uploads in one request rather than truncating in transit.
export const PORTABLE_ZIP_UPLOAD_LIMIT_BYTES = 128 * 1024 * 1024;
// The whole upload is buffered in memory and unzipped in one pass, so this cap
// bounds peak per-import memory (roughly the compressed size plus the inflated
// package). Operators who need more (or less) headroom can override it with
// PAPERCLIP_IMPORT_ZIP_MAX_BYTES; the import route scales its decompression-bomb
// guards from whatever value is in effect.
// The override is clamped to [1 byte, 64 GiB]: a fractional value would floor
// to a zero-byte limit that rejects every upload, and an astronomically large
// one would overflow the derived 4x decompression guard. 64 GiB is far beyond
// what the in-memory import pipeline can serve anyway.
const MAX_ZIP_UPLOAD_LIMIT_OVERRIDE_BYTES = 64 * 1024 * 1024 * 1024;
const zipUploadLimitOverride = Math.floor(Number(process.env.PAPERCLIP_IMPORT_ZIP_MAX_BYTES));
export const PORTABLE_ZIP_UPLOAD_LIMIT_BYTES =
Number.isFinite(zipUploadLimitOverride) && zipUploadLimitOverride >= 1
? Math.min(zipUploadLimitOverride, MAX_ZIP_UPLOAD_LIMIT_OVERRIDE_BYTES)
: 1024 * 1024 * 1024;

View File

@ -6,7 +6,11 @@ import { z } from "zod";
import type { Db } from "@paperclipai/db";
import { agents as agentsTable } from "@paperclipai/db";
import type { CompanyPortabilityImportResult } from "@paperclipai/shared";
import { readZipArchive } from "@paperclipai/shared/portability-zip";
import {
MAX_ZIP_ENTRY_DECOMPRESSED_BYTES,
MAX_ZIP_TOTAL_DECOMPRESSED_BYTES,
readZipArchive,
} from "@paperclipai/shared/portability-zip";
import {
DEFAULT_FEEDBACK_DATA_SHARING_TERMS_VERSION,
companyArtifactsQuerySchema,
@ -131,7 +135,9 @@ async function resolveImportPayload(req: Request, res: Response): Promise<unknow
} catch (error) {
if (error instanceof multer.MulterError) {
if (error.code === "LIMIT_FILE_SIZE") {
throw unprocessable(`Import package exceeds ${PORTABLE_ZIP_UPLOAD_LIMIT_BYTES} bytes`);
throw unprocessable(
`Import package exceeds the ${Math.floor(PORTABLE_ZIP_UPLOAD_LIMIT_BYTES / (1024 * 1024))} MB upload limit`,
);
}
throw badRequest(error.message);
}
@ -157,7 +163,16 @@ async function resolveImportPayload(req: Request, res: Response): Promise<unknow
}
let archive: Awaited<ReturnType<typeof readZipArchive>>;
try {
archive = await readZipArchive(zipBytes);
// Scale the bomb guards from the configured upload cap so a legitimately
// compressible package under the cap never trips them (~4x covers dense
// text; the per-entry ceiling is bounded by V8's string limit regardless).
archive = await readZipArchive(zipBytes, {
maxEntryDecompressedBytes: MAX_ZIP_ENTRY_DECOMPRESSED_BYTES,
maxTotalDecompressedBytes: Math.max(
MAX_ZIP_TOTAL_DECOMPRESSED_BYTES,
PORTABLE_ZIP_UPLOAD_LIMIT_BYTES * 4,
),
});
} catch (error) {
throw badRequest(`Import package could not be read: ${errorMessage(error)}`);
}

View File

@ -2,7 +2,8 @@ import type { CompanyPortabilityFileEntry } from "@paperclipai/shared";
// Inline imports post the whole parsed package as one JSON body, so oversized
// packages must be blocked before the request is built. Packages past this
// limit go through the CLI folder import today; a blob-store relay is planned.
// limit go through the zip upload path (a far higher, operator-configurable
// server cap); a blob-store relay is planned for arbitrarily large migrations.
export const INLINE_IMPORT_MAX_BYTES = 56 * 1024 * 1024;
export function isBlobStoreFilePath(filePath: string): boolean {

View File

@ -420,7 +420,7 @@ describe("CompanyImport", () => {
await flushReact();
expect(container.textContent).toContain("Preview failed: stream disconnected");
expect(container.textContent).toContain("Retry, or use the CLI folder import");
expect(container.textContent).toContain("Retry, or re-export the package without large attachments");
expect(mockPushToast).toHaveBeenCalledWith(expect.objectContaining({ tone: "error" }));
// Changing the package supersedes the failed request: the error panel resets.

View File

@ -1779,7 +1779,7 @@ export function CompanyImport() {
{previewMutation.error instanceof Error
? previewMutation.error.message
: "the request did not complete."}{" "}
Retry, or use the CLI folder import for very large packages.
Retry, or re-export the package without large attachments to shrink it.
</p>
</div>
)}