RomanNum3ral
/
paperclip
mirror of https://github.com/paperclipai/paperclip.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Upload company import packages as compressed zip uploads (fix large-company imports through Cloud) (#10531) 

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Company Import (#10507, hardened in #10523) lets a user upload a
company package on the Import page
> - The page expanded the user's `.zip` into a files map and POSTed it
as ONE inline JSON body — ~40MB for a real company because attachment
blobs get base64-inflated
> - On Paperclip Cloud that body travels browser → harness proxy →
tenant, where it truncated in transit → body-parser 400 → the browser
saw "Failed to fetch", and nothing imported
> - Two compounding causes: the giant inline body itself, and the board
async opt-in riding an `x-paperclip-cloud-*` header that the Cloud
harness strips as anti-spoofing (so async never engaged and the import
held one fragile synchronous connection)
> - This pull request uploads the raw compressed `.zip` as a multipart
request (about a third the size, already compressed) parsed server-side
into the same bundle the importer consumes, and moves the async opt-in
to a proxy-safe `?async=1`
> - The benefit is that a large-company import actually completes
through Cloud: a small compressed upload, a real async job that survives
dropped connections

## Linked Issues or Issue Description

- Refs #10507 / #10523 (Import/Export and its hardening). No open issue;
problem described above (large-company browser import through a proxy:
inline JSON body truncates → 400 → "Failed to fetch"; async opt-in
header stripped by the front door → async never engages).

## What Changed

- **Multipart zip transport.** The Import page uploads the raw `File` as
`multipart/form-data` (field `package`, import options in a JSON `meta`
field); the server unzips it into `{ rootPath, files }` and runs the
exact existing preview/import logic. The `application/json` inline path
is byte-identical for CLI/programmatic callers. Bare `application/zip`
(meta via `?meta=`) is also accepted for programmatic use.
- **Shared node zip reader.** `packages/shared/src/portability-zip.ts`
(node-only subpath, not re-exported to the browser bundle — same pattern
as `portability-hash.ts`); the CLI's `zip.ts` becomes a thin re-export.
Identical codec (STORE + DEFLATE via `inflateRawSync`, rejects data
descriptors/zip64).
- **Proxy-safe async signal.** `wantsAsyncImport` = `?async=1` (board
browsers, survives the harness) OR the existing
`x-paperclip-cloud-async-import` header (cloud tenants, set
server-side). The UI async client now uses `?async=1`. Backward
compatible.
- **Size + preflight.** New `PORTABLE_ZIP_UPLOAD_LIMIT_BYTES = 128MB`;
the inline 56MB preflight no longer gates the zip path (it shows the
compressed size instead). Async submit/poll/resume, the duplicate-guard
fingerprint (now over the resolved bundle), pause-on-import,
progress/error panels, and activation all apply to the multipart path.
- OpenAPI documents json + multipart + zip bodies and the `async` query
param.

## Verification

- Full typecheck chain (shared, server, ui, cli) clean.
- 152 tests across 8 files: new `portability-zip.test.ts`
(STORE/DEFLATE/base64-blob byte-exact round-trip, truncation throws,
data-descriptor rejection); `company-portability-routes.test.ts` +7
(multipart import+preview equals the inline bundle; async multipart
202→poll→success; board async via `?async=1` with no cloud header;
cloud-tenant async via header; sync fallback with neither; truncated-zip
400, nothing imported); `CompanyImport.test.tsx` asserts the local zip
sends the raw File and the inline preflight no longer blocks;
`openapi-routes.test.ts` green.
- NOT yet measured: the end-to-end browser upload through the live Cloud
harness — verified on staging after deploy before closing out.

## Risks

- Import semantics unchanged — only transport changed; the JSON inline
path is byte-identical, the cloud-tenant header async path untouched.
Multipart parsing is server-side (memory-bound: a ~13MB zip → ~30MB
files map, fine on the server).
- The bare `application/zip` path is programmatic-only and covered by
content-type dispatch but not a dedicated route test (the multipart path
is).

## Model Used

- Claude Fable 5 (`claude-fable-5`, Anthropic), Claude Code CLI,
extended thinking + tool use; root-caused against live logs/DB and the
harness proxy source.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Devin Foley 2026-07-30 18:48:22 -07:00 committed by GitHub
parent 740554acc6
commit 5ec7ce76e5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
11 changed files with 1172 additions and 274 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

162
cli/src/commands/client/zip.ts
View File

@ -1,152 +1,10 @@
import { inflateRawSync } from "node:zlib";
import path from "node:path";
import type { CompanyPortabilityFileEntry } from "@paperclipai/shared";
const textDecoder = new TextDecoder();
// ignoreBOM keeps a leading BOM in the decoded text so text entries
// re-encode to their original bytes; fatal surfaces invalid UTF-8.
const strictTextDecoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
export const binaryContentTypeByExtension: Record<string, string> = {
".gif": "image/gif",
".jpeg": "image/jpeg",
".jpg": "image/jpeg",
".png": "image/png",
".svg": "image/svg+xml",
".webp": "image/webp",
};
function normalizeArchivePath(pathValue: string) {
return pathValue
.replace(/\\/g, "/")
.split("/")
.filter(Boolean)
.join("/");
}
function readUint16(source: Uint8Array, offset: number) {
return source[offset]! | (source[offset + 1]! << 8);
}
function readUint32(source: Uint8Array, offset: number) {
return (
source[offset]! |
(source[offset + 1]! << 8) |
(source[offset + 2]! << 16) |
(source[offset + 3]! << 24)
) >>> 0;
}
function sharedArchiveRoot(paths: string[]) {
if (paths.length === 0) return null;
const firstSegments = paths
.map((entry) => normalizeArchivePath(entry).split("/").filter(Boolean))
.filter((parts) => parts.length > 0);
if (firstSegments.length === 0) return null;
const candidate = firstSegments[0]![0]!;
return firstSegments.every((parts) => parts.length > 1 && parts[0] === candidate)
? candidate
: null;
}
export function isBlobStorePath(pathValue: string) {
return /(^|\/)blobs\/[^/]+$/.test(normalizeArchivePath(pathValue));
}
function decodeStrictUtf8(bytes: Uint8Array): string | null {
let text: string;
try {
text = strictTextDecoder.decode(bytes);
} catch {
return null;
}
return Buffer.from(text, "utf8").equals(Buffer.from(bytes)) ? text : null;
}
export function bytesToPortableFileEntry(pathValue: string, bytes: Uint8Array): CompanyPortabilityFileEntry {
// Content-addressed blob entries are opaque bytes regardless of extension.
if (isBlobStorePath(pathValue)) {
return { encoding: "base64", data: Buffer.from(bytes).toString("base64"), contentType: "application/octet-stream" };
}
const contentType = binaryContentTypeByExtension[path.extname(pathValue).toLowerCase()];
if (contentType) {
return { encoding: "base64", data: Buffer.from(bytes).toString("base64"), contentType };
}
const text = decodeStrictUtf8(bytes);
if (text !== null) return text;
// Bytes that are not valid UTF-8 must not be decoded lossily; fall back
// to base64 so they round-trip exactly.
return { encoding: "base64", data: Buffer.from(bytes).toString("base64"), contentType: "application/octet-stream" };
}
async function inflateZipEntry(compressionMethod: number, bytes: Uint8Array) {
if (compressionMethod === 0) return bytes;
if (compressionMethod !== 8) {
throw new Error("Unsupported zip archive: only STORE and DEFLATE entries are supported.");
}
return new Uint8Array(inflateRawSync(bytes));
}
export async function readZipArchive(source: ArrayBuffer | Uint8Array): Promise<{
rootPath: string | null;
files: Record<string, CompanyPortabilityFileEntry>;
}> {
const bytes = source instanceof Uint8Array ? source : new Uint8Array(source);
const entries: Array<{ path: string; body: CompanyPortabilityFileEntry }> = [];
let offset = 0;
while (offset + 4 <= bytes.length) {
const signature = readUint32(bytes, offset);
if (signature === 0x02014b50 || signature === 0x06054b50) break;
if (signature !== 0x04034b50) {
throw new Error("Invalid zip archive: unsupported local file header.");
}
if (offset + 30 > bytes.length) {
throw new Error("Invalid zip archive: truncated local file header.");
}
const generalPurposeFlag = readUint16(bytes, offset + 6);
const compressionMethod = readUint16(bytes, offset + 8);
const compressedSize = readUint32(bytes, offset + 18);
const fileNameLength = readUint16(bytes, offset + 26);
const extraFieldLength = readUint16(bytes, offset + 28);
if ((generalPurposeFlag & 0x0008) !== 0) {
throw new Error("Unsupported zip archive: data descriptors are not supported.");
}
const nameOffset = offset + 30;
const bodyOffset = nameOffset + fileNameLength + extraFieldLength;
const bodyEnd = bodyOffset + compressedSize;
if (bodyEnd > bytes.length) {
throw new Error("Invalid zip archive: truncated file contents.");
}
const rawArchivePath = textDecoder.decode(bytes.slice(nameOffset, nameOffset + fileNameLength));
const archivePath = normalizeArchivePath(rawArchivePath);
const isDirectoryEntry = /\/$/.test(rawArchivePath.replace(/\\/g, "/"));
if (archivePath && !isDirectoryEntry) {
const entryBytes = await inflateZipEntry(compressionMethod, bytes.slice(bodyOffset, bodyEnd));
entries.push({
path: archivePath,
body: bytesToPortableFileEntry(archivePath, entryBytes),
});
}
offset = bodyEnd;
}
const rootPath = sharedArchiveRoot(entries.map((entry) => entry.path));
const files: Record<string, CompanyPortabilityFileEntry> = {};
for (const entry of entries) {
const normalizedPath =
rootPath && entry.path.startsWith(`${rootPath}/`)
? entry.path.slice(rootPath.length + 1)
: entry.path;
if (!normalizedPath) continue;
files[normalizedPath] = entry.body;
}
return { rootPath, files };
}
// The node-side portability zip reader lives in @paperclipai/shared so the
// server can consume the same codec (a raw uploaded zip is unzipped into the
// exact `{ rootPath, files }` bundle the inline import source carries). This
// module re-exports it to keep the CLI's existing import paths stable.
export {
binaryContentTypeByExtension,
bytesToPortableFileEntry,
isBlobStorePath,
readZipArchive,
} from "@paperclipai/shared/portability-zip";

303
packages/shared/src/portability-zip.test.ts Normal file
View File

@ -0,0 +1,303 @@
import { deflateRawSync } from "node:zlib";
import { describe, expect, it } from "vitest";
import {
bytesToPortableFileEntry,
isBlobStorePath,
readZipArchive,
} from "./portability-zip.js";
// A minimal, faithful zip writer so the node reader can be round-tripped
// against both STORE (method 0) and DEFLATE (method 8) entries. The layout
// matches the browser writer in ui/src/lib/zip.ts: local file headers, then a
// central directory, then the end-of-central-directory record.
const crcTable = (() => {
const table = new Uint32Array(256);
for (let i = 0; i < 256; i += 1) {
let crc = i;
for (let bit = 0; bit < 8; bit += 1) {
crc = (crc & 1) === 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
}
table[i] = crc >>> 0;
}
return table;
})();
function crc32(bytes: Uint8Array) {
let crc = 0xffffffff;
for (const byte of bytes) crc = (crc >>> 8) ^ crcTable[(crc ^ byte) & 0xff]!;
return (crc ^ 0xffffffff) >>> 0;
}
interface ZipInput {
path: string;
bytes: Uint8Array;
method?: 0 | 8;
}
function buildZip(entries: ZipInput[], rootPath: string): Uint8Array {
const encoder = new TextEncoder();
const localChunks: Buffer[] = [];
const centralChunks: Buffer[] = [];
let localOffset = 0;
for (const entry of entries) {
const method = entry.method ?? 0;
const fileName = encoder.encode(`${rootPath}/${entry.path}`);
const checksum = crc32(entry.bytes);
const body = method === 8 ? deflateRawSync(Buffer.from(entry.bytes)) : Buffer.from(entry.bytes);
const localHeader = Buffer.alloc(30 + fileName.length);
localHeader.writeUInt32LE(0x04034b50, 0);
localHeader.writeUInt16LE(20, 4);
localHeader.writeUInt16LE(0x0800, 6);
localHeader.writeUInt16LE(method, 8);
localHeader.writeUInt32LE(checksum, 14);
localHeader.writeUInt32LE(body.length, 18);
localHeader.writeUInt32LE(entry.bytes.length, 22);
localHeader.writeUInt16LE(fileName.length, 26);
Buffer.from(fileName).copy(localHeader, 30);
const centralHeader = Buffer.alloc(46 + fileName.length);
centralHeader.writeUInt32LE(0x02014b50, 0);
centralHeader.writeUInt16LE(20, 4);
centralHeader.writeUInt16LE(20, 6);
centralHeader.writeUInt16LE(0x0800, 8);
centralHeader.writeUInt16LE(method, 10);
centralHeader.writeUInt32LE(checksum, 16);
centralHeader.writeUInt32LE(body.length, 20);
centralHeader.writeUInt32LE(entry.bytes.length, 24);
centralHeader.writeUInt16LE(fileName.length, 28);
centralHeader.writeUInt32LE(localOffset, 42);
Buffer.from(fileName).copy(centralHeader, 46);
localChunks.push(localHeader, body);
centralChunks.push(centralHeader);
localOffset += localHeader.length + body.length;
}
const centralDirectory = Buffer.concat(centralChunks);
const eocd = Buffer.alloc(22);
eocd.writeUInt32LE(0x06054b50, 0);
eocd.writeUInt16LE(entries.length, 8);
eocd.writeUInt16LE(entries.length, 10);
eocd.writeUInt32LE(centralDirectory.length, 12);
eocd.writeUInt32LE(localOffset, 16);
return new Uint8Array(Buffer.concat([...localChunks, centralDirectory, eocd]));
}
describe("isBlobStorePath", () => {
it("matches blobs/ entries at the archive root and under a package root", () => {
expect(isBlobStorePath("blobs/4f2d1c9a")).toBe(true);
expect(isBlobStorePath("paperclip-demo/blobs/4f2d1c9a")).toBe(true);
expect(isBlobStorePath("tasks/pap-1/TASK.md")).toBe(false);
});
});
describe("bytesToPortableFileEntry", () => {
it("keeps blobs/ entries as base64 octet streams regardless of extension", () => {
const bytes = new Uint8Array([0x00, 0x01, 0x80, 0xfe, 0xff]);
expect(bytesToPortableFileEntry("blobs/4f2d1c9a", bytes)).toEqual({
encoding: "base64",
data: Buffer.from(bytes).toString("base64"),
contentType: "application/octet-stream",
});
});
it("decodes valid UTF-8 entries to text and falls back to base64 for invalid bytes", () => {
const text = new TextEncoder().encode("# Notes\n\ncafé ✅\n");
expect(bytesToPortableFileEntry("tasks/pap-1/TASK.md", text)).toBe("# Notes\n\ncafé ✅\n");
const invalid = new Uint8Array([0x68, 0x69, 0xff, 0xfe, 0xc0]);
expect(bytesToPortableFileEntry("tasks/pap-1/raw", invalid)).toEqual({
encoding: "base64",
data: Buffer.from(invalid).toString("base64"),
contentType: "application/octet-stream",
});
});
});
describe("readZipArchive", () => {
it("round-trips STORE, DEFLATE, and base64 blob entries byte-exactly and strips the shared root", async () => {
const blobBytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x00, 0xff, 0x13, 0x37]);
// A text body large and repetitive enough that DEFLATE actually shrinks it,
// so the DEFLATE decode path is exercised, not just written.
const deflated = `# Weekly report\n${"paperclip ".repeat(512)}\n`;
const archive = buildZip(
[
{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n"), method: 0 },
{ path: "reports/weekly.md", bytes: new TextEncoder().encode(deflated), method: 8 },
{ path: "blobs/4f2d1c9a", bytes: blobBytes, method: 0 },
],
"paperclip-demo",
);
await expect(readZipArchive(archive)).resolves.toEqual({
rootPath: "paperclip-demo",
files: {
"COMPANY.md": "---\nname: Demo\n---\n",
"reports/weekly.md": deflated,
"blobs/4f2d1c9a": {
encoding: "base64",
data: Buffer.from(blobBytes).toString("base64"),
contentType: "application/octet-stream",
},
},
});
});
it("throws on a truncated archive so a partial upload fails closed", async () => {
const archive = buildZip(
[{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n") }],
"paperclip-demo",
);
// Chop the tail so a declared entry body runs past the end of the buffer.
const truncated = archive.slice(0, 40);
await expect(readZipArchive(truncated)).rejects.toThrow(/truncated|Invalid zip/i);
});
it("rejects data-descriptor entries the writer never emits", async () => {
const archive = buildZip(
[{ path: "COMPANY.md", bytes: new TextEncoder().encode("hi") }],
"paperclip-demo",
);
// Flip bit 0x0008 in the local header's general-purpose flag (offset 6).
archive[6] = archive[6]! | 0x08;
await expect(readZipArchive(archive)).rejects.toThrow(/data descriptors/i);
});
// Locate the first central-directory file header so a test can lop off the
// whole directory + EOCD, leaving only intact local entries.
function centralDirectoryOffset(archive: Uint8Array): number {
for (let i = 0; i + 4 <= archive.length; i += 1) {
if (archive[i] === 0x50 && archive[i + 1] === 0x4b && archive[i + 2] === 0x01 && archive[i + 3] === 0x02) {
return i;
}
}
return -1;
}
it("rejects an archive truncated before the central directory", async () => {
const archive = buildZip(
[
{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n") },
{ path: "agents/ceo/AGENTS.md", bytes: new TextEncoder().encode("---\nname: CEO\n---\n") },
],
"paperclip-demo",
);
// Keep every local entry intact but drop the directory + EOCD, mimicking an
// upload cut at a record boundary. The reader must not import the fragment.
const withoutDirectory = archive.slice(0, centralDirectoryOffset(archive));
await expect(readZipArchive(withoutDirectory)).rejects.toThrow(/truncated before the central directory/i);
});
it("rejects an archive whose end-of-central-directory record is missing", async () => {
const archive = buildZip(
[{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n") }],
"paperclip-demo",
);
// The central directory survives but the trailing 22-byte EOCD is gone.
await expect(readZipArchive(archive.slice(0, archive.length - 22))).rejects.toThrow(
/end-of-central-directory/i,
);
});
it("rejects an archive whose central directory count does not match the entries read", async () => {
const archive = buildZip(
[{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n") }],
"paperclip-demo",
);
// Overstate the total-entries field (EOCD offset +10 → last 12 bytes in) so a
// silently-dropped central-directory record is caught.
archive[archive.length - 12] = 5;
archive[archive.length - 11] = 0;
await expect(readZipArchive(archive)).rejects.toThrow(/central directory declares 5 entries/i);
});
it("rejects a truncated archive re-terminated with a forged EOCD that matches the surviving entries", async () => {
// The exact silent-partial-import this reader guards against: an archive is
// cut after a complete leading local entry (losing the real central directory
// and EOCD), then re-terminated with a hand-forged 22-byte EOCD whose entry
// count matches the surviving local entry. An entry-count-only check would
// wave it through; validating the central directory it points at must not.
const archive = buildZip(
[{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n") }],
"paperclip-demo",
);
// Keep only the intact local entry (everything before the first central record).
const localOnly = archive.slice(0, centralDirectoryOffset(archive));
const forgedEocd = Buffer.alloc(22);
forgedEocd.writeUInt32LE(0x06054b50, 0);
forgedEocd.writeUInt16LE(1, 8); // entries on this disk
forgedEocd.writeUInt16LE(1, 10); // total entries — matches the one surviving local entry
forgedEocd.writeUInt32LE(0, 12); // central directory size (forged)
forgedEocd.writeUInt32LE(0, 16); // central directory offset (forged)
const forged = new Uint8Array(Buffer.concat([Buffer.from(localOnly), forgedEocd]));
await expect(readZipArchive(forged)).rejects.toThrow(/central directory location is inconsistent/i);
});
it("rejects a forged EOCD whose central directory offset points at non-directory bytes", async () => {
const archive = buildZip(
[{ path: "COMPANY.md", bytes: new TextEncoder().encode("---\nname: Demo\n---\n") }],
"paperclip-demo",
);
const firstEntryOnly = archive.slice(0, centralDirectoryOffset(archive));
// A forged EOCD whose offset+size abut the record (passing the location
// check) but point into the local entry, which carries no directory signature.
const forgedEocd = Buffer.alloc(22);
forgedEocd.writeUInt32LE(0x06054b50, 0);
forgedEocd.writeUInt16LE(1, 8);
forgedEocd.writeUInt16LE(1, 10);
forgedEocd.writeUInt32LE(46, 12); // size
forgedEocd.writeUInt32LE(firstEntryOnly.length - 46, 16); // start = eocdOffset - size
const forged = new Uint8Array(Buffer.concat([Buffer.from(firstEntryOnly), forgedEocd]));
await expect(readZipArchive(forged)).rejects.toThrow(/malformed central directory record/i);
});
it("rejects two entries that normalize to the same path instead of silently overwriting", async () => {
const archive = buildZip(
[
{ path: "docs/x.md", bytes: new TextEncoder().encode("first") },
{ path: "docs//x.md", bytes: new TextEncoder().encode("second") },
],
"paperclip-demo",
);
await expect(readZipArchive(archive)).rejects.toThrow(/duplicate entry path "docs\/x\.md"/i);
});
it("bounds a highly compressible DEFLATE entry at the per-entry decompressed limit", async () => {
// Compresses tiny but expands to 8 KiB; a 1 KiB cap must reject it before it
// materializes. Real packages sit far under the 256 MB production default.
const bomb = new TextEncoder().encode("a".repeat(8 * 1024));
const archive = buildZip([{ path: "bomb.txt", bytes: bomb, method: 8 }], "paperclip-demo");
await expect(
readZipArchive(archive, { maxEntryDecompressedBytes: 1024, maxTotalDecompressedBytes: 1 << 30 }),
).rejects.toThrow(/per-entry limit/i);
});
it("bounds a stored entry at the per-entry decompressed limit", async () => {
const stored = new TextEncoder().encode("b".repeat(4 * 1024));
const archive = buildZip([{ path: "big.bin", bytes: stored, method: 0 }], "paperclip-demo");
await expect(
readZipArchive(archive, { maxEntryDecompressedBytes: 1024, maxTotalDecompressedBytes: 1 << 30 }),
).rejects.toThrow(/per-entry limit/i);
});
it("bounds the aggregate decompressed size across many entries", async () => {
const chunk = new TextEncoder().encode("c".repeat(600));
const archive = buildZip(
[
{ path: "a.txt", bytes: chunk, method: 0 },
{ path: "b.txt", bytes: chunk, method: 0 },
],
"paperclip-demo",
);
// Each entry is under the per-entry cap, but together they cross the total.
await expect(
readZipArchive(archive, { maxEntryDecompressedBytes: 4096, maxTotalDecompressedBytes: 1000 }),
).rejects.toThrow(/exceed the 1000-byte limit/i);
});
});

308
packages/shared/src/portability-zip.ts Normal file
View File

@ -0,0 +1,308 @@
import { inflateRawSync } from "node:zlib";
import path from "node:path";
import type { CompanyPortabilityFileEntry } from "./types/company-portability.js";
// Node-side reader for the company portability zip package. It produces the
// exact `{ rootPath, files }` shape the inline import source carries, so the
// server can accept a raw uploaded zip and feed the importer the same bundle
// the browser used to expand and post as inline JSON. The browser has its own
// reader in `ui/src/lib/zip.ts` (DecompressionStream); this one uses node zlib.
// Both must stay byte-compatible with the writer in `ui/src/lib/zip.ts`.
const textDecoder = new TextDecoder();
// ignoreBOM keeps a leading BOM in the decoded text so text entries
// re-encode to their original bytes; fatal surfaces invalid UTF-8.
const strictTextDecoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
// Decompression-bomb guards. The compressed upload is already capped upstream
// (multer/express.raw at PORTABLE_ZIP_UPLOAD_LIMIT_BYTES), but DEFLATE lets a
// small compressed archive expand to gigabytes. Bound the expansion so a
// malicious highly-compressible package cannot exhaust server memory: a
// per-entry ceiling (passed to zlib as maxOutputLength, so it fails before
// over-allocating) plus an aggregate ceiling across all entries. Both sit far
// above any real company package (inline JSON was historically capped at 64MB)
// yet far below what a bomb would need.
export const MAX_ZIP_ENTRY_DECOMPRESSED_BYTES = 256 * 1024 * 1024;
export const MAX_ZIP_TOTAL_DECOMPRESSED_BYTES = 512 * 1024 * 1024;
export const binaryContentTypeByExtension: Record<string, string> = {
".gif": "image/gif",
".jpeg": "image/jpeg",
".jpg": "image/jpeg",
".png": "image/png",
".svg": "image/svg+xml",
".webp": "image/webp",
};
function normalizeArchivePath(pathValue: string) {
return pathValue
.replace(/\\/g, "/")
.split("/")
.filter(Boolean)
.join("/");
}
function readUint16(source: Uint8Array, offset: number) {
return source[offset]! | (source[offset + 1]! << 8);
}
function readUint32(source: Uint8Array, offset: number) {
return (
source[offset]! |
(source[offset + 1]! << 8) |
(source[offset + 2]! << 16) |
(source[offset + 3]! << 24)
) >>> 0;
}
const LOCAL_FILE_SIGNATURE = 0x04034b50;
const CENTRAL_DIRECTORY_SIGNATURE = 0x02014b50;
const EOCD_SIGNATURE = 0x06054b50;
// Locate the end-of-central-directory record by scanning back from the tail.
// The record is 22 bytes plus an optional trailing comment (max 0xffff), so the
// signature lives within the last 22 + 0xffff bytes; returns -1 when absent
// (a truncated or non-zip upload), which the reader treats as fail-closed.
function findEndOfCentralDirectoryOffset(bytes: Uint8Array): number {
const minOffset = Math.max(0, bytes.length - (22 + 0xffff));
for (let offset = bytes.length - 22; offset >= minOffset; offset -= 1) {
if (readUint32(bytes, offset) === EOCD_SIGNATURE) return offset;
}
return -1;
}
// Fully validate the central directory the EOCD advertises against the local
// entries actually read from the archive body. Trusting only the EOCD's entry
// count is not enough: a truncated archive with a forged 22-byte EOCD whose
// count matches the surviving local entries would otherwise be accepted, and the
// importer would silently process a partial company package (dropping agents,
// issues, and so on). This walks the real central directory and requires that:
// • it is fully present and sits immediately before the EOCD (no gap, no
// pointer past the buffer — a truncated tail fails here);
// • every record carries the central-directory signature and its lengths sum
// to exactly the declared directory size; and
// • every record references a real local file header, and the record count
// equals both the EOCD's declared count and the local entries parsed.
function validateCentralDirectory(bytes: Uint8Array, eocdOffset: number, localHeaderCount: number) {
const declaredEntryCount = readUint16(bytes, eocdOffset + 10);
const centralDirectorySize = readUint32(bytes, eocdOffset + 12);
const centralDirectoryStart = readUint32(bytes, eocdOffset + 16);
if (centralDirectoryStart > eocdOffset || centralDirectoryStart + centralDirectorySize !== eocdOffset) {
throw new Error(
"Invalid zip archive: central directory location is inconsistent (truncated or forged).",
);
}
const directoryEnd = centralDirectoryStart + centralDirectorySize;
let cursor = centralDirectoryStart;
let recordCount = 0;
while (cursor < directoryEnd) {
if (cursor + 46 > directoryEnd || readUint32(bytes, cursor) !== CENTRAL_DIRECTORY_SIGNATURE) {
throw new Error("Invalid zip archive: malformed central directory record.");
}
const fileNameLength = readUint16(bytes, cursor + 28);
const extraFieldLength = readUint16(bytes, cursor + 30);
const commentLength = readUint16(bytes, cursor + 32);
const localHeaderOffset = readUint32(bytes, cursor + 42);
if (localHeaderOffset + 4 > bytes.length || readUint32(bytes, localHeaderOffset) !== LOCAL_FILE_SIGNATURE) {
throw new Error("Invalid zip archive: central directory references a missing local entry.");
}
cursor += 46 + fileNameLength + extraFieldLength + commentLength;
recordCount += 1;
}
if (cursor !== directoryEnd) {
throw new Error("Invalid zip archive: central directory size does not match its records.");
}
if (recordCount !== declaredEntryCount || recordCount !== localHeaderCount) {
throw new Error(
`Invalid zip archive: central directory declares ${declaredEntryCount} entries but ${localHeaderCount} local entries were read (truncated or corrupt).`,
);
}
}
function sharedArchiveRoot(paths: string[]) {
if (paths.length === 0) return null;
const firstSegments = paths
.map((entry) => normalizeArchivePath(entry).split("/").filter(Boolean))
.filter((parts) => parts.length > 0);
if (firstSegments.length === 0) return null;
const candidate = firstSegments[0]![0]!;
return firstSegments.every((parts) => parts.length > 1 && parts[0] === candidate)
? candidate
: null;
}
export function isBlobStorePath(pathValue: string) {
return /(^|\/)blobs\/[^/]+$/.test(normalizeArchivePath(pathValue));
}
function decodeStrictUtf8(bytes: Uint8Array): string | null {
let text: string;
try {
text = strictTextDecoder.decode(bytes);
} catch {
return null;
}
return Buffer.from(text, "utf8").equals(Buffer.from(bytes)) ? text : null;
}
export function bytesToPortableFileEntry(pathValue: string, bytes: Uint8Array): CompanyPortabilityFileEntry {
// Content-addressed blob entries are opaque bytes regardless of extension.
if (isBlobStorePath(pathValue)) {
return { encoding: "base64", data: Buffer.from(bytes).toString("base64"), contentType: "application/octet-stream" };
}
const contentType = binaryContentTypeByExtension[path.extname(pathValue).toLowerCase()];
if (contentType) {
return { encoding: "base64", data: Buffer.from(bytes).toString("base64"), contentType };
}
const text = decodeStrictUtf8(bytes);
if (text !== null) return text;
// Bytes that are not valid UTF-8 must not be decoded lossily; fall back
// to base64 so they round-trip exactly.
return { encoding: "base64", data: Buffer.from(bytes).toString("base64"), contentType: "application/octet-stream" };
}
// Size caps applied while expanding an archive. Callers use the module defaults;
// the limits are parameterizable so the guard can be exercised in tests without
// allocating hundreds of megabytes.
export interface ReadZipArchiveLimits {
maxEntryDecompressedBytes: number;
maxTotalDecompressedBytes: number;
}
const DEFAULT_ZIP_LIMITS: ReadZipArchiveLimits = {
maxEntryDecompressedBytes: MAX_ZIP_ENTRY_DECOMPRESSED_BYTES,
maxTotalDecompressedBytes: MAX_ZIP_TOTAL_DECOMPRESSED_BYTES,
};
function inflateZipEntry(compressionMethod: number, bytes: Uint8Array, maxEntryDecompressedBytes: number) {
if (compressionMethod === 0) {
if (bytes.length > maxEntryDecompressedBytes) {
throw new Error(
`Unsupported zip archive: a stored entry exceeds the ${maxEntryDecompressedBytes}-byte per-entry limit.`,
);
}
return bytes;
}
if (compressionMethod !== 8) {
throw new Error("Unsupported zip archive: only STORE and DEFLATE entries are supported.");
}
try {
// maxOutputLength makes zlib throw (ERR_BUFFER_TOO_LARGE) before it allocates
// past the per-entry ceiling, so a bomb entry never materializes in memory.
return new Uint8Array(inflateRawSync(bytes, { maxOutputLength: maxEntryDecompressedBytes }));
} catch (error) {
if ((error as NodeJS.ErrnoException | undefined)?.code === "ERR_BUFFER_TOO_LARGE") {
throw new Error(
`Unsupported zip archive: a compressed entry expands beyond the ${maxEntryDecompressedBytes}-byte per-entry limit.`,
);
}
throw error;
}
}
export async function readZipArchive(
source: ArrayBuffer | Uint8Array,
limits: ReadZipArchiveLimits = DEFAULT_ZIP_LIMITS,
): Promise<{
rootPath: string | null;
files: Record<string, CompanyPortabilityFileEntry>;
}> {
const { maxEntryDecompressedBytes, maxTotalDecompressedBytes } = limits;
const bytes = source instanceof Uint8Array ? source : new Uint8Array(source);
const entries: Array<{ path: string; body: CompanyPortabilityFileEntry }> = [];
let offset = 0;
// Count every local file header (including directory entries) so the tally can
// be reconciled against the central directory below; guard total expansion so
// a bomb split across many entries is still bounded.
let localHeaderCount = 0;
let totalDecompressedBytes = 0;
let reachedCentralDirectory = false;
while (offset + 4 <= bytes.length) {
const signature = readUint32(bytes, offset);
if (signature === CENTRAL_DIRECTORY_SIGNATURE || signature === EOCD_SIGNATURE) {
reachedCentralDirectory = true;
break;
}
if (signature !== LOCAL_FILE_SIGNATURE) {
throw new Error("Invalid zip archive: unsupported local file header.");
}
if (offset + 30 > bytes.length) {
throw new Error("Invalid zip archive: truncated local file header.");
}
const generalPurposeFlag = readUint16(bytes, offset + 6);
const compressionMethod = readUint16(bytes, offset + 8);
const compressedSize = readUint32(bytes, offset + 18);
const fileNameLength = readUint16(bytes, offset + 26);
const extraFieldLength = readUint16(bytes, offset + 28);
if ((generalPurposeFlag & 0x0008) !== 0) {
throw new Error("Unsupported zip archive: data descriptors are not supported.");
}
const nameOffset = offset + 30;
const bodyOffset = nameOffset + fileNameLength + extraFieldLength;
const bodyEnd = bodyOffset + compressedSize;
if (bodyEnd > bytes.length) {
throw new Error("Invalid zip archive: truncated file contents.");
}
localHeaderCount += 1;
const rawArchivePath = textDecoder.decode(bytes.slice(nameOffset, nameOffset + fileNameLength));
const archivePath = normalizeArchivePath(rawArchivePath);
const isDirectoryEntry = /\/$/.test(rawArchivePath.replace(/\\/g, "/"));
if (archivePath && !isDirectoryEntry) {
const entryBytes = inflateZipEntry(compressionMethod, bytes.slice(bodyOffset, bodyEnd), maxEntryDecompressedBytes);
totalDecompressedBytes += entryBytes.length;
if (totalDecompressedBytes > maxTotalDecompressedBytes) {
throw new Error(
`Unsupported zip archive: decompressed contents exceed the ${maxTotalDecompressedBytes}-byte limit.`,
);
}
entries.push({
path: archivePath,
body: bytesToPortableFileEntry(archivePath, entryBytes),
});
}
offset = bodyEnd;
}
// A complete archive always ends with a central directory after its local
// entries. If the scan ran off the end of the buffer without reaching one, the
// upload was truncated at a record boundary — fail closed rather than import a
// leading fragment. Then fully validate the central directory the EOCD points
// at so a truncated tail with a forged EOCD (whose count happens to match the
// surviving entries) cannot smuggle in a partial import.
if (!reachedCentralDirectory) {
throw new Error("Invalid zip archive: truncated before the central directory.");
}
const eocdOffset = findEndOfCentralDirectoryOffset(bytes);
if (eocdOffset === -1) {
throw new Error("Invalid zip archive: missing end-of-central-directory record.");
}
validateCentralDirectory(bytes, eocdOffset, localHeaderCount);
const rootPath = sharedArchiveRoot(entries.map((entry) => entry.path));
const files: Record<string, CompanyPortabilityFileEntry> = {};
for (const entry of entries) {
const normalizedPath =
rootPath && entry.path.startsWith(`${rootPath}/`)
? entry.path.slice(rootPath.length + 1)
: entry.path;
if (!normalizedPath) continue;
// Two entries that normalize to the same path (e.g. `a/b` and `a//b`) make
// the package ambiguous; reject it rather than silently letting the later
// entry's contents win over the earlier one.
if (Object.prototype.hasOwnProperty.call(files, normalizedPath)) {
throw new Error(`Invalid zip archive: duplicate entry path "${normalizedPath}".`);
}
files[normalizedPath] = entry.body;
}
return { rootPath, files };
}

205
server/src/__tests__/company-portability-routes.test.ts
View File

@ -260,6 +260,79 @@ async function waitForImportJobStatusAs(
throw new Error(`Timed out waiting for import job to reach ${status}`);
}
// A minimal STORE-only zip writer so the multipart/zip upload path can be
// exercised end-to-end: the route unzips this into the same inline bundle an
// application/json caller would send. Layout matches ui/src/lib/zip.ts (local
// file headers, central directory, end-of-central-directory).
function crc32(bytes: Uint8Array) {
let crc = 0xffffffff;
for (const byte of bytes) {
crc ^= byte;
for (let bit = 0; bit < 8; bit += 1) {
crc = (crc & 1) === 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
}
}
return (crc ^ 0xffffffff) >>> 0;
}
function buildStoreZip(files: Record<string, string>, rootPath: string): Buffer {
const encoder = new TextEncoder();
const localChunks: Buffer[] = [];
const centralChunks: Buffer[] = [];
let localOffset = 0;
const entries = Object.entries(files);
for (const [relativePath, content] of entries) {
const fileName = encoder.encode(`${rootPath}/${relativePath}`);
const body = Buffer.from(encoder.encode(content));
const checksum = crc32(body);
const localHeader = Buffer.alloc(30 + fileName.length);
localHeader.writeUInt32LE(0x04034b50, 0);
localHeader.writeUInt16LE(20, 4);
localHeader.writeUInt16LE(0x0800, 6);
localHeader.writeUInt32LE(checksum, 14);
localHeader.writeUInt32LE(body.length, 18);
localHeader.writeUInt32LE(body.length, 22);
localHeader.writeUInt16LE(fileName.length, 26);
Buffer.from(fileName).copy(localHeader, 30);
const centralHeader = Buffer.alloc(46 + fileName.length);
centralHeader.writeUInt32LE(0x02014b50, 0);
centralHeader.writeUInt16LE(20, 4);
centralHeader.writeUInt16LE(20, 6);
centralHeader.writeUInt16LE(0x0800, 8);
centralHeader.writeUInt32LE(checksum, 16);
centralHeader.writeUInt32LE(body.length, 20);
centralHeader.writeUInt32LE(body.length, 24);
centralHeader.writeUInt16LE(fileName.length, 28);
centralHeader.writeUInt32LE(localOffset, 42);
Buffer.from(fileName).copy(centralHeader, 46);
localChunks.push(localHeader, body);
centralChunks.push(centralHeader);
localOffset += localHeader.length + body.length;
}
const centralDirectory = Buffer.concat(centralChunks);
const eocd = Buffer.alloc(22);
eocd.writeUInt32LE(0x06054b50, 0);
eocd.writeUInt16LE(entries.length, 8);
eocd.writeUInt16LE(entries.length, 10);
eocd.writeUInt32LE(centralDirectory.length, 12);
eocd.writeUInt32LE(localOffset, 16);
return Buffer.concat([...localChunks, centralDirectory, eocd]);
}
// The import fields a multipart caller ships as the JSON `meta` field: the same
// object an inline caller sends, minus `source` (the source is the uploaded zip).
const importMeta = {
include: importRequest.include,
target: importRequest.target,
collisionStrategy: importRequest.collisionStrategy,
};
describe.sequential("company portability routes", () => {
beforeEach(() => {
vi.clearAllMocks();
@ -1007,4 +1080,136 @@ describe.sequential("company portability routes", () => {
{ mode: "agent_safe", sourceCompanyId: companyId, pauseAutomations: true },
);
});
it.sequential("imports a company from a multipart zip upload, unzipping into the same inline bundle", async () => {
const app = await createBoardApp();
const files = { "COMPANY.md": "---\nname: Test\n---\n", "agents/ceo/AGENTS.md": "---\nname: CEO\n---\n" };
const zip = buildStoreZip(files, "paperclip");
const res = await request(app)
.post("/api/companies/import")
.set(TEST_USER_HEADER, "board-user-a")
.field("meta", JSON.stringify(importMeta))
.attach("package", zip, "paperclip-demo.zip");
expect(res.status).toBe(200);
expect(mockCompanyPortabilityService.importBundle).toHaveBeenCalledTimes(1);
const call = mockCompanyPortabilityService.importBundle.mock.calls[0]!;
// The uploaded zip is unzipped into the exact inline source the importer
// consumes; the other import fields ride along from the JSON meta field.
expect(call[0]).toEqual({ ...importMeta, source: { type: "inline", rootPath: "paperclip", files } });
expect(call[1]).toBe("board-user-a");
expect(call[2]).toEqual({ pauseAutomations: false });
});
it.sequential("previews a company from a multipart zip upload", async () => {
const app = await createBoardApp();
const files = { "COMPANY.md": "---\nname: Test\n---\n" };
const zip = buildStoreZip(files, "paperclip");
const res = await request(app)
.post("/api/companies/import/preview")
.set(TEST_USER_HEADER, "board-user-a")
.field("meta", JSON.stringify(importMeta))
.attach("package", zip, "paperclip-demo.zip");
expect(res.status).toBe(200);
expect(mockCompanyPortabilityService.previewImport).toHaveBeenCalledTimes(1);
const call = mockCompanyPortabilityService.previewImport.mock.calls[0]!;
expect(call[0]).toEqual({ ...importMeta, source: { type: "inline", rootPath: "paperclip", files } });
});
it.sequential("runs a multipart zip import as an async board job via ?async=1", async () => {
let resolveImport: (value: ReturnType<typeof createImportResult>) => void = () => undefined;
const pendingImport = new Promise<ReturnType<typeof createImportResult>>((resolve) => {
resolveImport = resolve;
});
mockCompanyPortabilityService.importBundle.mockReturnValueOnce(pendingImport);
const app = await createBoardApp();
const files = { "COMPANY.md": "---\nname: Test\n---\n" };
const zip = buildStoreZip(files, "paperclip");
const accepted = await request(app)
.post("/api/companies/import?async=1")
.set(TEST_USER_HEADER, "board-user-a")
.field("meta", JSON.stringify(importMeta))
.attach("package", zip, "paperclip-demo.zip");
expect(accepted.status).toBe(202);
expect(accepted.body.job.status).toBe("running");
expect(accepted.body.statusUrl).toMatch(/^\/api\/companies\/import\/jobs\/import-/);
await waitForCondition(
() => mockCompanyPortabilityService.importBundle.mock.calls.length === 1,
"multipart async import start",
);
expect(mockCompanyPortabilityService.importBundle.mock.calls[0]![0]).toEqual({
...importMeta,
source: { type: "inline", rootPath: "paperclip", files },
});
const fullResult = createImportResult("created");
resolveImport(fullResult);
const succeeded = await waitForImportJobStatusAs(app, accepted.body.statusUrl, "succeeded", {
[TEST_USER_HEADER]: "board-user-a",
});
expect(succeeded.body.job.status).toBe("succeeded");
expect(succeeded.body.job.importResult).toEqual(fullResult);
});
it.sequential("engages the async path for board sessions via ?async=1, not the stripped cloud header", async () => {
mockCompanyPortabilityService.importBundle.mockReturnValueOnce(new Promise(() => undefined));
const app = await createBoardApp();
// The Cloud harness strips inbound x-paperclip-cloud-* headers, so a browser
// can only opt into async with the proxy-safe query parameter.
const accepted = await request(app)
.post("/api/companies/import?async=1")
.set(TEST_USER_HEADER, "board-user-a")
.send(importRequest);
expect(accepted.status).toBe(202);
expect(accepted.body.job.status).toBe("running");
expect(accepted.body.statusUrl).toMatch(/^\/api\/companies\/import\/jobs\/import-/);
});
it.sequential("keeps a board import synchronous when neither async signal is present", async () => {
const app = await createBoardApp();
const res = await request(app)
.post("/api/companies/import")
.set(TEST_USER_HEADER, "board-user-a")
.send(importRequest);
expect(res.status).toBe(200);
expect(res.body.company.id).toBe(companyId);
});
it.sequential("still engages the async path for cloud tenants via the x-paperclip-cloud-async-import header", async () => {
mockCompanyPortabilityService.importBundle.mockReturnValueOnce(new Promise(() => undefined));
const app = await createApp(cloudTenantActor());
const accepted = await request(app)
.post("/api/companies/import")
.set("x-paperclip-cloud-async-import", "1")
.set(cloudHeaders)
.send(importRequest);
expect(accepted.status).toBe(202);
expect(accepted.body.statusUrl).toMatch(/^\/api\/companies\/import\/jobs\/tenant-import-/);
});
it.sequential("rejects a truncated zip upload without importing anything", async () => {
const app = await createBoardApp();
const zip = buildStoreZip({ "COMPANY.md": "---\nname: Test\n---\n" }, "paperclip");
const truncated = zip.subarray(0, 40);
const res = await request(app)
.post("/api/companies/import")
.set(TEST_USER_HEADER, "board-user-a")
.field("meta", JSON.stringify(importMeta))
.attach("package", truncated, "paperclip-demo.zip");
expect(res.status).toBe(400);
expect(mockCompanyPortabilityService.importBundle).not.toHaveBeenCalled();
});
});

6
server/src/http/body-limits.ts
View File

@ -1,3 +1,9 @@
export const DEFAULT_JSON_BODY_LIMIT = "10mb";
export const PORTABLE_JSON_BODY_LIMIT = "64mb";
export const PORTABLE_JSON_BODY_LIMIT_BYTES = 64 * 1024 * 1024;
// A company import can also be uploaded as its raw compressed zip (multipart or
// application/zip) instead of an inflated inline JSON body. The compressed zip
// is roughly a third of the inline size, but the limit is kept generous so a
// large company package uploads in one request rather than truncating in transit.
export const PORTABLE_ZIP_UPLOAD_LIMIT_BYTES = 128 * 1024 * 1024;

158
server/src/routes/companies.ts
View File

@ -1,10 +1,12 @@
import { createHash, randomUUID } from "node:crypto";
import { Router, type Request } from "express";
import express, { Router, type Request, type Response } from "express";
import multer from "multer";
import { and, count as countFn, eq } from "drizzle-orm";
import { z } from "zod";
import type { Db } from "@paperclipai/db";
import { agents as agentsTable } from "@paperclipai/db";
import type { CompanyPortabilityImportResult } from "@paperclipai/shared";
import { readZipArchive } from "@paperclipai/shared/portability-zip";
import {
DEFAULT_FEEDBACK_DATA_SHARING_TERMS_VERSION,
companyArtifactsQuerySchema,
@ -18,7 +20,8 @@ import {
updateCompanyBrandingSchema,
updateCompanySchema,
} from "@paperclipai/shared";
import { badRequest, forbidden } from "../errors.js";
import { badRequest, forbidden, unprocessable } from "../errors.js";
import { PORTABLE_ZIP_UPLOAD_LIMIT_BYTES } from "../http/body-limits.js";
import { validate } from "../middleware/validate.js";
import {
accessService,
@ -37,6 +40,147 @@ import type { StorageService } from "../storage/types.js";
import { assertBoard, assertCompanyAccess, assertInstanceAdmin, getActorInfo } from "./authz.js";
import { COMPANY_IMPORT_ROUTE_PATH } from "./company-import-paths.js";
// A company import can arrive one of two ways on the import + preview routes:
// • application/json — the original inline body `{ source, target, ... }`,
// kept byte-identical for CLI and programmatic callers; or
// • the raw compressed zip, either as multipart/form-data (a `package` file
// field plus a JSON `meta` field) or a bare application/zip body. The zip
// is a third of the inline size and already gzip-friendly, so it survives
// the browser → edge → harness-proxy → tenant chain that truncated the
// inflated inline JSON on large companies.
// The zip is unzipped server-side into the exact `{ rootPath, files }` bundle
// the inline source carries, then fed through the unchanged preview/import
// logic, so import semantics are identical regardless of transport.
const PORTABLE_ZIP_CONTENT_TYPES = ["application/zip", "application/x-zip-compressed"] as const;
const zipPackageUpload = multer({
storage: multer.memoryStorage(),
limits: { fileSize: PORTABLE_ZIP_UPLOAD_LIMIT_BYTES, files: 1 },
});
const rawZipBodyParser = express.raw({
type: [...PORTABLE_ZIP_CONTENT_TYPES],
limit: PORTABLE_ZIP_UPLOAD_LIMIT_BYTES,
});
function requestContentType(req: Request) {
return (req.header("content-type") ?? "").toLowerCase();
}
function isMultipartImport(req: Request) {
return requestContentType(req).includes("multipart/form-data");
}
function isZipImport(req: Request) {
const contentType = requestContentType(req);
return PORTABLE_ZIP_CONTENT_TYPES.some((type) => contentType.includes(type));
}
function runMiddleware(
middleware: (req: Request, res: Response, next: (err?: unknown) => void) => void,
req: Request,
res: Response,
) {
return new Promise<void>((resolve, reject) => {
middleware(req, res, (err?: unknown) => (err ? reject(err) : resolve()));
});
}
/**
* Parse the `meta` form/query field into the object the import schemas expect.
* A client-declared `source` is ignored — the source is always the uploaded
* zip — so a multipart caller only ships the other import fields (include,
* target, collisionStrategy, nameOverrides, selectedFiles, adapterOverrides,
* pauseAutomations, ...).
*/
function parseImportMeta(metaRaw: string | undefined): Record<string, unknown> {
if (metaRaw === undefined || metaRaw.trim().length === 0) return {};
let parsed: unknown;
try {
parsed = JSON.parse(metaRaw);
} catch {
throw badRequest("Import package metadata was not valid JSON.");
}
if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {
throw badRequest("Import package metadata must be a JSON object.");
}
const { source: _ignoredSource, ...rest } = parsed as Record<string, unknown>;
return rest;
}
/**
* Resolve the object to hand the preview/import zod schemas. For a JSON request
* this is the inline body unchanged. For a multipart or application/zip request
* the uploaded zip is read into `{ rootPath, files }` and combined with the
* `meta` fields as `source = { type: "inline", rootPath, files }`. A truncated,
* corrupt, or empty zip fails closed with a 400/422 before anything is imported.
*/
async function resolveImportPayload(req: Request, res: Response): Promise<unknown> {
const multipart = isMultipartImport(req);
const rawZip = isZipImport(req);
if (!multipart && !rawZip) {
return req.body;
}
let zipBytes: Buffer | undefined;
let metaRaw: string | undefined;
if (multipart) {
try {
await runMiddleware(zipPackageUpload.single("package"), req, res);
} catch (error) {
if (error instanceof multer.MulterError) {
if (error.code === "LIMIT_FILE_SIZE") {
throw unprocessable(`Import package exceeds ${PORTABLE_ZIP_UPLOAD_LIMIT_BYTES} bytes`);
}
throw badRequest(error.message);
}
throw error;
}
const file = (req as Request & { file?: { buffer?: Buffer } }).file;
zipBytes = file?.buffer;
const metaField = (req.body as { meta?: unknown } | undefined)?.meta;
metaRaw = typeof metaField === "string" ? metaField : undefined;
} else {
try {
await runMiddleware(rawZipBodyParser, req, res);
} catch (error) {
throw badRequest(error instanceof Error ? error.message : "Invalid zip upload");
}
zipBytes = Buffer.isBuffer(req.body) ? req.body : undefined;
const metaQuery = req.query.meta;
metaRaw = typeof metaQuery === "string" ? metaQuery : undefined;
}
if (!zipBytes || zipBytes.length === 0) {
throw badRequest("Import package upload was empty.");
}
let archive: Awaited<ReturnType<typeof readZipArchive>>;
try {
archive = await readZipArchive(zipBytes);
} catch (error) {
throw badRequest(`Import package could not be read: ${errorMessage(error)}`);
}
if (Object.keys(archive.files).length === 0) {
throw badRequest("Import package contained no files.");
}
return {
...parseImportMeta(metaRaw),
source: { type: "inline", rootPath: archive.rootPath, files: archive.files },
};
}
/**
* Async job opt-in. Cloud tenants set the `x-paperclip-cloud-async-import`
* header server-side (they are not a browser, so it is never stripped). Board
* browsers cannot use that header — the Cloud harness proxy strips every
* inbound `x-paperclip-cloud-*` header as anti-spoofing — so they opt in with
* the proxy-safe `?async=1` query parameter instead. Either signal enters the
* async path.
*/
function wantsAsyncImport(req: Request) {
return req.query.async === "1" || req.header("x-paperclip-cloud-async-import") === "1";
}
export function companyRoutes(db: Db, storage?: StorageService) {
const router = Router();
const svc = companyService(db);
@ -264,7 +408,7 @@ export function companyRoutes(db: Db, storage?: StorageService) {
router.post("/import/preview", async (req, res) => {
assertBoard(req);
const body = companyPortabilityPreviewSchema.parse(req.body);
const body = companyPortabilityPreviewSchema.parse(await resolveImportPayload(req, res));
assertImportTargetAccess(req, body.target);
const preview = await portability.previewImport(body);
res.json(preview);
@ -288,10 +432,14 @@ export function companyRoutes(db: Db, storage?: StorageService) {
router.post(COMPANY_IMPORT_ROUTE_PATH, async (req, res) => {
assertBoard(req);
const rawImportBody: unknown = req.body;
// Resolve the request body up front: a JSON caller's inline body is used
// unchanged; a multipart/application-zip caller's uploaded zip is read into
// the same `{ source: { type: "inline", rootPath, files } }` bundle here,
// fast and in-memory, so the async job machinery below is transport-agnostic.
const rawImportBody: unknown = await resolveImportPayload(req, res);
const actor = getActorInfo(req);
const boardUserId = req.actor.type === "board" ? req.actor.userId : null;
if (req.header("x-paperclip-cloud-async-import") === "1") {
if (wantsAsyncImport(req)) {
// Async job path. Two kinds of callers opt in:
// - trusted Cloud tenants (original behavior, kept byte-identical),
// keyed by their tenant identity headers;

68
server/src/routes/openapi.ts
View File

@ -523,6 +523,40 @@ const jsonBody = (schema: z.ZodTypeAny) => ({
required: true as const,
});
// The company import + preview routes accept the inline JSON body or the raw
// company package as a compressed zip upload. Document both content types: the
// JSON variant keeps its zod schema; the multipart variant carries the zip in a
// `package` file field plus the other import fields as a JSON `meta` field.
const importRequestBody = (schema: z.ZodTypeAny) => ({
content: {
"application/json": { schema },
"multipart/form-data": {
schema: {
type: "object",
properties: {
package: {
type: "string",
format: "binary",
description: "The company package as a compressed .zip.",
},
meta: {
type: "string",
description:
"JSON-encoded import fields (include, target, collisionStrategy, and, for " +
"the apply route, nameOverrides / selectedFiles / adapterOverrides / " +
"pauseAutomations). A `source` here is ignored — the source is the zip.",
},
},
required: ["package"],
},
},
"application/zip": {
schema: { type: "string", format: "binary" },
},
},
required: true as const,
});
const r = responses;
const externalObjectSummariesBodySchema = z.object({
@ -5239,8 +5273,15 @@ registry.registerPath({
path: "/api/companies/import/preview",
tags: ["companies"],
summary: "Preview a company import (legacy route)",
request: { body: jsonBody(companyPortabilityPreviewSchema) },
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized },
description:
"Accepts either the inline JSON body (`application/json`) or the raw company " +
"package uploaded as a compressed zip (`multipart/form-data` with a `package` " +
"file field plus a JSON `meta` field carrying the other import fields, or a bare " +
"`application/zip` body with the `meta` JSON in the `meta` query parameter). The " +
"zip is unzipped server-side into the same `{ source: { type: \"inline\", ... } }` " +
"bundle the JSON body carries.",
request: { body: importRequestBody(companyPortabilityPreviewSchema) },
responses: { 200: r.ok(), 400: r.badRequest, 401: r.unauthorized, 422: r.unprocessable },
});
registry.registerPath({
@ -5249,17 +5290,28 @@ registry.registerPath({
tags: ["companies"],
summary: "Apply a company import (legacy route)",
description:
"Board sessions and trusted Cloud tenants can opt into asynchronous processing with the " +
"`x-paperclip-cloud-async-import: 1` header: the server responds 202 with a job id and status " +
"URL instead of holding the connection open for the whole import. While a board actor already " +
"has an async job running, a resubmit returns 409 carrying the running job's id and status URL. " +
"Jobs are held in memory and are lost on restart.",
request: { body: jsonBody(companyPortabilityImportSchema) },
"Accepts either the inline JSON body (`application/json`) or the raw company package " +
"uploaded as a compressed zip (`multipart/form-data` with a `package` file field plus a " +
"JSON `meta` field, or a bare `application/zip` body with the `meta` JSON in the `meta` " +
"query parameter); the zip is unzipped server-side into the same import bundle. " +
"Callers can opt into asynchronous processing: trusted Cloud tenants set the " +
"`x-paperclip-cloud-async-import: 1` header (browsers cannot — the Cloud harness proxy " +
"strips inbound `x-paperclip-cloud-*` headers), while board sessions use the proxy-safe " +
"`?async=1` query parameter. Either way the server responds 202 with a job id and status " +
"URL instead of holding the connection open for the whole import. While a board actor " +
"already has an async job running, a resubmit returns 409 carrying the running job's id " +
"and status URL. Jobs are held in memory and are lost on restart.",
request: {
query: z.object({ async: z.enum(["1"]).optional() }),
body: importRequestBody(companyPortabilityImportSchema),
},
responses: {
200: r.ok(),
202: { description: "Async import job accepted" },
400: r.badRequest,
401: r.unauthorized,
409: { description: "An async import job is already running for this actor" },
422: r.unprocessable,
},
});

9
ui/src/api/client.ts
View File

@ -155,7 +155,14 @@ export const api = {
...(options?.headers ? { headers: options.headers } : {}),
}),
postForm: <T>(path: string, body: FormData, options?: RequestOptions) =>
request<T>(path, { method: "POST", body, signal: options?.signal }),
request<T>(path, {
method: "POST",
body,
signal: options?.signal,
// Never set Content-Type here — the browser sets multipart/form-data with
// the boundary. Extra headers (e.g. an async opt-in) may still ride along.
...(options?.headers ? { headers: options.headers } : {}),
}),
put: <T>(path: string, body: unknown, options?: RequestOptions) =>
request<T>(path, { method: "PUT", body: JSON.stringify(body), signal: options?.signal }),
patch: <T>(path: string, body: unknown, options?: RequestOptions) =>

31
ui/src/api/companies.ts
View File

@ -14,6 +14,21 @@ import { api } from "./client";
export type CompanyStats = Record<string, { agentCount: number; issueCount: number }>;
/**
* Import fields for a zip package upload: everything the JSON request carries
* except `source` — the source is the uploaded zip, read server-side. Sent as a
* JSON-encoded `meta` form field alongside the raw `package` file.
*/
export type CompanyPortabilityPreviewMeta = Omit<CompanyPortabilityPreviewRequest, "source">;
export type CompanyPortabilityImportMeta = Omit<CompanyPortabilityImportRequest, "source">;
function importPackageForm(file: File, meta: CompanyPortabilityPreviewMeta | CompanyPortabilityImportMeta): FormData {
const form = new FormData();
form.append("package", file);
form.append("meta", JSON.stringify(meta));
return form;
}
export type CompanyImportJobState = "running" | "succeeded" | "failed";
/** 202 body from the async opt-in on POST /companies/import (and the 409 body when a job is already running). */
@ -82,13 +97,21 @@ export const companiesApi = {
api.get<ExportFidelityReport>(`/companies/${companyId}/export/fidelity`),
importPreview: (data: CompanyPortabilityPreviewRequest) =>
api.post<CompanyPortabilityPreviewResult>("/companies/import/preview", data),
/** Preview a local .zip package by uploading the raw compressed zip as multipart. */
importPreviewPackage: (file: File, meta: CompanyPortabilityPreviewMeta) =>
api.postForm<CompanyPortabilityPreviewResult>("/companies/import/preview", importPackageForm(file, meta)),
importBundle: (data: CompanyPortabilityImportRequest) =>
api.post<CompanyPortabilityImportResult>("/companies/import", data),
/** Submit an import as a server-side job: 202 with a job id to poll, or 409 with the already-running job. */
// Submit an import as a server-side job: 202 with a job id to poll, or 409
// with the already-running job. Board sessions opt in with the proxy-safe
// `?async=1` query parameter — the Cloud harness strips the inbound
// `x-paperclip-cloud-*` header a browser would otherwise use, so that header
// never survives to engage the async path.
importBundleAsync: (data: CompanyPortabilityImportRequest) =>
api.post<CompanyImportJobAccepted>("/companies/import", data, {
headers: { "x-paperclip-cloud-async-import": "1" },
}),
api.post<CompanyImportJobAccepted>("/companies/import?async=1", data),
/** Submit a local .zip package as an async import job by uploading the raw compressed zip as multipart. */
importBundlePackageAsync: (file: File, meta: CompanyPortabilityImportMeta) =>
api.postForm<CompanyImportJobAccepted>("/companies/import?async=1", importPackageForm(file, meta)),
getImportJob: (jobId: string) =>
api.get<CompanyImportJobStatus>(`/companies/import/jobs/${encodeURIComponent(jobId)}`),
};

52
ui/src/pages/CompanyImport.test.tsx
View File

@ -11,8 +11,10 @@ import { CompanyImport } from "./CompanyImport";
const mockCompaniesApi = vi.hoisted(() => ({
importPreview: vi.fn(),
importPreviewPackage: vi.fn(),
importBundle: vi.fn(),
importBundleAsync: vi.fn(),
importBundlePackageAsync: vi.fn(),
getImportJob: vi.fn(),
get: vi.fn(),
}));
@ -186,9 +188,11 @@ describe("CompanyImport", () => {
mockAgentsApi.resume.mockResolvedValue({ id: "agent-1", status: "idle" });
mockRoutinesApi.update.mockResolvedValue({ id: "routine-1", status: "active" });
mockCompaniesApi.importPreview.mockResolvedValue(buildPreviewResult());
mockCompaniesApi.importPreviewPackage.mockResolvedValue(buildPreviewResult());
// Default async flow: the submit is accepted (202) and the first poll finds
// the job already finished with the full result. Individual tests override.
mockCompaniesApi.importBundleAsync.mockResolvedValue(buildAccepted());
mockCompaniesApi.importBundlePackageAsync.mockResolvedValue(buildAccepted());
mockCompaniesApi.getImportJob.mockResolvedValue(buildSucceededJob());
mockCompaniesApi.get.mockResolvedValue({ id: "company-2", name: "Imported Test", issuePrefix: "IMP" });
mockSidebarPreferencesApi.updateProjectOrder.mockResolvedValue(undefined);
@ -303,9 +307,10 @@ describe("CompanyImport", () => {
expect(mockPushToast).toHaveBeenCalledWith(expect.objectContaining({ tone: "error" }));
});
it("blocks oversized local packages and sends the declared file count once attachments are dropped", async () => {
// A synthetic parsed package: the base64 blob payload alone exceeds the
// inline import limit, so no real 60MB zip needs to be built.
it("uploads a local .zip as a multipart package and never blocks on inline size", async () => {
// Even a package whose inflated inline JSON would blow past the old browser
// limit uploads fine now: the raw compressed zip goes up as multipart and is
// unzipped server-side, so the inline-size ceiling no longer gates it.
mockReadZipArchive.mockResolvedValue({
rootPath: "big-package",
files: {
@ -325,7 +330,7 @@ describe("CompanyImport", () => {
const fileInput = container.querySelector<HTMLInputElement>('input[type="file"]');
expect(fileInput).toBeTruthy();
const file = new File(["stub"], "big-package.zip", { type: "application/zip" });
const file = new File(["stub-zip-bytes"], "big-package.zip", { type: "application/zip" });
Object.defineProperty(file, "arrayBuffer", { value: async () => new ArrayBuffer(0) });
Object.defineProperty(fileInput!, "files", { value: [file] });
await act(async () => {
@ -333,32 +338,33 @@ describe("CompanyImport", () => {
});
await flushReact();
expect(container.textContent).toContain("CLI folder import");
expect(container.textContent).toContain("Package too large for browser import");
expect(findButton((text) => text === "Preview import")?.disabled).toBe(true);
await clickButton((text) => text === "Continue without attachments");
expect(container.textContent).not.toContain("CLI folder import");
// The inline preflight no longer blocks the zip path.
expect(container.textContent).not.toContain("Package too large for browser import");
expect(container.textContent).not.toContain("CLI folder import");
expect(findButton((text) => text === "Preview import")?.disabled).toBe(false);
await clickButton((text) => text === "Preview import");
// The button label reflects the preview's file count (3); the sent source
// is the local package, which is down to two files after the blob is
// dropped.
// The preview goes up as a multipart package (the raw File), not inline JSON.
expect(mockCompaniesApi.importPreviewPackage).toHaveBeenCalledTimes(1);
expect(mockCompaniesApi.importPreview).not.toHaveBeenCalled();
expect(mockCompaniesApi.importPreviewPackage.mock.calls[0]![0]).toBe(file);
await clickButton((text) => text.startsWith("Import 3 file"));
await settle();
expect(mockCompaniesApi.importBundleAsync).toHaveBeenCalledTimes(1);
const request = mockCompaniesApi.importBundleAsync.mock.calls[0]![0] as {
source: { type: string; files: Record<string, unknown>; expectedFileCount?: number };
};
expect(request.source.type).toBe("inline");
expect(Object.keys(request.source.files).sort()).toEqual([".paperclip.yaml", "COMPANY.md"]);
// The client declares the file count so the server can reject a truncated
// upload instead of importing a fragment.
expect(request.source.expectedFileCount).toBe(2);
// The apply also uploads the raw File as a multipart async job — never the
// inflated inline body that truncated in transit.
expect(mockCompaniesApi.importBundleAsync).not.toHaveBeenCalled();
expect(mockCompaniesApi.importBundlePackageAsync).toHaveBeenCalledTimes(1);
const [sentFile, meta] = mockCompaniesApi.importBundlePackageAsync.mock.calls[0]! as [
File,
{ pauseAutomations: boolean; target: { mode: string } },
];
expect(sentFile).toBe(file);
expect(sentFile.name).toBe("big-package.zip");
expect(meta.pauseAutomations).toBe(true);
// The bundle itself is never expanded into the request; only the raw zip travels.
expect(meta).not.toHaveProperty("source");
});
it("explains the disabled preview button until a package is chosen", async () => {

144
ui/src/pages/CompanyImport.tsx
View File

@ -50,13 +50,7 @@ import {
FileTree,
} from "../components/FileTree";
import { readZipArchive } from "../lib/zip";
import {
INLINE_IMPORT_MAX_BYTES,
buildInlineImportPreflight,
formatMegabytes,
isBlobStoreFilePath,
stripBlobFiles,
} from "../lib/import-preflight";
import { formatMegabytes } from "../lib/import-preflight";
import { getPortableFileDataUrl, getPortableFileText, isPortableImageFile } from "../lib/portable-files";
import {
clearStoredImportJob,
@ -667,6 +661,14 @@ function AdapterPickerList({
async function readLocalPackageZip(file: File): Promise<{
name: string;
/**
* The raw .zip File itself. Local imports upload this compressed file as
* multipart instead of inflating it into one large inline JSON body (which
* truncated in transit on big companies); the parsed `files` map below is
* kept only for the client-side preflight display and preview affordances.
*/
file: File;
compressedBytes: number;
rootPath: string | null;
files: Record<string, CompanyPortabilityFileEntry>;
}> {
@ -679,6 +681,8 @@ async function readLocalPackageZip(file: File): Promise<{
}
return {
name: file.name,
file,
compressedBytes: file.size,
rootPath: archive.rootPath,
files: archive.files,
};
@ -779,6 +783,8 @@ export function CompanyImport() {
const [importUrl, setImportUrl] = useState("");
const [localPackage, setLocalPackage] = useState<{
name: string;
file: File;
compressedBytes: number;
rootPath: string | null;
files: Record<string, CompanyPortabilityFileEntry>;
} | null>(null);
@ -845,24 +851,29 @@ export function CompanyImport() {
]);
}, [selectedCompany?.name, setBreadcrumbs]);
function buildSource(): CompanyPortabilitySource | null {
if (sourceMode === "local") {
if (!localPackage) return null;
// Declare how many files we are sending so the server can reject a
// truncated upload instead of importing a fragment (see the server-side
// completeness check in importBundle).
return {
type: "inline",
rootPath: localPackage.rootPath,
files: localPackage.files,
expectedFileCount: Object.keys(localPackage.files).length,
};
}
// The GitHub/URL source still travels inline (it is just a URL, so it never
// hits the inline-size ceiling). The local .zip source uploads its raw
// compressed file as multipart instead — see the preview/import mutations.
function buildGithubSource(): CompanyPortabilitySource | null {
const url = importUrl.trim();
if (!url) return null;
return { type: "github", url };
}
// Import fields shared by preview and apply, and by both transports. The
// multipart zip upload ships these as a JSON `meta` field; the inline GitHub
// request spreads them alongside its `source`.
function buildImportMetaCommon() {
return {
include: { company: true, agents: true, projects: true, issues: true },
target:
targetMode === "new"
? { mode: "new_company" as const, newCompanyName: newCompanyName || null }
: { mode: "existing_company" as const, companyId: selectedCompanyId! },
collisionStrategy,
};
}
// Monotonic id for preview requests. Structural configuration changes bump
// it, so an in-flight preview they supersede settles silently instead of
// publishing a result or error for a package that is no longer selected.
@ -873,17 +884,16 @@ export function CompanyImport() {
// Preview mutation
const previewMutation = useMutation({
mutationFn: (_generation: number) => {
const source = buildSource();
const meta = buildImportMetaCommon();
if (sourceMode === "local") {
if (!localPackage) throw new Error("No source configured.");
// Upload the raw compressed zip; the server unzips it into the same
// inline bundle the importer consumes.
return companiesApi.importPreviewPackage(localPackage.file, meta);
}
const source = buildGithubSource();
if (!source) throw new Error("No source configured.");
return companiesApi.importPreview({
source,
include: { company: true, agents: true, projects: true, issues: true },
target:
targetMode === "new"
? { mode: "new_company", newCompanyName: newCompanyName || null }
: { mode: "existing_company", companyId: selectedCompanyId! },
collisionStrategy,
});
return companiesApi.importPreview({ source, ...meta });
},
onSuccess: (result, generation) => {
if (generation !== previewGenerationRef.current) return;
@ -1001,24 +1011,24 @@ export function CompanyImport() {
if (variables.resume) {
return watchImportJob(variables.resume.jobId, variables.resume.storageKey);
}
const source = buildSource();
if (!source) throw new Error("No source configured.");
const meta = {
...buildImportMetaCommon(),
nameOverrides: buildFinalNameOverrides(),
selectedFiles: buildSelectedFiles(),
adapterOverrides: buildFinalAdapterOverrides(),
pauseAutomations: variables.pauseAutomations,
};
const localFile = sourceMode === "local" ? localPackage?.file : null;
const githubSource = sourceMode === "local" ? null : buildGithubSource();
if (sourceMode === "local" ? !localFile : !githubSource) {
throw new Error("No source configured.");
}
const storageKey = currentImportJobStorageKey();
let accepted: CompanyImportJobAccepted;
try {
accepted = await companiesApi.importBundleAsync({
source,
include: { company: true, agents: true, projects: true, issues: true },
target:
targetMode === "new"
? { mode: "new_company", newCompanyName: newCompanyName || null }
: { mode: "existing_company", companyId: selectedCompanyId! },
collisionStrategy,
nameOverrides: buildFinalNameOverrides(),
selectedFiles: buildSelectedFiles(),
adapterOverrides: buildFinalAdapterOverrides(),
pauseAutomations: variables.pauseAutomations,
});
accepted = localFile
? await companiesApi.importBundlePackageAsync(localFile, meta)
: await companiesApi.importBundleAsync({ source: githubSource!, ...meta });
} catch (err) {
// 409: this user's previous import is still running. Adopt that job
// and watch it — never fire a second import.
@ -1352,20 +1362,10 @@ export function CompanyImport() {
sourceMode === "local" ? !!localPackage : importUrl.trim().length > 0;
const hasErrors = importPreview ? importPreview.errors.length > 0 : false;
// Inline imports ship the parsed package as one JSON request; oversized
// local packages are blocked before any request is attempted.
const inlinePreflight = useMemo(
() => (sourceMode === "local" && localPackage ? buildInlineImportPreflight(localPackage.files) : null),
[sourceMode, localPackage],
);
const inlineImportBlocked = Boolean(inlinePreflight?.tooLarge);
function handleContinueWithoutAttachments() {
if (!localPackage) return;
resetMutationState();
setLocalPackage({ ...localPackage, files: stripBlobFiles(localPackage.files) });
setCheckedFiles((prev) => new Set([...prev].filter((filePath) => !isBlobStoreFilePath(filePath))));
}
// The local .zip uploads its raw compressed file as multipart and is unzipped
// server-side, so the old inline-size ceiling no longer gates it. Surface the
// compressed upload size instead of the inflated-JSON estimate.
const localCompressedBytes = sourceMode === "local" ? localPackage?.compressedBytes ?? null : null;
const previewContent = selectedFile && importPreview
? (() => {
@ -1555,6 +1555,7 @@ export function CompanyImport() {
{localPackage.name} with{" "}
{Object.keys(localPackage.files).length} file
{Object.keys(localPackage.files).length === 1 ? "" : "s"}
{localCompressedBytes !== null ? ` (${formatMegabytes(localCompressedBytes)} zip)` : ""}
</span>
)}
</div>
@ -1563,20 +1564,6 @@ export function CompanyImport() {
{localZipHelpText}
</p>
)}
{inlinePreflight?.tooLarge && (
<div className="mt-3 space-y-2 rounded-md border border-amber-500/30 bg-amber-500/5 px-3 py-2.5">
<p className="text-xs text-amber-500">
This package is about {formatMegabytes(inlinePreflight.estimatedBytes)} inline, which is
larger than the {formatMegabytes(INLINE_IMPORT_MAX_BYTES)} browser import limit. Large
packages need the CLI folder import today (paperclip company import).
</p>
{inlinePreflight.canDropAttachments && (
<Button size="sm" variant="outline" onClick={handleContinueWithoutAttachments}>
Continue without attachments
</Button>
)}
</div>
)}
</div>
) : (
<Field
@ -1657,7 +1644,7 @@ export function CompanyImport() {
variant="outline"
onClick={() => previewMutation.mutate(previewGenerationRef.current)}
disabled={
previewMutation.isPending || importMutation.isPending || !hasSource || inlineImportBlocked
previewMutation.isPending || importMutation.isPending || !hasSource
}
>
{previewMutation.isPending ? "Previewing..." : "Preview import"}
@ -1672,18 +1659,13 @@ export function CompanyImport() {
Import in progress — the package and settings unlock when it finishes.
</span>
)}
{inlineImportBlocked && (
<span className="text-xs text-amber-500">
Package too large for browser import — see the notice above.
</span>
)}
</div>
{previewMutation.isPending && (
<div className="mt-3 flex items-start gap-2 rounded-md border border-border bg-muted/30 px-3 py-2.5">
<Loader2 className="mt-0.5 h-3.5 w-3.5 shrink-0 animate-spin text-muted-foreground" />
<p className="text-xs text-muted-foreground">
Uploading and analyzing your package
{inlinePreflight ? ` (about ${formatMegabytes(inlinePreflight.estimatedBytes)})` : ""} — large
{localCompressedBytes !== null ? ` (${formatMegabytes(localCompressedBytes)} zip)` : ""} — large
packages can take a few minutes. Keep this page open.
</p>
</div>
@ -1767,7 +1749,7 @@ export function CompanyImport() {
<Button
size="sm"
onClick={() => importMutation.mutate({ previewForImport: importPreview, pauseAutomations })}
disabled={importMutation.isPending || hasErrors || selectedCount === 0 || inlineImportBlocked}
disabled={importMutation.isPending || hasErrors || selectedCount === 0}
>
<Download className="mr-1.5 h-3.5 w-3.5" />
{importMutation.isPending