perf(ci): balance general-server test shards by recorded suite duration (#9516)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its PR CI runs the general-server vitest lane pinned to `maxWorkers=1` and sharded across 3 runners (introduced in #8360) > - Suites were assigned to shards round-robin by sorted file index, so shard test time was unbalanced: a recent PR run split 73s / 153s / 115s, and the heaviest shard made "General tests (server 2/3)" the slowest check in the whole workflow at 314s wall > - The slowest shard sets the lane's wall time, so unbalanced partitions waste the other two runners and stretch the PR critical path > - This pull request replaces the round-robin assignment with a deterministic longest-processing-time partition weighted by a checked-in per-suite duration manifest > - The benefit is near-even shard weights (projected 113s / 113s / 113s with the current manifest), taking roughly 40s off the PR critical path with no reduction in coverage ## Linked Issues or Issue Description - Refs #8360 (introduced the 3-way general-server sharding this PR rebalances) - No public issue exists. Problem: the general-server test lane's round-robin shard assignment ignores per-suite duration, so one shard can carry multiple 30s+ suites while another finishes in half the time; the slowest shard alone determines the check's wall time. ## What Changed - `scripts/general-server-shard.mjs` (new): manifest loader and deterministic LPT (longest-processing-time) partitioner; suites missing from the manifest get the median recorded weight, and a missing or malformed manifest degrades to uniform weights so the lane never fails on stale data - `scripts/general-server-shard-durations.json` (new): per-suite duration manifest sampled from a real PR run (240 suites); the `$comment` field documents how to regenerate it - `scripts/run-vitest-stable.mjs`: both shard-selection sites (run and `--dry-run`) now use the balanced partition instead of index round-robin - `scripts/__tests__/run-vitest-stable-shard.test.mjs`: 6 new tests covering skew-balance vs round-robin, determinism, median fallback for unlisted suites, malformed-manifest degradation, manifest coverage of the current suite set, and real-partition balance - `server/src/__tests__/heartbeat-issue-rewake-throttle.test.ts`: hardened the `afterEach` sweep — post-run bookkeeping (run-event records, follow-up wake scheduling) can still insert rows briefly after a run reaches a terminal status, and a late insert landing between the `agent_wakeup_requests` and `agents` deletes failed teardown with a foreign-key violation on the first CI attempt of this PR; the sweep now retries so a late background write cannot take down the shard - `release-verify.yml` shares the same runner script and inherits the balancing with no workflow change ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9 pass (run against current master) - `npx vitest run src/__tests__/heartbeat-issue-rewake-throttle.test.ts` — 6/6 pass against embedded Postgres with the hardened teardown - `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2 pass - `node scripts/run-vitest-stable.mjs --dry-run` with each shard flag shows every suite assigned exactly once across the 3 shards, with projected weights ~113s each ## Risks - Low risk: partition changes which runner executes which suite, not what runs; a completeness test asserts every suite is assigned to exactly one shard - The duration manifest will drift as suites are added/changed; unlisted suites get the median weight and a coverage test flags when the manifest covers less than half the suite set, so drift degrades balance gracefully rather than breaking the lane > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic), extended thinking enabled, agentic tool use (file edits, shell, test execution) via Claude Code ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude (Paperclip SWE) <noreply@paperclip.ing>
This commit is contained in:
Devin Foley
GitHub
parent
b49d178c46
commit
ce7dedf33d
|
|
@ -4,8 +4,15 @@ import path from "node:path";
|
|||
import { fileURLToPath } from "node:url";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
defaultSuiteWeight,
|
||||
loadShardDurations,
|
||||
partitionGeneralServerSuites,
|
||||
} from "../general-server-shard.mjs";
|
||||
|
||||
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
|
||||
const script = path.join(repoRoot, "scripts", "run-vitest-stable.mjs");
|
||||
const durationsManifest = path.join(repoRoot, "scripts", "general-server-shard-durations.json");
|
||||
|
||||
function dryRun(args) {
|
||||
const result = spawnSync(process.execPath, [script, ...args, "--dry-run"], {
|
||||
|
|
@ -61,3 +68,80 @@ test("shard flags are rejected for the parallel workspace groups", () => {
|
|||
const result = dryRun(["--mode", "general", "--group", "general-workspaces-a", "--shard-index", "0", "--shard-count", "3"]);
|
||||
assert.notEqual(result.status, 0, "workspace groups must not accept shard flags");
|
||||
});
|
||||
|
||||
test("duration-aware partition balances skewed weights better than round-robin", () => {
|
||||
// Round-robin puts all three heavy suites on shard 0 (indexes 0, 3, 6).
|
||||
const files = ["a", "b", "c", "d", "e", "f", "g", "h", "i"];
|
||||
const durations = { a: 30000, d: 30000, g: 30000, b: 100, c: 100, e: 100, f: 100, h: 100, i: 100 };
|
||||
|
||||
const shards = partitionGeneralServerSuites(files, 3, durations);
|
||||
const totals = shards.map((shard) => shard.totalWeight);
|
||||
const maxTotal = Math.max(...totals);
|
||||
const minTotal = Math.min(...totals);
|
||||
assert.ok(
|
||||
maxTotal - minTotal <= 200,
|
||||
`expected near-even shard weights, got ${totals.join(", ")}`,
|
||||
);
|
||||
assert.equal(
|
||||
shards.flatMap((shard) => shard.files).sort().join(","),
|
||||
files.join(","),
|
||||
"partition must cover every file exactly once",
|
||||
);
|
||||
});
|
||||
|
||||
test("the partition is deterministic for identical inputs", () => {
|
||||
const files = Array.from({ length: 50 }, (_, index) => `suite-${index}.test.ts`);
|
||||
const durations = Object.fromEntries(files.map((file, index) => [file, (index * 37) % 5000]));
|
||||
|
||||
const first = partitionGeneralServerSuites(files, 3, durations);
|
||||
const second = partitionGeneralServerSuites(files, 3, durations);
|
||||
assert.deepEqual(first, second, "same inputs must always produce the same partition");
|
||||
});
|
||||
|
||||
test("suites missing from the manifest get the median weight", () => {
|
||||
assert.equal(defaultSuiteWeight({ a: 100, b: 300, c: 900 }), 300);
|
||||
assert.equal(defaultSuiteWeight({ a: 100, b: 300, c: 500, d: 900 }), 400);
|
||||
assert.equal(defaultSuiteWeight({}), 1000, "empty manifest falls back to a fixed weight");
|
||||
});
|
||||
|
||||
test("a missing or malformed manifest degrades to uniform weights", () => {
|
||||
assert.deepEqual(loadShardDurations(path.join(repoRoot, "scripts", "no-such-manifest.json")), {});
|
||||
|
||||
const files = ["a", "b", "c", "d"];
|
||||
const shards = partitionGeneralServerSuites(files, 2, {});
|
||||
assert.equal(shards[0].files.length + shards[1].files.length, files.length);
|
||||
assert.equal(Math.abs(shards[0].files.length - shards[1].files.length), 0);
|
||||
});
|
||||
|
||||
test("the checked-in manifest loads and covers most of the current suite set", () => {
|
||||
const durations = loadShardDurations(durationsManifest);
|
||||
assert.ok(Object.keys(durations).length > 0, "manifest must parse to a non-empty duration map");
|
||||
|
||||
const shard = dryRunJson(["--mode", "general", "--group", "general-server", "--shard-index", "0", "--shard-count", "1"]);
|
||||
const currentFiles = shard.selectedGeneralServerSuites;
|
||||
const known = currentFiles.filter((file) => durations[file] !== undefined).length;
|
||||
assert.ok(
|
||||
known / currentFiles.length >= 0.5,
|
||||
`manifest is stale: only ${known} of ${currentFiles.length} suites have recorded durations — regenerate it from a recent PR run (see the manifest's $comment)`,
|
||||
);
|
||||
});
|
||||
|
||||
test("the real shard partition is duration-balanced", () => {
|
||||
const durations = loadShardDurations(durationsManifest);
|
||||
const fallback = defaultSuiteWeight(durations);
|
||||
const shards = Array.from({ length: SHARD_COUNT }, (_, index) =>
|
||||
dryRunJson(["--mode", "general", "--group", "general-server", "--shard-index", String(index), "--shard-count", String(SHARD_COUNT)]),
|
||||
);
|
||||
|
||||
const totals = shards.map((shard) =>
|
||||
shard.selectedGeneralServerSuites.reduce((sum, file) => sum + (durations[file] ?? fallback), 0),
|
||||
);
|
||||
const maxTotal = Math.max(...totals);
|
||||
const minTotal = Math.min(...totals);
|
||||
// LPT keeps the spread within the heaviest single suite; use that as the bound.
|
||||
const heaviest = Math.max(...Object.values(durations));
|
||||
assert.ok(
|
||||
maxTotal - minTotal <= heaviest,
|
||||
`shard weight spread ${maxTotal - minTotal}ms exceeds heaviest suite ${heaviest}ms: ${totals.join(", ")}`,
|
||||
);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -0,0 +1,246 @@
|
|||
{
|
||||
"$comment": "Per-suite wall-clock durations (ms) for the general-server vitest lane, used by scripts/general-server-shard.mjs to balance suites across the PR shard matrix. Sampled from a real PR run of .github/workflows/pr.yml (actions run 29262068391, 2026-07-13). Suites missing here get the median weight, so the manifest only needs occasional refreshes: pull the 'Run grouped general test suites' logs from a recent PR run and record each suite's vitest-reported duration.",
|
||||
"unit": "ms",
|
||||
"durations": {
|
||||
"server/src/__tests__/access-service.test.ts": 2303,
|
||||
"server/src/__tests__/access-validators.test.ts": 7,
|
||||
"server/src/__tests__/activity-service.test.ts": 2390,
|
||||
"server/src/__tests__/adapter-models.test.ts": 61,
|
||||
"server/src/__tests__/adapter-registry.test.ts": 17,
|
||||
"server/src/__tests__/adapter-session-codecs.test.ts": 10,
|
||||
"server/src/__tests__/agent-auth-jwt.test.ts": 20,
|
||||
"server/src/__tests__/agent-auth-middleware.test.ts": 66,
|
||||
"server/src/__tests__/agent-instructions-service.test.ts": 58,
|
||||
"server/src/__tests__/agent-invokability.test.ts": 5,
|
||||
"server/src/__tests__/agent-permissions-service.test.ts": 6,
|
||||
"server/src/__tests__/agent-shortname-collision.test.ts": 5,
|
||||
"server/src/__tests__/agent-skill-contract.test.ts": 6,
|
||||
"server/src/__tests__/agents-pending-approval-config.test.ts": 2182,
|
||||
"server/src/__tests__/agents-service-clear-error.test.ts": 2258,
|
||||
"server/src/__tests__/agents-service-secret-bindings.test.ts": 2367,
|
||||
"server/src/__tests__/api-compression.test.ts": 77,
|
||||
"server/src/__tests__/app-hmr-port.test.ts": 4,
|
||||
"server/src/__tests__/app-private-hostname-gate.test.ts": 4,
|
||||
"server/src/__tests__/app-vite-dev-routing.test.ts": 4,
|
||||
"server/src/__tests__/approvals-service.test.ts": 12,
|
||||
"server/src/__tests__/attachment-types.test.ts": 9,
|
||||
"server/src/__tests__/attention-service.test.ts": 2626,
|
||||
"server/src/__tests__/authorization-service.test.ts": 2824,
|
||||
"server/src/__tests__/aws-secrets-manager-provider.test.ts": 39,
|
||||
"server/src/__tests__/better-auth.test.ts": 9,
|
||||
"server/src/__tests__/board-claim.test.ts": 2121,
|
||||
"server/src/__tests__/board-mutation-guard.test.ts": 50,
|
||||
"server/src/__tests__/body-limits.test.ts": 3,
|
||||
"server/src/__tests__/budgets-service.test.ts": 2211,
|
||||
"server/src/__tests__/built-in-agents.test.ts": 6157,
|
||||
"server/src/__tests__/change-consent-gate.test.ts": 2256,
|
||||
"server/src/__tests__/claude-local-adapter-environment.test.ts": 372,
|
||||
"server/src/__tests__/claude-local-adapter.test.ts": 8,
|
||||
"server/src/__tests__/claude-local-execute.test.ts": 2771,
|
||||
"server/src/__tests__/claude-local-skill-sync.test.ts": 21,
|
||||
"server/src/__tests__/cleanup-removal-service.test.ts": 2248,
|
||||
"server/src/__tests__/cloud-upstreams.test.ts": 2576,
|
||||
"server/src/__tests__/codex-auth-reconciliation.test.ts": 46,
|
||||
"server/src/__tests__/codex-local-adapter.test.ts": 9,
|
||||
"server/src/__tests__/codex-local-execute.test.ts": 974,
|
||||
"server/src/__tests__/codex-local-skill-injection.test.ts": 29,
|
||||
"server/src/__tests__/codex-local-skill-sync.test.ts": 18,
|
||||
"server/src/__tests__/companies-service.test.ts": 3104,
|
||||
"server/src/__tests__/company-artifacts-service.test.ts": 2531,
|
||||
"server/src/__tests__/company-search-service.test.ts": 3127,
|
||||
"server/src/__tests__/company-skill-test-runs-service.test.ts": 3331,
|
||||
"server/src/__tests__/company-skills-catalog-service.test.ts": 3584,
|
||||
"server/src/__tests__/company-skills-detail.test.ts": 2258,
|
||||
"server/src/__tests__/company-skills-service.test.ts": 5470,
|
||||
"server/src/__tests__/company-skills.test.ts": 33,
|
||||
"server/src/__tests__/cursor-local-adapter-environment.test.ts": 232,
|
||||
"server/src/__tests__/cursor-local-adapter.test.ts": 11,
|
||||
"server/src/__tests__/cursor-local-execute.test.ts": 1028,
|
||||
"server/src/__tests__/cursor-local-skill-injection.test.ts": 15,
|
||||
"server/src/__tests__/cursor-local-skill-sync.test.ts": 22,
|
||||
"server/src/__tests__/dashboard-service.test.ts": 2222,
|
||||
"server/src/__tests__/dev-runner-output.test.ts": 25,
|
||||
"server/src/__tests__/dev-runner-paths.test.ts": 4,
|
||||
"server/src/__tests__/dev-runner-snapshot.test.ts": 8,
|
||||
"server/src/__tests__/dev-runner-worktree.test.ts": 7,
|
||||
"server/src/__tests__/dev-server-status.test.ts": 7,
|
||||
"server/src/__tests__/dev-watch-ignore.test.ts": 7,
|
||||
"server/src/__tests__/docker-entrypoint.test.ts": 39,
|
||||
"server/src/__tests__/document-annotations-service.test.ts": 2528,
|
||||
"server/src/__tests__/documents-service.test.ts": 2253,
|
||||
"server/src/__tests__/documents.test.ts": 4,
|
||||
"server/src/__tests__/effective-run-config-fingerprints.test.ts": 10,
|
||||
"server/src/__tests__/environment-config.test.ts": 11,
|
||||
"server/src/__tests__/environment-custom-image-terminal-ws.test.ts": 2566,
|
||||
"server/src/__tests__/environment-custom-images-service.test.ts": 3889,
|
||||
"server/src/__tests__/environment-execution-target.test.ts": 7,
|
||||
"server/src/__tests__/environment-live-ssh.test.ts": 717,
|
||||
"server/src/__tests__/environment-probe.test.ts": 12,
|
||||
"server/src/__tests__/environment-run-orchestrator.test.ts": 13,
|
||||
"server/src/__tests__/environment-runtime-driver-contract.test.ts": 2723,
|
||||
"server/src/__tests__/environment-runtime.test.ts": 3287,
|
||||
"server/src/__tests__/environment-service.test.ts": 2266,
|
||||
"server/src/__tests__/environment-test-harness.test.ts": 14,
|
||||
"server/src/__tests__/error-handler.test.ts": 12,
|
||||
"server/src/__tests__/execution-lock-orphan-cleanup.test.ts": 2333,
|
||||
"server/src/__tests__/execution-workspace-policy.test.ts": 9,
|
||||
"server/src/__tests__/execution-workspaces-service.test.ts": 6035,
|
||||
"server/src/__tests__/external-objects-service.test.ts": 2282,
|
||||
"server/src/__tests__/feedback-flush-controller.test.ts": 4,
|
||||
"server/src/__tests__/feedback-service.test.ts": 2520,
|
||||
"server/src/__tests__/feedback-share-client.test.ts": 9,
|
||||
"server/src/__tests__/file-resources.test.ts": 4757,
|
||||
"server/src/__tests__/first-admin-claim.test.ts": 2117,
|
||||
"server/src/__tests__/forbidden-tokens.test.ts": 6,
|
||||
"server/src/__tests__/gemini-local-adapter-environment.test.ts": 80,
|
||||
"server/src/__tests__/gemini-local-adapter.test.ts": 9,
|
||||
"server/src/__tests__/gemini-local-execute.test.ts": 220,
|
||||
"server/src/__tests__/gemini-local-skill-sync.test.ts": 18,
|
||||
"server/src/__tests__/grok-local-skill-sync.test.ts": 15,
|
||||
"server/src/__tests__/heartbeat-accepted-plan-workspace-refresh.test.ts": 5034,
|
||||
"server/src/__tests__/heartbeat-active-run-output-watchdog.test.ts": 49231,
|
||||
"server/src/__tests__/heartbeat-archived-company-guard.test.ts": 2303,
|
||||
"server/src/__tests__/heartbeat-auto-checkout.test.ts": 5,
|
||||
"server/src/__tests__/heartbeat-comment-wake-batching.test.ts": 5030,
|
||||
"server/src/__tests__/heartbeat-context-summary.test.ts": 8,
|
||||
"server/src/__tests__/heartbeat-issue-rewake-throttle.test.ts": 3198,
|
||||
"server/src/__tests__/heartbeat-list.test.ts": 2240,
|
||||
"server/src/__tests__/heartbeat-local-environment.test.ts": 2504,
|
||||
"server/src/__tests__/heartbeat-lock-release-on-reassignment.test.ts": 2129,
|
||||
"server/src/__tests__/heartbeat-model-profile.test.ts": 6,
|
||||
"server/src/__tests__/heartbeat-plugin-environment.test.ts": 2310,
|
||||
"server/src/__tests__/heartbeat-project-env.test.ts": 24,
|
||||
"server/src/__tests__/heartbeat-responsible-user-invariant.test.ts": 6285,
|
||||
"server/src/__tests__/heartbeat-retry-scheduling.test.ts": 3373,
|
||||
"server/src/__tests__/heartbeat-run-log.test.ts": 5,
|
||||
"server/src/__tests__/heartbeat-run-summary.test.ts": 6,
|
||||
"server/src/__tests__/heartbeat-runtime-skills.test.ts": 2681,
|
||||
"server/src/__tests__/heartbeat-runtime-state.test.ts": 2322,
|
||||
"server/src/__tests__/heartbeat-scheduling-suppression.test.ts": 5,
|
||||
"server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts": 11573,
|
||||
"server/src/__tests__/heartbeat-start-lock.test.ts": 8,
|
||||
"server/src/__tests__/heartbeat-timer-wake-session-reset-pf4.test.ts": 6,
|
||||
"server/src/__tests__/heartbeat-workspace-branch-containment.test.ts": 6144,
|
||||
"server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts": 3599,
|
||||
"server/src/__tests__/heartbeat-workspace-session.test.ts": 185,
|
||||
"server/src/__tests__/heartbeat-worktree-suppression.test.ts": 2665,
|
||||
"server/src/__tests__/heartbeat-zombie-guard.test.ts": 5,
|
||||
"server/src/__tests__/hire-hook.test.ts": 14,
|
||||
"server/src/__tests__/http-log-policy.test.ts": 5,
|
||||
"server/src/__tests__/inbox-dismissals.test.ts": 2110,
|
||||
"server/src/__tests__/instance-settings-service.test.ts": 15,
|
||||
"server/src/__tests__/instrumentation.test.ts": 27,
|
||||
"server/src/__tests__/invite-join-grants.test.ts": 6,
|
||||
"server/src/__tests__/issue-blocker-attention.test.ts": 3329,
|
||||
"server/src/__tests__/issue-comment-redaction.test.ts": 2383,
|
||||
"server/src/__tests__/issue-continuation-summary.test.ts": 6,
|
||||
"server/src/__tests__/issue-execution-policy.test.ts": 35,
|
||||
"server/src/__tests__/issue-goal-fallback.test.ts": 5,
|
||||
"server/src/__tests__/issue-liveness.test.ts": 14,
|
||||
"server/src/__tests__/issue-monitor-scheduler.test.ts": 6810,
|
||||
"server/src/__tests__/issue-recovery-actions.test.ts": 3556,
|
||||
"server/src/__tests__/issue-references-service.test.ts": 2184,
|
||||
"server/src/__tests__/issue-rewake-throttle.test.ts": 7,
|
||||
"server/src/__tests__/issue-thread-interactions-service.test.ts": 3053,
|
||||
"server/src/__tests__/issue-thread-interactions-telemetry.test.ts": 2209,
|
||||
"server/src/__tests__/issue-tree-control-service-unit.test.ts": 6,
|
||||
"server/src/__tests__/issue-tree-control-service.test.ts": 2498,
|
||||
"server/src/__tests__/issues-list-query-parsing.test.ts": 42,
|
||||
"server/src/__tests__/issues-user-context.test.ts": 5,
|
||||
"server/src/__tests__/join-request-dedupe.test.ts": 4,
|
||||
"server/src/__tests__/json-schema-secret-refs.test.ts": 4,
|
||||
"server/src/__tests__/live-events-ws.test.ts": 21,
|
||||
"server/src/__tests__/log-redaction.test.ts": 6,
|
||||
"server/src/__tests__/logger-tz.test.ts": 24,
|
||||
"server/src/__tests__/monthly-spend-service.test.ts": 29,
|
||||
"server/src/__tests__/openclaw-gateway-adapter.test.ts": 52,
|
||||
"server/src/__tests__/opencode-local-adapter.test.ts": 8,
|
||||
"server/src/__tests__/opencode-local-skill-sync.test.ts": 17,
|
||||
"server/src/__tests__/paperclip-env.test.ts": 5,
|
||||
"server/src/__tests__/paperclip-skill-utils.test.ts": 18,
|
||||
"server/src/__tests__/parse-status-filter.test.ts": 6,
|
||||
"server/src/__tests__/pi-local-adapter-environment.test.ts": 128,
|
||||
"server/src/__tests__/pi-local-execute.test.ts": 228,
|
||||
"server/src/__tests__/pi-local-skill-sync.test.ts": 17,
|
||||
"server/src/__tests__/pipelines-service.test.ts": 4478,
|
||||
"server/src/__tests__/plugin-access-authorization-host-services.test.ts": 2259,
|
||||
"server/src/__tests__/plugin-database.test.ts": 2422,
|
||||
"server/src/__tests__/plugin-dev-watcher.test.ts": 67,
|
||||
"server/src/__tests__/plugin-environment-driver-seam.test.ts": 55,
|
||||
"server/src/__tests__/plugin-execution-workspace-bridge.test.ts": 7,
|
||||
"server/src/__tests__/plugin-install-autobuild.test.ts": 4336,
|
||||
"server/src/__tests__/plugin-lifecycle-restart.test.ts": 7,
|
||||
"server/src/__tests__/plugin-local-folders.test.ts": 63,
|
||||
"server/src/__tests__/plugin-managed-agents.test.ts": 2407,
|
||||
"server/src/__tests__/plugin-managed-routines.test.ts": 2371,
|
||||
"server/src/__tests__/plugin-managed-skills.test.ts": 2600,
|
||||
"server/src/__tests__/plugin-orchestration-apis.test.ts": 2531,
|
||||
"server/src/__tests__/plugin-sdk-orchestration-contract.test.ts": 13,
|
||||
"server/src/__tests__/plugin-sdk-testing.test.ts": 10,
|
||||
"server/src/__tests__/plugin-secrets-handler.test.ts": 5,
|
||||
"server/src/__tests__/plugin-telemetry-bridge.test.ts": 36,
|
||||
"server/src/__tests__/plugin-tenant-isolation.test.ts": 2264,
|
||||
"server/src/__tests__/plugin-tool-dispatcher-pluginDbId.test.ts": 12,
|
||||
"server/src/__tests__/plugin-worker-manager.test.ts": 355,
|
||||
"server/src/__tests__/private-hostname-guard.test.ts": 39,
|
||||
"server/src/__tests__/productivity-review-service.test.ts": 5180,
|
||||
"server/src/__tests__/project-icon-persistence.test.ts": 2091,
|
||||
"server/src/__tests__/project-list-metrics.test.ts": 5,
|
||||
"server/src/__tests__/project-shortname-resolution.test.ts": 5,
|
||||
"server/src/__tests__/qa-routine-secrets-e2e.test.ts": 2364,
|
||||
"server/src/__tests__/quota-windows-service.test.ts": 7,
|
||||
"server/src/__tests__/quota-windows.test.ts": 59,
|
||||
"server/src/__tests__/recovery-classifiers.test.ts": 9,
|
||||
"server/src/__tests__/recovery-stale-issue-lock-sweep.test.ts": 2250,
|
||||
"server/src/__tests__/redact-sensitive.test.ts": 7,
|
||||
"server/src/__tests__/routine-run-telemetry.test.ts": 2157,
|
||||
"server/src/__tests__/routines-service.test.ts": 5411,
|
||||
"server/src/__tests__/run-continuations.test.ts": 7,
|
||||
"server/src/__tests__/run-liveness.test.ts": 11,
|
||||
"server/src/__tests__/runtime-api.test.ts": 6,
|
||||
"server/src/__tests__/sandbox-provider-runtime.test.ts": 9,
|
||||
"server/src/__tests__/secret-provider-registry.test.ts": 6,
|
||||
"server/src/__tests__/secrets-service.test.ts": 3281,
|
||||
"server/src/__tests__/server-info.test.ts": 300,
|
||||
"server/src/__tests__/server-package-build-script.test.ts": 3,
|
||||
"server/src/__tests__/server-startup-feedback-export.test.ts": 27,
|
||||
"server/src/__tests__/shared-telemetry-events.test.ts": 8,
|
||||
"server/src/__tests__/skills-catalog-service.test.ts": 39,
|
||||
"server/src/__tests__/source-trust.test.ts": 2204,
|
||||
"server/src/__tests__/static-index-html.test.ts": 28,
|
||||
"server/src/__tests__/storage-local-provider.test.ts": 23,
|
||||
"server/src/__tests__/task-watchdogs-classifier.test.ts": 9,
|
||||
"server/src/__tests__/task-watchdogs-scheduler.test.ts": 3245,
|
||||
"server/src/__tests__/teams-catalog-install-no-overrides.test.ts": 2995,
|
||||
"server/src/__tests__/teams-catalog-service.test.ts": 75,
|
||||
"server/src/__tests__/telemetry-client-flush.test.ts": 12,
|
||||
"server/src/__tests__/trust-preset-resolver.test.ts": 10,
|
||||
"server/src/__tests__/trust-proxy.test.ts": 11,
|
||||
"server/src/__tests__/ui-branding.test.ts": 6,
|
||||
"server/src/__tests__/version.test.ts": 11,
|
||||
"server/src/__tests__/vite-html-renderer.test.ts": 6,
|
||||
"server/src/__tests__/work-products.test.ts": 6,
|
||||
"server/src/__tests__/work-timeline-service.test.ts": 2541,
|
||||
"server/src/__tests__/workspace-runtime.test.ts": 28685,
|
||||
"server/src/__tests__/worktree-config.test.ts": 29,
|
||||
"server/src/adapters/claude-agent-id-header.test.ts": 6,
|
||||
"server/src/adapters/http/execute.test.ts": 7,
|
||||
"server/src/middleware/cloud-tenant-actor.test.ts": 10,
|
||||
"server/src/services/adapter-models-env.test.ts": 5,
|
||||
"server/src/services/adapter-registry-bootstrap.reconcile.test.ts": 7,
|
||||
"server/src/services/adapter-registry-bootstrap.test.ts": 7,
|
||||
"server/src/services/environment-custom-image-terminal-sessions.test.ts": 11,
|
||||
"server/src/services/execution-allowlist.test.ts": 6,
|
||||
"server/src/services/execution-policy-bootstrap.test.ts": 15,
|
||||
"server/src/services/heartbeat-run-runtime-status.test.ts": 9,
|
||||
"server/src/services/heartbeat-stop-metadata.test.ts": 6,
|
||||
"server/src/services/issue-thread-interactions.test.ts": 757,
|
||||
"server/src/services/recovery/model-profile-hint.test.ts": 4,
|
||||
"server/src/services/recovery/service.pause-durability.test.ts": 4,
|
||||
"server/src/services/recovery/successful-run-handoff.test.ts": 10,
|
||||
"server/src/services/responsible-user-denial-run-outcomes.test.ts": 8,
|
||||
"server/src/services/routines-formatter-cache.test.ts": 1029,
|
||||
"server/src/services/run-scratch.test.ts": 16,
|
||||
"server/src/services/session-workspace-cwd.test.ts": 4
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
import { readFileSync } from "node:fs";
|
||||
|
||||
// Fallback weight (ms) when the duration manifest is missing or empty.
|
||||
const FALLBACK_SUITE_WEIGHT_MS = 1000;
|
||||
|
||||
// Loads the per-suite duration manifest produced from a real PR run (see the
|
||||
// $comment field in scripts/general-server-shard-durations.json). Returns an
|
||||
// empty map on any read/parse problem so sharding degrades to uniform weights
|
||||
// instead of failing the test lane.
|
||||
export function loadShardDurations(manifestPath) {
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(readFileSync(manifestPath, "utf8"));
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
|
||||
const durations = parsed?.durations;
|
||||
if (!durations || typeof durations !== "object" || Array.isArray(durations)) {
|
||||
return {};
|
||||
}
|
||||
|
||||
const result = {};
|
||||
for (const [file, ms] of Object.entries(durations)) {
|
||||
if (typeof ms === "number" && Number.isFinite(ms) && ms >= 0) {
|
||||
result[file] = ms;
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// Weight assigned to suites absent from the manifest (new or renamed files).
|
||||
// The median keeps one unknown suite from skewing a shard the way a mean
|
||||
// dragged up by a few 30s+ suites would.
|
||||
export function defaultSuiteWeight(durations) {
|
||||
const values = Object.values(durations).sort((a, b) => a - b);
|
||||
if (values.length === 0) {
|
||||
return FALLBACK_SUITE_WEIGHT_MS;
|
||||
}
|
||||
const mid = Math.floor(values.length / 2);
|
||||
return values.length % 2 === 1 ? values[mid] : (values[mid - 1] + values[mid]) / 2;
|
||||
}
|
||||
|
||||
// Deterministic longest-processing-time partition: heaviest suite first, each
|
||||
// assigned to the currently lightest shard. Ties break by file path and then
|
||||
// by shard index, so every runner in the matrix computes the identical
|
||||
// partition from the same checkout — that invariant is what makes the shards
|
||||
// a complete, non-overlapping cover of the suite set.
|
||||
export function partitionGeneralServerSuites(files, shardCount, durations = {}) {
|
||||
const fallbackWeight = defaultSuiteWeight(durations);
|
||||
const weighted = files
|
||||
.map((file) => ({ file, weight: durations[file] ?? fallbackWeight }))
|
||||
.sort((a, b) => b.weight - a.weight || a.file.localeCompare(b.file));
|
||||
|
||||
const shards = Array.from({ length: shardCount }, () => ({ files: [], totalWeight: 0 }));
|
||||
for (const { file, weight } of weighted) {
|
||||
let target = 0;
|
||||
for (let index = 1; index < shards.length; index += 1) {
|
||||
if (shards[index].totalWeight < shards[target].totalWeight) {
|
||||
target = index;
|
||||
}
|
||||
}
|
||||
shards[target].files.push(file);
|
||||
shards[target].totalWeight += weight;
|
||||
}
|
||||
|
||||
for (const shard of shards) {
|
||||
shard.files.sort((a, b) => a.localeCompare(b));
|
||||
}
|
||||
return shards;
|
||||
}
|
||||
|
||||
export function selectGeneralServerShard(files, shardIndex, shardCount, durations = {}) {
|
||||
return partitionGeneralServerSuites(files, shardCount, durations)[shardIndex].files;
|
||||
}
|
||||
|
|
@ -3,8 +3,14 @@ import { spawnSync } from "node:child_process";
|
|||
import { mkdirSync, mkdtempSync, readdirSync, statSync } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { loadShardDurations, selectGeneralServerShard } from "./general-server-shard.mjs";
|
||||
|
||||
const repoRoot = process.cwd();
|
||||
const scriptsDir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const generalServerShardDurations = loadShardDurations(
|
||||
path.join(scriptsDir, "general-server-shard-durations.json"),
|
||||
);
|
||||
const serverRoot = path.join(repoRoot, "server");
|
||||
const serverSrcDir = path.join(repoRoot, "server", "src");
|
||||
const serverTestsDir = path.join(repoRoot, "server", "src", "__tests__");
|
||||
|
|
@ -288,8 +294,11 @@ function runProjectGroup(projects, groupName) {
|
|||
function runGeneralGroup(routeTests, groupName, shardIndex = null, shardCount = null) {
|
||||
if (groupName === generalServerGroupName) {
|
||||
if (shardCount !== null && shardCount > 1) {
|
||||
const shardFiles = generalServerTestFiles.filter(
|
||||
(_, index) => index % shardCount === shardIndex,
|
||||
const shardFiles = selectGeneralServerShard(
|
||||
generalServerTestFiles,
|
||||
shardIndex,
|
||||
shardCount,
|
||||
generalServerShardDurations,
|
||||
);
|
||||
console.log(
|
||||
`\n[test:run] general-server shard ${shardIndex + 1}/${shardCount} running ${shardFiles.length} of ${generalServerTestFiles.length} suites`,
|
||||
|
|
@ -369,6 +378,8 @@ const routeTests = walk(serverTestsDir)
|
|||
// dedicated serialized shards. Sharding this list across runners is what keeps
|
||||
// the general-server lane from becoming the PR critical path: the server vitest
|
||||
// config pins maxWorkers to 1, so the only way to parallelize is across jobs.
|
||||
// Suites are partitioned by recorded duration (scripts/general-server-shard.mjs)
|
||||
// rather than round-robin, so one slow suite cluster can't stretch a single shard.
|
||||
const generalServerTestFiles = walk(serverSrcDir)
|
||||
.map((file) => toRepoPath(file))
|
||||
.filter((repoPath) => repoPath.endsWith(".test.ts"))
|
||||
|
|
@ -396,8 +407,11 @@ if (options.dryRun) {
|
|||
options.mode === generalModeName &&
|
||||
options.group === generalServerGroupName &&
|
||||
options.shardCount !== null
|
||||
? generalServerTestFiles.filter(
|
||||
(_, index) => index % options.shardCount === options.shardIndex,
|
||||
? selectGeneralServerShard(
|
||||
generalServerTestFiles,
|
||||
options.shardIndex,
|
||||
options.shardCount,
|
||||
generalServerShardDurations,
|
||||
)
|
||||
: null,
|
||||
},
|
||||
|
|
|
|||
|
|
@ -74,19 +74,31 @@ describeEmbeddedPostgres("heartbeat issue rewake throttle", () => {
|
|||
if (!runs.some((run) => run.status === "queued" || run.status === "running")) break;
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
await db.delete(environmentLeases);
|
||||
await db.delete(issueComments);
|
||||
await db.delete(issues);
|
||||
await db.delete(heartbeatRunEvents);
|
||||
await db.delete(activityLog);
|
||||
await db.delete(heartbeatRuns);
|
||||
await db.delete(agentWakeupRequests);
|
||||
await db.delete(agentRuntimeState);
|
||||
await db.delete(agents);
|
||||
await db.delete(environments);
|
||||
await db.delete(executionWorkspaces);
|
||||
await db.delete(companySkills);
|
||||
await db.delete(companies);
|
||||
// Post-run bookkeeping (run-event records, follow-up wake scheduling) can
|
||||
// still write for a moment after a run reaches a terminal status, so a
|
||||
// single delete sweep can hit a foreign-key violation when a late insert
|
||||
// lands between two deletes. Retry the sweep until it goes through clean.
|
||||
for (let attempt = 0; ; attempt += 1) {
|
||||
try {
|
||||
await db.delete(environmentLeases);
|
||||
await db.delete(issueComments);
|
||||
await db.delete(issues);
|
||||
await db.delete(heartbeatRunEvents);
|
||||
await db.delete(activityLog);
|
||||
await db.delete(heartbeatRuns);
|
||||
await db.delete(agentWakeupRequests);
|
||||
await db.delete(agentRuntimeState);
|
||||
await db.delete(agents);
|
||||
await db.delete(environments);
|
||||
await db.delete(executionWorkspaces);
|
||||
await db.delete(companySkills);
|
||||
await db.delete(companies);
|
||||
break;
|
||||
} catch (error) {
|
||||
if (attempt >= 4) throw error;
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
|
|
|
|||
Loading…
Reference in New Issue