## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud deployments wait for verified source commits.
> - Verification splits serialized server tests across independent
runners.
> - The shard duration estimates came from August and no longer match
current tests.
> - Stale estimates put much more work on one runner than the others.
> - This PR refreshes the estimates from a complete successful run to
balance the existing runners.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The time spent waiting for the slowest serialized server-test shard in
PR and release verification.
**Current behavior**
In [Cloud readiness run
34705914878](https://github.com/paperclipai/paperclip/actions/runs/34705914878),
the five serialized shards spent 479, 371, 322, 390, and 365 seconds
running tests. The recovery suite had a 55-second estimate but now takes
about 156 seconds including process overhead.
**Proposed behavior**
Use fresh per-suite measurements with the existing deterministic
duration balancer. Applying the same measured costs to the new
assignment gives 385, 385, 386, 385, and 385 seconds. This predicts
about 93 seconds less waiting for the slowest shard, before runner/setup
overhead. Live CI will confirm the result.
**Reason and benefit**
Use the existing runners more evenly. No extra runner, test parallelism,
cache, timeout, or routing change is needed.
**Breaking changes**
Suite-to-shard assignments change. The full suite set, assertions, and
per-suite process isolation stay the same.
**Additional context**
Searched related CI and shard PRs. This updates the existing duration
manifest, without duplicating a pending sharding implementation.
## What Changed
- Refresh all 145 serialized suite weights from the same successful
release verification run.
- Record source job IDs and the measurement method in the manifest.
Durations include process startup, imports, collection, tests, and
shutdown.
## Verification
- Passed all 30 shard and release-workflow tests: `node --test
scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs`.
- Confirmed every measured suite appears exactly once across all five
source logs.
- Compared old and new assignments using the same measured weights. The
maximum fell from 478862ms to 385551ms.
- In [PR CI run
34710696242](https://github.com/paperclipai/paperclip/actions/runs/34710696242),
all five serialized jobs passed in 7m05s–7m27s including setup. The
measured assignment is now balanced in a live run.
- The same run passed full typecheck, all grouped tests, native
verification, build, release dry run, and browser checks. Local
full-suite verification on this base was limited by disk exhaustion;
local typecheck and targeted shard tests passed.
- Latest-head Greptile is 5/5 with no open findings. Every current-head
CI check must be green or intentionally skipped before merge.
## Risks
- Individual durations vary with load and future test changes. These
estimates affect assignment only; missing or renamed suites receive the
existing median weight.
- Both PR and release verification read this manifest, so both receive
the new assignments. Each suite still runs in its own serialized Vitest
process.
## Model Used
OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100). -->
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The PR verify workflow gates every pull request; its wall-clock time
sets the feedback loop for all contributors
> - In a recent successful PR run (actions run 32012408876), the slowest
check was "Verify serialized server suites (1/5)" at 337s, while its
four sibling shards finished in 212-238s
> - The serialized lane assigns suites to shards round-robin over an
alphabetical list, so the heavy heartbeat and issues suites cluster on
one runner
> - The general-server lane already solves this with a duration-aware
LPT partition backed by a recorded manifest
> - This pull request reuses that partitioner for the serialized lane
with a fresh per-suite duration manifest
> - The benefit is a balanced serialized matrix: the measured 968s suite
total levels to about 194s per shard, which removes about 80-100s from
the run's slowest check
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The `Verify serialized server suites` shard matrix in
`.github/workflows/pr.yml` distributes route/authz test suites across
five runners.
**Subsystem affected**
CI / test infrastructure (`scripts/run-vitest-stable.mjs`).
**Current behavior**
`selectSerializedSuites` assigns suites round-robin (`index %
shardCount`) over the alphabetically sorted file list. The heavy suites
cluster on shard 1/5. In actions run 32012408876, shard 1/5 spent 291s
in its test step while the other shards spent 170-201s, which made that
job (337s total) the slowest check of the whole PR run.
**Proposed behavior**
Partition the serialized suites with the same duration-aware LPT
algorithm the general-server lane already uses
(`scripts/general-server-shard.mjs`), backed by a new per-suite duration
manifest. All five shards then carry about 194s of measured test time.
**Reason and benefit**
The slowest check bounds PR feedback time. Balancing the serialized
matrix removes about 80-100s from that bound without adding runners.
**Breaking changes**
None. The partition remains deterministic, complete, and
non-overlapping; suites missing from the manifest get the median weight.
## What Changed
- Added `scripts/serialized-shard-durations.json`: per-suite wall-clock
durations (ms) for all 134 serialized suites, sampled from actions run
32012408876 by diffing consecutive per-suite label timestamps in the
shard logs (captures vitest spawn overhead, not just reported test time)
- `scripts/run-vitest-stable.mjs`: `selectSerializedSuites` now uses the
existing LPT partitioner (`selectGeneralServerShard`) with the new
manifest instead of round-robin
- `scripts/__tests__/run-vitest-stable-shard.test.mjs`: added a
manifest-freshness test and a shard-balance test for the serialized
lane, mirroring the general-server ones
- `.github/workflows/pr.yml`: updated the serialized matrix comment with
the new measurement and mechanism
## Verification
- `node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs`
passes (13 tests), including the existing test that the serialized
shards form a complete, non-overlapping partition
- Dry-run of all five shards shows estimated totals of
194/194/194/194/193s (round-robin was 276/175/160/172/187s):
`node scripts/run-vitest-stable.mjs --mode serialized --shard-index N
--shard-count 5 --dry-run`
- The `Verify serialized server suites` jobs on this PR run the real
partition end to end
## Risks
- Low risk. Selection logic only; the vitest invocation per suite is
unchanged
- A stale manifest degrades gracefully: unknown suites get the median
weight, and a dedicated test fails if fewer than half the current suites
have recorded durations
## Model Used
- Claude (Anthropic), model ID `claude-fable-5`, agentic coding session
with tool use (Claude Code / Claude Agent SDK); no extended-thinking
mode
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Related prior work: #10923 (split serialized tests into five shards),
#10925 (general-server duration manifest), #11156 (workspaces-a native
shards).
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Devin Foley