Commit Graph

1 Commits

Author SHA1 Message Date
Dotta 4a7172f5ac
feat(runner-e2e): improve matrix report browsing (#12889)
## Thinking Path

> - Paperclip is the open source app that people use to manage AI agents
for work.
> - The runner E2E system verifies agent profiles across supported
execution environments.
> - Its generated report is the main surface for inspecting those
results and their visual evidence.
> - Expanded matcher content could change the matrix column widths and
make comparisons difficult.
> - Screenshots also required extra navigation, and the report did not
have current-result search and filters.
> - This pull request stabilizes the matrix layout and makes visual
evidence directly browsable.
> - The benefit is faster inspection of retained test evidence without
another paid matrix run.

## Linked Issues or Issue Description

**What happened?**

The runner E2E report changed matrix column widths when a matcher table
expanded. The report also made screenshot comparison and current-result
discovery slower than necessary.

**Expected behavior**

The matrix columns must remain stable. Each retained screenshot must
appear as a thumbnail. The gallery must support keyboard navigation and
show the relevant execution metadata. The report must support
client-side search and filters.

**Steps to reproduce**

1. Open a runner E2E matrix report that contains retained screenshots.
2. Expand the matcher details in a matrix cell.
3. Observe the matrix column movement in the old report.

**Paperclip version or commit**

`8430bd897`

**Deployment mode**

Generated static runner E2E report.

## What Changed

- Keep matrix and matcher table widths stable when details expand.
- Show retained screenshot thumbnails in each test card.
- Add a full-screen evidence gallery with mouse, keyboard, and swipe
navigation.
- Show agent, environment, runtime, status, duration, token, and matcher
data in the gallery header.
- Add client-side search and profile, environment, suite, and status
filters below the report section tabs.
- Keep the filters in normal document flow while the report tabs remain
sticky.
- Add report generator assertions for the new layout and controls.
- Add Playwright coverage for filtering, stable matcher expansion, and
filtered gallery navigation.

## Verification

- `pnpm test:e2e:runner:typecheck`
- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/report.test.ts`
- `PAPERCLIP_PLAYWRIGHT_CHANNEL=chrome pnpm exec playwright test
--config tests/e2e/playwright.config.ts
tests/e2e/runner-e2e-dashboard.spec.ts`
- `node --test ./scripts/__tests__/e2e-shard.test.mjs`
- `pnpm check:token-gates`
- `pnpm -r typecheck`
- `pnpm test:run`
- `pnpm build`
- Regenerated the report from GitHub Actions run `33963318820` without
rerunning the matrix.
- Verified 121 retained thumbnails across 66 results in a local browser.
- Verified that expanded matchers keep matrix widths at 260, 486, and
486 pixels in a 1280-pixel viewport.
- Verified search, filters, modal metadata, and arrow-key gallery
navigation.

## Risks

Low risk. This change only modifies the static runner E2E report
generator and its tests. It does not change runner execution or retained
evidence data.

> This work is focused report polish. It does not duplicate planned core
work in `ROADMAP.md`.

## Model Used

OpenAI Codex with `gpt-5.6-sol`. Codex desktop managed the context
window. The model used reasoning, filesystem tools, code execution,
GitHub CLI access, and in-app browser verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-05 07:46:05 -05:00