fix: improve task search relevance with a PostgreSQL rubric (#13335)
Unify full and quick task search around PostgreSQL term coverage, explicit relevance bands, and conservative typo recovery. Preserve matching evidence and navigation, and add a judged corpus, regression tests, and documented performance measurements. Validation: local typecheck, build, focused PostgreSQL tests, and task-list tests pass. All final-head CI gates pass and Greptile is 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
GitHub
parent
04e364236b
commit
1652a5c7f2
|
|
@ -352,6 +352,19 @@ These browser suites are intended for targeted local verification and CI, not th
|
|||
|
||||
For normal issue work, start with the smallest targeted check that proves the change. Reserve repo-wide typecheck/build/test runs for PR-ready handoff or changes broad enough that narrow checks do not cover the risk.
|
||||
|
||||
### Task search evaluation
|
||||
|
||||
The task search relevance rubric and regression corpus are documented in
|
||||
[SEARCH.md](SEARCH.md). Run the real PostgreSQL relevance suite with:
|
||||
|
||||
```sh
|
||||
pnpm exec vitest run server/src/__tests__/task-search-quality.test.ts
|
||||
```
|
||||
|
||||
Set `SEARCH_EVAL_SCALE=1` to additionally measure a disposable 10,000-task,
|
||||
30,000-comment dataset. `SEARCH_EVAL_REPORT=/tmp/search-quality.json` saves
|
||||
per-query results and latency measurements; scale measurements are opt-in.
|
||||
|
||||
### Recent task ordering
|
||||
|
||||
The streamlined sidebar keeps five recent tasks per company and account in browser
|
||||
|
|
|
|||
|
|
@ -0,0 +1,200 @@
|
|||
# Task search relevance
|
||||
|
||||
## Product rubric
|
||||
|
||||
Search should help someone reopen work they remember, using whatever fragment
|
||||
stuck in memory: an ID, a few title words, a technical name, or something in the
|
||||
conversation. The first screen should contain plausible answers, with enough
|
||||
context to explain each match.
|
||||
|
||||
| Intent | Good result | Failure |
|
||||
|---|---|---|
|
||||
| Known task ID | Exact ID first, case-insensitive; accept `PAP-42`, `pap42`, `PAP 42` | A mention or neighboring ID beats the task |
|
||||
| Remembered title | Exact title, phrase, then all title words in any order | A recent comment mentioning those words beats the title |
|
||||
| Several concepts | Every meaningful query term contributes, including short terms such as API/UI | A task matches only one common word |
|
||||
| Exact phrase | Quoted text stays together and literal | Quotes silently behave like OR or fuzzy search |
|
||||
| Thread memory | Find words across task text, comments and current documents | Relevant content exists but the task cannot be found |
|
||||
| Technical text | Preserve underscores, percent signs, paths and numbers | SQL wildcard expansion or fuzzy IDs return unrelated work |
|
||||
| Typo | Conservative title-word correction; all other terms still required | Ignoring a short term changes the query's meaning |
|
||||
| Result explanation | Show the best evidence and link to its source | A title hit jumps into an unrelated comment |
|
||||
| Old work | Strong completed-task matches remain ahead of weak recent hits | Recency/activity replaces relevance |
|
||||
| Boundaries | Company, visibility, deletion and explicit filters always apply | Content leaks through counts, snippets or typo matches |
|
||||
| Operations | PostgreSQL only, synchronous current-row reads, bounded query/page sizes | A worker, remote index or eventual-consistency repair is required |
|
||||
|
||||
Judge results on a 0–3 scale: **3** directly answers the remembered task intent,
|
||||
**2** is useful related work, **1** is only an incidental mention, **0** is
|
||||
irrelevant. Ambiguous short queries may have several grade-3 answers; do not
|
||||
invent a unique intended task for them.
|
||||
|
||||
Acceptance gates:
|
||||
|
||||
- Every unambiguous known-task case returns its intended task first.
|
||||
- Every grade-3 result in the small judged corpus appears in the first five.
|
||||
- All explicit negative, visibility, filter, freshness and literal-query cases pass.
|
||||
- Report mean reciprocal rank (first grade-3 result) and nDCG@5 (graded ordering
|
||||
and recall). Target MRR ≥ 0.95 and nDCG@5 ≥ 0.90 on the authored corpus.
|
||||
- Measure both the full search page and the command-palette/task-list API.
|
||||
- Measure database-backed latency separately from relevance. Report dataset
|
||||
size, warm/cold assumptions and hardware; a small fixture is not scale proof.
|
||||
Initial target: warm p95 ≤ 250 ms at 10,000 tasks and 30,000 short comments.
|
||||
A regression greater than 20% from baseline requires investigation and an
|
||||
explicit explanation of the cost; do not describe a quality improvement as
|
||||
latency-neutral when it is not.
|
||||
|
||||
The initial corpus is synthetic and deliberately adversarial. It includes
|
||||
plausible distractors and gives older completed tasks strong relevance labels.
|
||||
It is not evidence that every real user's search is solved. Add real failed
|
||||
queries and human judgments as they become available. Do not adjust judgments
|
||||
just to improve a score.
|
||||
|
||||
## Previous behavior
|
||||
|
||||
The command palette calls the issue-list endpoint. It searched one literal
|
||||
substring across title, identifier, description and comments, prioritized titles
|
||||
before identifiers, and did not search documents or recover typos. Reordered
|
||||
words commonly returned no result.
|
||||
|
||||
Company search used a different algorithm: any token admitted a result, bonuses
|
||||
from titles, comments and documents accumulated, and title-only token coverage
|
||||
was indistinguishable from words scattered across a long thread. It ran edit
|
||||
distance for title words, discarded short terms from fuzzy matching, and also
|
||||
fuzzed identifiers. Quotes were tokenized but did not constrain other matches.
|
||||
|
||||
## Matching contract
|
||||
|
||||
Both task search paths use `server/src/services/task-search.ts`. Search is lexical:
|
||||
trim/collapse whitespace, normalize case, keep quoted phrases, remove a small
|
||||
set of unquoted grammatical filler words, deduplicate terms, and retain up to
|
||||
8 terms within the existing 200-character query bound. All-filler queries keep
|
||||
their terms. No synonym service, embedding model or language-specific stemming
|
||||
is involved.
|
||||
|
||||
All retained terms must match. Full search and task lists allow terms to occur
|
||||
across task text and current, undeleted conversation/document content. The
|
||||
Tasks scope requires coverage in task text. Comments and Documents require a
|
||||
participating match in that source, while retaining the task context. Exact/prefix
|
||||
identifiers and conservative title-word typo matches are additional task matches.
|
||||
Typo matching runs only when no literal match satisfies the requested filters.
|
||||
It never guesses task numbers, loosens a quoted phrase or drops a short query
|
||||
term. Alphabetic terms of one to three characters must begin a word, so `UI`
|
||||
does not match `build`, while incomplete longer words still support typeahead.
|
||||
|
||||
Ranking uses disjoint bands: exact ID, ID prefix, exact title, title phrase,
|
||||
all title terms, all task-text terms, all thread terms, then title typo recovery.
|
||||
Whole-word title matches and title prefixes break close ties; status has only a
|
||||
small effect within a band. Full search uses recency and stable IDs for remaining
|
||||
ties; task lists retain their existing priority/activity tie-breaking. Explicit
|
||||
created/updated/priority sort modes retain their documented behavior. Other
|
||||
entity types retain their existing scoring rules, rescaled to keep exact names
|
||||
ahead of speculative task typo matches. The UI displays the server's order
|
||||
without regrouping results by source.
|
||||
|
||||
The existing `pg_trgm` indexes support literal substring retrieval. Tagged
|
||||
comment/document match sets are computed once per search with separate indexed
|
||||
patterns. Ranking stages carry compact flags; descriptions and matching snippets
|
||||
are fetched for the result window. The database reads current rows, so creates, edits, deletions and
|
||||
hidden-task changes take effect without indexing jobs. Bounded edit-distance
|
||||
checks operate on titles only, run only as a zero-result fallback, and guard
|
||||
fuzzystrmatch's 255-character argument limit. There is no schema migration or
|
||||
new extension in this change.
|
||||
|
||||
PostgreSQL documents the existing index support in
|
||||
[pg_trgm](https://www.postgresql.org/docs/17/pgtrgm.html).
|
||||
|
||||
## Reproduce the evaluation
|
||||
|
||||
```sh
|
||||
pnpm exec vitest run server/src/__tests__/task-search-quality.test.ts
|
||||
# Also write per-query rankings and metrics for inspection:
|
||||
SEARCH_EVAL_REPORT=/tmp/search-quality.json pnpm exec vitest run server/src/__tests__/task-search-quality.test.ts
|
||||
# Include the larger latency dataset and query plans:
|
||||
SEARCH_EVAL_SCALE=1 SEARCH_EVAL_REPORT=/tmp/search-scale.json pnpm exec vitest run server/src/__tests__/task-search-quality.test.ts
|
||||
```
|
||||
|
||||
The fixture is `server/src/__tests__/fixtures/task-search-corpus.ts`. Tests run
|
||||
the real services against a temporary embedded PostgreSQL database with the
|
||||
normal migrations. `SEARCH_EVAL_BASELINE=1` records judgments without asserting
|
||||
improved behavior. To compare another revision, copy this test, its fixture and
|
||||
`task-search.ts` into a separate worktree for that revision, leave its actual
|
||||
`company-search.ts` and `issues.ts` services unchanged, and run with
|
||||
`SEARCH_EVAL_BASELINE=1`. The copied helper is not used by the baseline services;
|
||||
its query-plan branch is disabled in baseline mode.
|
||||
|
||||
## Initial evaluation — 2026-09-12
|
||||
|
||||
Compared against `2083bf6f9` using the same 31-task corpus and 24 queries (23
|
||||
queries with intended answers, plus one no-result query).
|
||||
|
||||
| Surface | Intended answer first, before → after | MRR, before → after | nDCG@5, before → after |
|
||||
|---|---|---|---|
|
||||
| Full search | 17/23 → 23/23 | 0.828 → 1.000 | 0.904 → 0.999 |
|
||||
| Quick search / task list | 5/23 → 23/23 | 0.268 → 1.000 | 0.339 → 0.999 |
|
||||
|
||||
The relevance gates pass. These results measure the authored corpus, not general
|
||||
search accuracy. The no-result query also returns no tasks in both surfaces.
|
||||
|
||||
The scale run adds 10,000 tasks with ~300-character descriptions and 30,000
|
||||
~345-character comments. Measurements call the real service methods (including
|
||||
facets/snippets or task-list hydration), excluding HTTP and UI debounce. Each
|
||||
query has one separately recorded first request and 20 warm repetitions; p95
|
||||
is the 19th sorted warm sample. This is not a cold-disk test. Both revisions used
|
||||
PostgreSQL 18.1, default planner/memory settings, and `ANALYZE` after seeding.
|
||||
The host was an Apple M5 Max with 128 GiB RAM, running an x86_64 PostgreSQL
|
||||
binary and other development tests concurrently. Treat timing deltas as local
|
||||
measurements, not production capacity or a controlled concurrency benchmark.
|
||||
|
||||
| Query | Full p95 before → after (ms) | Quick p95 before → after (ms) |
|
||||
|---|---|---|
|
||||
| `GitHub OAuth` | 139 → 95 | 39 → 128 |
|
||||
| `OAuth callback GitHub` | 209 → 81 | 26 → 134 |
|
||||
| `mibile api` | 153 → 131 | 19 → 111 |
|
||||
| `search` | 172 → 41 | 22 → 67 |
|
||||
| `quasarxylophone` | 154 → 105 | 18 → 218 |
|
||||
| `routine` (matches all 10,000 added tasks) | 239 → 253 | 152 → 371 |
|
||||
|
||||
Selective full searches improved. Quick search is more expensive: it now
|
||||
evaluates term coverage, searches documents, and can scan company titles for
|
||||
typo recovery. The old quick search returned no answers for the reordered and
|
||||
typo queries, so its lower cost did not deliver equivalent results. The relative
|
||||
regression threshold is triggered, and broad-query p95 does **not** meet the
|
||||
initial 250 ms target. This is an explicit performance limitation of this pass.
|
||||
The implementation adds no operational service, but it is not latency-neutral.
|
||||
|
||||
`EXPLAIN (ANALYZE, BUFFERS)` confirmed existing trigram indexes on selective
|
||||
comment/document retrieval and zero fuzzy-branch executions for successful
|
||||
literal searches. Removing descriptions from intermediate materialized rows
|
||||
eliminated 1,699 temporary blocks (~13 MiB) of writes in the broad-query core
|
||||
plan; its final measured execution was 139 ms with no temporary writes. The
|
||||
quick-search endpoint also performs its existing activity sorting and task
|
||||
hydration. Larger companies, long threads and sustained concurrent searches
|
||||
still need production-shaped measurement before a stronger latency claim.
|
||||
|
||||
Browser acceptance used the real built app against an isolated PostgreSQL
|
||||
database containing this corpus. Starting from the dashboard, the Search link
|
||||
found an older completed task from reordered title words and opened that task.
|
||||
Command-K ranked `PAP-42` first for `pap42`; `mibile api` recovered only the
|
||||
intended mobile API task and carried the query into full search. Quoted
|
||||
`"connection timeout"` excluded scattered words. `Hermes parser` showed the
|
||||
document title as evidence and opened the correct plan in the task's side panel.
|
||||
The desktop result layout was visually inspected. Mobile layout, continuous
|
||||
transition timing and production data were not part of this walkthrough.
|
||||
|
||||
For a future failed search, record the query, what the person remembered, and
|
||||
the intended task IDs. Grade the old top five and any missed intended tasks
|
||||
before changing the ranker, add realistic distractors, then run both entry
|
||||
points. Keep these judgments independent of the ranking constants.
|
||||
|
||||
Verification: 78 search/parser tests (including the real PostgreSQL scale run),
|
||||
14 existing task-list search/filter tests, and 29 Search/CommandPalette UI tests
|
||||
passed. Workspace typecheck, the final server typecheck, production build,
|
||||
Storybook build and token gates passed. The full repository test run was stopped
|
||||
after `chat-channels.integration.test.ts` reported one failure in “publishes a
|
||||
closed-choice question, settles its Slack card, and delivers its exact
|
||||
continuation response”; that test passed when rerun alone. The remaining broad
|
||||
suite was not completed, so this is not a claim of a green repository-wide run.
|
||||
|
||||
The API response contracts and company authorization stay unchanged. Artifact,
|
||||
agent and project ranking are separate from the task relevance rubric. Extraction
|
||||
search and the specialized blocked-attention queue retain their existing
|
||||
literal matching. Pure semantic paraphrases and
|
||||
language-specific word inflections are outside this first lexical rubric.
|
||||
|
|
@ -1142,6 +1142,9 @@ The current app also exposes V1-supporting surfaces for:
|
|||
- company-scoped summary slots for projects, the workspaces overview, project workspaces, and individual execution workspaces; execution-workspace slots are keyed by execution workspace id so a new workspace never inherits another workspace's summary
|
||||
- issue thread interactions (`suggest_tasks`, `ask_user_questions`, `request_confirmation`, `request_checkbox_confirmation`, `request_item_verdicts`) with the open-default resolver contract in §9.8.1
|
||||
- issue approvals, issue references/search, labels, read state, inbox/archive state, and work products
|
||||
- task search uses shared PostgreSQL matching/ranking for company search and task-list quick search;
|
||||
all query terms contribute, quoted phrases stay literal, exact identifiers and direct title matches
|
||||
lead relevance ordering, and the UI preserves server result order (see `doc/SEARCH.md`)
|
||||
- company search through `GET /companies/:companyId/search` plus agent-oriented bulk extraction through
|
||||
`GET /companies/:companyId/search/extract`; extraction accepts a server-escaped literal `contains`, optional
|
||||
server-owned URL expansion, issue/comment/document scopes, status/date filters, issue-level pagination, a
|
||||
|
|
|
|||
|
|
@ -586,3 +586,12 @@ allocation retains one selected number and individually enabled groups.
|
|||
|
||||
See [iMessage Photon](connections/IMESSAGE-PHOTON.md) for the implementation
|
||||
contract, setup, recovery, boundaries, and qualification status.
|
||||
|
||||
## Task search relevance
|
||||
|
||||
Task discovery uses PostgreSQL and the existing search indexes, with no external
|
||||
search service or background indexing job. The task-list quick search and full
|
||||
company search share lexical matching and ranking. Known identifiers and direct
|
||||
title matches lead; current conversation and document content supplies supporting
|
||||
evidence. See [Task search relevance](SEARCH.md) for the evaluation rubric,
|
||||
matching contract and reproducible quality tests.
|
||||
|
|
|
|||
|
|
@ -163,6 +163,19 @@ describeEmbeddedPostgres("companySearchService", () => {
|
|||
return id;
|
||||
}
|
||||
|
||||
it("keeps exact entity names ahead of speculative task typos and rejects empty quotes", async () => {
|
||||
const companyId = await createCompany();
|
||||
const agentId = await createAgent(companyId, { name: "Mibile" });
|
||||
const projectId = await createProject(companyId, { name: "Mibile" });
|
||||
const taskId = await createIssue(companyId, { title: "Mobile navigation" });
|
||||
const result = await svc.search(companyId, companySearchQuerySchema.parse({ q: "mibile" }));
|
||||
const ids = result.results.map((row) => row.id);
|
||||
expect(ids).toContain(taskId);
|
||||
expect(ids.indexOf(agentId)).toBeLessThan(ids.indexOf(taskId));
|
||||
expect(ids.indexOf(projectId)).toBeLessThan(ids.indexOf(taskId));
|
||||
expect((await svc.search(companyId, companySearchQuerySchema.parse({ q: '""' }))).results).toEqual([]);
|
||||
});
|
||||
|
||||
it("ranks exact issue identifiers before weaker title matches", async () => {
|
||||
const companyId = await createCompany();
|
||||
const exactId = await createIssue(companyId, {
|
||||
|
|
@ -180,7 +193,7 @@ describeEmbeddedPostgres("companySearchService", () => {
|
|||
expect(result.results[0]?.matchedFields).toContain("identifier");
|
||||
});
|
||||
|
||||
it("ranks phrase and all-token issue matches before partial scattered-token matches", async () => {
|
||||
it("ranks phrase before reordered title words and rejects partial matches", async () => {
|
||||
const companyId = await createCompany();
|
||||
const base = new Date("2026-01-01T00:00:00.000Z").getTime();
|
||||
const partialTokenId = await createIssue(companyId, {
|
||||
|
|
@ -201,7 +214,8 @@ describeEmbeddedPostgres("companySearchService", () => {
|
|||
|
||||
const result = await svc.search(companyId, companySearchQuerySchema.parse({ q: "alpha beta", scope: "issues" }));
|
||||
|
||||
expect(result.results.map((row) => row.id)).toEqual([phraseId, allTokenId, partialTokenId]);
|
||||
expect(result.results.map((row) => row.id)).toEqual([phraseId, allTokenId]);
|
||||
expect(result.results.map((row) => row.id)).not.toContain(partialTokenId);
|
||||
});
|
||||
|
||||
it("matches multiple tokens across the same issue thread and returns comment snippets", async () => {
|
||||
|
|
@ -682,6 +696,83 @@ describeEmbeddedPostgres("companySearchService", () => {
|
|||
}
|
||||
});
|
||||
|
||||
it("does not interpret short UI terms as the middle of unrelated words", async () => {
|
||||
const companyId = await createCompany();
|
||||
const target = await createIssue(companyId, { title: "Improve mobile UI" });
|
||||
await createIssue(companyId, { title: "Build billing reports" });
|
||||
const result = await svc.search(companyId, companySearchQuerySchema.parse({ q: "UI" }));
|
||||
expect(result.results.map((row) => row.id)).toEqual([target]);
|
||||
});
|
||||
|
||||
it("keeps typo fallback inside the requested filters", async () => {
|
||||
const companyId = await createCompany();
|
||||
await createIssue(companyId, { title: "Mibile API", status: "done" });
|
||||
const target = await createIssue(companyId, { title: "Mobile API", status: "todo" });
|
||||
const result = await svc.search(companyId, companySearchQuerySchema.parse({ q: "mibile api", status: "todo" }));
|
||||
expect(result.results.map((row) => row.id)).toEqual([target]);
|
||||
});
|
||||
|
||||
it("keeps exact title hits on the task and chooses the most complete context evidence", async () => {
|
||||
const companyId = await createCompany();
|
||||
const task = await createIssue(companyId, { title: "Aurora callback" });
|
||||
await db.insert(issueComments).values({ companyId, issueId: task, body: "Aurora callback is mentioned here too." });
|
||||
const exact = await svc.search(companyId, companySearchQuerySchema.parse({ q: "Aurora callback" }));
|
||||
expect(exact.results[0]?.href).not.toContain("#comment-");
|
||||
|
||||
const holder = await createIssue(companyId, { title: "Connection investigation" });
|
||||
await db.insert(issueComments).values({ companyId, issueId: holder, body: "Aurora was discussed.", updatedAt: new Date("2026-06-01") });
|
||||
const strongest = randomUUID();
|
||||
await db.insert(issueComments).values({ id: strongest, companyId, issueId: holder,
|
||||
body: "Aurora loses the callback state.", updatedAt: new Date("2026-01-01") });
|
||||
const result = await svc.search(companyId, companySearchQuerySchema.parse({ q: "aurora state callback" }));
|
||||
expect(result.results[0]?.href).toContain(`#comment-${strongest}`);
|
||||
expect(result.results[0]?.snippet).toContain("state");
|
||||
});
|
||||
|
||||
it.each(["comment", "document"] as const)("preserves %s evidence when title and identifier both match", async (source) => {
|
||||
const companyId = await createCompany();
|
||||
const task = await createIssue(companyId, {
|
||||
identifier: "CTX-123", title: "CTX callback investigation", description: "CTX callback details",
|
||||
});
|
||||
const sourceId = randomUUID();
|
||||
if (source === "comment") {
|
||||
await db.insert(issueComments).values({ id: sourceId, companyId, issueId: task, body: "The missing signal is quasar." });
|
||||
} else {
|
||||
await db.insert(documents).values({ id: sourceId, companyId, title: "Investigation plan", latestBody: "The missing signal is quasar.", format: "markdown" });
|
||||
await db.insert(issueDocuments).values({ companyId, issueId: task, documentId: sourceId, key: "plan" });
|
||||
}
|
||||
|
||||
const result = await svc.search(companyId, companySearchQuerySchema.parse({ q: "CTX quasar" }));
|
||||
const match = result.results.find((row) => row.id === task)!;
|
||||
expect(match.matchedFields).toEqual(expect.arrayContaining(["identifier", "title", source]));
|
||||
expect(match.score).toBeGreaterThanOrEqual(2000);
|
||||
expect(match.score).toBeLessThan(3000);
|
||||
expect(match.snippets).toHaveLength(2);
|
||||
expect(match.snippets[0]).toMatchObject({ field: source, text: expect.stringContaining("quasar") });
|
||||
expect(match.snippet).toContain("quasar");
|
||||
expect(match.href).toContain(source === "comment" ? `#comment-${sourceId}` : "#document-plan");
|
||||
});
|
||||
|
||||
it("reflects edits, deleted comments and document updates immediately", async () => {
|
||||
const companyId = await createCompany();
|
||||
const task = await createIssue(companyId, { title: "Old uniquequartz title" });
|
||||
const find = () => svc.search(companyId, companySearchQuerySchema.parse({ q: '"uniquequartz"' }));
|
||||
expect((await find()).results.map((row) => row.id)).toEqual([task]);
|
||||
await db.update(issues).set({ title: "New title" }).where(sql`${issues.id} = ${task}`);
|
||||
expect((await find()).results).toEqual([]);
|
||||
const comment = randomUUID();
|
||||
await db.insert(issueComments).values({ id: comment, companyId, issueId: task, body: "uniquequartz" });
|
||||
expect((await find()).results.map((row) => row.id)).toEqual([task]);
|
||||
await db.update(issueComments).set({ deletedAt: new Date() }).where(sql`${issueComments.id} = ${comment}`);
|
||||
expect((await find()).results).toEqual([]);
|
||||
const doc = randomUUID();
|
||||
await db.insert(documents).values({ id: doc, companyId, title: "Findings", latestBody: "uniquequartz", format: "markdown" });
|
||||
await db.insert(issueDocuments).values({ companyId, issueId: task, documentId: doc, key: "plan" });
|
||||
expect((await find()).results.map((row) => row.id)).toEqual([task]);
|
||||
await db.update(documents).set({ latestBody: "Updated findings" }).where(sql`${documents.id} = ${doc}`);
|
||||
expect((await find()).results).toEqual([]);
|
||||
});
|
||||
|
||||
it("uses pg_trgm for conservative fuzzy title matches", async () => {
|
||||
const companyId = await createCompany();
|
||||
const issueId = await createIssue(companyId, {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,77 @@
|
|||
// Authored relevance judgments, independent of the ranker's scoring constants.
|
||||
export const taskSearchCorpus = [
|
||||
{ key: "id", identifier: "PAP-42", title: "Repair callback state", status: "done" },
|
||||
{ key: "id-mention", title: "PAP-42 follow-up discussion" },
|
||||
{ key: "id-neighbor", identifier: "PAP-420", title: "Repair callback state later" },
|
||||
{ key: "oauth", title: "Fix GitHub OAuth callback", status: "done" },
|
||||
{ key: "oauth-noise", title: "GitHub release checklist", comments: ["OAuth is mentioned in an unrelated weekly update."] },
|
||||
{ key: "oauth-partial", title: "GitHub repository badges" },
|
||||
{ key: "oauth-body", title: "Repair the connection flow", description: "GitHub OAuth callback loses state on redirect." },
|
||||
{ key: "oauth-comment", title: "Investigate sign-in", comments: ["The GitHub OAuth callback loses state on redirect."] },
|
||||
{ key: "oauth-doc", title: "Connection investigation", document: { title: "GitHub OAuth callback findings", body: "State is lost on redirect." } },
|
||||
{ key: "search", title: "Improve search performance" },
|
||||
{ key: "search-noise", title: "Improve exports", description: "A search for performance numbers was included in the meeting." },
|
||||
{ key: "search-partial", title: "Search typography" },
|
||||
{ key: "mobile", title: "Polish mobile navigation" },
|
||||
{ key: "mobile-api", title: "Build mobile API" },
|
||||
{ key: "mobile-ui", title: "Build mobile UI" },
|
||||
{ key: "onboarding", title: "Onboarding wizard polish" },
|
||||
{ key: "quoted", title: "Repair connection timeout handling" },
|
||||
{ key: "quoted-scattered", title: "Connection retry after timeout" },
|
||||
{ key: "cross", title: "Checkout ownership", comments: ["A concurrency race needs a regression test."] },
|
||||
{ key: "cross-partial", title: "Checkout style guide" },
|
||||
{ key: "document", title: "Adapter investigation", document: { title: "Hermes parser plan", body: "Discover plugins from their package manifest." } },
|
||||
{ key: "percentage", title: "Release 100% checklist" },
|
||||
{ key: "percentage-decoy", title: "Release 1000 checklist" },
|
||||
{ key: "path", title: "Fix foo_bar lookup" },
|
||||
{ key: "path-decoy", title: "Fix fooXbar lookup" },
|
||||
{ key: "unicode", title: "Réparer navigation mobile" },
|
||||
{ key: "identifier-code", title: "Document heartbeat_run_events retention" },
|
||||
{ key: "word", title: "Fix API authentication" },
|
||||
{ key: "word-decoy", title: "Capistrano migration", description: "API details appear here.", comments: ["API is an incidental mention."] },
|
||||
{ key: "freshness", title: "Reconcile billing ledger", status: "done" },
|
||||
{ key: "freshness-noise", title: "Weekly financial update", comments: ["Reconcile billing ledger was one of many completed projects."] },
|
||||
] as const;
|
||||
|
||||
export type TaskSearchCase = {
|
||||
name: string;
|
||||
q: string;
|
||||
relevant: Record<string, number>; // 3 = intended task; 2 = useful; 1 = incidental; absent = irrelevant
|
||||
first?: string;
|
||||
absent?: string[];
|
||||
scope?: "all" | "issues" | "comments" | "documents";
|
||||
};
|
||||
export const taskSearchCases: TaskSearchCase[] = [
|
||||
{ name: "exact identifier", q: "PAP-42", relevant: { id: 3, "id-mention": 1, "id-neighbor": 1 }, first: "id" },
|
||||
{ name: "identifier case", q: "pap-42", relevant: { id: 3, "id-mention": 1, "id-neighbor": 1 }, first: "id" },
|
||||
{ name: "compact identifier", q: "pap42", relevant: { id: 3, "id-neighbor": 1 }, first: "id" },
|
||||
{ name: "spaced identifier", q: "PAP 42", relevant: { id: 3, "id-mention": 1, "id-neighbor": 1 }, first: "id" },
|
||||
{ name: "title phrase", q: "GitHub OAuth", relevant: { oauth: 3, "oauth-body": 2, "oauth-comment": 2, "oauth-doc": 2, "oauth-noise": 1 }, first: "oauth", absent: ["oauth-partial"] },
|
||||
{ name: "reordered title words", q: "OAuth GitHub callback", relevant: { oauth: 3, "oauth-body": 2, "oauth-comment": 2, "oauth-doc": 2 }, first: "oauth", absent: ["oauth-partial", "oauth-noise"] },
|
||||
{ name: "title beats body chatter", q: "performance search", relevant: { search: 3, "search-noise": 1 }, first: "search", absent: ["search-partial"] },
|
||||
{ name: "title beats incidental comment", q: "billing ledger", relevant: { freshness: 3, "freshness-noise": 1 }, first: "freshness" },
|
||||
{ name: "filler words", q: "the GitHub OAuth callback", relevant: { oauth: 3, "oauth-body": 2, "oauth-comment": 2, "oauth-doc": 2 }, first: "oauth", absent: ["oauth-noise"] },
|
||||
{ name: "quoted phrase", q: '"connection timeout"', relevant: { quoted: 3 }, first: "quoted", absent: ["quoted-scattered"] },
|
||||
{ name: "quoted phrase plus term", q: 'repair "connection timeout"', relevant: { quoted: 3 }, first: "quoted", absent: ["quoted-scattered"] },
|
||||
{ name: "transposition", q: "serach", relevant: { search: 3, "search-partial": 3 } },
|
||||
{ name: "substitution", q: "mibile navigation", relevant: { mobile: 3, unicode: 3 }, absent: ["mobile-api", "mobile-ui"] },
|
||||
{ name: "two missing letters", q: "onbordng wizard", relevant: { onboarding: 3 }, first: "onboarding" },
|
||||
{ name: "short token constrains typo", q: "mibile api", relevant: { "mobile-api": 3 }, first: "mobile-api", absent: ["mobile", "mobile-ui"] },
|
||||
{ name: "cross-field thread", q: "checkout concurrency", relevant: { cross: 3 }, first: "cross", absent: ["cross-partial"] },
|
||||
{ name: "document title", q: "Hermes parser", relevant: { document: 3 }, first: "document" },
|
||||
{ name: "document body", q: "plugins manifest", relevant: { document: 3 }, first: "document" },
|
||||
{ name: "literal percent", q: "100%", relevant: { percentage: 3 }, first: "percentage", absent: ["percentage-decoy"] },
|
||||
{ name: "literal underscore", q: "foo_bar", relevant: { path: 3 }, first: "path", absent: ["path-decoy"] },
|
||||
{ name: "code identifier", q: "heartbeat_run_events", relevant: { "identifier-code": 3 }, first: "identifier-code" },
|
||||
{ name: "unicode", q: "réparer mobile", relevant: { unicode: 3 }, first: "unicode" },
|
||||
{ name: "whole word title", q: "api", relevant: { word: 3, "mobile-api": 3, "word-decoy": 1 } },
|
||||
{ name: "no result", q: "quasarxylophone", relevant: {} },
|
||||
];
|
||||
|
||||
export function searchQualityMetrics(keys: string[], relevant: Record<string, number>) {
|
||||
const gain = (grade: number, index: number) => (2 ** grade - 1) / Math.log2(index + 2);
|
||||
const dcg = keys.slice(0, 5).reduce((sum, key, index) => sum + gain(relevant[key] ?? 0, index), 0);
|
||||
const ideal = Object.values(relevant).sort((a, b) => b - a).slice(0, 5).reduce((sum, grade, index) => sum + gain(grade, index), 0);
|
||||
const rank = keys.findIndex((key) => relevant[key] === 3);
|
||||
return { ndcg5: ideal === 0 ? Number(keys.length === 0) : dcg / ideal, reciprocalRank: rank < 0 ? 0 : 1 / (rank + 1) };
|
||||
}
|
||||
|
|
@ -1466,7 +1466,7 @@ describeEmbeddedPostgres("issueService.list participantAgentId", () => {
|
|||
expect(result.map((issue) => issue.id)).toEqual([recentMediumIssueId]);
|
||||
});
|
||||
|
||||
it("ranks comment matches ahead of description-only matches", async () => {
|
||||
it("ranks direct description matches ahead of comment-only matches", async () => {
|
||||
const companyId = randomUUID();
|
||||
const commentMatchId = randomUUID();
|
||||
const descriptionMatchId = randomUUID();
|
||||
|
|
@ -1508,7 +1508,7 @@ describeEmbeddedPostgres("issueService.list participantAgentId", () => {
|
|||
includeRoutineExecutions: true,
|
||||
});
|
||||
|
||||
expect(result.map((issue) => issue.id)).toEqual([commentMatchId, descriptionMatchId]);
|
||||
expect(result.map((issue) => issue.id)).toEqual([descriptionMatchId, commentMatchId]);
|
||||
});
|
||||
|
||||
it("filters issue lists to the full descendant tree for a root issue", async () => {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,147 @@
|
|||
import { randomUUID } from "node:crypto";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { cpus, platform, release, totalmem } from "node:os";
|
||||
import { sql } from "drizzle-orm";
|
||||
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||
import { companies, createDb, documents, issueComments, issueDocuments, issues, getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "@paperclipai/db";
|
||||
import { companySearchQuerySchema } from "@paperclipai/shared";
|
||||
import { companySearchService } from "../services/company-search.js";
|
||||
import { parseTaskSearch, taskSearchCtes, taskSearchScore } from "../services/task-search.js";
|
||||
import { issueService } from "../services/issues.js";
|
||||
import { searchQualityMetrics, taskSearchCases, taskSearchCorpus } from "./fixtures/task-search-corpus.js";
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
const baseline = process.env.SEARCH_EVAL_BASELINE === "1";
|
||||
|
||||
describe.skipIf(!support.supported)("task search relevance rubric (real PostgreSQL)", () => {
|
||||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
const companyId = randomUUID();
|
||||
const keys = new Map<string, string>();
|
||||
const plans: Record<string, unknown> = {};
|
||||
const latency: Array<{ engine: string; q: string; firstMs: number; p95Ms: number; warmMs: number[] }> = [];
|
||||
let postgresVersion = "";
|
||||
const report: Array<{ engine: string; name: string; q: string; keys: string[]; ndcg5: number; reciprocalRank: number; ms: number }> = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-search-quality-");
|
||||
db = createDb(tempDb.connectionString);
|
||||
postgresVersion = String((await db.execute(sql`SELECT version()`))[0]!.version);
|
||||
await db.insert(companies).values({ id: companyId, name: "Search benchmark", issuePrefix: "EVAL" });
|
||||
for (const [index, entry] of taskSearchCorpus.entries()) {
|
||||
const id = randomUUID();
|
||||
keys.set(id, entry.key);
|
||||
const updatedAt = new Date(Date.UTC(2026, 0, 1) + index * 60_000);
|
||||
await db.insert(issues).values({ id, companyId, title: entry.title,
|
||||
identifier: "identifier" in entry ? entry.identifier : `EVAL-${index + 1}`,
|
||||
description: "description" in entry ? entry.description : null,
|
||||
status: "status" in entry ? entry.status : "todo", updatedAt, createdAt: updatedAt });
|
||||
if ("comments" in entry) {
|
||||
for (const body of entry.comments) await db.insert(issueComments).values({ companyId, issueId: id, body, updatedAt });
|
||||
}
|
||||
if ("document" in entry) {
|
||||
const documentId = randomUUID();
|
||||
await db.insert(documents).values({ id: documentId, companyId, title: entry.document.title, latestBody: entry.document.body, format: "markdown" });
|
||||
await db.insert(issueDocuments).values({ companyId, issueId: id, documentId, key: "plan" });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
const summary = ["full", "quick"].map((engine) => {
|
||||
const rows = report.filter((row) => row.engine === engine);
|
||||
const known = rows.filter((row) => Object.keys(taskSearchCases.find((item) => item.name === row.name)!.relevant).length > 0);
|
||||
return { engine, queries: rows.length, ndcg5: rows.reduce((sum, row) => sum + row.ndcg5, 0) / rows.length,
|
||||
mrr: known.reduce((sum, row) => sum + row.reciprocalRank, 0) / known.length };
|
||||
});
|
||||
console.log("SEARCH QUALITY", JSON.stringify(summary));
|
||||
if (process.env.SEARCH_EVAL_REPORT) await writeFile(process.env.SEARCH_EVAL_REPORT, JSON.stringify({
|
||||
environment: { postgresVersion, platform: platform(), release: release(), cpu: cpus()[0]?.model, memoryBytes: totalmem(), warmSamples: 20 },
|
||||
summary, queries: report, latency, plans,
|
||||
}, null, 2));
|
||||
await tempDb?.cleanup();
|
||||
});
|
||||
|
||||
for (const engine of ["full", "quick"] as const) {
|
||||
for (const testCase of taskSearchCases) {
|
||||
it(`${engine}: ${testCase.name}`, async () => {
|
||||
const start = performance.now();
|
||||
const rows = engine === "full"
|
||||
? (await companySearchService(db).search(companyId, companySearchQuerySchema.parse({ q: testCase.q }))).results.filter((row) => row.type === "issue")
|
||||
: await issueService(db).list(companyId, { q: testCase.q, limit: 50 });
|
||||
const resultKeys = rows.map((row) => keys.get(row.id)!);
|
||||
report.push({ engine, name: testCase.name, q: testCase.q, keys: resultKeys, ...searchQualityMetrics(resultKeys, testCase.relevant), ms: performance.now() - start });
|
||||
if (baseline) return;
|
||||
if (testCase.first) expect(resultKeys[0], JSON.stringify(resultKeys)).toBe(testCase.first);
|
||||
for (const absent of testCase.absent ?? []) expect(resultKeys).not.toContain(absent);
|
||||
for (const [key, grade] of Object.entries(testCase.relevant)) if (grade === 3) expect(resultKeys.slice(0, 5)).toContain(key);
|
||||
if (Object.keys(testCase.relevant).length === 0) expect(resultKeys).toEqual([]);
|
||||
});
|
||||
}
|
||||
}
|
||||
it("meets the aggregate relevance gates", () => {
|
||||
if (baseline) return;
|
||||
for (const engine of ["full", "quick"]) {
|
||||
const rows = report.filter((row) => row.engine === engine);
|
||||
const known = rows.filter((row) => taskSearchCases.find((entry) => entry.name === row.name)!.q !== "quasarxylophone");
|
||||
expect(known.reduce((sum, row) => sum + row.reciprocalRank, 0) / known.length).toBeGreaterThanOrEqual(0.95);
|
||||
expect(rows.reduce((sum, row) => sum + row.ndcg5, 0) / rows.length).toBeGreaterThanOrEqual(0.90);
|
||||
}
|
||||
});
|
||||
|
||||
it("handles empty quotes, literal punctuation, and oversized title words without errors", async () => {
|
||||
if (baseline) return;
|
||||
await db.insert(issues).values({ companyId, title: "x".repeat(300) });
|
||||
for (const q of ['""', "%", "_", "\\", "z".repeat(200)]) {
|
||||
const full = await companySearchService(db).search(companyId, companySearchQuerySchema.parse({ q }));
|
||||
const quick = await issueService(db).list(companyId, { q, limit: 50 });
|
||||
if (q === '""' || q.startsWith("z")) {
|
||||
expect(full.results).toEqual([]);
|
||||
expect(quick).toEqual([]);
|
||||
} else {
|
||||
for (const row of quick) expect(row.title).toContain(q);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it.runIf(process.env.SEARCH_EVAL_SCALE === "1")("measures 10k tasks / 30k comments", async () => {
|
||||
await db.execute(sql`
|
||||
INSERT INTO issues (company_id, title, description, identifier)
|
||||
SELECT ${companyId}, 'Routine deployment checkpoint ' || n,
|
||||
repeat('Review the build output and update the deployment checklist. ', 5), 'SCALE-' || n
|
||||
FROM generate_series(1, 10000) n
|
||||
`);
|
||||
await db.execute(sql`
|
||||
INSERT INTO issue_comments (company_id, issue_id, body)
|
||||
SELECT ${companyId}, id, repeat('Routine progress report: verified the output and recorded the findings. ', 5)
|
||||
FROM issues CROSS JOIN generate_series(1, 3) n
|
||||
WHERE company_id = ${companyId} AND identifier LIKE 'SCALE-%'
|
||||
`);
|
||||
await db.execute(sql`ANALYZE issues`);
|
||||
await db.execute(sql`ANALYZE issue_comments`);
|
||||
for (const engine of ["full", "quick"] as const) {
|
||||
for (const q of ["GitHub OAuth", "OAuth callback GitHub", "mibile api", "search", "quasarxylophone", "routine"]) {
|
||||
const durations: number[] = [];
|
||||
for (let i = 0; i < 21; i++) {
|
||||
const start = performance.now();
|
||||
if (engine === "full") await companySearchService(db).search(companyId, companySearchQuerySchema.parse({ q }));
|
||||
else await issueService(db).list(companyId, { q, limit: 20 });
|
||||
durations.push(performance.now() - start);
|
||||
}
|
||||
const warmMs = durations.slice(1);
|
||||
latency.push({ engine, q, firstMs: durations[0]!, p95Ms: [...warmMs].sort((a, b) => a - b)[18]!, warmMs });
|
||||
}
|
||||
}
|
||||
for (const q of ["GitHub OAuth", "mibile api", "routine"]) {
|
||||
const search = parseTaskSearch(q);
|
||||
if (!baseline) plans[q] = await db.execute(sql`
|
||||
EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)
|
||||
${taskSearchCtes(companyId, search)}
|
||||
SELECT m.id, ${taskSearchScore(search)} AS score FROM matched m ORDER BY score DESC LIMIT 20
|
||||
`);
|
||||
}
|
||||
console.log("SEARCH LATENCY", JSON.stringify(latency.map(({ warmMs: _warmMs, ...row }) => row)));
|
||||
}, 120_000);
|
||||
|
||||
});
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import { parseTaskSearch } from "../services/task-search.js";
|
||||
|
||||
describe("task search query intent", () => {
|
||||
it("keeps negation, short domain terms and quoted filler", () => {
|
||||
expect(parseTaskSearch('the API is not "in the UI"').tokens).toEqual(["api", "is", "not", "in the ui"]);
|
||||
expect(parseTaskSearch("the and").tokens).toEqual(["the", "and"]);
|
||||
});
|
||||
|
||||
it("retains the strictest intent for repeated terms", () => {
|
||||
expect(parseTaskSearch('callback "callback"').terms).toEqual([{ text: "callback", quoted: true }]);
|
||||
});
|
||||
|
||||
it("normalizes task identifiers without guessing their numbers", () => {
|
||||
for (const q of ["PAP-42", "pap42", "PAP 42"]) expect(parseTaskSearch(q).identifierQuery).toBe("pap-42");
|
||||
expect(parseTaskSearch("T123-42").identifierQuery).toBe("t123-42");
|
||||
expect(parseTaskSearch("PAP-420").identifierQuery).toBe("pap-420");
|
||||
});
|
||||
});
|
||||
|
|
@ -15,7 +15,6 @@ import {
|
|||
import {
|
||||
COMPANY_SEARCH_MAX_LIMIT,
|
||||
COMPANY_SEARCH_MAX_OFFSET,
|
||||
COMPANY_SEARCH_MAX_TOKENS,
|
||||
COMPANY_SEARCH_UPDATED_WITHIN_OPTIONS,
|
||||
COMPANY_ARTIFACTS_MAX_LIMIT,
|
||||
COMPANY_ARTIFACTS_MAX_QUERY_LENGTH,
|
||||
|
|
@ -39,18 +38,8 @@ import {
|
|||
import { companyArtifactsService } from "./company-artifacts.js";
|
||||
import { companySearchExtractService } from "./company-search-extract.js";
|
||||
import { visibleIssueCondition } from "./issue-visibility.js";
|
||||
import { parseTaskSearch, taskSearchCtes, taskSearchScore, taskSearchFieldMatch, taskSearchTermMatch } from "./task-search.js";
|
||||
|
||||
const MIN_TOKEN_LENGTH = 2;
|
||||
const MIN_FUZZY_QUERY_LENGTH = 4;
|
||||
const MIN_FUZZY_TOKEN_LENGTH = 4;
|
||||
// Cap fuzzy edits using the shorter of (query token, title word) so common
|
||||
// 4–5 letter English words don't sweep in noise (e.g. "serach" vs "each").
|
||||
const FUZZY_PAIR_LONG_LENGTH = 6;
|
||||
const FUZZY_PAIR_LONG_MAX_EDITS = 2;
|
||||
const FUZZY_PAIR_MEDIUM_LENGTH = 5;
|
||||
const FUZZY_PAIR_MEDIUM_MAX_EDITS = 1;
|
||||
const FUZZY_PAIR_SHORT_MAX_EDITS = 0;
|
||||
const FUZZY_IDENTIFIER_SIMILARITY_THRESHOLD = 0.45;
|
||||
const SNIPPET_MAX_CHARS = 240;
|
||||
export const COMPANY_SEARCH_BRANCH_FETCH_LIMIT = COMPANY_SEARCH_MAX_OFFSET + COMPANY_SEARCH_MAX_LIMIT + 1;
|
||||
|
||||
|
|
@ -95,30 +84,6 @@ type SearchAggregateRow = {
|
|||
count: number | string;
|
||||
};
|
||||
|
||||
function normalizeQuery(query: string) {
|
||||
return query.trim().replace(/\s+/g, " ").toLowerCase();
|
||||
}
|
||||
|
||||
function escapeLikePattern(value: string): string {
|
||||
return value.replace(/[\\%_]/g, "\\$&");
|
||||
}
|
||||
|
||||
function tokenizeQuery(normalizedQuery: string) {
|
||||
const matches = normalizedQuery.match(/"[^"]+"|[^\s]+/g) ?? [];
|
||||
const tokens: string[] = [];
|
||||
for (const match of matches) {
|
||||
const token = match.replace(/^"|"$/g, "").replace(/^[^\p{L}\p{N}%_\\-]+|[^\p{L}\p{N}%_\\-]+$/gu, "");
|
||||
if (token.length < MIN_TOKEN_LENGTH) continue;
|
||||
if (!tokens.includes(token)) tokens.push(token);
|
||||
if (tokens.length >= COMPANY_SEARCH_MAX_TOKENS) break;
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function fuzzyEligibleTokens(tokens: string[]): string[] {
|
||||
return tokens.filter((token) => token.length >= MIN_FUZZY_TOKEN_LENGTH);
|
||||
}
|
||||
|
||||
function sqlTextArray(values: string[]) {
|
||||
if (values.length === 0) return sql`ARRAY[]::text[]`;
|
||||
return sql`ARRAY[${sql.join(values.map((value) => sql`${value}`), sql`, `)}]::text[]`;
|
||||
|
|
@ -138,7 +103,7 @@ function plainText(value: string | null | undefined) {
|
|||
.replace(/```[\s\S]*?```/g, " ")
|
||||
.replace(/`([^`]+)`/g, "$1")
|
||||
.replace(/\[([^\]]+)\]\([^)]+\)/g, "$1")
|
||||
.replace(/[#>*_~|]+/g, " ")
|
||||
.replace(/(^|\s)[#>*_~|]+|[#>*_~|]+(?=\s|$)/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
}
|
||||
|
|
@ -431,6 +396,8 @@ function scopeIncludesProjects(scope: CompanySearchScope) {
|
|||
|
||||
function selectPrimarySnippets(row: IssueSearchRow, normalizedQuery: string, tokens: string[]) {
|
||||
const terms = matchTerms(normalizedQuery, tokens);
|
||||
const identifierQuery = parseTaskSearch(normalizedQuery).identifierQuery;
|
||||
if (!terms.includes(identifierQuery)) terms.push(identifierQuery);
|
||||
const matchedFields = new Set(row.matchedFields ?? []);
|
||||
const candidates: Array<CompanySearchSnippet | null> = [];
|
||||
if (matchedFields.has("identifier")) {
|
||||
|
|
@ -439,15 +406,22 @@ function selectPrimarySnippets(row: IssueSearchRow, normalizedQuery: string, tok
|
|||
if (matchedFields.has("title")) {
|
||||
candidates.push(createSnippet("title", "Title", row.title, terms));
|
||||
}
|
||||
if (matchedFields.has("comment")) {
|
||||
candidates.push(createSnippet("comment", "Comment", row.commentSnippet, terms));
|
||||
}
|
||||
if (matchedFields.has("document")) {
|
||||
candidates.push(createSnippet("document", row.documentTitle || "Document", row.documentSnippet, terms));
|
||||
}
|
||||
if (matchedFields.has("description")) {
|
||||
candidates.push(createSnippet("description", "Description", row.description, terms));
|
||||
}
|
||||
const description = matchedFields.has("description")
|
||||
? createSnippet("description", "Description", row.description, terms) : null;
|
||||
const directMatch = Number(row.score) >= 3000 || Number(row.score) < 2000;
|
||||
if (directMatch) candidates.push(description);
|
||||
const context = [
|
||||
{ field: "comment", label: "Comment", text: row.commentSnippet },
|
||||
{ field: "document", label: row.documentTitle || "Document", text: [row.documentTitle, row.documentSnippet].filter(Boolean).join(" ") },
|
||||
].filter((source) => matchedFields.has(source.field));
|
||||
const coverage = (text: string | null) => tokens.filter((term) => (text ?? "").toLowerCase().includes(term)).length;
|
||||
context.sort((left, right) => coverage(right.text) - coverage(left.text));
|
||||
const contextSnippets = context.map((source) => createSnippet(source.field, source.label, source.text, terms));
|
||||
// The title and identifier are already visible in the row. For thread-only
|
||||
// coverage, preserve the evidence before applying the two-snippet limit.
|
||||
if (directMatch) candidates.push(...contextSnippets);
|
||||
else candidates.unshift(...contextSnippets);
|
||||
if (!directMatch) candidates.push(description);
|
||||
return candidates.filter((snippet): snippet is CompanySearchSnippet => Boolean(snippet)).slice(0, 2);
|
||||
}
|
||||
|
||||
|
|
@ -456,7 +430,11 @@ function issueResult(row: IssueSearchRow, prefix: string, normalizedQuery: strin
|
|||
const sourceLabel = snippets[0]?.label ?? null;
|
||||
const documentSuffix = row.documentKey ? `#document-${encodeURIComponent(row.documentKey)}` : "";
|
||||
const commentSuffix = row.commentId ? `#comment-${encodeURIComponent(row.commentId)}` : "";
|
||||
const suffix = row.commentId ? commentSuffix : documentSuffix;
|
||||
// Direct task matches open the task; context matches open the evidence shown.
|
||||
const directMatch = Number(row.score) >= 3000 || Number(row.score) < 2000;
|
||||
const evidence = snippets.find((snippet) => snippet.field === "comment" || snippet.field === "document");
|
||||
const suffix = directMatch ? "" : evidence?.field === "comment" ? commentSuffix
|
||||
: evidence?.field === "document" ? documentSuffix : "";
|
||||
const issue: CompanySearchIssueSummary = {
|
||||
id: row.id,
|
||||
identifier: row.identifier,
|
||||
|
|
@ -495,7 +473,9 @@ function scoreSimpleRow(row: SimpleSearchRow, normalizedQuery: string, tokens: s
|
|||
if (haystack.includes(token)) score += 20;
|
||||
}
|
||||
if (row.title.toLowerCase().startsWith(normalizedQuery)) score += 80;
|
||||
return score;
|
||||
// Keep other entity types on the same scale as the task relevance bands:
|
||||
// an exact agent/project name must still outrank a speculative task typo.
|
||||
return score * 10;
|
||||
}
|
||||
|
||||
function artifactResult(artifact: CompanyArtifact, normalizedQuery: string, tokens: string[]): CompanySearchResult {
|
||||
|
|
@ -559,9 +539,10 @@ export function companySearchService(db: Db) {
|
|||
return {
|
||||
extract: extractService.extract,
|
||||
search: async (companyId: string, query: CompanySearchQuery): Promise<CompanySearchResponse> => {
|
||||
const normalizedQuery = normalizeQuery(query.q);
|
||||
const taskSearch = parseTaskSearch(query.q);
|
||||
const normalizedQuery = taskSearch.normalizedQuery;
|
||||
const hasSearchText = normalizedQuery.length > 0;
|
||||
const tokens = tokenizeQuery(normalizedQuery);
|
||||
const tokens = taskSearch.tokens;
|
||||
const scope = query.scope;
|
||||
const sort = query.sort;
|
||||
const limit = query.limit;
|
||||
|
|
@ -583,147 +564,20 @@ export function companySearchService(db: Db) {
|
|||
}
|
||||
|
||||
const fetchLimit = companySearchBranchFetchLimit(limit, offset);
|
||||
const escapedTokens = tokens.map(escapeLikePattern);
|
||||
// LIKE/ILIKE both treat backslash as the default escape character, so the
|
||||
// escaped tokens stay literal inside ILIKE ANY(...) patterns too.
|
||||
const tokenPatterns = escapedTokens.map((token) => `%${token}%`);
|
||||
const tokenPatternArray = sqlTextArray(tokenPatterns);
|
||||
const fuzzyTokens = fuzzyEligibleTokens(tokens);
|
||||
const fuzzyTokenArray = sqlTextArray(fuzzyTokens);
|
||||
const escapedQuery = escapeLikePattern(normalizedQuery);
|
||||
const containsPattern = hasSearchText ? `%${escapedQuery}%` : "__paperclip_no_match__";
|
||||
const startsWithPattern = hasSearchText ? `${escapedQuery}%` : "__paperclip_no_match__";
|
||||
const fuzzyEnabled = hasSearchText && normalizedQuery.length >= MIN_FUZZY_QUERY_LENGTH && !/[\\%_]/.test(normalizedQuery);
|
||||
const fuzzyTokensEnabled = fuzzyEnabled && fuzzyTokens.length > 0;
|
||||
const tokenPatternArray = sqlTextArray(taskSearch.patterns);
|
||||
const containsPattern = hasSearchText && tokens.length > 0 ? taskSearch.containsPattern : "__paperclip_no_match__";
|
||||
const tokenCount = tokens.length;
|
||||
|
||||
// --- shared match expressions against the `issues` table -------------
|
||||
// Raw-column ILIKE keeps the predicates compatible with the existing
|
||||
// pg_trgm GIN indexes (lower(col) LIKE expressions cannot use them).
|
||||
const titlePhraseMatch = hasSearchText ? sql<boolean>`issues.title ILIKE ${containsPattern}` : noMatchSql();
|
||||
const titleStartsWith = hasSearchText ? sql<boolean>`issues.title ILIKE ${startsWithPattern}` : noMatchSql();
|
||||
const titleExactMatch = hasSearchText ? sql<boolean>`lower(issues.title) = ${normalizedQuery}` : noMatchSql();
|
||||
const identifierPhraseMatch = hasSearchText ? sql<boolean>`coalesce(issues.identifier, '') ILIKE ${containsPattern}` : noMatchSql();
|
||||
const identifierStartsWith = hasSearchText ? sql<boolean>`coalesce(issues.identifier, '') ILIKE ${startsWithPattern}` : noMatchSql();
|
||||
const identifierExactMatch = hasSearchText ? sql<boolean>`lower(coalesce(issues.identifier, '')) = ${normalizedQuery}` : noMatchSql();
|
||||
const descriptionPhraseMatch = hasSearchText ? sql<boolean>`coalesce(issues.description, '') ILIKE ${containsPattern}` : noMatchSql();
|
||||
const titleTokenMatch = tokenCount > 0 ? sql<boolean>`issues.title ILIKE ANY(${tokenPatternArray})` : noMatchSql();
|
||||
const identifierTokenMatch = tokenCount > 0 ? sql<boolean>`coalesce(issues.identifier, '') ILIKE ANY(${tokenPatternArray})` : noMatchSql();
|
||||
const descriptionTokenMatch = tokenCount > 0 ? sql<boolean>`coalesce(issues.description, '') ILIKE ANY(${tokenPatternArray})` : noMatchSql();
|
||||
// Comment/document matches are computed once per request into tagged
|
||||
// CTEs (issue_id, ord) where ord 1 is the phrase pattern and ord k+1 is
|
||||
// token k. Flags and per-token coverage become cheap hashed IN probes
|
||||
// against those sets instead of per-issue-row correlated subqueries.
|
||||
// Single-pattern queries stay a bare `col ILIKE pattern` so the pg_trgm
|
||||
// GIN indexes can bitmap-scan them; multi-pattern queries use one tagged
|
||||
// pass over the table (an OR/ANY form would seq-scan anyway).
|
||||
const matchPatterns = hasSearchText
|
||||
? [containsPattern, ...tokenPatterns.filter((pattern) => pattern !== containsPattern)]
|
||||
: [];
|
||||
const matchPatternOrdinal = (pattern: string) => matchPatterns.indexOf(pattern) + 1;
|
||||
const matchPatternArray = sqlTextArray(matchPatterns);
|
||||
const commentMatchesCte = !hasSearchText
|
||||
? sql`SELECT NULL::uuid AS issue_id, 0 AS ord WHERE false`
|
||||
: matchPatterns.length === 1
|
||||
? sql`
|
||||
SELECT search_comments.issue_id, 1 AS ord
|
||||
FROM issue_comments search_comments
|
||||
WHERE search_comments.company_id = ${companyId}
|
||||
AND search_comments.deleted_at IS NULL
|
||||
AND search_comments.body ILIKE ${matchPatterns[0]!}
|
||||
GROUP BY 1, 2
|
||||
`
|
||||
: sql`
|
||||
SELECT search_comments.issue_id, pat.ord::int AS ord
|
||||
FROM issue_comments search_comments
|
||||
INNER JOIN unnest(${matchPatternArray}) WITH ORDINALITY AS pat(pattern, ord)
|
||||
ON search_comments.body ILIKE pat.pattern
|
||||
WHERE search_comments.company_id = ${companyId}
|
||||
AND search_comments.deleted_at IS NULL
|
||||
GROUP BY 1, 2
|
||||
`;
|
||||
// Documents get one UNION ALL arm per pattern (each arm a bare
|
||||
// `col ILIKE pattern`) so the planner can pick a pg_trgm bitmap scan per
|
||||
// pattern; latest_body is large enough that skipping the seq scan for
|
||||
// selective patterns dwarfs the duplicate-recheck cost on common ones.
|
||||
const documentMatchesCte = !hasSearchText
|
||||
? sql`SELECT NULL::uuid AS issue_id, 0 AS ord WHERE false`
|
||||
: sql.join(matchPatterns.map((pattern, index) => sql`
|
||||
SELECT search_issue_documents.issue_id, ${index + 1}::int AS ord
|
||||
FROM issue_documents search_issue_documents
|
||||
INNER JOIN documents search_documents
|
||||
ON search_documents.id = search_issue_documents.document_id
|
||||
AND search_documents.company_id = search_issue_documents.company_id
|
||||
WHERE search_issue_documents.company_id = ${companyId}
|
||||
AND (
|
||||
search_documents.title ILIKE ${pattern}
|
||||
OR search_documents.latest_body ILIKE ${pattern}
|
||||
)
|
||||
GROUP BY 1, 2
|
||||
`), sql` UNION ALL `);
|
||||
const commentMatch = hasSearchText
|
||||
? sql<boolean>`issues.id IN (SELECT comment_matches.issue_id FROM comment_matches)`
|
||||
: noMatchSql();
|
||||
const documentMatch = hasSearchText
|
||||
? sql<boolean>`issues.id IN (SELECT document_matches.issue_id FROM document_matches)`
|
||||
: noMatchSql();
|
||||
// Each query token (length >= MIN_FUZZY_TOKEN_LENGTH) must have at least
|
||||
// one title word within Levenshtein edit distance. This handles typos
|
||||
// like "serach" -> "search" (transposition) and "mibile" -> "mobile"
|
||||
// (substitution) without the trigram noise that drop-character variants
|
||||
// produced (e.g. "serac" matching "service"). Edit budget is gated on
|
||||
// the SHORTER of the two strings so 4–5 letter English words don't get
|
||||
// swept in by lev=2 collisions.
|
||||
const fuzzyMaxEditsExpr = sql.raw(
|
||||
`CASE
|
||||
WHEN least(length(qt.value), length(title_word.value)) >= ${FUZZY_PAIR_LONG_LENGTH} THEN ${FUZZY_PAIR_LONG_MAX_EDITS}
|
||||
WHEN least(length(qt.value), length(title_word.value)) >= ${FUZZY_PAIR_MEDIUM_LENGTH} THEN ${FUZZY_PAIR_MEDIUM_MAX_EDITS}
|
||||
ELSE ${FUZZY_PAIR_SHORT_MAX_EDITS}
|
||||
END`,
|
||||
);
|
||||
const fuzzyMinTitleWordLengthExpr = sql.raw(`${MIN_FUZZY_TOKEN_LENGTH}`);
|
||||
const fuzzyTokenTitleMatch = fuzzyTokensEnabled
|
||||
? sql<boolean>`
|
||||
coalesce((
|
||||
SELECT bool_and(
|
||||
EXISTS (
|
||||
SELECT 1
|
||||
FROM regexp_split_to_table(lower(issues.title), '[^a-z0-9]+') AS title_word(value)
|
||||
WHERE length(title_word.value) >= ${fuzzyMinTitleWordLengthExpr}
|
||||
AND levenshtein_less_equal(qt.value, title_word.value, ${fuzzyMaxEditsExpr}) <= ${fuzzyMaxEditsExpr}
|
||||
)
|
||||
)
|
||||
FROM unnest(${fuzzyTokenArray}) AS qt(value)
|
||||
), false)
|
||||
`
|
||||
: noMatchSql();
|
||||
const fuzzyIdentifierMatch = fuzzyEnabled
|
||||
? sql<boolean>`similarity(lower(coalesce(issues.identifier, '')), ${normalizedQuery}) >= ${FUZZY_IDENTIFIER_SIMILARITY_THRESHOLD}`
|
||||
: noMatchSql();
|
||||
|
||||
const issueTextMatch = sql<boolean>`(
|
||||
${titlePhraseMatch}
|
||||
OR ${identifierPhraseMatch}
|
||||
OR ${descriptionPhraseMatch}
|
||||
OR ${titleTokenMatch}
|
||||
OR ${identifierTokenMatch}
|
||||
OR ${descriptionTokenMatch}
|
||||
)`;
|
||||
const fuzzyMatch = sql<boolean>`(${fuzzyTokenTitleMatch} OR ${fuzzyIdentifierMatch})`;
|
||||
const anySearchMatch = sql<boolean>`(${issueTextMatch} OR ${commentMatch} OR ${documentMatch} OR ${fuzzyMatch})`;
|
||||
|
||||
const issueFilters = issueFilterConditions(companyId, query);
|
||||
const hasIssueOnlyFilters = issueOnlyFiltersActive(query);
|
||||
|
||||
// Scope conditions over precomputed flag columns (alias-qualified).
|
||||
function flagTextMatch(alias: string) {
|
||||
return sql<boolean>`(
|
||||
${sql.raw(alias)}.title_phrase OR ${sql.raw(alias)}.ident_phrase OR ${sql.raw(alias)}.desc_phrase
|
||||
OR ${sql.raw(alias)}.title_token OR ${sql.raw(alias)}.ident_token OR ${sql.raw(alias)}.desc_token
|
||||
)`;
|
||||
return sql<boolean>`(${sql.raw(alias)}.issue_coverage = ${tokenCount}
|
||||
OR ${sql.raw(alias)}.ident_exact OR ${sql.raw(alias)}.ident_starts)`;
|
||||
}
|
||||
function flagFuzzyMatch(alias: string) {
|
||||
return sql<boolean>`(${sql.raw(alias)}.fuzzy_title OR ${sql.raw(alias)}.fuzzy_ident)`;
|
||||
return sql<boolean>`${sql.raw(alias)}.fuzzy_title`;
|
||||
}
|
||||
function flagScopeCondition(alias: string, forScope: CompanySearchScope): SQL<boolean> {
|
||||
if (!hasSearchText) {
|
||||
|
|
@ -732,7 +586,7 @@ export function companySearchService(db: Db) {
|
|||
if (forScope === "comments") return sql<boolean>`${sql.raw(alias)}.comment_match`;
|
||||
if (forScope === "documents") return sql<boolean>`${sql.raw(alias)}.document_match`;
|
||||
if (forScope === "issues") return sql<boolean>`(${flagTextMatch(alias)} OR ${flagFuzzyMatch(alias)})`;
|
||||
return sql<boolean>`(${flagTextMatch(alias)} OR ${sql.raw(alias)}.comment_match OR ${sql.raw(alias)}.document_match OR ${flagFuzzyMatch(alias)})`;
|
||||
return sql<boolean>`true`;
|
||||
}
|
||||
|
||||
// --- combined issue results + aggregates statement ---------------------
|
||||
|
|
@ -764,24 +618,7 @@ export function companySearchService(db: Db) {
|
|||
const wantResultRows = scopeIncludesIssues(scope)
|
||||
&& !(!hasSearchText && (scope === "comments" || scope === "documents"));
|
||||
if (wantResultRows) {
|
||||
const allTokensBonus = tokenCount > 0
|
||||
? sql`CASE WHEN m.token_coverage = ${tokenCount} THEN 260 ELSE 0 END`
|
||||
: sql`0`;
|
||||
const scoreSql = sql`(
|
||||
CASE WHEN m.ident_exact THEN 1200 ELSE 0 END
|
||||
+ CASE WHEN m.ident_starts THEN 700 ELSE 0 END
|
||||
+ CASE WHEN m.title_exact THEN 900 ELSE 0 END
|
||||
+ CASE WHEN m.title_starts THEN 550 ELSE 0 END
|
||||
+ CASE WHEN m.title_phrase THEN 350 ELSE 0 END
|
||||
+ CASE WHEN m.ident_phrase THEN 320 ELSE 0 END
|
||||
+ CASE WHEN m.comment_match THEN 180 ELSE 0 END
|
||||
+ CASE WHEN m.document_match THEN 170 ELSE 0 END
|
||||
+ CASE WHEN m.desc_phrase THEN 120 ELSE 0 END
|
||||
+ ${allTokensBonus}
|
||||
+ (m.token_coverage * 70)
|
||||
+ CASE WHEN (m.fuzzy_title OR m.fuzzy_ident) THEN 110 ELSE 0 END
|
||||
+ CASE m.status WHEN 'done' THEN 0 WHEN 'cancelled' THEN -30 ELSE 20 END
|
||||
)::double precision`;
|
||||
const scoreSql = taskSearchScore(taskSearch);
|
||||
const priorityOrderSql = sql`CASE m.priority WHEN 'critical' THEN 0 WHEN 'high' THEN 1 WHEN 'medium' THEN 2 WHEN 'low' THEN 3 ELSE 4 END`;
|
||||
const orderBySql = sort === "updated"
|
||||
? sql`m.updated_at DESC, score DESC, m.id DESC`
|
||||
|
|
@ -798,7 +635,8 @@ export function companySearchService(db: Db) {
|
|||
m.id,
|
||||
m.identifier,
|
||||
m.title,
|
||||
m.description,
|
||||
(SELECT issue_text.description FROM issues issue_text
|
||||
WHERE issue_text.id = m.id AND issue_text.company_id = ${companyId}) AS description,
|
||||
m.status,
|
||||
m.priority,
|
||||
m.assignee_agent_id AS "assigneeAgentId",
|
||||
|
|
@ -808,9 +646,9 @@ export function companySearchService(db: Db) {
|
|||
m.updated_at AS "updatedAt",
|
||||
${scoreSql} AS score,
|
||||
array_remove(ARRAY[
|
||||
CASE WHEN m.ident_phrase OR m.ident_token OR m.fuzzy_ident THEN 'identifier' END,
|
||||
CASE WHEN m.title_phrase OR m.title_token OR m.fuzzy_title THEN 'title' END,
|
||||
CASE WHEN m.desc_phrase OR m.desc_token THEN 'description' END,
|
||||
CASE WHEN m.ident_exact OR m.ident_starts OR m.ident_phrase OR m.ident_token THEN 'identifier' END,
|
||||
CASE WHEN m.title_token OR m.fuzzy_title THEN 'title' END,
|
||||
CASE WHEN m.desc_token THEN 'description' END,
|
||||
CASE WHEN m.comment_match THEN 'comment' END,
|
||||
CASE WHEN m.document_match THEN 'document' END
|
||||
], NULL)::text[] AS "matchedFields"
|
||||
|
|
@ -825,10 +663,10 @@ export function companySearchService(db: Db) {
|
|||
branches.push(sql`SELECT 'type:issue' AS kind, NULL::text AS value, count(*)::int AS count ${countTail} FROM matched m ${branchWhere([...facetsAll, titleCond])}`);
|
||||
}
|
||||
if (hasSearchText && (scope === "all" || scope === "comments")) {
|
||||
branches.push(sql`SELECT 'type:comment' AS kind, NULL::text AS value, count(*)::int AS count ${countTail} FROM matched m ${branchWhere([...facetsAll, sql`m.comment_match`])}`);
|
||||
branches.push(sql`SELECT 'type:comment' AS kind, NULL::text AS value, count(*)::int AS count ${countTail} FROM matched m ${branchWhere([...facetsAll, flagScopeCondition("m", "comments")])}`);
|
||||
}
|
||||
if (hasSearchText && (scope === "all" || scope === "documents")) {
|
||||
branches.push(sql`SELECT 'type:document' AS kind, NULL::text AS value, count(*)::int AS count ${countTail} FROM matched m ${branchWhere([...facetsAll, sql`m.document_match`])}`);
|
||||
branches.push(sql`SELECT 'type:document' AS kind, NULL::text AS value, count(*)::int AS count ${countTail} FROM matched m ${branchWhere([...facetsAll, flagScopeCondition("m", "documents")])}`);
|
||||
}
|
||||
|
||||
const facetBranch = (kind: string, valueSql: SQL, omit: CompanySearchIssueFilterKey, extra: SQL[] = []) => sql`
|
||||
|
|
@ -878,59 +716,8 @@ export function companySearchService(db: Db) {
|
|||
}
|
||||
}
|
||||
|
||||
// Per-token coverage counts matches across issue text and the tagged
|
||||
// comment/document match sets (hashed IN probes, one set per token).
|
||||
const coverageSql = tokenCount > 0
|
||||
? sql`(${sql.join(tokens.map((_, index) => {
|
||||
const pattern = tokenPatterns[index]!;
|
||||
const ord = matchPatternOrdinal(pattern);
|
||||
return sql`(CASE WHEN
|
||||
issues.title ILIKE ${pattern}
|
||||
OR coalesce(issues.identifier, '') ILIKE ${pattern}
|
||||
OR coalesce(issues.description, '') ILIKE ${pattern}
|
||||
OR issues.id IN (SELECT comment_matches.issue_id FROM comment_matches WHERE comment_matches.ord = ${ord})
|
||||
OR issues.id IN (SELECT document_matches.issue_id FROM document_matches WHERE document_matches.ord = ${ord})
|
||||
THEN 1 ELSE 0 END)`;
|
||||
}), sql` + `)})`
|
||||
: sql`0`;
|
||||
|
||||
const matchedWhere = hasSearchText ? sql` AND ${anySearchMatch}` : sql``;
|
||||
const resultRows = await db.execute(sql`
|
||||
WITH comment_matches AS MATERIALIZED (${commentMatchesCte}),
|
||||
document_matches AS MATERIALIZED (${documentMatchesCte}),
|
||||
matched AS MATERIALIZED (
|
||||
SELECT
|
||||
issues.id,
|
||||
issues.identifier,
|
||||
issues.title,
|
||||
issues.description,
|
||||
issues.status,
|
||||
issues.priority,
|
||||
issues.assignee_agent_id,
|
||||
issues.assignee_user_id,
|
||||
issues.project_id,
|
||||
issues.created_at,
|
||||
issues.updated_at,
|
||||
${titlePhraseMatch} AS title_phrase,
|
||||
${titleStartsWith} AS title_starts,
|
||||
${titleExactMatch} AS title_exact,
|
||||
${identifierPhraseMatch} AS ident_phrase,
|
||||
${identifierStartsWith} AS ident_starts,
|
||||
${identifierExactMatch} AS ident_exact,
|
||||
${descriptionPhraseMatch} AS desc_phrase,
|
||||
${titleTokenMatch} AS title_token,
|
||||
${identifierTokenMatch} AS ident_token,
|
||||
${descriptionTokenMatch} AS desc_token,
|
||||
${commentMatch} AS comment_match,
|
||||
${documentMatch} AS document_match,
|
||||
${fuzzyTokenTitleMatch} AS fuzzy_title,
|
||||
${fuzzyIdentifierMatch} AS fuzzy_ident,
|
||||
${coverageSql} AS token_coverage
|
||||
FROM issues
|
||||
WHERE issues.company_id = ${companyId}
|
||||
AND ${visibleIssueCondition()}
|
||||
${matchedWhere}
|
||||
)
|
||||
${taskSearchCtes(companyId, taskSearch, scope !== "issues", and(...issueFilters))}
|
||||
${sql.join(branches, sql` UNION ALL `)}
|
||||
`) as unknown as Array<SearchAggregateRow & Omit<IssueSearchRow, "commentSnippet" | "commentId" | "documentSnippet" | "documentTitle" | "documentKey">>;
|
||||
|
||||
|
|
@ -1014,11 +801,11 @@ export function companySearchService(db: Db) {
|
|||
AND search_comments.issue_id = target.id
|
||||
AND search_comments.deleted_at IS NULL
|
||||
AND (
|
||||
search_comments.body ILIKE ${containsPattern}
|
||||
OR search_comments.body ILIKE ANY(${tokenPatternArray})
|
||||
${taskSearchFieldMatch(sql`search_comments.body`, taskSearch)}
|
||||
)
|
||||
ORDER BY
|
||||
CASE WHEN search_comments.body ILIKE ${containsPattern} THEN 0 ELSE 1 END,
|
||||
${sql.join(tokens.map((_, index) => sql`CASE WHEN ${taskSearchTermMatch(sql`search_comments.body`, taskSearch, index)} THEN 1 ELSE 0 END`), sql` + `)} DESC,
|
||||
search_comments.updated_at DESC,
|
||||
search_comments.id DESC
|
||||
LIMIT 1
|
||||
|
|
@ -1032,10 +819,8 @@ export function companySearchService(db: Db) {
|
|||
WHERE search_issue_documents.company_id = ${companyId}
|
||||
AND search_issue_documents.issue_id = target.id
|
||||
AND (
|
||||
coalesce(search_documents.title, '') ILIKE ${containsPattern}
|
||||
OR search_documents.latest_body ILIKE ${containsPattern}
|
||||
OR coalesce(search_documents.title, '') ILIKE ANY(${tokenPatternArray})
|
||||
OR search_documents.latest_body ILIKE ANY(${tokenPatternArray})
|
||||
${taskSearchFieldMatch(sql`search_documents.title`, taskSearch)}
|
||||
OR ${taskSearchFieldMatch(sql`search_documents.latest_body`, taskSearch)}
|
||||
)
|
||||
ORDER BY
|
||||
CASE
|
||||
|
|
@ -1043,6 +828,7 @@ export function companySearchService(db: Db) {
|
|||
WHEN search_documents.latest_body ILIKE ${containsPattern} THEN 1
|
||||
ELSE 2
|
||||
END,
|
||||
${sql.join(tokens.map((_, index) => sql`CASE WHEN ${taskSearchTermMatch(sql`search_documents.title`, taskSearch, index)} OR ${taskSearchTermMatch(sql`search_documents.latest_body`, taskSearch, index)} THEN 1 ELSE 0 END`), sql` + `)} DESC,
|
||||
search_documents.updated_at DESC,
|
||||
search_documents.id DESC
|
||||
LIMIT 1
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import { documentService } from "./documents.js";
|
||||
import { parseTaskSearch, taskSearchCtes, taskSearchScore } from "./task-search.js";
|
||||
import { createdFromIssueCondition } from "./issue-creation-origin.js";
|
||||
import { executionProjectionsForRuns } from "./execution-projection.js";
|
||||
import type { ExecutionProjection } from "@paperclipai/shared";
|
||||
|
|
@ -7810,24 +7811,7 @@ export function issueService(db: Db) {
|
|||
filters?.includeLiveDescendantSummary === true;
|
||||
const rawSearch = filters?.q?.trim() ?? "";
|
||||
const hasSearch = rawSearch.length > 0;
|
||||
const escapedSearch = hasSearch ? escapeLikePattern(rawSearch) : "";
|
||||
const startsWithPattern = `${escapedSearch}%`;
|
||||
const containsPattern = `%${escapedSearch}%`;
|
||||
const titleStartsWithMatch = sql<boolean>`${issues.title} ILIKE ${startsWithPattern} ESCAPE '\\'`;
|
||||
const titleContainsMatch = sql<boolean>`${issues.title} ILIKE ${containsPattern} ESCAPE '\\'`;
|
||||
const identifierStartsWithMatch = sql<boolean>`${issues.identifier} ILIKE ${startsWithPattern} ESCAPE '\\'`;
|
||||
const identifierContainsMatch = sql<boolean>`${issues.identifier} ILIKE ${containsPattern} ESCAPE '\\'`;
|
||||
const descriptionContainsMatch = sql<boolean>`${issues.description} ILIKE ${containsPattern} ESCAPE '\\'`;
|
||||
const commentContainsMatch = sql<boolean>`
|
||||
EXISTS (
|
||||
SELECT 1
|
||||
FROM ${issueComments}
|
||||
WHERE ${issueComments.issueId} = ${issues.id}
|
||||
AND ${issueComments.companyId} = ${companyId}
|
||||
AND ${issueComments.deletedAt} IS NULL
|
||||
AND ${issueComments.body} ILIKE ${containsPattern} ESCAPE '\\'
|
||||
)
|
||||
`;
|
||||
const taskSearch = parseTaskSearch(rawSearch);
|
||||
if (filters?.createdFromIssueId) {
|
||||
conditions.push(createdFromIssueCondition(companyId, filters.createdFromIssueId));
|
||||
}
|
||||
|
|
@ -7935,16 +7919,6 @@ export function issueService(db: Db) {
|
|||
),
|
||||
);
|
||||
}
|
||||
if (hasSearch) {
|
||||
conditions.push(
|
||||
or(
|
||||
titleContainsMatch,
|
||||
identifierContainsMatch,
|
||||
descriptionContainsMatch,
|
||||
commentContainsMatch,
|
||||
)!,
|
||||
);
|
||||
}
|
||||
if (filters?.updatedSince) {
|
||||
const since = new Date(filters.updatedSince);
|
||||
if (Number.isFinite(since.getTime())) {
|
||||
|
|
@ -7959,20 +7933,15 @@ export function issueService(db: Db) {
|
|||
conditions.push(ne(issues.originKind, "routine_execution"));
|
||||
}
|
||||
const priorityOrder = sql`CASE ${issues.priority} WHEN 'critical' THEN 0 WHEN 'high' THEN 1 WHEN 'medium' THEN 2 WHEN 'low' THEN 3 ELSE 4 END`;
|
||||
const searchOrder = sql<number>`
|
||||
CASE
|
||||
WHEN ${titleStartsWithMatch} THEN 0
|
||||
WHEN ${titleContainsMatch} THEN 1
|
||||
WHEN ${identifierStartsWithMatch} THEN 2
|
||||
WHEN ${identifierContainsMatch} THEN 3
|
||||
WHEN ${commentContainsMatch} THEN 4
|
||||
WHEN ${descriptionContainsMatch} THEN 5
|
||||
ELSE 6
|
||||
END
|
||||
`;
|
||||
const baseQuery = db
|
||||
.select(issueListSelect)
|
||||
.from(issues)
|
||||
const searchOrder = sql<number>`-task_search.score`;
|
||||
const issueSource = db.select(issueListSelect).from(issues);
|
||||
const searchedSource = hasSearch
|
||||
? issueSource.innerJoin(sql`(
|
||||
${taskSearchCtes(companyId, taskSearch, true, and(...conditions))}
|
||||
SELECT m.id, ${taskSearchScore(taskSearch)} AS score FROM matched m
|
||||
) task_search`, sql`task_search.id = ${issues.id}`)
|
||||
: issueSource;
|
||||
const baseQuery = searchedSource
|
||||
.where(and(...conditions))
|
||||
.orderBy(
|
||||
...issueListOrderBy(companyId, {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,196 @@
|
|||
import { sql, type SQL } from "drizzle-orm";
|
||||
import { COMPANY_SEARCH_MAX_QUERY_LENGTH, COMPANY_SEARCH_MAX_TOKENS } from "@paperclipai/shared";
|
||||
import { visibleIssueCondition } from "./issue-visibility.js";
|
||||
|
||||
// Only grammatical filler is ignored, only in multi-term queries, and never
|
||||
// inside quotes. Keep negation and domain words (API, UI, PR, etc.) meaningful.
|
||||
const FILLER = new Set(["a", "an", "the", "and", "of", "to", "for", "in", "on", "with"]);
|
||||
export function escapeTaskSearchPattern(value: string) {
|
||||
return value.replace(/[\\%_]/g, "\\$&");
|
||||
}
|
||||
|
||||
export function parseTaskSearch(text: string) {
|
||||
const normalizedQuery = text.slice(0, COMPANY_SEARCH_MAX_QUERY_LENGTH).trim().replace(/\s+/g, " ").toLowerCase();
|
||||
const parsed = Array.from(normalizedQuery.matchAll(/"([^"]+)"|([^\s"]+)/g), (match) => ({
|
||||
text: match[1] ?? match[2]!, quoted: match[1] !== undefined,
|
||||
}));
|
||||
const meaningful = parsed.filter((term) => term.quoted || !FILLER.has(term.text));
|
||||
const uniqueTerms = new Map<string, { text: string; quoted: boolean }>();
|
||||
for (const term of meaningful.length > 0 ? meaningful : parsed) {
|
||||
uniqueTerms.set(term.text, { ...term, quoted: term.quoted || uniqueTerms.get(term.text)?.quoted === true });
|
||||
}
|
||||
const terms = [...uniqueTerms.values()].slice(0, COMPANY_SEARCH_MAX_TOKENS);
|
||||
const tokens = terms.map((term) => term.text);
|
||||
const phrase = tokens.join(" ");
|
||||
// A copied/typed task identifier is navigation, never a fuzzy number match.
|
||||
const identifier = /^([a-z][a-z0-9]*)[- ](\d+)$/i.exec(normalizedQuery) ?? /^([a-z]+)(\d+)$/i.exec(normalizedQuery);
|
||||
const identifierQuery = identifier ? `${identifier[1]}-${identifier[2]}` : normalizedQuery;
|
||||
const patterns = tokens.map((token) => `%${escapeTaskSearchPattern(token)}%`);
|
||||
const containsPattern = `%${escapeTaskSearchPattern(phrase)}%`;
|
||||
const startsWithPattern = `${escapeTaskSearchPattern(phrase)}%`;
|
||||
return { normalizedQuery, terms, tokens, phrase, identifierQuery, patterns, containsPattern, startsWithPattern };
|
||||
}
|
||||
export type TaskSearch = ReturnType<typeof parseTaskSearch>;
|
||||
|
||||
function taskSearchAny(field: SQL, search: TaskSearch): SQL<boolean> {
|
||||
return search.patterns.length === 0 ? sql`false`
|
||||
: sql`(${sql.join(search.patterns.map((pattern) => sql`${field} ILIKE ${pattern}`), sql` OR `)})`;
|
||||
}
|
||||
// Short typeahead terms must start a word: UI must not match "build", and
|
||||
// API must not match "Capistrano". Keep an indexable literal precondition.
|
||||
export function taskSearchTermMatch(field: SQL, search: TaskSearch, index: number): SQL<boolean> {
|
||||
const term = search.tokens[index]!;
|
||||
const literal = sql<boolean>`${field} ILIKE ${search.patterns[index]!}`;
|
||||
return /^[\p{L}]{1,3}$/u.test(term)
|
||||
? sql`(${literal} AND ${field} ~* ${`(^|[^[:alnum:]])${term}`})`
|
||||
: literal;
|
||||
}
|
||||
export function taskSearchFieldMatch(field: SQL, search: TaskSearch): SQL<boolean> {
|
||||
return search.tokens.length === 0 ? sql`false`
|
||||
: sql`(${sql.join(search.tokens.map((_, index) => taskSearchTermMatch(field, search, index)), sql` OR `)})`;
|
||||
}
|
||||
function coverage(matches: SQL[]): SQL<number> {
|
||||
return matches.length === 0 ? sql`0`
|
||||
: sql`(${sql.join(matches.map((match) => sql`CASE WHEN ${match} THEN 1 ELSE 0 END`), sql` + `)})`;
|
||||
}
|
||||
|
||||
// Score bands are deliberately disjoint. Incidental comments, repeated terms,
|
||||
// status and recency cannot outweigh a stronger kind of match.
|
||||
export function taskSearchScore(search: TaskSearch): SQL<number> {
|
||||
const n = search.tokens.length;
|
||||
if (n === 0) return sql`0`;
|
||||
return sql`(
|
||||
CASE
|
||||
WHEN m.ident_exact THEN 8000
|
||||
WHEN m.ident_starts THEN 7000
|
||||
WHEN m.title_exact THEN 6000
|
||||
WHEN m.title_phrase AND m.title_coverage = ${n} THEN 5000
|
||||
WHEN m.title_coverage = ${n} THEN 4000
|
||||
WHEN m.issue_coverage = ${n} THEN 3000
|
||||
WHEN m.token_coverage = ${n} THEN 2000
|
||||
WHEN m.fuzzy_title THEN 1000
|
||||
ELSE 0
|
||||
END
|
||||
+ m.title_word_coverage * 10
|
||||
+ CASE WHEN m.title_starts THEN 30 ELSE 0 END
|
||||
+ CASE m.status WHEN 'done' THEN 0 WHEN 'cancelled' THEN 0 ELSE 10 END
|
||||
)::double precision`;
|
||||
}
|
||||
|
||||
/** Shared task retrieval for company search and issue-list/command-palette search.
|
||||
* Uses existing pg_trgm indexes and current rows: no derived corpus or worker.
|
||||
* The tagged comment/document sets are evaluated once, not once per task.
|
||||
*/
|
||||
export function taskSearchCtes(companyId: string, search: TaskSearch, includeContext = true, fallbackFilters?: SQL): SQL {
|
||||
const n = search.tokens.length;
|
||||
const comments = n === 0 || !includeContext ? sql`SELECT NULL::uuid AS issue_id, 0 AS ord WHERE false`
|
||||
: sql.join(search.patterns.map((_, index) => sql`
|
||||
SELECT c.issue_id, ${index}::int AS ord FROM issue_comments c
|
||||
WHERE c.company_id = ${companyId} AND c.deleted_at IS NULL AND ${taskSearchTermMatch(sql`c.body`, search, index)}
|
||||
GROUP BY c.issue_id
|
||||
`), sql` UNION ALL `);
|
||||
const documents = n === 0 || !includeContext ? sql`SELECT NULL::uuid AS issue_id, 0 AS ord WHERE false`
|
||||
: sql.join(search.patterns.map((_, index) => sql`
|
||||
SELECT d.issue_id, ${index}::int AS ord FROM issue_documents d
|
||||
JOIN documents body ON body.id = d.document_id AND body.company_id = d.company_id
|
||||
WHERE d.company_id = ${companyId} AND (${taskSearchTermMatch(sql`body.title`, search, index)} OR ${taskSearchTermMatch(sql`body.latest_body`, search, index)})
|
||||
GROUP BY d.issue_id
|
||||
`), sql` UNION ALL `);
|
||||
const titleTerms = search.patterns.map((_, index) => taskSearchTermMatch(sql`issues.title`, search, index));
|
||||
const issueTerms = search.patterns.map((_, index) => sql`(
|
||||
${titleTerms[index]!} OR ${taskSearchTermMatch(sql`issues.identifier`, search, index)}
|
||||
OR ${taskSearchTermMatch(sql`issues.description`, search, index)}
|
||||
)`);
|
||||
const commentTerms = search.patterns.map((_, index) => sql`issues.id IN (SELECT issue_id FROM comment_matches WHERE ord = ${index})`);
|
||||
const documentTerms = search.patterns.map((_, index) => sql`issues.id IN (SELECT issue_id FROM document_matches WHERE ord = ${index})`);
|
||||
const allTerms = issueTerms.map((term, index) => sql`(${term} OR ${commentTerms[index]!} OR ${documentTerms[index]!})`);
|
||||
const phraseMatch = (field: SQL) => n > 0 ? sql`coalesce(${field} ILIKE ${search.containsPattern}, false)` : sql`false`;
|
||||
const identExact = n > 0 ? sql`lower(issues.identifier) = ${search.identifierQuery}` : sql`false`;
|
||||
const identStarts = n > 0 ? sql`issues.identifier ILIKE ${escapeTaskSearchPattern(search.identifierQuery) + "%"}` : sql`false`;
|
||||
const fuzzyAllowed = !search.terms.some((term) => term.quoted)
|
||||
&& search.terms.some((term) => /^[\p{L}]{4,255}$/u.test(term.text))
|
||||
&& !/^[a-z][a-z0-9]*[- ]?\d+$/i.test(search.normalizedQuery);
|
||||
const fuzzyTerms = search.terms.map((term, index) => {
|
||||
if (!/^[\p{L}]{4,255}$/u.test(term.text)) return titleTerms[index]!;
|
||||
// Bound both arguments before calling fuzzystrmatch (255-character limit).
|
||||
// Cheap length checks prune word pairs before bounded edit-distance work.
|
||||
const edits = sql`CASE WHEN least(char_length(word), ${Array.from(term.text).length}) >= 6 THEN 2
|
||||
WHEN least(char_length(word), ${Array.from(term.text).length}) >= 5 THEN 1 ELSE 0 END`;
|
||||
return sql`(${titleTerms[index]!} OR EXISTS (
|
||||
SELECT 1 FROM regexp_split_to_table(lower(issues.title), '[^[:alnum:]]+') AS word
|
||||
WHERE CASE WHEN char_length(word) BETWEEN 4 AND 255
|
||||
AND abs(char_length(word) - ${Array.from(term.text).length}) <= ${edits}
|
||||
THEN levenshtein_less_equal(${term.text}, word, ${edits}) <= ${edits}
|
||||
ELSE false END
|
||||
))`;
|
||||
});
|
||||
const fuzzy = fuzzyAllowed ? sql`CASE WHEN ${coverage(titleTerms)} = ${n} THEN false
|
||||
ELSE (${sql.join(fuzzyTerms, sql` AND `)}) END` : sql`false`;
|
||||
const wordTerms = search.tokens.map((token) => {
|
||||
const escaped = token.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
return sql`issues.title ~* ${`(^|[^[:alnum:]_])${escaped}($|[^[:alnum:]_])`}`;
|
||||
});
|
||||
// Carry flags, not potentially large bodies, through materialized stages.
|
||||
// The search page fetches descriptions only for its result window.
|
||||
const flags = (fuzzyMatch: SQL) => sql`
|
||||
SELECT issues.id, issues.identifier, issues.title,
|
||||
issues.status, issues.priority, issues.assignee_agent_id, issues.assignee_user_id,
|
||||
issues.project_id, issues.created_at, issues.updated_at,
|
||||
${identExact} AS ident_exact, ${identStarts} AS ident_starts,
|
||||
${phraseMatch(sql`issues.identifier`)} AS ident_phrase,
|
||||
${taskSearchFieldMatch(sql`issues.identifier`, search)} AS ident_token,
|
||||
${n > 0 ? sql`lower(issues.title) = ${search.phrase}` : sql`false`} AS title_exact,
|
||||
${n > 0 ? sql`issues.title ILIKE ${search.startsWithPattern}` : sql`false`} AS title_starts,
|
||||
${phraseMatch(sql`issues.title`)} AS title_phrase,
|
||||
${taskSearchFieldMatch(sql`issues.title`, search)} AS title_token,
|
||||
${phraseMatch(sql`issues.description`)} AS desc_phrase,
|
||||
${taskSearchFieldMatch(sql`issues.description`, search)} AS desc_token,
|
||||
${coverage(titleTerms)} AS title_coverage,
|
||||
${coverage(wordTerms)} AS title_word_coverage,
|
||||
${coverage(issueTerms)} AS issue_coverage,
|
||||
${coverage(commentTerms)} AS comment_coverage,
|
||||
${coverage(documentTerms)} AS document_coverage,
|
||||
${coverage(allTerms)} AS token_coverage,
|
||||
${fuzzyMatch} AS fuzzy_title,
|
||||
issues.id IN (SELECT issue_id FROM comment_matches) AS comment_match,
|
||||
issues.id IN (SELECT issue_id FROM document_matches) AS document_match
|
||||
FROM issues
|
||||
`;
|
||||
return sql`
|
||||
WITH comment_matches AS MATERIALIZED (${comments}),
|
||||
document_matches AS MATERIALIZED (${documents}),
|
||||
literal_candidates AS MATERIALIZED (
|
||||
SELECT issues.id FROM issues
|
||||
WHERE issues.company_id = ${companyId} AND ${visibleIssueCondition()}
|
||||
AND ${n === 0 ? sql`${search.normalizedQuery.length === 0}` : sql`(
|
||||
${taskSearchAny(sql`issues.title`, search)}
|
||||
OR ${taskSearchAny(sql`issues.identifier`, search)}
|
||||
OR ${taskSearchAny(sql`issues.description`, search)}
|
||||
OR ${identStarts}
|
||||
)`}
|
||||
UNION SELECT issue_id FROM comment_matches
|
||||
UNION SELECT issue_id FROM document_matches
|
||||
), search_flags AS MATERIALIZED (
|
||||
${flags(sql`false`)}
|
||||
WHERE issues.company_id = ${companyId} AND ${visibleIssueCondition()}
|
||||
AND issues.id IN (SELECT id FROM literal_candidates)
|
||||
), literal_matches AS MATERIALIZED (
|
||||
SELECT * FROM search_flags
|
||||
WHERE ${n === 0 ? sql`true` : sql`token_coverage = ${n} OR ident_exact OR ident_starts`}
|
||||
), fuzzy_candidates AS MATERIALIZED (
|
||||
SELECT issues.id FROM issues
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM literal_matches literal
|
||||
JOIN issues ON issues.id = literal.id
|
||||
${fallbackFilters ? sql`WHERE ${fallbackFilters}` : sql``}
|
||||
)
|
||||
AND issues.company_id = ${companyId} AND ${visibleIssueCondition()}
|
||||
AND ${fuzzy}
|
||||
), matched AS MATERIALIZED (
|
||||
SELECT * FROM literal_matches
|
||||
UNION ALL
|
||||
${flags(sql`true`)}
|
||||
WHERE issues.id IN (SELECT id FROM fuzzy_candidates)
|
||||
)
|
||||
`;
|
||||
}
|
||||
|
|
@ -263,6 +263,30 @@ describe("Search page", () => {
|
|||
});
|
||||
});
|
||||
|
||||
it.each(["relevance", "updated"])("preserves server %s order across result sources", async (sort) => {
|
||||
const results = ["document", "title", "comment"].map((field, index) => ({
|
||||
id: `rank-${index}`, type: "issue", score: 300 - index,
|
||||
title: `Rank ${index}`, href: `/PAP/issues/rank-${index}`,
|
||||
matchedFields: [field], sourceLabel: field, snippet: field,
|
||||
snippets: [{ field, label: field, text: field, highlights: [] }],
|
||||
updatedAt: "2026-01-01T00:00:00.000Z",
|
||||
issue: { id: `rank-${index}`, identifier: `RANK-${index}`, title: `Rank ${index}`,
|
||||
status: "todo", priority: "medium", assigneeAgentId: null, assigneeUserId: null,
|
||||
projectId: null, updatedAt: "2026-01-01T00:00:00.000Z" },
|
||||
}));
|
||||
searchApiMock.search.mockResolvedValue({ query: "rank", normalizedQuery: "rank", scope: "all",
|
||||
sort, limit: 20, offset: 0, hasMore: false, zeroResults: null, results,
|
||||
countsByType: { issue: 1, comment: 1, document: 1, artifact: 0, agent: 0, project: 0 },
|
||||
filterOptionCounts: { status: {}, priority: {}, assigneeAgentId: {}, assigneeUserId: {}, projectId: {}, labelId: {}, updatedWithin: {} },
|
||||
});
|
||||
const { root } = renderSearch(`/search?q=rank&sort=${sort}`, container);
|
||||
await waitForAssertion(() => {
|
||||
const links = Array.from(container.querySelectorAll('[data-testid="search-results"] a[data-result-type]'));
|
||||
expect(links.map((link) => link.getAttribute("href"))).toEqual(results.map((result) => result.href));
|
||||
});
|
||||
flushSync(() => root.unmount());
|
||||
});
|
||||
|
||||
it("renders artifact search results in the company search surface", async () => {
|
||||
searchApiMock.search.mockResolvedValueOnce({
|
||||
query: "launch brief",
|
||||
|
|
|
|||
|
|
@ -15,7 +15,6 @@ import { Input } from "@/components/ui/input";
|
|||
import { Button } from "@/components/ui/button";
|
||||
import { Skeleton } from "@/components/ui/skeleton";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useNavigate, useSearchParams } from "@/lib/router";
|
||||
import { useCompany } from "../context/CompanyContext";
|
||||
import { useBreadcrumbs } from "../context/BreadcrumbContext";
|
||||
|
|
@ -39,7 +38,6 @@ import {
|
|||
type ParsedSearchQuery,
|
||||
type SearchQueryParserContext,
|
||||
} from "../lib/search-query-parser";
|
||||
import { IssueGroupHeader } from "../components/IssueGroupHeader";
|
||||
import { SearchResultRow } from "../components/search/SearchResultRow";
|
||||
import { SearchFilterBar, type SearchFilterDataProps } from "../components/search/SearchFilterBar";
|
||||
import { SearchFilterChips } from "../components/search/SearchFilterChips";
|
||||
|
|
@ -69,44 +67,6 @@ const SCOPE_LABELS: Record<CompanySearchScope, string> = {
|
|||
projects: "Projects",
|
||||
};
|
||||
|
||||
type SubGroupKey = "issues" | "comments" | "documents" | "artifacts" | "agents" | "projects";
|
||||
|
||||
const SUBGROUP_ORDER: SubGroupKey[] = ["issues", "comments", "documents", "artifacts", "agents", "projects"];
|
||||
|
||||
const SUBGROUP_LABELS: Record<SubGroupKey, string> = {
|
||||
issues: "Tasks",
|
||||
comments: "Comments",
|
||||
documents: "Documents",
|
||||
artifacts: "Artifacts",
|
||||
agents: "Agents",
|
||||
projects: "Projects",
|
||||
};
|
||||
|
||||
function classifyResult(result: CompanySearchResult): SubGroupKey {
|
||||
if (result.type === "artifact") return "artifacts";
|
||||
if (result.type === "agent") return "agents";
|
||||
if (result.type === "project") return "projects";
|
||||
const matched = new Set(result.matchedFields);
|
||||
if (matched.has("title") || matched.has("identifier") || matched.has("description")) return "issues";
|
||||
if (matched.has("comment")) return "comments";
|
||||
if (matched.has("document")) return "documents";
|
||||
return "issues";
|
||||
}
|
||||
|
||||
function buildSubgroups(results: CompanySearchResult[]): Array<{ key: SubGroupKey; results: CompanySearchResult[] }> {
|
||||
const buckets = new Map<SubGroupKey, CompanySearchResult[]>();
|
||||
for (const result of results) {
|
||||
const key = classifyResult(result);
|
||||
const list = buckets.get(key) ?? [];
|
||||
list.push(result);
|
||||
buckets.set(key, list);
|
||||
}
|
||||
return SUBGROUP_ORDER.filter((key) => (buckets.get(key)?.length ?? 0) > 0).map((key) => ({
|
||||
key,
|
||||
results: buckets.get(key) ?? [],
|
||||
}));
|
||||
}
|
||||
|
||||
function isCompanySearchScope(value: string | null): value is CompanySearchScope {
|
||||
return Boolean(value) && (COMPANY_SEARCH_SCOPES as readonly string[]).includes(value as string);
|
||||
}
|
||||
|
|
@ -533,8 +493,6 @@ export function Search() {
|
|||
});
|
||||
}, [counts, data, filtersActive]);
|
||||
|
||||
const subgroups = useMemo(() => buildSubgroups(data?.results ?? []), [data?.results]);
|
||||
|
||||
const operatorPills = useMemo(() => searchFilterPills(draftFilters, parserContext), [draftFilters, parserContext]);
|
||||
const operatorSuggestions = useMemo(
|
||||
() => (inputFocused ? searchOperatorSuggestions(draftQuery, 4) : []),
|
||||
|
|
@ -721,7 +679,7 @@ export function Search() {
|
|||
refetch={() => void refetch()}
|
||||
recentSearches={recentSearches}
|
||||
onRecentClick={handleRecentClick}
|
||||
subgroups={subgroups}
|
||||
results={data?.results ?? []}
|
||||
totalResults={totalResults}
|
||||
allMatchTotal={allMatchTotal}
|
||||
activeFilterCount={activeFilterCount}
|
||||
|
|
@ -767,7 +725,7 @@ interface SearchTabContentProps {
|
|||
refetch: () => void;
|
||||
recentSearches: string[];
|
||||
onRecentClick: (query: string) => void;
|
||||
subgroups: Array<{ key: SubGroupKey; results: CompanySearchResult[] }>;
|
||||
results: CompanySearchResult[];
|
||||
totalResults: number;
|
||||
allMatchTotal: number;
|
||||
activeFilterCount: number;
|
||||
|
|
@ -792,7 +750,7 @@ function SearchTabContent({
|
|||
refetch,
|
||||
recentSearches,
|
||||
onRecentClick,
|
||||
subgroups,
|
||||
results,
|
||||
totalResults,
|
||||
allMatchTotal,
|
||||
activeFilterCount,
|
||||
|
|
@ -950,47 +908,14 @@ function SearchTabContent({
|
|||
</span>
|
||||
{isFetching ? <span aria-live="polite" className="normal-case tracking-normal">Updating…</span> : null}
|
||||
</div>
|
||||
<div className="flex flex-col pb-10">
|
||||
{scope === "all" ? (
|
||||
subgroups.map((group, groupIndex) => (
|
||||
<section
|
||||
key={group.key}
|
||||
aria-label={SUBGROUP_LABELS[group.key]}
|
||||
className={cn("flex flex-col", groupIndex > 0 && "mt-6")}
|
||||
>
|
||||
<IssueGroupHeader
|
||||
label={SUBGROUP_LABELS[group.key]}
|
||||
trailing={
|
||||
<span className="text-xs font-normal tabular-nums text-muted-foreground">
|
||||
{group.results.length}
|
||||
</span>
|
||||
}
|
||||
className="pt-2 pb-1 text-(length:--text-micro) tracking-wider text-muted-foreground"
|
||||
/>
|
||||
<div className="flex flex-col gap-y-1">
|
||||
{group.results.map((result) => (
|
||||
<SearchResultRow
|
||||
key={`${result.type}:${result.id}:${result.href}`}
|
||||
result={result}
|
||||
agentsById={agentsById}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
</section>
|
||||
))
|
||||
) : (
|
||||
<div className="flex flex-col gap-y-1">
|
||||
{subgroups
|
||||
.flatMap((group) => group.results)
|
||||
.map((result) => (
|
||||
<SearchResultRow
|
||||
key={`${result.type}:${result.id}:${result.href}`}
|
||||
result={result}
|
||||
agentsById={agentsById}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
<div className="flex flex-col gap-y-1 pb-10">
|
||||
{results.map((result) => (
|
||||
<SearchResultRow
|
||||
key={`${result.type}:${result.id}:${result.href}`}
|
||||
result={result}
|
||||
agentsById={agentsById}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@ import type {
|
|||
CompanySearchZeroResults,
|
||||
} from "@paperclipai/shared";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import { IssueGroupHeader } from "@/components/IssueGroupHeader";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { PageTabBar, type PageTabItem } from "@/components/PageTabBar";
|
||||
import { MatchSourceChip } from "@/components/search/MatchSourceChip";
|
||||
|
|
@ -296,58 +295,11 @@ function SearchPagePreview({
|
|||
<div className="flex items-center justify-between py-2 text-[11px] uppercase tracking-wide text-muted-foreground">
|
||||
<span>{response.results.length} results · sorted by relevance</span>
|
||||
</div>
|
||||
<section aria-label="Issues" className="flex flex-col">
|
||||
<IssueGroupHeader
|
||||
label="Issues"
|
||||
trailing={
|
||||
<span className="text-xs font-normal tabular-nums text-muted-foreground">
|
||||
{fixtureResults.length}
|
||||
</span>
|
||||
}
|
||||
className="pt-2 pb-1 text-[11px] tracking-wider text-muted-foreground"
|
||||
/>
|
||||
<div className="flex flex-col gap-y-1">
|
||||
{fixtureResults.map((result) => (
|
||||
<SearchResultRow
|
||||
key={result.id}
|
||||
result={result}
|
||||
agentsById={agentsById}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
</section>
|
||||
<section aria-label="Agents" className="mt-6 flex flex-col">
|
||||
<IssueGroupHeader
|
||||
label="Agents"
|
||||
trailing={
|
||||
<span className="text-xs font-normal tabular-nums text-muted-foreground">
|
||||
{fixtureAgents.length}
|
||||
</span>
|
||||
}
|
||||
className="pt-2 pb-1 text-[11px] tracking-wider text-muted-foreground"
|
||||
/>
|
||||
<div className="flex flex-col gap-y-1">
|
||||
{fixtureAgents.map((result) => (
|
||||
<SearchResultRow key={result.id} result={result} />
|
||||
))}
|
||||
</div>
|
||||
</section>
|
||||
<section aria-label="Projects" className="mt-6 flex flex-col">
|
||||
<IssueGroupHeader
|
||||
label="Projects"
|
||||
trailing={
|
||||
<span className="text-xs font-normal tabular-nums text-muted-foreground">
|
||||
{fixtureProjects.length}
|
||||
</span>
|
||||
}
|
||||
className="pt-2 pb-1 text-[11px] tracking-wider text-muted-foreground"
|
||||
/>
|
||||
<div className="flex flex-col gap-y-1">
|
||||
{fixtureProjects.map((result) => (
|
||||
<SearchResultRow key={result.id} result={result} />
|
||||
))}
|
||||
</div>
|
||||
</section>
|
||||
<div className="flex flex-col gap-y-1">
|
||||
{response.results.map((result) => (
|
||||
<SearchResultRow key={result.id} result={result} agentsById={agentsById} />
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
|
|
@ -770,7 +722,7 @@ const meta = {
|
|||
docs: {
|
||||
description: {
|
||||
component:
|
||||
"Full search page surfaces and Command K Search-all handoff. Reuses StatusIcon, StatusBadge, Identity, IssueGroupHeader, and PageTabBar; adds MatchSourceChip + SearchResultRow.",
|
||||
"Full search page surfaces and Command K Search-all handoff. Reuses StatusIcon, StatusBadge, Identity and PageTabBar; adds MatchSourceChip + SearchResultRow.",
|
||||
},
|
||||
},
|
||||
},
|
||||
|
|
|
|||
Loading…
Reference in New Issue