whisper-money/tests/Browser
Víctor Falcón 6727a9c64a
feat(ai): learn from category corrections so the AI stops repeating the same mistake (#608)
## Problem

Users keep correcting the same transactions over and over. The AI
mislabels a merchant (e.g. supermarket → fuel), the user fixes it, and
the next near-identical transaction from that merchant gets mislabeled
the same way again. Today a correction is logged and the offending ai
rule is self-healed, but the system only *forgets* its mistake — it
never *remembers* the user's fix.

## Approach

A correction now becomes a deterministic, forward-looking
`AutomationRule` (new `RuleOrigin::Correction`). The next matching
transaction is categorized by that rule **before the model ever runs**
(`ApplyAutomationRules` is synchronous and runs ahead of AI
categorization), ending the loop. Zero model cost, instant, reuses the
existing rule engine.

**Matching key** (in order):
1. **Merchant** (`creditor_name`/`debtor_name`, exact `==`) when present
— stable even as the description varies.
2. Otherwise the **description's distinctive tokens** (`in` /
AND-of-`in`), extracted by the shared `DescriptionTokenizer` (noise
tokens dropped by document frequency, language-agnostic), **guarded**
against over-broad rules that could silently mis-file en masse. If
guarded out → nothing is learned, silently.

## Deliberate decisions (from a design walkthrough)

- **Forward-only**: never retroactively re-categorizes existing
transactions.
- **Learn only from system categorizations** (AI label, ai rule, or a
prior correction rule) — never from one-off manual filing, bank
categories, or the user's own hand-authored rules.
- **A key lives in exactly one correction rule**, so changing your mind
moves it to the new category. Correcting a transaction a prior
correction rule categorized is also learnable, so correction rules stay
fixable in-flow.
- Correcting to *uncategorized* learns nothing but still self-heals the
ai rule.
- **Safety net**: correction rules are visible/editable in
`settings/automation-rules` (marked with the AI sparkle, tooltip
"Learned from your correction"); the transactions table shows a toast
with an instant **Undo**.

## Review pass (two independent agents + live QA)

- **HIGH fix** (`67fc4293`): an ai rule could out-rank a freshly learned
correction and re-apply the wrong category (when the corrected
transaction was a *direct* model label with no rule id). Now every ai
rule holding the merchant is swept on correction. Regression test added.
- **Refactor** (`ca743fde`): collapsed duplicated clause-append logic;
removed a speculative unused enum helper.
- **Coverage** (`12ceb0f7`): debtor_name path, single-token description
clause, encrypted-description fail-safe.
- **Toast conflict fix** (`382c8169`, found in live QA): correcting an
AI transaction fired both the new "Learned …" toast and the pre-existing
"Transaction categorized → Automatize" prompt, which invited the user to
manually create the rule the correction had just created. Made them
mutually exclusive. Adds a browser test for the inline-correction flow.
- **Settings icon** (`f3f882b6`): correction rules now show the AI
sparkle in settings, like ai rules.

## Verified end-to-end (against a running instance)

Drove the real UI with a browser: correcting an AI-mislabeled
transaction creates the correction rule, shows the "Learned · Undo"
toast (no competing Automatize prompt), and Undo deletes the rule while
keeping the correction. Confirmed for **merchant** keys and the
**description-only** path — including that a later "practically
identical" description (different surrounding text, no merchant) is
caught by the rule, while a near-miss sharing only one distinctive token
is correctly **not** caught.

## Open question for reviewers

**`debtor_name` (P2P) as a rule key.** For incoming transfers the
merchant key falls back to the sender's name, so correcting one can
create a rule keyed on a person's name (useful for recurring transfers
from a roommate, but a possible privacy surprise; the name appears in
the rule title in settings). This matches the existing tier-2 learner's
behaviour. Keep as-is, or restrict correction rules to `creditor_name`
only? Happy to change.

## Testing

- `tests/Feature/Ai/CategoryOverrideHandlerTest.php`: merchant +
description learning, next-transaction match, over-broad rejection,
change-of-mind move, correct-to-null self-heal, the HIGH regression,
debtor_name, single-token, encrypted fail-safe.
- `tests/Browser/CategoryCorrectionLearningTest.php`: inline correction
→ toast → learned rule → undo.
- `automation-rule-title.test.tsx`: the AI sparkle shows for `ai` and
`correction`, not `user`.
- Full AI suite green (106 tests); transaction/bulk-update suites green
(62). Pint + Prettier + ESLint clean.

No new dependency, no migration (the `origin` column is a free-text
string). The feature is implicitly gated by AI categorization — with no
AI categorization there is nothing to correct and nothing is learned.
2026-06-29 19:12:15 +02:00
..
assets Improve automation rules and sync error handling 2025-12-01 10:30:22 +01:00
live feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
.gitignore Add category type field support (#2) 2025-12-01 20:19:47 +01:00
AccountsPageTest.php test: expand browser coverage for accounts and linked loans (#277) 2026-04-13 09:33:18 +01:00
AmountInputTest.php fix(transactions): only auto-select account on account pages (#549) 2026-06-17 11:28:31 +00:00
AuthenticationTest.php Support soft-deleted users with reusable emails (#316) 2026-04-22 11:41:41 +01:00
AutomationRuleBuilderTest.php feat: parent/child category tree (#474) 2026-06-03 19:30:12 +02:00
BankAccountsTest.php feat(loans): backfill historical balances on loan creation (#322) 2026-04-24 13:09:34 +01:00
BankConnectionFlowTest.php test(open-banking): e2e coverage for Enable Banking connection flows (#509) 2026-06-09 11:58:50 +02:00
BudgetCrudTest.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
BudgetsFeatureNavigationTest.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
CashflowCategoryNavigationTest.php feat(cashflow): make income/expense category rows clickable to transactions (#234) 2026-03-17 10:57:40 +00:00
CatchAllBudgetCrudTest.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
CategoriesTest.php fix(categories): expose cashflow setting on create (#448) 2026-05-29 11:05:04 +02:00
CategoryCorrectionLearningTest.php feat(ai): learn from category corrections so the AI stops repeating the same mistake (#608) 2026-06-29 19:12:15 +02:00
ImportTransactionsTest.php Remove encryption from browser tests and demo user (#129) 2026-02-17 11:45:27 +01:00
OnboardingFlowTest.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
RealEstateAccountTest.php refactor(real-estate): remove Pennant gating (#308) 2026-04-20 13:31:49 +01:00
SubscriptionRefundTest.php feat(subscriptions): trial/pricing A/B/C experiment (#600) 2026-06-27 18:00:15 +02:00
TransactionsTest.php Fix label creation dropdown refresh (#363) 2026-05-07 10:26:43 +01:00