## The bug
Sentry
[PHP-LARAVEL-53](https://whisper-money.sentry.io/issues/PHP-LARAVEL-53)
— `SQLSTATE[22001]: Data too long for column 'title'` on `PATCH
/transactions/{transaction}`. 6 events, 1 user.
Changing a transaction's category learns a forward-looking automation
rule, and `AiRuleLearner::title()` rebuilds the rule's title from the
merchant names it matches. This user's bank puts the **entire statement
line** in `creditor_name`:
```
15/06 12/06 Pago Con Tarjeta En Discos, Libros, Fotos Y Pc's -59,00 189,27 | 4940197152806468 Classpass* Monthly
```
Two corrections into the same category produced a 279-char title against
a `varchar(255)` column, so `$rule->save()` threw and took the whole
request with it. Since `creditor_name` is itself capped at 255 on
ingest, a single correction can overflow it too.
## The fix
1. **`AiRuleLearner::title()`** caps each merchant token (40 chars) and
the category name (60), so the title stays readable instead of merely
short — the `→ Category` tail is the part that carries the meaning.
2. **A `title` mutator on `AutomationRule`** truncates to the column
width. Both reviews flagged that fixing the learner alone leaves the
sibling generator, `ApplyRuleSuggestions::title()`, unguarded: it joins
three AI match tokens that are each validated at `max:255`, so it can
overflow the same column and 500 the accept-suggestions step **during
onboarding**. The mutator covers every generator, present and future, in
one place. User-authored titles never reach it —
`StoreAutomationRuleRequest` already rejects longer ones with a 422.
3. **Learning can no longer abort the correction.**
`CategoryOverrideHandler::record()` runs before the transaction is saved
and outside any DB transaction, so the incident left the correction
logged and the ai rules stripped while the category the user asked for
was never written. `bulkUpdate` is worse: it records every transaction
before the mass update, so one throw mutated rules for the earlier ones
and categorized none of them. A learning failure is now reported and
swallowed — both callers already handle "nothing learned".
## Tests
- `AiRuleLearnerTest` — replays the exact production payload; reproduces
the identical `SQLSTATE[22001]` on the pre-fix code and asserts the
title stays inside the column with the category still visible.
- `AutomationRuleTest` — the mutator truncates rather than failing the
write.
- `CategoryOverrideHandlerTest` — a throwing learner is reported, not
propagated.
191 tests green locally (`tests/Feature/Ai` + the three automation-rule
suites) against the MySQL testcontainer; `pint --test` clean.
## Deliberately out of scope
- **Dead-weight rules.** With such a merchant name the learned clause is
`creditor_name == "…-59,00 189,27 | …"` — it contains amounts and a
running balance, so it can never match again, yet it accumulates one
clause per correction and the UI toasts *"Learned: similar transactions
will be categorized automatically"*. The review's prod data says length
is the wrong guard (62 long merchant names DO repeat, across 490
transactions); the right one is rejecting the statement-line *signature*
(an embedded `dd/mm` or `nn,nn`) in `merchantKey()` so it falls through
to the already-guarded description-token path. Worth its own PR.
- **Rules list UI**: a long title sits in a `whitespace-nowrap` cell and
pushes the actions column off-screen (`settings/automation-rules.tsx`).
- **Unbounded writes of the same class elsewhere**, found while
reviewing: `transactions.currency_code` `varchar(3)` written unvalidated
from the Enable Banking payload (`TransactionSyncService.php:179`), and
`banking_connections.aspsp_name`/`aspsp_logo` with no `max:` rule in
`StartAuthorizationRequest`.
Fixes PHP-LARAVEL-53
## Problem
Users keep correcting the same transactions over and over. The AI
mislabels a merchant (e.g. supermarket → fuel), the user fixes it, and
the next near-identical transaction from that merchant gets mislabeled
the same way again. Today a correction is logged and the offending ai
rule is self-healed, but the system only *forgets* its mistake — it
never *remembers* the user's fix.
## Approach
A correction now becomes a deterministic, forward-looking
`AutomationRule` (new `RuleOrigin::Correction`). The next matching
transaction is categorized by that rule **before the model ever runs**
(`ApplyAutomationRules` is synchronous and runs ahead of AI
categorization), ending the loop. Zero model cost, instant, reuses the
existing rule engine.
**Matching key** (in order):
1. **Merchant** (`creditor_name`/`debtor_name`, exact `==`) when present
— stable even as the description varies.
2. Otherwise the **description's distinctive tokens** (`in` /
AND-of-`in`), extracted by the shared `DescriptionTokenizer` (noise
tokens dropped by document frequency, language-agnostic), **guarded**
against over-broad rules that could silently mis-file en masse. If
guarded out → nothing is learned, silently.
## Deliberate decisions (from a design walkthrough)
- **Forward-only**: never retroactively re-categorizes existing
transactions.
- **Learn only from system categorizations** (AI label, ai rule, or a
prior correction rule) — never from one-off manual filing, bank
categories, or the user's own hand-authored rules.
- **A key lives in exactly one correction rule**, so changing your mind
moves it to the new category. Correcting a transaction a prior
correction rule categorized is also learnable, so correction rules stay
fixable in-flow.
- Correcting to *uncategorized* learns nothing but still self-heals the
ai rule.
- **Safety net**: correction rules are visible/editable in
`settings/automation-rules` (marked with the AI sparkle, tooltip
"Learned from your correction"); the transactions table shows a toast
with an instant **Undo**.
## Review pass (two independent agents + live QA)
- **HIGH fix** (`67fc4293`): an ai rule could out-rank a freshly learned
correction and re-apply the wrong category (when the corrected
transaction was a *direct* model label with no rule id). Now every ai
rule holding the merchant is swept on correction. Regression test added.
- **Refactor** (`ca743fde`): collapsed duplicated clause-append logic;
removed a speculative unused enum helper.
- **Coverage** (`12ceb0f7`): debtor_name path, single-token description
clause, encrypted-description fail-safe.
- **Toast conflict fix** (`382c8169`, found in live QA): correcting an
AI transaction fired both the new "Learned …" toast and the pre-existing
"Transaction categorized → Automatize" prompt, which invited the user to
manually create the rule the correction had just created. Made them
mutually exclusive. Adds a browser test for the inline-correction flow.
- **Settings icon** (`f3f882b6`): correction rules now show the AI
sparkle in settings, like ai rules.
## Verified end-to-end (against a running instance)
Drove the real UI with a browser: correcting an AI-mislabeled
transaction creates the correction rule, shows the "Learned · Undo"
toast (no competing Automatize prompt), and Undo deletes the rule while
keeping the correction. Confirmed for **merchant** keys and the
**description-only** path — including that a later "practically
identical" description (different surrounding text, no merchant) is
caught by the rule, while a near-miss sharing only one distinctive token
is correctly **not** caught.
## Open question for reviewers
**`debtor_name` (P2P) as a rule key.** For incoming transfers the
merchant key falls back to the sender's name, so correcting one can
create a rule keyed on a person's name (useful for recurring transfers
from a roommate, but a possible privacy surprise; the name appears in
the rule title in settings). This matches the existing tier-2 learner's
behaviour. Keep as-is, or restrict correction rules to `creditor_name`
only? Happy to change.
## Testing
- `tests/Feature/Ai/CategoryOverrideHandlerTest.php`: merchant +
description learning, next-transaction match, over-broad rejection,
change-of-mind move, correct-to-null self-heal, the HIGH regression,
debtor_name, single-token, encrypted fail-safe.
- `tests/Browser/CategoryCorrectionLearningTest.php`: inline correction
→ toast → learned rule → undo.
- `automation-rule-title.test.tsx`: the AI sparkle shows for `ai` and
`correction`, not `user`.
- Full AI suite green (106 tests); transaction/bulk-update suites green
(62). Pint + Prettier + ESLint clean.
No new dependency, no migration (the `origin` column is a free-text
string). The feature is implicitly gated by AI categorization — with no
AI categorization there is nothing to correct and nothing is learned.
## What
Auto-categorizes transactions with AI (Gemini) for **pro +
AI-consented** users when no automation rule already matched. Ships the
full backend **behind a Pennant flag, off by default**, so it's
mergeable and testable in isolation; the UI is a deliberate follow-up.
## Why / cost
Prod check first: ~20k txns/month, **~52% of pro-user transactions are
uncategorized** after rules. At Gemini Flash-Lite rates the cost is a
**rounding error** — ~$0.13–$0.75/month for all pro users, single-digit
dollars even on full Flash. So the model is chosen for accuracy, not
price; the real constraints are trust, accuracy and privacy.
## How it works
**Two tiers** (every transaction is covered, rules are an optimization
on top):
- **Tier 1 – label** — a queued listener runs *after* the synchronous
rules; if still uncategorized and the user is eligible, the model picks
a **leaf** category (referenced by numeric index, never a UUID, so it
can't hallucinate one). Auto-applied only above the **label bar**
(`0.7`); below → left blank, no nag. Tagged `category_source = ai` +
`ai_confidence`, fully reversible.
- **Tier 2 – learn** — above the higher **rule bar** (`0.85`) *and* a
clean merchant key *and* the model flags the merchant unambiguous → the
merchant is appended to a single **ai-owned** automation rule for that
category (OR'd conditions, not rule-sprawl), so future transactions
match for free and consistently. AI rules sit at the lowest priority;
**user-owned rules are never touched**.
**Self-heal + signal** — when a user overrides an AI category, a
`category_correction` is logged (calibration signal, bucketable by
confidence) and the offending merchant condition is dropped from the ai
rule (deleted if empty). User rules and manual categories are untouched.
**Safety** — config kill switch + pro + active consent + gradual Pennant
rollout. Dedicated `ai` queue so Gemini never blocks bank syncs.
Encrypted (client-side) transactions are never sent.
**Backfill** — `ai:categorize-backfill {user}`, explicit opt-in,
batched, learns rules as it goes.
## Data model
- `transactions`: `category_source`, `ai_confidence`,
`categorized_by_rule_id`
- `automation_rules`: `origin` (`user`/`ai`)
- new `category_corrections` table
## Screenshots
<img width="921" height="384" alt="image"
src="https://github.com/user-attachments/assets/f04c2a03-b39e-4a3d-81eb-ecf26eaefb83"
/>
Víctor Falcón