The confidence bar that decides whether the AI's suggested category is
auto-applied to a transaction was a fixed config value
(ai_categorization.label_confidence, 0.7). Let each user tune it with a
slider in Settings > Manage Plan, next to the AI categorization toggle.
- Add nullable user_settings.ai_confidence_threshold (percent, 0-100);
null falls back to the global config default.
- User::aiConfidenceThresholdPercent() centralises the "user value else
config default" resolution used by both the billing page and the
categorization engine.
- CategorizeTransactions reads the per-user bar instead of the config.
- Gate tier-2 rule learning on the outcome being applied, so the
effective rule bar becomes max(label bar, rule_confidence): raising the
threshold to reduce automation now also holds back forward rules
(previously a suggestion above 0.85 could still spawn a rule even when
the user's raised bar left the transaction uncategorized).
- Slider (50-95%, debounced save) rendered when AI consent is on.
## Why
Two issues surfaced from a real mis-categorization corrected via the
**Edit Transaction** modal (not the table quick-edit).
### 1. The modal never told the user the AI learned
Both the table quick-edit and the edit modal hit the same backend
(`PATCH /transactions/{id}` → `CategoryOverrideHandler` →
`AiRuleLearner`), so both learn a forward rule from a correction. But
the modal **discarded** the update response and never read
`learned_rule`:
- No *"Learned: similar transactions will be categorized automatically"*
confirmation.
- No **Undo**.
- It still showed the generic *"Automatize"* prompt — offering to create
a rule that already exists.
The modal now reads `learned_rule` and mirrors the table's toast + undo,
skipping the automatize prompt when a rule was learned. Same i18n
strings as the table (no new keys).
### 2. The learner could key a rule on a single weak token
A correction on `"Pago en MADRID SUC 04 MADRID ES"` produced a rule
keyed on `suc` alone. The tokenizer drops noise by **per-user** document
frequency, so `madrid` (20% of this user's descriptions) is correctly
dropped — but `suc` (0.14%) survives, even though it is a generic
banking abbreviation that matches broadly as a substring. The per-user
filter can't see that `suc` is generic in general.
`AiRuleLearner::descriptionClause()` now refuses a single distinctive
token shorter than 5 characters. It learns only when there are **two or
more** distinctive tokens, or a single token of **at least 5
characters**. Merchant-keyed rules are unaffected.
## Tests
- `does not learn a description rule from a single short token`
- `learns a description rule from a single sufficiently long token`
- `learns a description rule from two short tokens`
All `AiRuleLearnerTest` pass (13/13).
## Not in this PR
The one already-learned prod rule still carries the stale `suc`
clause/title (low impact, matches ~2 txns). Cleaning that data is a
separate manual step.
> **Draft on purpose — a partial fix + a design for the full one.**
These commits are safe, behavior-preserving reductions of the N+1, but
they do **not** fully resolve PHP-LARAVEL-40. The complete fix is a
batch-aware refactor of the AI rule-learning path, which is delicate
(getting it wrong silently mis-categorizes users' transactions). I've
written the design below for a human to implement and diff against the
current per-transaction behavior. Current user impact is low (~170 ms
request, 1 event), so there's no urgency to rush the risky part.
## What
Sentry **PHP-LARAVEL-40** — N+1 in `TransactionController@bulkUpdate`
(PATCH `/transactions/bulk`). A bulk category update calls
`CategoryOverrideHandler::record()` once per transaction, and each call
runs the full AI rule-learning pipeline
(`AiRuleLearner::forgetFromAiRules()` + `learnFromCorrection()`):
loading all the user's AI rules, plucking every description, running
matcher `count(*)` probes, and inserting a `category_corrections` row.
For a 112-transaction batch that's ~112× each. The offending span is the
per-transaction `category_corrections` insert.
## What this branch does (safe, partial)
Two behavior-preserving reductions, each validated by the existing
learning tests:
1. `perf(transactions): resolve the override handler once for bulk
updates` — `bulkUpdate()` re-resolved `CategoryOverrideHandler` from the
container every iteration. Resolve it once (also required for #2 to take
effect).
2. `perf(ai): memoize the description corpus per user in AiRuleLearner`
— `learnFromCorrection` reloaded and re-tokenized every one of the
user's descriptions on each transaction. The corpus is immutable while
only categories change, so memoize the document-frequency map + count
per user. Removes one `SELECT` (and its tokenization) per transaction.
3. `test(ai): assert batch corrections learn correctly through the
memoized corpus` — proves the second, memoized-corpus learning still
yields a correct, distinct clause (not just that the query count
dropped).
4. `test(ai): guard AiRuleLearner against a singleton binding` — the
cache has no invalidation and is only safe while the learner is resolved
fresh per request; a test now fails if it is ever bound singleton.
## What it does NOT do
It does **not** resolve the flagged N+1. Still per-transaction: the
`category_corrections` insert, `forgetFromAiRules`' reload of all AI
rules, the matcher `total()`/`countMatchingAll()` overbroad probes,
`releaseClauseFromOtherCorrectionRules`, and `existingCorrectionRule`. A
learnable transaction still costs ~6–10 queries. Treat PHP-LARAVEL-40 as
**reduced, not closed** (hence no `Fixes` keyword).
## Reviewed by two independent agents (architecture +
product/correctness)
Both verified the two perf commits are **behavior-preserving and safe**:
the memo cannot go stale (nothing in the correction path mutates
descriptions; the bulk `update()` only writes
`category_id`/`category_source`/`ai_confidence`/`categorized_by_rule_id`,
and runs after the loop), no cross-user leak (keyed by user_id; learner
is transient, one user per request, no Octane), and the handler is
stateless. The existing 34 learning tests exercise the corpus→rule
computation on fresh loads.
## Proposed full fix (for a human to implement)
Exploit that a bulk update sends **all transactions to the same category
for one user.** Add `CategoryOverrideHandler::recordBulk(Collection,
?string)` backed by `AiRuleLearner::learnFromCorrections(array,
string)`; load user-level data once, mutate in memory, write once:
1. **Batch-resolve** `categorized_by_rule_id` via one `whereIn` instead
of per-txn `find()`; apply the **identical** per-txn learnable test
against the map.
2. **Batch-insert** corrections: snapshot each AI-driven txn's *old*
`from_category_id`/`source`/`confidence` during the loop, collect rows,
one insert. (Note: raw `insert()` skips model events/UUID/timestamps —
verify `CategoryCorrection` has no `creating` hook, else chunked
`create` in one transaction.)
3. **Forget once**: union all learnable txns' merchant tokens, load AI
rules once, apply the **whole union** to each rule *before* the
delete-vs-save decision, save/delete each once.
4. **Learn once**: compute each clause (merchant or memoized-corpus
tokens), dedupe within the batch and against the target rule, run
`releaseClauseFromOtherCorrectionRules` for the union once, load/create
the single target correction rule once, append all, save once.
5. Wrap 2–4 in one `DB::transaction`, and route the single-txn path
through the same method (one-element collection) so the two can't drift.
**Correctness invariants to preserve (call these out in review):**
- Apply-all-then-decide for both `forget` and `releaseClause` deletes —
an incremental forget/save/forget would delete a rule a later txn's
clause should have kept.
- Corrections must read each txn's **old** category/source/confidence —
run before the bulk `update()`, as today.
- Clause dedup must match `appendClause`'s loose `==` array comparison.
- Validate with a golden-set test diffing learned/forgotten rules
before/after against the current per-txn path over a mixed batch
(merchant txns, description txns, encrypted/no-merchant skips,
re-corrections that move a key between categories).
## Testing
- `tests/Feature/Ai/` + `BulkUpdateTransactionsTest` + IDOR/decrypt:
153/153 green. `pint` clean.
Refs PHP-LARAVEL-40
## What
Auto-categorizes transactions with AI (Gemini) for **pro +
AI-consented** users when no automation rule already matched. Ships the
full backend **behind a Pennant flag, off by default**, so it's
mergeable and testable in isolation; the UI is a deliberate follow-up.
## Why / cost
Prod check first: ~20k txns/month, **~52% of pro-user transactions are
uncategorized** after rules. At Gemini Flash-Lite rates the cost is a
**rounding error** — ~$0.13–$0.75/month for all pro users, single-digit
dollars even on full Flash. So the model is chosen for accuracy, not
price; the real constraints are trust, accuracy and privacy.
## How it works
**Two tiers** (every transaction is covered, rules are an optimization
on top):
- **Tier 1 – label** — a queued listener runs *after* the synchronous
rules; if still uncategorized and the user is eligible, the model picks
a **leaf** category (referenced by numeric index, never a UUID, so it
can't hallucinate one). Auto-applied only above the **label bar**
(`0.7`); below → left blank, no nag. Tagged `category_source = ai` +
`ai_confidence`, fully reversible.
- **Tier 2 – learn** — above the higher **rule bar** (`0.85`) *and* a
clean merchant key *and* the model flags the merchant unambiguous → the
merchant is appended to a single **ai-owned** automation rule for that
category (OR'd conditions, not rule-sprawl), so future transactions
match for free and consistently. AI rules sit at the lowest priority;
**user-owned rules are never touched**.
**Self-heal + signal** — when a user overrides an AI category, a
`category_correction` is logged (calibration signal, bucketable by
confidence) and the offending merchant condition is dropped from the ai
rule (deleted if empty). User rules and manual categories are untouched.
**Safety** — config kill switch + pro + active consent + gradual Pennant
rollout. Dedicated `ai` queue so Gemini never blocks bank syncs.
Encrypted (client-side) transactions are never sent.
**Backfill** — `ai:categorize-backfill {user}`, explicit opt-in,
batched, learns rules as it goes.
## Data model
- `transactions`: `category_source`, `ai_confidence`,
`categorized_by_rule_id`
- `automation_rules`: `origin` (`user`/`ai`)
- new `category_corrections` table
## Screenshots
<img width="921" height="384" alt="image"
src="https://github.com/user-attachments/assets/f04c2a03-b39e-4a3d-81eb-ecf26eaefb83"
/>
Víctor Falcón