## Contexto
El categorizador solo persistía un resultado cuando la confianza
superaba la barra (`ai_categorization.label_confidence`, 0.7). Las
sugerencias por debajo del umbral se calculaban y se descartaban,
dejándonos sin ninguna visibilidad de qué proponía el modelo para las
transacciones que no se autocategorizaban.
## Cambio
Se guarda la sugerencia del modelo en **toda** transacción que el modelo
coloca, mediante columnas tipadas (en vez de un blob JSON):
- `ai_suggested_category_id` (FK a `categories`, nullable)
- `ai_suggested_category_at` (timestamp, nullable)
- `ai_confidence` (ya existía) — ahora se rellena **siempre**, no solo
al aplicar
Por debajo de la barra la transacción sigue sin categorizar, pero la
sugerencia queda guardada; a partir de la barra, además se autoaplica la
categoría como hasta ahora.
## Para qué sirve
- **Afinar el umbral 0.7 con datos reales**: `where
ai_suggested_category_id is not null and category_id is null` muestra
exactamente qué nos perdemos.
- Habilita una futura UI de "la IA cree que es X, ¿confirmas?"
directamente desde el FK.
## Tests
- `CategorizeTransactionsTest`: el caso sub-umbral ahora verifica que la
sugerencia se persiste; el caso aplicado verifica que además se guardan
los campos de sugerencia.
- Suite de IA + tests que tocan transacciones en verde (111 tests).
## What
During onboarding, the per-transaction AI categorization listener no
longer runs. The AI automation rules generated in the suggestions step
cover the bulk of the imported transactions (~70%). On completion, a
single batch pass categorizes whatever is still uncategorized.
## Why
Running per-transaction categorization on every transaction created
during onboarding is wasteful: most are about to be covered by the AI
rules generated at the end of the flow. Defer the AI spend to one batch
over the remainder.
## How
- **`CategorizeTransactionWithAi` listener** — skips when `!
$user->isOnboarded()`. Covers every txn-creation path, since it hangs
off the model `created` event.
- **`CategorizeOnboardingTransactionsJob`** (new) — dispatched from
`OnboardingController::complete` after `onboarded_at` is set. Snapshots
`whereNull('category_id')->whereNull('description_iv')`, so transactions
already categorized by the AI rules (or manually in the categorize step)
are excluded. Chunks by `group_batch_size` and runs `AiCategorizer` per
chunk. Runs on the dedicated `ai` queue.
- Gated by `AiCategorizationGate::allows()` (rollout flag) — automatic
system behavior, gated like the real-time tier, not the explicit
backfill.
## Flow
1. `syncing` → transactions created, **no** per-transaction AI
2. `ai-suggestions` → AI rules cover the bulk
3. `categorize-transactions` → user manually categorizes a few
4. `complete` → batch-categorize the rest, skipping anything already
labeled
## Tests
- Listener skips transactions created while onboarding; still
categorizes post-onboarding.
- Job categorizes the remainder, leaves rule/manual categories
untouched, no-ops for ineligible users.
- `complete` marks the user onboarded and queues the batch job.
Full Ai + Onboarding suites green (112/112). Pint clean.
## What
Auto-categorizes transactions with AI (Gemini) for **pro +
AI-consented** users when no automation rule already matched. Ships the
full backend **behind a Pennant flag, off by default**, so it's
mergeable and testable in isolation; the UI is a deliberate follow-up.
## Why / cost
Prod check first: ~20k txns/month, **~52% of pro-user transactions are
uncategorized** after rules. At Gemini Flash-Lite rates the cost is a
**rounding error** — ~$0.13–$0.75/month for all pro users, single-digit
dollars even on full Flash. So the model is chosen for accuracy, not
price; the real constraints are trust, accuracy and privacy.
## How it works
**Two tiers** (every transaction is covered, rules are an optimization
on top):
- **Tier 1 – label** — a queued listener runs *after* the synchronous
rules; if still uncategorized and the user is eligible, the model picks
a **leaf** category (referenced by numeric index, never a UUID, so it
can't hallucinate one). Auto-applied only above the **label bar**
(`0.7`); below → left blank, no nag. Tagged `category_source = ai` +
`ai_confidence`, fully reversible.
- **Tier 2 – learn** — above the higher **rule bar** (`0.85`) *and* a
clean merchant key *and* the model flags the merchant unambiguous → the
merchant is appended to a single **ai-owned** automation rule for that
category (OR'd conditions, not rule-sprawl), so future transactions
match for free and consistently. AI rules sit at the lowest priority;
**user-owned rules are never touched**.
**Self-heal + signal** — when a user overrides an AI category, a
`category_correction` is logged (calibration signal, bucketable by
confidence) and the offending merchant condition is dropped from the ai
rule (deleted if empty). User rules and manual categories are untouched.
**Safety** — config kill switch + pro + active consent + gradual Pennant
rollout. Dedicated `ai` queue so Gemini never blocks bank syncs.
Encrypted (client-side) transactions are never sent.
**Backfill** — `ai:categorize-backfill {user}`, explicit opt-in,
batched, learns rules as it goes.
## Data model
- `transactions`: `category_source`, `ai_confidence`,
`categorized_by_rule_id`
- `automation_rules`: `origin` (`user`/`ai`)
- new `category_corrections` table
## Screenshots
<img width="921" height="384" alt="image"
src="https://github.com/user-attachments/assets/f04c2a03-b39e-4a3d-81eb-ecf26eaefb83"
/>
Suggests transaction categorization rules during onboarding.
After a sync or import, it groups the uncategorized transactions, asks
Gemini (via laravel/ai) to map the common merchants to categories, and
shows the results for review. The user edits or drops any and creates
the ones they want. During onboarding the accepted rules also categorize
existing transactions right away.
Off by default: it needs the `AiRuleSuggestions` Pennant flag and a
per-user AI consent. The model and thresholds are config-driven.
`ai:suggest-rules {user}` prints what a user would get.
The settings-page surface and monthly regeneration are a follow-up.
Víctor Falcón