## Why
Sentry issue **PHP-LARAVEL-3S** (`ProviderOverloadedException: AI
provider [gemini] is overloaded`) recurs whenever Gemini returns 503/429
under high demand — 7 events in 3h during the last surge, 0 users
impacted.
Two problems behind it:
1. **Sentry noise.** `CategorizeTransactions::resolveChunkWithRetry`
retries, `laravel/ai` fails over providers, and a still-failing chunk is
deliberately dropped so the rest of the backfill proceeds. But the catch
reported **every** dropped chunk via `report()`, so an expected,
self-healing transient condition floods Sentry and buries real bugs.
2. **Silently lost work.** A dropped chunk leaves those transactions
uncategorized (`category_id NULL`) with nothing to re-trigger them: the
backfill jobs are one-shot (`tries = 1`), there is no scheduled
backfill, and the real-time listener only handles *new* transactions.
They stay uncategorized until someone manually re-runs a backfill.
## What
**1. Stop reporting transient failures.** Catch `FailoverableException`
(the marker interface for `ProviderOverloadedException` /
`RateLimitedException`) separately and log a warning instead of
reporting it. Everything else still goes to `report()` unchanged, so
real failures (malformed responses, insufficient credits, …) keep
surfacing.
**2. Retry the dropped work.** On a transient failure, schedule a
deferred, per-user `RetryTransientAiCategorizationJob` that re-reads the
user's still-pending transactions once the provider has had time to
recover (`ai_categorization.retry_delay`, default 10 min).
- `ShouldBeUnique` per user collapses a surge of dropped chunks into a
**single** retry.
- The unique lock is held through processing, so a retry that overloads
again **cannot chain another** — exactly one deferred attempt per
failure, no infinite loop. Failed 503s aren't billed.
- Wired in `resolve()`, so it covers every entry point (backfill,
onboarding, real-time listener, admin command).
- Model cost is negligible (per config), so the retry re-runs the
existing backfill rather than tracking which exact chunks failed.
## Tests
- Transient overload → chunk dropped, nothing reported, retry scheduled
for the user.
- Unexpected failure → reported, **no** retry scheduled.
- Retry job → categorizes still-pending transactions for a consenting
user; no-op without consent (agent never prompted).
Full `tests/Feature/Ai` + listeners suite green (111 tests); Pint clean.
## Existing prod backlog
The transactions already dropped before this ships stay `pending`; an
`ai:categorize-backfill <user>` recovers them on demand.
Fixes PHP-LARAVEL-3S
## Why
We pay for AI transaction categorization on Gemini, but we couldn't tell
which model produced any given suggestion. The `gemini-flash-latest`
alias used by categorization is **floating** — it silently drifted from
the `flash-lite` tier up to `gemini-3.5-flash`, which is now the
dominant cost on the Gemini dashboard. Before swapping categorization to
a cheaper pinned model we need to measure accuracy per model, and for
that we have to know which model labelled each row.
We already store the AI-suggested category and its confidence on
`transactions`; this adds the missing third axis: the model.
## What
- New nullable `ai_model` column on `transactions`, written by
`CategorizeTransactions::recordOutcome` from
`config('ai_categorization.model')` whenever the model produces a
suggestion (above or below the label bar).
- The migration backfills existing AI-categorized rows
(`ai_suggested_category_at` not null) with `gemini-3.5-flash` — the
model `gemini-flash-latest` resolved to when those suggestions were
made. Done as a raw update so it doesn't bump every row's `updated_at`.
- `ai_model` added to `$fillable` and to `$hidden` (pure ops/measurement
metadata, no need to ship it to the frontend).
## Testing
- Extended `CategorizeTransactionsTest` to assert `ai_model` is stored
on both the auto-applied and below-bar paths.
- `php artisan test tests/Feature/Ai/CategorizeTransactionsTest.php` → 5
passed.
## Follow-ups (not in this PR)
- Pin `AI_CATEGORIZATION_MODEL` to an explicit cheaper tier (mirror the
existing `AI_SUGGESTIONS_MODEL=gemini-2.5-flash-lite`) and kill the
floating alias in the config defaults.
- Capture token usage from the SDK response for cost tracking.
## Contexto
El categorizador solo persistía un resultado cuando la confianza
superaba la barra (`ai_categorization.label_confidence`, 0.7). Las
sugerencias por debajo del umbral se calculaban y se descartaban,
dejándonos sin ninguna visibilidad de qué proponía el modelo para las
transacciones que no se autocategorizaban.
## Cambio
Se guarda la sugerencia del modelo en **toda** transacción que el modelo
coloca, mediante columnas tipadas (en vez de un blob JSON):
- `ai_suggested_category_id` (FK a `categories`, nullable)
- `ai_suggested_category_at` (timestamp, nullable)
- `ai_confidence` (ya existía) — ahora se rellena **siempre**, no solo
al aplicar
Por debajo de la barra la transacción sigue sin categorizar, pero la
sugerencia queda guardada; a partir de la barra, además se autoaplica la
categoría como hasta ahora.
## Para qué sirve
- **Afinar el umbral 0.7 con datos reales**: `where
ai_suggested_category_id is not null and category_id is null` muestra
exactamente qué nos perdemos.
- Habilita una futura UI de "la IA cree que es X, ¿confirmas?"
directamente desde el FK.
## Tests
- `CategorizeTransactionsTest`: el caso sub-umbral ahora verifica que la
sugerencia se persiste; el caso aplicado verifica que además se guardan
los campos de sugerencia.
- Suite de IA + tests que tocan transacciones en verde (111 tests).
## What
Auto-categorizes transactions with AI (Gemini) for **pro +
AI-consented** users when no automation rule already matched. Ships the
full backend **behind a Pennant flag, off by default**, so it's
mergeable and testable in isolation; the UI is a deliberate follow-up.
## Why / cost
Prod check first: ~20k txns/month, **~52% of pro-user transactions are
uncategorized** after rules. At Gemini Flash-Lite rates the cost is a
**rounding error** — ~$0.13–$0.75/month for all pro users, single-digit
dollars even on full Flash. So the model is chosen for accuracy, not
price; the real constraints are trust, accuracy and privacy.
## How it works
**Two tiers** (every transaction is covered, rules are an optimization
on top):
- **Tier 1 – label** — a queued listener runs *after* the synchronous
rules; if still uncategorized and the user is eligible, the model picks
a **leaf** category (referenced by numeric index, never a UUID, so it
can't hallucinate one). Auto-applied only above the **label bar**
(`0.7`); below → left blank, no nag. Tagged `category_source = ai` +
`ai_confidence`, fully reversible.
- **Tier 2 – learn** — above the higher **rule bar** (`0.85`) *and* a
clean merchant key *and* the model flags the merchant unambiguous → the
merchant is appended to a single **ai-owned** automation rule for that
category (OR'd conditions, not rule-sprawl), so future transactions
match for free and consistently. AI rules sit at the lowest priority;
**user-owned rules are never touched**.
**Self-heal + signal** — when a user overrides an AI category, a
`category_correction` is logged (calibration signal, bucketable by
confidence) and the offending merchant condition is dropped from the ai
rule (deleted if empty). User rules and manual categories are untouched.
**Safety** — config kill switch + pro + active consent + gradual Pennant
rollout. Dedicated `ai` queue so Gemini never blocks bank syncs.
Encrypted (client-side) transactions are never sent.
**Backfill** — `ai:categorize-backfill {user}`, explicit opt-in,
batched, learns rules as it goes.
## Data model
- `transactions`: `category_source`, `ai_confidence`,
`categorized_by_rule_id`
- `automation_rules`: `origin` (`user`/`ai`)
- new `category_corrections` table
## Screenshots
<img width="921" height="384" alt="image"
src="https://github.com/user-attachments/assets/f04c2a03-b39e-4a3d-81eb-ecf26eaefb83"
/>
Víctor Falcón