whisper-money/tests/Feature
Víctor Falcón 4038e60fbc
fix(ai): handle transient AI provider overloads — stop the Sentry noise and retry the dropped work (#595)
## Why

Sentry issue **PHP-LARAVEL-3S** (`ProviderOverloadedException: AI
provider [gemini] is overloaded`) recurs whenever Gemini returns 503/429
under high demand — 7 events in 3h during the last surge, 0 users
impacted.

Two problems behind it:

1. **Sentry noise.** `CategorizeTransactions::resolveChunkWithRetry`
retries, `laravel/ai` fails over providers, and a still-failing chunk is
deliberately dropped so the rest of the backfill proceeds. But the catch
reported **every** dropped chunk via `report()`, so an expected,
self-healing transient condition floods Sentry and buries real bugs.
2. **Silently lost work.** A dropped chunk leaves those transactions
uncategorized (`category_id NULL`) with nothing to re-trigger them: the
backfill jobs are one-shot (`tries = 1`), there is no scheduled
backfill, and the real-time listener only handles *new* transactions.
They stay uncategorized until someone manually re-runs a backfill.

## What

**1. Stop reporting transient failures.** Catch `FailoverableException`
(the marker interface for `ProviderOverloadedException` /
`RateLimitedException`) separately and log a warning instead of
reporting it. Everything else still goes to `report()` unchanged, so
real failures (malformed responses, insufficient credits, …) keep
surfacing.

**2. Retry the dropped work.** On a transient failure, schedule a
deferred, per-user `RetryTransientAiCategorizationJob` that re-reads the
user's still-pending transactions once the provider has had time to
recover (`ai_categorization.retry_delay`, default 10 min).
- `ShouldBeUnique` per user collapses a surge of dropped chunks into a
**single** retry.
- The unique lock is held through processing, so a retry that overloads
again **cannot chain another** — exactly one deferred attempt per
failure, no infinite loop. Failed 503s aren't billed.
- Wired in `resolve()`, so it covers every entry point (backfill,
onboarding, real-time listener, admin command).
- Model cost is negligible (per config), so the retry re-runs the
existing backfill rather than tracking which exact chunks failed.

## Tests

- Transient overload → chunk dropped, nothing reported, retry scheduled
for the user.
- Unexpected failure → reported, **no** retry scheduled.
- Retry job → categorizes still-pending transactions for a consenting
user; no-op without consent (agent never prompted).

Full `tests/Feature/Ai` + listeners suite green (111 tests); Pint clean.

## Existing prod backlog

The transactions already dropped before this ships stay `pending`; an
`ai:categorize-backfill <user>` recovers them on demand.

Fixes PHP-LARAVEL-3S
2026-06-26 20:07:06 +02:00
..
Ai fix(ai): handle transient AI provider overloads — stop the Sentry noise and retry the dropped work (#595) 2026-06-26 20:07:06 +02:00
Api Remove encryption from bank account names (#104) 2026-02-09 14:15:26 +01:00
Auth feat(users): track last login and last active timestamps (#516) 2026-06-10 11:01:30 +02:00
Commands feat(leads): add user lead re-invite campaign (#432) 2026-05-26 08:35:31 +02:00
Console feat(features): support percentage rollouts in feature:enable (#592) 2026-06-25 10:46:30 +00:00
Jobs feat(drip): email users stuck on the paywall a day after onboarding (#562) 2026-06-19 14:11:12 +00:00
Listeners feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
Mail feat(leads): add user lead re-invite campaign (#432) 2026-05-26 08:35:31 +02:00
Onboarding feat(ai): defer per-transaction categorization until onboarding completes (#536) 2026-06-15 17:33:26 +02:00
OpenBanking feat(banking): enable Interactive Brokers for all users (#593) 2026-06-26 11:03:21 +02:00
Services feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
Settings fix(account): block deletion while subscription or trial is active (#531) 2026-06-14 20:46:44 +02:00
Sync refactor: Simplify transaction endpoints architecture (#76) 2026-01-25 16:15:17 +01:00
AccountBalanceControllerTest.php feat: investment benefits — show gains/losses on investment accounts (#140) 2026-02-23 13:59:10 +01:00
AccountControllerTest.php feat(accounts): reorder accounts with drag-and-drop (#575) 2026-06-21 11:17:45 +02:00
AiConsentSettingsTest.php feat(ai): manage AI consent outside onboarding with live backfill (#591) 2026-06-25 10:50:35 +02:00
AiConsentTest.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
AlignAccountsEncryptedFlagMigrationTest.php refactor(encryption): strip client-side transaction encryption (#514) 2026-06-20 16:13:26 +00:00
ApplyRealEstateRevaluationTest.php fix(real-estate): compound annual revaluation monthly (#337) 2026-04-27 07:35:51 +01:00
AuthenticatedLayoutSafeAreaTest.php fix(layout): keep bottom padding while floating nav is visible (#537) 2026-06-15 18:23:26 +02:00
AutomationRuleApplicationTest.php feat(transactions): add counterparty fields (#440) 2026-05-27 16:20:55 +02:00
AutomationRuleEvaluationTest.php feat(transactions): add counterparty fields (#440) 2026-05-27 16:20:55 +02:00
AutomationRuleTest.php refactor(api): standardize serialization via model $hidden (#492) 2026-06-05 13:57:34 +02:00
BackfillAccountIbansCommandTest.php chore: upgrade Laravel 12 to 13 (#242) 2026-03-25 12:56:33 +00:00
BalanceLookupTest.php feat: investment benefits — show gains/losses on investment accounts (#140) 2026-02-23 13:59:10 +01:00
BudgetHistoricalAssignmentTest.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
BudgetPeriodDateTest.php Remove budgets feature flag (#108) 2026-02-12 09:58:01 +01:00
BudgetPeriodServiceTest.php fix(budgets): make period generation idempotent (#533) 2026-06-15 12:44:44 +02:00
BudgetTest.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
BudgetTransactionServiceTest.php feat: parent/child category tree (#474) 2026-06-03 19:30:12 +02:00
BulkUpdateTransactionsTest.php refactor: Simplify transaction endpoints architecture (#76) 2026-01-25 16:15:17 +01:00
CancelFreeEnableBankingConnectionsCommandTest.php Cancel Enable Banking connections for free users (#289) 2026-04-15 16:23:03 +02:00
CashflowAnalyticsTest.php fix(cashflow): bound trend window to prevent request timeout (#534) 2026-06-15 12:47:27 +02:00
CashflowPageTest.php feat(cashflow): add savings and period views (#424) 2026-05-25 16:41:00 +02:00
CatchAllBudgetTest.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
CategoryMonthlyBreakdownTest.php feat(analysis): per-category 12-month spending drawer (#519) 2026-06-11 09:52:53 +02:00
CurrencyConversionServiceTest.php fix(currency): make rate fetching resilient to slow CDN (#502) 2026-06-08 09:10:38 +02:00
DashboardAnalyticsTest.php feat: expand parent categories inline in breakdowns (#486) 2026-06-04 11:19:21 +02:00
DashboardTest.php feat: parent/child category tree (#474) 2026-06-03 19:30:12 +02:00
DecryptTransactionsTest.php refactor(encryption): strip client-side transaction encryption (#514) 2026-06-20 16:13:26 +00:00
DeleteUserCommandTest.php Support soft-deleted users with reusable emails (#316) 2026-04-22 11:41:41 +01:00
DemoAccountRestrictionsTest.php feat(demo): gate demo account access behind a config flag (#580) 2026-06-22 11:01:27 +00:00
DisconnectBankingConnectionsCommandTest.php feat(banking): add command to disconnect connections by id (#497) 2026-06-06 11:16:01 +02:00
DiscordWebhookTest.php feat: add Discord admin feed for daily stats and Stripe events (#458) 2026-05-30 18:14:46 +02:00
EncryptionTest.php refactor(encryption): strip client-side transaction encryption (#514) 2026-06-20 16:13:26 +00:00
ExampleTest.php Install Pest 2025-11-07 12:01:58 +00:00
ExchangeRateServiceTest.php Fix PHP-LARAVEL-1V exchange rate cache race (#383) 2026-05-12 12:45:45 +02:00
GenerateStripePromotionCodesCommandTest.php feat(stripe): add promo code generator (#311) 2026-04-20 18:15:28 +01:00
IdorVulnerabilityTest.php refactor: Simplify transaction endpoints architecture (#76) 2026-01-25 16:15:17 +01:00
ImportDataTest.php fix: Apply automation rule labels on transaction creation and import (#79) 2026-01-27 11:11:29 +01:00
InertiaSharedDataTest.php feat(banking): enable Interactive Brokers for all users (#593) 2026-06-26 11:03:21 +02:00
IntegrationRequestTest.php fix(integration-requests): freeze votes on not-doable requests (#555) 2026-06-18 10:54:56 +00:00
LabelTest.php refactor(api): standardize serialization via model $hidden (#492) 2026-06-05 13:57:34 +02:00
LandingAuthOverrideTest.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LoanTest.php feat(discord): enrich Stripe event messages and dedupe deliveries (#460) 2026-05-31 12:50:37 +02:00
LocalizationTest.php feat(i18n): add French translation support (#532) 2026-06-15 19:15:43 +02:00
LoggingConfigTest.php fix(logging): keep laravel.log writable across container UIDs (#451) 2026-05-29 15:10:50 +02:00
MailSenderTest.php Cancel Enable Banking connections for free users (#289) 2026-04-15 16:23:03 +02:00
PlaintextTransactionsTest.php Remove plaintext-transactions feature flag & E2E references (#116) 2026-02-13 11:10:21 +01:00
PlanFeatureTest.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
PopoverSafeAreaTest.php fix: keep iOS popovers below the notch (#282) 2026-04-13 15:19:56 +01:00
PostStripeEventToDiscordTest.php fix(discord): skip zero-amount payment stats messages (#540) 2026-06-15 19:21:47 +02:00
PwaTest.php Harden browser storage and PostHog recording (#402) 2026-05-14 15:57:48 +02:00
ReEvaluateTransactionRulesTest.php chore: upgrade Laravel 12 to 13 (#242) 2026-03-25 12:56:33 +00:00
RealEstateAvailabilityTest.php refactor(real-estate): remove Pennant gating (#308) 2026-04-20 13:31:49 +01:00
RealEstateTest.php feat(accounts): reorder accounts with drag-and-drop (#575) 2026-06-21 11:17:45 +02:00
ResendLeadVerificationEmailsCommandTest.php feat: resend verification emails to unverified leads (#287) 2026-04-15 09:13:27 +01:00
ResendSyncCommandTest.php feat: Sync new users to Resend contacts (#85) 2026-01-28 21:25:58 +01:00
ResendSyncLeadsCommandTest.php perf(resend): default sync-leads to last 24h window (#354) 2026-05-05 09:57:25 +01:00
RouteNotificationForMailTest.php fix(notifications): skip mail dispatch when recipient email is invalid (#387) 2026-05-13 09:47:50 +01:00
SavedFilterTest.php feat(analysis): project-aware transaction analysis (#513) 2026-06-09 15:32:07 +02:00
SendAiCohortReportCommandTest.php feat(ai): add weekly AI-suggestions cohort report (#530) 2026-06-13 23:23:34 +02:00
SendAiConsentFollowUpEmailsCommandTest.php feat(email): follow up after post-onboarding AI consent (#596) 2026-06-26 17:56:06 +00:00
SendDailyStatsReportCommandTest.php feat(discord): enrich Stripe event messages and dedupe deliveries (#460) 2026-05-31 12:50:37 +02:00
SendStuckCohortReportCommandTest.php feat(stats): weekly paywall stuck-cohort report to Discord (#563) 2026-06-19 14:12:51 +00:00
SentryConfigTest.php ci: remove production deploy job (#574) 2026-06-20 18:17:14 +00:00
SentryUserMiddlewareTest.php Add Sentry user context (#348) 2026-05-04 13:26:50 +01:00
SetLocaleTest.php feat(i18n): add French translation support (#532) 2026-06-15 19:15:43 +02:00
SitemapTest.php User Onboarding Flow (#23) 2025-12-12 13:06:08 +01:00
StripeSubscriptionStatsCommandTest.php feat: add Discord admin feed for daily stats and Stripe events (#458) 2026-05-30 18:14:46 +02:00
SubscriptionTest.php feat(paywall): require a plan when the user has accepted AI (#564) 2026-06-19 14:18:49 +00:00
SuggestionPersistenceTest.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
SyncBankingConnectionsCommandTest.php chore: Remove account-mapping feature flag (#252) 2026-04-01 12:09:22 +02:00
SyncStripePricesCommandTest.php fix(pricing): update final release prices (#288) 2026-04-15 14:49:02 +01:00
TrackLastActiveAtTest.php feat(users): track last login and last active timestamps (#516) 2026-06-10 11:01:30 +02:00
TransactionAnalysisTest.php feat(transactions): release transaction analysis to all users (#579) 2026-06-22 16:09:09 +02:00
TransactionFilterTest.php fix(transactions): prevent crash when sorting by nullable column (#501) 2026-06-06 17:23:21 +02:00
TransactionTest.php feat(transactions): default balance toggle on and apply it server-side (#566) 2026-06-19 15:01:04 +00:00
UserLeadTest.php feat: verify waitlist leads (#285) 2026-04-14 11:26:01 +01:00
WelcomeBanksOrderingTest.php feat(open-banking): remove feature flag gating (#297) 2026-04-17 10:20:05 +02:00