whisper-money/app/Services
Víctor Falcón 08d367a5c8
fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661)
## What & why

A queued historical-balance generator job exhausted the worker's 128MB
PHP memory limit and **died on every retry** — Sentry `PHP-LARAVEL-49` +
`PHP-LARAVEL-4A` (two fingerprints of the *same* poison message:
identical `trace_id`/`message.id`, `retry.count` 1 then 2, OOM at
slightly different `Arr.php` lines).

### Root cause (trace-confirmed)
The Sentry trace localized the OOMing message to
`App\Jobs\GenerateHistoricalRealEstateBalancesJob`, dispatched from
`Settings\AccountController::store` for the "older than 12 months" slice
of history. The fatal frame is `Arr::map` inside Eloquent's
`AccountBalance::upsert()` value-prep (`addUniqueIds`/`addTimestamps`
per-row `array_merge`, `Query\Builder::upsert` `Arr::flatten` bindings,
and a compiled multi-row SQL string — all **O(n)** over the rows).

`purchase_date` (`before_or_equal:today`) and `loan_start_date` (no
constraint) had **no lower bound**, so a mistyped/ancient year (e.g.
`0201`, `1080`) made the generator build a **multi-century monthly
series** and hand it all to a single `upsert`, blowing past 128MB.

## The fix (two commits)

1. **`fix(balances): upsert historical balances in batches`** — build +
upsert in batches of 500 (`UPSERT_CHUNK_SIZE`) in both
`RealEstateBalanceGeneratorService` and `LoanBalanceGeneratorService`,
instead of one giant operation. Same rows, same `(account_id,
balance_date)` conflict target — peak memory is now bounded regardless
of series length. Defense-in-depth.
2. **`fix(accounts): floor purchase/loan start dates at 1900`** — the
actual root cause: add `after_or_equal:1900-01-01` to `purchase_date`
and `loan_start_date`. Rejects the data-entry typos that drive the
runaway series (and the resulting DB bloat / misleading multi-century
net-worth ramp) while accepting any legitimately old asset in a
personal-finance app.

## Review (two agents, before this PR)
Both reviewed against the live trace and the code.

- **Root cause & fix confirmed** — the batching caps exactly the
`Arr::map`-over-N-rows frame from the trace; no independent OOM source
exists (no per-row model events; `upsert` bypasses events).
- **Chunking is correct** — the unique index `(account_id,
balance_date)` exists, so the conflict target is real; `buildDateList`
yields a strictly-increasing de-duplicated list, so batches are disjoint
(no dropped/duplicated dates, no off-by-one). The fresh per-row `id`
UUID is discarded on conflict → idempotent retries.
- **No material user-facing regression.** Balance consumption
(dashboard/net-worth) is date-range-clamped. Batching introduces a
theoretical partial-write-on-mid-job-failure window, but it's idempotent
and self-heals on retry — agents advised **against** wrapping it in a
transaction (holds locks, doesn't help memory).

Applied recommendation #2 (the date floor) as its own commit. Deferred
(both agents agree): extracting the two near-identical generators into a
shared base/trait — a larger refactor that shouldn't ride a hotfix.

## Tests
- Regression test in each service: generate a ~46-year (>500-point)
series and assert the batched writes produce no dropped/duplicated dates
and keep endpoints anchored (crosses multiple batches).
- Validation test: a pre-1900 `purchase_date` is rejected at
`accounts.store`.

Note: the PHP feature suite boots a MySQL testcontainer (Docker) that
isn't available in my environment, so these were validated by
static/`pint`/`php -l` locally and rely on CI for execution — auto-merge
is gated on green CI.

Fixes PHP-LARAVEL-49
Fixes PHP-LARAVEL-4A
2026-07-08 21:34:43 +00:00
..
Ai fix(ai): don't report expected transient provider overloads (PHP-LARAVEL-44) (#655) 2026-07-07 10:55:02 +00:00
Banking fix(banking): recover from EnableBanking 422 wrong-period instead of crashing the sync (PHP-LARAVEL-42) (#653) 2026-07-07 08:57:23 +02:00
Demo Remove encryption from browser tests and demo user (#129) 2026-02-17 11:45:27 +01:00
Discord feat: add Discord admin feed for daily stats and Stripe events (#458) 2026-05-30 18:14:46 +02:00
Stats feat(subscriptions): trial/pricing A/B/C experiment (#600) 2026-06-27 18:00:15 +02:00
Stripe feat(discord): enrich Stripe event messages and dedupe deliveries (#460) 2026-05-31 12:50:37 +02:00
Subscriptions feat(subscriptions): trial/pricing A/B/C experiment (#600) 2026-06-27 18:00:15 +02:00
AccountMetricsService.php feat(dashboard): add accounts manager dialog with visibility toggle and reorder (#604) 2026-06-27 16:11:25 +00:00
AccountUserCurrencyService.php fix(open-banking): stop storing the XXX no-currency placeholder on accounts (#602) 2026-06-27 16:01:21 +00:00
AuthEntryPointService.php fix: route new PWA guests to signup (#313) 2026-04-21 10:53:05 +01:00
AutomationRuleService.php refactor: consolidate duplicated financial calculations (#643) 2026-07-04 22:26:44 +02:00
BalanceLookup.php fix(static-analysis): clear phpstan-baseline by fixing all suppressed errors (#183) 2026-03-02 12:22:30 +00:00
BudgetPeriodService.php fix(budgets): make period generation idempotent (#533) 2026-06-15 12:44:44 +02:00
BudgetTransactionService.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
CashflowSummaryService.php refactor: consolidate duplicated financial calculations (#643) 2026-07-04 22:26:44 +02:00
CategorySpendingService.php refactor: consolidate duplicated financial calculations (#643) 2026-07-04 22:26:44 +02:00
CategoryTree.php feat(analysis): break down spending by sub-category (#508) 2026-06-08 14:53:36 +02:00
CurrencyConversionService.php fix(currency): make rate fetching resilient to slow CDN (#502) 2026-06-08 09:10:38 +02:00
CurrencyOptions.php feat(settings): centralize currency options and split profile/account support (#256) 2026-04-02 19:23:10 +02:00
ExchangeRateService.php Fix PHP-LARAVEL-1V exchange rate cache race (#383) 2026-05-12 12:45:45 +02:00
LandingAuthOverrideService.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LeadCohortResolver.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LeadPromoCodeAllocator.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LoanAmortizationService.php fix(dashboard): avoid month overflow in real estate projection (#340) 2026-04-30 13:53:01 +00:00
LoanBalanceGeneratorService.php fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661) 2026-07-08 21:34:43 +00:00
ManualBalanceAdjuster.php feat(transactions): default balance toggle on and apply it server-side (#566) 2026-06-19 15:01:04 +00:00
PeriodComparator.php feat(cashflow): add savings and period views (#424) 2026-05-25 16:41:00 +02:00
RealEstateBalanceGeneratorService.php fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661) 2026-07-08 21:34:43 +00:00
ResendService.php feat: sync user leads to resend (#283) 2026-04-13 19:56:08 +01:00