whisper-money/app/Http/Requests
Víctor Falcón 08d367a5c8
fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661)
## What & why

A queued historical-balance generator job exhausted the worker's 128MB
PHP memory limit and **died on every retry** — Sentry `PHP-LARAVEL-49` +
`PHP-LARAVEL-4A` (two fingerprints of the *same* poison message:
identical `trace_id`/`message.id`, `retry.count` 1 then 2, OOM at
slightly different `Arr.php` lines).

### Root cause (trace-confirmed)
The Sentry trace localized the OOMing message to
`App\Jobs\GenerateHistoricalRealEstateBalancesJob`, dispatched from
`Settings\AccountController::store` for the "older than 12 months" slice
of history. The fatal frame is `Arr::map` inside Eloquent's
`AccountBalance::upsert()` value-prep (`addUniqueIds`/`addTimestamps`
per-row `array_merge`, `Query\Builder::upsert` `Arr::flatten` bindings,
and a compiled multi-row SQL string — all **O(n)** over the rows).

`purchase_date` (`before_or_equal:today`) and `loan_start_date` (no
constraint) had **no lower bound**, so a mistyped/ancient year (e.g.
`0201`, `1080`) made the generator build a **multi-century monthly
series** and hand it all to a single `upsert`, blowing past 128MB.

## The fix (two commits)

1. **`fix(balances): upsert historical balances in batches`** — build +
upsert in batches of 500 (`UPSERT_CHUNK_SIZE`) in both
`RealEstateBalanceGeneratorService` and `LoanBalanceGeneratorService`,
instead of one giant operation. Same rows, same `(account_id,
balance_date)` conflict target — peak memory is now bounded regardless
of series length. Defense-in-depth.
2. **`fix(accounts): floor purchase/loan start dates at 1900`** — the
actual root cause: add `after_or_equal:1900-01-01` to `purchase_date`
and `loan_start_date`. Rejects the data-entry typos that drive the
runaway series (and the resulting DB bloat / misleading multi-century
net-worth ramp) while accepting any legitimately old asset in a
personal-finance app.

## Review (two agents, before this PR)
Both reviewed against the live trace and the code.

- **Root cause & fix confirmed** — the batching caps exactly the
`Arr::map`-over-N-rows frame from the trace; no independent OOM source
exists (no per-row model events; `upsert` bypasses events).
- **Chunking is correct** — the unique index `(account_id,
balance_date)` exists, so the conflict target is real; `buildDateList`
yields a strictly-increasing de-duplicated list, so batches are disjoint
(no dropped/duplicated dates, no off-by-one). The fresh per-row `id`
UUID is discarded on conflict → idempotent retries.
- **No material user-facing regression.** Balance consumption
(dashboard/net-worth) is date-range-clamped. Batching introduces a
theoretical partial-write-on-mid-job-failure window, but it's idempotent
and self-heals on retry — agents advised **against** wrapping it in a
transaction (holds locks, doesn't help memory).

Applied recommendation #2 (the date floor) as its own commit. Deferred
(both agents agree): extracting the two near-identical generators into a
shared base/trait — a larger refactor that shouldn't ride a hotfix.

## Tests
- Regression test in each service: generate a ~46-year (>500-point)
series and assert the batched writes produce no dropped/duplicated dates
and keep endpoints anchored (crosses multiple batches).
- Validation test: a pre-1900 `purchase_date` is rejected at
`accounts.store`.

Note: the PHP feature suite boots a MySQL testcontainer (Docker) that
isn't available in my environment, so these were validated by
static/`pint`/`php -l` locally and rely on CI for execution — auto-merge
is gated on green CI.

Fixes PHP-LARAVEL-49
Fixes PHP-LARAVEL-4A
2026-07-08 21:34:43 +00:00
..
Ai feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
Api feat(transactions): serve import dedup and account ledger from the backend (#631) 2026-07-03 16:49:59 +02:00
Concerns fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661) 2026-07-08 21:34:43 +00:00
OpenBanking feat(banking): add Interactive Brokers sync via Flex Web Service (#581) 2026-06-23 11:39:24 +02:00
Settings refactor: centralize duplicated provider & locale keys into enums (#543) 2026-06-16 13:43:14 +00:00
BulkReEvaluateRulesRequest.php feat(transactions): add counterparty fields (#440) 2026-05-27 16:20:55 +02:00
BulkUpdateTransactionsRequest.php refactor(requests): extract user-owned exists rules to trait (#477) 2026-06-03 17:33:54 +02:00
IndexTransactionRequest.php feat(ai): auto-categorize transactions with AI (behind flag) (#535) 2026-06-15 16:35:20 +02:00
ReorderAccountsRequest.php feat(accounts): reorder accounts with drag-and-drop (#575) 2026-06-21 11:17:45 +02:00
StoreAccountBalanceRequest.php feat: investment benefits — show gains/losses on investment accounts (#140) 2026-02-23 13:59:10 +01:00
StoreBudgetRequest.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
StoreIntegrationRequestRequest.php feat(integration-requests): community board to request & vote bank integrations (#550) 2026-06-17 12:50:51 +00:00
StoreRealEstateDetailRequest.php refactor(requests): share account detail validation rules (#481) 2026-06-03 19:01:03 +02:00
StoreTransactionRequest.php refactor(requests): extract user-owned exists rules to trait (#477) 2026-06-03 17:33:54 +02:00
StoreUserLeadRequest.php feat(i18n): add French translation support (#532) 2026-06-15 19:15:43 +02:00
UpdateAccountVisibilityRequest.php feat(dashboard): add accounts manager dialog with visibility toggle and reorder (#604) 2026-06-27 16:11:25 +00:00
UpdateBudgetRequest.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
UpdateCurrentAccountBalanceRequest.php feat: investment benefits — show gains/losses on investment accounts (#140) 2026-02-23 13:59:10 +01:00
UpdateLoanDetailRequest.php feat(accounts): add loan amortization projections for loan accounts (#246) 2026-03-26 15:06:09 +01:00
UpdateRealEstateDetailRequest.php refactor(requests): share account detail validation rules (#481) 2026-06-03 19:01:03 +02:00
UpdateTransactionRequest.php refactor(requests): extract user-owned exists rules to trait (#477) 2026-06-03 17:33:54 +02:00