whisper-money/app/Services
Víctor Falcón cd918523e8
fix(banking): stop duplicating EnableBanking transactions with positional entry_reference (#669)
## Problem

A user reported the same bank transactions appearing twice.
Investigation in prod showed this is a systematic dedup bug in the
EnableBanking sync affecting a few hundred rows across several
users/accounts.

## Root cause

The affected banks don't return a real `transaction_id`; their only id
is a **positional** `entry_reference` of the form
`{booking_date}.{index}` (e.g. `YYYY-MM-DD.0`). That field is **null the
day a transaction first appears** and only **populated on a later
sync**.

`TransactionFingerprint::for()` preferred `entry_reference` when present
and fell back to a content hash when absent. So the same transaction
produced two different fingerprints:

| | First sync (same day) | Later sync |
|---|---|---|
| `entry_reference` | `null` | `YYYY-MM-DD.0` |
| `dedup_fingerprint` | content-based | id-based |
| `external_transaction_id` | `null` | `YYYY-MM-DD.0` |

Neither the fingerprint nor the `external_transaction_id` dedup path
matched across the two syncs → duplicate row. The one-shot historical
first sync has no duplicates; the duplicates start with the daily
incremental syncs.

## Fix

Treat a positional (`^\d{4}-\d{2}-\d{2}\.\d+$`) `entry_reference` as "no
stable id" and fall through to the content hash, which is identical on
both syncs. Genuine `transaction_id` and non-positional
`entry_reference` keep keying exactly as before.

## Trade-off (accepted, follow-up tracked)

The positional index is also the only field that distinguishes two
genuinely-distinct same-day transactions with byte-identical content.
Falling to the content hash collapses them to one fingerprint, so only
the first is kept — a rare silent under-count. We accept it here over
the systematic duplication it fixes; **existing rows are unaffected**
(their distinct positional value is still stored in
`external_transaction_id` and caught by the fallback dedup path on
re-sync). Fixing both cases needs occurrence-aware dedup in the consumer
(a schema change) — left as a follow-up and documented in the code.

## Tests

- Regression: a positional `entry_reference` matches the same
transaction seen earlier without one.
- Boundary: non-positional references still key on `entry_reference`.
- `php artisan test
tests/Unit/Services/Banking/TransactionFingerprintTest.php` → 6 passed.
`pint` clean.

## Data cleanup (separate, after deploy)

The already-duplicated rows still need a one-off cleanup (soft-delete
the later copy per group, keeping the content-fingerprint original). It
must run **after** this fix ships, otherwise the next daily sync
re-creates them. Not included in this PR.
2026-07-12 16:25:13 +00:00
..
Ai fix(ai): don't report expected transient provider overloads (PHP-LARAVEL-44) (#655) 2026-07-07 10:55:02 +00:00
Banking fix(banking): stop duplicating EnableBanking transactions with positional entry_reference (#669) 2026-07-12 16:25:13 +00:00
Demo Remove encryption from browser tests and demo user (#129) 2026-02-17 11:45:27 +01:00
Discord feat: add Discord admin feed for daily stats and Stripe events (#458) 2026-05-30 18:14:46 +02:00
Stats feat(stats): add unit-economics funnel + connection cost to experiment report (#666) 2026-07-10 18:06:01 +00:00
Stripe feat(discord): enrich Stripe event messages and dedupe deliveries (#460) 2026-05-31 12:50:37 +02:00
Subscriptions feat(subscriptions): trial/pricing A/B/C experiment (#600) 2026-06-27 18:00:15 +02:00
AccountMetricsService.php feat(dashboard): add accounts manager dialog with visibility toggle and reorder (#604) 2026-06-27 16:11:25 +00:00
AccountUserCurrencyService.php fix(open-banking): stop storing the XXX no-currency placeholder on accounts (#602) 2026-06-27 16:01:21 +00:00
AuthEntryPointService.php fix: route new PWA guests to signup (#313) 2026-04-21 10:53:05 +01:00
AutomationRuleService.php refactor: consolidate duplicated financial calculations (#643) 2026-07-04 22:26:44 +02:00
BalanceLookup.php fix(static-analysis): clear phpstan-baseline by fixing all suppressed errors (#183) 2026-03-02 12:22:30 +00:00
BudgetPeriodService.php fix(budgets): make period generation idempotent (#533) 2026-06-15 12:44:44 +02:00
BudgetTransactionService.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
CashflowSummaryService.php refactor: consolidate duplicated financial calculations (#643) 2026-07-04 22:26:44 +02:00
CategorySpendingService.php refactor: consolidate duplicated financial calculations (#643) 2026-07-04 22:26:44 +02:00
CategoryTree.php feat(analysis): break down spending by sub-category (#508) 2026-06-08 14:53:36 +02:00
CurrencyConversionService.php fix(currency): make rate fetching resilient to slow CDN (#502) 2026-06-08 09:10:38 +02:00
CurrencyOptions.php feat(settings): centralize currency options and split profile/account support (#256) 2026-04-02 19:23:10 +02:00
ExchangeRateService.php Fix PHP-LARAVEL-1V exchange rate cache race (#383) 2026-05-12 12:45:45 +02:00
LandingAuthOverrideService.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LeadCohortResolver.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LeadPromoCodeAllocator.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
LoanAmortizationService.php fix(dashboard): avoid month overflow in real estate projection (#340) 2026-04-30 13:53:01 +00:00
LoanBalanceGeneratorService.php fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661) 2026-07-08 21:34:43 +00:00
ManualBalanceAdjuster.php feat(transactions): default balance toggle on and apply it server-side (#566) 2026-06-19 15:01:04 +00:00
PeriodComparator.php feat(cashflow): add savings and period views (#424) 2026-05-25 16:41:00 +02:00
RealEstateBalanceGeneratorService.php fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661) 2026-07-08 21:34:43 +00:00
ResendService.php feat: sync user leads to resend (#283) 2026-04-13 19:56:08 +01:00