whisper-money/app
Víctor Falcón cd918523e8
fix(banking): stop duplicating EnableBanking transactions with positional entry_reference (#669)
## Problem

A user reported the same bank transactions appearing twice.
Investigation in prod showed this is a systematic dedup bug in the
EnableBanking sync affecting a few hundred rows across several
users/accounts.

## Root cause

The affected banks don't return a real `transaction_id`; their only id
is a **positional** `entry_reference` of the form
`{booking_date}.{index}` (e.g. `YYYY-MM-DD.0`). That field is **null the
day a transaction first appears** and only **populated on a later
sync**.

`TransactionFingerprint::for()` preferred `entry_reference` when present
and fell back to a content hash when absent. So the same transaction
produced two different fingerprints:

| | First sync (same day) | Later sync |
|---|---|---|
| `entry_reference` | `null` | `YYYY-MM-DD.0` |
| `dedup_fingerprint` | content-based | id-based |
| `external_transaction_id` | `null` | `YYYY-MM-DD.0` |

Neither the fingerprint nor the `external_transaction_id` dedup path
matched across the two syncs → duplicate row. The one-shot historical
first sync has no duplicates; the duplicates start with the daily
incremental syncs.

## Fix

Treat a positional (`^\d{4}-\d{2}-\d{2}\.\d+$`) `entry_reference` as "no
stable id" and fall through to the content hash, which is identical on
both syncs. Genuine `transaction_id` and non-positional
`entry_reference` keep keying exactly as before.

## Trade-off (accepted, follow-up tracked)

The positional index is also the only field that distinguishes two
genuinely-distinct same-day transactions with byte-identical content.
Falling to the content hash collapses them to one fingerprint, so only
the first is kept — a rare silent under-count. We accept it here over
the systematic duplication it fixes; **existing rows are unaffected**
(their distinct positional value is still stored in
`external_transaction_id` and caught by the fallback dedup path on
re-sync). Fixing both cases needs occurrence-aware dedup in the consumer
(a schema change) — left as a follow-up and documented in the code.

## Tests

- Regression: a positional `entry_reference` matches the same
transaction seen earlier without one.
- Boundary: non-positional references still key on `entry_reference`.
- `php artisan test
tests/Unit/Services/Banking/TransactionFingerprintTest.php` → 6 passed.
`pint` clean.

## Data cleanup (separate, after deploy)

The already-duplicated rows still need a one-off cleanup (soft-delete
the later copy per group, keeping the content-fingerprint original). It
must run **after** this fix ships, otherwise the next daily sync
re-creates them. Not included in this PR.
2026-07-12 16:25:13 +00:00
..
Actions feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
Ai/Agents feat(ai): auto-categorize transactions with AI (behind flag) (#535) 2026-06-15 16:35:20 +02:00
Console/Commands feat(stats): add unit-economics funnel + connection cost to experiment report (#666) 2026-07-10 18:06:01 +00:00
Contracts feat(connections): manage which accounts a bank connection syncs (#558) 2026-06-18 16:22:49 +02:00
Enums feat(transactions): serve import dedup and account ledger from the backend (#631) 2026-07-03 16:49:59 +02:00
Events Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
Exceptions/Banking fix(banking): recover from EnableBanking 422 wrong-period instead of crashing the sync (PHP-LARAVEL-42) (#653) 2026-07-07 08:57:23 +02:00
Features refactor(ai): remove AiConsentSettings feature flag (#619) 2026-07-01 09:47:55 +02:00
Http fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661) 2026-07-08 21:34:43 +00:00
Jobs fix(queue): raise retry_after above the longest job timeout (PHP-LARAVEL-2D) (#645) 2026-07-05 09:31:23 +00:00
Listeners fix(discord): show old → new plan on plan change notification (#637) 2026-07-04 15:49:09 +00:00
Mail refactor(drip): extract base mailable and job for the drip email family (#641) 2026-07-04 20:51:38 +02:00
Models feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650) 2026-07-09 14:26:07 +02:00
Notifications fix: verify email via signed link without requiring login (#490) 2026-06-05 10:01:32 +02:00
Policies refactor(policies): extract HandlesUserOwnership trait (#478) 2026-06-03 17:43:30 +02:00
Providers fix: stop double-dispatching transaction listeners (N+1 insert into jobs) (#620) 2026-07-02 13:26:45 +00:00
Rules feat: parent/child category tree (#474) 2026-06-03 19:30:12 +02:00
Services fix(banking): stop duplicating EnableBanking transactions with positional entry_reference (#669) 2026-07-12 16:25:13 +00:00