## Problem
Production user reported "inaccurate expenses, some appear multiple
times". DB inspection of their active BNP Paribas Fortis connection
confirmed duplicates growing every sync:
- `-3840` on `2026-05-11` x5
- `-2900` on `2026-05-08` x5
- `-2470` on `2026-05-12` x4
- 56 of 776 rows on the account had `external_transaction_id IS NULL`
- 16 (date, amount) duplicate groups, all with NULL upstream id
## Root cause
`TransactionSyncService::importTransaction()` short-circuited dedup when
both `transaction_id` and `entry_reference` were missing:
```php
$externalId = $data['transaction_id'] ?? $data['entry_reference'] ?? null;
if ($externalId) {
// dedup check
}
// else: fall through, always insert
```
BNP returns no stable id for certain card transactions (`status:
"OTHR"`, `bank_transaction_code.code: "CCRD"`, foreign currency). Every
cron tick (every 6h) re-inserts a fresh copy.
Confirmed with prod data: of 16 NULL-id duplicate groups on this user,
**zero** ever got upgraded to a real id later. BNP simply doesn't issue
one.
## Fix
Deterministic per-transaction fingerprint, persisted in a new column,
protected by a unique index.
- **Migration**: adds `transactions.dedup_fingerprint` (nullable string,
80) and unique index on `(account_id, dedup_fingerprint)`. The unique
index is the real source of truth — it also closes the race between
overlapping sync runs that the prior `exists()` + `create()` pattern
couldn't.
- **`TransactionFingerprint::for($data)`**: two-mode fingerprint. If
`transaction_id` or `entry_reference` exists, the fingerprint is based
only on that canonical upstream id. If no upstream id exists, the
fallback fingerprint uses the prod-verified stable fields:
`booking_date`, amount, currency, credit/debit indicator,
creditor/debtor names + accounts, bank tx codes, reference number, and
remittance info. It intentionally excludes volatile fields (`status`,
`value_date`, raw `transaction_date`). Prefix `fp_` avoids mixing with
bank-issued ids.
- **`TransactionSyncService`**:
- Always computes the fingerprint and writes it on every insert.
- Dedup lookup checks the fingerprint **and** (as a fallback) the legacy
`external_transaction_id` to gracefully handle rows imported before the
backfill runs.
- Wraps the insert in a `try/catch UniqueConstraintViolationException`
so concurrent syncs that pass `exists()` together don't crash.
## Rollout plan
Ship migration + service change → new duplicates stop. Existing
duplicate cleanup is intentionally out of scope for this PR.
## Tradeoff
Two genuinely distinct same-day, same-amount card transactions from the
same merchant on the same card collapse into one (no time-of-day in
raw_data for this BNP class). Today we over-count by 3–5x; after the fix
we may rarely under-count by 1. Acceptable net win, can monitor via logs
if needed.
## Tests
- `tests/Unit/Services/Banking/TransactionFingerprintTest.php` — 4 new
tests covering canonical id behavior and volatile-field exclusion.
- `tests/Feature/OpenBanking/TransactionSyncServiceTest.php` — 3 new
tests:
- Dedupes payloads without an upstream id across consecutive syncs.
- Doesn't crash when a payload arrives later with an upstream id
(bounded behavior).
- Dedupes against soft-deleted fingerprinted rows.
- Targeted suite green locally: **17 passed**.
## Files
-
`database/migrations/2026_05_13_085027_add_dedup_fingerprint_to_transactions_table.php`
*(new)*
- `app/Services/Banking/TransactionFingerprint.php` *(new)*
- `app/Services/Banking/TransactionSyncService.php`
- `app/Models/Transaction.php` (fillable)
- Tests above
## Out of scope
A smaller secondary pattern exists where BNP emitted distinct
`transaction_id`s for the same booking (Feb–Apr only, ~10 groups on the
reporter). Not addressed here because: (a) fingerprint includes
`transaction_id` so different ids = different fingerprints, (b) no
recent occurrences, (c) needs API-level analysis to determine when this
is genuine vs noise.
## Summary
- Adds a `BankFormatter` interface with a `BbvaFormatter` implementation
that transforms ALL CAPS `//`-separated BBVA descriptions into readable
Title Case (preserving acronyms like SEPA, S.A., BIZUM and Spanish
stopwords)
- Stores the raw description in a new `original_description` column when
formatting changes it, keeping sync matching intact
- Existing non-BBVA banks and null-bank transactions pass through
unchanged
## Test plan
- [x] Unit tests for `BbvaFormatter` (11 tests: title case, acronyms,
stopwords, reference numbers, mixed-case passthrough, whitespace)
- [x] Unit tests for `TransactionDescriptionFormatter` dispatcher (4
tests: BBVA match, no-match, null bank)
- [x] Feature tests for BBVA sync integration (formatted description +
original stored)
- [x] Feature tests for non-BBVA sync (description unchanged, original
null)
- [x] All 25 existing + new tests pass
- [x] Pint formatting clean
## Summary
- Added `withTrashed()` to the duplicate check in
`TransactionSyncService::importTransaction()` so soft-deleted
transactions are not re-created on subsequent bank syncs.
- Added test covering the soft-deleted transaction deduplication
scenario.
## Test plan
- [x] `php artisan test --filter=TransactionSyncServiceTest` — all 8
tests pass
## Summary
- Adds **EnableBanking** as the first open banking provider, allowing
users to connect real bank accounts and automatically sync transactions
and balances
- Uses a **`BankingProviderInterface`** contract so future providers
(Plaid, GoCardless, etc.) can be added by implementing the same
interface
- Feature-flagged behind the **`open-banking`** Pennant flag (default:
off)
- Connected accounts are **unencrypted** and transactions have `source =
'enablebanking'`
### What's included
**Backend:**
- `BankingProviderInterface` contract + `EnableBankingProvider`
implementation (JWT RS256 auth)
- `BankingConnection` model with full lifecycle (pending →
awaiting_mapping → active → expired/revoked/error)
- `TransactionSyncService` — pagination, deduplication by
`external_transaction_id`, amount/date mapping
- `BalanceSyncService` — preferred balance type selection (CLBD → ITAV
fallback)
- Authorization flow: start auth → bank redirect → callback → session
creation → account mapping → sync
- `SyncBankingConnectionJob` (unique per connection, 3 retries) +
scheduled every 6 hours
- `banking:sync` artisan command
- 5 migrations: `banking_connections` table, account fields, transaction
`external_transaction_id`, `pending_accounts_data`, `linked_at`
**Frontend:**
- Manual vs Connected account choice in the create account dialog
- Multi-step bank connection dialog (country → bank selection →
confirmation → redirect)
- Account mapping page — map discovered bank accounts to existing
accounts, create new ones, or skip
- Settings/Connections page with status badges, sync/disconnect actions
- "Connected" badge on linked accounts in settings
**Tests:**
- 49 tests covering feature flags, controllers, account mapping,
transaction sync, balance sync, deduplication, and pagination
### Feature Flags
This PR introduces **two Pennant feature flags**:
1. **`open-banking`** — Gates the entire open banking feature
(institutions endpoint, authorization flow, connections page). When
disabled, all open banking routes return 404.
2. **`account-mapping`** — Controls whether users see an intermediate
account mapping step after connecting a bank. When **enabled**, users
are redirected to a mapping page where they can choose to create new
accounts, link to existing ones, or skip each discovered bank account.
When **disabled**, all discovered accounts are automatically created
(original behavior). Linked accounts only sync transactions from their
last transaction date and only update the current balance from the
provider (no historical balance calculation or daily balance tracking).
Enable per-user:
```bash
php artisan feature:enable open-banking user@example.com
php artisan feature:enable account-mapping user@example.com
```
Enable for all users:
```bash
php artisan feature:enable open-banking all
php artisan feature:enable account-mapping all
```
## Test plan
- [x] Enable feature flag: `php artisan feature:enable open-banking`
- [x] Verify "Connected" option appears in create account dialog
- [x] Start authorization flow and verify redirect to bank
- [x] With `account-mapping` **disabled**: verify callback creates
accounts directly and dispatches sync
- [x] With `account-mapping` **enabled**: verify callback redirects to
mapping page
- [x] Test mapping page: create new, link to existing, and skip actions
- [x] Verify linked accounts sync only from last transaction date and
only update current balance
- [x] Verify connections page shows "Setup Required" badge for
awaiting_mapping status
- [x] Run `php artisan banking:sync` and verify transactions sync
- [x] Verify connections page shows status, sync, and disconnect actions
- [x] Run full test suite: `php artisan test --compact`
Víctor Falcón