## Problem
A user reported the same bank transactions appearing twice.
Investigation in prod showed this is a systematic dedup bug in the
EnableBanking sync affecting a few hundred rows across several
users/accounts.
## Root cause
The affected banks don't return a real `transaction_id`; their only id
is a **positional** `entry_reference` of the form
`{booking_date}.{index}` (e.g. `YYYY-MM-DD.0`). That field is **null the
day a transaction first appears** and only **populated on a later
sync**.
`TransactionFingerprint::for()` preferred `entry_reference` when present
and fell back to a content hash when absent. So the same transaction
produced two different fingerprints:
| | First sync (same day) | Later sync |
|---|---|---|
| `entry_reference` | `null` | `YYYY-MM-DD.0` |
| `dedup_fingerprint` | content-based | id-based |
| `external_transaction_id` | `null` | `YYYY-MM-DD.0` |
Neither the fingerprint nor the `external_transaction_id` dedup path
matched across the two syncs → duplicate row. The one-shot historical
first sync has no duplicates; the duplicates start with the daily
incremental syncs.
## Fix
Treat a positional (`^\d{4}-\d{2}-\d{2}\.\d+$`) `entry_reference` as "no
stable id" and fall through to the content hash, which is identical on
both syncs. Genuine `transaction_id` and non-positional
`entry_reference` keep keying exactly as before.
## Trade-off (accepted, follow-up tracked)
The positional index is also the only field that distinguishes two
genuinely-distinct same-day transactions with byte-identical content.
Falling to the content hash collapses them to one fingerprint, so only
the first is kept — a rare silent under-count. We accept it here over
the systematic duplication it fixes; **existing rows are unaffected**
(their distinct positional value is still stored in
`external_transaction_id` and caught by the fallback dedup path on
re-sync). Fixing both cases needs occurrence-aware dedup in the consumer
(a schema change) — left as a follow-up and documented in the code.
## Tests
- Regression: a positional `entry_reference` matches the same
transaction seen earlier without one.
- Boundary: non-positional references still key on `entry_reference`.
- `php artisan test
tests/Unit/Services/Banking/TransactionFingerprintTest.php` → 6 passed.
`pint` clean.
## Data cleanup (separate, after deploy)
The already-duplicated rows still need a one-off cleanup (soft-delete
the later copy per group, keeping the content-fingerprint original). It
must run **after** this fix ships, otherwise the next daily sync
re-creates them. Not included in this PR.
## Summary
Wave 2 structural refactor: the same financial math was copy-pasted
across the
dashboard and the analytics API endpoints, so it could silently diverge
between
screens — a real risk in a finance app. This consolidates the duplicated
calculations into single sources of truth, kills a net-worth N+1, and
aligns
the PHP/TS rule engines and the `TransactionSource` enum. Every change
is
**behavior-preserving**; the numeric outputs of every endpoint are
unchanged and
are locked down with characterization/parity tests.
Builds on merged #640 (Wave 1); no file overlap. No dependencies
changed.
## Changes (per commit)
- **Consolidate savings-rate math into `CashflowSummaryService`** —
`savings_rate`
and `net` were byte-identical inline in `DashboardController` and
`Api/CashflowAnalyticsController`. Extracted to
`CashflowSummaryService::summarize(income, expense)`;
both controllers now spread its result (same keys, same values).
- **Move income/expense-side classification onto the `Transaction`
model** —
the income/expense side test was reimplemented in three places
(`Api/TransactionAnalysisController`, `Api/CashflowAnalyticsController`,
`DashboardController`). Now `Transaction::isIncomeSide()` /
`isExpenseSide()`.
- **Extract duplicated `getCategorySpending` into
`CategorySpendingService`** —
the tree-rollup expense-spending query was duplicated verbatim between
`DashboardController` and `Api/DashboardAnalyticsController`. Moved to
`CategorySpendingService::forPeriod()` (drill-parent parameterized).
- **Batch net-worth balance lookups to kill the per-account N+1** —
`Api/DashboardAnalyticsController::calculateNetWorthAt` ran one
`AccountBalance` query per account per compared period. Now uses the
existing
`BalanceLookup::forAccounts()` (fixed 3 queries via carry-forward seed +
in-range records), reproducing the exact "latest balance <= date, else
0"
semantics.
- **Align server rule normalization with the client and lock it with
parity
fixtures** — `AutomationRuleService::normalizeRuleJson` protected only
`['description','notes']` while `rule-engine.ts` also protected
`creditor_name`/`debtor_name`. Aligned to the superset (structurally a
no-op
since those var names are already lowercase, so no matching change) and
added
shared PHP+TS parity fixtures so the two engines can never drift
unnoticed.
- **Add missing `TransactionSource` cases to the TS type** —
`transaction.ts`
was missing `enablebanking`/`wise`; now mirrors
`App\Enums\TransactionSource`.
- **Guard `sumTransactions` against unsupported category types**
(reviewer fix) —
replaced the income/expense ternary that silently treated any non-Income
type
as expense with a `match` that throws on Savings/Investment/Transfer.
- **Document category eager-load expectation on `Transaction` side
methods**
(reviewer fix) — doc-only note to prevent a future N+1.
## Test plan
New tests:
- `tests/Unit/Services/CashflowSummaryServiceTest.php` —
net/savings-rate/rounding/div-by-zero.
- `tests/Feature/TransactionSideClassificationTest.php` — income/expense
side across signs, uncategorized, and transfer/savings/investment =
neither side.
- `tests/Feature/DashboardAnalyticsTest.php` — new net-worth test
asserts both the values (600000 / 540000) and a flat balance-query count
(<= 3) regardless of account count.
- `tests/Feature/RuleEngineParityTest.php` +
`resources/js/lib/rule-engine-parity.test.ts` +
`tests/Fixtures/rule-engine-parity.json` — one shared fixture set
driving both the PHP and TS rule engines.
Results (targeted, local):
- `--filter=Cashflow` (exclude Browser): 57/57 passed
- `--filter=DashboardAnalytics`: 41/41 passed
- `--filter=AutomationRule`: 60/60 passed
-
`--filter=TransactionSideClassification|CashflowSummaryService|RuleEngineParity`:
21/21 passed
- `bun run test rule-engine` (vitest): 13/13 passed
- `vendor/bin/pint --test`: pass; `bun run lint`: 0 errors; `bun run
format:check`: clean
- `bun run types`: 157 errors (unchanged pre-existing baseline), 0 in
touched files
Note: the 6 `Cashflow*` Browser tests fail locally only on "Vite
manifest not found" (no build present); they are environmental, not
logic, and pass in CI.
## Reviewer findings
Two read-only reviewers (architecture/quality and product/behavior)
reviewed the diff.
**Addressed**
- Both flagged that `sumTransactions` silently treated any non-Income
type as expense — added a throwing `match` guard.
- Eager-load expectation documented on the `Transaction` side methods.
**Verified identical** (behavior reviewer):
income/expense/net/savings_rate across all three endpoints; net worth
for both compared dates including no-record / all-records-after-range /
same-date edge cases; category spending (uncategorized excluded,
soft-deleted categories excluded, rollup/drill preserved); rule-engine
normalization output; multi-currency conversion.
**Deferred (documented)**
- The income/expense **summation** itself is still computed three ways
with differing uncategorized-transaction handling (dashboard
`whereExists` + sign vs analytics `join` excluding uncategorized vs
in-memory `isIncomeSide`). Unifying it would change numbers, so it is
out of scope for this behavior-preserving PR — worth a dedicated
follow-up.
- `savings_rate` keeps its `int|float` union (int `0` when income is 0).
Intentionally preserved to keep JSON output byte-identical.
- `BalanceLookup`'s `empty()` guard never short-circuits a `Collection`,
so a zero-account user runs 3 empty (harmless) queries. Left untouched —
it lives in a shared, unchanged service and only affects a no-account
edge case.
Do not merge before Wave 1 (#640) is in main.
## Summary
- add creditor/debtor fields to transactions with raw_data backfill
- store counterparties on bank sync and CSV/XLS imports
- add creditor/debtor filters plus hidden table columns
## Tests
- php artisan test --compact tests/Feature/LocalizationTest.php
tests/Feature/OpenBanking/TransactionSyncServiceTest.php
tests/Feature/TransactionFilterTest.php
- npm test -- resources/js/lib/file-parser.test.ts --run
- vendor/bin/pint --dirty --format agent
Note: `npm run types` still has pre-existing unrelated errors; no
creditor/debtor/import-related errors remained in filtered output.
## Screenshot
<img width="1241" height="676" alt="8odYWtFcvUM"
src="https://github.com/user-attachments/assets/55653485-d588-4beb-9e6a-5c7c81ba7cf8"
/>
## Problem
Production user reported "inaccurate expenses, some appear multiple
times". DB inspection of their active BNP Paribas Fortis connection
confirmed duplicates growing every sync:
- `-3840` on `2026-05-11` x5
- `-2900` on `2026-05-08` x5
- `-2470` on `2026-05-12` x4
- 56 of 776 rows on the account had `external_transaction_id IS NULL`
- 16 (date, amount) duplicate groups, all with NULL upstream id
## Root cause
`TransactionSyncService::importTransaction()` short-circuited dedup when
both `transaction_id` and `entry_reference` were missing:
```php
$externalId = $data['transaction_id'] ?? $data['entry_reference'] ?? null;
if ($externalId) {
// dedup check
}
// else: fall through, always insert
```
BNP returns no stable id for certain card transactions (`status:
"OTHR"`, `bank_transaction_code.code: "CCRD"`, foreign currency). Every
cron tick (every 6h) re-inserts a fresh copy.
Confirmed with prod data: of 16 NULL-id duplicate groups on this user,
**zero** ever got upgraded to a real id later. BNP simply doesn't issue
one.
## Fix
Deterministic per-transaction fingerprint, persisted in a new column,
protected by a unique index.
- **Migration**: adds `transactions.dedup_fingerprint` (nullable string,
80) and unique index on `(account_id, dedup_fingerprint)`. The unique
index is the real source of truth — it also closes the race between
overlapping sync runs that the prior `exists()` + `create()` pattern
couldn't.
- **`TransactionFingerprint::for($data)`**: two-mode fingerprint. If
`transaction_id` or `entry_reference` exists, the fingerprint is based
only on that canonical upstream id. If no upstream id exists, the
fallback fingerprint uses the prod-verified stable fields:
`booking_date`, amount, currency, credit/debit indicator,
creditor/debtor names + accounts, bank tx codes, reference number, and
remittance info. It intentionally excludes volatile fields (`status`,
`value_date`, raw `transaction_date`). Prefix `fp_` avoids mixing with
bank-issued ids.
- **`TransactionSyncService`**:
- Always computes the fingerprint and writes it on every insert.
- Dedup lookup checks the fingerprint **and** (as a fallback) the legacy
`external_transaction_id` to gracefully handle rows imported before the
backfill runs.
- Wraps the insert in a `try/catch UniqueConstraintViolationException`
so concurrent syncs that pass `exists()` together don't crash.
## Rollout plan
Ship migration + service change → new duplicates stop. Existing
duplicate cleanup is intentionally out of scope for this PR.
## Tradeoff
Two genuinely distinct same-day, same-amount card transactions from the
same merchant on the same card collapse into one (no time-of-day in
raw_data for this BNP class). Today we over-count by 3–5x; after the fix
we may rarely under-count by 1. Acceptable net win, can monitor via logs
if needed.
## Tests
- `tests/Unit/Services/Banking/TransactionFingerprintTest.php` — 4 new
tests covering canonical id behavior and volatile-field exclusion.
- `tests/Feature/OpenBanking/TransactionSyncServiceTest.php` — 3 new
tests:
- Dedupes payloads without an upstream id across consecutive syncs.
- Doesn't crash when a payload arrives later with an upstream id
(bounded behavior).
- Dedupes against soft-deleted fingerprinted rows.
- Targeted suite green locally: **17 passed**.
## Files
-
`database/migrations/2026_05_13_085027_add_dedup_fingerprint_to_transactions_table.php`
*(new)*
- `app/Services/Banking/TransactionFingerprint.php` *(new)*
- `app/Services/Banking/TransactionSyncService.php`
- `app/Models/Transaction.php` (fillable)
- Tests above
## Out of scope
A smaller secondary pattern exists where BNP emitted distinct
`transaction_id`s for the same booking (Feb–Apr only, ~10 groups on the
reporter). Not addressed here because: (a) fingerprint includes
`transaction_id` so different ids = different fingerprints, (b) no
recent occurrences, (c) needs API-level analysis to determine when this
is genuine vs noise.
## Summary
- Adds loan detail tracking (interest rate, term, start date, original
amount) to loan-type accounts, following the existing
`real_estate_details` pattern
- Implements `LoanAmortizationService` with standard amortization math
to automatically project month-to-month balance evolution
- Shows projected future balances as a dashed line on the account
balance chart, visually distinct from historical data
- Adds a scheduled command (`loans:generate-balances`) to auto-generate
monthly balance entries for loan accounts
- Includes 36 tests (10 unit for pure math, 26 feature for CRUD, API
projections, command, and authorization)
## Changes
### Backend
- **Migration**: `loan_details` table (UUID PK, unique `account_id` FK,
interest rate, term, start date, original amount)
- **LoanDetail model** + factory with `Account::loanDetail()` HasOne
relationship
- **LoanAmortizationService**: `calculateMonthlyPayment`,
`calculateRemainingBalance`, `generateProjection`, `projectFromBalance`,
`calculateRemainingMonths`, `getBalanceAtDate`
- **GenerateMonthlyLoanBalances** command: runs monthly on 1st, skips
existing entries
- **LoanDetailController**: PATCH endpoint for editing loan details from
show page
- **Settings\AccountController**: creates/updates `LoanDetail` on
store/update
- **DashboardAnalyticsController**: appends projected data points with
`projected: true` flag
- **Validation**: loan-specific rules in StoreAccountRequest and
UpdateAccountRequest
### Frontend
- **AccountForm**: loan fields (rate, term, start date, original amount)
shown when `type === 'loan'`
- **Create/Edit dialogs**: pass loan data fields in POST/PATCH payloads
- **Show page**: `LoanDetailsCard` component with edit/read modes,
computed monthly payment and remaining months
- **Balance chart**: new `ComposedChart` branch with dashed `Line`
overlay for projected data points
- **Types**: `LoanDetail` interface in `account.ts`
## Summary
- Adds a `BankFormatter` interface with a `BbvaFormatter` implementation
that transforms ALL CAPS `//`-separated BBVA descriptions into readable
Title Case (preserving acronyms like SEPA, S.A., BIZUM and Spanish
stopwords)
- Stores the raw description in a new `original_description` column when
formatting changes it, keeping sync matching intact
- Existing non-BBVA banks and null-bank transactions pass through
unchanged
## Test plan
- [x] Unit tests for `BbvaFormatter` (11 tests: title case, acronyms,
stopwords, reference numbers, mixed-case passthrough, whitespace)
- [x] Unit tests for `TransactionDescriptionFormatter` dispatcher (4
tests: BBVA match, no-match, null bank)
- [x] Feature tests for BBVA sync integration (formatted description +
original stored)
- [x] Feature tests for non-BBVA sync (description unchanged, original
null)
- [x] All 25 existing + new tests pass
- [x] Pint formatting clean
Víctor Falcón