## Why
A support ticket: Bankinter transactions arrive with the raw ISO 20022
remittance tag in front of the text, so AI categorization reads the tag
instead of the merchant.
```
/TXT/D|SumUp *GELATERIA SALV
/TXT/H|TRANSF NOMI /AIGUA DE RIGAT, S ← the payroll that got categorized as Fuel
/TXT/CONDIS SANT JUST DESV07/07/26|20260714
```
`/TXT/` is the unstructured remittance tag, `D|`/`H|` is the debe/haber
marker (which only repeats the sign of the amount), and card payments
append the purchase and settlement dates. None of it describes the
transaction.
In production this hits **7297 transactions across 20 users** —
Bankinter (6569) and Unicaja Banco (728), which ships the same shape.
## What
**`RemittanceTagFormatter`** strips the tag, the credit/debit marker,
the card dates and the `#` marker on card charges. The tag identifies
itself, so the formatter is keyed on the **description** rather than on
a bank name — it works for any bank shipping the same shape instead of
needing a new class per bank. `BankFormatter::matches()` now takes the
description as well; `BbvaFormatter` keeps matching on the bank name.
**`banking:backfill-descriptions`** fixes the rows that are already
imported. It also rewrites the automation rules that match on the raw
text: **48 user-authored rules across 4 users** contain literals like
`/TXT/D|RECIBO VISA CLASICA`, and rewriting descriptions without
rewriting those rules would silently stop them from ever matching again.
A test pins that a rule still matches its transaction after both are
rewritten.
```
banking:backfill-descriptions [--user=email] [--dry-run] [-v]
```
## QA
Ran the formatter over **all 2755 distinct tagged descriptions in
production**: 0 no-ops, 0 leftover tags/dates/markers, 0 degenerate
output.
Ran the command against a database seeded with production-shaped rows
and rules:
```
===== DRY RUN (-v) =====
DRY RUN — no changes will be saved.
/TXT/D|SumUp *GELATERIA SALV → SumUp *GELATERIA SALV
/TXT/H|TRANSF NOMI /AIGUA DE RIGAT, S → TRANSF NOMI /AIGUA DE RIGAT, S
/TXT/EL CLANDESTI 27/07/26|20260803 → EL CLANDESTI
/TXT/CONDIS SANT JUST DESV07/07/26|20260714 → CONDIS SANT JUST DESV
/TXT/RECIBO MES TARJETA|20260806 → RECIBO MES TARJETA
/TXT/D|#RECOBRO RECIBO VISA → RECOBRO RECIBO VISA
/TXT/OPENAI *CHATGPT SUBSCR MP → OPENAI *CHATGPT SUBSCR MP
rule …080: {"in":["RECIBO VISA CLASICA",{"var":"description"}]}
8 transaction(s) and 2 automation rule(s) would be reformatted.
===== SECOND RUN =====
0 transaction(s) and 0 automation rule(s) reformatted. ← idempotent
```
The raw text is kept in `original_description`; an already-stored
original is never overwritten; untagged descriptions and rules are left
alone; transactions the server cannot read (`description_iv`) are
skipped.
## Not in this PR
**Already-categorized transactions keep their category.** The backfill
fixes the text, not the past AI decisions — 4005 of the affected rows
already have a category (1024 of them AI-sourced), and
`ai:categorize-backfill` only picks up uncategorized ones.
Rule-categorized rows are unaffected because the rules are migrated. If
we want the AI ones re-run, that's a separate deliberate step: clear
`category_source = 'ai'` on the affected rows, then run the existing
backfill command.
## Rollout
```bash
php artisan banking:backfill-descriptions --dry-run -v # inspect
php artisan banking:backfill-descriptions # apply
```
## Problem
A user reported the same bank transactions appearing twice.
Investigation in prod showed this is a systematic dedup bug in the
EnableBanking sync affecting a few hundred rows across several
users/accounts.
## Root cause
The affected banks don't return a real `transaction_id`; their only id
is a **positional** `entry_reference` of the form
`{booking_date}.{index}` (e.g. `YYYY-MM-DD.0`). That field is **null the
day a transaction first appears** and only **populated on a later
sync**.
`TransactionFingerprint::for()` preferred `entry_reference` when present
and fell back to a content hash when absent. So the same transaction
produced two different fingerprints:
| | First sync (same day) | Later sync |
|---|---|---|
| `entry_reference` | `null` | `YYYY-MM-DD.0` |
| `dedup_fingerprint` | content-based | id-based |
| `external_transaction_id` | `null` | `YYYY-MM-DD.0` |
Neither the fingerprint nor the `external_transaction_id` dedup path
matched across the two syncs → duplicate row. The one-shot historical
first sync has no duplicates; the duplicates start with the daily
incremental syncs.
## Fix
Treat a positional (`^\d{4}-\d{2}-\d{2}\.\d+$`) `entry_reference` as "no
stable id" and fall through to the content hash, which is identical on
both syncs. Genuine `transaction_id` and non-positional
`entry_reference` keep keying exactly as before.
## Trade-off (accepted, follow-up tracked)
The positional index is also the only field that distinguishes two
genuinely-distinct same-day transactions with byte-identical content.
Falling to the content hash collapses them to one fingerprint, so only
the first is kept — a rare silent under-count. We accept it here over
the systematic duplication it fixes; **existing rows are unaffected**
(their distinct positional value is still stored in
`external_transaction_id` and caught by the fallback dedup path on
re-sync). Fixing both cases needs occurrence-aware dedup in the consumer
(a schema change) — left as a follow-up and documented in the code.
## Tests
- Regression: a positional `entry_reference` matches the same
transaction seen earlier without one.
- Boundary: non-positional references still key on `entry_reference`.
- `php artisan test
tests/Unit/Services/Banking/TransactionFingerprintTest.php` → 6 passed.
`pint` clean.
## Data cleanup (separate, after deploy)
The already-duplicated rows still need a one-off cleanup (soft-delete
the later copy per group, keeping the content-fingerprint original). It
must run **after** this fix ships, otherwise the next daily sync
re-creates them. Not included in this PR.
## Summary
Wave 2 structural refactor: the same financial math was copy-pasted
across the
dashboard and the analytics API endpoints, so it could silently diverge
between
screens — a real risk in a finance app. This consolidates the duplicated
calculations into single sources of truth, kills a net-worth N+1, and
aligns
the PHP/TS rule engines and the `TransactionSource` enum. Every change
is
**behavior-preserving**; the numeric outputs of every endpoint are
unchanged and
are locked down with characterization/parity tests.
Builds on merged #640 (Wave 1); no file overlap. No dependencies
changed.
## Changes (per commit)
- **Consolidate savings-rate math into `CashflowSummaryService`** —
`savings_rate`
and `net` were byte-identical inline in `DashboardController` and
`Api/CashflowAnalyticsController`. Extracted to
`CashflowSummaryService::summarize(income, expense)`;
both controllers now spread its result (same keys, same values).
- **Move income/expense-side classification onto the `Transaction`
model** —
the income/expense side test was reimplemented in three places
(`Api/TransactionAnalysisController`, `Api/CashflowAnalyticsController`,
`DashboardController`). Now `Transaction::isIncomeSide()` /
`isExpenseSide()`.
- **Extract duplicated `getCategorySpending` into
`CategorySpendingService`** —
the tree-rollup expense-spending query was duplicated verbatim between
`DashboardController` and `Api/DashboardAnalyticsController`. Moved to
`CategorySpendingService::forPeriod()` (drill-parent parameterized).
- **Batch net-worth balance lookups to kill the per-account N+1** —
`Api/DashboardAnalyticsController::calculateNetWorthAt` ran one
`AccountBalance` query per account per compared period. Now uses the
existing
`BalanceLookup::forAccounts()` (fixed 3 queries via carry-forward seed +
in-range records), reproducing the exact "latest balance <= date, else
0"
semantics.
- **Align server rule normalization with the client and lock it with
parity
fixtures** — `AutomationRuleService::normalizeRuleJson` protected only
`['description','notes']` while `rule-engine.ts` also protected
`creditor_name`/`debtor_name`. Aligned to the superset (structurally a
no-op
since those var names are already lowercase, so no matching change) and
added
shared PHP+TS parity fixtures so the two engines can never drift
unnoticed.
- **Add missing `TransactionSource` cases to the TS type** —
`transaction.ts`
was missing `enablebanking`/`wise`; now mirrors
`App\Enums\TransactionSource`.
- **Guard `sumTransactions` against unsupported category types**
(reviewer fix) —
replaced the income/expense ternary that silently treated any non-Income
type
as expense with a `match` that throws on Savings/Investment/Transfer.
- **Document category eager-load expectation on `Transaction` side
methods**
(reviewer fix) — doc-only note to prevent a future N+1.
## Test plan
New tests:
- `tests/Unit/Services/CashflowSummaryServiceTest.php` —
net/savings-rate/rounding/div-by-zero.
- `tests/Feature/TransactionSideClassificationTest.php` — income/expense
side across signs, uncategorized, and transfer/savings/investment =
neither side.
- `tests/Feature/DashboardAnalyticsTest.php` — new net-worth test
asserts both the values (600000 / 540000) and a flat balance-query count
(<= 3) regardless of account count.
- `tests/Feature/RuleEngineParityTest.php` +
`resources/js/lib/rule-engine-parity.test.ts` +
`tests/Fixtures/rule-engine-parity.json` — one shared fixture set
driving both the PHP and TS rule engines.
Results (targeted, local):
- `--filter=Cashflow` (exclude Browser): 57/57 passed
- `--filter=DashboardAnalytics`: 41/41 passed
- `--filter=AutomationRule`: 60/60 passed
-
`--filter=TransactionSideClassification|CashflowSummaryService|RuleEngineParity`:
21/21 passed
- `bun run test rule-engine` (vitest): 13/13 passed
- `vendor/bin/pint --test`: pass; `bun run lint`: 0 errors; `bun run
format:check`: clean
- `bun run types`: 157 errors (unchanged pre-existing baseline), 0 in
touched files
Note: the 6 `Cashflow*` Browser tests fail locally only on "Vite
manifest not found" (no build present); they are environmental, not
logic, and pass in CI.
## Reviewer findings
Two read-only reviewers (architecture/quality and product/behavior)
reviewed the diff.
**Addressed**
- Both flagged that `sumTransactions` silently treated any non-Income
type as expense — added a throwing `match` guard.
- Eager-load expectation documented on the `Transaction` side methods.
**Verified identical** (behavior reviewer):
income/expense/net/savings_rate across all three endpoints; net worth
for both compared dates including no-record / all-records-after-range /
same-date edge cases; category spending (uncategorized excluded,
soft-deleted categories excluded, rollup/drill preserved); rule-engine
normalization output; multi-currency conversion.
**Deferred (documented)**
- The income/expense **summation** itself is still computed three ways
with differing uncategorized-transaction handling (dashboard
`whereExists` + sign vs analytics `join` excluding uncategorized vs
in-memory `isIncomeSide`). Unifying it would change numbers, so it is
out of scope for this behavior-preserving PR — worth a dedicated
follow-up.
- `savings_rate` keeps its `int|float` union (int `0` when income is 0).
Intentionally preserved to keep JSON output byte-identical.
- `BalanceLookup`'s `empty()` guard never short-circuits a `Collection`,
so a zero-account user runs 3 empty (harmless) queries. Left untouched —
it lives in a shared, unchanged service and only affects a no-account
edge case.
Do not merge before Wave 1 (#640) is in main.
## Summary
- add creditor/debtor fields to transactions with raw_data backfill
- store counterparties on bank sync and CSV/XLS imports
- add creditor/debtor filters plus hidden table columns
## Tests
- php artisan test --compact tests/Feature/LocalizationTest.php
tests/Feature/OpenBanking/TransactionSyncServiceTest.php
tests/Feature/TransactionFilterTest.php
- npm test -- resources/js/lib/file-parser.test.ts --run
- vendor/bin/pint --dirty --format agent
Note: `npm run types` still has pre-existing unrelated errors; no
creditor/debtor/import-related errors remained in filtered output.
## Screenshot
<img width="1241" height="676" alt="8odYWtFcvUM"
src="https://github.com/user-attachments/assets/55653485-d588-4beb-9e6a-5c7c81ba7cf8"
/>
## Problem
Production user reported "inaccurate expenses, some appear multiple
times". DB inspection of their active BNP Paribas Fortis connection
confirmed duplicates growing every sync:
- `-3840` on `2026-05-11` x5
- `-2900` on `2026-05-08` x5
- `-2470` on `2026-05-12` x4
- 56 of 776 rows on the account had `external_transaction_id IS NULL`
- 16 (date, amount) duplicate groups, all with NULL upstream id
## Root cause
`TransactionSyncService::importTransaction()` short-circuited dedup when
both `transaction_id` and `entry_reference` were missing:
```php
$externalId = $data['transaction_id'] ?? $data['entry_reference'] ?? null;
if ($externalId) {
// dedup check
}
// else: fall through, always insert
```
BNP returns no stable id for certain card transactions (`status:
"OTHR"`, `bank_transaction_code.code: "CCRD"`, foreign currency). Every
cron tick (every 6h) re-inserts a fresh copy.
Confirmed with prod data: of 16 NULL-id duplicate groups on this user,
**zero** ever got upgraded to a real id later. BNP simply doesn't issue
one.
## Fix
Deterministic per-transaction fingerprint, persisted in a new column,
protected by a unique index.
- **Migration**: adds `transactions.dedup_fingerprint` (nullable string,
80) and unique index on `(account_id, dedup_fingerprint)`. The unique
index is the real source of truth — it also closes the race between
overlapping sync runs that the prior `exists()` + `create()` pattern
couldn't.
- **`TransactionFingerprint::for($data)`**: two-mode fingerprint. If
`transaction_id` or `entry_reference` exists, the fingerprint is based
only on that canonical upstream id. If no upstream id exists, the
fallback fingerprint uses the prod-verified stable fields:
`booking_date`, amount, currency, credit/debit indicator,
creditor/debtor names + accounts, bank tx codes, reference number, and
remittance info. It intentionally excludes volatile fields (`status`,
`value_date`, raw `transaction_date`). Prefix `fp_` avoids mixing with
bank-issued ids.
- **`TransactionSyncService`**:
- Always computes the fingerprint and writes it on every insert.
- Dedup lookup checks the fingerprint **and** (as a fallback) the legacy
`external_transaction_id` to gracefully handle rows imported before the
backfill runs.
- Wraps the insert in a `try/catch UniqueConstraintViolationException`
so concurrent syncs that pass `exists()` together don't crash.
## Rollout plan
Ship migration + service change → new duplicates stop. Existing
duplicate cleanup is intentionally out of scope for this PR.
## Tradeoff
Two genuinely distinct same-day, same-amount card transactions from the
same merchant on the same card collapse into one (no time-of-day in
raw_data for this BNP class). Today we over-count by 3–5x; after the fix
we may rarely under-count by 1. Acceptable net win, can monitor via logs
if needed.
## Tests
- `tests/Unit/Services/Banking/TransactionFingerprintTest.php` — 4 new
tests covering canonical id behavior and volatile-field exclusion.
- `tests/Feature/OpenBanking/TransactionSyncServiceTest.php` — 3 new
tests:
- Dedupes payloads without an upstream id across consecutive syncs.
- Doesn't crash when a payload arrives later with an upstream id
(bounded behavior).
- Dedupes against soft-deleted fingerprinted rows.
- Targeted suite green locally: **17 passed**.
## Files
-
`database/migrations/2026_05_13_085027_add_dedup_fingerprint_to_transactions_table.php`
*(new)*
- `app/Services/Banking/TransactionFingerprint.php` *(new)*
- `app/Services/Banking/TransactionSyncService.php`
- `app/Models/Transaction.php` (fillable)
- Tests above
## Out of scope
A smaller secondary pattern exists where BNP emitted distinct
`transaction_id`s for the same booking (Feb–Apr only, ~10 groups on the
reporter). Not addressed here because: (a) fingerprint includes
`transaction_id` so different ids = different fingerprints, (b) no
recent occurrences, (c) needs API-level analysis to determine when this
is genuine vs noise.
## Summary
- Adds loan detail tracking (interest rate, term, start date, original
amount) to loan-type accounts, following the existing
`real_estate_details` pattern
- Implements `LoanAmortizationService` with standard amortization math
to automatically project month-to-month balance evolution
- Shows projected future balances as a dashed line on the account
balance chart, visually distinct from historical data
- Adds a scheduled command (`loans:generate-balances`) to auto-generate
monthly balance entries for loan accounts
- Includes 36 tests (10 unit for pure math, 26 feature for CRUD, API
projections, command, and authorization)
## Changes
### Backend
- **Migration**: `loan_details` table (UUID PK, unique `account_id` FK,
interest rate, term, start date, original amount)
- **LoanDetail model** + factory with `Account::loanDetail()` HasOne
relationship
- **LoanAmortizationService**: `calculateMonthlyPayment`,
`calculateRemainingBalance`, `generateProjection`, `projectFromBalance`,
`calculateRemainingMonths`, `getBalanceAtDate`
- **GenerateMonthlyLoanBalances** command: runs monthly on 1st, skips
existing entries
- **LoanDetailController**: PATCH endpoint for editing loan details from
show page
- **Settings\AccountController**: creates/updates `LoanDetail` on
store/update
- **DashboardAnalyticsController**: appends projected data points with
`projected: true` flag
- **Validation**: loan-specific rules in StoreAccountRequest and
UpdateAccountRequest
### Frontend
- **AccountForm**: loan fields (rate, term, start date, original amount)
shown when `type === 'loan'`
- **Create/Edit dialogs**: pass loan data fields in POST/PATCH payloads
- **Show page**: `LoanDetailsCard` component with edit/read modes,
computed monthly payment and remaining months
- **Balance chart**: new `ComposedChart` branch with dashed `Line`
overlay for projected data points
- **Types**: `LoanDetail` interface in `account.ts`
## Summary
- Adds a `BankFormatter` interface with a `BbvaFormatter` implementation
that transforms ALL CAPS `//`-separated BBVA descriptions into readable
Title Case (preserving acronyms like SEPA, S.A., BIZUM and Spanish
stopwords)
- Stores the raw description in a new `original_description` column when
formatting changes it, keeping sync matching intact
- Existing non-BBVA banks and null-bank transactions pass through
unchanged
## Test plan
- [x] Unit tests for `BbvaFormatter` (11 tests: title case, acronyms,
stopwords, reference numbers, mixed-case passthrough, whitespace)
- [x] Unit tests for `TransactionDescriptionFormatter` dispatcher (4
tests: BBVA match, no-match, null bank)
- [x] Feature tests for BBVA sync integration (formatted description +
original stored)
- [x] Feature tests for non-BBVA sync (description unchanged, original
null)
- [x] All 25 existing + new tests pass
- [x] Pint formatting clean
Víctor Falcón