Commit Graph

107 Commits

Author SHA1 Message Date
Víctor Falcón 9312d24f93
fix(balances): propagate retroactive transaction changes to later balances (#682)
## Problem

When a transaction is created (or deleted) with the **update balance**
option on
a **past date**, only that day's balance snapshot was adjusted. Balances
are
stored as sparse per-date snapshots read with carry-forward semantics
(`BalanceLookup::getBalanceAt` returns the most recent snapshot
on/before a
date), so later snapshots — most importantly *today's current balance* —
kept
their stale value.

Reproduction: yesterday = 10, today = 10. Add a −5 expense dated
yesterday with
"update balance" on. Before this fix, yesterday became 5 but today
stayed 10.
Expected: both become 5.

## Fix

`ManualBalanceAdjuster` now shifts the transaction's own day **and every
later
snapshot** by the same delta, on both create and delete:

- `applyCreatedTransaction` seeds a snapshot on the transaction's date
from the
carried-forward balance (when none exists yet), then increments that
date and
  all later snapshots by `+amount`.
- `reverseDeletedTransaction` now applies the exact inverse from the
transaction's own date forward (previously it only ever adjusted
*today*,
which left the past day stale and would corrupt a create→delete
round-trip of
  a back-dated transaction).

The shared forward-shift is extracted into `shiftBalancesFrom(delta)`,
used by
both paths.

## Also in this PR (small UI fix)

The balance history modal (`BalancesModal`) rendered each row's
date/balance
text top-aligned while the action icons were centered, because the base
`TableCell` defaults to `align-top`. Center the data-row cells so every
column
lines up. Scoped to this modal — the shared `TableCell` is untouched.

## Notes / intended semantics

- **Later user-set balances shift too.** Snapshots are treated as points
on one
running balance, so backfilling/removing a past transaction moves
today's
balance as well. That is exactly the reported expectation ("today should
also
change"). We deliberately do **not** try to distinguish "anchor"
snapshots
(a value the user reconciled by hand) from transaction-derived ones —
there is
no such distinction in the schema, and adding one is a separate product
  decision.
- **Pre-existing, out of scope:** editing a transaction's amount/date
never
  adjusts balances (`amount`/`transaction_date` aren't editable via
`UpdateTransactionRequest`), and the create vs. delete "update balance"
toggles are independent — creating with the box off but deleting with it
on
  can over-correct. Neither is introduced here.

## Tests

- New: creating a past-dated transaction updates that date **and** every
later
  balance (the exact reported bug).
- Rewrote the delete test to assert the reverse propagates from the
transaction's date forward (the old test encoded the now-removed "always
write
  today" behavior).
- Full `TransactionTest` suite green (49 passed).

## QA

Verified end-to-end in the running app on a manual account (yesterday =
10.00,
today = 10.00): created a −5.00 expense dated yesterday with "update
balance"
on. The balance history modal shows **both** records — yesterday and
today —
drop from 10.00 to 5.00, and the DB confirms both snapshots = 500.

## Demo

https://github.com/user-attachments/assets/18aa043d-37aa-4073-990c-a269c982a818
2026-07-16 08:38:02 +02:00
Víctor Falcón 3db03de86d
fix(stats): correct the trial/pricing experiment funnel report (#679)
## Summary

Audited `stats:experiment-funnel` end-to-end (data acquisition, queries,
per-row counting, financial math, and statistics) and fixed the issues
that made it misleading for the win/no-win decision. The report used to
rank variants by an **absolute** contribution-margin total that
mechanically favoured whichever variant matured fastest, and it declared
statistical significance with a normal approximation that is invalid at
the small conversion counts this experiment has.

## What changed

**Presentation & comparability**
- Replace `A2P%` (numerator not a subset of the denominator → could
exceed 100%) with `Conv%` = conversions ÷ matured-assigned — always
≤100% and comparable across variants.
- Print `MatU` (matured cohort size) so the rate denominators are
visible and the table reconciles; revive `ARPU`.
- Reframe the guidance: compare on per-user `Conv%`/`ARPU`, not the
absolute `MRR`/`Cost`/`Burn`/`CM` totals (which scale with `MatU`).

**Data correctness**
- Count soft-deleted users (`withTrashed`) — they were assigned a
variant and their connections incurred real cost.
- Attribute by the deterministic `SubscriptionExperiment::bucket()`
instead of the resolved Pennant flag, so setting `force_variant` (the
winner rollout switch) no longer collapses the whole report onto one
variant. Also stops writing Pennant rows as a side effect.
- Resolve `MRR` for subscriptions on rotated/archived Stripe price ids
(fetch prices by product), warn on any net-active sub whose price is
unmapped, round yearly ÷ 12, and skip foreign-currency prices.
- `Burn` counts only users who never earned net revenue (no
subscription, or paid-then-refunded) — a paid-then-churned user is no
longer booked as connect-and-leave leak.

**Statistics**
- Measure conversion as "ever charged, net of refund" (time-invariant)
instead of a live active-now snapshot that biases older cohorts (which
have had longer to churn).
- Decide significance with **Fisher's exact test** (exact at any sample
size), Bonferroni-corrected over the three arms, instead of the
normal-approx z that overstates evidence when expected cell counts fall
below 5. Add a Newcombe difference-of-proportions CI and a small-sample
caveat.
- Extract the inference into `App\Services\Stats\ProportionSignificance`
(+ a `BinomialProportion` value object), with unit tests that pin the
exact interval and p-value numbers.

## Validation

Reconstructed every column in raw SQL against a production dump — all
reconcile **to the cent / to the row**. Independent recomputation of the
statistics (Wilson, Fisher exact, Newcombe, z) matches the command's
output.

## Testing

26 tests (20 feature + 6 unit, 94 assertions). `pint` and
`phpstan`/larastan green.

## Note

This branch also carries two small pre-existing commits unrelated to the
funnel (`fix(categories): fall back to gray…`, `chore(schedule): stop
scheduling the stuck cohort report`). Happy to split them into their own
PR if preferred.
2026-07-15 09:17:32 +02:00
Víctor Falcón d7963736d1
fix(banking): treat EnableBanking upstream 5xx as transient, not reportable (#678)
## Problem

`EnableBankingProvider::getTransactions` and `getBalances` classify a
set of expected `RequestException`s (401 expired session, 400
inaccessible account, 422 wrong period, 400 ASPSP_ERROR) but let
**everything else** fall through to a raw `throw $e`. An upstream **500
`Internal server error`** — from EnableBanking itself or the ASPSP
behind it — therefore propagates as a plain `RequestException` and gets
`report()`ed as an app error.

**Sentry:** `PHP-LARAVEL-3J` — `Illuminate\Http\Client\RequestException:
HTTP request returned status code 500` in
`EnableBankingProvider::getTransactions`, **36 events / 3 users** since
2026-06-16 (regressed 2026-07-15). Nothing per-event our code can do
about the bank's server erroring.

## Fix

A 5xx is a transient server-side failure — the same class as a
`ConnectionException` (no response), which is **already** wrapped as
`TransientBankingProviderException` (logged at `warning`, retried,
self-healing, `ShouldntReport`). Classify any upstream 5xx the same way
in both `getTransactions` and `getBalances`, via a small
`isTransientServerError()` helper placed alongside the other
classifiers:

```php
private function isTransientServerError(RequestException $e): bool
{
    return $e->response->status() >= 500;
}
```

So provider outages retry/self-heal instead of paging Sentry.

## Not changed

All non-5xx paths are untouched and stay reportable — 401/400/422/ASPSP
keep their dedicated exceptions, and genuine 4xx client errors (e.g. the
existing "keeps non-ASPSP client errors reportable" / "keeps unrelated
422 validation errors reportable" tests) still surface.

## Test

Added two cases to `EnableBankingProviderTest.php` (an upstream 500 on
`getTransactions` and on `getBalances` →
`TransientBankingProviderException` / `ShouldntReport`). Full provider
suite: **14/14 green** locally.

🤖 Found and fixed autonomously via the Sentry monitoring loop.
2026-07-15 06:40:55 +00:00
Víctor Falcón cd918523e8
fix(banking): stop duplicating EnableBanking transactions with positional entry_reference (#669)
## Problem

A user reported the same bank transactions appearing twice.
Investigation in prod showed this is a systematic dedup bug in the
EnableBanking sync affecting a few hundred rows across several
users/accounts.

## Root cause

The affected banks don't return a real `transaction_id`; their only id
is a **positional** `entry_reference` of the form
`{booking_date}.{index}` (e.g. `YYYY-MM-DD.0`). That field is **null the
day a transaction first appears** and only **populated on a later
sync**.

`TransactionFingerprint::for()` preferred `entry_reference` when present
and fell back to a content hash when absent. So the same transaction
produced two different fingerprints:

| | First sync (same day) | Later sync |
|---|---|---|
| `entry_reference` | `null` | `YYYY-MM-DD.0` |
| `dedup_fingerprint` | content-based | id-based |
| `external_transaction_id` | `null` | `YYYY-MM-DD.0` |

Neither the fingerprint nor the `external_transaction_id` dedup path
matched across the two syncs → duplicate row. The one-shot historical
first sync has no duplicates; the duplicates start with the daily
incremental syncs.

## Fix

Treat a positional (`^\d{4}-\d{2}-\d{2}\.\d+$`) `entry_reference` as "no
stable id" and fall through to the content hash, which is identical on
both syncs. Genuine `transaction_id` and non-positional
`entry_reference` keep keying exactly as before.

## Trade-off (accepted, follow-up tracked)

The positional index is also the only field that distinguishes two
genuinely-distinct same-day transactions with byte-identical content.
Falling to the content hash collapses them to one fingerprint, so only
the first is kept — a rare silent under-count. We accept it here over
the systematic duplication it fixes; **existing rows are unaffected**
(their distinct positional value is still stored in
`external_transaction_id` and caught by the fallback dedup path on
re-sync). Fixing both cases needs occurrence-aware dedup in the consumer
(a schema change) — left as a follow-up and documented in the code.

## Tests

- Regression: a positional `entry_reference` matches the same
transaction seen earlier without one.
- Boundary: non-positional references still key on `entry_reference`.
- `php artisan test
tests/Unit/Services/Banking/TransactionFingerprintTest.php` → 6 passed.
`pint` clean.

## Data cleanup (separate, after deploy)

The already-duplicated rows still need a one-off cleanup (soft-delete
the later copy per group, keeping the content-fingerprint original). It
must run **after** this fix ships, otherwise the next daily sync
re-creates them. Not included in this PR.
2026-07-12 16:25:13 +00:00
Víctor Falcón dada23cd84
feat(stats): add unit-economics funnel + connection cost to experiment report (#666)
## What

Reworks `stats:experiment-funnel` from a pure conversion funnel into a
**contribution-margin** one, so the 3-way trial/pricing experiment can
be judged on margin — not just conversion. Signups aren't free: each
bank connection costs money per month, so trials that connect banks and
never pay are burned cash, which the old report was blind to.

## Funnel

Per variant: **assigned → activated → carded → net-paying**

- **activated** = connected ≥1 bank connection **OR** gave AI consent
(triggered paid infrastructure), whether or not they paid.
- **carded** = completed Stripe Checkout (card on file).
- **net-paying** = live, non-refunded, mature subscription.

The **activated → carded** gap is where a user connects a bank and walks
away without paying — the exact leak the report now puts a number on.

## Metrics

| Column | Meaning |
|---|---|
| `A2P%` | net-paying ÷ activated (mature) |
| `Cost` | mature-cohort connections × cost/connection |
| `Burn` | connection cost of mature non-payers (money lost) |
| `CM` | MRR − Cost (the decision metric) |

- New `--cost-per-connection` option, default **0.40** per connection.
- Connection count includes soft-deleted/revoked connections (they still
cost money).
- All money/rate columns are gated on each variant's maturity window;
`Cost`/`Burn`/`CM` print `—` until a variant has mature volume. Legend
notes that raw `Assg`/`Actd`/`Card` are lifetime counts while
rates/money are mature-cohort only.
- Existing MRR/ARPU/Net% fields stay on the collector; dropped from the
printed table.

## Tests

Added coverage for activation (bank OR AI),
cost/burn/contribution-margin math, and the cost-per-connection
argument. Full file green (12 passed).
2026-07-10 18:06:01 +00:00
Víctor Falcón 815ca6244c
feat(stats): surface trials scheduled to cancel in experiment funnel (#665)
## Why

In the experiment funnel report, variants with a trial (control,
reduced) show `0` under Actv/Cncl/Rfnd while their signups are still
mid-trial — so the table looked empty even though many of those trials
have already been canceled by the user (Cashier keeps serving the trial
until it ends, then simply doesn't charge). That "already lost, just not
settled yet" cohort was invisible.

On prod right now: 28 subscriptions in `trialing`, of which **10 are
already scheduled to cancel** — a leading churn signal the report was
hiding.

## What

- `ExperimentFunnelCollector`: new per-variant `trialingCanceling`
counter = `stripe_status === 'trialing'` **and** `ends_at !== null`.
- Report table: two new columns — `Trl` (currently in trial, previously
collected but never printed) and `TrlX` (of those, already scheduled to
cancel and won't convert). Discord legend updated.
- Test covering the new counter.

`Trl`/`TrlX` are orthogonal leading indicators; the mature Net%/MRR/ARPU
metrics are unchanged.

## Testing

`php artisan test
tests/Feature/SendExperimentFunnelReportCommandTest.php` — 9 passed.
2026-07-10 19:22:58 +02:00
Víctor Falcón 6f72c43cce
feat(spaces): phase 0 — multi-tenant Space foundation (no behaviour change) (#650)
## Spaces / Business plan — Phase 0: invisible foundation

First of **three stacked PRs** introducing multi-tenant **Spaces** (the
basis for the Business plan). This one is a **pure, behaviour-preserving
foundation**: it can ship to production on its own with zero
user-visible change.

- **Stacked PRs:** this → `enterprise-spaces-ui` (Phase 1+2) →
`enterprise-spaces-invitations` (Phase 3).

### What a Space is
A Space groups its own accounts, connections, transactions, categories,
labels, budgets and rules. **Every user gets one invisible "Personal"
space**, provisioned automatically on creation — so the architecture is
identical for free, Standard and Business accounts, even though only
Business will ever see more than one.

### What this PR does (no behaviour change)
- `spaces`, `space_user`, `space_invitations` tables;
`users.current_space_id`; a nullable, indexed `space_id` on the 8 owned
tables (plain column, **no FK** — avoids a validating table-scan/lock on
`transactions` during a phased rollout).
- `Space` model + `BelongsToSpace` trait that stamps `space_id` on
create (from the row's user's current space; a transaction inherits its
account's space, so bank-sync lands rows correctly).
- Idempotent, chunked `spaces:backfill` command (run from a migration)
that gives every existing user a personal space and stamps their rows —
so the read switch in the next PR is safe.
- **Reads are untouched here** (still user-scoped); every user has
exactly one space, so behaviour is identical.

### Testing
- New `tests/Feature/Spaces/SpaceFoundationTest.php` (provisioning,
default-space stamping, account-anchored transaction space,
stale-pointer self-heal, backfill).
- Full suite green.

### Notes / deliberate simplifications
- `space_id` stays **nullable** (populated by backfill + on every
write); the NOT NULL constraint is deferred until prod is confirmed
fully backfilled.
- For very large `transactions` tables, `spaces:backfill` can be run
out-of-band before deploy so the migration's call is a no-op.
2026-07-09 14:26:07 +02:00
Víctor Falcón 08d367a5c8
fix(balances): stop historical-balance generator OOM on ancient purchase/loan dates (PHP-LARAVEL-49) (#661)
## What & why

A queued historical-balance generator job exhausted the worker's 128MB
PHP memory limit and **died on every retry** — Sentry `PHP-LARAVEL-49` +
`PHP-LARAVEL-4A` (two fingerprints of the *same* poison message:
identical `trace_id`/`message.id`, `retry.count` 1 then 2, OOM at
slightly different `Arr.php` lines).

### Root cause (trace-confirmed)
The Sentry trace localized the OOMing message to
`App\Jobs\GenerateHistoricalRealEstateBalancesJob`, dispatched from
`Settings\AccountController::store` for the "older than 12 months" slice
of history. The fatal frame is `Arr::map` inside Eloquent's
`AccountBalance::upsert()` value-prep (`addUniqueIds`/`addTimestamps`
per-row `array_merge`, `Query\Builder::upsert` `Arr::flatten` bindings,
and a compiled multi-row SQL string — all **O(n)** over the rows).

`purchase_date` (`before_or_equal:today`) and `loan_start_date` (no
constraint) had **no lower bound**, so a mistyped/ancient year (e.g.
`0201`, `1080`) made the generator build a **multi-century monthly
series** and hand it all to a single `upsert`, blowing past 128MB.

## The fix (two commits)

1. **`fix(balances): upsert historical balances in batches`** — build +
upsert in batches of 500 (`UPSERT_CHUNK_SIZE`) in both
`RealEstateBalanceGeneratorService` and `LoanBalanceGeneratorService`,
instead of one giant operation. Same rows, same `(account_id,
balance_date)` conflict target — peak memory is now bounded regardless
of series length. Defense-in-depth.
2. **`fix(accounts): floor purchase/loan start dates at 1900`** — the
actual root cause: add `after_or_equal:1900-01-01` to `purchase_date`
and `loan_start_date`. Rejects the data-entry typos that drive the
runaway series (and the resulting DB bloat / misleading multi-century
net-worth ramp) while accepting any legitimately old asset in a
personal-finance app.

## Review (two agents, before this PR)
Both reviewed against the live trace and the code.

- **Root cause & fix confirmed** — the batching caps exactly the
`Arr::map`-over-N-rows frame from the trace; no independent OOM source
exists (no per-row model events; `upsert` bypasses events).
- **Chunking is correct** — the unique index `(account_id,
balance_date)` exists, so the conflict target is real; `buildDateList`
yields a strictly-increasing de-duplicated list, so batches are disjoint
(no dropped/duplicated dates, no off-by-one). The fresh per-row `id`
UUID is discarded on conflict → idempotent retries.
- **No material user-facing regression.** Balance consumption
(dashboard/net-worth) is date-range-clamped. Batching introduces a
theoretical partial-write-on-mid-job-failure window, but it's idempotent
and self-heals on retry — agents advised **against** wrapping it in a
transaction (holds locks, doesn't help memory).

Applied recommendation #2 (the date floor) as its own commit. Deferred
(both agents agree): extracting the two near-identical generators into a
shared base/trait — a larger refactor that shouldn't ride a hotfix.

## Tests
- Regression test in each service: generate a ~46-year (>500-point)
series and assert the batched writes produce no dropped/duplicated dates
and keep endpoints anchored (crosses multiple batches).
- Validation test: a pre-1900 `purchase_date` is rejected at
`accounts.store`.

Note: the PHP feature suite boots a MySQL testcontainer (Docker) that
isn't available in my environment, so these were validated by
static/`pint`/`php -l` locally and rely on CI for execution — auto-merge
is gated on green CI.

Fixes PHP-LARAVEL-49
Fixes PHP-LARAVEL-4A
2026-07-08 21:34:43 +00:00
Víctor Falcón d504e70309
fix(ai): don't report expected transient provider overloads (PHP-LARAVEL-44) (#655)
## Issue (Sentry PHP-LARAVEL-44 — low volume: 0 users, 1 event)

`Laravel\Ai\Exceptions\ProviderOverloadedException: AI provider [gemini]
is overloaded` (underlying Gemini HTTP 503 "high demand"), thrown from
the AI rule-suggestion generator. This is an expected, transient,
self-healing provider overload — but
`LaravelAiRuleSuggestionGenerator::generate()` called `report()` on
every failed batch, so a transient hiccup surfaced in Sentry as an
error.

## Fix

Split the per-batch catch so a `FailoverableException` (overload /
rate-limit / insufficient-credits) is logged at `warning` **without**
reporting, while every other `Throwable` is still `report()`ed. This
mirrors the existing, test-enforced handling in the sibling
`CategorizeTransactions` service.

Unchanged (deliberately preserved):
- **Partial tolerance** — suggestions from batches that succeeded are
still kept.
- **Rethrow-when-all-fail** — a genuine total failure still rethrows, so
the run is marked `Failed` (the onboarding "Try again / Skip" UI) and is
reported **once** at the run level. Non-transient batch errors are still
reported immediately.

## What I deliberately did NOT do

Two review agents converged that adding a **retry backoff** (the other
candidate fix) is not worth it here and carries real risk:
- The run **already self-heals**: failed/empty runs don't count toward
the throttle, so the user's "Try again" button re-runs immediately at
zero cost.
- A synchronous per-batch `sleep` on the single-worker `database` queue,
across up to ~10 batches, could push the run past its **120s job timeout
(failing more often, not less)** and starve other queued jobs during a
provider brownout.

So this PR is scoped to the safe, high-signal change: stop reporting an
expected transient as an error. Genuine misconfigurations (bad
model/key/quota) throw non-`FailoverableException` types and remain
fully reported.

## Testing

- New: an expected transient overload returns the successful batches'
suggestions and is **not** reported
(`Exceptions::assertNothingReported()`); an unexpected batch failure
**is** reported (`assertReported(RuntimeException::class)`).
- Full `tests/Feature/Ai` suite green (121 tests); Pint and Larastan
clean.

Fixes PHP-LARAVEL-44

---
🤖 Opened by the autonomous Sentry-triage loop. Auto-merge enabled:
low-risk, mirrors an existing tested pattern, keeps genuine failures
visible.
2026-07-07 10:55:02 +00:00
Víctor Falcón 9b7632f585
fix(banking): recover from EnableBanking 422 wrong-period instead of crashing the sync (PHP-LARAVEL-42) (#653)
## Problem (Sentry PHP-LARAVEL-42)

EnableBanking's `GET /accounts/{id}/transactions` returns **HTTP 422
"Wrong transactions period requested"** when the requested date range is
wider than the bank is willing to serve. The catch-ladder in
`EnableBankingProvider::getTransactions()` only matched
401/EXPIRED_SESSION, 400/AccountNotAccessible and 400/ASPSP_ERROR, so
the 422 rethrew a **raw `RequestException`**. That:

1. escaped the per-account loop in `EnableBankingSyncer::sync` (which
only skips `InaccessibleBankAccountException`), so **every remaining
account in the connection stopped syncing too**;
2. hit the job's generic `catch (\Throwable)` → retried 3×
(deterministic, always the same 422) → connection marked **Error** and
**reported to Sentry**;
3. after the scheduled-retry budget (`consecutive_sync_failures >= 3`)
the whole connection was **dropped from scheduled sync** until a manual
reconnect.

Real user impacted (active connection). The failing request was a
~92-day window on the incremental/linked path.

## Fix (3 commits)

1. **Classify the 422** — new `WrongTransactionsPeriodException`
(`ShouldntReport`) thrown from `getTransactions()` when status is 422
and the message names the period. Also stop logging handled 422s at
`error` level in the HTTP client callback.
2. **Clamp + retry** — on that exception, `TransactionSyncService::sync`
restarts the account from page 1 with a progressively narrower window
(`90 → 30 → 7` days before `date_to`), so the user keeps as much history
as the bank will serve. `strategy='longest'` is dropped on the narrowed
retry so the explicit `date_from` is honoured; re-fetched pages are
idempotent (fingerprint dedup + date-keyed daily balances).
3. **Graceful skip** — if even the narrowest window is refused, the
syncer skips just that account (like an inaccessible account) and keeps
the connection Active, instead of failing the whole sync.

## Why draft (needs a human call, per two review agents)

The crash fix itself is well-covered and safe. What needs sign-off is
the **product tradeoff** the clamp introduces:

- **First-sync history truncation.** First sync requests
`now()->subYear()`. If the bank refuses a year, we narrow to ≤90 days
and there is **no back-fill path** (incremental syncs only move
`date_from` forward), so the skipped history is lost. This is bounded
and logged, but it is a deliberate behaviour change.
- **Incremental catch-up gap.** If the watermark is older than the
bank's servable window, the days between the watermark and the clamp are
never fetched (silent, but logged).
- **Heuristics worth a human eye:** the ladder values `[90, 30, 7]`,
dropping `strategy='longest'` on retry, and detecting the error by
`status 422 + message contains "period"` (ideally confirmed against
EnableBanking docs / a stable error code).

Low-risk per review: duplicate imports (fingerprint + `(account_id,
dedup_fingerprint)` unique index are robust to overlapping windows) and
balances (truncated, not corrupted).

## Testing

- Provider: 422 wrong-period → `WrongTransactionsPeriodException`;
unrelated 422 stays a raw `RequestException`.
- Service: clamps + retries once (asserts the clamped date and the
`strategy` drop); gives up after the ladder is exhausted; does not retry
an already-narrow window.
- Syncer: a refused account is skipped, connection stays
Active/unreported, siblings still sync.
- Full `tests/Feature/OpenBanking` + `tests/Feature/Sync` green (315
tests); Pint and Larastan clean.

Fixes PHP-LARAVEL-42

---
🤖 Opened by the autonomous Sentry-triage loop. Draft on purpose — the
data-truncation tradeoff above is a product decision for a human.
2026-07-07 08:57:23 +02:00
Víctor Falcón 26875bbfff
refactor: consolidate duplicated financial calculations (#643)
## Summary

Wave 2 structural refactor: the same financial math was copy-pasted
across the
dashboard and the analytics API endpoints, so it could silently diverge
between
screens — a real risk in a finance app. This consolidates the duplicated
calculations into single sources of truth, kills a net-worth N+1, and
aligns
the PHP/TS rule engines and the `TransactionSource` enum. Every change
is
**behavior-preserving**; the numeric outputs of every endpoint are
unchanged and
are locked down with characterization/parity tests.

Builds on merged #640 (Wave 1); no file overlap. No dependencies
changed.

## Changes (per commit)

- **Consolidate savings-rate math into `CashflowSummaryService`** —
`savings_rate`
  and `net` were byte-identical inline in `DashboardController` and
`Api/CashflowAnalyticsController`. Extracted to
`CashflowSummaryService::summarize(income, expense)`;
  both controllers now spread its result (same keys, same values).
- **Move income/expense-side classification onto the `Transaction`
model** —
  the income/expense side test was reimplemented in three places
(`Api/TransactionAnalysisController`, `Api/CashflowAnalyticsController`,
`DashboardController`). Now `Transaction::isIncomeSide()` /
`isExpenseSide()`.
- **Extract duplicated `getCategorySpending` into
`CategorySpendingService`** —
  the tree-rollup expense-spending query was duplicated verbatim between
  `DashboardController` and `Api/DashboardAnalyticsController`. Moved to
  `CategorySpendingService::forPeriod()` (drill-parent parameterized).
- **Batch net-worth balance lookups to kill the per-account N+1** —
  `Api/DashboardAnalyticsController::calculateNetWorthAt` ran one
`AccountBalance` query per account per compared period. Now uses the
existing
`BalanceLookup::forAccounts()` (fixed 3 queries via carry-forward seed +
in-range records), reproducing the exact "latest balance <= date, else
0"
  semantics.
- **Align server rule normalization with the client and lock it with
parity
  fixtures** — `AutomationRuleService::normalizeRuleJson` protected only
  `['description','notes']` while `rule-engine.ts` also protected
`creditor_name`/`debtor_name`. Aligned to the superset (structurally a
no-op
since those var names are already lowercase, so no matching change) and
added
shared PHP+TS parity fixtures so the two engines can never drift
unnoticed.
- **Add missing `TransactionSource` cases to the TS type** —
`transaction.ts`
was missing `enablebanking`/`wise`; now mirrors
`App\Enums\TransactionSource`.
- **Guard `sumTransactions` against unsupported category types**
(reviewer fix) —
replaced the income/expense ternary that silently treated any non-Income
type
  as expense with a `match` that throws on Savings/Investment/Transfer.
- **Document category eager-load expectation on `Transaction` side
methods**
  (reviewer fix) — doc-only note to prevent a future N+1.

## Test plan

New tests:
- `tests/Unit/Services/CashflowSummaryServiceTest.php` —
net/savings-rate/rounding/div-by-zero.
- `tests/Feature/TransactionSideClassificationTest.php` — income/expense
side across signs, uncategorized, and transfer/savings/investment =
neither side.
- `tests/Feature/DashboardAnalyticsTest.php` — new net-worth test
asserts both the values (600000 / 540000) and a flat balance-query count
(<= 3) regardless of account count.
- `tests/Feature/RuleEngineParityTest.php` +
`resources/js/lib/rule-engine-parity.test.ts` +
`tests/Fixtures/rule-engine-parity.json` — one shared fixture set
driving both the PHP and TS rule engines.

Results (targeted, local):
- `--filter=Cashflow` (exclude Browser): 57/57 passed
- `--filter=DashboardAnalytics`: 41/41 passed
- `--filter=AutomationRule`: 60/60 passed
-
`--filter=TransactionSideClassification|CashflowSummaryService|RuleEngineParity`:
21/21 passed
- `bun run test rule-engine` (vitest): 13/13 passed
- `vendor/bin/pint --test`: pass; `bun run lint`: 0 errors; `bun run
format:check`: clean
- `bun run types`: 157 errors (unchanged pre-existing baseline), 0 in
touched files

Note: the 6 `Cashflow*` Browser tests fail locally only on "Vite
manifest not found" (no build present); they are environmental, not
logic, and pass in CI.

## Reviewer findings

Two read-only reviewers (architecture/quality and product/behavior)
reviewed the diff.

**Addressed**
- Both flagged that `sumTransactions` silently treated any non-Income
type as expense — added a throwing `match` guard.
- Eager-load expectation documented on the `Transaction` side methods.

**Verified identical** (behavior reviewer):
income/expense/net/savings_rate across all three endpoints; net worth
for both compared dates including no-record / all-records-after-range /
same-date edge cases; category spending (uncategorized excluded,
soft-deleted categories excluded, rollup/drill preserved); rule-engine
normalization output; multi-currency conversion.

**Deferred (documented)**
- The income/expense **summation** itself is still computed three ways
with differing uncategorized-transaction handling (dashboard
`whereExists` + sign vs analytics `join` excluding uncategorized vs
in-memory `isIncomeSide`). Unifying it would change numbers, so it is
out of scope for this behavior-preserving PR — worth a dedicated
follow-up.
- `savings_rate` keeps its `int|float` union (int `0` when income is 0).
Intentionally preserved to keep JSON output byte-identical.
- `BalanceLookup`'s `empty()` guard never short-circuits a `Collection`,
so a zero-account user runs 3 empty (harmless) queries. Left untouched —
it lives in a shared, unchanged service and only affects a no-account
edge case.

Do not merge before Wave 1 (#640) is in main.
2026-07-04 22:26:44 +02:00
Víctor Falcón c159e8782e
fix(ai): surface learned-rule toast in edit modal and guard weak description keys (#635)
## Why

Two issues surfaced from a real mis-categorization corrected via the
**Edit Transaction** modal (not the table quick-edit).

### 1. The modal never told the user the AI learned

Both the table quick-edit and the edit modal hit the same backend
(`PATCH /transactions/{id}` → `CategoryOverrideHandler` →
`AiRuleLearner`), so both learn a forward rule from a correction. But
the modal **discarded** the update response and never read
`learned_rule`:

- No *"Learned: similar transactions will be categorized automatically"*
confirmation.
- No **Undo**.
- It still showed the generic *"Automatize"* prompt — offering to create
a rule that already exists.

The modal now reads `learned_rule` and mirrors the table's toast + undo,
skipping the automatize prompt when a rule was learned. Same i18n
strings as the table (no new keys).

### 2. The learner could key a rule on a single weak token

A correction on `"Pago en MADRID SUC 04 MADRID ES"` produced a rule
keyed on `suc` alone. The tokenizer drops noise by **per-user** document
frequency, so `madrid` (20% of this user's descriptions) is correctly
dropped — but `suc` (0.14%) survives, even though it is a generic
banking abbreviation that matches broadly as a substring. The per-user
filter can't see that `suc` is generic in general.

`AiRuleLearner::descriptionClause()` now refuses a single distinctive
token shorter than 5 characters. It learns only when there are **two or
more** distinctive tokens, or a single token of **at least 5
characters**. Merchant-keyed rules are unaffected.

## Tests

- `does not learn a description rule from a single short token`
- `learns a description rule from a single sufficiently long token`
- `learns a description rule from two short tokens`

All `AiRuleLearnerTest` pass (13/13).

## Not in this PR

The one already-learned prod rule still carries the stale `suc`
clause/title (low impact, matches ~2 txns). Cleaning that data is a
separate manual step.
2026-07-04 11:00:15 +00:00
Víctor Falcón d55c3f41df
fix(security): scope job-status endpoints to owner + feature-area fixes (#627)
## Summary

Starts from a security finding on the background-job status endpoints
and folds in the most important issues surfaced by a follow-up review of
the same feature area (categorization backfill, bulk rule re-evaluation,
automation-rule apply).

Each change is its own commit.

## Changes

### 1. `fix(security)` — scope job-status cache keys to the owning user
The categorization, bulk re-evaluation, and apply status endpoints
looked jobs up by a bare job UUID with no user scoping. Any
authenticated user who obtained another user's job id could poll its
progress payload. Cache keys now include the owning user's id, so a
status request keyed by the polling user's id resolves only that user's
own jobs — a mismatched owner falls through to the existing 404. No
ownership store or extra lookup. Cross-user isolation tests added for
all three endpoints.

### 2. `fix(automation-rules)` — re-check `only_uncategorized` at apply
time
The apply flow cached a match snapshot for up to 15 min (keyed only by
`rule.updated_at`) and applied the rule's category to every id in it
without re-checking eligibility. A transaction categorized *after* the
snapshot (by the user, a sibling rule, or a concurrent AI backfill) was
silently re-categorized and stamped `category_source=Rule`. Now
re-filtered through `shouldSkipForOnlyUncategorized()` at apply time;
skipped rows are no longer counted as changed.

### 3. `perf(automation-rules)` — memoize rule set per user in
`applyRules()`
Bulk re-evaluation calls `applyRules()` once per transaction, and each
call re-queried the user's whole rule set + labels — an N+1 scaling with
transaction count (the sibling apply job already loaded rules once).
Memoized per user for the service instance's lifetime (resolved fresh
per job, so rules created mid-run are intentionally not seen).
Query-count test asserts one `automation_rules` query regardless of
transaction count.

### 4. `fix(transactions)` — stop polling when the job never starts
The re-evaluate and apply pollers rescheduled on any non-terminal
status, including `pending`. If the queue worker is down the job never
runs, `failed()` never fires, and the client polled for the full
hour-long TTL with a stuck spinner. Now gives up after 30 consecutive
`pending` ticks (~30s), mirroring the guard the AI-categorization poller
already has. Long `processing` runs are unaffected.

### 5. `test(automation-rules)` — cover apply job execution and failure
branches
`ApplySingleAutomationRuleJob` was only asserted to be *pushed*; its
`handle()` body and `failed()` branch had no coverage, and
`ReEvaluateTransactionRulesJob::failed()` was untested too. Added direct
`handle()`/`failed()` tests pinning the progress-cache payloads.

## Deferred follow-ups (surfaced by review, not in this PR)

- **Consolidate the three frontend pollers onto `usePollJobStatus`** —
they hand-roll `setTimeout` loops and don't tear down on unmount (can
`setState`/`onClose` after the dialog closes). The hook already exists
and is unmount-safe; routing all three through it would also make the
`pending` cap unit-testable. The cap in change #4 currently mirrors the
already-shipped AI poller and is not separately unit-tested.
- **`noteAlreadyPresent()` uses a substring match**
(`AutomationRuleService`) — a rule note that is a substring of an
existing note is silently not appended. Should compare note lines
exactly.
- **No dedup/lock on concurrent apply/re-evaluate jobs** — note appends
are a non-atomic read-then-write, so two concurrent jobs can duplicate a
note. Category/label writes are already idempotent.
- **Transient 404 → false "failed"** — a mid-run cache
eviction/TTL-expiry (or the deploy window of change #1, where an
in-flight job's pre-deploy key is orphaned) makes the poller surface a
false failure. Retry a few times on transient errors before giving up.
- **`matches()` pagination** advances `next_offset` by fetched rows, not
page-window size — deleted ids can stall infinite scroll before reaching
`total`.
- **"Apply to N" count / encrypted-skip counts** can be stale or
under-report vs what's actually processed.
- **Job-trait convention drift** between the three jobs (modern
`Queueable` vs legacy trait stack).

## Testing
- `vendor/bin/pint --test` — pass
- `bun run format` / `bun run lint` — pass (one pre-existing unrelated
warning in `chart.tsx`)
- Affected Pest suites (apply, re-evaluate, evaluation, rule,
categorization, apply-rule-suggestions) — 94 tests pass
2026-07-03 14:49:32 +02:00
Víctor Falcón 3d3f6daa77
perf(ai): reduce N+1 in bulk category updates (PHP-LARAVEL-40, partial) (#624)
> **Draft on purpose — a partial fix + a design for the full one.**
These commits are safe, behavior-preserving reductions of the N+1, but
they do **not** fully resolve PHP-LARAVEL-40. The complete fix is a
batch-aware refactor of the AI rule-learning path, which is delicate
(getting it wrong silently mis-categorizes users' transactions). I've
written the design below for a human to implement and diff against the
current per-transaction behavior. Current user impact is low (~170 ms
request, 1 event), so there's no urgency to rush the risky part.

## What

Sentry **PHP-LARAVEL-40** — N+1 in `TransactionController@bulkUpdate`
(PATCH `/transactions/bulk`). A bulk category update calls
`CategoryOverrideHandler::record()` once per transaction, and each call
runs the full AI rule-learning pipeline
(`AiRuleLearner::forgetFromAiRules()` + `learnFromCorrection()`):
loading all the user's AI rules, plucking every description, running
matcher `count(*)` probes, and inserting a `category_corrections` row.
For a 112-transaction batch that's ~112× each. The offending span is the
per-transaction `category_corrections` insert.

## What this branch does (safe, partial)

Two behavior-preserving reductions, each validated by the existing
learning tests:

1. `perf(transactions): resolve the override handler once for bulk
updates` — `bulkUpdate()` re-resolved `CategoryOverrideHandler` from the
container every iteration. Resolve it once (also required for #2 to take
effect).
2. `perf(ai): memoize the description corpus per user in AiRuleLearner`
— `learnFromCorrection` reloaded and re-tokenized every one of the
user's descriptions on each transaction. The corpus is immutable while
only categories change, so memoize the document-frequency map + count
per user. Removes one `SELECT` (and its tokenization) per transaction.
3. `test(ai): assert batch corrections learn correctly through the
memoized corpus` — proves the second, memoized-corpus learning still
yields a correct, distinct clause (not just that the query count
dropped).
4. `test(ai): guard AiRuleLearner against a singleton binding` — the
cache has no invalidation and is only safe while the learner is resolved
fresh per request; a test now fails if it is ever bound singleton.

## What it does NOT do

It does **not** resolve the flagged N+1. Still per-transaction: the
`category_corrections` insert, `forgetFromAiRules`' reload of all AI
rules, the matcher `total()`/`countMatchingAll()` overbroad probes,
`releaseClauseFromOtherCorrectionRules`, and `existingCorrectionRule`. A
learnable transaction still costs ~6–10 queries. Treat PHP-LARAVEL-40 as
**reduced, not closed** (hence no `Fixes` keyword).

## Reviewed by two independent agents (architecture +
product/correctness)

Both verified the two perf commits are **behavior-preserving and safe**:
the memo cannot go stale (nothing in the correction path mutates
descriptions; the bulk `update()` only writes
`category_id`/`category_source`/`ai_confidence`/`categorized_by_rule_id`,
and runs after the loop), no cross-user leak (keyed by user_id; learner
is transient, one user per request, no Octane), and the handler is
stateless. The existing 34 learning tests exercise the corpus→rule
computation on fresh loads.

## Proposed full fix (for a human to implement)

Exploit that a bulk update sends **all transactions to the same category
for one user.** Add `CategoryOverrideHandler::recordBulk(Collection,
?string)` backed by `AiRuleLearner::learnFromCorrections(array,
string)`; load user-level data once, mutate in memory, write once:

1. **Batch-resolve** `categorized_by_rule_id` via one `whereIn` instead
of per-txn `find()`; apply the **identical** per-txn learnable test
against the map.
2. **Batch-insert** corrections: snapshot each AI-driven txn's *old*
`from_category_id`/`source`/`confidence` during the loop, collect rows,
one insert. (Note: raw `insert()` skips model events/UUID/timestamps —
verify `CategoryCorrection` has no `creating` hook, else chunked
`create` in one transaction.)
3. **Forget once**: union all learnable txns' merchant tokens, load AI
rules once, apply the **whole union** to each rule *before* the
delete-vs-save decision, save/delete each once.
4. **Learn once**: compute each clause (merchant or memoized-corpus
tokens), dedupe within the batch and against the target rule, run
`releaseClauseFromOtherCorrectionRules` for the union once, load/create
the single target correction rule once, append all, save once.
5. Wrap 2–4 in one `DB::transaction`, and route the single-txn path
through the same method (one-element collection) so the two can't drift.

**Correctness invariants to preserve (call these out in review):**
- Apply-all-then-decide for both `forget` and `releaseClause` deletes —
an incremental forget/save/forget would delete a rule a later txn's
clause should have kept.
- Corrections must read each txn's **old** category/source/confidence —
run before the bulk `update()`, as today.
- Clause dedup must match `appendClause`'s loose `==` array comparison.
- Validate with a golden-set test diffing learned/forgotten rules
before/after against the current per-txn path over a mixed batch
(merchant txns, description txns, encrypted/no-merchant skips,
re-corrections that move a key between categories).

## Testing

- `tests/Feature/Ai/` + `BulkUpdateTransactionsTest` + IDOR/decrypt:
153/153 green. `pint` clean.

Refs PHP-LARAVEL-40
2026-07-03 07:52:19 +02:00
Víctor Falcón 84bad76316
perf(banking): kill per-transaction dedup N+1 in bank sync (PHP-LARAVEL-3Y) (#621)
## What

Fixes the N+1 flagged by Sentry **PHP-LARAVEL-3Y** in
`App\Jobs\SyncBankingConnectionJob`.

`TransactionSyncService::importTransaction()` ran one `exists()` probe
**per incoming bank transaction** to detect duplicates:

```sql
select exists(
  select * from transactions
  where account_id = ?
    and (dedup_fingerprint = ? or external_transaction_id = ?)
) as exists
```

A sync importing N transactions issued N dedup `SELECT`s. As transaction
volume per sync grows, so does the query count.

## How

Preload the account's existing dedup keys **once** at the start of
`sync()` and match against in-memory sets:

- `loadExistingDedupKeys()` streams (`cursor()`) the two dedup columns
for the account — including soft-deleted rows (`withTrashed()`) — into
two keyed sets. One query instead of N.
- `importTransaction()` checks membership in memory (`isset()`), an
exact mirror of the old `fingerprint OR external_id` predicate.
- Keys from successful inserts are folded back into the sets, so
duplicates **within the same sync** (e.g. the same transaction repeated
across pages) are still caught in memory.
- The `(account_id, dedup_fingerprint)` unique index +
`UniqueConstraintViolationException` catch still backstop concurrent
syncs (`SyncBankingConnectionJob` is already `ShouldBeUnique` per
connection).

## Commits

1. `perf(banking): preload dedup keys to kill per-transaction N+1` — the
core fix + a query-count regression test.
2. `perf(banking): stream dedup key preload with a cursor` — avoids
buffering every historical row into a Collection on top of the sets
(memory guard for very large accounts).
3. `fix(banking): match external ids case-insensitively in dedup` — the
old query compared `external_transaction_id` under the production
`utf8mb4_unicode_ci` collation; the in-memory `isset()` is byte-exact.
With no unique index on `external_transaction_id`, a legacy id stored as
`ABC` vs an incoming `abc` would have silently double-imported.
Normalized with `mb_strtolower()` on both sides + a test.
4. `test(banking): cover intra-run cross-page dedup` — new coverage for
the same transaction appearing on two pages of one sync.

## Testing

- 303/303 in `tests/Feature/OpenBanking/` green; full backend suite
green except one **pre-existing, unrelated** failure on `main`
(`DashboardTest` returns 409 locally — an Inertia asset-version artifact
that resolves once `bun run build` runs in CI).
- New tests: dedup runs as a single preload SELECT regardless of batch
size; case-insensitive external-id dedup; cross-page intra-run dedup.
- `pint` clean; `larastan` clean on the changed file.

## Reviewed by two independent agents (architecture +
product/correctness)

Both rated the change behavior-preserving and shippable. Follow-ups they
surfaced, intentionally **not** bundled here to keep this fix focused
and low-risk:

- **`WiseTransactionSyncService` has the identical latent N+1**
(`:149-158`). It is not currently firing in Sentry (lower volume). Worth
a separate PR — possibly extracting a shared dedup-set helper once both
call sites need it.
- **Residual collation divergence**: accent/width folding from
`utf8mb4_unicode_ci` is not replicated in PHP. Irrelevant for real bank
reference ids (ASCII), accepted.
- The test comment referencing a "cleanup command" that repoints orphan
fingerprinted rows describes a command that does not exist in the repo —
pre-existing, worth correcting separately.

Fixes PHP-LARAVEL-3Y
2026-07-02 13:34:29 +00:00
Víctor Falcón 6727a9c64a
feat(ai): learn from category corrections so the AI stops repeating the same mistake (#608)
## Problem

Users keep correcting the same transactions over and over. The AI
mislabels a merchant (e.g. supermarket → fuel), the user fixes it, and
the next near-identical transaction from that merchant gets mislabeled
the same way again. Today a correction is logged and the offending ai
rule is self-healed, but the system only *forgets* its mistake — it
never *remembers* the user's fix.

## Approach

A correction now becomes a deterministic, forward-looking
`AutomationRule` (new `RuleOrigin::Correction`). The next matching
transaction is categorized by that rule **before the model ever runs**
(`ApplyAutomationRules` is synchronous and runs ahead of AI
categorization), ending the loop. Zero model cost, instant, reuses the
existing rule engine.

**Matching key** (in order):
1. **Merchant** (`creditor_name`/`debtor_name`, exact `==`) when present
— stable even as the description varies.
2. Otherwise the **description's distinctive tokens** (`in` /
AND-of-`in`), extracted by the shared `DescriptionTokenizer` (noise
tokens dropped by document frequency, language-agnostic), **guarded**
against over-broad rules that could silently mis-file en masse. If
guarded out → nothing is learned, silently.

## Deliberate decisions (from a design walkthrough)

- **Forward-only**: never retroactively re-categorizes existing
transactions.
- **Learn only from system categorizations** (AI label, ai rule, or a
prior correction rule) — never from one-off manual filing, bank
categories, or the user's own hand-authored rules.
- **A key lives in exactly one correction rule**, so changing your mind
moves it to the new category. Correcting a transaction a prior
correction rule categorized is also learnable, so correction rules stay
fixable in-flow.
- Correcting to *uncategorized* learns nothing but still self-heals the
ai rule.
- **Safety net**: correction rules are visible/editable in
`settings/automation-rules` (marked with the AI sparkle, tooltip
"Learned from your correction"); the transactions table shows a toast
with an instant **Undo**.

## Review pass (two independent agents + live QA)

- **HIGH fix** (`67fc4293`): an ai rule could out-rank a freshly learned
correction and re-apply the wrong category (when the corrected
transaction was a *direct* model label with no rule id). Now every ai
rule holding the merchant is swept on correction. Regression test added.
- **Refactor** (`ca743fde`): collapsed duplicated clause-append logic;
removed a speculative unused enum helper.
- **Coverage** (`12ceb0f7`): debtor_name path, single-token description
clause, encrypted-description fail-safe.
- **Toast conflict fix** (`382c8169`, found in live QA): correcting an
AI transaction fired both the new "Learned …" toast and the pre-existing
"Transaction categorized → Automatize" prompt, which invited the user to
manually create the rule the correction had just created. Made them
mutually exclusive. Adds a browser test for the inline-correction flow.
- **Settings icon** (`f3f882b6`): correction rules now show the AI
sparkle in settings, like ai rules.

## Verified end-to-end (against a running instance)

Drove the real UI with a browser: correcting an AI-mislabeled
transaction creates the correction rule, shows the "Learned · Undo"
toast (no competing Automatize prompt), and Undo deletes the rule while
keeping the correction. Confirmed for **merchant** keys and the
**description-only** path — including that a later "practically
identical" description (different surrounding text, no merchant) is
caught by the rule, while a near-miss sharing only one distinctive token
is correctly **not** caught.

## Open question for reviewers

**`debtor_name` (P2P) as a rule key.** For incoming transfers the
merchant key falls back to the sender's name, so correcting one can
create a rule keyed on a person's name (useful for recurring transfers
from a roommate, but a possible privacy surprise; the name appears in
the rule title in settings). This matches the existing tier-2 learner's
behaviour. Keep as-is, or restrict correction rules to `creditor_name`
only? Happy to change.

## Testing

- `tests/Feature/Ai/CategoryOverrideHandlerTest.php`: merchant +
description learning, next-transaction match, over-broad rejection,
change-of-mind move, correct-to-null self-heal, the HIGH regression,
debtor_name, single-token, encrypted fail-safe.
- `tests/Browser/CategoryCorrectionLearningTest.php`: inline correction
→ toast → learned rule → undo.
- `automation-rule-title.test.tsx`: the AI sparkle shows for `ai` and
`correction`, not `user`.
- Full AI suite green (106 tests); transaction/bulk-update suites green
(62). Pint + Prettier + ESLint clean.

No new dependency, no migration (the `origin` column is a free-text
string). The feature is implicitly gated by AI categorization — with no
AI categorization there is nothing to correct and nothing is learned.
2026-06-29 19:12:15 +02:00
Víctor Falcón 777dfc07b2
feat(dashboard): add accounts manager dialog with visibility toggle and reorder (#604)
## What

Adds an **Edit** button (icon-only, top-right) to the dashboard accounts
grid, next to a new **Accounts** section title. It opens a dialog for
managing accounts:

```
Accounts                     [edit]

[ acc 1        ] [ acc 2        ]
[              ] [              ]
```

Each dialog row shows:
- the bank logo + account name
- an **eye toggle** (open/closed) to hide or show the account on the
dashboard grid
- a **drag handle** to reorder

## Behavior

- **Visibility**: hidden accounts disappear from the grid but stay in
the net worth chart/totals and remain in the dialog so they can be
re-enabled. Backed by a new `hidden_on_dashboard` column.
- **Reorder**: moved entirely into the dialog (over *all* accounts,
including hidden) instead of dragging the cards. This keeps a single
source of truth for `position` — dragging visible-only cards would have
left hidden accounts' positions inconsistent. The card-level drag handle
was removed.
- Both actions use optimistic UI, mirroring the existing reorder flow.

## Changes

- Migration: `hidden_on_dashboard` boolean on `accounts` (hidden from
default serialization like `position`, surfaced explicitly in the net
worth evolution payload).
- `AccountController::updateVisibility` +
`UpdateAccountVisibilityRequest` (owner-authorized) and the
`accounts.visibility` route.
- New `AccountsManagerDialog` component; dashboard header + plain grid;
removed the now-unused `dragHandle` prop from `AccountBalanceCard`.
- New `es.json` keys.

## Tests

Added feature tests for the visibility endpoint (toggle, validation,
ownership). Full `AccountControllerTest` and `DashboardAnalyticsTest`
suites pass; lint and types are clean on the touched files.
2026-06-27 16:11:25 +00:00
Víctor Falcón bc57eae5c3
fix(open-banking): stop storing the XXX no-currency placeholder on accounts (#602)
## Why

Prod warning logs were flooded with `Exchange rate not found, returning
unconverted amount` (≈500 lines from a single dashboard load). The
`ExchangeRateService` was behaving correctly — the source currency was
the ISO 4217 placeholder **`XXX`** ("no currency"), which has no
exchange rate, so conversions silently fell back to an **unconverted
(wrong) amount**.

Root cause: when importing accounts from a banking connection (Enable
Banking can report `currency: "XXX"`), the code did
`$accountData['currency'] ?? 'EUR'`. The `??` only catches
`null`/missing, not the literal `"XXX"`, so the placeholder was
persisted as `currency_code`.

Prod data confirmed `XXX` is the **only** account currency not covered
by the rates table (342 currencies, incl. BTC/VES/exotics, are all
present): 18 accounts (16 from bank links, 2 manual) across 14 users,
plus **5 users** whose base currency had itself become `XXX` (a
first-account `XXX` propagates to the user via `syncFromFirstAccount`).

## What

1. **Fix the source** —
`AccountUserCurrencyService::resolveImportedCurrency()` resolves a
bank-reported currency to: bank value → user base currency → app default
(`cashier.currency`), treating `XXX`/empty/missing as "no currency".
Wired into both creation paths (`CreatesAccountsFromPending`,
`AccountMappingController`); also covers Interactive Brokers imports.
2. **Backfill** — migration fixes existing rows in order: `XXX` users →
app default first, then `XXX` accounts → their owner's currency.

## Tests

- `AccountUserCurrencyServiceTest` — resolver chain (valid / uppercasing
/ XXX·empty·null → user / both missing → default).
- `AccountMappingTest` — mapping flow falls back to the user currency
when the bank reports `XXX`.
- `BackfillXxxAccountCurrenciesTest` — migration resolves both XXX
owners and XXX accounts end-to-end.

23 tests green; Pint and Larastan clean.
2026-06-27 16:01:21 +00:00
Víctor Falcón e5350ff1a6
feat(subscriptions): trial/pricing A/B/C experiment (#600)
## What

A 3-way experiment on how the paid plan is offered, plus per-variant
measurement. New signups (on/after `SUBSCRIPTION_EXPERIMENT_STARTED_AT`)
are split evenly into:

- **control** — current 15-day trial.
- **reduced_trial** — shorter trial: 3 days monthly, 7 days yearly.
- **pay_now** — charged immediately (no trial), with a self-service
money-back guarantee for the first 3 days.

Earlier users stay **legacy** and keep the 15-day trial. **While
`started_at` is null the experiment is off and everyone behaves like
control — inert until activated via env.**

## How it works

- **Assignment** — `App\Features\SubscriptionExperiment` (Pennant),
deterministic even split by a stable hash of the user id. QA can force a
variant with `feature:enable`.
- **Offer policy** — `ExperimentOffer` is the single source of truth for
trial days per plan, the pay-now flag, the refund window and refund
eligibility; shared by checkout, paywall and billing.
- **Checkout** — trial length comes from the variant (`trialDays(0)` for
pay_now → immediate charge).
- **Onboarding clarity** — the paywall states the exact terms above the
CTA: trial length for the selected plan, or "charged €X today + 3-day
money-back guarantee" for pay_now.
- **Self-service refund (pay_now)** — Settings → Billing, within the
window: refunds the upfront charge, `cancelNow`, revokes bank
connections keeping imported data. `refunded_at` records it and blocks a
second refund. Crash-safe ordering: the refund is stamped before
cancel/disconnect, which run best-effort in a try/catch.

## Measurement

`stats:experiment-funnel` (weekly → Discord): per-variant funnel
(assigned, subscribed, status breakdown, refunds) with a **net-active
rate** gated by each variant's decision window (control 15d / reduced 7d
/ pay_now 3d) so cohorts are read at equal age. Attribution reads the
variant Pennant actually served each user, so the report can't drift
from what users experienced. It also reports **MRR** (monthly run-rate
of mature net-active subs, yearly normalised ÷12) and **ARPU** (MRR ÷
assigned) per variant — ARPU is the revenue metric for the winner
decision. Plus a winner can be pinned org-wide with
`SUBSCRIPTION_EXPERIMENT_FORCE_VARIANT` (env, no deploy).

## Config (env)

- `SUBSCRIPTION_EXPERIMENT_STARTED_AT` — activates the experiment
(launch date). Null = off.
- `SUBSCRIPTION_EXPERIMENT_REDUCED_TRIAL_MONTHLY` (3), `..._YEARLY` (7),
`..._REFUND_WINDOW_DAYS` (3)

## Tests

- **Feature/unit:** assignment, offer policy, checkout wiring, refund
eligibility, the refund action incl. idempotency + crash-safe ordering
(Stripe mocked), and the funnel collector/command. ES + FR translations.
- **Browser** (`tests/Browser/SubscriptionRefundTest.php`): the
self-service refund UX end to end — card visibility + deadline, two-step
confirm, back-out, the refund control disappearing after confirming, and
gating (window passed / non-pay_now hidden). The `RefundSelfServe`
action is doubled so it never hits Stripe but applies the same DB
effect. Screenshots: `refund-card-visible`, `refund-confirm-step`,
`refund-completed`.
- Full non-browser suite green (the one failing `DashboardTest` is
pre-existing on `main` — Inertia 409 from the unbuilt local manifest).
Pint + ESLint + tsc (changed files) clean.

## Two independent reviews — acted on

**Fixed:** refund atomicity/idempotency (major) · funnel attribution now
reads Pennant's served value instead of recomputing, killing
report-vs-runtime drift (major) · pay_now copy shows the exact amount
charged · throttle + block-demo on the refund route · `resolve(?User)`
nullable · French translations.

**Reviewer notes (deferred, low value):**
- `refunded_at` is not cast to Carbon on Cashier's `Subscription` (safe
today — only null-compared; would need a custom Cashier model).
- `ExperimentFunnelCollector` walks users in PHP via `chunkById`; fine
at current volume, can move to grouped SQL if it grows.

## Confidence: 85 / 100

The critical money path is now **verified live against the Stripe
sandbox** (see below), which removes the earlier cap. All gates are
green and the acceptance criteria are met. Held at 85 (not higher)
because the browser UI test runs in CI rather than locally, and the
pay_now *hosted-checkout + webhook* leg reuses the standard Cashier
checkout already proven by the control flow (only `trialDays(0)`
differs) but wasn't re-driven through the hosted page. Given it moves
money + disconnects accounts, a human glance is still warranted before
enabling.

## Sandbox verification (live Stripe test mode)

`php artisan stripe:verify-refund` creates a real immediately-charged
subscription with a test card, runs the actual `RefundSelfServe`, and
checks the Stripe API. Result:

```
PASS  subscription active after immediate charge   (pay_now, no trial)
PASS  canSelfRefund is true before refund
PASS  latestPayment() resolves a payment intent
PASS  refunded_at is stamped
PASS  subscription is canceled
PASS  canSelfRefund is false after refund
PASS  Stripe charge shows a full refund   (refunded=true)
```

The command is committed and guarded to Stripe test keys /
non-production, so it can be re-run before each launch toggle.


## Launch checklist

1. Stripe-sandbox smoke: `php artisan stripe:verify-refund` (done —
passing). Optionally also drive the hosted pay_now checkout once for
monthly + yearly to confirm the webhook leg.
2. Set `SUBSCRIPTION_EXPERIMENT_STARTED_AT` to the launch date (set
once; don't backdate).
3. Watch `stats:experiment-funnel`; a clean cohort baseline lands once
each variant's window matures.
2026-06-27 18:00:15 +02:00
Víctor Falcón 756b4816a6
feat(stats): add weekly subscription funnel report (#599)
## What

Adds a weekly **registration → subscription → paid** funnel so we can
baseline subscription conversion and, later, measure A/B tests against
it.

- `stats:subscription-funnel` artisan command — prints the cohort table
and posts it to Discord (same webhook as the other weekly reports).
- `SubscriptionFunnelCollector` — builds the age-normalized weekly
cohort series.
- Scheduled weekly, Mondays 09:15, next to the existing cohort reports.

## How the funnel is defined

Every stage is measured from each user's **own signup**, so weekly
cohorts are compared at the same age regardless of the calendar.

- **Registered** — signups that week.
- **Subscribed** — started a `default` plan ≤30d after signup (entered
checkout/trial).
- **Paid** — that plan billed past the 15d trial: currently `active`, or
`canceled` only after outliving the trial. `trial_ends_at` is cleared on
cancel, so subscription lifespan is the reliable "did it ever bill"
signal.

Both stages are bounded by the same subscription-creation window, which
guarantees the funnel invariant **registered ≥ subscribed ≥ paid** (a
test enforces it — I caught and fixed a window bug where `paid` could
exceed `subscribed`). Cohorts too young to score show `pend`; signup
surges (launch/marketing waves) are flagged with .

## Current baseline (prod, 2026-06-27)

| Stage | Count | Rate |
|---|---|---|
| Registered | 1,289 | — |
| Subscribed | 158 | 12.3% of registered |
| Trial resolved | 128 | conversion denominator |
| Paid after trial | 91 | **71%** of resolved · 7.1% of registered |
| Currently active | 75 | 82% of payers retained |

⚠️ ~940 of the 1,289 signups arrived in the last ~5 weeks (a
late-May/June surge); their trials haven't fully resolved, so
cohort-level conversion for that wave isn't measurable yet. The 71% is
status-based — a clean cohort baseline lands in ~4–6 weeks.

## Tests

5 Pest tests covering cohort counting, the attribution window, the
funnel invariant, maturity gating, and Discord delivery. All green, Pint
clean.
2026-06-27 15:40:11 +02:00
Víctor Falcón 4038e60fbc
fix(ai): handle transient AI provider overloads — stop the Sentry noise and retry the dropped work (#595)
## Why

Sentry issue **PHP-LARAVEL-3S** (`ProviderOverloadedException: AI
provider [gemini] is overloaded`) recurs whenever Gemini returns 503/429
under high demand — 7 events in 3h during the last surge, 0 users
impacted.

Two problems behind it:

1. **Sentry noise.** `CategorizeTransactions::resolveChunkWithRetry`
retries, `laravel/ai` fails over providers, and a still-failing chunk is
deliberately dropped so the rest of the backfill proceeds. But the catch
reported **every** dropped chunk via `report()`, so an expected,
self-healing transient condition floods Sentry and buries real bugs.
2. **Silently lost work.** A dropped chunk leaves those transactions
uncategorized (`category_id NULL`) with nothing to re-trigger them: the
backfill jobs are one-shot (`tries = 1`), there is no scheduled
backfill, and the real-time listener only handles *new* transactions.
They stay uncategorized until someone manually re-runs a backfill.

## What

**1. Stop reporting transient failures.** Catch `FailoverableException`
(the marker interface for `ProviderOverloadedException` /
`RateLimitedException`) separately and log a warning instead of
reporting it. Everything else still goes to `report()` unchanged, so
real failures (malformed responses, insufficient credits, …) keep
surfacing.

**2. Retry the dropped work.** On a transient failure, schedule a
deferred, per-user `RetryTransientAiCategorizationJob` that re-reads the
user's still-pending transactions once the provider has had time to
recover (`ai_categorization.retry_delay`, default 10 min).
- `ShouldBeUnique` per user collapses a surge of dropped chunks into a
**single** retry.
- The unique lock is held through processing, so a retry that overloads
again **cannot chain another** — exactly one deferred attempt per
failure, no infinite loop. Failed 503s aren't billed.
- Wired in `resolve()`, so it covers every entry point (backfill,
onboarding, real-time listener, admin command).
- Model cost is negligible (per config), so the retry re-runs the
existing backfill rather than tracking which exact chunks failed.

## Tests

- Transient overload → chunk dropped, nothing reported, retry scheduled
for the user.
- Unexpected failure → reported, **no** retry scheduled.
- Retry job → categorizes still-pending transactions for a consenting
user; no-op without consent (agent never prompted).

Full `tests/Feature/Ai` + listeners suite green (111 tests); Pint clean.

## Existing prod backlog

The transactions already dropped before this ships stay `pending`; an
`ai:categorize-backfill <user>` recovers them on demand.

Fixes PHP-LARAVEL-3S
2026-06-26 20:07:06 +02:00
Víctor Falcón 291cfbe261
feat(ai): record the model behind each AI categorization (#594)
## Why

We pay for AI transaction categorization on Gemini, but we couldn't tell
which model produced any given suggestion. The `gemini-flash-latest`
alias used by categorization is **floating** — it silently drifted from
the `flash-lite` tier up to `gemini-3.5-flash`, which is now the
dominant cost on the Gemini dashboard. Before swapping categorization to
a cheaper pinned model we need to measure accuracy per model, and for
that we have to know which model labelled each row.

We already store the AI-suggested category and its confidence on
`transactions`; this adds the missing third axis: the model.

## What

- New nullable `ai_model` column on `transactions`, written by
`CategorizeTransactions::recordOutcome` from
`config('ai_categorization.model')` whenever the model produces a
suggestion (above or below the label bar).
- The migration backfills existing AI-categorized rows
(`ai_suggested_category_at` not null) with `gemini-3.5-flash` — the
model `gemini-flash-latest` resolved to when those suggestions were
made. Done as a raw update so it doesn't bump every row's `updated_at`.
- `ai_model` added to `$fillable` and to `$hidden` (pure ops/measurement
metadata, no need to ship it to the frontend).

## Testing

- Extended `CategorizeTransactionsTest` to assert `ai_model` is stored
on both the auto-applied and below-bar paths.
- `php artisan test tests/Feature/Ai/CategorizeTransactionsTest.php` → 5
passed.

## Follow-ups (not in this PR)

- Pin `AI_CATEGORIZATION_MODEL` to an explicit cheaper tier (mirror the
existing `AI_SUGGESTIONS_MODEL=gemini-2.5-flash-lite`) and kill the
floating alias in the config defaults.
- Capture token usage from the SDK response for cost tracking.
2026-06-26 12:53:55 +00:00
Víctor Falcón 9a458b1031
feat(ai): manage AI consent outside onboarding with live backfill (#591)
## Summary

Lets existing (non-onboarding) users discover and manage AI
categorization, gated behind a new Pennant feature flag. When a flagged
paid user grants consent, every uncategorized transaction is categorized
in the background with live, on-screen feedback.

Enable per user: `php artisan feature:enable AiConsentSettings
user@example.com`

## What's included

**Feature flag**
- New `AiConsentSettings` Pennant flag (off by default), exposed to the
frontend via the shared `features` prop.

**Billing settings (`/settings/billing`)**
- When the flag is active, a section to grant or revoke AI consent
(checkbox → `POST`/`DELETE /ai/consent`).

**Transactions page**
- A consent prompt rendered as the first row of the transactions table
(new `DataTable` `topRow` slot), styled with the shared gradient
`AiSparkleIcon`. Only shown to paid users with the flag who haven't
consented yet.
- An inline "Enable AI" button records consent without leaving the page.

**Backfill on consent**
- Granting consent dispatches a queued
`CategorizeUncategorizedTransactionsJob` that categorizes all of the
user's uncategorized transactions, **most recent first**, recording
progress in the cache.
- The transactions page polls a new status endpoint (`GET
/ai/categorization/{jobId}/status`): visible uncategorized rows show a
spinner + pulse while the backfill runs, categories stream in via
partial reloads, and a toast tracks `processed / total` until
completion.

## Notes / decisions

- **Polling, not websockets** — the project has no broadcasting
configured; this mirrors the existing `ReEvaluateTransactionRules` job +
status-endpoint pattern.
- The backfill job is kept separate from
`CategorizeOnboardingTransactionsJob` so the onboarding pass stays
progress-free (small, commented duplication).
- Reuses the existing `AiCategorizationGate` (config kill-switch + pro
plan + active consent) — free users and disabled-AI never dispatch the
job.
- Not included (YAGNI): cancelling an in-flight backfill on revoke;
persisting the spinner across full page reloads.

## Testing

- New Pest coverage: feature-flag exposure, billing/transactions consent
props, consent-triggered dispatch + gating, the status endpoint
(200/404), job progress recording, and recent-first ordering.
- `php artisan test --exclude-testsuite=Browser` passes (one unrelated,
pre-existing `DashboardTest` failure on `main`).
- Pint, Prettier and ESLint clean.
2026-06-25 10:50:35 +02:00
Víctor Falcón f60e6d7035
feat(banking): add Interactive Brokers sync via Flex Web Service (#581)
## What

Adds **Interactive Brokers** as a banking sync provider (investment
account, balances only), mirroring the Indexa Capital integration.

It uses the **Flex Web Service** rather than IBKR's Web API: the Web API
requires registering as an IBKR third party (business entity, Compliance
approval, RSA-signed OAuth, ~3-5 weeks), which is overkill for read-only
balance sync. Flex is a read-only token + Query ID model that fits our
existing API-key provider shape.

### How the sync works
- The user creates an Activity Flex Query (NAV + Open Positions) and a
Flex Web Service token in their IBKR Client Portal, then pastes both.
- Client flow: `SendRequest` → reference code → poll `GetStatement` →
parse the XML statement.
- Mapping: `EquitySummaryByReportDateInBase@total` → `balance` (daily
rows give historical backfill on first sync); `Σ(OpenPosition
costBasisMoney × fxRateToBase) + cash` → `invested_amount`, so **profit
derives as `balance − invested_amount`** (unrealized P&L), like Indexa
Capital. Everything is already in base currency, so no FX conversion is
needed.
- One statement covers every account, so the syncer fetches once per
connection to respect IB's per-query rate limit.
- IB returns HTTP 200 with an error XML, so the client translates Flex
error codes into the exceptions the sync job already understands:
`RequestException(401)` for token problems, `RequestException(429)` for
throttling, `TransientBankingProviderException` otherwise.

### Connect flow
- New connect/update-credentials endpoints validate the credentials by
pulling a statement, then build pending accounts from it.
- Credentials reuse the encrypted `api_token` (Flex token) and
`api_secret` (Flex Query ID) columns — **no migration**.
- The IB option (two fields) is added to the connect dialog, inline
connect flow, and update-credentials dialog, with Spanish translations.

## Feature flag (why this is a draft)

Gated behind a Pennant feature `App\Features\InteractiveBrokers` (off by
default). It was built against documented/open-source Flex XML fixtures,
**not a live IBKR account** (we don't have one). Before enabling,
validate against a real account (beta tester or a free IBKR account):

```
php artisan feature:enable InteractiveBrokers user@example.com
```

If the parser needs tweaks against real XML, they should be minor
(field-name level).

## Tests
- Client + balance sync: NAV → balance, invested/profit, daily backfill,
since-date incremental, multi-account, GetStatement polling, token-401 /
rate-limit-429 mapping.
- Controller: feature-flag gate (403), valid/invalid credentials,
subscription gate, onboarding auto-create, validation.
- Factory wiring, enum cases, job-level sync, feature-flag visibility
(vitest).
- Spanish translations added (enforced by `LocalizationTest`).

All green: `pint --test`, `phpstan`, OpenBanking + localization suite,
vitest.
2026-06-23 11:39:24 +02:00
Víctor Falcón b76a0de074
feat(transactions): default balance toggle on and apply it server-side (#566)
## What

In the add-transaction modal, the **Update account balance** checkbox
now defaults to **on**. An explicit opt-out is still remembered in
`localStorage`, so users who turned it off keep their preference.

Balance updates also move from the client to the backend, mirroring the
existing delete flow
(`ManualBalanceAdjuster::reverseDeletedTransaction`).

## How the balance is applied

When the toggle is on, `TransactionController::store` calls the new
`ManualBalanceAdjuster::applyCreatedTransaction`, which adjusts the
balance on the **transaction's own date**:

- A balance already on that date → increased by the transaction amount.
- No balance on that date but an earlier one exists → a new balance for
that date is created from the **closest earlier balance** plus the
amount.
- No prior balances at all (first transaction on the account) → the
balance is set to the transaction amount.

Connected accounts are skipped, since their balances come from bank
sync.

## Why

The previous client-side flow fetched all balances, computed from the
**latest balance overall** (not the transaction's date), and stored via
a separate request. This replaces those three round-trips with one
atomic server-side write and fixes the wrong base balance.

## Tests

- Feature tests for all three balance cases plus the not-requested and
connected-account guards (`tests/Feature/TransactionTest.php`).
- Frontend test asserting the toggle is checked by default in create
mode.

Full suite green locally: Pest (1690 passed), vitest (221 passed), Pint,
ESLint, Prettier.
2026-06-19 15:01:04 +00:00
Víctor Falcón 57f8c93e28
feat(stats): weekly paywall stuck-cohort report to Discord (#563)
## What

Adds a weekly `stats:stuck-cohort-report` command that measures the
percentage of users "stuck" on the paywall, compares it to the previous
week, and posts the result to Discord.

## Definition of "stuck"

A user with a non-deleted banking connection and **no** valid
subscription:

- has a banking connection (`bankingConnections()` — SoftDeletes already
excludes deleted)
- has no subscription with `stripe_status` in `active` / `trialing` /
`past_due`, nor a `canceled` one still in grace (`ends_at > now()`)

Built with Eloquent (`whereHas` / `whereDoesntHave`), no raw SQL.

## Metric

`stuck_pct = stuck / onboarded users` (`onboarded_at` not null — the
population that reached the paywall). The Discord message also shows the
raw counts, not just the percentage.

## Week-over-week

Each run snapshots `date`, `onboarded_count`, `stuck_count`, `stuck_pct`
into a new `stuck_cohort_snapshots` table (`updateOrCreate` per day,
idempotent), then compares against the most recent prior snapshot to
report the delta in percentage points and stuck count. First run has no
prior → reports current only.

## Discord

Reuses the `SendAiCohortReportCommand` pattern: posts an embed via
`DiscordWebhook` to `config('services.discord.ai_cohort_webhook_url')`
(fallback `webhook_url`). No direct `env()`.

## Schedule


`Schedule::command('stats:stuck-cohort-report')->weekly()->mondays()->at('09:00')->timezone('Europe/Madrid')`
in `routes/console.php`.

## Tests

7 tests (percentage calc across subscription statuses incl. canceled
in/out of grace, soft-deleted connection, non-onboarded excluded from
denominator, snapshot persistence, delta vs previous, single upsert per
day, Discord POST asserted via `Http::fake`). `pint` clean.
2026-06-19 14:12:51 +00:00
Víctor Falcón ae59c90f2c
AI auto-categorization: open to pro + consent, nudge free users (#561)
## What

Two related changes to the AI auto-categorization feature.

### 1. Open the gate to pro + consent (drop the new-signups-only cohort)

Eligibility for AI auto-categorization was: kill switch **+ pro plan +
active AI consent + a Pennant rollout flag** that only resolved for
users created after `ai_categorization.rollout_after`. That last cohort
gate limited the feature to a handful of recent signups (6 eligible
users in prod).

The gate is now just **kill switch + pro plan + active AI consent**, so
every consented pro user is eligible regardless of signup date.

- `allows()` and `allowsBackfill()` became identical and collapse into a
single `allows()`; `CategorizeBackfillCommand` calls it.
- The `AiCategorization` Pennant feature and the
`ai_categorization.rollout_after` config are now dead and removed. The
weekly cohort report already derives its release marker from the first
`ai_consents.accepted_at`, so nothing depends on the config.

> Note: the `AI_CATEGORIZATION_ROLLOUT_AFTER` env var must be removed
from the production environment — it is no longer read.

### 2. Nudge free users that AI could categorize their transactions

Free-plan users now see a subtle AI sparkle on uncategorized rows,
reusing the trailing icon slot already in `CategoryCell` (no layout
change). It only shows when subscriptions are enforced and the user is
not pro; clicking it routes to `/settings/billing`.

To keep it subtle, the sparkle is sampled to a share of rows via a
deterministic function of the transaction id (its last byte mapped onto
a 0-100 threshold), so the same rows decide the same way across reloads
instead of flickering or marking every row.

The share is **configurable** via `ai_categorization.upsell_sample_rate`
(env `AI_CATEGORIZATION_UPSELL_SAMPLE_RATE`, default 40), exposed to the
frontend as the `aiCategorizationUpsellRate` Inertia prop — no rebuild
needed to retune it.

## Tests

- PHP: `AiCategorizationGateTest` updated (rollout case dropped);
job/listener tests no longer activate the removed feature.
- JS: `ai-upsell-sample.test.ts` covers determinism, the 0/100 bounds,
the threshold boundary, and the per-rate split.
- Manually verified in-app (Playwright): nudge renders for a free,
consented, bankless user; rate matches config.

## Follow-ups (not in this PR)

- Remove the `AI_CATEGORIZATION_ROLLOUT_AFTER` prod env var.
- Backfilling existing pro+consent users' history still requires running
`ai:categorize-backfill {user}`.
- The nudge only reaches free users who pass the paywall (no bank
connection + paywall already seen); free users with a bank connection
never reach `/transactions`. Consider surfacing it on the paywall too.
2026-06-19 14:08:40 +00:00
Víctor Falcón 46568700b2
fix(banking): skip inaccessible EnableBanking accounts instead of failing the connection (#559)
## Problem

Sentry issue
[PHP-LARAVEL-3J](https://whisper-money.sentry.io/issues/PHP-LARAVEL-3J)
started escalating again — but with a **different** error than the 401
expired-session fixed in #557. Sentry groups both under the same
`getTransactions` culprit:

```
RequestException: HTTP request returned status code 400:
{"code":400,"message":"Account not found","detail":{"message":"Account not found","error_name":"AccountNotAccessibleException"}}
```

When EnableBanking returns `400 AccountNotAccessibleException` for
**one** account (closed, or no longer covered by the consent), the raw
`RequestException` was re-thrown. This:

1. **Aborted the whole connection's account loop** in
`EnableBankingSyncer` — accounts after the dead one never synced.
2. Marked the **entire connection** `Error` (`friendlyErrorMessage`
default), even though its other accounts were fine.
3. **Reported to Sentry** on every scheduled retry (escalating noise).

Confirmed in production (`agent:db --prod`): connection `019ea143-…` is
`active` with `valid_until` in the future (2026-09-05) and **6
accounts**; one (`08bbcdf9-…`) returns the 400, and the connection is
now stuck in `error` with the generic "try again later" message. 2 users
affected.

## Fix

A single account the bank no longer exposes must not break the rest of
the connection.

- New domain exception `InaccessibleBankAccountException` (implements
`ShouldntReport`, like `TransientBankingProviderException` /
`ExpiredBankingSessionException`).
- `EnableBankingProvider::getTransactions()` / `getBalances()` wrap a
`400 AccountNotAccessibleException` in it (keyed on `detail.error_name`)
instead of re-throwing the raw `RequestException`.
- `EnableBankingSyncer` catches it **per account**, logs a warning, and
`continue`s — the remaining accounts sync and the connection stays
`Active`.

Connections currently stuck in `error` from this self-heal on the next
scheduled sync (they're `Error` + under the retry cap + `valid_until`
future, so the scheduler re-dispatches them; the dead account is now
skipped).

Composes with #558: a user can permanently *stop syncing* such an
account from the manage-accounts screen. This PR only stops one dead
account from breaking automated syncs in the meantime.

## Tests

- `EnableBankingProviderTest`: `getTransactions` / `getBalances` wrap a
`400 AccountNotAccessibleException` as a non-reportable
`InaccessibleBankAccountException`.
- `SyncRetryAndLoggingTest`: with one inaccessible and one good account,
the good one still syncs, the connection stays `Active`, and nothing is
thrown.

All three failed before the fix. Full `tests/Feature/OpenBanking` suite
passes (276), pint clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-19 08:59:05 +02:00
Víctor Falcón c36df98d32
fix(banking): handle EnableBanking expired sessions as reconnect, not error (#557)
## Problem

Sentry issue
[PHP-LARAVEL-3J](https://whisper-money.sentry.io/issues/PHP-LARAVEL-3J)
— `RequestException: HTTP 401 {"error":"EXPIRED_SESSION"}` in
`EnableBankingProvider::getTransactions` — was escalating (26 events, 2
users).

When an EnableBanking **session** expires *before* its 90-day consent
window (`valid_until`), the sync hit a `401 EXPIRED_SESSION` that was
not classified as transient or ASPSP, so it was re-thrown as a raw
`RequestException`. Consequences:

1. **Sentry noise** — reported as an error on every scheduled retry
until the failure cap.
2. **Silent breakage for the user** — EnableBanking's syncer returns
`notifiesOnAuthFailure() === false`, so the 401 went through the generic
permanent-error path: connection set to `Error` (not `Expired`) with
**no reconnect email**.

Confirmed in production: **10 of 17** EnableBanking connections in
`error` status are actually expired sessions (`valid_until` in the
future + "Authentication failed" message), all with no notification
sent.

## Fix

Treat a `401 EXPIRED_SESSION` as the expected lifecycle event it is:

- New domain exception `ExpiredBankingSessionException` (implements
`ShouldntReport`, like `TransientBankingProviderException`).
- `EnableBankingProvider::getTransactions()` / `getBalances()` wrap the
`401 EXPIRED_SESSION` in it instead of re-throwing the raw
`RequestException`.
- `SyncBankingConnectionJob` catches it and routes through the existing
expiry handling (extracted to `markExpired()`): marks the connection
`Expired`, sends `BankingConnectionExpiredEmail`, logs a skipped
attempt, returns **without throwing**.
- The provider's HTTP error logger downgrades the expected expiry from
`error` → `warning`.

`Expired` connections are not re-dispatched by
`SyncAllBankingConnectionsJob`, so retries stop and the user is prompted
to reconnect.

## Tests

- `EnableBankingProviderTest`: `getTransactions` / `getBalances` wrap a
`401 EXPIRED_SESSION` as a non-reportable
`ExpiredBankingSessionException`.
- `SyncRetryAndLoggingTest`: an expired session marks the connection
`Expired`, queues the reconnect email, and does not throw.

All three failed before the fix. Full `tests/Feature/OpenBanking` suite
passes (264 tests).

## Follow-up (not in this PR)

The 10 connections already stuck in `Error` from this bug are past the
retry cap and won't self-heal. A one-off command to reclassify them to
`Expired` and send the reconnect email would recover those users — worth
a separate, reviewed change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-18 13:48:13 +00:00
Víctor Falcón 9328cd3e1b
feat(ai): persist AI categorization suggestions below the label bar (#547)
## Contexto

El categorizador solo persistía un resultado cuando la confianza
superaba la barra (`ai_categorization.label_confidence`, 0.7). Las
sugerencias por debajo del umbral se calculaban y se descartaban,
dejándonos sin ninguna visibilidad de qué proponía el modelo para las
transacciones que no se autocategorizaban.

## Cambio

Se guarda la sugerencia del modelo en **toda** transacción que el modelo
coloca, mediante columnas tipadas (en vez de un blob JSON):

- `ai_suggested_category_id` (FK a `categories`, nullable)
- `ai_suggested_category_at` (timestamp, nullable)
- `ai_confidence` (ya existía) — ahora se rellena **siempre**, no solo
al aplicar

Por debajo de la barra la transacción sigue sin categorizar, pero la
sugerencia queda guardada; a partir de la barra, además se autoaplica la
categoría como hasta ahora.

## Para qué sirve

- **Afinar el umbral 0.7 con datos reales**: `where
ai_suggested_category_id is not null and category_id is null` muestra
exactamente qué nos perdemos.
- Habilita una futura UI de "la IA cree que es X, ¿confirmas?"
directamente desde el FK.

## Tests

- `CategorizeTransactionsTest`: el caso sub-umbral ahora verifica que la
sugerencia se persiste; el caso aplicado verifica que además se guardan
los campos de sugerencia.
- Suite de IA + tests que tocan transacciones en verde (111 tests).
2026-06-17 08:24:30 +00:00
Víctor Falcón 220b1e11f1
refactor(banking): scalable per-provider sync via a syncer factory (#545)
## Why

Syncing was a growing `if/elseif` chain in
`SyncBankingConnectionJob::handle()` plus one private method per
provider. Every new provider meant editing the job, and the file mixed
provider-specific logic with cross-cutting orchestration. This does not
scale to dozens/hundreds of providers.

## What

Introduces a **factory + strategy** pattern:

- **`App\Contracts\BankingConnectionSyncer`** — interface: `sync()`,
`expires()`, `notifiesOnAuthFailure()`.
- **`AbstractBankingConnectionSyncer`** — defaults for the common case
(API-key provider: never expires, notifies on auth failure). A new
provider usually only implements `sync()`.
- **`BankingConnectionSyncerFactory`** — maps `connection.provider` to
its syncer (resolved from the container, so dependencies are injected).
- **Six syncers** — `IndexaCapitalSyncer`, `BinanceSyncer`,
`WiseSyncer`, `BitpandaSyncer`, `CoinbaseSyncer`, `EnableBankingSyncer`.
Each fully owns its provider behavior, including EnableBanking's daily
email and first-sync cutoff.

`SyncBankingConnectionJob` drops from ~520 to ~190 lines and now only
orchestrates: deleted-user/expiry/rate-limit checks, error handling,
retries, logging, and status updates. It asks the syncer `expires()` /
`notifiesOnAuthFailure()` instead of hardcoding provider lists.

### Adding a provider now

1. Create `XSyncer extends AbstractBankingConnectionSyncer` implementing
`sync()`.
2. Add one line to the factory map.

No changes to the job.

## Tests

- Existing job tests (`SyncBankingConnectionJobTest`,
`SyncRetryAndLoggingTest`) migrated through a shared `runSync()` helper
in `tests/Pest.php`; all original assertions preserved.
- New `BankingConnectionSyncerFactoryTest`: provider→syncer resolution,
unknown-provider exception, and the capability flags.
- Full suite green: **1604 passed**, 0 failures. PHPStan clean, Pint
applied.

## Docs

Adds `docs/adding-a-banking-provider.md` — a step-by-step developer
guide (usable by a human or an AI agent) covering the sync provider and
the broader end-to-end integration touchpoints.
2026-06-16 16:25:29 +02:00
Toni Grunwald 1c5a76a3a4
feat: add Wise open banking integration with balance sync (#525)
## Summary
Adds **Wise** as an API-token banking provider — connect flow,
transaction sync, and per-wallet balance sync — and fixes two bugs that
kept business/extra profiles and balances from working.

## Changes
- **Balance sync** — new `WiseBalanceSyncService` pulls
borderless-account balances on every sync. Previously Wise balances were
never fetched, so the displayed balance went stale (frozen at the last
value).
- **Multi-profile connect** — `WiseController` now creates accounts for
**every** profile on the token (personal *and* business), not just one.
- **ID-format fix** — `external_account_id` is now
`"{profileId}:{currency}"`. The connect flow previously wrote the
**borderless-account id**, but both sync services parse the **profile
id** — so any UI-connected account (and every business profile) silently
failed to sync.
- **Tests** — `WiseBalanceSyncTest` (currency matching, idempotent
today-balance, skip-when-unmapped) and `WiseControllerTest`
(multi-profile pending accounts, onboarding auto-create, invalid token).

## Test plan
- [x] `php artisan test
tests/Feature/OpenBanking/WiseBalanceSyncTest.php
tests/Feature/OpenBanking/WiseControllerTest.php` — 6 passing / 28
assertions
- [x] `vendor/bin/pint` — clean
- [ ] CI (full pest + lint + build)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Víctor Falcón <victoor89@gmail.com>
2026-06-16 13:23:25 +00:00
Toni Grunwald dbec1c4c13
feat: add catch-all budgets (#527)
## Summary
Adds **catch-all budgets** — a budget flagged `is_catch_all` absorbs
every expense-category transaction not already claimed by another
(non-catch-all) budget, so out-of-budget spending is still tracked.

## Changes
- Migration: `is_catch_all` boolean (default false) on `budgets`.
- `Budget` model: fillable + boolean cast.
- `BudgetTransactionService`: a catch-all budget matches expense
transactions whose `category_id` is not claimed by any non-catch-all
budget; period assignment mirrors the same rule.
- `BudgetController`: supports the flag.

## Notes
- Pre-existing WIP committed as-is; CI is the validation gate.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Víctor Falcón <victoor89@gmail.com>
2026-06-15 16:07:19 +00:00
Víctor Falcón 8013a0b6f2
feat(ai): auto-categorize transactions with AI (behind flag) (#535)
## What

Auto-categorizes transactions with AI (Gemini) for **pro +
AI-consented** users when no automation rule already matched. Ships the
full backend **behind a Pennant flag, off by default**, so it's
mergeable and testable in isolation; the UI is a deliberate follow-up.

## Why / cost

Prod check first: ~20k txns/month, **~52% of pro-user transactions are
uncategorized** after rules. At Gemini Flash-Lite rates the cost is a
**rounding error** — ~$0.13–$0.75/month for all pro users, single-digit
dollars even on full Flash. So the model is chosen for accuracy, not
price; the real constraints are trust, accuracy and privacy.

## How it works

**Two tiers** (every transaction is covered, rules are an optimization
on top):
- **Tier 1 – label** — a queued listener runs *after* the synchronous
rules; if still uncategorized and the user is eligible, the model picks
a **leaf** category (referenced by numeric index, never a UUID, so it
can't hallucinate one). Auto-applied only above the **label bar**
(`0.7`); below → left blank, no nag. Tagged `category_source = ai` +
`ai_confidence`, fully reversible.
- **Tier 2 – learn** — above the higher **rule bar** (`0.85`) *and* a
clean merchant key *and* the model flags the merchant unambiguous → the
merchant is appended to a single **ai-owned** automation rule for that
category (OR'd conditions, not rule-sprawl), so future transactions
match for free and consistently. AI rules sit at the lowest priority;
**user-owned rules are never touched**.

**Self-heal + signal** — when a user overrides an AI category, a
`category_correction` is logged (calibration signal, bucketable by
confidence) and the offending merchant condition is dropped from the ai
rule (deleted if empty). User rules and manual categories are untouched.

**Safety** — config kill switch + pro + active consent + gradual Pennant
rollout. Dedicated `ai` queue so Gemini never blocks bank syncs.
Encrypted (client-side) transactions are never sent.

**Backfill** — `ai:categorize-backfill {user}`, explicit opt-in,
batched, learns rules as it goes.

## Data model

- `transactions`: `category_source`, `ai_confidence`,
`categorized_by_rule_id`
- `automation_rules`: `origin` (`user`/`ai`)
- new `category_corrections` table

## Screenshots

<img width="921" height="384" alt="image"
src="https://github.com/user-attachments/assets/f04c2a03-b39e-4a3d-81eb-ecf26eaefb83"
/>
2026-06-15 16:35:20 +02:00
Víctor Falcón cd323bbe52
fix(budgets): make period generation idempotent (#533)
## Sentry
Fixes
**[PHP-LARAVEL-39](https://whisper-money.sentry.io/issues/PHP-LARAVEL-39)**
— `UniqueConstraintViolationException` (1062) on
`budget_periods_budget_id_start_date_unique`. 7 events over 6 days, 0
users (background command), regressed.

## Root cause
`BudgetPeriodService::generatePeriod()` did a blind
`BudgetPeriod::create()`. The scheduled `budgets:generate-periods`
command reaches `generatePeriod()` from three paths (`handle()` +
`closePeriod()`), each computing the next start date as `max(end_date) +
1 day`. Across **overlapping or repeated runs** two paths compute the
same `start_date`, and the second insert collides with the unique key:

```
insert into budget_periods (budget_id, start_date, ...) values (019ea713..., 2026-06-30 00:00:00, ...)
-> 1062 Duplicate entry '019ea713...-2026-06-30'
```

## Fix
Make creation idempotent on the `(budget_id, start_date)` unique key via
`firstOrCreate()`. It delegates to Laravel's `createOrFirst()`, which
catches the unique violation and re-queries — so the call is
**concurrency-safe** and the command is safe to re-run. Period dates are
normalized to start-of-day so the lookup matches the stored date key.

## Test
`generatePeriod is idempotent when a period already exists for the start
date` — reproduces the 1062 (verified failing against the pre-fix code
with the exact `BudgetPeriodService.php:26` stack), passes after the
fix. Returns the existing period, no duplicate row.

## Notes
Behavior change: when the next period already exists, `generatePeriod()`
now returns it unchanged instead of throwing. `closePeriod()` still
updates `carried_over_amount` on the returned period.
2026-06-15 12:44:44 +02:00
Víctor Falcón 906e3cc2b4
feat(ai): add weekly AI-suggestions cohort report (#530)
## What

Adds `stats:ai-cohort-report` — a monthly scheduled command that posts a
**weekly per-cohort** retention/conversion time series for the
onboarding AI suggestions feature to Discord.

## Design (pre/post, directional — not causal)

We measure whether shipping AI suggestions moves retention/conversion
among the users who can actually receive it. This is an **observational
pre/post** readout, deliberately chosen over a randomized A/B holdout
(low signup volume; we don't want to withhold the feature). It reports a
**correlation**, never a cause.

- **Cohort (ITT):** users who imported **≥50 transactions in their first
7 days** (a pre-treatment trait). We do *not* condition on who accepted
AI — that would reintroduce self-selection.
- **Release anchor `R`:** `MIN(ai_consents.accepted_at)`, planted by
self-accepting on deploy.
- **Metrics, all fixed-horizon from each user's own signup:**
  - Retention — active ≥14d (`last_active_at ≥ signup+14d`)
  - Trial — subscribed ≤14d
  - Paid — `active` subscription ≤30d
  - AI-acceptance — funnel-health line (descriptive, not causal)
- **Right-censoring:** too-young cohorts render as `pend`, never `0`.
- **Surge flagging:** weeks with outlier eligible volume (>2.5× median)
are flagged  so a one-time acquisition spike (e.g. the launch/YouTube
surge) isn't misread as an organic trend.

Staff/test accounts (incl. the one planting the anchor consent) are
excluded via `config('ai_suggestions.report.excluded_emails')`. Output
goes to `DISCORD_AI_COHORT_WEBHOOK_URL`, falling back to the shared
`DISCORD_WEBHOOK_URL`.

## Honest caveat (baked into every report footer)

A one-time launch+YouTube signup surge sits right before release and
there's no acquisition-source tracking, so cross-release comparisons mix
channels. Read **organic-vs-organic cohort trends over a quarter**, not
a month-one before/after diff.

## Deploy steps

1. Self-accept the AI consent in prod to plant `R`.
2. Add your email (+ staff) to `AI_SUGGESTIONS_REPORT_EXCLUDED_EMAILS`.
3. Set `DISCORD_AI_COHORT_WEBHOOK_URL` (or rely on the existing admin
webhook).

## Tests

`tests/Feature/SendAiCohortReportCommandTest.php` — 7 tests / 25
assertions covering eligibility, retention, trial/paid windows, release
anchor + pre/post split, maturity censoring, surge flagging, and Discord
delivery (incl. webhook fallback). Pint clean.
2026-06-13 23:23:34 +02:00
Víctor Falcón 8056ede636
feat(ai): suggest automation rules during onboarding (#523)
Suggests transaction categorization rules during onboarding.

After a sync or import, it groups the uncategorized transactions, asks
Gemini (via laravel/ai) to map the common merchants to categories, and
shows the results for review. The user edits or drops any and creates
the ones they want. During onboarding the accepted rules also categorize
existing transactions right away.

Off by default: it needs the `AiRuleSuggestions` Pennant flag and a
per-user AI consent. The model and thresholds are config-driven.
`ai:suggest-rules {user}` prints what a user would get.

The settings-page surface and monthly regeneration are a follow-up.
2026-06-13 22:51:15 +02:00
Víctor Falcón c944a2c37e
feat(analysis): break down spending by sub-category (#508)
## What

Clicking **Analysis** on the transactions page opens a chart of spending
by category. Two improvements:

### Sub-category breakdown
Previously every expense rolled up to its top-level category. Now each
parent shows its total with the sub-categories that carry spending
nested beneath it.

- New `CategoryTree::spendingBreakdown()` builds a two-level tree: each
root with its rolled-up total + immediate sub-categories. Grand-children
fold into their level-2 ancestor. Spend booked directly on a parent that
is *also* split across sub-categories surfaces as a **Direct** child, so
the children always sum to the parent total.
- `by_category` slices now carry a `children[]` array.
- The legend renders each parent row, then indented sub-category rows
(name, % of parent, amount). The donut stays at parent level.

### Clearer colors
Monochrome schemes (blue, pink, neutral) run darkest→lightest, so
adjacent slices were nearly identical. A new `alternateContrast()` zips
the two palette halves (`chart-1,5,2,6,3,7,4,8`) so neighbouring slices
alternate between dark and light ends. Sub-category dots reuse the
parent color at reduced opacity.

## Tests
Added coverage for sub-category nesting, the Direct child, grand-child
folding, and the flat (no-children) case. All analysis + CategoryTree
tests pass.
2026-06-08 14:53:36 +02:00
Víctor Falcón 4b90bcfc96
fix(currency): make rate fetching resilient to slow CDN (#502)
## What

Hardens `CurrencyConversionService` against a slow/unreachable external
FX-rate CDN, which was crashing `/api/cashflow/trend`.

## Sentry

- Fixes PHP-LARAVEL-2X — `RuntimeException: Failed to fetch currency
rates ... cURL error 28: Operation timed out`
- Fixes PHP-LARAVEL-31 — `FatalError: Maximum execution time of 30
seconds exceeded` (Guzzle CurlFactory)

## Root cause

For a historical date the service walked up to 8 lookback dates × 2 URLs
at a **10s timeout each** (~160s worst case → 30s PHP fatal), and a
single network timeout threw a `RuntimeException` that 500'd the whole
request.

## Changes

- **Bounded timeouts**: 3s connect / 5s total per request.
- **Abort on unreachable source**: a connection/timeout error stops the
date walk (after trying the fallback host) instead of repeating the same
timeout for every candidate date. 404s still walk back to earlier
releases.
- **Graceful degradation**: failures return an empty rate map instead of
throwing — conversions already fall back to `0.0` on missing rates, so
the trend endpoint no longer 500s.
- **Persistent cache**: historical releases cached 30d (immutable),
`latest` 6h, unavailable results 10m (dampens retries during an outage).

## Tests

- Updated the former "throws" test to assert graceful degradation to
`0.0`.
- Added: timeout aborts the date walk after 2 attempts; rates cache
across service instances.
- All 12 tests pass.
2026-06-08 09:10:38 +02:00
Víctor Falcón a7dde5fbc5
feat(open-banking): finalize bank connection without a session via state token (#498)
## Why

iOS PWAs hand the bank redirect back to the system browser (Safari),
where the app session does not exist. The old `/open-banking/callback`
sat behind `auth` + `verified` middleware, so Safari hit it with no
session → bounced to `/login`, the callback body never ran, and the
connection stayed `pending` forever (4 stale rows observed in prod).

## What

Make the bank connection flow session-independent end to end.

### 1. State-token callback (session-independent)
- `store()` generates `Str::random(40)`, persists it on the connection,
and passes it to `startAuthorization` as `state`.
- `/open-banking/callback` is now **public**. It resolves the connection
from `state_token` (`resolveConnectionFromState`) and derives the owner
from the connection, not the session.
- Connection is finalized server-side (`session_id`, status,
`state_token = NULL`) before any redirect.
- New migration adds the nullable, unique `state_token` column; the
column is hidden on the model.

### 2. Completion page instead of login bounce
When the callback runs without a session (the Safari case), the
connection is already finalized server-side. Instead of bouncing the
user to `/login`, render a standalone "go back to the app" page — their
session lives in the PWA, not here.

- New page `resources/js/pages/open-banking/connection-complete.tsx`
(success / error states, dark mode).
- `finishRedirect()` renders the page when `!Auth::check()`; logged-in
users still get the normal redirect.

### 3. Onboarding: land on the connections step + poll cross-browser
- A logged-in user finishing the flow lands straight on the onboarding
connections step (`?step=create-account`), now resolved server-side via
a validated `initialStep` prop so the step is deterministic.
- While on that step the page polls every 4s (`usePoll`, `only:
['accounts']`), so a connection finalized in another browser (PWA →
Safari) shows up without a manual refresh.

## Flow

```
POST /authorize → stateToken = Str::random(40)
  → startAuthorization(state = stateToken)
  → create BankingConnection(pending, state_token)
EnableBanking → callback?code&state=stateToken  (PUBLIC, no session needed)
  → find connection WHERE state_token = state
  → createSession(code) → finalize (session_id, status, state_token = NULL)
  → session present? redirect to connections step / mapping
                   : render "go back to the app" completion page
PWA (logged in) → onboarding?step=create-account → polls accounts every 4s
```

## Tests

36 passing. Covers:
- state-token persistence and session-less finalization
- owner resolved from the connection regardless of who is authenticated
- completion page rendered (success + error) when no session; login
fallback when no connection resolves
- logged-in user redirected directly to the onboarding connections step
- `?step=create-account` lands on that step; unknown step falls back to
default
2026-06-06 12:12:07 +02:00
Víctor Falcón 45e25e018d
feat: optionally update manual account balance on transaction delete (#491)
## What

When deleting a transaction that belongs to a **manual**
(non-bank-connected) account, the user can now opt to update the
account's **current balance** for today, so it stays accurate without
re-syncing.

- Deleting an **expense** (amount < 0) → balance **increases** by the
amount.
- Deleting **income** (amount > 0) → balance **decreases** by the
amount.
- Formula: `new_balance = current_balance − transaction.amount`. If
today has no balance row, it's seeded from the latest known balance.
- Available for **single delete** and **bulk delete** (a checkbox in the
confirmation dialog).
- Connected accounts are never touched — their balances come from bank
sync.
- On the account page, the balance display refreshes automatically after
the delete (no page reload).

## How

- New `ManualBalanceAdjuster` service; `TransactionController@destroy`
calls it when the request carries `update_balance` and the account is
manual.
- Frontend `transactionSyncService.delete/deleteMany` accept an
`updateBalance` option; both delete dialogs show the checkbox only when
a manual account is affected.
- `TransactionList` gains an `onBalanceUpdated` callback;
`Accounts/Show` uses it to bump the balance chart's refresh key.
- Added `banking_connection_id` to the accounts payload (transactions,
budgets, and the shared Inertia props feeding accounts/show) so the
frontend can identify manual accounts.

<img width="896" height="400" alt="image"
src="https://github.com/user-attachments/assets/e7f35ccb-13a1-40a1-bf04-615752bddf1a"
/>
2026-06-05 11:30:31 +02:00
Víctor Falcón 53d051800b
feat: expand parent categories inline in breakdowns (#486)
## What

Expand/collapse child categories inline in three places:
- Dashboard → **Top spending categories**
- Cashflow → **Income sources**
- Cashflow → **Expense categories**

## How it looks

<img width="1225" height="928" alt="image"
src="https://github.com/user-attachments/assets/aadce904-bfdd-46eb-b919-8695577845d7"
/>

## Implementation

**Frontend**
- `useExpandableCategories` hook: tracks expanded set, lazy-fetches
children once, caches, resets on period change.
- `AnimatedCollapse` component for the height animation.
- Recursive rows in `BreakdownCard` and `TopCategoriesCard`.

**Backend**
- Extracted `rollUpByTree`/`displayNodeFor` out of
`CashflowAnalyticsController` into `CategoryTree::rollUp` (now shared by
cashflow + dashboard).
- Dashboard `top-categories` emits `has_children`/`is_direct` and
accepts `?parent` to drill into a category's children (children + a
direct "Parent" node), mirroring the existing cashflow breakdown drill.
- Added Spanish translations for the new toggle labels.

## Tests
- Backend: `has_children` true/false + parent-drill split (children +
direct node) for dashboard top categories.
- Component: chevron expands, lazily fetches `parent=…`, renders the
child, flips to "Hide subcategories".
- Full suite green; pint / eslint / prettier clean.

> Note: children fetch lazily on first expand (same pattern as the
Sankey inline expand).
2026-06-04 11:19:21 +02:00
Víctor Falcón 1cc10566a3
feat: parent/child category tree (#474)
## Summary

Adds nested categories (parent → child, up to **3 levels**) across the
app. Children inherit their parent's type and cashflow direction, and
every category selector now renders the hierarchy as an indented tree.
Gated behind the `CategoryTree` Pennant flag (off by default).

## Backend

- **Migration**: nullable self-referencing `parent_id`; uniqueness
scoped per-parent via a `parent_unique_marker` virtual column (root
names stay unique). New composite unique created before dropping the old
one so the `user_id` FK keeps a supporting index.
- **`CategoryTree` service**: descendant/ancestor resolution, depth &
cycle checks, type cascade, subtree deletion.
- **Validation**: depth limit, cycle prevention, inherited + locked
child type.
- **Delete strategies**: reparent (default), promote to root, or cascade
(uncategorizes affected transactions).
- **Transaction filter** expands a selected parent to its descendants.
- **Cashflow Sankey & breakdown** roll up to top-level parents with
click-to-drill (children + a parent "direct" node).
- **Budgets** tracking a parent also count their children's
transactions.
- **Unified** the frontend category query behind
`Category::forDisplay()` / `FRONTEND_COLUMNS` (8 call sites) so every
selector receives the full Category shape, including `parent_id`.

## Frontend

- New `category-tree.ts` helpers (build/flatten/descendants/path,
tree-aware selection toggle + tri-state).
- **Settings page**: indented tree, sortable by name/color/type
(siblings sorted, hierarchy preserved), parent picker, delete-strategy
dialog.
- **Combobox** (transaction table cell + edit/create modal + parent
picker): indented tree, search keeps matches with their ancestors.
- **Transaction filter**: indented tree, tri-state cascading selection
(parent ↔ children), themed checkboxes, selected branches float to top
on open, ancestor-aware search.
- **Categorizer palette** and **budget multi-select**: same indented
tree + ancestor-aware search.
- **Sankey**: click a parent node to drill into its children, with
breadcrumb.
- Pennant `CategoryTree` flag gates the parent UI.

## Tests

- Pest: model/validation, delete strategies, filter expansion, cashflow
rollup/drill, budget child inclusion.
- Vitest: tree-aware selection logic.
- All green; Pint / ESLint / Prettier clean.

## Rollout

Flag is off by default — enable per user with:
\`\`\`
php artisan feature:enable "App\\Features\\CategoryTree" you@example.com
\`\`\`
2026-06-03 19:30:12 +02:00
Víctor Falcón 71dd6e2b7f
feat(budgets): track multiple categories and labels per budget (#466)
## Summary

Budgets previously tracked a single category **or** label (mutually
exclusive). This lets a budget span **multiple** categories and labels
at once, all pooling spend against one allocated amount per period.

Scope decided with the requester:
- **Shared pool** — one allocated amount; any tracked category/label
counts against it.
- **Multi categories + multi labels** on a single budget.
- **Create-only** — tracking is chosen at creation and locked afterward
(edit dialog shows it read-only).

## Changes

**Backend**
- New `budget_category` + `budget_label` pivot tables; data migration
copies existing `category_id`/`label_id` into them, then drops those
columns.
- `Budget` model: `categories()` / `labels()` belongsToMany.
- `BudgetTransactionService` matches transactions across **all** tracked
categories OR labels (live assignment + historical backfill).
- `StoreBudgetRequest` accepts `category_ids` / `label_ids` arrays,
requires ≥1 across both, validates ownership. `update` no longer touches
tracking.

**Frontend**
- Reusable `MultiSelect` (popover + command + badge).
- Create dialog uses multi-selects; cards and show page render tracked
categories/labels as badges; edit dialog shows them read-only.
- Dexie bumped to v10 (drops unused per-category allocations table, adds
`budget_labels`).

## Testing
- Updated all budget/transaction/listener/browser tests for the pivot
model; added cases for multi-category matching, mixed category+label
pooling, and store validation (empty selection + foreign-ownership).
- Added the new `__()` strings to `lang/es.json`.
- Local `pint`, `lint`, `format` pass. Relying on CI for the full suite.
2026-06-01 12:32:23 +02:00
Víctor Falcón f9bf0ea5ff
feat(discord): enrich Stripe event messages and dedupe deliveries (#460)
## Why

The Stripe subscription events posted to the Discord admin feed weren't
useful (only status + raw `cus_123` id) and sometimes arrived duplicated
— including two cancellation messages for the same subscription.

## What changed

**Richer messages** (`PostStripeEventToDiscord`)
- Customer name + email (resolved via the Stripe API), plan + interval
(`€19.99 / month`), status, subscription id.
- A `Changed` field built from Stripe's `previous_attributes` diff (e.g.
`Status: trialing → active`).
- Cancellation reason, trial-end and period-end dates where relevant.
- Invoices now show invoice number + subscription id.

**Deduplication (two causes fixed)**
- *Webhook/queue retries*: guard on the Stripe event id via
`Cache::add(..., 24h)` — first delivery wins, retries skipped.
- *Double cancellation*: Stripe fires `subscription.updated` (cancel
scheduled) **then** `subscription.deleted`. We now label the scheduled
one `🗓️ Cancellation scheduled` distinctly from `👋 Subscription
cancelled`, and only post `subscription.updated` when something
meaningful changed (status, plan, or cancellation), dropping trivial
churn.

**New** `StripeCustomerResolver` service — resolves `Name (email)` from
a customer id, falls back to the id on any error. Injectable so it's
stubbed in tests (no network).

## Tests

9 cases covering rich fields, dedup, scheduled-vs-deleted cancellation,
status-change diff, trivial-update skip, deletion reason, and invoice
amounts. All passing.

Note: this adds one Stripe API call per subscription/invoice event, made
inside the queued listener (no webhook latency).
2026-05-31 12:50:37 +02:00
Víctor Falcón 0b528b7902
feat: add Discord admin feed for daily stats and Stripe events (#458)
## What

Adds a private Discord admin feed with two flows, both posting through
one webhook (`DISCORD_WEBHOOK_URL`):

### 1. Daily stats — `stats:daily-report`
Scheduled **09:00 Europe/Madrid**. Posts an embed with:
- New users created **yesterday** (Madrid calendar day, converted to UTC
for the query)
- Total users
- Active/trialing counts + current & projected **MRR / ARR** per
currency

Reuses the #457 Stripe stats logic, extracted into
`SubscriptionStatsCollector` so the existing `stripe:subscription-stats`
command and this report share one source of truth.

### 2. Stripe events — `PostStripeEventToDiscord`
Queued listener on Cashier's `WebhookReceived`. Posts on:
- `customer.subscription.created` / `updated` / `deleted`
- `invoice.payment_succeeded` / `payment_failed`

## Setup
- `DISCORD_WEBHOOK_URL` — added to prod  (set locally to test)
- Stripe dashboard must send the 5 event types above to the existing
Cashier `/stripe/webhook` endpoint
- A queue worker must be running (listener is `ShouldQueue`)

## Tests
13 passing: Discord client (3), daily report (2), Stripe event listener
(4), plus the existing stats command (4) still green after the refactor.
2026-05-30 18:14:46 +02:00
Víctor Falcón 64b78e3680
fix(banking): handle balance-fetch timeouts and silence handled retries (#450)
Fixes two linked production banking-sync issues.

## PHP-LARAVEL-W — `cURL error 28: timed out` in
`EnableBankingProvider::getBalances` (8 events, 4 users, High)

`getBalances` called `$response->throw()` raw, so a connection timeout
(or ASPSP error) escaped as an **unhandled**
`ConnectionException`/`RequestException` and crashed the sync.
`getTransactions` already wraps these in
`TransientBankingProviderException` (which `implements ShouldntReport`
and is handled as a transient, retryable error in
`SyncBankingConnectionJob`).

→ `getBalances` now follows the exact same pattern. Genuine validation
errors (non-ASPSP 4xx) stay reportable.

## PHP-LARAVEL-2D — `SyncBankingConnectionJob has been attempted too
many times` (High, regressed)

The hanging balance call above pushed the job past its 120s `timeout`,
the worker was killed mid-job, and the retry tripped a
`MaxAttemptsExceededException`. That exception is thrown by the queue
worker (not catchable in `handle()`), and the job's `failed()` handler
**already** records the terminal `Error` state on the connection — so
the Sentry report is redundant operational noise.

→ Fixing W removes the main cause of the timeout. Additionally,
`MaxAttemptsExceededException` is no longer reported **for this job
only** (scoped via `dontReportWhen` on `$e->job?->resolveName()`); other
jobs still report it.

## Tests
- `getBalances` wraps connection failures and ASPSP errors as
non-reportable transient errors; keeps non-ASPSP client errors
reportable.
- `MaxAttemptsExceededException` is not reported for
`SyncBankingConnectionJob`, but still reported for other jobs.

Fixes PHP-LARAVEL-W, PHP-LARAVEL-2D.
2026-05-29 14:58:38 +02:00
Víctor Falcón bef657c2ed
fix(currency): degrade gracefully when rates return 404 (#449)
## Problem

`PHP-LARAVEL-2M` (High, escalating, 38 events) — `RuntimeException:
Failed to fetch currency rates for xxx on 2025-12-31: HTTP 404` crashing
`/dashboard`.

A user holds an account in currency `xxx` (ISO 4217 "no/unknown
currency"). No rate file exists for it, so every candidate URL returns
404. `CurrencyConversionService::fetchRates` threw an unhandled
`RuntimeException` that bubbled past the graceful-degradation logic in
`ExchangeRateService::convert` and broke the whole dashboard render.

## Fix

Distinguish **permanent** 404s from **transient** failures:
- All sources return 404 → log a warning and return `[]`. Callers
already handle empty rates (return unconverted / zero amount).
- Timeouts / 5xx / connection errors → still throw, so real outages stay
visible.

## Tests

- New: unknown base currency with all-404 sources returns `0.0` instead
of throwing.
- Existing "throws when both primary and fallback fail" (500s) still
passes — transient errors still surface.

Fixes PHP-LARAVEL-2M.
2026-05-29 14:44:56 +02:00
Víctor Falcón 10da06ed84
feat(transactions): add counterparty fields (#440)
## Summary
- add creditor/debtor fields to transactions with raw_data backfill
- store counterparties on bank sync and CSV/XLS imports
- add creditor/debtor filters plus hidden table columns

## Tests
- php artisan test --compact tests/Feature/LocalizationTest.php
tests/Feature/OpenBanking/TransactionSyncServiceTest.php
tests/Feature/TransactionFilterTest.php
- npm test -- resources/js/lib/file-parser.test.ts --run
- vendor/bin/pint --dirty --format agent

Note: `npm run types` still has pre-existing unrelated errors; no
creditor/debtor/import-related errors remained in filtered output.

## Screenshot
<img width="1241" height="676" alt="8odYWtFcvUM"
src="https://github.com/user-attachments/assets/55653485-d588-4beb-9e6a-5c7c81ba7cf8"
/>
2026-05-27 16:20:55 +02:00
Víctor Falcón 606093d311
fix: batch automation rule application (#435)
## Sentry
- Issue: PHP-LARAVEL-2C
- URL: https://whisper-money.sentry.io/issues/122336983/

## Root cause
Applying an automation rule to multiple transactions used
per-transaction `saveQuietly()` and `syncWithoutDetaching()`. Sentry
detected repeated transaction updates and pivot lookups/inserts on
`/settings/automation-rules/{automationRule}/apply`.

## Fix
- Added batched automation rule application for transaction collections.
- Bulk updates category changes in one query.
- Bulk inserts missing label pivots with `insertOrIgnore`.
- Reused batching from sync controller path and queued job chunks.

## Verification
- `vendor/bin/pint --dirty --format agent`
- `php artisan test --compact
tests/Feature/AutomationRuleApplicationTest.php`
2026-05-26 10:45:37 +02:00