## Problem
Since #781 a catch-all budget ("Not budgeted") yields to any budget that
tracks a
transaction by category **or** label in a period covering its date.
Reassignment is
driven by `TransactionCreated` / `TransactionUpdated`, whose listener is
`AssignTransactionToBudget`.
Pivot writes and query-builder mass updates fire no model event. So
every path that
attaches a label without a real `save()` left the transaction assigned
to whatever it
was before — typically stuck in the catch-all and missing from the
budget that tracks
its label.
In production, 44 expenses carrying one label never entered their label
budget:
`label_transaction.created_at` is ~10 minutes later than
`transactions.updated_at` on
each of them, i.e. the listener ran before the label existed and nothing
ran after.
## The paths that were broken
| Path | Write | Event |
|---|---|---|
| `AutomationRuleService::applyActions` | `saveQuietly()` +
`syncWithoutDetaching` | none |
| `AutomationRuleService::applyRuleActionsToTransactions` |
`LabelTransaction::insertOrIgnore` + mass category `update()` | none |
| `Mcp\Tools\LabelTransaction` | `syncWithoutDetaching` / `detach` |
none |
| `TransactionController::bulkUpdate` | mass category `update()` +
`labels()->sync()` | none |
The last one was found during review and is the surface users hit most —
the
bulk-actions bar in the transactions table. Its `syncLabels()` helper
does
`sync()` then `save()`, but those models are loaded before the mass
update and
nothing dirties them, so `Model::save()` skips `performUpdate()` and
fires nothing.
The docblock claiming the `save()` bumped `updated_at` was wrong and is
corrected.
## The fix
A dedicated `ReassignTransactionsToBudgets` job, dispatched from each
path, rather
than re-broadcasting `TransactionUpdated` — which would also re-run the
automation
rules that dispatched it. The job takes ids, not models, so a retry
never works from
a stale payload.
Batching, so no path queues one job per row:
- the bulk rule apply dispatches per batch of 500 ids (the AI suggestion
path is the
one caller that can hand it an unbounded list)
- `ReEvaluateTransactionRulesJob` loops `applyRules()` over the user's
whole history,
so `applyRules()` takes `reassignBudgets` and that job batches one job
per chunk
- `TransactionController::bulkUpdate` dispatches once for the whole
selection
Notifications are suppressed (`notify: false`) everywhere the change is
an
administrative edit rather than new spending: bulk rule applies,
re-evaluating rules
over history, the bulk-actions bar, and MCP relabelling. Otherwise
removing a label
would announce a months-old expense as new in the catch-all budget. The
single
transaction matched by a rule at creation time keeps notifications on.
Suppressing
them leaves `close_to_limit_notified` / `over_limit_notified` unclaimed,
so the next
genuine transaction into that budget still alerts.
A note never changes which budget counts a transaction, so a note-only
rule no longer
earns a reassignment on either path.
`applyRuleActionsToTransactions` was doing category, note and label work
inline; the
three branches are extracted so the added dispatch keeps the method
under the
repo's complexity-10 gate.
## Deploy step
This stops the state from going stale again, it does not repair what is
already
stale. After deploy:
```
php artisan budgets:reassign-labeled [--user=<email>] [--dry-run]
```
## Tests
`tests/Feature/LabelBudgetReassignmentTest.php` covers all four paths
plus the
re-evaluate batching, each asserting the transaction actually leaves the
catch-all
and lands in the label budget. All seven fail on `main` and pass here.
`AutomationRuleApplicationTest` now pins that the bulk apply queues
exactly one
reassignment job and that it is silent.
## QA
Verified in the browser against the running app with a throwaway
account: a
catch-all budget and a label budget over the same period, four expenses
starting in
the catch-all.
- Selecting all four in the transactions table and applying the label
moves
**Not budgeted $540.00 → $0.00** and **Miami 26 $0.00 → $540.00**,
confirmed in
`budget_transactions`.
- "Remove all labels" hands them back to the catch-all.
## Demo
<!-- PLACEHOLDER: drag the QA video here -->
https://github.com/user-attachments/assets/5a135257-42cf-42a7-a66a-3f76bf773e0c
## Follow-ups (not in this PR)
- Soft-deleting a label leaves its budget still counting the
transactions; the
catch-all never takes them back.
- `CategoryTree::deleteSubtree` mass-nulls `category_id` with no event,
so cascade
category deletion leaves transactions in their old category budget.
- `BudgetService::create` only adds rows for a new budget's historical
transactions;
it never removes their catch-all rows, so creating a label budget next
to an
existing catch-all double-counts until something else reassigns them.
## Summary
Wave 2 structural refactor: the same financial math was copy-pasted
across the
dashboard and the analytics API endpoints, so it could silently diverge
between
screens — a real risk in a finance app. This consolidates the duplicated
calculations into single sources of truth, kills a net-worth N+1, and
aligns
the PHP/TS rule engines and the `TransactionSource` enum. Every change
is
**behavior-preserving**; the numeric outputs of every endpoint are
unchanged and
are locked down with characterization/parity tests.
Builds on merged #640 (Wave 1); no file overlap. No dependencies
changed.
## Changes (per commit)
- **Consolidate savings-rate math into `CashflowSummaryService`** —
`savings_rate`
and `net` were byte-identical inline in `DashboardController` and
`Api/CashflowAnalyticsController`. Extracted to
`CashflowSummaryService::summarize(income, expense)`;
both controllers now spread its result (same keys, same values).
- **Move income/expense-side classification onto the `Transaction`
model** —
the income/expense side test was reimplemented in three places
(`Api/TransactionAnalysisController`, `Api/CashflowAnalyticsController`,
`DashboardController`). Now `Transaction::isIncomeSide()` /
`isExpenseSide()`.
- **Extract duplicated `getCategorySpending` into
`CategorySpendingService`** —
the tree-rollup expense-spending query was duplicated verbatim between
`DashboardController` and `Api/DashboardAnalyticsController`. Moved to
`CategorySpendingService::forPeriod()` (drill-parent parameterized).
- **Batch net-worth balance lookups to kill the per-account N+1** —
`Api/DashboardAnalyticsController::calculateNetWorthAt` ran one
`AccountBalance` query per account per compared period. Now uses the
existing
`BalanceLookup::forAccounts()` (fixed 3 queries via carry-forward seed +
in-range records), reproducing the exact "latest balance <= date, else
0"
semantics.
- **Align server rule normalization with the client and lock it with
parity
fixtures** — `AutomationRuleService::normalizeRuleJson` protected only
`['description','notes']` while `rule-engine.ts` also protected
`creditor_name`/`debtor_name`. Aligned to the superset (structurally a
no-op
since those var names are already lowercase, so no matching change) and
added
shared PHP+TS parity fixtures so the two engines can never drift
unnoticed.
- **Add missing `TransactionSource` cases to the TS type** —
`transaction.ts`
was missing `enablebanking`/`wise`; now mirrors
`App\Enums\TransactionSource`.
- **Guard `sumTransactions` against unsupported category types**
(reviewer fix) —
replaced the income/expense ternary that silently treated any non-Income
type
as expense with a `match` that throws on Savings/Investment/Transfer.
- **Document category eager-load expectation on `Transaction` side
methods**
(reviewer fix) — doc-only note to prevent a future N+1.
## Test plan
New tests:
- `tests/Unit/Services/CashflowSummaryServiceTest.php` —
net/savings-rate/rounding/div-by-zero.
- `tests/Feature/TransactionSideClassificationTest.php` — income/expense
side across signs, uncategorized, and transfer/savings/investment =
neither side.
- `tests/Feature/DashboardAnalyticsTest.php` — new net-worth test
asserts both the values (600000 / 540000) and a flat balance-query count
(<= 3) regardless of account count.
- `tests/Feature/RuleEngineParityTest.php` +
`resources/js/lib/rule-engine-parity.test.ts` +
`tests/Fixtures/rule-engine-parity.json` — one shared fixture set
driving both the PHP and TS rule engines.
Results (targeted, local):
- `--filter=Cashflow` (exclude Browser): 57/57 passed
- `--filter=DashboardAnalytics`: 41/41 passed
- `--filter=AutomationRule`: 60/60 passed
-
`--filter=TransactionSideClassification|CashflowSummaryService|RuleEngineParity`:
21/21 passed
- `bun run test rule-engine` (vitest): 13/13 passed
- `vendor/bin/pint --test`: pass; `bun run lint`: 0 errors; `bun run
format:check`: clean
- `bun run types`: 157 errors (unchanged pre-existing baseline), 0 in
touched files
Note: the 6 `Cashflow*` Browser tests fail locally only on "Vite
manifest not found" (no build present); they are environmental, not
logic, and pass in CI.
## Reviewer findings
Two read-only reviewers (architecture/quality and product/behavior)
reviewed the diff.
**Addressed**
- Both flagged that `sumTransactions` silently treated any non-Income
type as expense — added a throwing `match` guard.
- Eager-load expectation documented on the `Transaction` side methods.
**Verified identical** (behavior reviewer):
income/expense/net/savings_rate across all three endpoints; net worth
for both compared dates including no-record / all-records-after-range /
same-date edge cases; category spending (uncategorized excluded,
soft-deleted categories excluded, rollup/drill preserved); rule-engine
normalization output; multi-currency conversion.
**Deferred (documented)**
- The income/expense **summation** itself is still computed three ways
with differing uncategorized-transaction handling (dashboard
`whereExists` + sign vs analytics `join` excluding uncategorized vs
in-memory `isIncomeSide`). Unifying it would change numbers, so it is
out of scope for this behavior-preserving PR — worth a dedicated
follow-up.
- `savings_rate` keeps its `int|float` union (int `0` when income is 0).
Intentionally preserved to keep JSON output byte-identical.
- `BalanceLookup`'s `empty()` guard never short-circuits a `Collection`,
so a zero-account user runs 3 empty (harmless) queries. Left untouched —
it lives in a shared, unchanged service and only affects a no-account
edge case.
Do not merge before Wave 1 (#640) is in main.
## Summary
Starts from a security finding on the background-job status endpoints
and folds in the most important issues surfaced by a follow-up review of
the same feature area (categorization backfill, bulk rule re-evaluation,
automation-rule apply).
Each change is its own commit.
## Changes
### 1. `fix(security)` — scope job-status cache keys to the owning user
The categorization, bulk re-evaluation, and apply status endpoints
looked jobs up by a bare job UUID with no user scoping. Any
authenticated user who obtained another user's job id could poll its
progress payload. Cache keys now include the owning user's id, so a
status request keyed by the polling user's id resolves only that user's
own jobs — a mismatched owner falls through to the existing 404. No
ownership store or extra lookup. Cross-user isolation tests added for
all three endpoints.
### 2. `fix(automation-rules)` — re-check `only_uncategorized` at apply
time
The apply flow cached a match snapshot for up to 15 min (keyed only by
`rule.updated_at`) and applied the rule's category to every id in it
without re-checking eligibility. A transaction categorized *after* the
snapshot (by the user, a sibling rule, or a concurrent AI backfill) was
silently re-categorized and stamped `category_source=Rule`. Now
re-filtered through `shouldSkipForOnlyUncategorized()` at apply time;
skipped rows are no longer counted as changed.
### 3. `perf(automation-rules)` — memoize rule set per user in
`applyRules()`
Bulk re-evaluation calls `applyRules()` once per transaction, and each
call re-queried the user's whole rule set + labels — an N+1 scaling with
transaction count (the sibling apply job already loaded rules once).
Memoized per user for the service instance's lifetime (resolved fresh
per job, so rules created mid-run are intentionally not seen).
Query-count test asserts one `automation_rules` query regardless of
transaction count.
### 4. `fix(transactions)` — stop polling when the job never starts
The re-evaluate and apply pollers rescheduled on any non-terminal
status, including `pending`. If the queue worker is down the job never
runs, `failed()` never fires, and the client polled for the full
hour-long TTL with a stuck spinner. Now gives up after 30 consecutive
`pending` ticks (~30s), mirroring the guard the AI-categorization poller
already has. Long `processing` runs are unaffected.
### 5. `test(automation-rules)` — cover apply job execution and failure
branches
`ApplySingleAutomationRuleJob` was only asserted to be *pushed*; its
`handle()` body and `failed()` branch had no coverage, and
`ReEvaluateTransactionRulesJob::failed()` was untested too. Added direct
`handle()`/`failed()` tests pinning the progress-cache payloads.
## Deferred follow-ups (surfaced by review, not in this PR)
- **Consolidate the three frontend pollers onto `usePollJobStatus`** —
they hand-roll `setTimeout` loops and don't tear down on unmount (can
`setState`/`onClose` after the dialog closes). The hook already exists
and is unmount-safe; routing all three through it would also make the
`pending` cap unit-testable. The cap in change #4 currently mirrors the
already-shipped AI poller and is not separately unit-tested.
- **`noteAlreadyPresent()` uses a substring match**
(`AutomationRuleService`) — a rule note that is a substring of an
existing note is silently not appended. Should compare note lines
exactly.
- **No dedup/lock on concurrent apply/re-evaluate jobs** — note appends
are a non-atomic read-then-write, so two concurrent jobs can duplicate a
note. Category/label writes are already idempotent.
- **Transient 404 → false "failed"** — a mid-run cache
eviction/TTL-expiry (or the deploy window of change #1, where an
in-flight job's pre-deploy key is orphaned) makes the poller surface a
false failure. Retry a few times on transient errors before giving up.
- **`matches()` pagination** advances `next_offset` by fetched rows, not
page-window size — deleted ids can stall infinite scroll before reaching
`total`.
- **"Apply to N" count / encrypted-skip counts** can be stale or
under-report vs what's actually processed.
- **Job-trait convention drift** between the three jobs (modern
`Queueable` vs legacy trait stack).
## Testing
- `vendor/bin/pint --test` — pass
- `bun run format` / `bun run lint` — pass (one pre-existing unrelated
warning in `chart.tsx`)
- Affected Pest suites (apply, re-evaluate, evaluation, rule,
categorization, apply-rule-suggestions) — 94 tests pass
## What
Auto-categorizes transactions with AI (Gemini) for **pro +
AI-consented** users when no automation rule already matched. Ships the
full backend **behind a Pennant flag, off by default**, so it's
mergeable and testable in isolation; the UI is a deliberate follow-up.
## Why / cost
Prod check first: ~20k txns/month, **~52% of pro-user transactions are
uncategorized** after rules. At Gemini Flash-Lite rates the cost is a
**rounding error** — ~$0.13–$0.75/month for all pro users, single-digit
dollars even on full Flash. So the model is chosen for accuracy, not
price; the real constraints are trust, accuracy and privacy.
## How it works
**Two tiers** (every transaction is covered, rules are an optimization
on top):
- **Tier 1 – label** — a queued listener runs *after* the synchronous
rules; if still uncategorized and the user is eligible, the model picks
a **leaf** category (referenced by numeric index, never a UUID, so it
can't hallucinate one). Auto-applied only above the **label bar**
(`0.7`); below → left blank, no nag. Tagged `category_source = ai` +
`ai_confidence`, fully reversible.
- **Tier 2 – learn** — above the higher **rule bar** (`0.85`) *and* a
clean merchant key *and* the model flags the merchant unambiguous → the
merchant is appended to a single **ai-owned** automation rule for that
category (OR'd conditions, not rule-sprawl), so future transactions
match for free and consistently. AI rules sit at the lowest priority;
**user-owned rules are never touched**.
**Self-heal + signal** — when a user overrides an AI category, a
`category_correction` is logged (calibration signal, bucketable by
confidence) and the offending merchant condition is dropped from the ai
rule (deleted if empty). User rules and manual categories are untouched.
**Safety** — config kill switch + pro + active consent + gradual Pennant
rollout. Dedicated `ai` queue so Gemini never blocks bank syncs.
Encrypted (client-side) transactions are never sent.
**Backfill** — `ai:categorize-backfill {user}`, explicit opt-in,
batched, learns rules as it goes.
## Data model
- `transactions`: `category_source`, `ai_confidence`,
`categorized_by_rule_id`
- `automation_rules`: `origin` (`user`/`ai`)
- new `category_corrections` table
## Screenshots
<img width="921" height="384" alt="image"
src="https://github.com/user-attachments/assets/f04c2a03-b39e-4a3d-81eb-ecf26eaefb83"
/>
## Summary
- add creditor/debtor fields to transactions with raw_data backfill
- store counterparties on bank sync and CSV/XLS imports
- add creditor/debtor filters plus hidden table columns
## Tests
- php artisan test --compact tests/Feature/LocalizationTest.php
tests/Feature/OpenBanking/TransactionSyncServiceTest.php
tests/Feature/TransactionFilterTest.php
- npm test -- resources/js/lib/file-parser.test.ts --run
- vendor/bin/pint --dirty --format agent
Note: `npm run types` still has pre-existing unrelated errors; no
creditor/debtor/import-related errors remained in filtered output.
## Screenshot
<img width="1241" height="676" alt="8odYWtFcvUM"
src="https://github.com/user-attachments/assets/55653485-d588-4beb-9e6a-5c7c81ba7cf8"
/>
## Sentry issue
- PHP-LARAVEL-2F: https://whisper-money.sentry.io/issues/122581787/
## Root cause
- Automation rule match preview eagerly loaded account, bank, category,
and labels for every 500-transaction chunk, even for rules that only
inspect description fields.
- Sentry flagged repeated account/bank eager-load queries across chunks
as an N+1 pattern on
`/settings/automation-rules/{automationRule}/matches`.
## Fix
- Detect variables used by an automation rule and eager load only
relationships required for evaluation.
- Keep label eager loading only when label-only skip logic needs it.
- Avoid lazy loading unused relationships while preserving full data
shape for rule evaluation.
- Update the Sentry prompt to use the `sentry` CLI workflow.
## Verification
- `vendor/bin/pint --dirty --format agent`
- `php artisan test --compact
tests/Feature/AutomationRuleApplicationTest.php`
- `php artisan test --compact
tests/Feature/AutomationRuleEvaluationTest.php
tests/Feature/AutomationRuleApplicationTest.php`
## 🚪 Why?
### Problem
Automation rule evaluation was happening entirely on the frontend,
relying on the client to fetch rules, loop over transactions, and apply
category/label/note changes. This meant evaluation was tied to
browser-side decryption logic, couldn't be triggered server-side, and
had no reliable progress reporting for bulk operations.
## 🔑 What?
### Changes
- **Single re-evaluation**: `POST
/transactions/{transaction}/re-evaluate-rules` applies all matching
rules immediately and returns the updated transaction; the frontend
updates local state from the response.
- **Bulk re-evaluation**: `POST /transactions/re-evaluate-rules`
dispatches a queued job and returns a `job_id`; the frontend polls `GET
/transactions/re-evaluate-rules/status/{jobId}` ~every second, updates a
progress toast, and refreshes the list when done.
- **Note support**: `AutomationRuleService` now applies plain
(unencrypted) `action_note` actions; encrypted notes are skipped since
the backend cannot decrypt them.
- **Dirty-only saves**: `AutomationRuleService::applyActions()` now
tracks whether any field changed and only calls `saveQuietly()` when
needed, avoiding unnecessary writes.
- **Encrypted transactions skipped**: Transactions with `description_iv`
are excluded from bulk queries and silently skipped on single
re-evaluation (existing behavior preserved).
- **Frontend cleanup**: Removed `evaluateRules`,
`appendNoteIfNotPresent`, and related crypto imports from
`transaction-list.tsx`, `index.tsx`,
`use-re-evaluate-all-transactions.tsx`, and
`transaction-actions-menu.tsx`.
## ✅ Verification
### Tests
- New: `tests/Feature/ReEvaluateTransactionRulesTest.php` — 14 passing
tests covering:
- Single re-evaluation applies category, label, and note
- Single re-evaluation skips encrypted transactions
- Bulk job dispatched and returns 202 with job_id
- Bulk status polling returns `processing` and `done`
- Partial bulk (specific transaction_ids) only processes selected
transactions
- Encrypted transactions excluded from bulk processing
- Unauthenticated requests are rejected (401)
### Manual Verification
- Row dropdown "Re-evaluate rules" calls backend and updates the row in
place
- Bulk "Re-evaluate All Expenses" dispatches job, shows progress toast,
refreshes list on completion
- Encrypted transactions are silently skipped in both flows
Víctor Falcón