fix(automation): avoid rule preview n+1 (#431)

## Sentry issue
- PHP-LARAVEL-2F: https://whisper-money.sentry.io/issues/122581787/

## Root cause
- Automation rule match preview eagerly loaded account, bank, category,
and labels for every 500-transaction chunk, even for rules that only
inspect description fields.
- Sentry flagged repeated account/bank eager-load queries across chunks
as an N+1 pattern on
`/settings/automation-rules/{automationRule}/matches`.

## Fix
- Detect variables used by an automation rule and eager load only
relationships required for evaluation.
- Keep label eager loading only when label-only skip logic needs it.
- Avoid lazy loading unused relationships while preserving full data
shape for rule evaluation.
- Update the Sentry prompt to use the `sentry` CLI workflow.

## Verification
- `vendor/bin/pint --dirty --format agent`
- `php artisan test --compact
tests/Feature/AutomationRuleApplicationTest.php`
- `php artisan test --compact
tests/Feature/AutomationRuleEvaluationTest.php
tests/Feature/AutomationRuleApplicationTest.php`
This commit is contained in:
Víctor Falcón 2026-05-26 08:02:46 +02:00 committed by GitHub
parent 0d4c68361a
commit fd67cf7c72
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 108 additions and 13 deletions

View File

@ -15,7 +15,8 @@ Goal:
Rules:
- Protect local work. Start with `git status --short`. If uncommitted changes exist, stop and ask before branching.
- Never expose secrets. Do not print tokens, `.env`, Sentry auth, DB URLs, or PII.
- Use Sentry MCP tools first: `find_organizations`, `find_projects`, `search_issues`, `get_sentry_resource`, `search_issue_events`, `get_issue_tag_values`, `get_replay_details`, `get_profile_details`, `analyze_issue_with_seer` when root cause unclear.
- Use the Sentry CLI (`sentry`) first. Prefer stored OAuth login over env build tokens; if `SENTRY_AUTH_TOKEN` is invalid or too narrow, run CLI commands as `env -u SENTRY_AUTH_TOKEN sentry ...` unless `SENTRY_FORCE_ENV_TOKEN=1` is intentionally set.
- Never print raw Sentry JSON that may contain PII. Redact emails, user IDs when summarizing. Keep secrets out of output.
- For Laravel ecosystem changes, use `application-info` and `search-docs` before code changes.
- Activate/read relevant project skills when touched: Pest tests, Inertia React, Wayfinder, Tailwind, Fortify, Pennant.
- Every code change needs programmatic verification. Add or update a focused Pest/test when feasible. Run minimum affected tests. Run `vendor/bin/pint --dirty --format agent` after PHP edits.
@ -23,17 +24,24 @@ Rules:
Workflow:
1. Identify issue:
- If `$ARGUMENTS` is a Sentry URL, fetch it with `get_sentry_resource(url=...)`.
- If `$ARGUMENTS` is a bare issue id, discover org/project if needed, then fetch issue with `get_sentry_resource(resourceType="issue", resourceId=<id>, organizationSlug=<org>)`.
- If no args, find org/project, search unresolved production issues with both frequency and user sorting, compare top results, and pick one with best impact score: high event count, high user count, recent lastSeen, production environment, clear actionable stack.
- Confirm auth and org/project access with `sentry auth status`, `sentry org list --json`, and `sentry project list <org> --json` when needed.
- If `$ARGUMENTS` is a Sentry URL, extract `/issues/<numeric-id>/` or the visible short issue id, then fetch it with `sentry issue view <issue> --json --fields id,shortId,title,culprit,permalink,level,status,substatus,count,userCount,firstSeen,lastSeen,project,metadata,priority,platform,isUnhandled`.
- If `$ARGUMENTS` is a bare issue id or short id, fetch it with `sentry issue view <issue> --json --fields id,shortId,title,culprit,permalink,level,status,substatus,count,userCount,firstSeen,lastSeen,project,metadata,priority,platform,isUnhandled`.
- If no args, list unresolved production issues with both frequency and user sorting, then compare top results:
- `sentry issue list <org>/<project> --query 'is:unresolved environment:production' --sort freq --limit 25 --json --fields id,shortId,title,culprit,count,userCount,lastSeen,permalink,priority,metadata,project`
- `sentry issue list <org>/<project> --query 'is:unresolved environment:production' --sort user --limit 25 --json --fields id,shortId,title,culprit,count,userCount,lastSeen,permalink,priority,metadata,project`
- Pick best impact score: high event count, high user count, recent lastSeen, production environment, clear actionable stack.
- Record chosen short issue id and Sentry URL in notes.
2. Branch:
- Derive branch name from Sentry short issue id only, e.g. `WHISPER-MONEY-123`.
- Run `git switch -c <issue-id>`; if branch exists, `git switch <issue-id>`.
3. Investigate:
- Fetch latest events, stack trace, breadcrumbs, environment, release, tags, URLs/routes, affected users count, and relevant traces/replays/profiles if present.
- Fetch latest issue details and spans with `sentry issue view <issue> --spans all --json`.
- Fetch recent events with `sentry issue events <issue> --limit 10 --full --json`.
- Extract stack trace, breadcrumbs, environment, release, tags, URLs/routes, affected users count, trace/span data, and suspect queries. Redact PII in notes.
- For event details if needed, use `sentry event view <event-id> --json` or `sentry api <endpoint> --json`.
- Reproduce locally using tests or focused command. Inspect app logs/browser logs if relevant.
- If root cause not obvious after issue/event data, run Seer analysis.
- If root cause not obvious after issue/event data, run Seer with `sentry issue explain <issue> --json` and/or `sentry issue plan <issue> --json`.
4. Fix:
- Read nearby code and conventions first.
- Implement minimal fix.

View File

@ -177,10 +177,15 @@ class AutomationRuleApplicationController extends Controller
$ids = [];
$eagerLoads = $service->eagerLoadsForRuleEvaluation($rule);
if ($onlyUncategorized && $rule->action_category_id === null) {
$eagerLoads[] = 'labels';
}
Transaction::query()
->where('user_id', $rule->user_id)
->whereNull('description_iv')
->with(['account.bank', 'category', 'labels'])
->with(array_values(array_unique($eagerLoads)))
->orderByDesc('transaction_date')
->orderByDesc('created_at')
->chunkById(500, function ($transactions) use ($rule, $service, $onlyUncategorized, &$ids) {

View File

@ -45,7 +45,7 @@ class AutomationRuleService
return false;
}
$transactionData = $this->prepareTransactionData($transaction);
$transactionData = $this->prepareTransactionData($transaction, $rule);
try {
$normalizedRulesJson = $this->normalizeRuleJson($rule->rules_json);
@ -91,15 +91,37 @@ class AutomationRuleService
return empty(array_diff($ruleLabelIds, $transactionLabelIds));
}
/**
* @return array<int, string>
*/
public function eagerLoadsForRuleEvaluation(AutomationRule $rule): array
{
$variables = $this->ruleVariables($rule);
$eagerLoads = [];
if (in_array('bank_name', $variables, true)) {
$eagerLoads[] = 'account.bank';
} elseif (in_array('account_name', $variables, true)) {
$eagerLoads[] = 'account';
}
if (in_array('category', $variables, true)) {
$eagerLoads[] = 'category';
}
return $eagerLoads;
}
/**
* @return array{description: string, amount: float, transaction_date: string, bank_name: string, account_name: string, category: string|null, notes: string|null}
*/
private function prepareTransactionData(Transaction $transaction): array
private function prepareTransactionData(Transaction $transaction, ?AutomationRule $rule = null): array
{
$transaction->loadMissing(['account.bank', 'category']);
$transaction->loadMissing($rule ? $this->eagerLoadsForRuleEvaluation($rule) : ['account.bank', 'category']);
$account = $transaction->account;
$bank = $account?->bank;
$account = $transaction->relationLoaded('account') ? $transaction->account : null;
$bank = $account?->relationLoaded('bank') ? $account->bank : null;
$category = $transaction->relationLoaded('category') ? $transaction->category : null;
$accountName = '';
if ($account && ! $account->encrypted) {
@ -112,7 +134,7 @@ class AutomationRuleService
'transaction_date' => $transaction->transaction_date->format('Y-m-d'),
'bank_name' => mb_strtolower($bank->name ?? ''),
'account_name' => mb_strtolower($accountName),
'category' => $transaction->category?->name,
'category' => $category?->name,
'notes' => $transaction->notes
? $this->normalizeWhitespace(mb_strtolower($transaction->notes))
: null,
@ -141,6 +163,35 @@ class AutomationRuleService
return null;
}
/**
* @return array<int, string>
*/
private function ruleVariables(AutomationRule $rule): array
{
$variables = [];
$this->collectRuleVariables($this->normalizeRuleJson($rule->rules_json), $variables);
return array_values(array_unique($variables));
}
/**
* @param array<int, string> $variables
*/
private function collectRuleVariables(mixed $ruleJson, array &$variables): void
{
if (! is_array($ruleJson)) {
return;
}
if (isset($ruleJson['var']) && is_string($ruleJson['var'])) {
$variables[] = $ruleJson['var'];
}
foreach ($ruleJson as $value) {
$this->collectRuleVariables($value, $variables);
}
}
private function applyActions(Transaction $transaction, AutomationRule $rule): bool
{
$changed = false;

View File

@ -11,6 +11,7 @@ use App\Models\Label;
use App\Models\Transaction;
use App\Models\User;
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Event;
use Illuminate\Support\Facades\Queue;
@ -86,6 +87,36 @@ test('matches endpoint skips already categorized when only_uncategorized is true
$allResponse->assertOk()->assertJsonPath('total', 2);
});
test('matches endpoint avoids repeated relationship queries for description-only rules', function () {
Transaction::factory()->enableBanking()->count(501)->create([
'user_id' => $this->user->id,
'account_id' => $this->account->id,
'category_id' => null,
'description' => 'Grocery Store',
'amount' => -1000,
]);
$queries = [];
DB::listen(function ($query) use (&$queries): void {
$queries[] = $query->sql;
});
$this->actingAs($this->user)
->getJson(route('automation-rules.matches', $this->rule))
->assertOk()
->assertJsonPath('total', 501);
$accountEagerLoadQueries = collect($queries)
->filter(fn (string $query): bool => (str_contains($query, 'from "accounts"') || str_contains($query, 'from `accounts`'))
&& (str_contains($query, '"accounts"."id" in') || str_contains($query, '`accounts`.`id` in')));
$bankEagerLoadQueries = collect($queries)
->filter(fn (string $query): bool => (str_contains($query, 'from "banks"') || str_contains($query, 'from `banks`'))
&& (str_contains($query, '"banks"."id" in') || str_contains($query, '`banks`.`id` in')));
expect($accountEagerLoadQueries)->toHaveCount(1)
->and($bankEagerLoadQueries)->toHaveCount(1);
});
test('matches endpoint deduplicates cached matching transaction ids', function () {
$transaction = Transaction::factory()->enableBanking()->create([
'user_id' => $this->user->id,