fix(ai): don't report expected transient provider overloads (PHP-LARAVEL-44) (#655)
## Issue (Sentry PHP-LARAVEL-44 — low volume: 0 users, 1 event)
`Laravel\Ai\Exceptions\ProviderOverloadedException: AI provider [gemini]
is overloaded` (underlying Gemini HTTP 503 "high demand"), thrown from
the AI rule-suggestion generator. This is an expected, transient,
self-healing provider overload — but
`LaravelAiRuleSuggestionGenerator::generate()` called `report()` on
every failed batch, so a transient hiccup surfaced in Sentry as an
error.
## Fix
Split the per-batch catch so a `FailoverableException` (overload /
rate-limit / insufficient-credits) is logged at `warning` **without**
reporting, while every other `Throwable` is still `report()`ed. This
mirrors the existing, test-enforced handling in the sibling
`CategorizeTransactions` service.
Unchanged (deliberately preserved):
- **Partial tolerance** — suggestions from batches that succeeded are
still kept.
- **Rethrow-when-all-fail** — a genuine total failure still rethrows, so
the run is marked `Failed` (the onboarding "Try again / Skip" UI) and is
reported **once** at the run level. Non-transient batch errors are still
reported immediately.
## What I deliberately did NOT do
Two review agents converged that adding a **retry backoff** (the other
candidate fix) is not worth it here and carries real risk:
- The run **already self-heals**: failed/empty runs don't count toward
the throttle, so the user's "Try again" button re-runs immediately at
zero cost.
- A synchronous per-batch `sleep` on the single-worker `database` queue,
across up to ~10 batches, could push the run past its **120s job timeout
(failing more often, not less)** and starve other queued jobs during a
provider brownout.
So this PR is scoped to the safe, high-signal change: stop reporting an
expected transient as an error. Genuine misconfigurations (bad
model/key/quota) throw non-`FailoverableException` types and remain
fully reported.
## Testing
- New: an expected transient overload returns the successful batches'
suggestions and is **not** reported
(`Exceptions::assertNothingReported()`); an unexpected batch failure
**is** reported (`assertReported(RuntimeException::class)`).
- Full `tests/Feature/Ai` suite green (121 tests); Pint and Larastan
clean.
Fixes PHP-LARAVEL-44
---
🤖 Opened by the autonomous Sentry-triage loop. Auto-merge enabled:
low-risk, mirrors an existing tested pattern, keeps genuine failures
visible.
This commit is contained in:
parent
9b7632f585
commit
d504e70309
|
|
@ -4,7 +4,9 @@ namespace App\Services\Ai;
|
|||
|
||||
use App\Ai\Agents\RuleSuggestionAgent;
|
||||
use App\Services\Ai\Contracts\RuleSuggestionGenerator;
|
||||
use Illuminate\Support\Facades\Log;
|
||||
use Laravel\Ai\Enums\Lab;
|
||||
use Laravel\Ai\Exceptions\FailoverableException;
|
||||
use Throwable;
|
||||
|
||||
class LaravelAiRuleSuggestionGenerator implements RuleSuggestionGenerator
|
||||
|
|
@ -27,6 +29,17 @@ class LaravelAiRuleSuggestionGenerator implements RuleSuggestionGenerator
|
|||
foreach ($this->generateBatchWithRetry($batch, $categoryOptions) as $suggestion) {
|
||||
$suggestions[] = $suggestion;
|
||||
}
|
||||
} catch (FailoverableException $exception) {
|
||||
// An overloaded or rate-limited provider is an expected transient
|
||||
// condition, not a bug (PHP-LARAVEL-44). Count it as a failure so
|
||||
// an all-transient run still surfaces, but don't report the
|
||||
// per-batch noise — a genuine total failure is still reported once
|
||||
// by the run-level handler. Mirrors CategorizeTransactions.
|
||||
$failures++;
|
||||
$lastError = $exception;
|
||||
Log::warning('AI rule-suggestion batch dropped: provider transient failure.', [
|
||||
'exception' => $exception->getMessage(),
|
||||
]);
|
||||
} catch (Throwable $exception) {
|
||||
// A single batch failing must not discard the suggestions from
|
||||
// the batches that did succeed (a run can span many batches).
|
||||
|
|
|
|||
|
|
@ -2,6 +2,8 @@
|
|||
|
||||
use App\Ai\Agents\RuleSuggestionAgent;
|
||||
use App\Services\Ai\LaravelAiRuleSuggestionGenerator;
|
||||
use Illuminate\Support\Facades\Exceptions;
|
||||
use Laravel\Ai\Exceptions\ProviderOverloadedException;
|
||||
|
||||
it('returns the structured suggestions produced by the model', function () {
|
||||
RuleSuggestionAgent::fake([
|
||||
|
|
@ -95,6 +97,55 @@ it('keeps successful batches when one batch fails after retry', function () {
|
|||
->and($suggestions[0]['match_token'])->toBe('okkey');
|
||||
});
|
||||
|
||||
it('does not report an expected transient provider overload', function () {
|
||||
config()->set('ai_suggestions.group_batch_size', 1);
|
||||
|
||||
Exceptions::fake();
|
||||
|
||||
RuleSuggestionAgent::fake(function (string $prompt) {
|
||||
if (str_contains($prompt, 'boomtoken')) {
|
||||
throw ProviderOverloadedException::forProvider('gemini');
|
||||
}
|
||||
|
||||
return ['suggestions' => [genSuggestion('okkey')]];
|
||||
});
|
||||
|
||||
$generator = new LaravelAiRuleSuggestionGenerator;
|
||||
|
||||
$suggestions = $generator->generate(
|
||||
groups: [benchGroup('boomtoken'), benchGroup('goodkey')],
|
||||
categoryOptions: [['id' => 'cat-1', 'name' => 'X', 'path' => 'X', 'type' => 'expense', 'direction' => 'outflow', 'is_leaf' => true]],
|
||||
);
|
||||
|
||||
expect($suggestions)->toHaveCount(1)
|
||||
->and($suggestions[0]['match_token'])->toBe('okkey');
|
||||
|
||||
Exceptions::assertNothingReported();
|
||||
});
|
||||
|
||||
it('reports an unexpected batch failure', function () {
|
||||
config()->set('ai_suggestions.group_batch_size', 1);
|
||||
|
||||
Exceptions::fake();
|
||||
|
||||
RuleSuggestionAgent::fake(function (string $prompt) {
|
||||
if (str_contains($prompt, 'boomtoken')) {
|
||||
throw new RuntimeException('batch failed');
|
||||
}
|
||||
|
||||
return ['suggestions' => [genSuggestion('okkey')]];
|
||||
});
|
||||
|
||||
$generator = new LaravelAiRuleSuggestionGenerator;
|
||||
|
||||
$generator->generate(
|
||||
groups: [benchGroup('boomtoken'), benchGroup('goodkey')],
|
||||
categoryOptions: [['id' => 'cat-1', 'name' => 'X', 'path' => 'X', 'type' => 'expense', 'direction' => 'outflow', 'is_leaf' => true]],
|
||||
);
|
||||
|
||||
Exceptions::assertReported(RuntimeException::class);
|
||||
});
|
||||
|
||||
it('rethrows when every batch fails', function () {
|
||||
config()->set('ai_suggestions.group_batch_size', 1);
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue