RomanNum3ral
/
whisper-money
mirror of https://github.com/whisper-money/whisper-money.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

AI auto-categorization: open to pro + consent, nudge free users (#561) 

## What

Two related changes to the AI auto-categorization feature.

### 1. Open the gate to pro + consent (drop the new-signups-only cohort)

Eligibility for AI auto-categorization was: kill switch **+ pro plan +
active AI consent + a Pennant rollout flag** that only resolved for
users created after `ai_categorization.rollout_after`. That last cohort
gate limited the feature to a handful of recent signups (6 eligible
users in prod).

The gate is now just **kill switch + pro plan + active AI consent**, so
every consented pro user is eligible regardless of signup date.

- `allows()` and `allowsBackfill()` became identical and collapse into a
single `allows()`; `CategorizeBackfillCommand` calls it.
- The `AiCategorization` Pennant feature and the
`ai_categorization.rollout_after` config are now dead and removed. The
weekly cohort report already derives its release marker from the first
`ai_consents.accepted_at`, so nothing depends on the config.

> Note: the `AI_CATEGORIZATION_ROLLOUT_AFTER` env var must be removed
from the production environment — it is no longer read.

### 2. Nudge free users that AI could categorize their transactions

Free-plan users now see a subtle AI sparkle on uncategorized rows,
reusing the trailing icon slot already in `CategoryCell` (no layout
change). It only shows when subscriptions are enforced and the user is
not pro; clicking it routes to `/settings/billing`.

To keep it subtle, the sparkle is sampled to a share of rows via a
deterministic function of the transaction id (its last byte mapped onto
a 0-100 threshold), so the same rows decide the same way across reloads
instead of flickering or marking every row.

The share is **configurable** via `ai_categorization.upsell_sample_rate`
(env `AI_CATEGORIZATION_UPSELL_SAMPLE_RATE`, default 40), exposed to the
frontend as the `aiCategorizationUpsellRate` Inertia prop — no rebuild
needed to retune it.

## Tests

- PHP: `AiCategorizationGateTest` updated (rollout case dropped);
job/listener tests no longer activate the removed feature.
- JS: `ai-upsell-sample.test.ts` covers determinism, the 0/100 bounds,
the threshold boundary, and the per-rate split.
- Manually verified in-app (Playwright): nudge renders for a free,
consented, bankless user; rate matches config.

## Follow-ups (not in this PR)

- Remove the `AI_CATEGORIZATION_ROLLOUT_AFTER` prod env var.
- Backfilling existing pro+consent users' history still requires running
`ai:categorize-backfill {user}`.
- The nudge only reaches free users who pass the paywall (no bank
connection + paywall already seen); free users with a bank connection
never reach `/transactions`. Consider surfacing it on the paywall too.
This commit is contained in:
Víctor Falcón 2026-06-19 16:08:40 +02:00 committed by GitHub
parent 6cb8d11563
commit ae59c90f2c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
14 changed files with 127 additions and 128 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
app/Console/Commands/CategorizeBackfillCommand.php
View File

@ -27,7 +27,7 @@ class CategorizeBackfillCommand extends Command
return self::FAILURE;
}
if (! $gate->allowsBackfill($user)) {
if (! $gate->allows($user)) {
$this->warn('User is not eligible (needs the feature enabled, a pro plan and active AI consent).');
return self::FAILURE;

35
app/Features/AiCategorization.php
View File

@ -1,35 +0,0 @@
<?php
namespace App\Features;
use App\Models\User;
use Illuminate\Support\Carbon;
/**
* Gates AI auto-categorization of transactions. Rolled out to users who signed
* up after `ai_categorization.rollout_after`; combined with the pro plan and
* AI consent checks in AiCategorizationGate.
*
* @api
*/
class AiCategorization
{
/**
* Resolve the feature's initial value.
*/
public function resolve(?User $user): bool
{
if ($user === null) {
return false;
}
$rolloutAfter = config('ai_categorization.rollout_after');
if (blank($rolloutAfter)) {
return false;
}
return $user->created_at !== null
&& $user->created_at->gt(Carbon::parse($rolloutAfter));
}
}

1
app/Http/Middleware/HandleInertiaRequests.php
View File

@ -92,6 +92,7 @@ class HandleInertiaRequests extends Middleware
'password' => config('app.demo.password'),
] : null,
'subscriptionsEnabled' => config('subscriptions.enabled', false),
'aiCategorizationUpsellRate' => (int) config('ai_categorization.upsell_sample_rate'),
'pricing' => [
'plans' => config('subscriptions.plans', []),
'defaultPlan' => config('subscriptions.default_plan', 'monthly'),

27
app/Services/Ai/AiCategorizationGate.php
View File

@ -2,39 +2,16 @@
namespace App\Services\Ai;
use App\Features\AiCategorization;
use App\Models\User;
use Laravel\Pennant\Feature;
/**
* The eligibility gate for AI auto-categorization: a hard config kill switch, a
* pro subscription, an active (current-version) AI consent, and the per-user
* Pennant rollout flag. All four must hold before any transaction is sent.
* pro subscription and an active (current-version) AI consent. All three must
* hold before any transaction is sent.
*/
class AiCategorizationGate
{
public function allows(User $user): bool
{
if (! (bool) config('ai_categorization.enabled')) {
return false;
}
if (! $user->hasProPlan()) {
return false;
}
if (! $user->hasActiveAiConsent()) {
return false;
}
return Feature::for($user)->active(AiCategorization::class);
}
/**
* Backfill is an explicit, user-initiated action, so it requires the kill
* switch, a pro plan and active consent — but not the gradual rollout flag.
*/
public function allowsBackfill(User $user): bool
{
return (bool) config('ai_categorization.enabled')
&& $user->hasProPlan()

28
config/ai_categorization.php
View File

@ -27,20 +27,6 @@ return [
'enabled' => (bool) env('AI_CATEGORIZATION_ENABLED', true),
/*
|--------------------------------------------------------------------------
| Rollout cohort
|--------------------------------------------------------------------------
|
| The AiCategorization Pennant feature resolves to active for users created
| strictly after this timestamp (parsed in the app timezone). This rolls the
| feature out to new signups only. Set to null to deactivate the cohort
| (the flag can still be activated per-user via Pennant directly).
|
*/
'rollout_after' => env('AI_CATEGORIZATION_ROLLOUT_AFTER', '2026-06-13 21:00:00'),
/*
|--------------------------------------------------------------------------
| Confidence bars
@ -83,4 +69,18 @@ return [
'queue' => env('AI_CATEGORIZATION_QUEUE', 'ai'),
/*
|--------------------------------------------------------------------------
| Free-plan upsell nudge
|--------------------------------------------------------------------------
|
| Percentage (0-100) of a free user's uncategorized transactions that show
| the "AI could categorize this" sparkle. Sampled deterministically by
| transaction id so the same rows always decide the same way. Exposed to the
| frontend as the `aiCategorizationUpsellRate` Inertia prop.
|
*/
'upsell_sample_rate' => (int) env('AI_CATEGORIZATION_UPSELL_SAMPLE_RATE', 40),
];

1
lang/es.json
View File

@ -31,6 +31,7 @@
"You have reached your monthly limit of :count integration actions. Try again next month.": "Has alcanzado tu límite mensual de :count acciones de integración. Inténtalo de nuevo el mes que viene.",
"Remove one vote": "Quitar un voto",
"Categorized by AI": "Categorizado por IA",
"Let AI categorize your transactions": "Deja que la IA categorice tus transacciones",
"Categorized by AI with :confidence% confident": "Categorizado por IA con un :confidence % de confianza",
"Only show AI guesses": "Mostrar solo sugerencias de IA",
"Created by AI": "Creado por IA",

42
resources/js/components/transactions/ai-upsell-sample.test.ts Normal file
View File

@ -0,0 +1,42 @@
import { describe, expect, it } from 'vitest';
import { showsAiUpsell } from './ai-upsell-sample';
describe('showsAiUpsell', () => {
it('is deterministic for the same id and rate', () => {
const id = '019ec83c-9837-7245-8717-ae66b6dd9957';
expect(showsAiUpsell(id, 33)).toBe(showsAiUpsell(id, 33));
});
it('shows nothing at 0 and everything at 100', () => {
const id = '019ec83c-9837-7245-8717-ae66b6dd9957';
expect(showsAiUpsell(id, 0)).toBe(false);
expect(showsAiUpsell(id, 100)).toBe(true);
});
it('thresholds on the id last byte', () => {
// 0x54 = 84, below the 33% cutoff (84.48) -> shown
expect(showsAiUpsell('00000000-0000-0000-0000-000000000054', 33)).toBe(
true,
);
// 0x55 = 85, at/above the cutoff -> hidden
expect(showsAiUpsell('00000000-0000-0000-0000-000000000055', 33)).toBe(
false,
);
});
it('samples about the configured rate across all 256 final bytes', () => {
const rateShown = (pct: number) =>
Array.from({ length: 256 }, (_, n) =>
showsAiUpsell(
`00000000-0000-0000-0000-0000000000${n.toString(16).padStart(2, '0')}`,
pct,
),
).filter(Boolean).length;
expect(rateShown(33)).toBe(85); // bytes 0..84
expect(rateShown(50)).toBe(128); // bytes 0..127
expect(rateShown(75)).toBe(192); // bytes 0..191
});
});

21
resources/js/components/transactions/ai-upsell-sample.ts Normal file
View File

@ -0,0 +1,21 @@
/**
* Deterministic sampler keyed on a transaction id, so the AI-categorization
* upsell sparkle shows on the same rows across reloads instead of flickering and
* does not appear on every uncategorized row.
*
* The share is configurable: `samplePercent` (0-100) comes from the
* `aiCategorizationUpsellRate` Inertia prop, backed by
* `ai_categorization.upsell_sample_rate` (env: AI_CATEGORIZATION_UPSELL_SAMPLE_RATE).
*
* ponytail: map the id's last byte (0-255) onto the 0-100 threshold. Uniform
* enough for a cosmetic nudge; swap for a real hash if the split must be exact.
*/
export function showsAiUpsell(
transactionId: string,
samplePercent: number,
): boolean {
const lastByte = parseInt(transactionId.replace(/-/g, '').slice(-2), 16);
const byte = Number.isNaN(lastByte) ? 256 : lastByte; // 256 → never shows
return byte < (samplePercent / 100) * 256;
}

46
resources/js/components/transactions/category-cell.tsx
View File

@ -1,3 +1,4 @@
import { showsAiUpsell } from '@/components/transactions/ai-upsell-sample';
import { CategorySelect } from '@/components/transactions/category-select';
import { AiSparkleIcon } from '@/components/ui/ai-sparkle-icon';
import {
@ -8,11 +9,14 @@ import {
} from '@/components/ui/tooltip';
import { useIsMobile } from '@/hooks/use-mobile';
import { cn } from '@/lib/utils';
import { billing } from '@/routes/settings';
import { transactionSyncService } from '@/services/transaction-sync';
import { type SharedData } from '@/types';
import { type Account, type Bank } from '@/types/account';
import { type Category } from '@/types/category';
import { type DecryptedTransaction } from '@/types/transaction';
import { __ } from '@/utils/i18n';
import { router, usePage } from '@inertiajs/react';
import { useState } from 'react';
interface CategoryCellProps {
@ -42,6 +46,16 @@ export function CategoryCell({
}: CategoryCellProps) {
const [isUpdating, setIsUpdating] = useState(false);
const isMobile = useIsMobile();
const { auth, subscriptionsEnabled, aiCategorizationUpsellRate } =
usePage<SharedData>().props;
// Free-plan nudge: AI could categorize this row. Sampled to a configurable
// share of rows so it stays subtle instead of marking every uncategorized one.
const showAiUpsell =
!transaction.category_id &&
subscriptionsEnabled &&
!auth.hasProPlan &&
showsAiUpsell(transaction.id, aiCategorizationUpsellRate);
async function handleCategoryChange(value: string) {
const categoryId = value === 'null' ? null : value;
@ -123,6 +137,29 @@ export function CategoryCell({
</span>
);
const aiUpsell = (
<TooltipProvider>
<Tooltip>
<TooltipTrigger asChild>
<button
type="button"
onClick={(event) => {
event.stopPropagation();
router.visit(billing.url());
}}
className="inline-flex"
aria-label={__('Let AI categorize your transactions')}
>
<AiSparkleIcon className="h-3.5 w-3.5 opacity-50 transition-opacity hover:opacity-100" />
</button>
</TooltipTrigger>
<TooltipContent>
{__('Let AI categorize your transactions')}
</TooltipContent>
</Tooltip>
</TooltipProvider>
);
return (
<div className="flex w-full items-center gap-1">
<div className="min-w-0 flex-1">
@ -151,8 +188,8 @@ export function CategoryCell({
horizontally on every row. Desktop shows confidence on hover;
mobile relies on the dropdown header. */}
<span className="flex w-3.5 shrink-0 items-center justify-center">
{isAiCategorized &&
(isMobile ? (
{isAiCategorized ? (
isMobile ? (
aiIcon
) : (
<TooltipProvider>
@ -163,7 +200,10 @@ export function CategoryCell({
<TooltipContent>{aiNote}</TooltipContent>
</Tooltip>
</TooltipProvider>
))}
)
) : showAiUpsell ? (
aiUpsell
) : null}
</span>
</div>
);

1
resources/js/types/index.d.ts vendored
View File

@ -78,6 +78,7 @@ export interface SharedData {
includeLoansInNetWorthChart: boolean;
includeRealEstateInNetWorthChart: boolean;
subscriptionsEnabled: boolean;
aiCategorizationUpsellRate: number;
subscriptionPaymentIssue: SubscriptionPaymentIssueNotification | null;
pricing: PricingConfig;
sidebarOpen: boolean;

29
tests/Feature/Ai/AiCategorizationFeatureTest.php
View File

@ -1,29 +0,0 @@
<?php
use App\Features\AiCategorization;
use App\Models\User;
use Laravel\Pennant\Feature;
beforeEach(function () {
config()->set('ai_categorization.rollout_after', '2026-06-13 21:00:00');
});
it('activates for users created after the rollout cutoff', function () {
$user = User::factory()->create(['created_at' => '2026-06-13 21:00:01']);
expect(Feature::for($user)->active(AiCategorization::class))->toBeTrue();
});
it('stays inactive for users created before the rollout cutoff', function () {
$user = User::factory()->create(['created_at' => '2026-06-13 20:59:59']);
expect(Feature::for($user)->active(AiCategorization::class))->toBeFalse();
});
it('is inactive when no rollout cutoff is configured', function () {
config()->set('ai_categorization.rollout_after', null);
$user = User::factory()->create(['created_at' => '2026-06-14 00:00:00']);
expect(Feature::for($user)->active(AiCategorization::class))->toBeFalse();
});

16
tests/Feature/Ai/AiCategorizationGateTest.php
View File

@ -1,20 +1,17 @@
<?php
use App\Features\AiCategorization;
use App\Models\User;
use App\Services\Ai\AiCategorizationGate;
use Laravel\Pennant\Feature;
function eligibleUser(): User
{
$user = User::factory()->create();
$user->recordAiConsent();
Feature::for($user)->activate(AiCategorization::class);
return $user;
}
it('allows an enabled, pro, consented, flagged user', function () {
it('allows an enabled, pro, consented user', function () {
expect(app(AiCategorizationGate::class)->allows(eligibleUser()))->toBeTrue();
});
@ -26,17 +23,6 @@ it('denies when the master kill switch is off', function () {
it('denies a user without active AI consent', function () {
$user = User::factory()->create();
Feature::for($user)->activate(AiCategorization::class);
expect(app(AiCategorizationGate::class)->allows($user))->toBeFalse();
});
it('denies a user the rollout flag is not active for', function () {
config()->set('ai_categorization.rollout_after', '2026-06-13 21:00:00');
// Signed up before the rollout cutoff, so the feature does not resolve on.
$user = User::factory()->create(['created_at' => '2026-06-13 20:00:00']);
$user->recordAiConsent();
expect(app(AiCategorizationGate::class)->allows($user))->toBeFalse();
});

3
tests/Feature/Ai/CategorizeOnboardingTransactionsJobTest.php
View File

@ -4,19 +4,16 @@ use App\Ai\Agents\TransactionCategorizationAgent;
use App\Enums\CategoryCashflowDirection;
use App\Enums\CategorySource;
use App\Enums\CategoryType;
use App\Features\AiCategorization;
use App\Jobs\CategorizeOnboardingTransactionsJob;
use App\Models\Category;
use App\Models\Transaction;
use App\Models\User;
use App\Services\Ai\CategoryCatalog;
use Laravel\Pennant\Feature;
function eligibleOnboardedUser(): User
{
$user = User::factory()->onboarded()->create();
$user->recordAiConsent();
Feature::for($user)->activate(AiCategorization::class);
return $user;
}

3
tests/Feature/Ai/CategorizeTransactionWithAiTest.php
View File

@ -4,18 +4,15 @@ use App\Ai\Agents\TransactionCategorizationAgent;
use App\Enums\CategoryCashflowDirection;
use App\Enums\CategorySource;
use App\Enums\CategoryType;
use App\Features\AiCategorization;
use App\Models\Category;
use App\Models\Transaction;
use App\Models\User;
use App\Services\Ai\CategoryCatalog;
use Laravel\Pennant\Feature;
function eligible(): User
{
$user = User::factory()->onboarded()->create();
$user->recordAiConsent();
Feature::for($user)->activate(AiCategorization::class);
return $user;
}