whisper-money/app/Console/Commands
Víctor Falcón 906e3cc2b4
feat(ai): add weekly AI-suggestions cohort report (#530)
## What

Adds `stats:ai-cohort-report` — a monthly scheduled command that posts a
**weekly per-cohort** retention/conversion time series for the
onboarding AI suggestions feature to Discord.

## Design (pre/post, directional — not causal)

We measure whether shipping AI suggestions moves retention/conversion
among the users who can actually receive it. This is an **observational
pre/post** readout, deliberately chosen over a randomized A/B holdout
(low signup volume; we don't want to withhold the feature). It reports a
**correlation**, never a cause.

- **Cohort (ITT):** users who imported **≥50 transactions in their first
7 days** (a pre-treatment trait). We do *not* condition on who accepted
AI — that would reintroduce self-selection.
- **Release anchor `R`:** `MIN(ai_consents.accepted_at)`, planted by
self-accepting on deploy.
- **Metrics, all fixed-horizon from each user's own signup:**
  - Retention — active ≥14d (`last_active_at ≥ signup+14d`)
  - Trial — subscribed ≤14d
  - Paid — `active` subscription ≤30d
  - AI-acceptance — funnel-health line (descriptive, not causal)
- **Right-censoring:** too-young cohorts render as `pend`, never `0`.
- **Surge flagging:** weeks with outlier eligible volume (>2.5× median)
are flagged  so a one-time acquisition spike (e.g. the launch/YouTube
surge) isn't misread as an organic trend.

Staff/test accounts (incl. the one planting the anchor consent) are
excluded via `config('ai_suggestions.report.excluded_emails')`. Output
goes to `DISCORD_AI_COHORT_WEBHOOK_URL`, falling back to the shared
`DISCORD_WEBHOOK_URL`.

## Honest caveat (baked into every report footer)

A one-time launch+YouTube signup surge sits right before release and
there's no acquisition-source tracking, so cross-release comparisons mix
channels. Read **organic-vs-organic cohort trends over a quarter**, not
a month-one before/after diff.

## Deploy steps

1. Self-accept the AI consent in prod to plant `R`.
2. Add your email (+ staff) to `AI_SUGGESTIONS_REPORT_EXCLUDED_EMAILS`.
3. Set `DISCORD_AI_COHORT_WEBHOOK_URL` (or rely on the existing admin
webhook).

## Tests

`tests/Feature/SendAiCohortReportCommandTest.php` — 7 tests / 25
assertions covering eligibility, retention, trial/paid windows, release
anchor + pre/post split, maturity censoring, surge flagging, and Discord
delivery (incl. webhook fallback). Pint clean.
2026-06-13 23:23:34 +02:00
..
Concerns refactor(real-estate): remove Pennant gating (#308) 2026-04-20 13:31:49 +01:00
AgentDatabaseCommand.php feat(console): add agent:db command for querying local and prod DB (#522) 2026-06-12 18:35:14 +02:00
ApplyRealEstateRevaluationCommand.php fix(real-estate): compound annual revaluation monthly (#337) 2026-04-27 07:35:51 +01:00
BackfillAccountIbans.php fix(banking): correct backfill-ibans endpoint and handle expired sessions gracefully (#222) 2026-03-12 11:57:54 +00:00
BackfillUserCurrencyCode.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
CancelFreeEnableBankingConnectionsCommand.php Support soft-deleted users with reusable emails (#316) 2026-04-22 11:41:41 +01:00
CheckBankLogosCommand.php Add weekly bank logo audit command (#211) 2026-03-06 11:05:13 +00:00
DeleteUserCommand.php Support soft-deleted users with reusable emails (#316) 2026-04-22 11:41:41 +01:00
DisconnectBankingConnectionsCommand.php feat(banking): add command to disconnect connections by id (#497) 2026-06-06 11:16:01 +02:00
E2eBankingFixtureCommand.php test(open-banking): e2e coverage for Enable Banking connection flows (#509) 2026-06-09 11:58:50 +02:00
EnsureLaunchCouponsCommand.php feat(leads): cohort-based launch invitations with per-user Stripe coupons (#333) 2026-04-30 15:10:28 +01:00
FeatureDisableCommand.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
FeatureEnableCommand.php Add Budgeting Feature to Track and Manage Spending (#36) 2026-01-21 15:25:50 +01:00
GenerateBudgetPeriods.php fix: Budget period not found on last day of period (#91) 2026-02-01 11:33:10 +01:00
GenerateLandingAuthLinkCommand.php feat(landing): add signed auth links (#312) 2026-04-21 08:28:59 +01:00
GenerateMonthlyLoanBalances.php feat(accounts): add loan amortization projections for loan accounts (#246) 2026-03-26 15:06:09 +01:00
GenerateStripePromotionCodesCommand.php feat(stripe): add promo code generator (#311) 2026-04-20 18:15:28 +01:00
ResendLeadVerificationEmailsCommand.php feat: resend verification emails to unverified leads (#287) 2026-04-15 09:13:27 +01:00
ResendSyncCommand.php feat: Sync new users to Resend contacts (#85) 2026-01-28 21:25:58 +01:00
ResendSyncLeadsCommand.php perf(resend): default sync-leads to last 24h window (#354) 2026-05-05 09:57:25 +01:00
ResetDemoAccountCommand.php fix(budgets): remove Custom period type to fix duplicate-key crash (#355) 2026-05-05 16:07:16 +02:00
ResetUserCategories.php Refactor: Optimize category management in CreateDefaultCategories and ResetUserCategories 2025-12-01 15:37:53 +01:00
RetryLeadFailedJobsCommand.php feat: selective retry of failed lead email jobs (#286) 2026-04-15 08:00:29 +01:00
SendAiCohortReportCommand.php feat(ai): add weekly AI-suggestions cohort report (#530) 2026-06-13 23:23:34 +02:00
SendDailyStatsReportCommand.php feat(discord): enrich Stripe event messages and dedupe deliveries (#460) 2026-05-31 12:50:37 +02:00
SendSubscriptionCancelledEmailCommand.php Add subscription cancellation email with manual trigger command (#47) 2025-12-30 20:32:44 +01:00
SendTestEmails.php chore(commands): unify command signatures 2025-12-30 07:22:18 +01:00
SendUpdateEmailCommand.php fix: delay emails to avoid reaching daily resend limit 2026-01-09 11:11:38 +01:00
SendUserLeadInvitations.php fix: keep lead invite command aliases (#406) 2026-05-20 12:26:21 +02:00
SendUserLeadReInvitations.php feat(leads): add user lead re-invite campaign (#432) 2026-05-26 08:35:31 +02:00
SetBankLogoCommand.php fix(banks:set-logo): add JPEG support test coverage and prompt for missing arguments (#214) 2026-03-07 16:33:54 +00:00
SetupMainUser.php fix(static-analysis): clear phpstan-baseline by fixing all suppressed errors (#183) 2026-03-02 12:22:30 +00:00
StripeSubscriptionStatsCommand.php feat: add Discord admin feed for daily stats and Stripe events (#458) 2026-05-30 18:14:46 +02:00
SuggestRulesCommand.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
SyncBankingConnections.php Notify users about expired bank connections (#404) 2026-05-20 09:20:13 +02:00
SyncStripePricesCommand.php chore: upgrade Laravel 12 to 13 (#242) 2026-03-25 12:56:33 +00:00
TestScheduledDripEmailCommand.php Test mail drip command 2025-12-30 07:22:18 +01:00