whisper-money/config
Víctor Falcón 906e3cc2b4
feat(ai): add weekly AI-suggestions cohort report (#530)
## What

Adds `stats:ai-cohort-report` — a monthly scheduled command that posts a
**weekly per-cohort** retention/conversion time series for the
onboarding AI suggestions feature to Discord.

## Design (pre/post, directional — not causal)

We measure whether shipping AI suggestions moves retention/conversion
among the users who can actually receive it. This is an **observational
pre/post** readout, deliberately chosen over a randomized A/B holdout
(low signup volume; we don't want to withhold the feature). It reports a
**correlation**, never a cause.

- **Cohort (ITT):** users who imported **≥50 transactions in their first
7 days** (a pre-treatment trait). We do *not* condition on who accepted
AI — that would reintroduce self-selection.
- **Release anchor `R`:** `MIN(ai_consents.accepted_at)`, planted by
self-accepting on deploy.
- **Metrics, all fixed-horizon from each user's own signup:**
  - Retention — active ≥14d (`last_active_at ≥ signup+14d`)
  - Trial — subscribed ≤14d
  - Paid — `active` subscription ≤30d
  - AI-acceptance — funnel-health line (descriptive, not causal)
- **Right-censoring:** too-young cohorts render as `pend`, never `0`.
- **Surge flagging:** weeks with outlier eligible volume (>2.5× median)
are flagged ⚡ so a one-time acquisition spike (e.g. the launch/YouTube
surge) isn't misread as an organic trend.

Staff/test accounts (incl. the one planting the anchor consent) are
excluded via `config('ai_suggestions.report.excluded_emails')`. Output
goes to `DISCORD_AI_COHORT_WEBHOOK_URL`, falling back to the shared
`DISCORD_WEBHOOK_URL`.

## Honest caveat (baked into every report footer)

A one-time launch+YouTube signup surge sits right before release and
there's no acquisition-source tracking, so cross-release comparisons mix
channels. Read **organic-vs-organic cohort trends over a quarter**, not
a month-one before/after diff.

## Deploy steps

1. Self-accept the AI consent in prod to plant `R`.
2. Add your email (+ staff) to `AI_SUGGESTIONS_REPORT_EXCLUDED_EMAILS`.
3. Set `DISCORD_AI_COHORT_WEBHOOK_URL` (or rely on the existing admin
webhook).

## Tests

`tests/Feature/SendAiCohortReportCommandTest.php` — 7 tests / 25
assertions covering eligibility, retention, trial/paid windows, release
anchor + pre/post split, maturity censoring, surge flagging, and Discord
delivery (incl. webhook fallback). Pint clean.
2026-06-13 23:23:34 +02:00
..
ai_suggestions.php feat(ai): add weekly AI-suggestions cohort report (#530) 2026-06-13 23:23:34 +02:00
app.php Remove encryption from browser tests and demo user (#129) 2026-02-17 11:45:27 +01:00
auth.php chore: upgrade Laravel 12 to 13 (#242) 2026-03-25 12:56:33 +00:00
cache.php chore: upgrade Laravel 12 to 13 (#242) 2026-03-25 12:56:33 +00:00
cashier.php feat(pricing): dynamic Stripe pricing with locale-aware formatting (#204) 2026-03-05 11:41:59 +00:00
currencies.php feat(currency): add NZD (New Zealand Dollar) (#504) 2026-06-08 09:18:17 +02:00
database.php feat(console): add agent:db command for querying local and prod DB (#522) 2026-06-12 18:35:14 +02:00
filesystems.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
fortify.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
inertia.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
landing.php feat(landing): add signed auth links (#312) 2026-04-21 08:28:59 +01:00
logging.php fix(logging): keep laravel.log writable across container UIDs (#451) 2026-05-29 15:10:50 +02:00
mail.php feat(mail): use AWS SES for email delivery (#422) 2026-05-25 10:59:57 +02:00
pennant.php Add Cashflow Analytics Feature (#49) 2026-01-05 13:06:50 +01:00
queue.php Reapply "swap horizon -> queue:work on mysql" 2025-12-30 07:22:18 +01:00
sentry.php fix(sentry): only report errors in production (#467) 2026-06-01 12:41:31 +02:00
services.php feat(ai): add weekly AI-suggestions cohort report (#530) 2026-06-13 23:23:34 +02:00
session.php Set up a fresh Laravel app 2025-11-07 12:01:36 +00:00
subscriptions.php feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00