whisper-money/tests/Browser
Víctor Falcón b6bc7756bc
feat(subscriptions): end the trial experiment and make the trial length per plan (#762)
Ends the trial/pricing A/B/C experiment. Everyone gets the control offer
— a free trial — and the trial length becomes a per-plan setting.

## Trial length

| Plan | Before | Now | Env override |
|---|---|---|---|
| Yearly | 15 days | **15 days** | `STRIPE_PRO_YEARLY_TRIAL_DAYS` |
| Monthly | 15 days | **7 days** | `STRIPE_PRO_MONTHLY_TRIAL_DAYS` |

**Note that monthly 15 → 7 is a new bet, not a rollback.** The control
arm was 15 days on both plans, and 7 days on monthly is a value the
experiment never tested (`reduced_trial` was monthly 3 / yearly 7). The
rationale is that the longer commitment earns the longer trial; it ships
here at the same time as the instrument that could measure it is
removed, so it will not be measurable as an isolated effect.

## Final experiment numbers

Archived here because `stats:experiment-funnel` and its collector are
deleted by this PR and the purge migration's `down()` is a no-op.

| Variant | Assigned | Subscribed | Active | Refunded |
|---|---|---|---|---|
| control | 597 | 46 | 13 | 0 |
| reduced_trial | 590 | 50 | 10 | 0 |
| pay_now | 609 | 41 | 21 | 18 |
| legacy | 94 | 55 | 31 | 0 |

## What is deleted

- `App\Features\SubscriptionExperiment` (the Pennant A/B/C assignment)
and the `ExperimentOffer` service.
- The `pay_now` self-service refund: `RefundSelfServe`, the
`settings.billing.refund` route, the controller actions and Discord
embeds, the money-back card in billing settings, and the
`stripe:verify-refund` sandbox command.
- The weekly `stats:experiment-funnel` report, its collector, and the
`ProportionSignificance` / `BinomialProportion` helpers it was the only
caller of, plus its schedule entry.
- The `subscriptions.experiment.*` config block and the orphaned
`es`/`fr` translation strings.
- A data migration purges the ~1,890 stored Pennant assignments.

`subscriptions.refunded_at` is deliberately **kept**: nothing reads it
anymore, but it is the only record of the 18 refunds the experiment
issued. The migration carries a comment saying so.

## Fixes found in review

- **The surviving funnel report was mis-scoring conversions.**
`SubscriptionFunnelCollector` compared every cancellation to one global
trial length. With trials now diverging per plan, a monthly subscriber
who was billed and cancelled on day 10 was scored as never having paid.
It now reads each subscription's own `trial_ends_at`, and the longest
plan trial is used only for deciding when a cohort is old enough to
score. Covered by two new tests.
- **The trial length swapped silently.** It lived on a single line under
the plan selector, which rewrote itself when the user switched plan. Now
that the plans genuinely differ, each plan card shows its own length.
- The report legend no longer quotes a single trial length for both
plans, and warns that the experiment weeks are still inside its window.

## Before merging

- [x] **Unset `SUBSCRIPTION_EXPERIMENT_STARTED_AT` in production** so no
new `pay_now` assignment happens while this waits. Anyone who checks out
under `pay_now` between now and the deploy is charged upfront and then
loses the one-tap refund they were promised at the point of payment.
Checked just before opening this PR: **0 `pay_now` subscriptions
currently inside the 3-day window**, so nobody is stranded today.
- [x] Drop the now-orphaned `SUBSCRIPTION_EXPERIMENT_*` variables from
the production env with the deploy.
- [x] If old containers are still serving while the purge migration
runs, a few assignments can be re-resolved and reappear. Harmless —
re-run `php artisan pennant:purge "App\Features\SubscriptionExperiment"`
once the deploy settles if you want the table clean.

Support note: a manual Stripe refund for a former `pay_now` user will
not disconnect their bank connections, which the automated flow used to
do.

## Demo


https://github.com/user-attachments/assets/3614d488-05c6-405d-a687-bbf45746879a


<!-- PLACEHOLDER: drag the QA video here -->

## QA

Browser-tested against the running app:

- Paywall: annual card shows "15 days free", monthly card "7 days free";
the terms line under the selector follows the selected plan (15 ↔ 7);
mobile viewport renders fine.
- Billing settings: no money-back card for a free user or an active
subscriber; `POST /settings/billing/refund` returns 404.
- No console or network errors on any screen.
- `stats:subscription-funnel` still renders and posts.
- The purge migration leaves 0 `SubscriptionExperiment` rows.

Full suite green (2045 tests) apart from the known local-only
`DashboardTest` 409; `pint`, `lint`, `format` and `build` all clean.
2026-08-12 10:59:55 +02:00
..
assets Improve automation rules and sync error handling 2025-12-01 10:30:22 +01:00
live feat(ai): suggest automation rules during onboarding (#523) 2026-06-13 22:51:15 +02:00
.gitignore Add category type field support (#2) 2025-12-01 20:19:47 +01:00
AccountsPageTest.php test: expand browser coverage for accounts and linked loans (#277) 2026-04-13 09:33:18 +01:00
AmountInputTest.php fix(transactions): only auto-select account on account pages (#549) 2026-06-17 11:28:31 +00:00
AuthenticationTest.php Support soft-deleted users with reusable emails (#316) 2026-04-22 11:41:41 +01:00
AutomationRuleBuilderTest.php feat: parent/child category tree (#474) 2026-06-03 19:30:12 +02:00
BankAccountsTest.php feat(loans): backfill historical balances on loan creation (#322) 2026-04-24 13:09:34 +01:00
BankConnectionFlowTest.php test(open-banking): e2e coverage for Enable Banking connection flows (#509) 2026-06-09 11:58:50 +02:00
BudgetCrudTest.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
BudgetsFeatureNavigationTest.php feat(budgets): track multiple categories and labels per budget (#466) 2026-06-01 12:32:23 +02:00
CashflowCategoryNavigationTest.php feat(cashflow): make income/expense category rows clickable to transactions (#234) 2026-03-17 10:57:40 +00:00
CatchAllBudgetCrudTest.php feat: add catch-all budgets (#527) 2026-06-15 16:07:19 +00:00
CategoriesTest.php fix(categories): expose cashflow setting on create (#448) 2026-05-29 11:05:04 +02:00
CategoryCorrectionLearningTest.php feat(ai): learn from category corrections so the AI stops repeating the same mistake (#608) 2026-06-29 19:12:15 +02:00
ImportTransactionsTest.php Remove encryption from browser tests and demo user (#129) 2026-02-17 11:45:27 +01:00
LandingPageTest.php feat(landing): stop auto-redirecting signed-in visitors to the dashboard (#777)a 2026-08-11 14:02:18 +00:00
OnboardingFlowTest.php feat(onboarding): auto-enable AI for connected banks, ask the rest (#618) 2026-07-01 07:26:52 +00:00
RealEstateAccountTest.php refactor(real-estate): remove Pennant gating (#308) 2026-04-20 13:31:49 +01:00
TransactionsTest.php Fix label creation dropdown refresh (#363) 2026-05-07 10:26:43 +01:00