Ends the trial/pricing A/B/C experiment. Everyone gets the control offer
— a free trial — and the trial length becomes a per-plan setting.
## Trial length
| Plan | Before | Now | Env override |
|---|---|---|---|
| Yearly | 15 days | **15 days** | `STRIPE_PRO_YEARLY_TRIAL_DAYS` |
| Monthly | 15 days | **7 days** | `STRIPE_PRO_MONTHLY_TRIAL_DAYS` |
**Note that monthly 15 → 7 is a new bet, not a rollback.** The control
arm was 15 days on both plans, and 7 days on monthly is a value the
experiment never tested (`reduced_trial` was monthly 3 / yearly 7). The
rationale is that the longer commitment earns the longer trial; it ships
here at the same time as the instrument that could measure it is
removed, so it will not be measurable as an isolated effect.
## Final experiment numbers
Archived here because `stats:experiment-funnel` and its collector are
deleted by this PR and the purge migration's `down()` is a no-op.
| Variant | Assigned | Subscribed | Active | Refunded |
|---|---|---|---|---|
| control | 597 | 46 | 13 | 0 |
| reduced_trial | 590 | 50 | 10 | 0 |
| pay_now | 609 | 41 | 21 | 18 |
| legacy | 94 | 55 | 31 | 0 |
## What is deleted
- `App\Features\SubscriptionExperiment` (the Pennant A/B/C assignment)
and the `ExperimentOffer` service.
- The `pay_now` self-service refund: `RefundSelfServe`, the
`settings.billing.refund` route, the controller actions and Discord
embeds, the money-back card in billing settings, and the
`stripe:verify-refund` sandbox command.
- The weekly `stats:experiment-funnel` report, its collector, and the
`ProportionSignificance` / `BinomialProportion` helpers it was the only
caller of, plus its schedule entry.
- The `subscriptions.experiment.*` config block and the orphaned
`es`/`fr` translation strings.
- A data migration purges the ~1,890 stored Pennant assignments.
`subscriptions.refunded_at` is deliberately **kept**: nothing reads it
anymore, but it is the only record of the 18 refunds the experiment
issued. The migration carries a comment saying so.
## Fixes found in review
- **The surviving funnel report was mis-scoring conversions.**
`SubscriptionFunnelCollector` compared every cancellation to one global
trial length. With trials now diverging per plan, a monthly subscriber
who was billed and cancelled on day 10 was scored as never having paid.
It now reads each subscription's own `trial_ends_at`, and the longest
plan trial is used only for deciding when a cohort is old enough to
score. Covered by two new tests.
- **The trial length swapped silently.** It lived on a single line under
the plan selector, which rewrote itself when the user switched plan. Now
that the plans genuinely differ, each plan card shows its own length.
- The report legend no longer quotes a single trial length for both
plans, and warns that the experiment weeks are still inside its window.
## Before merging
- [x] **Unset `SUBSCRIPTION_EXPERIMENT_STARTED_AT` in production** so no
new `pay_now` assignment happens while this waits. Anyone who checks out
under `pay_now` between now and the deploy is charged upfront and then
loses the one-tap refund they were promised at the point of payment.
Checked just before opening this PR: **0 `pay_now` subscriptions
currently inside the 3-day window**, so nobody is stranded today.
- [x] Drop the now-orphaned `SUBSCRIPTION_EXPERIMENT_*` variables from
the production env with the deploy.
- [x] If old containers are still serving while the purge migration
runs, a few assignments can be re-resolved and reappear. Harmless —
re-run `php artisan pennant:purge "App\Features\SubscriptionExperiment"`
once the deploy settles if you want the table clean.
Support note: a manual Stripe refund for a former `pay_now` user will
not disconnect their bank connections, which the automated flow used to
do.
## Demo
https://github.com/user-attachments/assets/3614d488-05c6-405d-a687-bbf45746879a
<!-- PLACEHOLDER: drag the QA video here -->
## QA
Browser-tested against the running app:
- Paywall: annual card shows "15 days free", monthly card "7 days free";
the terms line under the selector follows the selected plan (15 ↔ 7);
mobile viewport renders fine.
- Billing settings: no money-back card for a free user or an active
subscriber; `POST /settings/billing/refund` returns 404.
- No console or network errors on any screen.
- `stats:subscription-funnel` still renders and posts.
- The purge migration leaves 0 `SubscriptionExperiment` rows.
Full suite green (2045 tests) apart from the known local-only
`DashboardTest` 409; `pint`, `lint`, `format` and `build` all clean.
## What
The admin Discord channel is Spanish, but the four scheduled `stats:*`
reports posted English. They now post Spanish, and the three
cohort/experiment reports open with a short AI-written summary so a
reader understands the situation without decoding the table.
### 1. Spanish reports
`stats:daily-report`, `stats:subscription-funnel`,
`stats:experiment-funnel` and `stats:ai-cohort-report` now post Spanish
embed titles, field names, ASCII table headers, legends and disclaimers,
with Spanish dates (`sáb., 13 jun. 2026`). Table headers and cells stay
ASCII (`Semana`, `Variante`, `UMad`, `pdte`, …) so `sprintf`'s byte
padding keeps the columns aligned inside the code block.
Hardcoded, not `__()`: this is an internal channel, not user-facing UI,
so it never needs a second language and doesn't belong in
`lang/es.json`. Everything else — code, comments, PHPDoc, command
descriptions, `$this->info()` — stays English.
### 2. AI summary (best-effort)
New `ReportSummarizer` + `ReportSummaryAgent` (laravel/ai, same
Gemini-Flash pattern and `AI_PROVIDER` switch as the other AI features,
config in `config/ai_reports.php`). It prepends a few sentences to the
embed description that compare the current period against the previous
one (week over week for the two weekly reports, month over month for the
monthly cohort one), say what got better or worse, and call out
explicitly when a figure isn't conclusive — small sample, immature
cohort, signup surge week, or no previous period.
- **Data**: only what each collector already computes. The two cohort
reports pass their weekly series with the per-metric maturity flags; the
experiment report passes the per-variant figures plus the
already-rendered significance verdict (one source of truth), with money
figures nulled exactly where the table renders `—`, so the summary can't
report a zero where the reader sees no data.
- **Previous period**: the experiment report has no time series, so each
run caches its figures (with a capture timestamp) as the next run's
baseline. A same-day manual re-run doesn't overwrite it, and the model
is told to flag a gap that isn't roughly one period.
- **Degrades safely**: no API key, a provider outage, a bad provider
name or a slow response (30s timeout) → the report is posted unchanged,
without the summary. Transient provider errors are logged; anything else
is reported, matching `CategorizeTransactions` /
`LaravelAiRuleSuggestionGenerator`.
### 3. Fix found while reviewing: Discord's embed limits
The translated "Cómo leerlo" field came out at 1152 characters, past
Discord's 1024-character cap on a field value — Discord rejects the
**whole** payload, so the experiment funnel would have silently stopped
appearing in the channel every Monday (`DiscordWebhook` only logs the
400). Both long fields are now tighter (745 and 898 chars),
`DiscordWebhook` trims anything still over the limit instead of losing
the report, and a test measures every scheduled report's embed so
growing copy fails in CI rather than in production.
## Testing
- `tests/Feature/Ai/ReportSummarizerTest.php`: baseline in/out, same-day
re-run, dry run, backtick stripping, truncation, empty answer,
transient-vs-reported failures.
- `tests/Feature/DiscordReportEmbedLimitsTest.php`: all four embeds
inside Discord's limits, plus the trimming fallback.
- The four command tests: Spanish assertions, summary is the first thing
in the description, and the report is still posted when the AI throws.
- 64 tests / 338 assertions pass locally, plus PHPStan and Pint.
## QA
Ran all four commands against the local DB with `--no-discord` (and
dumped the real webhook payloads with the Discord call faked). Real
Gemini output, e.g. the experiment funnel on the second run:
> No se ha producido ningún cambio en las métricas del experimento
respecto a la ejecución previa del 10 de agosto de 2026. La tasa de
conversión sobre usuarios maduros se mantiene en el 4,7 % para la
variante reducida, el 4,0 % para pay_now y el 3,7 % para el control. Las
cifras no son conclusivas ya que las diferencias entre variantes no son
estadísticamente significativas, con un p-valor de 0,593 frente al
umbral α de 0,017.
Also verified with an unreachable provider: the report renders in full,
without the summary.
## Not included
- `stats:stuck-cohort-report` is still English. It posts to the same
webhook but isn't scheduled in `routes/console.php`, so it was out of
the four active reports; worth its own decision.
- The `—` and `⚡` cells are still a couple of bytes off inside the code
block (multibyte in `sprintf`). Pre-existing on `main`, unchanged here.
Víctor Falcón