Ends the trial/pricing A/B/C experiment. Everyone gets the control offer
— a free trial — and the trial length becomes a per-plan setting.
## Trial length
| Plan | Before | Now | Env override |
|---|---|---|---|
| Yearly | 15 days | **15 days** | `STRIPE_PRO_YEARLY_TRIAL_DAYS` |
| Monthly | 15 days | **7 days** | `STRIPE_PRO_MONTHLY_TRIAL_DAYS` |
**Note that monthly 15 → 7 is a new bet, not a rollback.** The control
arm was 15 days on both plans, and 7 days on monthly is a value the
experiment never tested (`reduced_trial` was monthly 3 / yearly 7). The
rationale is that the longer commitment earns the longer trial; it ships
here at the same time as the instrument that could measure it is
removed, so it will not be measurable as an isolated effect.
## Final experiment numbers
Archived here because `stats:experiment-funnel` and its collector are
deleted by this PR and the purge migration's `down()` is a no-op.
| Variant | Assigned | Subscribed | Active | Refunded |
|---|---|---|---|---|
| control | 597 | 46 | 13 | 0 |
| reduced_trial | 590 | 50 | 10 | 0 |
| pay_now | 609 | 41 | 21 | 18 |
| legacy | 94 | 55 | 31 | 0 |
## What is deleted
- `App\Features\SubscriptionExperiment` (the Pennant A/B/C assignment)
and the `ExperimentOffer` service.
- The `pay_now` self-service refund: `RefundSelfServe`, the
`settings.billing.refund` route, the controller actions and Discord
embeds, the money-back card in billing settings, and the
`stripe:verify-refund` sandbox command.
- The weekly `stats:experiment-funnel` report, its collector, and the
`ProportionSignificance` / `BinomialProportion` helpers it was the only
caller of, plus its schedule entry.
- The `subscriptions.experiment.*` config block and the orphaned
`es`/`fr` translation strings.
- A data migration purges the ~1,890 stored Pennant assignments.
`subscriptions.refunded_at` is deliberately **kept**: nothing reads it
anymore, but it is the only record of the 18 refunds the experiment
issued. The migration carries a comment saying so.
## Fixes found in review
- **The surviving funnel report was mis-scoring conversions.**
`SubscriptionFunnelCollector` compared every cancellation to one global
trial length. With trials now diverging per plan, a monthly subscriber
who was billed and cancelled on day 10 was scored as never having paid.
It now reads each subscription's own `trial_ends_at`, and the longest
plan trial is used only for deciding when a cohort is old enough to
score. Covered by two new tests.
- **The trial length swapped silently.** It lived on a single line under
the plan selector, which rewrote itself when the user switched plan. Now
that the plans genuinely differ, each plan card shows its own length.
- The report legend no longer quotes a single trial length for both
plans, and warns that the experiment weeks are still inside its window.
## Before merging
- [x] **Unset `SUBSCRIPTION_EXPERIMENT_STARTED_AT` in production** so no
new `pay_now` assignment happens while this waits. Anyone who checks out
under `pay_now` between now and the deploy is charged upfront and then
loses the one-tap refund they were promised at the point of payment.
Checked just before opening this PR: **0 `pay_now` subscriptions
currently inside the 3-day window**, so nobody is stranded today.
- [x] Drop the now-orphaned `SUBSCRIPTION_EXPERIMENT_*` variables from
the production env with the deploy.
- [x] If old containers are still serving while the purge migration
runs, a few assignments can be re-resolved and reappear. Harmless —
re-run `php artisan pennant:purge "App\Features\SubscriptionExperiment"`
once the deploy settles if you want the table clean.
Support note: a manual Stripe refund for a former `pay_now` user will
not disconnect their bank connections, which the automated flow used to
do.
## Demo
https://github.com/user-attachments/assets/3614d488-05c6-405d-a687-bbf45746879a
<!-- PLACEHOLDER: drag the QA video here -->
## QA
Browser-tested against the running app:
- Paywall: annual card shows "15 days free", monthly card "7 days free";
the terms line under the selector follows the selected plan (15 ↔ 7);
mobile viewport renders fine.
- Billing settings: no money-back card for a free user or an active
subscriber; `POST /settings/billing/refund` returns 404.
- No console or network errors on any screen.
- `stats:subscription-funnel` still renders and posts.
- The purge migration leaves 0 `SubscriptionExperiment` rows.
Full suite green (2045 tests) apart from the known local-only
`DashboardTest` 409; `pint`, `lint`, `format` and `build` all clean.
## What
The admin Discord channel is Spanish, but the four scheduled `stats:*`
reports posted English. They now post Spanish, and the three
cohort/experiment reports open with a short AI-written summary so a
reader understands the situation without decoding the table.
### 1. Spanish reports
`stats:daily-report`, `stats:subscription-funnel`,
`stats:experiment-funnel` and `stats:ai-cohort-report` now post Spanish
embed titles, field names, ASCII table headers, legends and disclaimers,
with Spanish dates (`sáb., 13 jun. 2026`). Table headers and cells stay
ASCII (`Semana`, `Variante`, `UMad`, `pdte`, …) so `sprintf`'s byte
padding keeps the columns aligned inside the code block.
Hardcoded, not `__()`: this is an internal channel, not user-facing UI,
so it never needs a second language and doesn't belong in
`lang/es.json`. Everything else — code, comments, PHPDoc, command
descriptions, `$this->info()` — stays English.
### 2. AI summary (best-effort)
New `ReportSummarizer` + `ReportSummaryAgent` (laravel/ai, same
Gemini-Flash pattern and `AI_PROVIDER` switch as the other AI features,
config in `config/ai_reports.php`). It prepends a few sentences to the
embed description that compare the current period against the previous
one (week over week for the two weekly reports, month over month for the
monthly cohort one), say what got better or worse, and call out
explicitly when a figure isn't conclusive — small sample, immature
cohort, signup surge week, or no previous period.
- **Data**: only what each collector already computes. The two cohort
reports pass their weekly series with the per-metric maturity flags; the
experiment report passes the per-variant figures plus the
already-rendered significance verdict (one source of truth), with money
figures nulled exactly where the table renders `—`, so the summary can't
report a zero where the reader sees no data.
- **Previous period**: the experiment report has no time series, so each
run caches its figures (with a capture timestamp) as the next run's
baseline. A same-day manual re-run doesn't overwrite it, and the model
is told to flag a gap that isn't roughly one period.
- **Degrades safely**: no API key, a provider outage, a bad provider
name or a slow response (30s timeout) → the report is posted unchanged,
without the summary. Transient provider errors are logged; anything else
is reported, matching `CategorizeTransactions` /
`LaravelAiRuleSuggestionGenerator`.
### 3. Fix found while reviewing: Discord's embed limits
The translated "Cómo leerlo" field came out at 1152 characters, past
Discord's 1024-character cap on a field value — Discord rejects the
**whole** payload, so the experiment funnel would have silently stopped
appearing in the channel every Monday (`DiscordWebhook` only logs the
400). Both long fields are now tighter (745 and 898 chars),
`DiscordWebhook` trims anything still over the limit instead of losing
the report, and a test measures every scheduled report's embed so
growing copy fails in CI rather than in production.
## Testing
- `tests/Feature/Ai/ReportSummarizerTest.php`: baseline in/out, same-day
re-run, dry run, backtick stripping, truncation, empty answer,
transient-vs-reported failures.
- `tests/Feature/DiscordReportEmbedLimitsTest.php`: all four embeds
inside Discord's limits, plus the trimming fallback.
- The four command tests: Spanish assertions, summary is the first thing
in the description, and the report is still posted when the AI throws.
- 64 tests / 338 assertions pass locally, plus PHPStan and Pint.
## QA
Ran all four commands against the local DB with `--no-discord` (and
dumped the real webhook payloads with the Discord call faked). Real
Gemini output, e.g. the experiment funnel on the second run:
> No se ha producido ningún cambio en las métricas del experimento
respecto a la ejecución previa del 10 de agosto de 2026. La tasa de
conversión sobre usuarios maduros se mantiene en el 4,7 % para la
variante reducida, el 4,0 % para pay_now y el 3,7 % para el control. Las
cifras no son conclusivas ya que las diferencias entre variantes no son
estadísticamente significativas, con un p-valor de 0,593 frente al
umbral α de 0,017.
Also verified with an unreachable provider: the report renders in full,
without the summary.
## Not included
- `stats:stuck-cohort-report` is still English. It posts to the same
webhook but isn't scheduled in `routes/console.php`, so it was out of
the four active reports; worth its own decision.
- The `—` and `⚡` cells are still a couple of bytes off inside the code
block (multibyte in `sprintf`). Pre-existing on `main`, unchanged here.
## Problem
The production image runs the queue workers via supervisor but **nothing
runs the Laravel scheduler** (there is no `schedule:work` program and no
cron running `schedule:run`). As a result, none of the commands in
`routes/console.php` ever fire automatically — most notably
`banking:sync` (every 6h), so self-hosted users have to trigger bank
synchronization by hand. The manual "sync" button works because it
dispatches the same job to the queue, which the existing workers pick
up; only the scheduled trigger was missing.
## Changes
- **`docker/supervisor/supervisord.conf`**: add a `[program:scheduler]`
running `php artisan schedule:work`, mirroring the existing worker
programs (autostart/autorestart, dedicated log). This is what makes
every scheduled command fire on its own cadence.
- **Gate the operator-only stats reports** (`stats:daily-report`,
`stats:subscription-funnel`, `stats:ai-cohort-report`,
`stats:experiment-funnel`) behind `config('subscriptions.enabled')` with
an early return. These are subscription-conversion telemetry destined
for the admin Discord; on a self-hosted instance
(`subscriptions.enabled=false`) `stats:daily-report` would otherwise
make an unnecessary live Stripe API call, and the others would build
reports for an unconfigured Discord channel. On the SaaS (subscriptions
enabled) behavior is unchanged.
The guard lives **inside each command** (not in `routes/console.php`) so
it stays unit-testable. `banking:cancel-free-enablebanking` was left
alone (already a safe no-op via `hasProPlan()` when subscriptions are
off), as were the drip-email commands (already gated by
`mail.drip_emails_enabled`).
## Testing
- `vendor/bin/pint --test` passes.
- New Pest test per gated command asserting it skips and makes no
external call when subscriptions are disabled; existing tests updated to
enable subscriptions so they still exercise the real path.
- Full suite green except 5 unrelated pre-existing environment failures
(timezone alias, dashboard 409, Passport OAuth key) that fail
identically on `main`.
- Verified end-to-end with a real supervisor run: the `scheduler`
program reaches `RUNNING`, `schedule:work` boots, and autorestart
resurrects it; `php artisan schedule:test --name=banking:sync`
dispatches successfully.
> Takes effect after rebuilding the production image.
## What
Follow-up to the `--no-discord` flag added to `stats:experiment-funnel`.
Extends the same flag to the **other stats report commands that post to
Discord**, so any of them can be run on demand (e.g. inside the prod
container) and just printed to the console without posting:
```bash
php artisan stats:subscription-funnel --no-discord
php artisan stats:ai-cohort-report --no-discord
php artisan stats:stuck-cohort-report --no-discord
php artisan stats:daily-report --no-discord
```
## How
- The two **funnel** commands (`subscription-funnel`) already printed
their table to the console, so `--no-discord` just short-circuits the
`->send()`.
- The **cohort/daily** commands only ever built a Discord embed. A small
`App\Console\Commands\Concerns\RendersReportToConsole` trait renders an
embed (title / description / fields) as plain text, so `--no-discord`
prints a readable report to the console instead of posting.
- Default behaviour is unchanged: without the flag (and on the scheduled
weekly/daily runs) they post to Discord exactly as before.
## Tests
A `--no-discord` test added to each command (`Http::assertNothingSent()`
+ the command succeeds). Pint + Larastan + all four command suites
green.
> Note: the commands query the default DB connection, so for
**production** numbers run them inside the prod container (Coolify
terminal).
## What
Adds a weekly **registration → subscription → paid** funnel so we can
baseline subscription conversion and, later, measure A/B tests against
it.
- `stats:subscription-funnel` artisan command — prints the cohort table
and posts it to Discord (same webhook as the other weekly reports).
- `SubscriptionFunnelCollector` — builds the age-normalized weekly
cohort series.
- Scheduled weekly, Mondays 09:15, next to the existing cohort reports.
## How the funnel is defined
Every stage is measured from each user's **own signup**, so weekly
cohorts are compared at the same age regardless of the calendar.
- **Registered** — signups that week.
- **Subscribed** — started a `default` plan ≤30d after signup (entered
checkout/trial).
- **Paid** — that plan billed past the 15d trial: currently `active`, or
`canceled` only after outliving the trial. `trial_ends_at` is cleared on
cancel, so subscription lifespan is the reliable "did it ever bill"
signal.
Both stages are bounded by the same subscription-creation window, which
guarantees the funnel invariant **registered ≥ subscribed ≥ paid** (a
test enforces it — I caught and fixed a window bug where `paid` could
exceed `subscribed`). Cohorts too young to score show `pend`; signup
surges (launch/marketing waves) are flagged with ⚡.
## Current baseline (prod, 2026-06-27)
| Stage | Count | Rate |
|---|---|---|
| Registered | 1,289 | — |
| Subscribed | 158 | 12.3% of registered |
| Trial resolved | 128 | conversion denominator |
| Paid after trial | 91 | **71%** of resolved · 7.1% of registered |
| Currently active | 75 | 82% of payers retained |
⚠️ ~940 of the 1,289 signups arrived in the last ~5 weeks (a
late-May/June surge); their trials haven't fully resolved, so
cohort-level conversion for that wave isn't measurable yet. The 71% is
status-based — a clean cohort baseline lands in ~4–6 weeks.
## Tests
5 Pest tests covering cohort counting, the attribution window, the
funnel invariant, maturity gating, and Discord delivery. All green, Pint
clean.
Víctor Falcón
Jesús Mejías Leiva