Burn incremented for every matured user who was not currently net-active,
which wrongly booked users who paid and later churned as connect-and-leave
leak (and mislabelled it 'never converted'). Restrict burn to users who never
earned net revenue: no subscription at all, or paid then refunded. A paid-then-
canceled user converted, so their connection cost is no longer burn.
Reading Feature::value() routed attribution through the before() hook, so
setting SUBSCRIPTION_EXPERIMENT_FORCE_VARIANT (the documented winner-rollout
switch) collapsed the whole weekly report onto one variant. Attribute by
SubscriptionExperiment::bucket() instead — the deterministic assignment every
in-window user was served — which is immune to force_variant and to started_at
moves, stops the report writing Pennant rows as a side effect, and makes the
feature's own docblock true again.
monthlyEquivByPriceId keyed the map by the current price ids (fetched by
lookup key), so a subscription on a rotated/archived price id matched no key
and silently contributed 0 to MRR/CM (prod already spans 4 distinct active
price ids). Fetch prices by product instead, which includes archived ids, and
warn if any net-active sub still has an unmapped price so it can never be
silent again. Also fix the yearly divide to round (was intdiv, always-down)
and skip foreign-currency/one-off prices.
The base user query dropped soft-deleted accounts (SoftDeletes scope), so
their variant assignment, connection cost and burn vanished from the report
- biased asymmetrically (prod: 8/9/10 deleted across control/reduced/pay_now)
and precisely on the cohort that most embodies connect-and-leave, since
deleting the account is the strongest form of it. Add withTrashed().
Replace A2P% (Paid / activated, which could exceed 100% because a paid
user need not have activated) with Conv% (Paid / matured-assigned, always
<=100% and comparable across variants). Print the matured cohort size
(MatU) so the rate denominators are visible and the table reconciles.
Revive ARPU (revenue per matured user) as the normalized revenue metric.
Reframe the guidance: compare variants on the per-matured-user Conv%/ARPU,
not on the absolute MRR/Cost/Burn/CM totals, which scale with MatU and so
mechanically favour whichever variant matured more (pay_now matures ~3x
faster than control).
## What
Reworks `stats:experiment-funnel` from a pure conversion funnel into a
**contribution-margin** one, so the 3-way trial/pricing experiment can
be judged on margin — not just conversion. Signups aren't free: each
bank connection costs money per month, so trials that connect banks and
never pay are burned cash, which the old report was blind to.
## Funnel
Per variant: **assigned → activated → carded → net-paying**
- **activated** = connected ≥1 bank connection **OR** gave AI consent
(triggered paid infrastructure), whether or not they paid.
- **carded** = completed Stripe Checkout (card on file).
- **net-paying** = live, non-refunded, mature subscription.
The **activated → carded** gap is where a user connects a bank and walks
away without paying — the exact leak the report now puts a number on.
## Metrics
| Column | Meaning |
|---|---|
| `A2P%` | net-paying ÷ activated (mature) |
| `Cost` | mature-cohort connections × cost/connection |
| `Burn` | connection cost of mature non-payers (money lost) |
| `CM` | MRR − Cost (the decision metric) |
- New `--cost-per-connection` option, default **0.40** per connection.
- Connection count includes soft-deleted/revoked connections (they still
cost money).
- All money/rate columns are gated on each variant's maturity window;
`Cost`/`Burn`/`CM` print `—` until a variant has mature volume. Legend
notes that raw `Assg`/`Actd`/`Card` are lifetime counts while
rates/money are mature-cohort only.
- Existing MRR/ARPU/Net% fields stay on the collector; dropped from the
printed table.
## Tests
Added coverage for activation (bank OR AI),
cost/burn/contribution-margin math, and the cost-per-connection
argument. Full file green (12 passed).
## Why
In the experiment funnel report, variants with a trial (control,
reduced) show `0` under Actv/Cncl/Rfnd while their signups are still
mid-trial — so the table looked empty even though many of those trials
have already been canceled by the user (Cashier keeps serving the trial
until it ends, then simply doesn't charge). That "already lost, just not
settled yet" cohort was invisible.
On prod right now: 28 subscriptions in `trialing`, of which **10 are
already scheduled to cancel** — a leading churn signal the report was
hiding.
## What
- `ExperimentFunnelCollector`: new per-variant `trialingCanceling`
counter = `stripe_status === 'trialing'` **and** `ends_at !== null`.
- Report table: two new columns — `Trl` (currently in trial, previously
collected but never printed) and `TrlX` (of those, already scheduled to
cancel and won't convert). Discord legend updated.
- Test covering the new counter.
`Trl`/`TrlX` are orthogonal leading indicators; the mature Net%/MRR/ARPU
metrics are unchanged.
## Testing
`php artisan test
tests/Feature/SendExperimentFunnelReportCommandTest.php` — 9 passed.
## What
Adds a `--no-discord` flag to `stats:experiment-funnel` so the
per-variant report can be run **on demand (e.g. in production) and only
printed to the console**, without posting to the Discord channel.
```bash
php artisan stats:experiment-funnel --no-discord
```
## Why
Handy for ad-hoc checks of the live experiment without spamming the
ops/Stripe Discord channel. The scheduled weekly run (and any run
without the flag) still posts to Discord exactly as before — default
behaviour is unchanged.
## Notes
- The console table already prints before the Discord step; the flag
just short-circuits the `->send()` and prints `Skipped Discord
(--no-discord).`
- Test added: with `--no-discord`, the command succeeds, prints the
table, and `Http::assertNothingSent()`.
- Pint + Larastan + the command's Pest suite green.
> Tip: the command queries the default DB connection, so to see
**production** numbers run it inside the prod container (e.g. via the
Coolify terminal).
## What
A 3-way experiment on how the paid plan is offered, plus per-variant
measurement. New signups (on/after `SUBSCRIPTION_EXPERIMENT_STARTED_AT`)
are split evenly into:
- **control** — current 15-day trial.
- **reduced_trial** — shorter trial: 3 days monthly, 7 days yearly.
- **pay_now** — charged immediately (no trial), with a self-service
money-back guarantee for the first 3 days.
Earlier users stay **legacy** and keep the 15-day trial. **While
`started_at` is null the experiment is off and everyone behaves like
control — inert until activated via env.**
## How it works
- **Assignment** — `App\Features\SubscriptionExperiment` (Pennant),
deterministic even split by a stable hash of the user id. QA can force a
variant with `feature:enable`.
- **Offer policy** — `ExperimentOffer` is the single source of truth for
trial days per plan, the pay-now flag, the refund window and refund
eligibility; shared by checkout, paywall and billing.
- **Checkout** — trial length comes from the variant (`trialDays(0)` for
pay_now → immediate charge).
- **Onboarding clarity** — the paywall states the exact terms above the
CTA: trial length for the selected plan, or "charged €X today + 3-day
money-back guarantee" for pay_now.
- **Self-service refund (pay_now)** — Settings → Billing, within the
window: refunds the upfront charge, `cancelNow`, revokes bank
connections keeping imported data. `refunded_at` records it and blocks a
second refund. Crash-safe ordering: the refund is stamped before
cancel/disconnect, which run best-effort in a try/catch.
## Measurement
`stats:experiment-funnel` (weekly → Discord): per-variant funnel
(assigned, subscribed, status breakdown, refunds) with a **net-active
rate** gated by each variant's decision window (control 15d / reduced 7d
/ pay_now 3d) so cohorts are read at equal age. Attribution reads the
variant Pennant actually served each user, so the report can't drift
from what users experienced. It also reports **MRR** (monthly run-rate
of mature net-active subs, yearly normalised ÷12) and **ARPU** (MRR ÷
assigned) per variant — ARPU is the revenue metric for the winner
decision. Plus a winner can be pinned org-wide with
`SUBSCRIPTION_EXPERIMENT_FORCE_VARIANT` (env, no deploy).
## Config (env)
- `SUBSCRIPTION_EXPERIMENT_STARTED_AT` — activates the experiment
(launch date). Null = off.
- `SUBSCRIPTION_EXPERIMENT_REDUCED_TRIAL_MONTHLY` (3), `..._YEARLY` (7),
`..._REFUND_WINDOW_DAYS` (3)
## Tests
- **Feature/unit:** assignment, offer policy, checkout wiring, refund
eligibility, the refund action incl. idempotency + crash-safe ordering
(Stripe mocked), and the funnel collector/command. ES + FR translations.
- **Browser** (`tests/Browser/SubscriptionRefundTest.php`): the
self-service refund UX end to end — card visibility + deadline, two-step
confirm, back-out, the refund control disappearing after confirming, and
gating (window passed / non-pay_now hidden). The `RefundSelfServe`
action is doubled so it never hits Stripe but applies the same DB
effect. Screenshots: `refund-card-visible`, `refund-confirm-step`,
`refund-completed`.
- Full non-browser suite green (the one failing `DashboardTest` is
pre-existing on `main` — Inertia 409 from the unbuilt local manifest).
Pint + ESLint + tsc (changed files) clean.
## Two independent reviews — acted on
**Fixed:** refund atomicity/idempotency (major) · funnel attribution now
reads Pennant's served value instead of recomputing, killing
report-vs-runtime drift (major) · pay_now copy shows the exact amount
charged · throttle + block-demo on the refund route · `resolve(?User)`
nullable · French translations.
**Reviewer notes (deferred, low value):**
- `refunded_at` is not cast to Carbon on Cashier's `Subscription` (safe
today — only null-compared; would need a custom Cashier model).
- `ExperimentFunnelCollector` walks users in PHP via `chunkById`; fine
at current volume, can move to grouped SQL if it grows.
## Confidence: 85 / 100
The critical money path is now **verified live against the Stripe
sandbox** (see below), which removes the earlier cap. All gates are
green and the acceptance criteria are met. Held at 85 (not higher)
because the browser UI test runs in CI rather than locally, and the
pay_now *hosted-checkout + webhook* leg reuses the standard Cashier
checkout already proven by the control flow (only `trialDays(0)`
differs) but wasn't re-driven through the hosted page. Given it moves
money + disconnects accounts, a human glance is still warranted before
enabling.
## Sandbox verification (live Stripe test mode)
`php artisan stripe:verify-refund` creates a real immediately-charged
subscription with a test card, runs the actual `RefundSelfServe`, and
checks the Stripe API. Result:
```
PASS subscription active after immediate charge (pay_now, no trial)
PASS canSelfRefund is true before refund
PASS latestPayment() resolves a payment intent
PASS refunded_at is stamped
PASS subscription is canceled
PASS canSelfRefund is false after refund
PASS Stripe charge shows a full refund (refunded=true)
```
The command is committed and guarded to Stripe test keys /
non-production, so it can be re-run before each launch toggle.
## Launch checklist
1. Stripe-sandbox smoke: `php artisan stripe:verify-refund` (done —
passing). Optionally also drive the hosted pay_now checkout once for
monthly + yearly to confirm the webhook leg.
2. Set `SUBSCRIPTION_EXPERIMENT_STARTED_AT` to the launch date (set
once; don't backdate).
3. Watch `stats:experiment-funnel`; a clean cohort baseline lands once
each variant's window matures.
Víctor Falcón