The CM/user p-value uses a normal approximation to Student's t, which is
anti-conservative at small n (thin tails can read as significant when a real
t-test would not). Below 30 matured users per arm the report now shows the effect
size and t but withholds the significance verdict, only declaring significance
once both arms clear the floor.
The collector now returns the price ids that carried net-active subscriptions but
were missing from the monthly-equivalent map, and the report prints a loud warning
(Stripe unreachable, or unmapped prices undercounting MRR as 0) where the
decision-maker reads it — previously this only went to the logs, so an unsynced
arm price with no current payers could bias CM unnoticed.
The report now runs a chi-square sample-ratio-mismatch test of the arm counts
against the expected 50/50 split, on both the raw assigned counts and the matured
subset (to catch differential attrition), flagging a low p as an imbalance to
investigate before trusting results. Extract a shared Normal::cdf so the SRM and
Welch tests use one erf implementation.
Add stats:price-experiment-funnel: a 2-arm funnel (control vs high) on one shared
decision window. The primary metric is contribution margin per assigned user,
tested with a Welch two-sample test (unequal variances, since the higher-priced
arm carries far more revenue variance); conversion is a Fisher-exact guardrail
against control. Cost uses currently-active connections to match the monthly MRR
flow. The report is framed monitoring-only: decide at the pre-registered horizon,
not on weekly peeks. Extract the shared MonthlyEquivalentPrices lookup so both
funnel collectors resolve Stripe prices through one source.
Víctor Falcón