whisper-money/tests/Feature
Víctor Falcón 8fef50f829
fix(subscriptions): assign the price arm before registration, not after (#792)
Fixes a design flaw in #700, before the experiment has any exposure.

## The problem

#700 drew the arm from `crc32('price:' . $user->id)`, which can only
happen once the user exists. But the landing quotes a price to anonymous
visitors, and `plansFor(null)` had no arm to apply — so **every visitor
saw €3.99**, and half of them were switched to €8.99 after registering.

That biases the result in both directions at once:

- **Against `high`** — it pays a penalty that isn't price sensitivity
but a price that moved after being advertised. Someone who'd have
happily paid €8.99 quoted upfront leaves because they feel baited.
- **In favour of `high`** — its funnel only contains people who already
decided to sign up under a €3.99 promise. In a world where we actually
charge €8.99, the landing says €8.99 and some of them never register at
all. The experiment is blind to that drop-off.

Neither bias is recoverable from the data, and they don't cancel: a
narrow loss for `high` would be uninterpretable.

## The fix

Draw the arm for the **anonymous visitor** on their first page view,
keep it in a year-long cookie, and freeze it onto `users.price_arm` at
registration. The landing quotes what checkout will charge, and the
whole funnel is measured under one price.

| | before | after |
|---|---|---|
| assigned at | registration | first page view |
| source | `crc32('price:' . id) % 2` | random 50/50 draw |
| stored in | nothing (recomputed) | cookie → `users.price_arm` |
| landing shows | always control | the visitor's arm |

Details worth a look in review:

- **The draw is random, not a hash.** There's no stable identifier
before the user exists. This is what forces the column — the arm has to
outlive the cookie.
- **The middleware writes the arm onto the current request**, not just
the response cookie. The cookie only reaches the browser *after* this
response, and the first view is the landing — the one page that most
needs the right price.
- **Checkout reads `users.price_arm` only, never the cookie.** Editing
your cookie after signing up doesn't get you the cheap price. There's a
test that asserts exactly this, with the cookie set to `control` and the
stored arm `high`.
- **No arm = control.** Users from before the experiment, cookies
blocked, arriving at a deep link. `sanitize()` narrows both the cookie
and the column, since a user controls the former.
- **`force_variant` now also stops the draw** — a winner being rolled
out means the split is over.
- **`price_arm` is in `$hidden`.** The frontend has no use for it, and
it needn't be visible in the Inertia payload.

## Exposure

**None.** The experiment started at 14:00 UTC and had **0 signups past
the cutoff** when this was written, so no arm needs reconciling and no
user changes price. This is the last moment this change is free.

## Reading results

The CRC32 expression from #700 is obsolete:

```sql
SELECT COALESCE(price_arm, 'legacy') AS arm, COUNT(*)
FROM users WHERE created_at >= '<started_at>' GROUP BY arm;
```

## Still open

The landing now shows €8.99 to half of anonymous visitors, which means
the experiment can finally affect signup volume itself. That's the point
— but it also means a drop in registrations is a *result*, not a bug,
and shouldn't be rolled back on reflex.

## Tests

19 tests in `PriceExperimentTest`, rewritten around the new mechanism:
the draw, the gate, the forced winner, the cookie→prop path on the first
visit, freezing at registration, and checkout ignoring the cookie.
`Auth`, `SubscriptionTest`, `InertiaSharedDataTest`, `CashflowPageTest`
and `SyncStripePricesCommandTest` all green locally (122 tests). PHPStan
and `crap` clean.
2026-08-13 08:14:37 +00:00
..
Ai feat(stats): post the Discord stats reports in Spanish, opened by an AI summary (#752) 2026-08-10 10:13:40 +02:00
Api
…
Auth
…
Console feat(reports): email a monthly CSV of active user emails to the owners (#783) 2026-08-12 11:29:28 +02:00
Events
…
Jobs
…
Listeners
…
Mcp feat(mcp): budget tools — read, create, edit and delete (#779) 2026-08-11 14:28:53 +00:00
Onboarding fix(onboarding): don't trap users on the syncing step when a bank sync fails (#745) 2026-08-09 18:40:49 +02:00
OpenBanking fix(wise): send the pagination cursor under the name Wise reads (#788) 2026-08-12 11:30:37 +00:00
Services
…
Settings
…
Spaces
…
Sync
…
AccountBalanceControllerTest.php
…
AccountControllerTest.php feat(accounts): count shared accounts at the owner's percentage (#750) 2026-08-11 13:26:54 +00:00
AccountImportConfigTest.php
…
AccountUserCurrencyServiceTest.php
…
AiConsentSettingsTest.php
…
AiConsentTest.php
…
AlignAccountsEncryptedFlagMigrationTest.php
…
ApplyRealEstateRevaluationTest.php
…
AuthenticatedLayoutSafeAreaTest.php
…
AutomationRuleApplicationTest.php fix(budgets): re-derive budget membership when labels are attached without a model event (#787) 2026-08-12 12:48:47 +02:00
AutomationRuleEvaluationTest.php
…
AutomationRuleTest.php fix(ai): stop a learned rule title from 500-ing the user's category change (#741) 2026-08-09 15:39:14 +00:00
BackfillAccountIbansCommandTest.php
…
BackfillXxxAccountCurrenciesTest.php
…
BalanceLookupTest.php
…
BudgetHistoricalAssignmentTest.php
…
BudgetNotificationTest.php
…
BudgetPeriodDateTest.php
…
BudgetPeriodServiceTest.php
…
BudgetTest.php feat(mcp): budget tools — read, create, edit and delete (#779) 2026-08-11 14:28:53 +00:00
BudgetTransactionServiceTest.php
…
BulkUpdateTransactionsTest.php refactor(transactions): build the bulk selection once in bulkUpdate (#774) 2026-08-11 13:14:19 +00:00
CancelFreeEnableBankingConnectionsCommandTest.php
…
CashflowAnalyticsTest.php
…
CashflowPageTest.php
…
CatchAllBudgetTest.php fix(budgets): keep labeled expenses out of the catch-all budget (#781) 2026-08-11 15:45:19 +00:00
CategoryMonthlyBreakdownTest.php
…
CrapCommandTest.php ci: add duplication and complexity quality checks (#765) 2026-08-11 13:29:37 +02:00
CurrencyConversionServiceTest.php
…
DashboardAnalyticsTest.php
…
DashboardTest.php
…
DecryptTransactionsTest.php
…
DeleteUserCommandTest.php
…
DemoAccountRestrictionsTest.php fix(demo): stop demo:reset from colliding on the fake Stripe subscription id (#756) 2026-08-10 12:48:13 +00:00
DisconnectBankingConnectionsCommandTest.php
…
DiscordReportEmbedLimitsTest.php feat(subscriptions): end the trial experiment and make the trial length per plan (#762) 2026-08-12 10:59:55 +02:00
DiscordWebhookTest.php
…
EncryptionTest.php
…
ExampleTest.php
…
ExchangeRateServiceTest.php
…
GenerateStripePromotionCodesCommandTest.php chore(deps): update composer dependencies to latest (#764) 2026-08-11 11:15:27 +00:00
IdorVulnerabilityTest.php
…
ImportDataTest.php
…
InertiaSharedDataTest.php
…
IntegrationRequestTest.php
…
LabelBudgetReassignmentTest.php fix(budgets): re-derive budget membership when labels are attached without a model event (#787) 2026-08-12 12:48:47 +02:00
LabelTest.php
…
LoanTest.php refactor(accounts): split store/update in the account controller (#776) 2026-08-11 13:58:47 +00:00
LocalizationTest.php feat(currencies): add the Danish Krone (DKK) (#754) 2026-08-10 12:08:56 +02:00
LoggingConfigTest.php
…
MailSenderTest.php feat(reports): email a monthly CSV of active user emails to the owners (#783) 2026-08-12 11:29:28 +02:00
NewTransactionsMarkerTest.php
…
OpenAiAppsChallengeTest.php feat(mcp): serve the ChatGPT app directory domain challenge (#749) 2026-08-10 07:46:49 +00:00
PersistUpsellSourceFromStripeTest.php
…
PlaintextTransactionsTest.php
…
PlanFeatureTest.php
…
PopoverSafeAreaTest.php
…
PostStripeEventToDiscordTest.php
…
PriceExperimentTest.php fix(subscriptions): assign the price arm before registration, not after (#792) 2026-08-13 08:14:37 +00:00
PurgeResidualEncryptionArtifactsJobTest.php
…
PwaTest.php
…
QueueConfigTest.php
…
ReEvaluateTransactionRulesTest.php
…
RealEstateAvailabilityTest.php
…
RealEstateTest.php
…
ResendSyncCommandTest.php
…
RoadmapTest.php feat(roadmap): add a public roadmap page mirrored from UserJot (#778) 2026-08-11 14:14:59 +00:00
RouteNotificationForMailTest.php
…
RuleEngineParityTest.php
…
SavedFilterTest.php feat(transactions): add a monthly trend view to the analysis drawer (#736) 2026-07-26 17:03:52 +02:00
SendAiCohortReportCommandTest.php feat(stats): post the Discord stats reports in Spanish, opened by an AI summary (#752) 2026-08-10 10:13:40 +02:00
SendAiConsentFollowUpEmailsCommandTest.php
…
SendDailyStatsReportCommandTest.php feat(stats): post the Discord stats reports in Spanish, opened by an AI summary (#752) 2026-08-10 10:13:40 +02:00
SendStuckCohortReportCommandTest.php
…
SendSubscriptionFunnelReportCommandTest.php feat(subscriptions): end the trial experiment and make the trial length per plan (#762) 2026-08-12 10:59:55 +02:00
SentryConfigTest.php
…
SentryUserMiddlewareTest.php
…
SetLocaleTest.php
…
SharedAccountOwnershipTest.php feat(budgets): count shared accounts at the owner's percentage (#786) 2026-08-12 12:47:43 +02:00
SitemapTest.php
…
StrayHttpRequestGuardTest.php
…
StripeSubscriptionStatsCommandTest.php
…
SubscriptionTest.php feat(subscriptions): end the trial experiment and make the trial length per plan (#762) 2026-08-12 10:59:55 +02:00
SuggestionPersistenceTest.php
…
SyncBankingConnectionsCommandTest.php
…
SyncStripePricesCommandTest.php feat(subscriptions): add an A/B price experiment (€3.99 control vs €8.99 high) (#700) 2026-08-12 13:36:49 +02:00
TrackLastActiveAtTest.php
…
TransactionAnalysisTest.php feat(transactions): add a monthly trend view to the analysis drawer (#736) 2026-07-26 17:03:52 +02:00
TransactionFilterTest.php
…
TransactionSideClassificationTest.php
…
TransactionTest.php
…
WeighBudgetTransactionsMigrationTest.php feat(budgets): count shared accounts at the owner's percentage (#786) 2026-08-12 12:47:43 +02:00
WelcomeBanksOrderingTest.php
…