From 03fd460b3f92d1deedcd829e99273e0d79d7137a Mon Sep 17 00:00:00 2001 From: Eugene Eisenstein Date: Thu, 3 Sep 2026 15:13:02 -0400 Subject: [PATCH] fix(sandbox): refuse seed and reset through the wrong provider MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Neither command recreates containers — seed uses `compose start` plus `--no-recreate` to avoid paying a container rebuild, and reset touches no container at all. So `--provider` on either could not change what the stack actually talked to, only what got recorded: seed would derive through the running provider and then stamp the requested one into the template, which is exactly what the fingerprint guard was supposed to catch. It compares the flag, not reality, so it matched. The real-stack-seeding-a-mock-template direction is the damaging one. It keeps real.env credentials on api and deriver, so it spends money, produces non-deterministic conclusions, labels them `mock`, and every later `reset --provider mock` restores those as the deterministic baseline that mock mode exists to provide. `up` now records the provider it created the stack with alongside the image in .state.env, and seed and reset refuse a mismatch pointing at `up --provider ...`. One mechanism covers both call sites and matches the file's existing posture of refusing rather than misleading. A .state.env with no provider line — written before this change — reads as unknown and is allowed; the next `up` fills it in. `status` also stopped reporting the flag as though it were the running stack. Verified against a live mock stack: a mismatch refuses from both seed and reset and exits 1; a match resets normally; `up` from a real-recorded state recreates, rewrites .state.env, and its internal reset does not trip the guard it just armed (3.7s end to end). --- sandbox/README.md | 11 ++++++++-- sandbox/sandbox.sh | 53 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 60 insertions(+), 4 deletions(-) diff --git a/sandbox/README.md b/sandbox/README.md index 1984db34..608c45a8 100644 --- a/sandbox/README.md +++ b/sandbox/README.md @@ -93,14 +93,21 @@ connection pools, and both reconnect by themselves — the api on its next reque next poll a quarter-second later. Each logs one `OperationalError` as its in-flight connection dies; that is expected, and it is the price of not paying ~9 seconds of container restart on every reset. -The cost of a snapshot is that it can go stale. Two guards: +The cost of a snapshot is that it can go stale. Three guards: - **Per-mode template names**, so a mock-seeded template can never be restored into a real-mode run. - **A fingerprint recorded inside each template** — the Alembic revision it was seeded at, a hash of `fixture.json` + `seed.py`, and the provider mode. `reset` compares and refuses with a "reseed" message rather than silently restoring a state that predates a migration. +- **A provider check against the running stack.** Neither `seed` nor `reset` recreates containers, so + `--provider` on either cannot change what the stack actually talks to — only what gets recorded. + `up` writes the provider it created the stack with to `sandbox/.state.env`, and `seed`/`reset` + refuse a mismatch and tell you to run `up --provider ...` first. Without it, + `seed --provider mock` against a running real stack would spend money on non-deterministic + conclusions and label them `mock`, and every later `reset` would restore those as the + deterministic baseline. -If you hit that refusal: `sandbox/sandbox.sh seed`. +If you hit the staleness refusal: `sandbox/sandbox.sh seed`. ## What's in the fixture diff --git a/sandbox/sandbox.sh b/sandbox/sandbox.sh index f87ef0a0..475efa2f 100755 --- a/sandbox/sandbox.sh +++ b/sandbox/sandbox.sh @@ -74,6 +74,22 @@ elif [ -f "$STATE" ]; then fi export HONCHO_SANDBOX_IMAGE +# The provider the running stack was actually created with, read on its own because +# the source above is skipped under --build. Empty means nothing has been started +# through `up` since the last `down`. +RUNNING_PROVIDER="" +if [ -f "$STATE" ]; then + RUNNING_PROVIDER="$(sed -n 's/^SANDBOX_RUNNING_PROVIDER=//p' "$STATE")" +fi + +write_state() { + cat > "$STATE" < "$STATE" + # Recorded before the seed/reset below, so their provider guard sees the stack + # that was just created rather than the one it replaced. + write_state if db_exists "$TEMPLATE"; then say "template $TEMPLATE already present — resetting to it" @@ -177,6 +220,7 @@ cmd_up() { cmd_seed() { preflight + require_running_provider # Seeding an already-seeded database appends to it — the fixture would land a # second time and the "expected 6 messages, found 12" check in seed.py would # (correctly) fail. Start from an empty, freshly migrated database every time so @@ -215,6 +259,7 @@ cmd_seed() { cmd_reset() { preflight + require_running_provider db_exists "$TEMPLATE" || die \ "no template for $PROVIDER mode. Run: sandbox/sandbox.sh seed --provider $PROVIDER" @@ -242,7 +287,11 @@ cmd_reset() { cmd_status() { compose ps echo - echo "provider: $PROVIDER" + if [ -n "$RUNNING_PROVIDER" ] && [ "$RUNNING_PROVIDER" != "$PROVIDER" ]; then + echo "provider: $RUNNING_PROVIDER (running) — this command asked for $PROVIDER" + else + echo "provider: $PROVIDER" + fi echo "image: $HONCHO_SANDBOX_IMAGE" echo "api: http://127.0.0.1:${SANDBOX_API_PORT:-18000}" for mode in mock real; do