fix(sandbox): refuse seed and reset through the wrong provider

Neither command recreates containers — seed uses `compose start` plus
`--no-recreate` to avoid paying a container rebuild, and reset touches no
container at all. So `--provider` on either could not change what the stack
actually talked to, only what got recorded: seed would derive through the
running provider and then stamp the requested one into the template, which
is exactly what the fingerprint guard was supposed to catch. It compares the
flag, not reality, so it matched.

The real-stack-seeding-a-mock-template direction is the damaging one. It
keeps real.env credentials on api and deriver, so it spends money, produces
non-deterministic conclusions, labels them `mock`, and every later
`reset --provider mock` restores those as the deterministic baseline that
mock mode exists to provide.

`up` now records the provider it created the stack with alongside the image
in .state.env, and seed and reset refuse a mismatch pointing at
`up --provider ...`. One mechanism covers both call sites and matches the
file's existing posture of refusing rather than misleading. A .state.env
with no provider line — written before this change — reads as unknown and
is allowed; the next `up` fills it in.

`status` also stopped reporting the flag as though it were the running
stack.

Verified against a live mock stack: a mismatch refuses from both seed and
reset and exits 1; a match resets normally; `up` from a real-recorded state
recreates, rewrites .state.env, and its internal reset does not trip the
guard it just armed (3.7s end to end).
This commit is contained in:
Eugene Eisenstein 2026-09-03 15:13:02 -04:00
parent 92df37ee60
commit 03fd460b3f
2 changed files with 60 additions and 4 deletions

View File

@ -93,14 +93,21 @@ connection pools, and both reconnect by themselves — the api on its next reque
next poll a quarter-second later. Each logs one `OperationalError` as its in-flight connection dies;
that is expected, and it is the price of not paying ~9 seconds of container restart on every reset.
The cost of a snapshot is that it can go stale. Two guards:
The cost of a snapshot is that it can go stale. Three guards:
- **Per-mode template names**, so a mock-seeded template can never be restored into a real-mode run.
- **A fingerprint recorded inside each template** — the Alembic revision it was seeded at, a hash of
`fixture.json` + `seed.py`, and the provider mode. `reset` compares and refuses with a "reseed"
message rather than silently restoring a state that predates a migration.
- **A provider check against the running stack.** Neither `seed` nor `reset` recreates containers, so
`--provider` on either cannot change what the stack actually talks to — only what gets recorded.
`up` writes the provider it created the stack with to `sandbox/.state.env`, and `seed`/`reset`
refuse a mismatch and tell you to run `up --provider ...` first. Without it,
`seed --provider mock` against a running real stack would spend money on non-deterministic
conclusions and label them `mock`, and every later `reset` would restore those as the
deterministic baseline.
If you hit that refusal: `sandbox/sandbox.sh seed`.
If you hit the staleness refusal: `sandbox/sandbox.sh seed`.
## What's in the fixture

View File

@ -74,6 +74,22 @@ elif [ -f "$STATE" ]; then
fi
export HONCHO_SANDBOX_IMAGE
# The provider the running stack was actually created with, read on its own because
# the source above is skipped under --build. Empty means nothing has been started
# through `up` since the last `down`.
RUNNING_PROVIDER=""
if [ -f "$STATE" ]; then
RUNNING_PROVIDER="$(sed -n 's/^SANDBOX_RUNNING_PROVIDER=//p' "$STATE")"
fi
write_state() {
cat > "$STATE" <<EOF
HONCHO_SANDBOX_IMAGE=$HONCHO_SANDBOX_IMAGE
SANDBOX_RUNNING_PROVIDER=$PROVIDER
EOF
RUNNING_PROVIDER="$PROVIDER"
}
compose() {
docker compose \
-f "$HERE/compose.yml" \
@ -134,6 +150,31 @@ MSG
fi
}
# Containers carry the provider wiring they were created with, and neither seed nor
# reset recreates them: seed uses `start` plus `--no-recreate` to avoid paying a
# container rebuild, and reset touches no container at all. So a --provider that
# disagrees with the running stack does not change what the stack talks to. It only
# changes what gets recorded — seed would derive through the running provider and
# then stamp the requested one into the template, defeating the fingerprint guard,
# which compares the flag rather than reality.
#
# The real-stack-seeding-a-mock-template case is the damaging one: it spends money,
# produces non-deterministic conclusions, and labels them `mock`, so every later
# reset restores that as the deterministic baseline. Refuse rather than mislead.
require_running_provider() {
[ -n "$RUNNING_PROVIDER" ] || return 0
[ "$RUNNING_PROVIDER" != "$PROVIDER" ] || return 0
die "$(cat <<MSG
the stack is running the $RUNNING_PROVIDER provider, but this is a $PROVIDER-mode command.
seed and reset do not recreate containers, so this would act through the
$RUNNING_PROVIDER provider while recording $PROVIDER. Switch the stack first:
sandbox/sandbox.sh up --provider $PROVIDER
MSG
)"
}
# --------------------------------------------------------------------------
# Commands
# --------------------------------------------------------------------------
@ -162,7 +203,9 @@ cmd_up() {
# mock to real leaves mock-provider running and otherwise unreferenced, where it
# would sit idle and make `status` misleading about what the stack is using.
compose up -d --wait --remove-orphans
echo "HONCHO_SANDBOX_IMAGE=$HONCHO_SANDBOX_IMAGE" > "$STATE"
# Recorded before the seed/reset below, so their provider guard sees the stack
# that was just created rather than the one it replaced.
write_state
if db_exists "$TEMPLATE"; then
say "template $TEMPLATE already present — resetting to it"
@ -177,6 +220,7 @@ cmd_up() {
cmd_seed() {
preflight
require_running_provider
# Seeding an already-seeded database appends to it — the fixture would land a
# second time and the "expected 6 messages, found 12" check in seed.py would
# (correctly) fail. Start from an empty, freshly migrated database every time so
@ -215,6 +259,7 @@ cmd_seed() {
cmd_reset() {
preflight
require_running_provider
db_exists "$TEMPLATE" || die \
"no template for $PROVIDER mode. Run: sandbox/sandbox.sh seed --provider $PROVIDER"
@ -242,7 +287,11 @@ cmd_reset() {
cmd_status() {
compose ps
echo
echo "provider: $PROVIDER"
if [ -n "$RUNNING_PROVIDER" ] && [ "$RUNNING_PROVIDER" != "$PROVIDER" ]; then
echo "provider: $RUNNING_PROVIDER (running) — this command asked for $PROVIDER"
else
echo "provider: $PROVIDER"
fi
echo "image: $HONCHO_SANDBOX_IMAGE"
echo "api: http://127.0.0.1:${SANDBOX_API_PORT:-18000}"
for mode in mock real; do