openGeoIP opens cfg.GeoIP.Path and returns a (Lookuper, closer)
pair; on open failure (missing/unreadable mmdb) it returns
geoip.NopService() and a no-op closer so the rest of the wire-up
sees a consistent non-nil interface and the binary stays healthy
in environments without a MaxMind license. buildEventStack now
accepts a geoip.Lookuper and threads it into event.ServiceConfig.
Close is deferred in run() so the mmdb file handle releases on
graceful shutdown; a close-error is logged at warn (not fatal).
- Rename admin.Handler.Register → RegisterRoutes for parity with
health.Handler.RegisterRoutes and token.Handler.RegisterAPIRoutes /
RegisterManageRoutes / RegisterTriggerRoutes. Spec §6.3 line 558
also illustrates the canonical wire-up as
`adminH.RegisterRoutes(api)`.
- Drop the local strItoa helper in handler_test.go; use strconv.Itoa.
The helper was a code-reviewer nit — equivalent stdlib already
exists; fewer lines, less surface to drift.
Both findings are non-blocking but covered by the fix-in-phase rule
(no backlog rot). Phase 12 audits otherwise PASS on first dispatch:
- superpowers:code-reviewer: PASS
- general-purpose spec-adherence: PASS
- cmd/canary/main.go: mountAdminRoutes helper keeps run() and
mountRouter under funlen 100. Admin routes mounted INSIDE the /api
subrouter so they inherit read-tier rate limit + CORS; gated by
OperatorBearer(cfg.Operator.Token). If OPERATOR_TOKEN is unset, the
entire mount is skipped (no Bearer middleware, no routes — admin
endpoints simply do not exist), and a structured warn log records
the reason. This is the production safety: misconfiguration leaves
the endpoint nonexistent rather than reachable.
- internal/admin/integration_test.go: testcontainers-backed end-to-end
tests exercising the full /api/admin/* stack:
* 404 on missing Authorization (no WWW-Authenticate leak)
* 404 on wrong token
* 200 on /stats with seeded mixed tokens + events
* 200 on /tokens?limit=2 with HasMore/NextOffset accuracy
* 204 on disable + DB-level verification token.Enabled flipped
* 404 on disable for a missing id
All requests go through the OperatorBearer middleware to verify the
wire-up matches the unit behavior.
- cmd/canary/main.go: tokenH.RegisterManageRoutes(api) mounts GET +
DELETE /m/{manage_id} inside the /api subrouter (read-tier rate
limit applies; no Turnstile, no create-tier limits — read-style
endpoints by spec §8.1).
- Extract buildHTTPDeps to keep run() under the funlen ceiling (was
102, now under 100). Pulls out registry build + tokenSvc + verifier
+ healthH + tokenH construction. Returns the four handles run()
needs to wire the router and shutdown path.
Integration tests (internal/event/integration_test.go,
testcontainers-Postgres + miniredis):
- TestIntegration_ManagePageReturnsTokenAndEventsAndSilencedCount:
POST creates a webbug, three triggers from the same CF-Connecting-IP
fire (one notification, two deduped per the Phase 10 dedup gate);
GET /api/m/{manage_id} returns token with TriggerCount=3,
EventsTotal=3, EventsSilencedActive=2, three event records with
exactly one NotifySent and two NotifyDeduped, no next page.
- TestIntegration_ManagePageReturns404OnUnknownManageID
- TestIntegration_ManageDeleteCascadesEvents: delete via DELETE
/api/m/{manage_id}, verify CountByToken == 0 (FK ON DELETE CASCADE)
and the token is gone (token.ErrNotFound).
Swaps the Phase-9 directEventRecorder/mysqlEventRecorder shims for
real event.Service.Record via an eventRecorderAdapter. Constructs
notify.Service with telegram + webhook senders. Spawns the retention
loop goroutine off the main wait group; gracefulShutdown now also
notifySvc.Wait()s for in-flight notifications to drain.
Closes deferred Phase-9 items 2 (real event recorder), 3 (nested
create-tier rate limits), 5 (FingerprintRecorder wiring), 7 (drop
required tag on TurnstileResp).
- cmd/canary/main.go:
* buildEventStack constructs telegram + webhook senders and registers
them on notify.Service. event.Service wired with eventRepo (Store +
StatusWriter), tokenRepo (TokenIncrementer), redis client, notifier.
* eventRecorderAdapter and a single mysqlEventRecorder both delegate
to event.Service.Record(ctx, t.NotifyInfo(), evt) — one path,
same dedup + notify semantics for HTTP triggers and mysql triggers.
* fingerprintRecorderAdapter wraps event.Repository.AttachFingerprint
with the configured window (defaults to 5 min) so the slow-redirect
fingerprint POST writes through.
* spawnRetentionLoop runs event.Service.RunRetentionLoop on the
shared wait group; configured interval + per-token limit gate it.
* /api/tokens POST gains nested create-tier limits per spec §11.3:
LimitCreateMin (5/min) + LimitCreateHour (20/hr) chained ahead of
Turnstile, with their own fingerprint-keyed Redis namespaces so
they don't share buckets with the read-tier limiter.
* Shutdown order: server → telemetry → redis → db, then
notifySvc.Wait() to drain in-flight goroutines, then wg.Wait()
for retention loop + mysql listener.
- internal/config/config.go:
* NotifyConfig: dedup_ttl, send_timeout, max_tries, max_elapsed,
initial_interval, retention_interval, retention_limit,
webhook_hmac_secret, telegram_api_base, fingerprint_window.
* RateLimitConfig: create_min_rate / burst, create_hour_rate / burst.
* Defaults + env mappings (NOTIFY_*, RATE_LIMIT_CREATE_*, plus the
spec'd WEBHOOK_HMAC_SECRET).
- internal/token/dto.go: drop the `validate:"required"` tag on
cf_turnstile_response. The middleware (TurnstileVerify) is the sole
authority — it bypasses on empty SecretKey for dev mode and would
reject empty-token submissions in prod via the Verifier's own
ErrEmptyToken path. The validator tag was forcing tests to pass
a placeholder string in dev mode which conflicted with the bypass.
Audit found that registry.Build was calling mysql.New() with hardcoded
localhost:3306 defaults, ignoring cfg.MySQL.PublicHost and PublicPort.
Result: every emitted mysql:// connection string said
"mysql://canary_<id>@localhost:3306/internal_db" regardless of actual
deployment address — useless for any non-localhost deployment.
Fix:
- registry.Config: adds MySQLPublicHost + MySQLPublicPort fields
- registry.Build: switches between mysql.New() and
mysql.NewWithAddress(host, port) based on config
- cmd/canary/main.go: passes cfg.MySQL.PublicHost/PublicPort
through to registry.Build
- registry_test.go: new TestBuild_MySQLUsesConfiguredPublicAddress
asserts the connection string reflects configured host/port
Other Phase 9 audit observations DEFERRED to Phase 10 (not blocking
rollup):
- §11.3 create-tier rate limit on POST /api/tokens (Turnstile
already provides spam protection; tiered limits are
belt-and-suspenders)
- scoping the gosec G107/G704 exclude to specific packages
(revisit when Phase 10's webhook sender lands and needs
case-by-case URL validation)
- TurnstileResp dto required-tag clashes with dev-mode bypass
(low severity — current behavior is sound)
- /api/m/{manage_id} GET/DELETE (spec §8.1 lines 740-741) — Phase 11
- FingerprintRecorder is nil at startup (slowredirect fingerprint
handler wiring) — Phase 10
- mysql_username metadata write in token.Service (mysql trigger
lookup goes by token ID, not username — low impact)
Pre-rollup gate clean at HEAD:
- go build ./... + go vet ./... clean
- go test -race -timeout=60s ./... all pass (~50 new test cases)
- go test -tags=integration -race -timeout=300s ./internal/token/...
./internal/event/... all pass
- golangci-lint run ./... → 0 issues
- grep -rn "//nolint" --include="*.go" returns nothing
Phase 9 tasks 9.7 + 9.8 + closes Phase 8 task 8.5 deferral.
config.Config additions:
- Canary {BaseURL, ManageURL} — public URLs for trigger
+ manage redirects
- Turnstile {SecretKey, SiteKey} — Cloudflare Turnstile
(empty SecretKey = dev bypass)
- MySQL {Enabled, Addr, PublicHost, — fake TCP listener config
PublicPort}
cmd/canary/main.go full wire-up per spec §6.3 + §11.1:
- Global middleware: RequestID → Logger → Recovery → SecurityHeaders
- /healthz: no further middleware (router-mounted directly)
- /c/{id}, /c/{id}/fingerprint, /k/{id}, /k/{id}/*: mounted at
router root, OUTSIDE /api subrouter, so they skip CORS + rate
limit (spec §11.1 line 1552 — we want to record EVERY hit)
- /api subrouter: CORS → RateLimit (KeyByFingerprint, FailOpen=true)
- /api/tokens/types: no further middleware (read-only public list)
- /api/tokens POST: + TurnstileVerify middleware
- mysql goroutine: spawned via spawnMySQLListener iff
cfg.MySQL.Enabled; uses mysqlTokenLookup + mysqlEventRecorder
adapters bridging mysql.{TokenLookup,EventRecorder} to
token.Service + event.Repository
Adapter types in cmd/canary/main.go:
- registryAdapter: bridges registry.Registry (map type) to
token.Registry interface (Get method)
- directEventRecorder: bridges token.EventRecorder to
event.Repository.Insert + token.Repository.IncrementTriggerCount
(synchronous; Phase 10 swaps in event.Service for dedup + async
notify)
- mysqlTokenLookup, mysqlEventRecorder: same shape for the TCP
mysql handler
webbug refactor (task 9.8):
- Delete local realIP/lastNonEmptyXFF/optionalHeader copies (~30 LOC)
- Use middleware.RealIP and middleware.OptionalHeader instead
- Tests still pass (the IP-precedence suite in webbug_test.go
exercises middleware.RealIP behavior end-to-end via the Trigger
contract)
Pre-rollup gate clean at HEAD:
- go build ./... + go vet ./... clean
- go test -race -timeout=60s ./... all pass
- go test -tags=integration -race -timeout=300s ./internal/token/...
./internal/event/... all pass
- golangci-lint run ./... → 0 issues
- grep -rn "//nolint" --include="*.go" returns nothing
DEFERRED (not in Phase 9 scope):
- Phase 9 task 9.9 integration test (full POST /api/tokens against
testcontainers) — defer to Phase 10's broader integration suite
along with event.Service + notify.Service end-to-end coverage
- Phase 10 brings: event.Service with dedup + async notify, swaps
in for the directEventRecorder/mysqlEventRecorder adapters
- Phase 13 brings: geoip.Service wired into event recording
Phase 5 first commit. Adds the committed template.pdf binary blob plus
the pure-Go one-shot utility that produced it, matching the docx Phase
4 pattern (cmd/build<format>template/main.go excluded from production).
cmd/buildpdftemplate/main.go assembles a minimal PDF-1.4 by hand:
- Header: %PDF-1.4 + 4 high-bit bytes to mark as binary
- Object 1: Catalog → Pages root
- Object 2: Pages → single Kids ref + Count 1
- Object 3: Page with /AA << /O << /Type /Action /S /URI /URI (...) >> >>
where the URI value is the 76-char placeholder
HONEY_TRACK_URL_PADDED_TO_FIXED_WIDTH______________________________________
- xref table with computed byte offsets
- trailer + startxref + %%EOF
The placeholder is a direct dictionary value (NOT inside a stream), so
substitution can be a plain byte-replace at runtime without breaking the
cross-reference table — spec §9.4 line 1133. No FlateDecode anywhere.
pdfcpu (v0.12.1) is used for validation only — the builder calls
api.Validate before writing, and tests will call it again on substituted
output to confirm the PDF stays well-formed.
Verification at HEAD:
- 477-byte template.pdf
- sha256: 70c6359016ceb780539f8c4497991b98ff82201e35a38d36090d88856027e103
- Reproducible: rebuild produces byte-identical output
- grep -aob HONEY_TRACK_URL → one offset (240)
- file: PDF document, version 1.4, 1 page(s)
- api.Validate passes
go.mod adds pdfcpu as a direct dep with its transitive deps via go mod
tidy. golang.org/x/crypto returns to direct (pdfcpu uses it for AES
encryption support).
Phase 4 Task 4.3+4.4 — docx Generator that produces a tracked Word
document by embedding template.docx and runtime-substituting the
HONEY_TRACK_URL placeholder in word/footer2.xml with the per-token
trigger URL. Preserves per-entry zip Method (STORE/DEFLATE) so
Word/LibreOffice still opens the result; non-footer entry bodies are
byte-identical to the template.
Generate returns {Kind: KindFile, Filename: t.Filename ?? "Document.docx",
Content: <patched zip>, ContentType: wordprocessingml MIME}. The
spec §9.3 reference snippet uses `t.Filename == ""` which won't
compile against the real schema — Token.Filename is *string. The
generator dereferences safely via resolveFilename, trimming and
falling back to the default for nil/empty/whitespace pointers.
Trigger mirrors webbug exactly per spec §9.3 line 1118: 200 + 43-byte
transparent GIF via pixel.Clone() + cache-control no-store + pragma
no-cache. Nil-token returns the same response with nil event
(spec §8.5 — no token-existence enumeration). realIP /
lastNonEmptyXFF / optionalHeader helpers are copied from webbug per
the standing rule (sanctioned duplication until Phase 9 middleware
extraction).
24 test cases including the six prescribed by implementation plan
§4.3 (OutputIsValidZip, FooterContainsTriggerURL,
FooterDoesNotContainPlaceholder, OtherEntriesUnchanged,
PreservesCompressionMethods, ReturnsGIFLikeWebbug) plus surrounding
correctness/regression coverage (type, kind, content type, filename
defaulting incl. whitespace, trigger-URL trailing-slash + subpath +
uniqueness, source-IP precedence with 9 subtests mirroring webbug,
GIF body independence per Trigger call, nil-token defense).
Also fixes pre-existing lint debt in cmd/builddocxtemplate from
commit f36cce9a: errcheck on the deferred f.Close (now propagates
via named return) and gosec G304 on os.OpenFile (now filepath.Clean
on the operator-supplied -out path). Rebuilding template.docx after
this change produces byte-identical output (verified via sha256sum).
Phase 4 Task 4.1+4.2 — pure-Go OOXML template builder under
backend/cmd/builddocxtemplate/. Run:
go run ./cmd/builddocxtemplate \
-out ./internal/token/generators/docx/template/template.docx
produces a minimal valid .docx (5 zip entries, mixed STORE/DEFLATE
compression) whose word/footer2.xml carries an INCLUDEPICTURE field
referencing the literal placeholder string HONEY_TRACK_URL with the
\\d switch (forces fetch-on-open, no local cache).
The committed template.docx is the embedded source for the Phase 4
docx generator (next commit). Mixed compression methods in the template
make Task 4.3's TestGenerate_PreservesCompressionMethods a meaningful
regression guard — a hardcode-DEFLATE generator would now mismatch on
[Content_Types].xml + word/_rels/document.xml.rels (both STORE).
cmd/builddocxtemplate is excluded from the production binary (separate
cmd/ subdir from cmd/canary).
Two audit agents (code-reviewer + spec-adherence) both returned PASS but
flagged substantive findings. Per fix-in-phase / no-backlog-rot, clearing
every MEDIUM + LOW in-phase.
Findings addressed:
MEDIUM — realIP RemoteAddr fallback returned "IP:port" verbatim, where
the spec wants the bare IP (geoip + downstream parsing assume host-only).
Now uses net.SplitHostPort with raw-string fallback if not host:port.
Adds 4 RemoteAddr cases: IPv4 strips port, IPv6 bracket form strips
brackets+port, loopback IPv6, and the port-less raw fallback.
MEDIUM — realIP XFF branch accepted whatever rightmost-comma-split
produced, so headers like "198.51.100.1, " (trailing comma) yielded
"" and skipped XRI/RemoteAddr entirely. Extracted into lastNonEmptyXFF
which walks right-to-left and falls through cleanly. Adds two cases:
trailing-comma falls through, all-empty entries fall through.
MEDIUM — pixel.TransparentGIF was an exported mutable []byte; any caller
could clobber it process-wide (mutating one response's body would affect
every subsequent webbug trigger). Renamed to unexported transparentGIF
+ exposed pixel.Clone() and pixel.Len(). Webbug now calls pixel.Clone()
per trigger; new test TestTrigger_ResponseBodyIsIndependentCopyPerCall
verifies two triggers return independent slices.
LOW (reviewer-escalated to HIGH-for-Phase-3) — Trigger on nil token
returned a non-nil event with empty TokenID. Since events.token_id is a
NOT NULL FK to tokens.id, the Phase 3 handler could not have persisted
that event anyway, and the contract was implicit (would have required
an inline comment, which the no-comments rule forbids). Now: nil token
in → nil event, non-nil response out. Contract is explicit in the
return shape. Test asserts evt is nil for nil-token path.
LOW — no IPv6 coverage in IP-precedence tests. Added IPv6 cases for
XFF rightmost and three RemoteAddr forms (bracketed, loopback, no-port).
NIT — gracefulShutdown swallowed every shutdown error, returning nil
unconditionally. Now collects with errors.Join so callers can detect
partial-shutdown for telemetry/exit-code purposes.
Audits accepted as-is (NIT, not blocking):
- Artifact discriminated-union shape (Phase 3+ concern when other Kinds
are produced)
- Registry returning bare map (read-only post-Build; concurrent reads
are safe by Go's memory model)
- Build(_ Config) ignoring its arg (signature reserved for future
stateful generators)
Quality verified post-fix: build/vet/lint clean (0 issues),
14 webbug tests + 6 pixel tests + 4 registry tests PASS under -race,
integration tests unchanged.
Pre-audit gate (`golangci-lint run`) at the start of Phase 2 surfaced 15
issues that should have been zeroed before Phase 1 rollup. Per the
fix-in-phase rule (no backlog rot), clearing everything before Phase 2
audit agents run on a green tree.
Config:
- .golangci.yml: migrate `issues.exclude-rules` and `issues.exclude-dirs`
to v2 syntax (`linters.exclusions.{rules,paths}`); test-file funlen/
dupl/goconst exclusion now actually applies under golangci-lint v2.10
- .golangci.yml: add G706 to gosec excludes — false-positive log-injection
reports on slog structured-logging call sites (slog separates message
from kv args, immune to log-line injection by construction)
errcheck (5):
- cmd/canary/main.go: `_ = telemetry.Shutdown(...)` → log on error
- internal/token/repository.go: `defer stmt.Close()` → log on close error
- internal/event/repository.go: same as token
- internal/testutil/postgres.go: `_ = pgContainer.Terminate(...)` and
`_ = db.Close()` → t.Logf on cleanup error (use distinct err names to
avoid govet shadow)
funlen (1):
- cmd/canary/main.go: split `run` into `run` + `initTelemetry` +
`mountRouter` + `gracefulShutdown` (was 55 statements, now under
the 50 cap; helpers are individually well under)
govet shadow (1):
- cmd/canary/main.go: migrations check switched from `if err :=` to
`if err =` (reuses outer err whose value was already consumed) — the
inner short-decl was lexically shadowing without intent
- cmd/canary/main.go: select-case `err` renamed to `startErr` to avoid
the same shadow pattern
golines (6, all auto-fixed by `golangci-lint run --fix`):
- main.go, config.go, telemetry.go, event/repository.go, token/dto.go,
token/repository.go — long lines wrapped to 80-col
Verified post-fix: build OK, vet OK, unit tests PASS under -race,
integration tests PASS under -tags=integration -race (12s testcontainers),
golangci-lint reports 0 issues.
CarterPerez-dev