RomanNum3ral
/
Cybersecurity-Projects
mirror of https://github.com/CarterPerez-dev/Cybersecurity-Projects.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

update

This commit is contained in:
CarterPerez-dev 2026-05-18 00:41:25 -04:00
parent de6ac71954
commit 615efe051d
7 changed files with 2052 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

151
PROJECTS/beginner/canary-token-generator/README.md Normal file
View File

@ -0,0 +1,151 @@
```yaml
██████╗ █████╗ ███╗ ██╗ █████╗ ██████╗ ██╗ ██╗
██╔════╝██╔══██╗████╗ ██║██╔══██╗██╔══██╗╚██╗ ██╔╝
██║ ███████║██╔██╗ ██║███████║██████╔╝ ╚████╔╝
██║ ██╔══██║██║╚██╗██║██╔══██║██╔══██╗ ╚██╔╝
╚██████╗██║ ██║██║ ╚████║██║ ██║██║ ██║ ██║
╚═════╝╚═╝ ╚═╝╚═╝ ╚═══╝╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝
```
[![Cybersecurity Projects](https://img.shields.io/badge/Cybersecurity--Projects-Project%20%2325-red?style=flat&logo=github)](https://github.com/CarterPerez-dev/Cybersecurity-Projects/tree/main/PROJECTS/beginner/canary-token-generator)
[![Go](https://img.shields.io/badge/Go-1.25+-00ADD8?style=flat&logo=go&logoColor=white)](https://go.dev)
[![React](https://img.shields.io/badge/React-19-61DAFB?style=flat&logo=react&logoColor=black)](https://react.dev)
[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-18-4169E1?style=flat&logo=postgresql&logoColor=white)](https://www.postgresql.org)
[![Redis](https://img.shields.io/badge/Redis-7-DC382D?style=flat&logo=redis&logoColor=white)](https://redis.io)
[![License: AGPLv3](https://img.shields.io/badge/License-AGPL_v3-purple.svg)](https://www.gnu.org/licenses/agpl-3.0)
[![Docker](https://img.shields.io/badge/Docker-ready-2496ED?style=flat&logo=docker)](https://www.docker.com)
[![MITRE Engage](https://img.shields.io/badge/MITRE-Engage-red?style=flat)](https://engage.mitre.org/)
> Self-hosted honeytoken generator. Mints seven kinds of tripwire artifacts — invisible web bugs, booby-trapped PDF/DOCX files, fake `.env` and kubeconfig credentials, and a real MySQL wire-protocol decoy — then alerts you on Telegram or a webhook the moment an attacker touches one.
*This is a quick overview — security theory, architecture, and full walkthroughs are in the [learn modules](#learn).*
## What It Does
- Seven token types, each disguised as something an attacker would actually try to use: `webbug`, `slowredirect`, `pdf`, `docx`, `envfile`, `kubeconfig`, and a real `mysql` listener that speaks the MySQL v10 handshake
- Per-token Telegram or webhook alerts the instant a token fires (HMAC-signed for webhooks)
- Async notification worker pool with per-channel timeouts and dedup gating (15-minute Redis silence window per `{token, source_ip}` so a curious attacker reloading the page doesn't spam you)
- GeoIP enrichment via MaxMind GeoLite2 (country, region, city, ASN, ASN org) attached to every event
- Browser fingerprint capture for `slowredirect` tokens via a 3-second JS-collection interstitial before the redirect resolves
- Public manage URL (UUID-gated) so you can share a single link with a teammate to view triggers without exposing operator credentials
- Operator-only admin API (constant-time bearer comparison) for global stats, token listing, and force-disable
- Cloudflare Turnstile on token creation, dual-window rate limiting (per-minute + per-hour) keyed by browser fingerprint
- Optional Cloudflare Tunnel overlay — expose the service publicly without opening a port or maintaining a TLS cert
- Defense-grade observability: OpenTelemetry traces, slog structured logs, `/healthz` liveness, graceful shutdown with load-balancer drain delay
## Quick Start
```bash
just init # generates .env, .env.development, randomised ports, operator token
just dev-up # launches nginx + Vite HMR + Go (Air hot-reload) + Postgres + Redis + Jaeger
```
Open the URL printed by `just init` (typically `http://localhost:22784`). Mint a token, watch the manage page, then trigger it from another tab and refresh.
> [!TIP]
> This project uses [`just`](https://github.com/casey/just) as a command runner. Type `just` to see every available recipe grouped by area.
>
> Install: `curl -sSf https://just.systems/install.sh | bash -s -- --to ~/.local/bin`
## Token Types
| Type | Artifact | Trigger Mechanism | Where You'd Plant It |
|------|----------|-------------------|----------------------|
| `webbug` | URL to a 1x1 JPEG | Any HTTP GET on the URL | HTML emails, internal wikis, "do-not-touch" docs |
| `slowredirect` | URL with a delayed redirect | Click-through; runs fingerprint JS before redirecting to a real destination | Phishing-bait links in honeyfile chat threads, fake admin panel URLs |
| `pdf` | Patched PDF | Acrobat opens, fires `/AA /O /URI` page-open action | `payroll-q3.pdf` on a shared drive, `vpn-creds.pdf` on a workstation |
| `docx` | Patched Word doc | Word/LibreOffice loads the footer, which contains a remote URI | `customer-list.docx`, `passwords.docx` in `Documents/` |
| `envfile` | Plain-text `.env` | Attacker `curl`s the fake `INTERNAL_METRICS_ENDPOINT` baked into the file | Repository roots, `~/.config/`, container `/app/` directories |
| `kubeconfig` | YAML kubeconfig | Attacker runs `kubectl --kubeconfig=stolen.yaml ...` and our server logs the bearer token | `~/.kube/config`, ops engineer laptops, CI runner home dirs |
| `mysql` | `mysql://...` connection string | Attacker connects with `mysql` CLI; our TCP listener replies with a real MySQL v10 handshake and an `Access denied` packet | `.env` files, `database.yml`, internal wiki snippets |
The `envfile` generator is the densest of the bunch. It picks recipes from `aws.go`, `db.go`, `github.go`, and `stripe.go`, shuffles the resulting sections, and buries a single canary line (`INTERNAL_METRICS_ENDPOINT=https://your-host/c/{tokenID}`) among plausible production config. The attacker harvesting the file gets a fistful of fake secrets to chase *and* trips the wire as soon as one of those secrets is touched.
## HTTP API
Token creation, manage view, and admin are mounted under `/api/`. Trigger routes live at the root so artifacts can carry short URLs.
| Method | Path | Auth | Purpose |
|--------|------|------|---------|
| `POST` | `/api/tokens` | Turnstile + rate limit | Mint a new token (`type`, `memo`, `alert_channel`, channel config, type-specific metadata) |
| `GET` | `/api/tokens/types` | Public | List available token types and their metadata schemas |
| `GET` | `/api/m/{manageId}` | Manage UUID | Token details + paginated event feed + dedup silence counter |
| `DELETE` | `/api/m/{manageId}` | Manage UUID | Soft-disable the token (events stop, history retained) |
| `GET` | `/api/admin/stats` | Bearer | Tokens count, events count, breakdowns by type and alert channel |
| `GET` | `/api/admin/tokens` | Bearer | All tokens (offset paginated) |
| `POST` | `/api/admin/tokens/{id}/disable` | Bearer | Force-disable any token |
| `GET` | `/healthz` | Public | Liveness + readiness probe (used by Docker healthchecks) |
| `GET` | `/c/{tokenID}` | Public | **Trigger route.** Records event, fires notification, returns artifact body (pixel, GIF, HTML interstitial, etc.) |
| `POST` | `/c/{tokenID}/fingerprint` | Public | Receives JSON fingerprint payload from the `slowredirect` interstitial |
| `*` | `/k/{tokenID}[/*]` | Public | Kubeconfig trigger — matches `kubectl`'s wildcard API paths |
The MySQL listener does not run on HTTP. It's a separate TCP server bound to a configurable address; an attacker using the connection string from the artifact will speak the MySQL wire protocol with our `protocol.go` handshake builder before getting denied.
## Stack
**Backend:** Go 1.25, chi router, pgx + sqlx, goose migrations, koanf config, slog, validator/v10, OpenTelemetry, pdfcpu, MaxMind GeoLite2, miniredis (tests), testcontainers (integration)
**Frontend:** React 19, TypeScript, Vite, TanStack Query, Zod, Axios, Biome, Stylelint
**Storage:** PostgreSQL 18 (`tokens`, `events` tables; INET + JSONB columns), Redis 7 (dedup gate + rate-limit token buckets)
**Infra:** Docker Compose (dev: nginx + Vite HMR + Air + Postgres + Redis + Jaeger; prod: nginx + Go binary + Postgres + Redis), optional `cloudflared` overlay
## Project Layout
```
canary-token-generator/
├── backend/
│ ├── cmd/canary/ main.go — wiring, signal handling, MySQL listener spawn, retention loop
│ └── internal/
│ ├── token/ Service, repository, handler, generator interface
│ │ └── generators/
│ │ ├── webbug/ Embedded JPEG pixel
│ │ ├── pixel/ Shared 1x1 GIF helper used by every "visited" response
│ │ ├── pdf/ Byte-exact PDF placeholder substitution (76-byte URL window)
│ │ ├── docx/ ZIP-aware footer rewrite
│ │ ├── envfile/ Recipe shuffler + canary line injection
│ │ │ └── recipes/ aws.go, db.go, github.go, stripe.go
│ │ ├── kubeconfig/ text/template renderer + wildcard /k/ handler
│ │ ├── mysql/ protocol.go (handshake + auth + err packets), server.go (TCP), handler.go
│ │ └── slowredirect/ HTML interstitial + fingerprint POST handler
│ ├── event/ Event entity, service (geo enrich → insert → dedup → notify), repository
│ ├── notify/ Worker pool, queue, status writer
│ │ ├── webhook/ HMAC-signed POSTs with exponential backoff
│ │ └── telegram/ Bot API client
│ ├── middleware/ request_id, logging, recovery, realip, fingerprint, ratelimit, turnstile, operator_bearer, headers
│ ├── geoip/ MaxMind MMDB lookup wrapper (nop when no DB present)
│ ├── turnstile/ Cloudflare Turnstile siteverify
│ ├── admin/ Stats, listing, force-disable
│ ├── core/ DB pool, Redis client, migrations, telemetry, errors, validation, response envelopes
│ ├── health/ /healthz handler with readiness/shutdown flags
│ └── server/ chi router shell with graceful shutdown + drain delay
├── frontend/
│ └── src/
│ ├── pages/landing/ Token creation form (type-aware metadata, Turnstile widget, artifact reveal)
│ └── pages/manage/ Token detail + event table (cursor paginated, GeoIP cells, dedup silence)
├── infra/
│ ├── nginx/ prod.nginx, dev.nginx (Vite proxy)
│ └── docker/ Dockerfiles for prod binary, Air hot-reload, Vite HMR
├── compose.yml Production stack
├── dev.compose.yml Dev stack with Jaeger
├── cloudflared.compose.yml Tunnel overlay
├── justfile Recipes grouped by frontend / backend / lint / compose / tunnel / dev / util
└── learn/ You are here
```
## Learn
This project includes step-by-step learning materials covering deception theory, token mechanics, system design, and a code walkthrough.
| Module | Topic |
|--------|-------|
| [00 - Overview](learn/00-OVERVIEW.md) | Prerequisites, quick start, project structure |
| [01 - Concepts](learn/01-CONCEPTS.md) | Honeytokens, deception defense, Thinkst Canary, MITRE Engage, real breaches |
| [02 - Architecture](learn/02-ARCHITECTURE.md) | System design, request lifecycle, schema, dedup gate, notification pipeline |
| [03 - Implementation](learn/03-IMPLEMENTATION.md) | Code walkthrough: generators, trigger handler, event service, MySQL protocol |
| [04 - Challenges](learn/04-CHALLENGES.md) | Extension ideas — new token types, alert channels, evasion-resistance |
## License
AGPL 3.0

207
PROJECTS/beginner/canary-token-generator/learn/00-OVERVIEW.md Normal file
View File

@ -0,0 +1,207 @@
<!-- © AngelaMos | 2026 | 00-OVERVIEW.md -->
# Canary Token Generator
## What This Is
A self-hosted honeytoken service. You mint a token through the web UI, drop the resulting artifact somewhere an attacker is likely to find it, and then sit back. The moment the artifact is opened, fetched, or used, the server records the event with IP, GeoIP, user agent, and (for some token types) a browser fingerprint — then fires a Telegram message or webhook to alert you.
It is the open-source spiritual cousin of Thinkst Canary and the Canarytokens.org service. Seven token types ship in the box: an invisible web bug, a slow-redirect link that captures a fingerprint before forwarding the victim onward, booby-trapped PDF and DOCX files, fake `.env` and kubeconfig credentials, and a real MySQL listener that completes the v10 handshake so an attacker's `mysql` CLI shows a plausible "Access denied" error while we log the bearer token they tried.
The backend is Go on chi, with PostgreSQL for tokens and events, Redis for dedup and rate limiting, MaxMind GeoLite2 for IP geolocation, and an async worker pool for outbound notifications. The frontend is React 19 with TanStack Query.
## Why This Matters
Mean time to detection for a breach is, depending on whose annual report you read, somewhere between two and eight *months*. The reason is straightforward: defenders watch the obvious signal sources — failed logins, IDS alerts, EDR telemetry — and skilled attackers know to stay out of those. By the time anyone notices, the attacker has been moving laterally for a long time.
Deception flips the asymmetry. A canary token has zero false-positive rate by construction. No legitimate user will ever connect to a database whose connection string only exists in a planted `.env` file. No real employee will open a PDF named `network-diagram-DO-NOT-SHARE.pdf` that nobody told them about. So when one of those events fires, you don't argue with it — you start the incident response clock and you already know the attacker is past your first-line controls.
This is not theoretical. The most famous canary-driven detection is probably the 2013 Target breach analysis, where investigators reconstructed the attacker's movement through credentialed pivots that would have been visible to honeytokens in those service accounts. More recently, MITRE published the Engage framework — a deliberate counterpart to ATT&CK that catalogues deception techniques and how defenders deploy them in real environments.
If you work on a blue team, this project shows you exactly how the primitives work: how a PDF can be patched byte-exact without breaking it, why the MySQL wire protocol is forgiving enough to fake, how dedup gates prevent a noisy attacker from drowning your alert channel, and how to enrich events with GeoIP and fingerprint data so the triage call goes faster.
## What You'll Learn
**Security Concepts:** Deception-based defense. Honeytoken mechanics across multiple file formats. The MITRE Engage framework and how it complements ATT&CK. Browser fingerprinting (the same technique advertisers use, but pointed at adversaries). Operational security for the canary infrastructure itself — why constant-time bearer comparison matters, why the trigger route returns the artifact body even when the token is invalid (enumeration resistance), and why dedup runs against `{token, source_ip}` rather than just `source_ip`. Detection patterns from real breach reports.
**Technical Skills:** Idiomatic Go service design — domain layout (`token/`, `event/`, `notify/`), generator interface with seven implementations, repository pattern over pgx/sqlx, slog for structured logging, koanf for layered config, goose for migrations, chi for routing. Async worker pools with bounded queues. Redis as a coordination primitive (atomic SETNX-based dedup, token-bucket rate limiting via `go-redis/redis_rate`). MySQL wire-protocol implementation from the spec. PDF byte-exact patching that preserves cross-reference offsets. DOCX manipulation as a ZIP archive. React 19 with TanStack Query, Zod, and a TypeScript-strict build. Docker Compose multi-service orchestration with health checks, drain delays, and a Cloudflare Tunnel overlay for zero-port-exposure deployments.
**Tools:** chi (router), pgx + sqlx (Postgres), goose (migrations), koanf (config), slog (logging), validator/v10 (request validation), pdfcpu (PDF parsing for the build-template command), MaxMind GeoLite2 (IP geolocation), Cloudflare Turnstile (bot mitigation on token creation), Cloudflare Tunnel (publication), miniredis + testcontainers (tests), OpenTelemetry + Jaeger (tracing), TanStack Query + Zod + Vite (frontend), Biome + Stylelint (frontend linting), Air (Go hot reload in dev).
## Prerequisites
### Required
- Go 1.25+
- Node.js 22+
- Docker and Docker Compose
- A passing familiarity with HTTP, TCP, and the request/response lifecycle
### Required Tools
- **just** as a command runner. Install: `curl -sSf https://just.systems/install.sh | bash -s -- --to ~/.local/bin`
- **pnpm** for Node package management (never npm). Install: `corepack enable && corepack prepare pnpm@latest --activate`
- **Docker Compose** (bundled with Docker Desktop or installed as a plugin)
### Optional But Useful
- A Telegram bot token + chat ID, or a webhook receiver (e.g. https://webhook.site), so you can see alerts arrive on a real channel
- A free MaxMind GeoLite2 account to download the City database — without it, events still record but the `geo_*` columns stay null
- A Cloudflare Turnstile site key + secret if you want to test the bot-mitigation flow on `POST /api/tokens`
- A Cloudflare Tunnel token if you want to expose your local instance to the public internet through `just tunnel-up`
## Quick Start
```bash
git clone https://github.com/CarterPerez-dev/Cybersecurity-Projects.git
cd Cybersecurity-Projects/PROJECTS/beginner/canary-token-generator
just init # generates .env + .env.development, picks free random ports, writes an operator token
just dev-up # nginx → Vite + Air-reloaded Go → Postgres → Redis → Jaeger
```
`just init` prints the URL it picked. Typically that's `http://localhost:22784`, but it will pick a different port if 22784 is taken.
Visit that URL. You'll land on the token creation form. Pick `Web Bug` from the dropdown, type "test token" in the memo, choose webhook as the alert channel, paste a https://webhook.site URL, then submit. The page renders the trigger URL and a manage URL.
Open the manage URL in a second tab. Open the trigger URL in a third tab — you should see a tiny broken-image placeholder (the 1x1 JPEG, which the browser does fetch but doesn't render meaningfully). Refresh the manage tab. The event table now has one row: your local IP, your user-agent, an empty GeoIP cell (you're on localhost), and a `pending` notify status that flips to `sent` once the webhook fires.
Check webhook.site. You should see a POST with the token ID, event ID, timestamp, source IP, user agent, and an HMAC-SHA256 signature in the `X-Canary-Signature` header.
Try the other token types from the dropdown. The `.env` and kubeconfig artifacts arrive as downloadable files. The PDF and DOCX arrive as base64-encoded content the UI converts to file downloads. The `slowredirect` token asks you for a destination URL; trigger it in another tab and you'll watch a 3-second interstitial run a fingerprinting script before redirecting you. Open the manage tab afterward — the event row now has a fingerprint JSON blob attached.
## Project Structure
```
canary-token-generator/
├── backend/
│ ├── cmd/canary/main.go Wiring: config → DB → Redis → GeoIP → notify worker pool → router → MySQL listener → retention loop
│ ├── cmd/buildpdftemplate/ One-shot tool: bakes the PDF placeholder into template.pdf
│ ├── cmd/builddocxtemplate/ One-shot tool: bakes the DOCX placeholder into the footer
│ ├── cmd/healthcheck/ Tiny Docker HEALTHCHECK binary (no curl needed in the image)
│ └── internal/
│ ├── token/
│ │ ├── entity.go Token domain type + NotifyInfo + Manage view
│ │ ├── repository.go pgx/sqlx-backed CRUD
│ │ ├── service.go Generate → persist → return artifact
│ │ ├── handler.go POST /api/tokens, GET/DELETE /api/m/{manageId}, GET /c/{id} trigger
│ │ ├── types.go Type constants (webbug, pdf, ...)
│ │ ├── dto.go Create/Manage request/response shapes + validation tags
│ │ ├── contract.go Interfaces for repository, generator registry, event recorder
│ │ └── generators/
│ │ ├── generator.go Generator interface + Artifact + TriggerResponse types
│ │ ├── registry/ Map of Type → Generator
│ │ ├── webbug/ Embedded JPEG pixel
│ │ ├── pixel/ Shared 1x1 GIF helper for "visited" responses
│ │ ├── pdf/ Byte-exact placeholder substitution against embedded template.pdf
│ │ ├── docx/ ZIP rewrite of word/footer2.xml
│ │ ├── envfile/ Recipe shuffler + canary line injection
│ │ │ └── recipes/ aws.go, db.go, github.go, stripe.go — produce realistic fake secrets
│ │ ├── kubeconfig/ text/template render + wildcard /k/ handler
│ │ ├── mysql/ protocol.go (handshake/auth/err), server.go (TCP), handler.go, generator.go
│ │ └── slowredirect/ HTML interstitial + POST /c/{id}/fingerprint handler
│ ├── event/ Event entity, service, repository, contract
│ ├── notify/ Async worker pool (queue + dispatcher), status writer
│ │ ├── webhook/ HMAC-signed POST with exponential backoff
│ │ └── telegram/ Bot sendMessage client
│ ├── middleware/ request_id, logging, recovery, realip, fingerprint, ratelimit, turnstile, operator_bearer, headers
│ ├── geoip/ MaxMind MMDB lookup (nop service when no DB)
│ ├── turnstile/ Cloudflare siteverify
│ ├── admin/ Stats, listing, force-disable
│ ├── core/ DB pool, Redis client, migrations driver, telemetry init, errors, response envelopes
│ ├── health/ Liveness + readiness flags
│ └── server/ chi router shell with drain delay + graceful shutdown
├── frontend/src/
│ ├── api/ Axios client, hooks, types (Zod schemas)
│ ├── pages/landing/ Token creation form, type-aware metadata, Turnstile widget, artifact reveal
│ └── pages/manage/ Token detail + event feed (cursor paginated, GeoIP, dedup silence count)
├── infra/
│ ├── nginx/ prod.nginx, dev.nginx (Vite HMR proxy)
│ └── docker/ Dockerfiles for prod binary, Air dev, Vite dev
├── compose.yml Production stack
├── dev.compose.yml Dev stack with Jaeger
├── cloudflared.compose.yml Tunnel overlay
├── justfile Recipes grouped by frontend / backend / lint / compose / tunnel / dev / util
└── learn/ You are here
```
## How It Works (Brief)
```
┌──────────────────────────────┐
│ Operator (browser) │
│ React 19 + TanStack Query │
└──────────────┬───────────────┘
│
POST /api/tokens
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ nginx → chi router (Go) │
│ Middleware chain: │
│ request_id → logger → recovery → SecurityHeaders → CORS → │
│ fingerprint-keyed rate limit → Turnstile → handler │
└──────────────────────────────┬──────────────────────────────────┘
│
token.Service.Create()
│
┌────────────────┼─────────────────┐
▼ ▼ ▼
generator.Generate repo.Insert return Artifact
(webbug | pdf | (Postgres) (URL | file |
docx | envfile | connection string)
kubeconfig |
slowredirect |
mysql)
--- attacker opens artifact ---
┌──────────────────────────────────────────────┐
│ GET /c/{tokenID} (or TCP for mysql) │
└──────────────────────┬───────────────────────┘
│
token.Handler.HandleTrigger
│
generator.Trigger(token, request)
│
event.Service.Record(notifyInfo, evt)
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
geo.Lookup(IP) repo.Insert(evt) Redis SETNX
(GeoLite2) (Postgres) dedup:{id}:{ip}
│ │
│ ┌──────────┴────────┐
│ ▼ ▼
│ first hit? duplicate?
│ │ │
│ notify.Notify UpdateNotifyStatus
│ │ (deduped)
│ ▼
│ worker pool (8) → telegram | webhook
│ │
▼ ▼
UpdateNotifyStatus(sent | failed)
```
The flow is deliberately small. The trigger handler does almost no work synchronously — it records the event and hands off to the notification worker pool. That asymmetry matters: a curious attacker auto-fetching the URL from a thousand processes won't slow the response and won't crash the alert path either, because the worker queue is bounded and drops with a `failed` status instead of blocking ingestion.
## Next Steps
- [01 - Concepts](01-CONCEPTS.md): The honeytoken idea, Thinkst Canary's commercial product, the MITRE Engage framework, real breaches that honeytokens caught (and ones they would have)
- [02 - Architecture](02-ARCHITECTURE.md): Schema, request lifecycle, the dedup gate, the notification pipeline, enumeration resistance, why MySQL gets its own listener
- [03 - Implementation](03-IMPLEMENTATION.md): Walkthroughs of the generators (PDF byte-exact substitution, DOCX ZIP rewrite, MySQL v10 handshake), the trigger handler, the event service, and the worker pool
- [04 - Challenges](04-CHALLENGES.md): New token types (DNS, SMTP, AWS-API), new alert channels (Slack, PagerDuty), evasion-resistance hardening
## Common Issues
**`just init` says "no free port found":** The init script tries ranges and gives up if everything is in use. Edit `scripts/init.sh` to widen the range, or pass `NGINX_HOST_PORT=...` explicitly in `.env`.
**The Vite dev server logs `optimizing dependencies` forever:** First-run Vite cold start. Wait. Subsequent starts are fast.
**Manage page shows empty GeoIP fields for an IP that's clearly not localhost:** You haven't mounted a GeoLite2 City MMDB. Get a free MaxMind license, drop the `.mmdb` in `data/geoip/GeoLite2-City.mmdb`, set `GEOLITE_PATH` in `.env`, and restart.
**Notification never arrives but `notify_status` says `sent`:** The worker succeeded but Telegram/your webhook receiver rejected silently. Inspect the canary logs (`just logs canary`) for the request body — slog logs every send at debug level.
**Notification stuck at `pending` and never moves:** The worker pool is wedged. Most likely cause is a webhook URL that hangs forever; the 30-second per-job timeout should release it. If it doesn't, you're looking at a goroutine leak — file an issue.
**MySQL listener won't bind:** Port 3306 is probably taken by a local MySQL install. Set `MYSQL_FAKE_PORT` to something else in `.env` (the artifact's connection string uses `MYSQL_PUBLIC_HOST:MYSQL_PUBLIC_PORT`, so the public-facing values can differ from the internal listener address — useful when you're behind a port-forwarder).

279
PROJECTS/beginner/canary-token-generator/learn/01-CONCEPTS.md Normal file
View File

@ -0,0 +1,279 @@
<!-- © AngelaMos | 2026 | 01-CONCEPTS.md -->
# Honeytoken Concepts
This document covers the security theory behind canary tokens — what they are, why they work, how they fit into the broader deception-defense playbook, and how each of the seven token types in this project maps to attacker behaviour we see in real incident reports. Read time is roughly 20-25 minutes.
---
## Deception-Based Defense
### The Core Idea
The standard mental model for defense is a perimeter: walls, gates, guards. Stop the attacker at the edge. The problem is that real adversaries get past the edge routinely. Phishing works. Vulnerable web apps get pwned. Insiders go rogue. By the time a sophisticated attacker is in your network, the perimeter is no longer the question — the question is how long they get to wander around before someone notices.
The industry numbers on that question are bad. The 2024 IBM "Cost of a Data Breach" report puts the global mean time to identify a breach at 194 days. The 2024 Verizon DBIR shows that for breaches involving stolen credentials — the most common pattern — the median time from compromise to detection is measured in months. The 2024 Mandiant M-Trends report puts the global median dwell time at 10 days, which is the first time it's dropped below two weeks, and even that improvement is mostly attributed to ransomware actors who detonate quickly enough to be loud. For stealthy financially-motivated and state-sponsored intrusions, dwell times of 6-12 months are still routine.
That gap is what deception is designed to close. The pitch is straightforward: plant artifacts that no legitimate user has any reason to touch, then watch them. If anyone touches one, you have a high-confidence signal that someone is in your environment who shouldn't be, and you have it independent of whether your EDR, SIEM, or NIDS produced anything.
The first formal articulation of this idea in modern security is usually credited to Clifford Stoll's 1989 book *The Cuckoo's Egg*, where he tracked a KGB-affiliated attacker through Lawrence Berkeley National Lab by leaving fake military documents on accessible systems and waiting for the attacker to read them. The contemporary commercial version is Thinkst Canary, which Haroon Meer started shipping in 2015 and which has had a noticeably outsized impact on how mature security programs think about detection.
### Honeypots vs Honeytokens
These get conflated a lot. They're related but operationally distinct.
A **honeypot** is a fake system — a fake SSH server, a fake industrial control device, a fake database. You stand it up somewhere, attackers find it, you watch what they do. The point is usually intelligence: understanding what the attacker tries, what tools they use, what credentials they have. Cowrie, T-Pot, and Conpot are well-known open-source honeypots.
A **honeytoken** is a fake artifact — a fake credential, a fake document, a fake URL. It lives inside your real environment. The point is usually detection: nobody has any reason to touch it, so the moment somebody does, you know they shouldn't be there.
The two complement each other. Honeypots tell you who's at your perimeter. Honeytokens tell you who's already inside. The tradeoff is mostly operational complexity — a honeypot is a system you have to maintain, monitor, and isolate from the rest of your network. A honeytoken is a file. You drop it where it belongs and forget about it until it fires.
This project builds honeytokens.
### Why Honeytokens Have Near-Zero False Positives
This is the killer feature. A typical EDR alert has a non-trivial false positive rate — legitimate admin behaviour looks weird, IT tools trigger heuristics, edge cases happen. SOC analysts spend a lot of their time chasing alerts that turn out to be benign, which creates alert fatigue, which is one of the ways real breaches go unnoticed.
Canary tokens are designed so that the false-positive rate is structurally close to zero. The token is something nobody has any reason to touch. The `payroll-q3.pdf` on the file share is a file that exists only as bait — no employee was told about it, no automated process indexes it, no backup job opens it. So when the page-open action fires from a workstation in Belarus at 2am, you don't argue with the alert. You wake up the on-call.
The qualifier "structurally close to zero" matters because the rate isn't actually zero. You get false positives when:
- An automated DLP or backup tool *does* open the file (DLP tools that "render" document previews are the classic offender). The fix is to exclude your tooling explicitly or to plant tokens where those tools won't reach.
- A search-engine crawler hits a URL token that leaked into a public page. The fix is to either tag the page `noindex` or accept the crawler as a benign trigger you've categorized.
- A staff member stumbles across the bait file out of curiosity. This is rare and is itself useful signal — it tells you a human is browsing places they don't belong.
The dedup gate in this project (15-minute Redis silence per `{token, source_ip}`) is partly a defense against the first two cases. If a crawler hits a URL token, you get one alert, not 200.
### MITRE Engage
MITRE Engage is the defensive companion to ATT&CK. ATT&CK is a taxonomy of what attackers do; Engage is a taxonomy of what defenders can do to deceive, deny, disrupt, and direct adversaries. It launched in 2021 (replacing the older Shield project) and lives at [https://engage.mitre.org/](https://engage.mitre.org/).
The relevant Engage activities for this project:
| Activity ID | Name | What It Means |
|-------------|------|---------------|
| EAC0002 | Detect | Get visibility into adversary activity through deception |
| EAC0003 | Direct | Steer the adversary toward a deception artifact you control |
| SAC0011 | Lures | Use artifacts to lure adversaries toward deceptive content |
| SAC0007 | Burn-In | Make deception artifacts look authentic enough to attract interest |
Every token type in this project maps to **SAC0011 Lures**. The `slowredirect` token additionally implements **EAC0003 Direct** by sending the attacker to a controlled destination after the fingerprint capture. Burn-in (SAC0007) — making the artifact look real — is the part of the work that's *not* in the code; it's in how you name your `.env` file, what you put in the memo of your kubeconfig, and how convincingly you place the `customer-list.docx` on the share.
---
## Honeytoken Mechanics
Each token type relies on a specific behaviour of a specific format or protocol. Understanding the mechanism makes it clear *why* the token works and which adversaries it catches.
### 1. Web Bug (HTTP-fetched pixel)
**File:** `backend/internal/token/generators/webbug/generator.go`
**Mechanism:** Any HTTP client that fetches the URL fires the token. The artifact is a URL pointing at a 1x1 JPEG. The response includes `Cache-Control: no-store, no-cache, must-revalidate, max-age=0` and `Pragma: no-cache` so that proxies and the browser cache don't suppress a second fetch.
**Why JPEG instead of the classic 1x1 GIF:** JPEGs are slightly larger and triggered some older anti-tracking heuristics less often during testing. The internal pixel helper in `backend/internal/token/generators/pixel/pixel.go` does provide a GIF, which is used by other generators that just need a non-empty "you visited" body (PDF, DOCX, envfile, slowredirect after fingerprint POST).
**Who you catch:** Anyone who renders an HTML email that contains the URL, anyone who follows a link from a Slack/Teams preview-bot, anyone running a vulnerability scanner that follows URLs. The technique is exactly the same as marketing email pixel-trackers — both sides of the deception/surveillance fence use it.
**Real-world parallel:** In 2016, Mossack Fonseca (the Panama Papers law firm) was breached, and one of the post-mortem recommendations from multiple sources was deploying tracker pixels on internal documents so that exfiltrated copies would phone home when the attackers opened them. Web bugs in the Thinkst Canary product fire this signal routinely in customer environments.
### 2. Slow Redirect (HTML interstitial with fingerprint capture)
**Files:** `backend/internal/token/generators/slowredirect/generator.go` and `fingerprint_handler.go`
**Mechanism:** The artifact is a URL the attacker clicks expecting a normal redirect. The server returns an HTML page (`template.html`) instead. That page runs a fingerprinting script that POSTs back to `/c/{tokenID}/fingerprint` with a JSON payload (canvas hash, WebGL renderer, font list, time-zone offset, CPU cores, etc.). After a 3-5 second delay the page redirects to the operator-specified destination so the attacker doesn't realise anything weird happened.
**Why a delay:** Two reasons. First, the JS needs time to run — fingerprinting is synchronous-ish and finishes in milliseconds, but networks aren't always fast. Second, an instantaneous redirect would feel suspicious; users expect link-shorteners and URL-cleaners to take a beat.
**Why CSP:** The response sets `Content-Security-Policy: default-src 'none'; script-src 'unsafe-inline'; connect-src 'self'` so the inline fingerprinting JS can run but the attacker's browser can't be coerced into loading external resources from our trigger page. Even though *we* control the HTML, treating it like an XSS sink is hygiene that matters when the page is technically attacker-influenced.
**Who you catch:** Anyone who clicks a phishing-bait link in a planted document, in a fake admin panel URL, in a "leaked" Slack message. The fingerprint is the differentiator — for the other URL-based token types you only get IP and User-Agent, both trivially spoofable. Canvas hashes, WebGL renderer strings, and font lists are much harder to spoof reliably and let you correlate the same attacker across multiple fires.
**Real-world parallel:** Browser fingerprinting is the technique advertising networks use to track users across sites without cookies. Same primitive, pointed at adversaries. Mozilla's research at [https://wiki.mozilla.org/Security/Fingerprinting](https://wiki.mozilla.org/Security/Fingerprinting) catalogues the surface in detail.
### 3. PDF (page-open URI action)
**File:** `backend/internal/token/generators/pdf/generator.go`, with embedded template at `template/template.pdf`
**Mechanism:** The PDF spec allows a page to declare an "Additional Action" (`/AA`) dictionary. The `/O` key inside that dictionary specifies an action to take when the page is opened. One legal action type is `/URI`, which causes the PDF reader to fetch a URL. When Acrobat opens the document, it follows the URI; that fires our trigger.
**Why a byte-exact substitution:** A PDF is fundamentally a stream of objects with byte-offset cross-references. If you rewrite a string and accidentally change the file length, every offset in the xref table is wrong and most readers will refuse to open the file. The trick: bake a fixed-width placeholder (`HONEY_TRACK_URL_PADDED_TO_FIXED_WIDTH`, 76 bytes) into the template, then replace it with a URL padded with `?p=____...` until it's exactly 76 bytes. Same length, same offsets, file still valid.
**Why 76 bytes:** Long enough to hold `https://canary.example.com/c/abc123xyz789` plus some headroom for longer hostnames. If the operator's `PUBLIC_BASE_URL` makes the URL exceed 76 characters, the generator returns `ErrTriggerURLTooLong` rather than corrupt the file.
**Caveat:** Chrome and Firefox's built-in PDF viewers (PDFium and PDF.js) deliberately strip Action objects. So this token type only fires when the attacker opens the document in Acrobat, Foxit, Preview, or any other "real" PDF reader. In practice that's most of the relevant cases — attackers exfiltrating documents typically look at them on a workstation with Acrobat or Preview, not in their browser.
**Real-world parallel:** Page-open URI actions are also the mechanism behind a class of PDF malware delivery, which is why some EDR products treat them as suspicious. We're using the same primitive defensively.
### 4. DOCX (footer with remote URI)
**File:** `backend/internal/token/generators/docx/generator.go`
**Mechanism:** A `.docx` file is a ZIP archive with a defined directory structure. The footer (typically `word/footer2.xml`) is an XML fragment that Word renders at the bottom of every page. We bake a placeholder URL into the footer and rewrite the ZIP archive entry with the real URL when the token is minted. When Word, LibreOffice, or Pages opens the document, it loads the footer, sees the URL, and fetches it.
**Why footer2.xml specifically:** Word's relationship files (`.rels`) decide which footer XML the document uses. Different templates use different paths (`footer1.xml`, `footer2.xml`, `footer3.xml`). Our `buildocxtemplate` build tool is responsible for picking the right one for the template we ship; the runtime generator just preserves whichever filename the template uses.
**Why ZIP-aware rewrite:** You cannot just `sed -i` a `.docx` file — the ZIP central directory records the size of each entry, and if you change the entry size without rewriting the central directory the file becomes corrupt. The generator opens the template as a `zip.Reader`, streams each entry to a `zip.Writer`, applies the URL substitution only to the footer entry, and lets the writer rebuild the central directory.
**Who you catch:** Anyone opening the stolen document in Word, LibreOffice, or Pages. Cloud-based viewers (Google Docs, Office Online) sometimes don't fire the request, which is a known limitation.
**Real-world parallel:** Office documents with remote-resource references have been a malware delivery vector for years (Follina / CVE-2022-30190 is the most famous recent example, though that one used MSDT not a footer URI). Same mechanism, pointed defensively.
### 5. Envfile (fake `.env` with embedded canary)
**Files:** `backend/internal/token/generators/envfile/generator.go` and `recipes/{aws,db,github,stripe}.go`
**Mechanism:** The artifact is a plain-text `.env` file containing a mix of fake credentials. The generator picks 2-4 recipes (each producing a realistic block of fake `AWS_ACCESS_KEY_ID=...`, database URL, GitHub token, or Stripe key), shuffles the sections in random order, and then appends a single canary line:
```
INTERNAL_METRICS_ENDPOINT=https://your-canary-host/c/{tokenID}
INTERNAL_METRICS_TOKEN=tok_live_{32-char-random}
```
The first time the attacker tries to use the `INTERNAL_METRICS_ENDPOINT` — typically by curling it as part of recon, or by running the application that consumed the `.env` and hit the endpoint as part of startup — the token fires.
**Why fake credentials matter:** Burn-in. An `.env` that's just `INTERNAL_METRICS_ENDPOINT=...` and nothing else screams honeypot. An `.env` with five sections of plausible AWS/DB/GitHub/Stripe credentials and *one* canary line buried in the middle looks like an unsanitised production config file the developer forgot to gitignore. The recipes are deliberately constructed to match real credential formats (AWS keys start with `AKIA`, Stripe live keys start with `sk_live_`, GitHub PATs start with `ghp_`, etc.).
**Real-world parallel:** Honey credentials in `.env` files are one of the highest-ROI deception placements in modern environments. The reason is that scraping `.env` files is one of the first things automated cloud-credential-stealer malware does (the [TeamTNT campaign](https://www.trendmicro.com/en_us/research/22/g/teamtnt-targeting-aws-alibaba.html) and various successors are well-documented examples). If you plant a single honey `.env` on a developer laptop or CI runner, you catch credential-stealer malware almost immediately on infection.
### 6. Kubeconfig (fake cluster credential)
**Files:** `backend/internal/token/generators/kubeconfig/generator.go`, `handler.go`, template at `template.yaml.tmpl`
**Mechanism:** The artifact is a Kubernetes `kubeconfig` YAML file pointing at our server's `/k/{tokenID}` path. The bearer token in the file is `{tokenID}` itself. When the attacker copies the file to their machine and runs `kubectl --kubeconfig=stolen.yaml get pods`, kubectl tries to authenticate against our server. Our wildcard `/k/{tokenID}/*` route catches the request — including the API path kubectl appends — and fires the trigger.
**Why a wildcard route:** kubectl doesn't just hit `/k/{tokenID}` — it hits `/k/{tokenID}/api/v1/namespaces/default/pods` or similar, depending on the command. The handler in `backend/internal/token/handler.go` registers `r.HandleFunc("/k/{tokenID}", ...)` and `r.HandleFunc("/k/{tokenID}/*", ...)` so kubectl's appended path doesn't bypass the trigger.
**Who you catch:** Anyone running `kubectl` against the stolen config. This is gold for catching post-exploitation lateral-movement attempts in cloud-native environments, where compromised CI runners and developer laptops typically have a `~/.kube/config` with cluster admin or near-admin permissions.
**Real-world parallel:** The [TeamTNT k8s-targeted campaigns](https://www.aquasec.com/blog/teamtnt-attacks-against-kubernetes-clusters/) actively scrape kubeconfig files from compromised hosts. A honey kubeconfig is one of the highest-precision detection signals available for cloud-native shops.
### 7. MySQL (real wire-protocol decoy)
**Files:** `backend/internal/token/generators/mysql/protocol.go`, `server.go`, `handler.go`, `generator.go`
**Mechanism:** This is the most ambitious token. The artifact is a connection string like `mysql://canary_abc123xyz789@db.your-host.com:3306/internal_db`. The username prefix `canary_` (defined in `backend/internal/token/generators/mysql/handler.go` as `mysqlUsernamePrefix`) is how the listener recognises one of our tokens — anything else gets silently dropped. When the attacker uses the `mysql` CLI or a programmatic MySQL client to connect, they hit a TCP listener on our server. Our listener speaks the **real** MySQL v10 wire protocol — sends a `HandshakeV10` packet with server version `5.7.40-canary`, reads the client's auth response, extracts the username, strips the prefix to recover the token ID, and replies with a properly-formatted `ERR_PACKET` carrying SQL state `28000` and the standard MySQL error message:
```
Access denied for user 'canary_abc123xyz789'@'attacker-ip' (using password: YES)
```
From the attacker's terminal, this looks exactly like a real MySQL server rejecting their password. They'll probably try a few more times with different passwords, then give up. Each attempt records an event.
**Why a real wire protocol:** Two reasons. First, it catches programmatic clients, not just CLI tools — a leaked connection string is more likely to be tried by an exfil tool's automated credential validator than by a human typing into a terminal, and those tools expect a real protocol. Second, the error message is the giveaway: a TCP listener that just closes the connection or returns garbage would tell the attacker immediately that something is off, and they might pivot to investigating. A real `Access denied` makes them assume the password is just wrong.
**The packet layout** (from `protocol.go`):
```
Packet:
[3 bytes payload length LE] [1 byte sequence id] [payload]
HandshakeV10 payload:
protocol version (0x0a) | server version "5.7.40-canary\0" |
connection id (4 bytes LE) | auth-plugin-data part 1 (8 bytes) | filler 0x00 |
capability flags lower (2 bytes LE = 0xf7ff) | charset (0x21 utf8mb4) |
status flags (2 bytes LE = 0x0002) | capability flags upper (2 bytes LE = 0x81ff) |
auth-plugin-data length (0x15) | reserved (10 bytes) |
auth-plugin-data part 2 (12 bytes) | filler 0x00 |
plugin name "mysql_native_password\0"
ERR packet payload:
0xff | error code 1045 (LE) | '#' | sql state "28000" |
"Access denied for user '<user>'@'<host>' (using password: YES)"
```
**Real-world parallel:** Connection strings in stolen `.env` files are the bread and butter of post-exploitation credential validation. An automated tool that finds `DATABASE_URL=mysql://...` and tries to connect to it is a routine part of modern attack pipelines. This token type catches that behaviour with high fidelity because it pretends to be exactly the system the attacker is testing for.
---
## Operational Considerations
### Burn-In (Making the Lure Believable)
A canary token only works if the attacker actually touches it. The technical mechanism is the easy part; the social engineering is the hard part. Every deployment decision should ask: "would a real attacker, mid-pivot, find this and decide it's worth investigating?"
A few rules from the Thinkst playbook and from real practitioners:
- **File names matter.** `password.docx` is a cliche and savvy attackers may skip it. `customer-q3-2026.docx` or `Vendor-Onboarding-2024.docx` blends in. Match your real document naming conventions.
- **Surrounding context matters.** A `.env` file alone in a directory looks staged. A `.env` next to `docker-compose.yml`, `app.py`, `requirements.txt`, and a `node_modules/` directory looks like a real repo somebody forgot to clean up.
- **Modification timestamps matter.** Use `touch -d "2 weeks ago"` on your honey files so they don't all share a fresh creation time.
- **Don't deploy too many.** If half the files on a share are canaries, the noise tells the attacker something. Plant a handful per host, in places the attacker has to look for.
### Dedup and Why It Matters
Every honeytoken deployment hits this problem within the first week: somebody scans the URL, or a backup process opens a file, or the attacker's tooling retries on failure, and your alert channel floods. The dedup gate in this project (`backend/internal/event/service.go` + Redis `dedup:trigger:{tokenID}:{sourceIP}`) silences the second-through-Nth event from the same source IP for 15 minutes after the first.
The events are still recorded in Postgres — you don't lose forensic data — but only the first one fires a notification. The manage page shows a "3 duplicate triggers silenced" counter so the operator knows there's noise to investigate.
15 minutes is a deliberate tradeoff. Long enough to cover most retry-loops and scan bursts. Short enough that a returning attacker after lunch produces a fresh alert.
### Enumeration Resistance
A naive trigger handler does this:
```
GET /c/abc → 404 Not Found
GET /c/abc123xyz789 → 200 OK + pixel + record event
```
An attacker who finds a leaked log entry mentioning a `/c/...` URL can now enumerate which token IDs are real by trying URLs and watching the status codes. That's bad — it lets them suppress the alert by not actually opening the file, or it tells them the system is a canary and they should pivot.
The trigger handler in this project does this instead:
```
GET /c/abc → 200 OK + pixel (no event recorded)
GET /c/abc123xyz789 → 200 OK + pixel + record event
```
Same response either way. The attacker cannot distinguish a real token from a fake one without seeing the alert side. For `slowredirect` specifically the response is more involved because the destination URL is required, but the same principle applies: the response for an invalid token serves a generic decoy destination of `/`.
### Operator Token Compromise
The admin API is gated by a single bearer token in `OPERATOR_TOKEN`. If that token leaks, an attacker can list every token you've deployed and which alert channels they fire on — i.e. they get a map of your entire deception infrastructure. The middleware compares with `subtle.ConstantTimeCompare` to prevent timing-based recovery of the token. The deployment guidance is: rotate `OPERATOR_TOKEN` periodically, log all `/api/admin/*` access at the reverse-proxy layer, and treat any unauthorized admin request the same way you'd treat unauthorized access to a SIEM.
---
## Real-World Incidents Where Honeytokens Mattered (Or Would Have)
A short selection from public post-mortems:
**The DNC breach (2016).** APT28 (Fancy Bear) had access to the DNC's network for months. The breach was eventually detected via CrowdStrike's response, not through DNC's own monitoring. Multiple public analyses noted that honey credentials in service accounts and file-share locations would have provided detection signal months earlier; service-account credentials in particular are textbook honeytoken candidates because no human ever logs in with them.
**SolarWinds / SUNBURST (2020).** The supply-chain compromise of the Orion update channel gave APT29 persistent access to thousands of customer networks for an average of ~9 months before public disclosure. The attack relied heavily on legitimate-looking outbound C2 traffic and credentialed lateral movement. Honey credentials in Active Directory service accounts and honey URLs in internal wikis would have given many of the victim networks a high-confidence detection. FireEye's own breach disclosure noted that one of the indicators that ultimately surfaced the campaign was unexpected access to a controlled account, which is functionally equivalent to a honeytoken trip.
**Twilio breach (2022).** A phishing campaign compromised employees' credentials and gave the attackers access to internal systems for several days. Honey credentials in internal tooling — and honey URLs in internal docs that the attackers would have browsed during reconnaissance — were among the post-mortem recommendations Twilio published.
**Uber breach (2022).** A social-engineering compromise of an Uber contractor's account let the attacker pivot to internal admin tools and dump credentials from a PowerShell script that contained a privileged Thycotic password. The attacker reportedly took several hours to escalate, during which honey credentials in the same PowerShell scripts they were combing through would have fired immediately.
The pattern: in every case, the attackers spent meaningful time inside the network looking for credentials, files, and infrastructure to pivot through. Every minute of that is a minute a planted honeytoken could fire.
---
## Mapping to MITRE ATT&CK (Defensive View)
Honeytokens don't map cleanly to ATT&CK because ATT&CK catalogues attacker behaviour, not defender behaviour. They map to MITRE Engage, which we covered above. But here is the inverse mapping — for each token type, the *attacker techniques* it detects:
| Token | Attacker Techniques Detected |
|-------|------------------------------|
| `webbug` | T1213 (Data from Information Repositories) — anyone reading planted internal docs; T1114 (Email Collection) if planted in email |
| `slowredirect` | T1566.002 (Spearphishing Link triage) — anyone clicking your honey-link; T1190 (Exploit Public-Facing Application) — scanners hitting fake admin URLs |
| `pdf`, `docx` | T1213 (Data from Information Repositories); T1005 (Data from Local System) — anyone exfiltrating planted files |
| `envfile` | T1552.001 (Credentials in Files); T1078 (Valid Accounts) — anyone using the fake creds |
| `kubeconfig` | T1552.001 (Credentials in Files); T1078.004 (Cloud Accounts); T1613 (Container and Resource Discovery) |
| `mysql` | T1552.001; T1078; T1110.001 (Brute Force: Password Guessing) — the password-spray after the first failure |
A defender mapping their detection coverage against ATT&CK can use this table to claim coverage on credential-access and discovery techniques that other detection sources (EDR, NIDS, SIEM) often miss because the attacker behaviour is too quiet to register on signature- or anomaly-based monitoring.
---
## Further Reading
- Thinkst Canary product documentation: [https://docs.canary.tools/](https://docs.canary.tools/) — the commercial reference for how this technology gets used in mature environments.
- Canarytokens (open-source predecessor): [https://canarytokens.org/generate](https://canarytokens.org/generate) — Thinkst's free hosted version, with a wider token catalogue than this project.
- MITRE Engage: [https://engage.mitre.org/matrix/](https://engage.mitre.org/matrix/) — the defensive deception matrix.
- Honeytokens chapter in *The Cuckoo's Egg* by Clifford Stoll (1989) — the original.
- "Practical Deception Engineering" by Haroon Meer (talks at BSidesLV and DEFCON over the past decade) — the modern doctrine, from the person who built Thinkst.
- Verizon 2024 DBIR: [https://www.verizon.com/business/resources/reports/dbir/](https://www.verizon.com/business/resources/reports/dbir/) — annual data on dwell time, breach causes, detection sources.
- IBM Cost of a Data Breach 2024: [https://www.ibm.com/reports/data-breach](https://www.ibm.com/reports/data-breach) — annual MTTI/MTTD numbers.
The architecture and code-walkthrough modules ([02 - Architecture](02-ARCHITECTURE.md), [03 - Implementation](03-IMPLEMENTATION.md)) take the theory above and show you exactly how each piece is implemented.

551
PROJECTS/beginner/canary-token-generator/learn/02-ARCHITECTURE.md Normal file
View File

@ -0,0 +1,551 @@
<!-- © AngelaMos | 2026 | 02-ARCHITECTURE.md -->
# Architecture
This document covers the system design. It walks through the request lifecycle for both token creation and token triggering, the schema, the dedup gate, the notification pipeline, and the deployment topology. Read time is roughly 20-25 minutes.
The point is not to recap the code — that's [03 - Implementation](03-IMPLEMENTATION.md) — but to explain *why* each piece exists and why it sits where it does. Architecture decisions only make sense once you understand the constraints they're optimising against.
---
## High-Level Topology
```
┌────────────────────────────┐
│ Cloudflare Tunnel │
│ (optional, public ingress)│
└─────────────┬──────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ nginx (reverse proxy) │
│ - Serves / → frontend static bundle (prod) or Vite (dev) │
│ - Proxies /api/* → canary backend │
│ - Proxies /c/*, /k/*, /healthz → canary backend │
└─────────────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ canary (Go binary, internal port 8080) │
│ │
│ chi router with middleware chain: │
│ CleanPath → StripSlashes → RequestID → Logger → Recovery → │
│ SecurityHeaders → (route-specific: CORS, rate limit, Turnstile, │
│ operator bearer) │
│ │
│ Domains: │
│ internal/token/ CRUD + trigger handler + generator registry │
│ internal/event/ ingestion service with dedup gate │
│ internal/notify/ async worker pool │
│ internal/admin/ bearer-gated operator endpoints │
│ internal/health/ /healthz │
│ │
│ Background goroutines: │
│ MySQL TCP listener (bound to MYSQL_FAKE_ADDR, accepts conns) │
│ Retention loop (prunes events to a per-token limit) │
└──────┬───────────────────────────────────────┬──────────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌───────────────────────────┐
│ PostgreSQL 18 │ │ Redis 7 │
│ tokens │ │ dedup:trigger:{id}:{ip} │
│ events │ │ dedup:active:{id} │
│ goose_db_version│ │ ratelimit:{scope}:{key} │
└──────────────────┘ └───────────────────────────┘
│
▼
┌───────────────────────┐
│ External destinations │
│ - Telegram Bot API │
│ - Webhook receivers │
│ - Jaeger (OTLP/gRPC) │
└───────────────────────┘
```
The shape is conventional for a Go web service: nginx in front, single Go binary in the middle, Postgres and Redis behind. The interesting parts are the dedup gate, the async notification pipeline, and the MySQL listener that lives next to the HTTP server in the same process.
---
## Request Lifecycle 1: Creating a Token
This is the request that runs when the operator clicks "Generate" in the web UI.
```
Operator (browser)
│
│ POST /api/tokens
│ Content-Type: application/json
│ Body: { "type": "envfile", "memo": "team-laptops",
│ "alert_channel": "telegram",
│ "telegram_bot": "...", "telegram_chat": "...",
│ "metadata": { "include_keys": ["aws", "github"] },
│ "cf_turnstile_response": "..." }
│
▼
nginx → canary:8080
│
▼
chi middleware chain (in order, see backend/cmd/canary/main.go:298-308):
CleanPath, StripSlashes normalise URL
RequestID attach uuid to ctx
Logger slog request_start / request_end
Recovery catch panics → 500 + structured log
SecurityHeaders CSP, HSTS, X-Frame-Options
CORS (per-route, /api/* only) allowed origins from config
RateLimiter (per-fingerprint) token bucket via Redis (KeyByFingerprint)
RateLimiter (per-minute, create scope) separate bucket for token creation
RateLimiter (per-hour, create scope) slow-bleed budget for creation
TurnstileVerify POST cf-turnstile-response to Cloudflare siteverify
│
▼
token.Handler.CreateToken
│
│ - validator/v10 on the CreateRequest DTO
│ - extract fingerprint = sha256(RealIP + UserAgent)[:16]
│
▼
token.Service.Create
│
├─► generator.Generator(type) → e.g. envfile.Generator
│ │
│ └─► Generate(ctx, token, baseURL) → Artifact
│ (envfile picks recipes, shuffles, injects canary line)
│
├─► token.Repository.Insert → PostgreSQL
│ INSERT INTO tokens (id, manage_id, type, memo, filename,
│ alert_channel, telegram_bot, telegram_chat, webhook_url,
│ created_at, created_ip, created_fp, enabled, metadata)
│ VALUES (...)
│
└─► Return CreateResponse { token, artifact }
│
▼
HTTP 201 Created
Body: {
"success": true,
"data": {
"token": {
"id": "abc123xyz789",
"manage_id": "9b1d...",
"type": "envfile",
"trigger_url": "https://canary.example.com/c/abc123xyz789",
"manage_url": "https://canary.example.com/m/9b1d...",
...
},
"artifact": {
"kind": "file",
"filename": ".env",
"content_b64": "..."
}
}
}
```
Notes on the chain that aren't obvious from the diagram:
- **Two separate rate limiters on creation.** The fingerprint limiter is wide (default 100/min) and applies to every `/api/*` request — it's there to keep a single tab from hammering the API. The create-min / create-hour limiters are tighter and apply *only* to `POST /api/tokens`. The hour budget exists because a script could happily spread its requests across many minutes and burn through unlimited token IDs if there were no longer-window cap.
- **Turnstile is verified last in the create chain.** That's so the cheap checks (rate limit) reject obvious abuse before we pay for the Cloudflare round-trip on every request.
- **The fingerprint is for rate limiting, not authentication.** It's `sha256(RealIP + UserAgent)[:16]` — trivially forgeable by an attacker who knows it exists. The point is to make casual abuse harder, not to bind identity. Authentication for admin endpoints is a separate concern (constant-time bearer compare).
---
## Request Lifecycle 2: Triggering a Token
This is the path that runs when an attacker opens the artifact. It's deliberately fast and free of synchronous side effects.
```
Attacker (or attacker's tool)
│
│ GET /c/abc123xyz789
│ User-Agent: curl/8.0 (or Acrobat, or Word, or kubectl, or ...)
│
▼
nginx → canary:8080
│
▼
Middleware chain (NO CORS, NO Turnstile, NO operator bearer — trigger routes are root-mounted)
│
▼
token.Handler.HandleTrigger (backend/internal/token/handler.go)
│
│ - Extract tokenID from URL param
│ - Look up token via Repository.GetByID (or treat as "fake" if not found)
│ - Resolve Generator by Type
│
▼
generator.Trigger(ctx, token, request) → Event, TriggerResponse, error
│
│ Generator-specific behaviour:
│ webbug → returns embedded JPEG + cache-control:no-store
│ pixel-using → returns embedded 1x1 GIF + cache-control:no-store
│ slowredirect → renders HTML interstitial; later POST /c/{id}/fingerprint
│ attaches fingerprint to the SAME event row's `extra` column
│ kubeconfig → same as webbug response
│ mysql → unreachable here (TCP listener handles MySQL triggers)
│
│ In every case: returns an Event{ TokenID, SourceIP, UserAgent, Referer, ... }
│ with no ID yet (DB assigns BIGSERIAL on insert)
│
▼
Handler writes the HTTP response IMMEDIATELY
│ (the attacker's browser gets its bytes before the next steps run)
│
▼
Handler dispatches event.Service.Record asynchronously (or inline, see note)
│
▼
event.Service.Record (backend/internal/event/service.go)
│
├─► geoip.Lookuper.Lookup(SourceIP) → enriches evt.Geo* fields
│
├─► Repository.Insert → PostgreSQL (assigns evt.ID)
│
├─► tokens.IncrementTriggerCount(tokenID) → atomic UPDATE
│
├─► dedupGate(tokenID, sourceIP):
│ key = "dedup:trigger:{tokenID}:{sourceIP}"
│ ok, _ = redis.SetNX(key, 1, 15min)
│ if ok: return true (first hit, will notify)
│ if !ok: redis.Incr(key)
│ redis.SAdd("dedup:active:{tokenID}", sourceIP)
│ redis.Expire("dedup:active:{tokenID}", 15min)
│ return false (suppress notification)
│
├─► if dedup says "first hit":
│ notify.Service.Notify(notifyInfo, evt) → enqueue dispatchJob
│ (returns immediately; worker pool handles delivery)
│
└─► if dedup says "duplicate":
repo.UpdateNotifyStatus(eventID, NotifyDeduped, nil)
(the event row stays in the DB for forensics; alert is suppressed)
```
An important subtlety in the codebase: the trigger handler records the event *synchronously* (DB insert + dedup + notify enqueue) before writing the HTTP response. That keeps the code paths simple and means we don't lose events if the process exits between response and DB insert. The bet is that the insert is fast enough (sub-10ms on a healthy Postgres) that the attacker's client doesn't notice the latency.
The notification *delivery* is async — `event.Service.Record` only *enqueues* a `dispatchJob` and returns immediately. The actual HTTP call to Telegram or the webhook receiver runs on a worker goroutine, so a slow alert destination cannot stall the trigger response. If you wanted to push event recording onto a background queue as well (to absorb DB latency spikes during a flood), the `notify` package's worker-pool pattern would be the model — apply it to event ingestion.
---
## Request Lifecycle 3: MySQL Trigger (TCP, not HTTP)
The MySQL token is special. The artifact is a connection string, not a URL. When the attacker connects, they speak the MySQL v10 wire protocol, not HTTP. So we run a separate TCP listener inside the same process.
```
Attacker
│
│ mysql -h db.canary.example.com -P 3306 \
│ -u canary_abc123xyz789 -p internal_db
│
▼
TCP listener (backend/internal/token/generators/mysql/server.go)
│
│ net.Listen("tcp", cfg.MySQL.FakeAddr)
│ accept loop spawns a goroutine per connection
│
▼
ConnectionHandler.HandleConnection
│
│ 1. Generate random connection ID + auth-plugin-data
│ (crypto/rand for both — must look real to MySQL clients)
│
│ 2. Write HandshakeV10 packet
│ (server version "5.7.40-canary", caps 0xf7ff/0x81ff, utf8mb4,
│ auth plugin "mysql_native_password")
│
│ 3. Read ClientAuth response
│ (32-bit caps, 32-bit max-packet, 1-byte charset, 23-byte filler,
│ null-terminated username)
│
│ 4. Extract token ID from username (strip "canary_" prefix)
│
│ 5. Look up token in DB
│ - if found and enabled: build Event, dispatch via event.Service.Record
│ - if not found: still respond with Access denied (enumeration resistance)
│
│ 6. Write ERR packet with sql state 28000
│ "Access denied for user 'canary_...'@'attacker-ip' (using password: YES)"
│
│ 7. Close TCP connection
│
▼
Attacker's terminal:
ERROR 1045 (28000): Access denied for user 'canary_abc123xyz789'@'1.2.3.4' (using password: YES)
```
A few design notes:
- **Same `event.Service.Record` path.** The MySQL listener doesn't bypass dedup, GeoIP, or notification. It builds the same `event.Event` struct with `extra = {"client_capabilities": ..., "client_charset": ...}` for forensic data, then hands off to the event service. That keeps the trigger logic uniform across all 7 token types.
- **No password verification, ever.** We never try to validate the password (we don't have one to validate against). We always return `Access denied`. If we *did* accept the connection, the attacker would expect to be able to run SQL, and we'd have to fake an entire MySQL server — out of scope.
- **Public vs internal address.** `MYSQL_FAKE_ADDR` is what the listener binds to (e.g. `0.0.0.0:3306` in the container). `MYSQL_PUBLIC_HOST` and `MYSQL_PUBLIC_PORT` are what go into the connection string we hand to the operator (e.g. `db.canary.example.com:3306`). Separating these is what lets the same instance be reachable through a reverse proxy or Cloudflare Tunnel that does port translation.
---
## The Dedup Gate
```
Attacker repeatedly opens /c/abc123xyz789 from IP 1.2.3.4
│
│ Hit 1 at T+0s:
│ SetNX("dedup:trigger:abc123xyz789:1.2.3.4", 1, 15min) → set=true
│ → record event, notify operator
│ event.notify_status = sent
│
│ Hit 2 at T+5s:
│ SetNX(...) → set=false (key already exists)
│ Incr("dedup:trigger:abc123xyz789:1.2.3.4") → 2
│ SAdd("dedup:active:abc123xyz789", "1.2.3.4")
│ Expire("dedup:active:abc123xyz789", 15min)
│ → record event in DB, mark notify_status = deduped
│ → no notification fired
│
│ Hit 3..N at T+5s..T+900s:
│ same as hit 2
│
│ Hit at T+901s:
│ SetNX(...) → set=true again (TTL expired)
│ → notify operator (fresh alert)
```
The dedup gate has three deliberate properties:
**It fails open.** If Redis is unreachable, `dedupGate` returns `true` and the event notifies. The reasoning: a Redis outage shouldn't suppress real attacker activity. The cost is alert noise during a Redis outage, which is preferable to silent failure.
**Per-`{token, source_ip}` not per-token.** Two different attackers hitting the same token from different IPs both fire notifications. That matters for shared honeyfile placements (e.g. a wiki page seen by multiple attackers in the same campaign) — you want each attacker to register as a fresh signal, not be suppressed by an earlier one.
**Events are still recorded.** The `events` table gets the row regardless of dedup outcome. The dedup gate only affects whether a notification is *fired*. The manage page displays a "N duplicate triggers silenced" badge so the operator knows there's quiet activity to investigate.
The `dedup:active:{tokenID}` Redis Set exists purely so the manage page can count distinct silenced IPs. `event.Service.CountActiveDedup` reads it via `SCARD`.
---
## The Notification Pipeline
```
event.Service.Record (decides to notify)
│
▼
notify.Service.Notify(info, evt)
│
│ - increment jobWg
│ - non-blocking send on s.queue (buffered chan dispatchJob, cap 256)
│ - if queue full: drop, mark NotifyFailed, decrement jobWg
│
▼
queue chan dispatchJob ◄── 8 worker goroutines consume
│
▼
worker.dispatch
│
│ ctx, cancel := context.WithTimeout(bg, 30s)
│
├─► sender = s.senders[info.AlertChannel]
│
├─► if not registered: log warn, mark NotifyFailed, return
│
├─► sender.Send(ctx, info, evt):
│ telegram.Send → POST api.telegram.org/bot{token}/sendMessage
│ webhook.Send → POST {webhook_url} (HMAC-signed if WEBHOOK_HMAC_SECRET set,
│ exponential backoff via cenkalti/backoff/v5)
│
├─► on error: log warn, mark NotifyFailed
│
└─► on success: mark NotifySent with notified_at = now
```
The pipeline has three knobs in config: worker count (default 8), queue size (default 256), and per-job timeout (default 30s). The defaults are sized for a single host with low-to-moderate trigger volume. If you operate a wide canary deployment that fires hundreds of events per minute, raise the worker count and queue size.
The bounded queue is important. An unbounded queue means a slow notification destination (e.g. a webhook receiver that hangs) eventually consumes all process memory. With the bounded queue, a hung sender backs up to the queue cap, then `Notify` starts dropping jobs and marking them `NotifyFailed`. The operator sees those in the manage UI and can decide what to do — typically: switch alert channels, kill the broken receiver, or scale up the worker pool.
The 30-second per-job timeout is what eventually unsticks a hung sender. Even if a webhook receiver TCP-accepts and then never reads, the worker abandons after 30s and moves on.
---
## Schema
### `tokens`
```sql
CREATE TABLE tokens (
id VARCHAR(12) PRIMARY KEY,
manage_id UUID UNIQUE NOT NULL,
type VARCHAR(32) NOT NULL,
memo TEXT NOT NULL DEFAULT '',
filename TEXT,
alert_channel VARCHAR(16) NOT NULL,
telegram_bot TEXT,
telegram_chat TEXT,
webhook_url TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
created_ip INET NOT NULL,
created_fp CHAR(16) NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
trigger_count BIGINT NOT NULL DEFAULT 0,
last_triggered TIMESTAMPTZ,
metadata JSONB NOT NULL DEFAULT '{}'::jsonb
);
```
A few schema decisions worth flagging:
- **`id` is a 12-char base62 string, not a UUID.** This is the value that goes into the trigger URL (`/c/{id}`) and the artifact body (e.g. the PDF placeholder substitution). UUIDs are 36 chars and would push the PDF placeholder to >76 bytes, breaking the byte-exact substitution. 12 chars at base62 gives ~71 bits of entropy — comfortably non-guessable.
- **`manage_id` is a UUID v4.** This is the *operator-facing* identifier, used in `/api/m/{manageId}` for viewing token + events. Splitting it from `id` means an attacker who triggers a token cannot also enumerate to the manage page from the trigger URL.
- **`created_ip` is `INET`.** Native Postgres type for IPv4/IPv6. Makes geographic queries via `INET <<= cidr` easy if you want to add them later.
- **`metadata` is `JSONB`.** Type-specific config lives here (e.g. `{"destination_url": "..."}` for slowredirect, `{"include_keys": ["aws","db"]}` for envfile). Keeping it out of typed columns means new token types don't need migrations.
### `events`
```sql
CREATE TABLE events (
id BIGSERIAL PRIMARY KEY,
token_id VARCHAR(12) NOT NULL REFERENCES tokens(id) ON DELETE CASCADE,
triggered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
source_ip INET NOT NULL,
user_agent TEXT,
referer TEXT,
geo_country CHAR(2),
geo_region VARCHAR(64),
geo_city VARCHAR(64),
geo_asn INT,
geo_asn_org VARCHAR(128),
extra JSONB NOT NULL DEFAULT '{}'::jsonb,
notify_status VARCHAR(16) NOT NULL DEFAULT 'pending',
notified_at TIMESTAMPTZ
);
```
- **`id` is BIGSERIAL.** Monotonic, dense, and ideal for cursor pagination. The manage page paginates with `WHERE id < cursor ORDER BY id DESC LIMIT N`.
- **`extra` is JSONB.** Two uses: (a) `slowredirect` token fingerprints get stored here as the raw JS-produced JSON blob; (b) MySQL events carry `{"client_capabilities": 0x..., "client_charset": 0x21}` so investigators can profile the attacking client.
- **`notify_status` ∈ {pending, sent, failed, deduped}.** The state machine: row inserted as `pending` → dedup decides → either `deduped` (skip notify) or queue for worker → worker writes `sent` or `failed` on completion. The status is what the manage UI uses to badge each row in the table.
### Indexes (migration `0003_indexes.sql`)
- `tokens.manage_id` is UNIQUE NOT NULL — implicit unique index.
- `events.token_id` — supports the manage page's `WHERE token_id = $1` filter.
- `events.triggered_at DESC` — supports admin "recent activity" queries.
- `events.notify_status` — supports filtering pending/failed for retry tooling.
---
## Frontend ↔ Backend Contract
The frontend is a single-page React 19 app. It has two routes:
```
/ Landing page (token creation form, artifact reveal after success)
/m/:manageId Manage page (token detail + paginated event feed)
```
State management is **TanStack Query only** — there's no Redux, no Zustand. Every server interaction is a query or mutation:
```typescript
useTokenTypes() // GET /api/tokens/types
useCreateToken() // POST /api/tokens
useManageToken(id) // GET /api/m/:id (cursor-paginated)
useDeleteToken(id) // DELETE /api/m/:id
```
Each hook validates the response body against a Zod schema before handing it to the component. That means the frontend types are the *actual* shape the backend returns, not what we hope it returns — Zod throws if the contract drifts.
The API client (`frontend/src/api/client.ts`) is a thin Axios wrapper with three job-specific behaviours:
1. **Turnstile injection.** If the user has completed the Turnstile widget, `POST /api/tokens` automatically gets the response token in the `CF-Turnstile-Response` header and the JSON body's `cf_turnstile_response` field (the backend accepts either).
2. **Error normalisation.** HTTP errors are mapped to typed error codes (`VALIDATION_ERROR`, `RATE_LIMITED`, `TURNSTILE_FAILED`, etc.) before being thrown, so components can branch on `err.code` instead of pattern-matching status codes.
3. **Timeout.** 15 seconds per request. Generation can take a couple of seconds for the PDF/DOCX types because of the embedded-template manipulation, but 15s is generous.
There is no WebSocket and no Server-Sent Events. The manage page polls (via TanStack Query's `refetchInterval`) every few seconds. For a low-volume canary deployment, this is plenty — events fire seconds-to-minutes apart, not milliseconds.
---
## Deployment Topology
### Production (`compose.yml`)
```
nginx ─┬─→ canary (Go binary)
│ ├─→ Postgres
│ └─→ Redis
│
│ static assets from frontend build (mounted in nginx image)
```
Four services, no Vite, no Air, no Jaeger. The Go binary is built with `CGO_ENABLED=0 -trimpath -ldflags="-s -w"` so the resulting image is a `gcr.io/distroless/static-debian12` with the single binary. The image is around 20 MB.
The `canary` container's `/healthz` is wired into the Docker healthcheck via the tiny `cmd/healthcheck` binary, which is a 1.2 MB statically-linked Go HTTP client. We ship that instead of putting `curl` in the image because distroless doesn't have curl.
### Production with Cloudflare Tunnel (`cloudflared.compose.yml`)
```
cloudflared (tunnel client) ─→ Cloudflare edge ─→ public internet
│
└─→ nginx (internal only, no exposed port)
```
The tunnel overlay swaps the nginx port-mapping for a `cloudflared` sidecar that maintains an outbound TLS connection to Cloudflare's edge. The instance becomes publicly reachable through `your-tunnel-name.your-domain` without opening any inbound port. That's especially useful for self-hosted deployments on residential connections or behind NAT.
### Development (`dev.compose.yml`)
```
nginx ─┬─→ frontend (Vite HMR)
│
└─→ canary (Air-reloaded Go)
├─→ Postgres
├─→ Redis
└─→ Jaeger (OTLP/gRPC + UI at :16686)
```
The dev compose:
- Mounts `frontend/` and `backend/` into the containers so file changes trigger reloads.
- Runs the backend with `air` (https://github.com/cosmtrek/air) for hot reload on `.go` file changes.
- Boots Jaeger so OpenTelemetry traces from the backend can be inspected at `http://localhost:16686`.
- Uses a separate compose project name (`canary-token-generator-dev`) so it doesn't conflict with a running production stack on the same host.
### Why Both Compose Files
A common pitfall is using a single compose file with optional services. The result is fragile: developers have to remember which `--profile` to pass, and CI scripts diverge from local commands. Two files keeps the boundary crisp — `just dev-up` is unambiguously the dev stack, `just up` is unambiguously prod. The `compose.yml` + `cloudflared.compose.yml` override is a separate concern (production with vs without public exposure).
---
## Configuration
The service is configured via `config.yaml` (defaults) overridden by environment variables, loaded with [koanf](https://github.com/knadh/koanf) (see `backend/internal/config/config.go`). The pattern is:
1. Load `config.yaml` for defaults.
2. Overlay env vars (matching the key path with `__` separators, e.g. `SERVER__PORT=8080` overrides `server.port`).
3. Validate the resulting struct against `validator/v10` tags.
The interesting env vars:
| Variable | Purpose |
|----------|---------|
| `PUBLIC_BASE_URL` | The hostname token URLs are minted against. **This is the thing that goes into your artifacts.** |
| `OPERATOR_TOKEN` | Bearer token for `/api/admin/*`. Unset → admin routes don't register. |
| `TURNSTILE_SECRET` | Cloudflare Turnstile server-side secret. Unset → Turnstile middleware is a no-op. |
| `WEBHOOK_HMAC_SECRET` | Used to sign outbound webhooks with HMAC-SHA256 in `X-Canary-Signature`. |
| `GEOLITE_PATH` | Path to the GeoLite2-City MMDB. Unset → GeoIP lookups are nop. |
| `MYSQL_FAKE_ENABLED` | Boolean. If false, the MySQL token type is hidden from `/api/tokens/types` and the listener doesn't spawn. |
| `MYSQL_FAKE_ADDR` | TCP listener bind address, e.g. `0.0.0.0:3306`. |
| `MYSQL_PUBLIC_HOST` / `MYSQL_PUBLIC_PORT` | What gets baked into the artifact connection string. |
| `EVENT_DEDUP_TTL` | Per-`{token, ip}` silence window. Default 15m. |
| `EVENT_RETENTION_PER_TOKEN` | The retention loop prunes events past this count per token. 0 = no pruning. |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Where to send OpenTelemetry traces. Unset → tracing disabled. |
| `LOG_LEVEL` / `LOG_FORMAT` | `debug|info|warn|error` and `json|text`. |
| `TRUSTED_PROXY_CIDRS` | Required for correct `RealIP` extraction behind nginx/Cloudflare. |
The `TRUSTED_PROXY_CIDRS` value matters more than people expect. The `RealIP` middleware reads `X-Forwarded-For` only if the immediate peer is in the trusted CIDR list. If you forget to set it for your reverse proxy, every event will show the proxy's IP as the source IP. If you set it too widely, an attacker can spoof their IP by forging `X-Forwarded-For`.
---
## What Isn't In This Architecture (And Why)
A few omissions are deliberate.
**No multi-tenancy.** A single deployment is one operator's canary deployment. If you want per-team isolation, run multiple deployments (cheap, since the binary is small and Postgres/Redis can be shared). Adding multi-tenancy would require auth on token creation, which adds a lot of complexity for a beginner project.
**No alert routing rules.** Every token has a single alert channel chosen at creation. No "if envfile then PagerDuty, else Telegram" routing. That kind of routing is something a real ops team builds on top of the webhook receiver, not in the canary itself.
**No replay or backfill.** If the worker pool drops jobs to `NotifyFailed`, there's no built-in retry loop. The data is in the DB; you can write a one-off script to re-process failed events through a fresh notification call. A cron-style retry job is in `04-CHALLENGES.md` as an extension exercise.
**No clustering.** One process, one host. The dedup gate uses Redis which is shareable, so multi-instance is *possible*, but the MySQL TCP listener and the retention loop aren't designed for leader election. If you genuinely need to scale this past one host, you've outgrown the project's intended audience.
The next document, [03 - Implementation](03-IMPLEMENTATION.md), takes the architecture above and points you at the exact code that implements each piece.

612
PROJECTS/beginner/canary-token-generator/learn/03-IMPLEMENTATION.md Normal file
View File

@ -0,0 +1,612 @@
<!-- © AngelaMos | 2026 | 03-IMPLEMENTATION.md -->
# Implementation Walkthrough
This document walks through the actual code that implements the architecture in [02 - Architecture](02-ARCHITECTURE.md). For each major piece I'll cite the file paths, the relevant function/type names, and the design decisions baked into the implementation. The point is not to read every line — it's to know exactly where to look when you want to extend or debug a specific behaviour. Read time is roughly 25-30 minutes.
---
## Wiring (`backend/cmd/canary/main.go`)
This is the file you read first to understand how the pieces connect. `run()` does the following, in order:
1. **Load config.** `config.Load(configPath)` reads `config.yaml`, overlays env vars via koanf, validates with `validator/v10`, returns a `*config.Config`.
2. **Set up logger.** `setupLogger(cfg.Log)` builds a `*slog.Logger` configured for JSON or text output at the configured level, then `slog.SetDefault(logger)` makes it the package-default.
3. **Configure trusted proxies.** `middleware.SetTrustedProxyCIDRs(cfg.Server.TrustedProxyCIDRs)` parses and stores the CIDRs that `RealIP` is allowed to trust.
4. **Initialise telemetry.** `initTelemetry(ctx, cfg, logger)` builds the OpenTelemetry SDK + OTLP gRPC exporter if `OTEL_EXPORTER_OTLP_ENDPOINT` is set; otherwise returns a nop tracer.
5. **Open Postgres.** `core.NewDatabase(ctx, cfg.Database)` creates the connection pool (pgx + sqlx). `core.RunMigrations(db.SQLDB())` runs the goose migrations in `backend/internal/core/migrations/`.
6. **Open Redis.** `core.NewRedis(ctx, cfg.Redis)` returns a `*redis.Client` with health-check ping at startup.
7. **Open GeoIP.** `openGeoIP(cfg, logger)` returns a `geoip.Lookuper` and a closer. If `GEOLITE_PATH` is unset or the MMDB fails to open, it returns `geoip.NopService()` so the event service keeps working with empty geo fields.
8. **Build event + notify stacks.** `buildEventStack` constructs the `notify.Service` worker pool, registers Telegram and webhook senders, and builds the `event.Service` that ties everything together.
9. **Build HTTP deps.** `buildHTTPDeps` constructs the Turnstile verifier, the health handler, the token service (with the generator registry), and the token HTTP handler.
10. **Mount the router.** `mountRouter` builds the `*server.Server` (chi router shell) and registers every route under the right middleware chain.
11. **Spawn the MySQL listener.** `spawnMySQLListener` opens a TCP listener if `cfg.MySQL.FakeEnabled` is true and runs the accept loop in a goroutine.
12. **Spawn the retention loop.** `spawnRetentionLoop` starts the periodic `event.Service.RunRetentionLoop` goroutine if `cfg.Event.RetentionPerToken > 0`.
13. **Serve.** `srv.Start()` runs in a goroutine; the main goroutine waits on the signal context (`signal.NotifyContext(SIGINT, SIGTERM)`).
14. **Graceful shutdown.** When a signal arrives, `srv.Shutdown(ctx, drainDelay)` flips the health endpoint to not-ready, waits `drainDelay` for load balancers to stop sending traffic, then shuts down the HTTP server. The MySQL listener and retention loop bound to the same context exit on their own.
If you only read one file in the codebase, this is the one.
---
## Generator Interface (`backend/internal/token/generators/generator.go`)
Every token type implements this interface:
```go
type Generator interface {
Type() token.Type
Generate(ctx context.Context, t *token.Token, baseURL string) (Artifact, error)
Trigger(ctx context.Context, t *token.Token, r *http.Request) (*event.Event, *TriggerResponse, error)
}
type Artifact struct {
Kind ArtifactKind // KindURL | KindFile | KindText | KindConnectionString
URL string
Filename string
Content []byte
ContentType string
ConnectionString string
}
type TriggerResponse struct {
StatusCode int
ContentType string
Body []byte
ExtraHeaders map[string]string
}
```
Generators are stateless beyond a few config values baked at construction (e.g. `mysql.Generator` keeps `publicHost`/`publicPort`/`database`). The registry at `backend/internal/token/generators/registry/registry.go` is just a `map[token.Type]Generator` populated at startup.
The same generator handles both creation (`Generate`) and trigger (`Trigger`). That keeps each type's logic in one file. The exceptions are `mysql` (HTTP `Trigger` returns `ErrHTTPTriggerNotSupported`; real triggering happens via TCP) and `slowredirect` (which adds a separate `POST /c/{id}/fingerprint` handler for capturing the fingerprint JSON after the interstitial).
---
## Generators, One By One
### `webbug` (the simplest one)
**File:** `backend/internal/token/generators/webbug/generator.go`
```go
//go:embed asset/pixel.jpg
var pixelBytes []byte
```
`Generate` returns a `KindURL` artifact pointing at `{baseURL}/c/{t.ID}`. `Trigger` returns the embedded JPEG with `Cache-Control: no-store, no-cache, must-revalidate, max-age=0` and `Pragma: no-cache`. The event captures `SourceIP` (via `middleware.RealIP(r)`), `UserAgent`, and `Referer`.
If `t == nil` (unknown token ID), `Trigger` still returns the pixel response but with no event — that's the enumeration-resistance behaviour described in 01-CONCEPTS.md.
This is the template every other HTTP-triggered generator follows.
### `pixel` (the shared helper)
**File:** `backend/internal/token/generators/pixel/pixel.go`
Not a generator; a shared 43-byte transparent GIF that every "visited" response body uses (PDF, DOCX, envfile, slowredirect-after-fingerprint, kubeconfig). The package exposes `ContentType = "image/gif"` and a `Clone()` helper that returns a fresh `[]byte` copy so handlers don't accidentally mutate the shared slice.
Why separate from webbug's JPEG? Two-pixel hygiene: the visible web bug is JPEG because some content-blocking heuristics treat 1x1 GIFs specifically as advertiser pixels; the "you opened a file" response is GIF because it's already an HTTP image response and nobody is going to render it anyway.
### `pdf` (byte-exact substitution)
**File:** `backend/internal/token/generators/pdf/generator.go`
The key constants:
```go
const (
placeholderRoot = "HONEY_TRACK_URL_PADDED_TO_FIXED_WIDTH"
PlaceholderLength = 76
padChar = "_"
queryPadPrefix = "?p="
)
//go:embed template/template.pdf
var pdfTemplate []byte
```
The template (built by `cmd/buildpdftemplate/main.go` with pdfcpu) has the literal `HONEY_TRACK_URL_PADDED_TO_FIXED_WIDTH___________________________________________________` (76 bytes) baked into a `/AA /O /URI` page-open action.
`Generate` builds `triggerURL`, refuses to proceed if it exceeds 76 bytes, pads it with `?p=____...` to be exactly 76 bytes, and runs `bytes.Replace(pdfTemplate, placeholder, padded, 1)`. Two defensive checks then guard against template corruption:
```go
if len(out) != len(pdfTemplate) {
return ..., fmt.Errorf("pdf: substitution changed byte length")
}
if !bytes.Contains(out, []byte(triggerURL)) {
return ..., fmt.Errorf("pdf: substitution did not embed trigger URL")
}
```
Both should be unreachable if the template is correct, but they catch a class of regressions where someone updates the template without re-running `buildpdftemplate`.
`padTriggerURL` is worth flagging because the padding strategy has a corner case: if the URL is exactly 75 bytes (one short of the placeholder length), `?p=` plus zero padding chars is impossible because `?p=` is 3 bytes itself. The function falls back to padding with underscores directly:
```go
case needed >= len(queryPadPrefix):
return triggerURL + queryPadPrefix +
strings.Repeat(padChar, needed-len(queryPadPrefix))
default:
return triggerURL + strings.Repeat(padChar, needed)
```
The plain-underscore tail is technically a malformed URL but PDF readers won't fetch it as a URL anyway — the page-open action invokes the URL via the `/URI` value verbatim, so the underscore-padded one resolves to the same path on our server with garbage trailing chars that we ignore.
### `docx` (ZIP rewrite)
**File:** `backend/internal/token/generators/docx/generator.go`
The whole thing is `patchTemplate(docxTemplate, triggerURL)`. It:
1. Opens the embedded `template.docx` as a `zip.Reader`.
2. Opens a `bytes.Buffer` and a `zip.Writer` writing into it.
3. Iterates every entry in the input ZIP.
4. If the entry name is `word/footer2.xml`, does a single `bytes.Replace(body, "HONEY_TRACK_URL", triggerURL, 1)` on its contents. Otherwise leaves the body alone.
5. Calls `w.CreateHeader(&zip.FileHeader{Name: f.Name, Method: f.Method})` and writes the body.
6. Closes the writer (which finalises the central directory).
The `Method: f.Method` part matters. ZIP entries can be `Store` (no compression) or `Deflate` (DEFLATE-compressed). DOCX uses Deflate for most XML entries. If you accidentally rewrite an entry under a different method, Word can usually open it but loses the streaming-decompress fast-path. Preserving the original method keeps the output byte-shape close to a normal DOCX.
Why no length check like in PDF? DOCX is forgiving — the central directory records sizes per-entry, so changing one entry's length doesn't break offsets in other entries. The `zip.Writer` recomputes the central directory from scratch. Word doesn't care.
### `envfile` (recipes + canary line)
**File:** `backend/internal/token/generators/envfile/generator.go`
`Generate` workflow:
```
keys = ExtractIncludeKeys(t.Metadata) // default ["aws", "db"] if absent
trigger = baseURL + "/c/" + t.ID
sections = BuildSections(keys, trigger) // per-recipe blocks + canary block
shuffle = ShuffleSections(sections) // crypto/rand Fisher-Yates
body = RenderSections(sections) // adds NODE_ENV=production header
```
The canary block is:
```go
sections = append(sections, []recipes.EnvLine{
{Comment: "Internal monitoring (Datadog-style integration)"},
{Key: "INTERNAL_METRICS_ENDPOINT", Value: triggerURL},
{Key: "INTERNAL_METRICS_TOKEN",
Value: "tok_live_" + recipes.RandomAlnumMixed(32)},
})
```
Note that the comment uses "Datadog-style integration" specifically — the burn-in is the comment that explains *why* this `.env` would contain a `INTERNAL_METRICS_ENDPOINT`, so an attacker reading the file accepts it as a plausible production config detail and tries to ping it.
**Recipes** (`backend/internal/token/generators/envfile/recipes/`):
| File | Produces |
|------|----------|
| `aws.go` | `AWS_ACCESS_KEY_ID=AKIA...` (20 chars), `AWS_SECRET_ACCESS_KEY=` (40 base64-ish chars), `AWS_REGION=us-east-1` etc. |
| `db.go` | `DATABASE_URL=postgresql://...:5432/...`, `REDIS_URL=redis://...`, etc. |
| `github.go` | `GITHUB_TOKEN=ghp_...` (36-char personal access token shape) |
| `stripe.go` | `STRIPE_SECRET_KEY=sk_live_...`, `STRIPE_PUBLISHABLE_KEY=pk_live_...` |
The fake values are deliberately constructed to *look* legitimate (AKIA-prefix for AWS, ghp_-prefix for GitHub, sk_live_-prefix for Stripe) without being valid credentials. An attacker who validates them against the real service gets a 401 from the real provider — which actually helps the deception, because their next move is usually "the creds are stale, let me try the next one in this file" and the next one is our canary.
`shuffleSections` is a crypto/rand-driven Fisher-Yates over the section slice. Using crypto/rand instead of `math/rand` is overkill for this purpose, but it gives an unpredictable section order that doesn't reveal a generator seed if an attacker collected many sample artifacts and pattern-analysed them.
### `kubeconfig` (text/template render)
**File:** `backend/internal/token/generators/kubeconfig/generator.go`, with template at `template.yaml.tmpl`
The template (simplified):
```yaml
apiVersion: v1
kind: Config
clusters:
- name: {{.ClusterName}}
cluster:
server: {{.APIServerURL}}
insecure-skip-tls-verify: true
contexts:
- name: {{.ContextName}}
context:
cluster: {{.ClusterName}}
user: {{.UserName}}
current-context: {{.ContextName}}
users:
- name: {{.UserName}}
user:
token: {{.Token}}
```
`Generate` fills in `ClusterName` (default `prod-cluster`, overridable via metadata), `UserName` (default `svc-backup-reader`), `APIServerURL = baseURL + "/k/" + t.ID`, and `Token = t.ID`. The default user name is the burn-in — `svc-backup-reader` sounds like a real service account, not "honeypot-user".
**The wildcard handler** (`handler.go` + `token/handler.go`):
```go
// in token/handler.go RegisterTriggerRoutes:
r.HandleFunc("/k/{tokenID}", h.HandleTrigger)
r.HandleFunc("/k/{tokenID}/*", h.HandleTrigger)
```
kubectl with a kubeconfig pointing at `https://canary.example.com/k/abc123xyz789` won't hit that path directly — it'll hit `https://canary.example.com/k/abc123xyz789/api/v1/namespaces/default/pods` or similar, depending on the command. The `/*` wildcard catches everything kubectl appends. The kubeconfig generator's `Trigger` returns the same pixel response as webbug (just to satisfy the contract — kubectl ignores the response body once it gets a non-2xx).
Actually, look at the response status more carefully: the handler returns 200 OK with a GIF body. kubectl typically expects JSON; it'll fail to parse and emit a connection error. From the attacker's perspective, the kubeconfig "works" enough to suggest the cluster is real but their auth is being rejected — same trick as the MySQL token.
### `mysql` (real wire protocol)
**Files:** `backend/internal/token/generators/mysql/protocol.go`, `server.go`, `handler.go`, `generator.go`
The four files split cleanly:
- `protocol.go` — pure functions: `BuildHandshakeV10`, `ReadClientAuth`, `BuildAccessDeniedErr`, `wrapPacket`, `readPacket`. No I/O dependencies; everything takes `io.Reader`/`io.Writer` or `[]byte`. This is testable in isolation and the test file (`protocol_test.go`) does exactly that — round-trips packets and checks every field.
- `server.go` — `Server.Run(ctx)` is the accept loop. It dials `net.Listen("tcp", addr)`, then loops `listener.Accept()` and spawns `go handler.HandleConnection(ctx, conn)` per connection. Shutdown is via `listener.Close()` from the parent goroutine.
- `handler.go` — `Handler.HandleConnection(ctx, conn)` is the per-connection logic.
- `generator.go` — `Generate` produces the connection string artifact. `Trigger` returns `ErrHTTPTriggerNotSupported` because MySQL doesn't go through the HTTP trigger path.
**The handshake sequence** in `handler.go`:
```go
func (h *Handler) HandleConnection(ctx context.Context, conn net.Conn) {
defer conn.Close()
conn.SetDeadline(time.Now().Add(10 * time.Second))
h.writeHandshake(conn) // send HandshakeV10
auth, _ := ReadClientAuth(conn) // read client auth response
if !strings.HasPrefix(auth.Username, "canary_") {
return // not our token, just drop
}
tokenID := strings.TrimPrefix(auth.Username, "canary_")
tok, _ := h.tokens.GetByID(ctx, tokenID)
if tok == nil {
return // unknown token, just drop
}
sourceHost := remoteHost(conn)
h.recordEvent(ctx, tok, sourceHost, auth)
h.writeAccessDenied(conn, auth.Username, sourceHost)
}
```
Notice the `canary_` prefix gate. Any client that doesn't send a `canary_*` username gets dropped silently — no handshake response, no error. That keeps generic MySQL scanners from extracting our `5.7.40-canary` server version banner via casual probing.
The `extra` JSONB stored on the event records the client's reported capabilities (e.g. `0x81bea285` for a typical `mysql` CLI) and charset (`0x21` for utf8mb4). Different attackers' tools have different capability fingerprints — a `mysql-cli` connection from a Linux host looks different from a Python `mysql-connector` connection looks different from a Go `database/sql` connection. The capability fingerprint is forensic gold.
**The error packet** in `protocol.go`:
```go
func BuildAccessDeniedErr(username, sourceHost string) ([]byte, error) {
msg := fmt.Sprintf(
`Access denied for user '%s'@'%s' (using password: YES)`,
username, sourceHost,
)
var payload bytes.Buffer
payload.WriteByte(0xff) // ERR packet marker
binary.LittleEndian.PutUint16(buf[:], 1045) // MySQL error 1045
payload.Write(buf[:])
payload.WriteByte('#') // SQL state marker
payload.WriteString("28000") // SQL state for auth fail
payload.WriteString(msg)
return wrapPacket(payload.Bytes(), seqIDServerErr)
}
```
Error code 1045 with SQL state 28000 is the standard MySQL auth-failure code. The message format is verbatim what a real MySQL server emits. If you ran a real MySQL and connected with a wrong password, you'd see the same text byte-for-byte.
### `slowredirect` (HTML interstitial + fingerprint)
**Files:** `backend/internal/token/generators/slowredirect/generator.go`, `fingerprint_handler.go`
**`Generate`** builds a `KindURL` artifact pointing at `/c/{tokenID}` (same as webbug). The interesting part is `Trigger`.
**`Trigger`** renders an HTML template (not embedded as a `[]byte` — embedded as `text/template.Template` parsed at startup) with:
```
{
"FingerprintURL": "/c/{tokenID}/fingerprint",
"Destination": "<operator-supplied destination URL>"
}
```
The template includes inline JS that:
1. Collects browser fingerprint signals (canvas, WebGL, fonts, timezone, etc.) using a small fingerprinting helper.
2. POSTs the JSON blob to `{FingerprintURL}`.
3. Waits a few seconds (template constant) so the fingerprint POST has time to complete.
4. Runs `window.location.replace(Destination)`.
The CSP header is set explicitly:
```
Content-Security-Policy: default-src 'none'; script-src 'unsafe-inline'; connect-src 'self'
```
This locks the page down so that even if the destination is an attacker-controlled URL passed through the operator's `metadata.destination_url`, the interstitial cannot be coerced into loading external scripts or making cross-origin XHRs.
**`fingerprint_handler.go`** is the POST handler:
```go
func (g *Generator) HandleFingerprint(w http.ResponseWriter, r *http.Request) {
tokenID := chi.URLParam(r, urlParamTokenID)
body, _ := io.ReadAll(io.LimitReader(r.Body, maxFingerprintBytes))
// Sanity-check that it's valid JSON; we don't parse the structure.
var dummy json.RawMessage
if err := json.Unmarshal(body, &dummy); err != nil {
w.WriteHeader(http.StatusNoContent)
return
}
g.fingerprintRecorder.AttachFingerprint(
r.Context(), tokenID, middleware.RealIP(r), body,
)
w.WriteHeader(http.StatusNoContent)
}
```
The fingerprint always returns 204 No Content — even on error. That's deliberate: if the attacker's network panel shows the fingerprint POST returning 400 or 500, they might notice. 204 with empty body is the most boring response possible.
`AttachFingerprint` writes the JSON blob to the *same event row* that the original trigger created, by updating `events.extra` for the most recent event matching `(token_id, source_ip)` within a short time window. That's why the manage page can show a fingerprint column alongside the trigger event: it's the same row.
**Enumeration resistance for slowredirect:** if the token ID is unknown, the response still renders a slowredirect HTML page — but with `Destination = "/"`. The attacker sees a redirect-to-root, which is indistinguishable from a token whose operator chose `/` as the destination. They can't tell the token is fake without seeing our alert side.
---
## The Trigger Handler (`backend/internal/token/handler.go`)
`HandleTrigger` is the function called for every `GET /c/{id}` and `* /k/{id}`. The shape:
```go
func (h *Handler) HandleTrigger(w http.ResponseWriter, r *http.Request) {
id := strings.TrimRight(chi.URLParam(r, urlParamTokenID), "_")
if id == "" {
http.NotFound(w, r)
return
}
tok, _ := h.svc.GetByID(r.Context(), id)
if tok != nil && !tok.Enabled {
tok = nil // treat disabled tokens like unknown ones
}
gen, ok := h.resolveGenerator(tok, r) // picks generator by path (/c/ vs /k/)
if !ok {
http.NotFound(w, r)
return
}
evt, resp, _ := gen.Trigger(r.Context(), tok, r)
if resp == nil {
http.NotFound(w, r)
return
}
if tok != nil && evt != nil {
h.events.Record(r.Context(), tok, evt)
}
h.writeTriggerResponse(w, r, resp)
}
```
Several things to note:
**Trailing underscore trim.** `strings.TrimRight(id, "_")` handles the PDF padded URL — recall that PDF tokens have `?p=____` padding appended. Most readers strip the query string but some malformed readers send the underscores as part of the path. Trimming `_` is a tolerance for that.
**Disabled tokens look unknown.** When the operator soft-disables a token (`DELETE /api/m/{manageId}`), the row stays in the DB but `enabled=false`. The trigger handler sets `tok = nil` for disabled tokens, which causes `resolveGenerator` to pick the default (webbug) and produces a pixel response with no event recorded — exactly the same behaviour as for a completely unknown token. From the attacker's perspective, there's no way to tell if a token was never minted or was minted-then-disabled.
**`resolveGenerator` picks by path prefix.** If `r.URL.Path` starts with `/k/`, the kubeconfig generator handles it. Otherwise the generator is looked up by `tok.Type`. This is how the wildcard `/k/.../*` route plumbs through to the kubeconfig generator without an explicit type lookup.
**Event recording is synchronous.** As covered in the architecture doc, `h.events.Record` runs *before* `writeTriggerResponse`. The trade-off is bounded by Postgres insert latency (typically sub-10ms).
---
## Event Service (`backend/internal/event/service.go`)
`Service.Record` is the choreographer. The flow:
```go
func (s *Service) Record(ctx context.Context, info NotifyInfo, evt *Event) error {
s.enrichGeo(evt) // attach GeoIP if available
s.repo.Insert(ctx, evt) // assigns evt.ID
s.tokens.IncrementTriggerCount(ctx, info.TokenID)
first := s.dedupGate(ctx, info.TokenID, evt.SourceIP)
if !first {
s.repo.UpdateNotifyStatus(ctx, evt.ID, NotifyDeduped, nil)
return nil
}
if s.notifier != nil {
s.notifier.Notify(info, evt) // enqueue, returns immediately
}
return nil
}
```
A subtle correctness property: the event row is inserted *before* the dedup gate check. That means even duplicate-suppressed events are persisted (with `notify_status = deduped`). The manage page can show "3 silenced triggers from 1.2.3.4" because the rows exist; it's just that no alerts fired for them.
The `dedupGate` function uses Redis `SETNX` for atomic first-acquire semantics:
```go
key := DedupKey(tokenID, sourceIP) // "dedup:trigger:abc:1.2.3.4"
set, _ := s.rdb.SetNX(ctx, key, 1, 15*time.Minute).Result()
if set { return true } // first hit
s.rdb.Incr(ctx, key)
s.rdb.SAdd(ctx, "dedup:active:"+tokenID, sourceIP)
s.rdb.Expire(ctx, "dedup:active:"+tokenID, 15*time.Minute)
return false // duplicate
```
The second Redis operation (`Incr` + `SAdd` + `Expire`) only runs on duplicates. The active-tracking set lets `CountActiveDedup` return a meaningful "N duplicate triggers silenced" badge to the manage page via `SCARD`.
The retention loop runs in its own goroutine (`RunRetentionLoop`) on a ticker. On each tick it calls `repo.PruneToLimit(perTokenLimit)` which deletes oldest events past the limit for each token. That bounds Postgres storage for high-volume tokens.
---
## Notification Service (`backend/internal/notify/service.go`)
The structure:
```go
type Service struct {
senders map[string]Sender // "telegram" → telegramSender, "webhook" → webhookSender
status StatusWriter // writes back to event.notify_status
sendTimeout time.Duration // 30s default
workers int // 8 default
queue chan dispatchJob // 256 buffered
workerWg sync.WaitGroup // tracks running workers
jobWg sync.WaitGroup // tracks in-flight + queued jobs
}
```
`Notify` is non-blocking:
```go
func (s *Service) Notify(info event.NotifyInfo, evt *event.Event) {
s.jobWg.Add(1)
select {
case s.queue <- dispatchJob{info, evt}:
// queued
default:
s.jobWg.Done()
s.markStatus(ctx, evt.ID, NotifyFailed, nil)
}
}
```
The non-blocking send is the load-shedding mechanism. If the queue is full (suggesting all 8 workers are stuck on slow senders), we drop the job and mark it failed rather than block the trigger handler.
Each worker is a simple consumer:
```go
func (s *Service) worker() {
defer s.workerWg.Done()
for job := range s.queue {
s.dispatch(job.info, job.evt)
s.jobWg.Done()
}
}
func (s *Service) dispatch(info, evt) {
ctx, cancel := context.WithTimeout(bg, 30*time.Second)
defer cancel()
sender, ok := s.senders[info.AlertChannel]
if !ok { mark failed; return }
if err := sender.Send(ctx, info, evt); err != nil { mark failed; return }
mark sent, time.Now().UTC()
}
```
The `closeOnce.Do(func() { close(s.queue) })` pattern in `Shutdown` ensures the queue is closed exactly once even if shutdown is called twice. After `close(queue)`, workers drain remaining jobs and exit when the channel reads return zero values.
`Wait()` is provided for test code to flush all pending notifications before assertions, via `jobWg.Wait()`.
### Webhook Sender (`backend/internal/notify/webhook/sender.go`)
POSTs a JSON body with `token_id`, `event_id`, `triggered_at`, `source_ip`, `user_agent`, GeoIP fields, and `extra`. If `WEBHOOK_HMAC_SECRET` is set in env, the body is signed with HMAC-SHA256 and the signature goes into `X-Canary-Signature: sha256={hex}`. Retries are via `cenkalti/backoff/v5` — exponential backoff with jitter, max 3 attempts within the 30-second timeout window, only retrying on transient errors (5xx, network errors, but not 4xx).
### Telegram Sender (`backend/internal/notify/telegram/sender.go`)
POSTs to `https://api.telegram.org/bot{TelegramBot}/sendMessage` with `chat_id={TelegramChat}` and a Markdown-formatted message. Retries are limited because Telegram returns proper status codes — a 403 means the bot doesn't have access to the chat, no amount of retries will help.
---
## Middleware (`backend/internal/middleware/`)
Each file is small and single-purpose:
| File | What It Does |
|------|--------------|
| `request_id.go` | Generates a UUID per request, attaches to context, exposes via `RequestID(ctx)` |
| `logging.go` | slog request_start / request_end with method, path, status, duration |
| `recovery.go` | `defer recover()` panic catcher; logs stack + returns 500 |
| `headers.go` | CSP, HSTS (prod only), X-Frame-Options, Referrer-Policy, X-Content-Type-Options; CORS middleware reads `cfg.CORS` |
| `realip.go` | Parses `X-Forwarded-For` and `X-Real-IP` only if the immediate peer is in `TRUSTED_PROXY_CIDRS` |
| `fingerprint.go` | `ExtractFingerprint(r) = sha256(RealIP(r) + r.UserAgent())[:16]` hex |
| `ratelimit.go` | Token bucket via Redis (`go-redis/redis_rate`) keyed by a `KeyFunc`; fail-open if Redis is down |
| `turnstile.go` | Reads response token from header `CF-Turnstile-Response` or JSON body field; calls `Verifier.Verify` |
| `operator_bearer.go` | Constant-time compare against `OPERATOR_TOKEN`; returns 404 on miss (not 401, to avoid revealing the endpoint exists) |
A few patterns worth absorbing:
**`OptionalHeader`** in `realip.go` (or wherever) returns `*string` — `nil` if empty, pointer to the value otherwise. This is for nullable DB columns (`user_agent`, `referer`) where an empty string is semantically different from "header was absent".
**Bearer middleware returns 404, not 401.** This is a small but deliberate choice. A 401 confirms the endpoint exists and just rejects your credentials. A 404 makes the endpoint look like it doesn't exist at all. An attacker scanning your service for admin endpoints can't distinguish "no admin endpoint here" from "I don't have the bearer". The cost is that legitimate operator typos look like 404s in logs — but operators don't make many typos on the admin endpoint URL.
**Rate limit fail-open.** If Redis is unreachable, the rate limiter logs a warning and lets the request through. The alternative (fail-closed) would mean a Redis outage immediately produces a service outage. For a canary service the trade-off is the right way around: a Redis outage shouldn't suppress real attacker activity.
---
## Frontend (`frontend/src/`)
The shape is two routes:
```
/ pages/landing/index.tsx
/m/:manageId pages/manage/index.tsx
```
**State management:** TanStack Query for *everything* server-touching. Each hook in `api/hooks/` validates with Zod:
```typescript
// frontend/src/api/hooks/useManageToken.ts
export function useManageToken(manageId: string, cursor?: number) {
return useQuery({
queryKey: ['manage', manageId, cursor],
queryFn: async () => {
const res = await api.get(`/m/${manageId}`, { params: { cursor } });
return ManageResponseSchema.parse(res.data);
},
refetchInterval: 5_000, // poll every 5s for new events
});
}
```
The Zod schema is the source of truth for the response shape. If the backend drifts, Zod throws on parse and the component error-boundary catches it — much better than silent type mismatches.
**Form state on the landing page** is plain `useState` per field. There's no `react-hook-form` or formal validation library; the input controls are typed (`<input type="url">`) and the backend re-validates on submit anyway. Beginner-readable.
**The artifact reveal** after `useCreateToken` mutates is conditional rendering on the artifact `kind`:
- `kind: "url"` → render `<CopyField>` with the URL
- `kind: "file"` → render a base64 → Blob download link
- `kind: "text"` → render an `<textarea readonly>` with the content plus a download button
- `kind: "connection_string"` → render `<CopyField>` with the connection string
**The manage page** is a single TanStack Query reading the paginated event feed. Pagination is cursor-based — the response includes `page.next_cursor` and `page.has_more`, and the "load more" button calls the same query with a higher cursor. Each event row renders source IP, GeoIP (country flag + city), user agent, timestamp, and `notify_status` as a coloured badge.
---
## Tests
The test stack is opinionated:
- **`stretchr/testify`** for assertions.
- **`miniredis/v2`** for in-memory Redis. Every test that touches Redis spins up a fresh miniredis on a random port — no shared state across tests.
- **`testcontainers-go` + `modules/postgres`** for Postgres integration tests. Tests that need a real Postgres get a real Postgres in Docker, migrated fresh per test package.
- **`testdata/` directories** alongside production code for fixture files (e.g. `backend/internal/token/generators/pdf/testdata/sample.pdf`).
Each generator package has both a unit test (`generator_test.go`) that exercises the pure logic against fixtures and an integration test where applicable. The MySQL package has the most thorough protocol tests — `protocol_test.go` builds packets from spec values and reads them back, asserting on every byte.
The `server` package has an `e2e_test.go` that spins up a real HTTP server with the full middleware chain and exercises the whole CRUD-and-trigger flow. That's the test you run to catch routing/middleware integration bugs.
---
## Where to Look First When Debugging
A short cheat-sheet:
| Symptom | First file |
|---------|------------|
| Token creation 400s with a strange message | `backend/internal/token/dto.go` (validator tags) |
| Token created but trigger 404s | `backend/internal/token/handler.go::HandleTrigger`, `resolveGenerator` |
| Trigger fires but no event in DB | `backend/internal/event/service.go::Record`, then `repository.go::Insert` |
| Event in DB but `notify_status = pending` forever | Worker pool is stuck. Check logs for `notify: send failed`. |
| Event in DB but `notify_status = failed` | Look at sender logs. Webhook URL bad? Telegram chat ID wrong? |
| Trigger response is the wrong content type | Generator's `Trigger` function — every type sets `ContentType` differently. |
| Manage page shows 0 events but they're in the DB | Cursor pagination bug. Check `parseCursor` in the handler and the `WHERE id < cursor` in `event/repository.go::ListByToken`. |
| `RealIP` middleware returns the proxy IP | `TRUSTED_PROXY_CIDRS` env var isn't set or is wrong. |
| PDF token won't open in Acrobat | Did the template change without re-running `cmd/buildpdftemplate`? Check `Generate`'s byte-length and contains-URL assertions. |
| DOCX opens but doesn't fire | Check the footer XML actually contains the placeholder. Different Word templates use different footer paths. |
| MySQL listener won't bind | Port collision. `MYSQL_FAKE_ADDR` is what the listener binds to. |
The next document, [04 - Challenges](04-CHALLENGES.md), proposes extensions you can build on top of this codebase.

251
PROJECTS/beginner/canary-token-generator/learn/04-CHALLENGES.md Normal file
View File

@ -0,0 +1,251 @@
<!-- © AngelaMos | 2026 | 04-CHALLENGES.md -->
# Challenges
This document proposes extensions to the canary token generator, organised by difficulty. Each challenge is sized to be a self-contained project on top of the existing codebase. The point is to deepen your understanding of the architecture by pushing it in directions the current design did or did not anticipate.
You don't need to do these in order. Pick ones that interest you.
---
## Easy (1-3 evenings each)
### 1. Add a Slack alert channel
Right now the project supports `telegram` and `webhook`. Slack has a different message format (rich attachments with colour-coded sidebars) and a different auth model (incoming webhook URLs, not bot tokens). Adding it cleanly means:
- New file `backend/internal/notify/slack/sender.go` implementing `notify.Sender` with `Channel() string { return "slack" }` and `Send(ctx, info, evt) error`.
- Extend `token.AlertChannel` validation in `backend/internal/token/dto.go` (validator tag is `oneof=telegram webhook`).
- Extend the React form on the landing page to render a Slack incoming-webhook URL field when the operator picks "Slack".
- Add a config field for an optional `SLACK_WEBHOOK_DEFAULT` if you want default routing.
- Register the sender in `backend/cmd/canary/main.go::buildEventStack`.
The natural test is `sender_test.go` with a `httptest.NewServer` that captures the POST body and asserts the rich-attachment JSON shape matches what Slack expects.
**Stretch:** add per-token Slack channel routing (the operator picks a channel suffix at token creation time, e.g. `#security-alerts` vs `#oncall`).
### 2. Per-token webhook signing secret
Currently the HMAC secret for webhook signing is a global env var (`WEBHOOK_HMAC_SECRET`). For multi-tenant deployments — or any deployment where different operators receive webhooks at different receivers — the secret should be per-token.
- Add a `webhook_hmac_secret` column to `tokens` (migration in `backend/internal/core/migrations/`).
- Plumb it through `event.NotifyInfo` and the webhook sender's `Send`.
- Generate a random secret automatically at token creation if `alert_channel = webhook`, and surface it once in the create response (operator copies it into their receiver configuration).
- Update the README to document the verification flow on the receiving side.
The interesting bit is the rotation story: if a secret leaks, how does the operator rotate it without redeploying the artifacts (whose URLs are immutable)? The cleanest answer is a `POST /api/m/{manageId}/rotate-secret` endpoint that issues a new secret and returns it once.
### 3. Filter events on the manage page
The manage page paginates events but doesn't filter. Common filter dimensions:
- By source IP (find all events from one IP)
- By country (find events from a specific country)
- By time range (last 24h, last 7d, custom)
- By notify status (show only `failed` to triage delivery problems)
The backend already has the data — `events` table is indexed on `triggered_at`. Add query params to `GET /api/m/{manageId}` and matching UI controls on the manage page. Use the same cursor-pagination pattern; just narrow the `WHERE` clause.
**Why this matters:** in a real deployment, a single canary token can fire hundreds of events over its lifetime (especially if it's a `webbug` planted in a public-ish location). The current "show me the most recent 20" view becomes useless quickly.
### 4. Event retention policy per-token
`event.Service.RunRetentionLoop` prunes events to a per-token global limit (`EVENT_RETENTION_PER_TOKEN`). In practice, different tokens want different retention. A `webbug` planted on a public link might fire frequently and need a rolling 30-day window; a `kubeconfig` planted on a service account might fire rarely and want forever-retention.
- Add a `retention_event_count` column (nullable) and a `retention_max_age` interval column to `tokens`.
- Modify `repo.PruneToLimit` (and consider renaming it `PruneByPolicy`) to honour per-token settings, falling back to the global default.
- Update the create form to optionally set retention.
This is a good exercise in handling SQL `NULL` cleanly in Go with `*int` / `*time.Duration` fields and `IS NOT NULL OR ...` in queries.
---
## Medium (a weekend each)
### 5. New token type: DNS resolution canary
A DNS canary fires when a unique subdomain is resolved. The artifact is the hostname; the trigger mechanism is *any* DNS query for that hostname hitting an authoritative server you control.
This requires:
- An authoritative DNS server, either delegated NS-records to your canary infrastructure for `*.canary.example.com`, or a separate listening port that you delegate from a public DNS host. Easiest path: a `miekg/dns` Go server on UDP/53 alongside the HTTP listener.
- A new generator `backend/internal/token/generators/dnscanary/` that:
- `Generate` returns a `KindURL` artifact containing the hostname.
- `Trigger` is unused on the HTTP side; the DNS server records the event directly via `event.Service.Record` when a query lands.
- A new handler that the DNS server calls per-query, extracting the token ID from the leftmost subdomain label.
**The clever part:** DNS queries are very chatty and most of them come from recursive resolvers, not the original querier. The event record should capture both the resolver's IP (from the query packet) and any EDNS Client Subnet (ECS) data, which leaks a /24 of the original querier in many cases. Most real-world canary tokens for DNS use this.
**Why it matters:** DNS canaries catch attackers using `nslookup` or `dig` against hostnames they find in a `.env` or config file. They also catch a class of pre-authentication enumeration tooling that resolves before connecting. Thinkst Canary's DNS token is one of their highest-value primitives.
### 6. New token type: AWS credential canary
A real AWS credential canary is a real `AKIA*` access key + matching secret. The "trigger" is a CloudTrail event for the IAM principal using that key. Real Thinkst tokens use AWS API Gateway + Lambda to detect the access attempts.
For this project, the simplified version is:
- Issue real (but heavily-permission-restricted) AWS access keys via the AWS SDK, scoped to a single IAM user that has no permissions except `sts:GetCallerIdentity` on its own ARN.
- Poll CloudTrail (or use EventBridge if the operator's account supports it) for `GetCallerIdentity` events using that key.
- Convert detected events into trigger events on your canary backend.
This is operationally heavier than the other tokens because it requires AWS API access from your canary infrastructure. The challenge is partly the AWS integration and partly the design decision about how the operator supplies their AWS credentials — direct keys, an assumed role, or a separate "AWS adapter" service.
**The cleaner alternative** for an educational project: generate a *fake* AKIA-style key, document the limitation that it only catches attackers who try to use it (not attackers who just exfiltrate it), and rely on a separate sub-token URL embedded in the key's "description" field that fires on retrieval.
### 7. JA3/JA4 fingerprinting on HTTPS trigger requests
Browser fingerprinting (the slowredirect type) requires JS to run. JA3/JA4 fingerprinting works at the TLS handshake level and catches every HTTPS client — including curl, wget, Python `requests`, and headless tools that won't run JS.
- Replace nginx in front with an HTTPS listener that exposes the TLS ClientHello bytes to the application layer. Options: terminate TLS in Go directly with `crypto/tls` (`GetClientHelloInfo` callback) or front with a JA-aware proxy like `Suricata` / `ja4-fingerprinter` / `caddy-tlsmonitor`.
- Extend the event payload's `extra` JSONB with `ja3_hash`, `ja4_hash`, and `tls_version`.
- Surface them in the manage page.
The forensic value is large: JA3/JA4 hashes are much harder to spoof than `User-Agent`, and they correlate the same attacker across multiple campaigns. The downside is that running your own TLS termination loses Cloudflare Tunnel as an option (Cloudflare terminates TLS at their edge before forwarding) — you'd need to deploy directly.
### 8. Bulk token operations
Power users want to mint 50 tokens at once with the same alert channel but different memos (one per workstation, say). Right now they have to click 50 times.
- Add `POST /api/tokens/bulk` accepting an array of `CreateRequest` objects.
- Rate-limit it independently from single-token creation (lower per-minute, but allow larger bursts of bulk requests).
- Frontend: a "Bulk" tab on the landing page with a CSV upload + per-row form.
- Return a zip of all artifacts in one response so the operator gets one download.
The interesting backend design problem is partial failure: if 49 tokens succeed and 1 fails, what's the response? Either you persist nothing on any failure (atomic) or you persist what succeeded and return a per-row status (best-effort). The codebase's domain layout suggests best-effort with per-row status is cleaner.
---
## Hard (a week or more each)
### 9. Detection-resistant artifact generation
Every artifact in this project has fingerprintable properties. The PDF placeholder padding produces a URL with a recognisable `?p=__________` query. The webbug pixel is byte-identical across all tokens of that type. The envfile canary uses a fixed `INTERNAL_METRICS_ENDPOINT` key.
A determined attacker who studies one of your tokens can identify others. Hardening:
- **PDF:** randomise the padding chars per-token (use printable URL-safe chars), or rotate among multiple template PDFs with placeholders in different fields (footer text, embedded form action, JS, etc.).
- **Webbug:** vary the pixel slightly per token. JPEG quantisation noise is enough to produce different SHA256 hashes for a "1x1 JPEG that looks the same".
- **Envfile:** randomise the canary key name from a list of plausible alternatives (`INTERNAL_METRICS_ENDPOINT`, `OBSERVABILITY_PUSH_URL`, `TELEMETRY_GATEWAY`, etc.). Randomise the position of the canary line within the file.
- **DOCX:** rotate among several footer styles or embed the URL in different document parts (settings.xml, footnotes, comments, etc.).
The exercise: pick one artifact type and design a configuration system that lets the operator dial how unique the per-token artifact should be (uniform vs randomised). Track the randomisation seed per token so you can reproduce the artifact later if needed.
**Why it matters:** Thinkst's commercial product is hardened against attacker fingerprinting in exactly this way. Open-source canaries are easier to fingerprint by definition (the source code is public), so this kind of work is the bridge between "educational project" and "operational deployment".
### 10. Operator-side retry pipeline for failed notifications
Right now if `notify_status = failed` (queue full, webhook receiver down, transient Telegram error), the event stays at that status forever. Operators have no built-in way to retry.
Build a retry pipeline:
- A new background goroutine (similar to the retention loop) that scans for `notify_status = failed` events with `notified_at IS NULL` and re-queues them on the notify service.
- Exponential backoff per event — first retry at 1 minute, then 5, then 30 — with a configurable max retry count.
- A new `retry_count` column on `events` to track attempts.
- Optional: a manual "retry" button on the manage page that bumps `retry_count = 0` and re-queues immediately.
The hardest design decision is when to give up. After N retries, do you alert the operator via a secondary channel (system-level email, dead-letter file write)? Do you just stop and require manual intervention? The right answer depends on the deployment model.
### 11. Multi-tenant deployment
The current architecture is single-operator. Multi-tenancy means:
- An auth model for token creation (not Turnstile — actual user accounts). Plug in WebAuthn for passwordless or use Cloudflare Access for SSO.
- A `users` table and a `user_id` foreign key on `tokens`.
- Manage routes (`/api/m/{manageId}`) become authorisation-aware: the requester must own the token (or use the public manage URL, which becomes a sharing primitive).
- Admin routes split into "global admin" (the deployment operator) and "per-user admin" (each tenant's view of their own tokens).
- Per-tenant rate limits — a tenant shouldn't be able to DOS others by exhausting the shared budget.
- Maybe per-tenant alert channel config (one tenant uses Telegram, another uses webhook, with separate validation).
This is a significant project — multi-tenancy is famously the rewrite that small services try to retrofit and end up rewriting twice. Doing it cleanly means starting from a clear data model and threading the tenant context through every layer.
### 12. Distributed deployment with leader election
The MySQL listener and the retention loop are designed assuming one process. If you want to run multiple `canary` instances behind a load balancer for HA, you need:
- **Leader election** for the retention loop (only one instance should be pruning). Easiest: a Redis key with `SETNX` + TTL renewal as the leader heartbeat. Slightly more robust: use `etcd` or `Consul`.
- **A shared MySQL listener address.** Each instance can bind to the same `0.0.0.0:3306` if you front them with an NLB; or they all bind to ephemeral ports and a service mesh routes by token-ID hash.
- **Shared dedup gate** — already works because the gate is in Redis.
- **Shared notification queue** — currently in-memory per process. Moving it to a Redis stream or a real broker (NATS, Kafka) is the right answer for actual scale.
This is the project where you'd genuinely benefit from running the existing one in production for a while first, hitting the limits, and then designing the shared infrastructure based on what actually breaks rather than what you imagine might.
---
## Defensive Drills (one-off exercises, hours each)
Not really "challenges" in the build-something sense; these are exercises to deepen your understanding by using the canary against yourself.
### A. Pwn-your-own-PDF
Generate a PDF token, save it, then open it in:
1. Chrome (built-in PDF viewer)
2. Firefox (PDF.js)
3. Acrobat Reader
4. macOS Preview
5. `pdftotext` (Poppler)
Which ones fire? Predict before checking the manage page. The answer is informative about how brittle PDF-based tokens are in practice.
### B. Pwn-your-own-DOCX
Same drill with a DOCX:
1. Microsoft Word (on Windows, on Mac)
2. LibreOffice
3. Pages
4. Google Docs (upload + view)
5. Office Online (upload + view)
6. `docx2txt` or `pandoc`
Which fire? What does the resulting fingerprint look like for each?
### C. Trace your own MySQL trigger
Connect to your local instance with the `mysql` CLI from a `canary_*` username. Watch the connection in Wireshark or `tcpdump -A -i lo0 -s 0 -X 'tcp port 3306'`. Identify each packet:
1. Server `HandshakeV10` (the server sends this first)
2. Client `HandshakeResponse`
3. Server `ERR_PACKET`
The point is to see how protocol-level the implementation really is. Read the bytes against the [MySQL Internals — Client/Server Protocol](https://dev.mysql.com/doc/internals/en/client-server-protocol.html) documentation.
### D. Try to enumerate
Set up the service. Mint one valid `webbug` token. Then write a script that hits `/c/{random}` URLs with random 12-char IDs and try to detect which ones are real based on:
- Response timing
- Response headers
- Response body bytes
If the script can reliably tell real from fake, the enumeration-resistance design has a hole. Find it. Fix it.
### E. Try to DOS the alert channel
Mint a token with a webhook alert channel pointing at a URL you control. Then write a script that hits the trigger URL from many source IPs (use a SOCKS pool, or just hit it from multiple cloud VMs).
How many concurrent IPs does it take before:
1. The notify queue fills up and starts dropping?
2. The webhook receiver gives up?
3. The operator notices?
Use the answer to inform the worker pool sizing for your real deployment.
---
## Reading List
If you want to deepen the conceptual side rather than the implementation side, these are the resources that informed the design of this project:
- **Thinkst Canary Tokens** documentation and Haroon Meer's talks (DEFCON, BSidesLV) — the modern canary canon.
- **MITRE Engage** matrix and example deployments — defensive deception taxonomy.
- **Tom Limoncelli's "How Complex Systems Fail"** — relevant when designing the dedup gate and failure modes.
- **The MySQL Internals docs** (Oracle) — needed if you want to extend the MySQL token to support more protocol features.
- **PDF 1.7 reference (ISO 32000-1)** — needed if you want to extend the PDF token beyond the URI page-open action.
- **Office Open XML format (ECMA-376)** — needed if you want to extend the DOCX token beyond the footer URI.
- **2024 Mandiant M-Trends report** and **2024 Verizon DBIR** — dwell-time numbers that make the case for deception.
---
Done. If you build any of these, the project is open to PRs — and if you build #5 (DNS canary) or #9 (detection-resistant artifacts), the maintainers want to talk to you.

2
README.md
View File

@ -62,7 +62,7 @@ Tools, courses, certifications, communities, and frameworks for cybersecurity pr
| **[Hash Cracker](./PROJECTS/beginner/hash-cracker)**<br>Dictionary and brute-force cracking | ![3-4h](https://img.shields.io/badge/⏱️_5--6h-blue) ![C++](https://img.shields.io/badge/C%2B%2B-00599C?logo=cplusplus&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Hash algorithms • Dictionary attacks • Password security<br>[Source Code](./PROJECTS/beginner/hash-cracker) \| [Docs](./PROJECTS/beginner/hash-cracker/learn) |
| **[Steganography Multi-Tool](./SYNOPSES/beginner/Steganography.Multi.Tool.md)**<br>Hide data in images, audio, QR, PDFs, text | ![2-3h](https://img.shields.io/badge/⏱️_8--10h-blue) ![Python](https://img.shields.io/badge/Python-3776AB?logo=python&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Multi-format steganography • Zero-width Unicode • Audio LSB • QR exploitation<br>[Learn More](./SYNOPSES/beginner/Steganography.Multi.Tool.md) |
| **[Ghost on the Wire](./SYNOPSES/beginner/Ghost.On.The.Wire.md)**<br>L2 attack & defense: MAC spoofing + ARP detection | ![2-3h](https://img.shields.io/badge/⏱️_6--8h-blue) ![Python](https://img.shields.io/badge/Python-3776AB?logo=python&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | ARP protocol • MAC spoofing • MITM detection • L2 trust mapping<br>[Learn More](./SYNOPSES/beginner/Ghost.On.The.Wire.md) |
| **[Canary Token Generator](./SYNOPSES/beginner/Canary.Token.Generator.md)**<br>Self-hosted honeytokens that alert on access | ![2-3h](https://img.shields.io/badge/⏱️_8--10h-blue) ![Go](https://img.shields.io/badge/Go-00ADD8?logo=go&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Deception defense • Honeytokens • Webhook alerting • Intrusion detection<br>[Learn More](./SYNOPSES/beginner/Canary.Token.Generator.md) |
| **[Canary Token Generator](./PROJECTS/beginner/canary-token-generator)**<br>Self-hosted honeytokens that alert on access | ![2-3h](https://img.shields.io/badge/⏱️_8--10h-blue) ![Go](https://img.shields.io/badge/Go-00ADD8?logo=go&logoColor=white) ![React](https://img.shields.io/badge/React-61DAFB?logo=react&logoColor=black) ![Docker](https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Deception defense • Honeytokens • MySQL wire protocol • PDF/DOCX patching • Webhook + Telegram alerting<br>[Source Code](./PROJECTS/beginner/canary-token-generator) \| [Docs](./PROJECTS/beginner/canary-token-generator/learn) |
| **[Security News Scraper](./SYNOPSES/beginner/Security.News.Scraper.md)**<br>Aggregate cybersecurity news | ![3-4h](https://img.shields.io/badge/⏱️_10--14h-blue) ![Python](https://img.shields.io/badge/Python-3776AB?logo=python&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Web scraping • CVE parsing • Database storage<br>[Learn More](./SYNOPSES/beginner/Security.News.Scraper.md) |
| **[Phishing Domain Generator & Quishing Scanner](./SYNOPSES/beginner/Phishing.Domain.Generator.And.Quishing.Scanner.md)**<br>Typosquat generation + QR phishing detection | ![2-3h](https://img.shields.io/badge/⏱️_6--8h-blue) ![Python](https://img.shields.io/badge/Python-3776AB?logo=python&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Homoglyph attacks • Typosquatting • QR code analysis • Domain intelligence<br>[Learn More](./SYNOPSES/beginner/Phishing.Domain.Generator.And.Quishing.Scanner.md) |
| **[SSH Brute Force Detector](./SYNOPSES/beginner/SSH.Brute.Force.Detector.md)**<br>Monitor and block SSH attacks | ![2-4h](https://img.shields.io/badge/⏱️_2--4h-blue) ![Python](https://img.shields.io/badge/Python-3776AB?logo=python&logoColor=white) ![Beginner](https://img.shields.io/badge/●_Beginner-green) | Log parsing • Attack detection • Firewall automation<br>[Learn More](./SYNOPSES/beginner/SSH.Brute.Force.Detector.md) |