Cybersecurity-Projects/PROJECTS/advanced/monitor-the-situation-dashb...
CarterPerez-dev d01fc57989 fix(monitor/frontend): DShield panel matches actual snapshot.scan_firehose shape
The panel crashed at runtime with "(ds.topports ?? []).slice is not a
function" because Plan 5 assumed shapes that didn't match what the Phase 2
DShield collector actually emits. Reality:

  topports: Record<string, Port>      // dict keyed "0".."N", not Array
  port field: targetport              // not "port"
  topips[].source                     // the IP — not "ip"
  topips[].reports                    // the count — not "records"
  topips                              // has no country/CC field
  dailysummary[].date                 // ISO date string, ascending order

toArray() helper accepts both Record<string,T> and T[] defensively (so
either shape works if the backend changes its mind). Both lists sort by
rank before slicing. The CC column is dropped (data doesn't carry it);
its slot becomes the per-source target count which the data does carry.
Daily summary picks the latest entry by date instead of index 0 (the
order is ascending, so [0] would be the oldest day).

Bigger discovery: most snapshot keys (cve_new / kev_added /
ransomware_victim / coinbase_price / etc.) are LATEST-event payloads,
not the recent-list arrays Plan 5 anticipated. Tasks 15-21 will need
per-topic Zustand stores that accumulate over time, populated by Task 24
from snapshot bootstrap + WS messages. Will discuss with Carter before
committing to that architecture.
2026-05-03 09:12:36 -04:00
..
backend fix(monitor/collectors/gdelt): GDELT 'value' field is float64 not int (caught in live verify) 2026-05-02 04:42:39 -04:00
conf fix(monitor/nginx): strip /api prefix on WS proxy_pass so backend route /v1/ws is reached 2026-05-01 21:38:04 -04:00
frontend fix(monitor/frontend): DShield panel matches actual snapshot.scan_firehose shape 2026-05-03 09:12:36 -04:00
.env.example fix(monitor): pick conflict-free host ports (8432/5432/4432/6432/3432); JWT keygen-on-boot, healthcheck path, baseline migration; ignore frontend/.pnpm-store; pre-commit excludes 2026-05-01 20:13:00 -04:00
.gitignore fix(monitor): pick conflict-free host ports (8432/5432/4432/6432/3432); JWT keygen-on-boot, healthcheck path, baseline migration; ignore frontend/.pnpm-store; pre-commit excludes 2026-05-01 20:13:00 -04:00
Justfile chore(monitor): replace Makefile with Justfile (project convention) 2026-05-01 14:53:28 -04:00
README.md docs(monitor): add four world-collector rows to data source matrix 2026-05-02 04:43:03 -04:00
cloudflared.compose.yml feat(monitor/infra): three compose files, Makefile, .env.example 2026-05-01 06:17:28 -04:00
compose.yml fix(monitor): pick conflict-free host ports (8432/5432/4432/6432/3432); JWT keygen-on-boot, healthcheck path, baseline migration; ignore frontend/.pnpm-store; pre-commit excludes 2026-05-01 20:13:00 -04:00
dev.compose.yml fix(monitor): pick conflict-free host ports (8432/5432/4432/6432/3432); JWT keygen-on-boot, healthcheck path, baseline migration; ignore frontend/.pnpm-store; pre-commit excludes 2026-05-01 20:13:00 -04:00

README.md

Monitor the Situation

Operator-grade real-time situational awareness dashboard. Single-binary Go backend, React 19 frontend, Postgres + Redis, fronted by nginx and (optionally) a Cloudflare Tunnel.

The phrase "Monitoring the situation" is a Twitter/X meme from June 2025. This is the version that actually monitors the situation.

Stack

Layer Tech
Backend Go 1.22, chi router, coder/websocket, goose migrations
Frontend React 19, Vite, TanStack Query, Zustand, MapLibre, D3
Storage Postgres 16 (BRIN-indexed time-series), Redis 7
Ingress nginx (dev + prod), Cloudflare Tunnel (prod)
Build / run just recipes, multi-stage Docker, air for live reload

Data sources

Panel Source Cadence Auth
Mass-scan firehose DShield (SANS ISC) 1h none
Internet outages + BGP hijacks Cloudflare Radar 5m CF_RADAR_TOKEN (Radar:Read scope)
CVE velocity + EPSS NVD CVE 2.0 + FIRST EPSS 2h NVD_API_KEY (optional, raises rate limit)
CISA KEV CISA KEV catalog 1h none
Ransomware victims ransomware.live 15m none
Live BTC + ETH ticks Coinbase Advanced Trade WS persistent none
Earthquakes (M2.5+) USGS GeoJSON feed 1m none
Space weather NOAA SWPC (5 endpoints) 1m / 3h none
World events Wikipedia ITN + GDELT v2 API 5m / 15m none
ISS position wheretheiss.at + CelesTrak 10s / 24h none
IP enrichment GreyNoise Community on-demand GREYNOISE_API_KEY (optional, free tier)

Quickstart (development)

cp .env.example .env
# fill .env: POSTGRES_PASSWORD, JWT_SECRET, NOTIFICATION_ENCRYPTION_KEY
just dev-start
just migrate-dev
open http://localhost:8432

JWT signing keys auto-generate at backend/keys/private.pem on first boot. The dev stack binds host ports 8432 (nginx) / 5432 (backend) / 4432 (postgres) / 6432 (redis) / 3432 (vite).

Smoke checks

curl -s http://localhost:8432/api/v1/healthz
curl -s http://localhost:8432/api/v1/snapshot | jq .
docker run --rm -i --network host ghcr.io/vi/websocat:latest \
    "ws://localhost:8432/api/v1/ws?topics=heartbeat"

Production (Cloudflare Tunnel)

cp .env.example .env
# fill production secrets including CLOUDFLARE_TUNNEL_TOKEN
just tunnel-start
just migrate

Tests

cd backend && go test -race ./...

Layout

backend/    Go services (cmd/api, internal/{events,bus,ws,snapshot,collectors,...})
frontend/   React 19 dashboard
conf/       nginx and per-environment Docker configs
migrations/ goose SQL migrations (mounted into the backend container)