Commit Graph

123 Commits

Author SHA1 Message Date
CarterPerez-dev f7f276df39 feat(monitor/frontend): dashboard WS client with reconnect + init handshake (TDD, 6 tests) 2026-05-02 05:35:18 -04:00
CarterPerez-dev 6aa3efacd6 feat(monitor/frontend): useSnapshot hook (single bootstrap, no refetch) 2026-05-02 05:33:53 -04:00
CarterPerez-dev 4216de3435 feat(monitor/frontend): ethos-aligned base styles (zero rounding, density, tabular nums) 2026-05-02 05:33:12 -04:00
CarterPerez-dev a34a175f14 feat(monitor/frontend): ethos design tokens (colors, type, density) 2026-05-02 05:32:26 -04:00
CarterPerez-dev 94d7e6035e docs(monitor/frontend): mirror ethos hard rules into SCSS comment block 2026-05-02 05:31:32 -04:00
CarterPerez-dev 2ac91da809 docs(monitor): add four world-collector rows to data source matrix 2026-05-02 04:43:03 -04:00
CarterPerez-dev 2f0c019f20 fix(monitor/collectors/gdelt): GDELT 'value' field is float64 not int (caught in live verify) 2026-05-02 04:42:39 -04:00
CarterPerez-dev d662a8c81a fix(monitor/collectors/wikipedia): move revid out of state: namespace (was leaking into snapshot) 2026-05-02 04:41:34 -04:00
CarterPerez-dev 994b510fa4 feat(monitor/main): wire usgs/swpc/wikipedia/gdelt/iss collectors under errgroup 2026-05-02 04:40:46 -04:00
CarterPerez-dev a7f0d5b097 feat(monitor/config): usgs/swpc/wikipedia/gdelt/iss config blocks 2026-05-02 04:39:51 -04:00
CarterPerez-dev ae01113086 feat(monitor/collectors/iss): 10s position poll + daily TLE refresh + propagator-ready collector 2026-05-02 04:39:07 -04:00
CarterPerez-dev 928db0befc feat(monitor/collectors/iss): SGP4 propagator wrapper (Position + LookAngles via go-satellite) 2026-05-02 04:37:25 -04:00
CarterPerez-dev 317fadeeef feat(monitor/collectors/gdelt): 15m collector with per-theme rolling z-score spike emit 2026-05-02 04:35:15 -04:00
CarterPerez-dev 066e2debc2 feat(monitor/collectors/gdelt): per-theme rolling z-score baseline detector 2026-05-02 04:31:10 -04:00
CarterPerez-dev e1d021a99a feat(monitor/collectors/wikipedia): 5m ITN collector with revid-based dedup and per-entry id hash 2026-05-02 04:30:14 -04:00
CarterPerez-dev 85626a0e11 feat(monitor/collectors/wikipedia): goquery-backed ITN parser (revid + entries with article slugs) 2026-05-02 04:28:28 -04:00
CarterPerez-dev 05ca2f529f feat(monitor/collectors/swpc): collector with 1m + 3h cadences and ring-buffered persistence 2026-05-02 04:27:13 -04:00
CarterPerez-dev 608a7937e7 feat(monitor/collectors/swpc): five-endpoint NOAA SWPC client (plasma/mag/kp/xray/alerts) 2026-05-02 04:25:17 -04:00
CarterPerez-dev fc3cfb523b feat(monitor/redisring): sorted-set ring buffer with score-based retention 2026-05-02 04:23:37 -04:00
CarterPerez-dev 2368ed8f18 feat(monitor/collectors/usgs): 1m collector with id-diff and earthquake emit 2026-05-02 04:22:37 -04:00
CarterPerez-dev 1d414ff341 feat(monitor/collectors/usgs): GeoJSON feed client (2.5_day, ms-since-epoch decoder) 2026-05-02 04:21:38 -04:00
CarterPerez-dev 11a68de2b7 feat(monitor/collectors/usgs): earthquakes repository (upsert, known-ids, recent-by-mag) 2026-05-02 04:20:49 -04:00
CarterPerez-dev d4995c1258 fix(monitor/collectors/coinbase): connection-global sequencer; gap is non-fatal log
Two related issues caught in live verification:

1. Coinbase's sequence_num is connection-global (not per-product). Previous
   per-product tracking flagged every cross-product frame as a gap.

2. Even after switching to global tracking, gaps still occur regularly under
   normal operation (Coinbase's exact sequencing pattern across channels is
   not strictly seq+1 frame-to-frame). Treating gaps as fatal kills the
   ReadLoop in seconds, never giving the minute aggregator a chance to cross
   a boundary, so btc_eth_minute stays empty.

Resolution: rewrite Sequencer for a single global counter and downgrade
gap-detection to a non-fatal Warn. ReadLoop continues; (symbol, ts) PK on
btc_eth_ticks is the safety net for any duplicate replay. Includes a
diagnostic log on loop exit (handleConn) for ops visibility.
2026-05-02 04:03:09 -04:00
CarterPerez-dev 989e53c958 fix(monitor/collectors/coinbase): parse Coinbase's Go-style time format in heartbeat current_time
Coinbase Advanced Trade WS leaks Go's time.Time.String() format in
heartbeat events (e.g. "2026-05-02 07:55:50.784... +0000 UTC m=+102632...")
which is not RFC 3339 — UnmarshalJSON into time.Time fails and the entire
envelope errors. Decode current_time as a string and parse it with a
fallback that handles both RFC 3339 and the Go-string variant (with the
m=+monotonic suffix stripped).
2026-05-02 04:02:49 -04:00
CarterPerez-dev 2ccc379255 fix(monitor/collectors/coinbase): exhaustive switches, %w double-wrap, require.ErrorIs (lint) 2026-05-02 03:42:59 -04:00
CarterPerez-dev d352a6bce6 chore(monitor): gofmt const-block alignment in phase-2 collectors 2026-05-02 03:39:51 -04:00
CarterPerez-dev 0027de6c72 docs(monitor): add coinbase row to cyber data source matrix 2026-05-02 03:37:56 -04:00
CarterPerez-dev 99f6e41845 feat(monitor/main): wire coinbase BTC/ETH collector under errgroup 2026-05-02 03:36:26 -04:00
CarterPerez-dev 9c35e14ae8 fix(monitor/collectors/coinbase): use envelope timestamp (per-ticker time field is not in real Coinbase frames) 2026-05-02 03:36:26 -04:00
CarterPerez-dev 0a6f4f1e09 feat(monitor/config): coinbase config block (enabled, url, product_ids, throttle) 2026-05-02 03:32:53 -04:00
CarterPerez-dev 89f1c10249 feat(monitor/collectors/coinbase): collector Run loop ties dial+readloop+aggregator+repo+throttled-emit 2026-05-02 03:32:16 -04:00
CarterPerez-dev f6192e9c04 feat(monitor/collectors/coinbase): per-product minute aggregator with rollover-emit 2026-05-02 03:30:24 -04:00
CarterPerez-dev 23a4e2ded4 feat(monitor/collectors/coinbase): gap-aware ReadLoop (snapshot resets sequencer; gap → ErrSequenceGap) 2026-05-02 03:29:43 -04:00
CarterPerez-dev 3be4c9d815 feat(monitor/collectors/coinbase): reconnect-with-backoff helper (1s init, 60s max, ctx-aware) 2026-05-02 03:28:24 -04:00
CarterPerez-dev 3e398da3fb feat(monitor/collectors/coinbase): per-product sequence-gap detector 2026-05-02 03:27:32 -04:00
CarterPerez-dev 10862314f0 feat(monitor/collectors/coinbase): WS dialer + frame decoder for ticker/heartbeats/snapshot 2026-05-02 03:26:58 -04:00
CarterPerez-dev ae5a83c2f8 feat(monitor/collectors/coinbase): ticks + minute OHLC repo with 1h history reads
Adds shopspring/decimal as a require (folded with the first consuming task per
Plan 2 convention). cenkalti/backoff/v4 is also now consumed transitively;
remains indirect until Task 5 (reconnect) imports it.
2026-05-02 03:24:55 -04:00
CarterPerez-dev 687e13389e chore(monitor): tidy headers, drop unused StateDown, gofmt config defaults 2026-05-02 03:01:48 -04:00
CarterPerez-dev b9c5fa1e13 docs(monitor): add cyber data source matrix to README 2026-05-01 22:53:58 -04:00
CarterPerez-dev c64141bc0b feat(monitor/main): wire dshield/cfradar/cve/kev/ransomware collectors under errgroup 2026-05-01 22:51:50 -04:00
CarterPerez-dev cb8cd902f6 feat(monitor/config): collectors config (toggles, intervals, API key bindings) 2026-05-01 22:50:55 -04:00
CarterPerez-dev 6cc3e681de feat(monitor/enrich/greynoise): on-demand IP lookup with 404→ErrUnknownIP via httpx.StatusError 2026-05-01 22:49:48 -04:00
CarterPerez-dev 894ecc8fe5 feat(monitor/collectors/ransomware): 15m collector with hash-diff and victim emit 2026-05-01 22:48:09 -04:00
CarterPerez-dev 3d71fd8c84 feat(monitor/collectors/ransomware): ransomware.live client with stable triple-hash ID
Hand-crafted fixture matches spec §8.5 schema. Live API endpoint may move; client
takes BaseURL config for swap-in once api.ransomware.live publishes a stable JSON
endpoint (current free tier returns HTML landing page on /recentvictims).
2026-05-01 22:47:19 -04:00
CarterPerez-dev 01a22dc7e4 feat(monitor/collectors/ransomware): victims repository with idempotent insert 2026-05-01 22:45:23 -04:00
CarterPerez-dev bdca22ee50 feat(monitor/collectors/kev): 1h KEV collector with set-diff and chime topic 2026-05-01 22:44:37 -04:00
CarterPerez-dev 7395f1064f feat(monitor/collectors/kev): CISA KEV catalog client 2026-05-01 22:43:11 -04:00
CarterPerez-dev 701bc8e790 feat(monitor/collectors/kev): kev_entries repository with diff lookup 2026-05-01 22:42:18 -04:00
CarterPerez-dev 5ad1ff8e14 feat(monitor/collectors/cve): 2h CVE collector (NVD pull + EPSS enrichment per tick) 2026-05-01 22:41:24 -04:00
CarterPerez-dev 35da1f45a6 feat(monitor/collectors/cve): FIRST EPSS client with 100-id batching 2026-05-01 22:40:15 -04:00
CarterPerez-dev f5e8cf5548 feat(monitor/collectors/cve): NVD CVE 2.0 client (apiKey header, last-modified window, NVDTime) 2026-05-01 22:39:29 -04:00
CarterPerez-dev 2a61953cb9 feat(monitor/collectors/cve): cve_events repository with EPSS-only patch 2026-05-01 22:38:08 -04:00
CarterPerez-dev ee358e38f1 feat(monitor/collectors/cfradar): 5m collector with diff-on-ID emitting outages + hijacks 2026-05-01 22:37:10 -04:00
CarterPerez-dev 428610b663 feat(monitor/collectors/cfradar): radar client (outages + bgp hijacks) with bearer auth 2026-05-01 22:35:26 -04:00
CarterPerez-dev ada39ef6d9 feat(monitor/collectors/cfradar): outages + hijacks repository with diff lookups 2026-05-01 22:34:03 -04:00
CarterPerez-dev fd861d587e feat(monitor/collectors/dshield): hourly tick collector emitting scan_firehose 2026-05-01 22:32:10 -04:00
CarterPerez-dev 808eb1b892 feat(monitor/collectors/dshield): http client + frozen fixtures (topports, topips, dailysummary) 2026-05-01 22:31:01 -04:00
CarterPerez-dev a80664b32a feat(monitor/collectors/dshield): snapshot repository (per-kind upsert + read-latest) 2026-05-01 22:28:42 -04:00
CarterPerez-dev 91d855f4ef feat(monitor/collectors/state): collector_state CRUD with healthy/degraded upserts 2026-05-01 22:26:57 -04:00
CarterPerez-dev c5cc7951a3 feat(monitor/httpx): rate-limited HTTP client with 429/5xx retry honoring Retry-After 2026-05-01 22:25:37 -04:00
CarterPerez-dev 33d15a6ab6 docs(monitor): quickstart README 2026-05-01 21:38:59 -04:00
CarterPerez-dev 4bf29da96f fix(monitor/nginx): strip /api prefix on WS proxy_pass so backend route /v1/ws is reached 2026-05-01 21:38:04 -04:00
CarterPerez-dev 92202b1c16 fix(monitor): pick conflict-free host ports (8432/5432/4432/6432/3432); JWT keygen-on-boot, healthcheck path, baseline migration; ignore frontend/.pnpm-store; pre-commit excludes 2026-05-01 20:13:00 -04:00
CarterPerez-dev ecaacc48fa chore(monitor): replace Makefile with Justfile (project convention) 2026-05-01 14:53:28 -04:00
CarterPerez-dev a67ef20680 feat(monitor/main): wire bus + hub + snapshot + heartbeat under errgroup; expose /v1/snapshot, /v1/ws, /v1/healthz 2026-05-01 13:53:16 -04:00
CarterPerez-dev a7865533cd feat(monitor/backend): ws hub, snapshot store, heartbeat collector with TDD 2026-05-01 07:09:28 -04:00
CarterPerez-dev bc9c2d06eb feat(monitor/backend): events, ratelimit, bus core packages with TDD 2026-05-01 07:03:47 -04:00
CarterPerez-dev 73c56333e6 feat(monitor/db): goose migrations 0001-0003 (alerts, panel data, collector state) 2026-05-01 06:28:51 -04:00
CarterPerez-dev 618aaf43e8 deps(monitor/backend): add coder/websocket, gobreaker/v2, errgroup, prometheus, testcontainers 2026-05-01 06:27:23 -04:00
CarterPerez-dev a627161b33 refactor(monitor/backend): rename module to monitor-the-situation/backend 2026-05-01 06:21:01 -04:00
CarterPerez-dev e46728bd80 feat(monitor/infra): three compose files, Makefile, .env.example 2026-05-01 06:17:28 -04:00
CarterPerez-dev a03c060b9f feat(monitor/nginx): base + dev + prod configs with WS upgrade routing 2026-05-01 06:16:12 -04:00
CarterPerez-dev 470533fd66 feat(monitor): infra scaffold — Dockerfiles, conf/ tree, .gitignore 2026-05-01 06:12:25 -04:00