Commit Graph

527 Commits

Author SHA1 Message Date
CarterPerez-dev 79cbdaaa51 feat(monitor/frontend): CVE velocity panel (1h/6h/24h KPIs + 24h sparkline + recent table)
CveStore (zustand) accumulates CveEvent items dedup-by-CveID, capped at
500. The panel seeds the store from snapshot.cve_new on mount via
useEffect — Task 24's WS routing will extend this with live deltas.

Three KPIs derived in-component from the store (1h/6h/24h counts, mono
tabular --type-num-l). 24h sparkline computed from per-hour bucketing
of items[].Published (one of the few places a chart is justified —
shape-of-CVE-velocity IS the insight per ethos rule on charts). Bottom
table shows the 5 most recent: CVE-ID mono, severity sans plain (NOT a
colored badge — severity stays monochrome per ethos), EPSS percentile
mono right-aligned, relative-time mono muted right-aligned. table-layout:
fixed + per-column widths + ellipsis to keep rows on a single line in
the 320px column.

Real CveEvent shape matches the backend Row struct (verified against
live snapshot): PascalCase keys CveID/Published/LastModified/Severity/
CVSS/EPSSScore/EPSSPercentile/InKEV. Earlier plan-author assumption of
{recent_24h:[]} was wrong — snapshot delivers a single latest event,
the running list builds in the store over time.

Plan 5 Task 15 Design QA: KPIs mono tabular, sparkline pure stroke
(currentColor, no fill/axes), severity NOT a colored badge, EPSS mono %,
ago muted, no "view all" CTA — operator clicks NVD via the raw link.
2026-05-03 09:20:58 -04:00
CarterPerez-dev 9ea40198d0 fix(monitor/frontend): stack DShield tables vertically (no horizontal overflow in 320px column)
Plan 5 said "two compact tables side-by-side" but didn't account for the
320px right-column width — tables overflowed and the source-IP table's
Reports/Tgt columns scrolled off-screen. Replaces grid 1fr 1.2fr with a
flex column so each table gets the full panel width: top ports on top,
top sources below.
2026-05-03 09:16:58 -04:00
CarterPerez-dev d01fc57989 fix(monitor/frontend): DShield panel matches actual snapshot.scan_firehose shape
The panel crashed at runtime with "(ds.topports ?? []).slice is not a
function" because Plan 5 assumed shapes that didn't match what the Phase 2
DShield collector actually emits. Reality:

  topports: Record<string, Port>      // dict keyed "0".."N", not Array
  port field: targetport              // not "port"
  topips[].source                     // the IP — not "ip"
  topips[].reports                    // the count — not "records"
  topips                              // has no country/CC field
  dailysummary[].date                 // ISO date string, ascending order

toArray() helper accepts both Record<string,T> and T[] defensively (so
either shape works if the backend changes its mind). Both lists sort by
rank before slicing. The CC column is dropped (data doesn't carry it);
its slot becomes the per-source target count which the data does carry.
Daily summary picks the latest entry by date instead of index 0 (the
order is ascending, so [0] would be the oldest day).

Bigger discovery: most snapshot keys (cve_new / kev_added /
ransomware_victim / coinbase_price / etc.) are LATEST-event payloads,
not the recent-list arrays Plan 5 anticipated. Tasks 15-21 will need
per-topic Zustand stores that accumulate over time, populated by Task 24
from snapshot bootstrap + WS messages. Will discuss with Carter before
committing to that architecture.
2026-05-03 09:12:36 -04:00
CarterPerez-dev a91f290595 feat(monitor/frontend): DShield panel (top ports + top source IPs, dense tables)
Two side-by-side tables fed from snapshot.scan_firehose: Top Ports (port +
24h hits) and Top Sources (IP + CC + hits). Below them a one-line muted
mono summary "Xk records · Yk sources · Z targets — last 24h." Tables, not
bar charts — operator wants to read the port number 22 / 3389, not stare
at a horizontal bar.

Port and IP and hit counts are mono with tabular-nums (digit alignment
matters). Country code stays sans (text label, not a number). Hover row
background var(--bg-row-hover) — subtle, no transform / no transition /
no theatre. Defensive empty-state — when snapshot has no scan_firehose
yet, tables render as empty (per ethos: no "no data" message).

PORT_ROW_LIMIT / SOURCE_ROW_LIMIT / THOUSAND / MILLION constants at module
scope (no inline magic). Renders in the right column as the bottom panel
per spec §11.1 column ordering.
2026-05-03 07:15:16 -04:00
CarterPerez-dev 44f0ff3c3d feat(monitor/frontend): shared Panel shell + KPI + Sparkline + StaleIndicator primitives
The shared shell every operator panel uses. 28px header strip with
UPPERCASE letter-spaced title (--type-label, --fg-2), optional muted
subtitle (--fg-3), StaleIndicator dot, and an external-link "raw"
icon → upstream source. Body fills remaining space and scrolls
internally. Border-bottom 1px --fg-4 separates panels in the column.

Primitives in shared/:
  KPI — value (mono, --type-num-l, tabular-nums, --fg-1) + label
        (UPPERCASE, --type-label, --fg-3). No card wrapper, no
        rounded corner, no background. The number IS the unit.
  Sparkline — d3-scale + d3-array driven inline SVG. Single stroke
        in currentColor, no axes / no fill / no animation. Used
        only when the shape IS the insight (per ethos). Returns
        null with <2 points so empty panels don't ship a flat line
        as a placeholder.
  StaleIndicator — 6px square dot. --ok green (fresh), --amber
        (stale, breached threshold), --fg-4 muted (unknown / no
        data yet). Title attribute carries last update UTC time
        for hover context.

RawLink not extracted yet — Panel inlines the icon since one raw
link per panel is the rule. Extract when a panel needs more.

Adds d3-scale 4.0.2 + d3-array 3.2.4 + their @types — small d3
modules (not the full umbrella) per spec §10.3 to keep bundle lean.

Plan 5 Task 13 Design QA: panel header 28px, title UPPERCASE
letter-spaced, sparkline pure stroke (no axes / fill), stale dot
6px, no border-radius anywhere, no hover scale.
2026-05-03 07:12:20 -04:00
CarterPerez-dev e3af0197b7 feat(monitor/frontend): globe centerpiece (merged-points, imperative camera, no decorative motion)
react-globe.gl 2.37 + three 0.184 added. Globe component is React.memo'd,
absolute-fills the .center grid cell, sizes itself via ResizeObserver on
the wrapper div (the library defaults to window.innerWidth/Height which
would blow past the 1247px cell). pointsMerge: true so hundreds of
points cost one ThreeJS Mesh draw call.

Layer dataset built by useGlobePoints / useGlobeRings hooks in
globeLayers.ts, projecting each store point onto color/altitude/radius
lookup tables keyed on the 6 GlobePointTypes:
  earthquake/ransomware/scan = monochrome (--fg-2/--fg-1/--fg-4)
  iss = --ok (live position is functional, not decorative)
  outage/hijack = --amber (the only globe surfaces with alarm color —
  earthquakes and ransomware stay monochrome because an M2.5 quake or
  daily victim is not user-actionable)

Rings are simple white pulses (no per-ring color decoration), single
pulse (ringRepeatPeriod=0). Camera panTo() in globeCamera.ts is the
only motion API; the dashboard lifecycle (Task 24) will trigger it on
high-mag quakes etc. Auto-rotate disabled on globe ready — motion is
meaning, a globe that spins for no reason violates the rule.

atmosphereAltitude=0.12, atmosphereColor=#1f2937 per spec §10.4 — a thin
glow that gives the sphere silhouette without becoming "3D space
ambiance." backgroundColor=rgba(0,0,0,0) so the panel cell --bg shows
through. No starfield, no nebulas. No globeImageUrl — keeps the build
self-contained (no CDN dep); we add a local texture later if geographic
context needs to be richer.

Plan 5 Task 12 Design QA: no starfield/nebula/extra glow, monochrome
quake dots, amber confined to outage/hijack, no hover tooltip on globe.
2026-05-03 07:09:44 -04:00
CarterPerez-dev 02ecc1c214 feat(monitor/frontend): F-key presentation mode + localStorage persistence
Press F → hide TopStrip + BottomTicker, drop the panel column borders, and
shrink panel columns from 320 to 280 so the globe gets more room. Esc →
exit. Mode persists to localStorage at STORAGE_KEYS.PRESENTATION_MODE so
refresh maintains it (use case: second-monitor wall).

Implementation: each chrome component (TopStrip, BottomTicker) reads
presentationMode from the ui store and early-returns null. Dashboard
applies a .presentation modifier to .root that re-templates the inner
.grid columns and zeroes the aside borders. CSS Modules-scoped — no
:global escape, no clsx dep.

isTypingInForm guards F so it doesn't fire when typing in inputs / textareas
/ contentEditable. Escape always exits — universal "get me out" semantics.
useUIStore.getState() inside the keydown handler avoids stale closure.

Plan 5 Task 11 Design QA: F hides chrome cleanly without animation flash,
globe area expands, panels stay readable, Esc restores, state persists.
2026-05-03 07:02:56 -04:00
CarterPerez-dev 92d8a1aabf feat(monitor/frontend): bottom ticker (world events marquee, 60s loop)
32px strip in the bottomticker grid-area that renders the ticker Zustand
store as a horizontally scrolling marquee — one of the few places motion
is allowed because new entries pushing older ones along IS data changing.
Source label uppercase + letter-spaced --fg-3, headline body --fg-2,
relative timestamp mono tabular --fg-3.

Empty state is a bare strip (no "no events" message, no illustration) —
operator UI doesn't onboard. The track only renders when items.length > 0
so the marquee animation isn't running over an empty list.

formatRel uses module-scope SECONDS_PER_MINUTE / SECONDS_PER_HOUR /
MS_PER_SECOND constants (no inline magic numbers). Animation uses kebab
keyframe name (stylelint keyframes-name-pattern).
2026-05-03 07:00:27 -04:00
CarterPerez-dev 0ba29226a2 feat(monitor/frontend): alert banner (one-amber-fill overlay, 30s auto-dismiss)
The only surface in the dashboard that uses amber as a fill color — the
one-alarm-color rule reserves this paint for "look here." 48px strip in
the alertbanner grid-area, dark text (--amber-fg #1a1a1a) on the amber
fill for cross-room legibility, ellipsis on long messages, dismiss button
at right. Returns null when the ui store has no currentAlert so the row
collapses to zero and the center grid sits flush with the top strip.

Auto-dismiss after 30s via setTimeout in useEffect; cleanup on unmount
or alert change. role="alert" for screen readers. focus-visible outline
1px --amber-fg.

Adds --amber-fg to ethos tokens — pairs explicitly with --amber so any
future amber-fill surface uses the same dark text. Also cleans up an
empty `//` line in the ethos comment block (stylelint scss/comment-no-empty).
2026-05-03 06:58:37 -04:00
CarterPerez-dev 5e33a4db22 feat(monitor/frontend): top strip (title + UTC clock + about/user icons)
Thin 36px strip in the topstrip grid-area: left = MONITORING THE SITUATION
label + about-trigger icon, center = HH:MM:SS UTC mono clock with tabular-nums
ticking once per second, right = settings gear (auth) or user (anonymous)
that navigates via ROUTES.{SETTINGS,LOGIN}. About icon dispatches openAbout
on the ui store; the modal itself is Task 23.

Pure monochrome — title --fg-2 letter-spaced uppercase --type-label, clock
--fg-2 mono, icons --fg-3 with --fg-1 hover color shift (no transition,
no scale, no rounded surface). Reads tmux-status-line, not SaaS app header.
focus-visible outline 1px --fg-2 for keyboard a11y. Dashboard.tsx renders
<TopStrip /> as a direct child of .root so it auto-places into the
topstrip named area.

Plan 5 Task 8 Design QA: strip height, label tier, mono tabular clock,
zero rounding, tmux feel — all verified.
2026-05-03 06:56:03 -04:00
CarterPerez-dev c7a0dd33a2 feat(monitor/frontend): dashboard 3-col grid layout (320 / 1fr / 320, no body scroll)
Replaces the demolition scaffold with the structural grid that Tasks 8-21
fill in. Root is a 4-row grid (topstrip / alertbanner / main / bottomticker)
with named grid-areas; the main row contains a 3-col sub-grid (left aside /
center section / right aside) at fixed 320px / 1fr / 320px. 100dvh, no body
scroll, ethos custom properties only — no template SCSS variables touched.

Adds --col-panel-width: 320px to the ethos token block (no magic number in
the layout module). index.tsx becomes a one-line re-export of Dashboard so
the lazy-route contract in routers.tsx stays unchanged.

No placeholder components. The empty <aside>/<section> elements are the
column containers themselves; subsequent tasks add real children inside
them rather than swap stub components.

Verified in browser at 1920x1080: rgb(10,10,10) bg, 320/1247/320 col rects,
1px var(--fg-4) rules between columns, body has no scroll. All 5 Plan 5
Task 7 Design QA boxes checked.
2026-05-03 06:49:16 -04:00
CarterPerez-dev 50b0e96735 chore(monitor/frontend): demolish template shell + landing for clean dashboard canvas
Cleared:
- core/app/shell.tsx + shell.module.scss (sidebar+header layout)
- core/app/toast.module.scss (replaced by inline ethos toaster styling)
- core/lib/shell.ui.store.ts (sidebar collapsed state, no longer needed)
- pages/landing/ (template marketing copy)
- pages/dashboard/ template placeholder (replaced with minimal scaffold)
- components/index.tsx (empty stub)

Kept:
- All auth infrastructure (apiClient, auth.store, ProtectedRoute)
- Auth pages (login/register/settings/admin) — they keep the template's
  visual style for now; not on the operator's hot path
- All my Phase 5 work (ethos tokens, snapshot/ws/stores)

Routes now: / -> dashboard scaffold (no Shell), /login + /register
standalone, /settings + /admin auth-gated standalone (no Shell wrapper).
2026-05-03 02:04:26 -04:00
CarterPerez-dev 7a97aac091 feat(monitor/frontend): zustand stores (globe / ticker / prices / ui / audio + ticker tests) 2026-05-02 05:36:18 -04:00
CarterPerez-dev f7f276df39 feat(monitor/frontend): dashboard WS client with reconnect + init handshake (TDD, 6 tests) 2026-05-02 05:35:18 -04:00
CarterPerez-dev 6aa3efacd6 feat(monitor/frontend): useSnapshot hook (single bootstrap, no refetch) 2026-05-02 05:33:53 -04:00
CarterPerez-dev 4216de3435 feat(monitor/frontend): ethos-aligned base styles (zero rounding, density, tabular nums) 2026-05-02 05:33:12 -04:00
CarterPerez-dev a34a175f14 feat(monitor/frontend): ethos design tokens (colors, type, density) 2026-05-02 05:32:26 -04:00
CarterPerez-dev 94d7e6035e docs(monitor/frontend): mirror ethos hard rules into SCSS comment block 2026-05-02 05:31:32 -04:00
CarterPerez-dev 2ac91da809 docs(monitor): add four world-collector rows to data source matrix 2026-05-02 04:43:03 -04:00
CarterPerez-dev 2f0c019f20 fix(monitor/collectors/gdelt): GDELT 'value' field is float64 not int (caught in live verify) 2026-05-02 04:42:39 -04:00
CarterPerez-dev d662a8c81a fix(monitor/collectors/wikipedia): move revid out of state: namespace (was leaking into snapshot) 2026-05-02 04:41:34 -04:00
CarterPerez-dev 994b510fa4 feat(monitor/main): wire usgs/swpc/wikipedia/gdelt/iss collectors under errgroup 2026-05-02 04:40:46 -04:00
CarterPerez-dev a7f0d5b097 feat(monitor/config): usgs/swpc/wikipedia/gdelt/iss config blocks 2026-05-02 04:39:51 -04:00
CarterPerez-dev ae01113086 feat(monitor/collectors/iss): 10s position poll + daily TLE refresh + propagator-ready collector 2026-05-02 04:39:07 -04:00
CarterPerez-dev 928db0befc feat(monitor/collectors/iss): SGP4 propagator wrapper (Position + LookAngles via go-satellite) 2026-05-02 04:37:25 -04:00
CarterPerez-dev 317fadeeef feat(monitor/collectors/gdelt): 15m collector with per-theme rolling z-score spike emit 2026-05-02 04:35:15 -04:00
CarterPerez-dev 066e2debc2 feat(monitor/collectors/gdelt): per-theme rolling z-score baseline detector 2026-05-02 04:31:10 -04:00
CarterPerez-dev e1d021a99a feat(monitor/collectors/wikipedia): 5m ITN collector with revid-based dedup and per-entry id hash 2026-05-02 04:30:14 -04:00
CarterPerez-dev 85626a0e11 feat(monitor/collectors/wikipedia): goquery-backed ITN parser (revid + entries with article slugs) 2026-05-02 04:28:28 -04:00
CarterPerez-dev 05ca2f529f feat(monitor/collectors/swpc): collector with 1m + 3h cadences and ring-buffered persistence 2026-05-02 04:27:13 -04:00
CarterPerez-dev 608a7937e7 feat(monitor/collectors/swpc): five-endpoint NOAA SWPC client (plasma/mag/kp/xray/alerts) 2026-05-02 04:25:17 -04:00
CarterPerez-dev fc3cfb523b feat(monitor/redisring): sorted-set ring buffer with score-based retention 2026-05-02 04:23:37 -04:00
CarterPerez-dev 2368ed8f18 feat(monitor/collectors/usgs): 1m collector with id-diff and earthquake emit 2026-05-02 04:22:37 -04:00
CarterPerez-dev 1d414ff341 feat(monitor/collectors/usgs): GeoJSON feed client (2.5_day, ms-since-epoch decoder) 2026-05-02 04:21:38 -04:00
CarterPerez-dev 11a68de2b7 feat(monitor/collectors/usgs): earthquakes repository (upsert, known-ids, recent-by-mag) 2026-05-02 04:20:49 -04:00
CarterPerez-dev d4995c1258 fix(monitor/collectors/coinbase): connection-global sequencer; gap is non-fatal log
Two related issues caught in live verification:

1. Coinbase's sequence_num is connection-global (not per-product). Previous
   per-product tracking flagged every cross-product frame as a gap.

2. Even after switching to global tracking, gaps still occur regularly under
   normal operation (Coinbase's exact sequencing pattern across channels is
   not strictly seq+1 frame-to-frame). Treating gaps as fatal kills the
   ReadLoop in seconds, never giving the minute aggregator a chance to cross
   a boundary, so btc_eth_minute stays empty.

Resolution: rewrite Sequencer for a single global counter and downgrade
gap-detection to a non-fatal Warn. ReadLoop continues; (symbol, ts) PK on
btc_eth_ticks is the safety net for any duplicate replay. Includes a
diagnostic log on loop exit (handleConn) for ops visibility.
2026-05-02 04:03:09 -04:00
CarterPerez-dev 989e53c958 fix(monitor/collectors/coinbase): parse Coinbase's Go-style time format in heartbeat current_time
Coinbase Advanced Trade WS leaks Go's time.Time.String() format in
heartbeat events (e.g. "2026-05-02 07:55:50.784... +0000 UTC m=+102632...")
which is not RFC 3339 — UnmarshalJSON into time.Time fails and the entire
envelope errors. Decode current_time as a string and parse it with a
fallback that handles both RFC 3339 and the Go-string variant (with the
m=+monotonic suffix stripped).
2026-05-02 04:02:49 -04:00
CarterPerez-dev 2ccc379255 fix(monitor/collectors/coinbase): exhaustive switches, %w double-wrap, require.ErrorIs (lint) 2026-05-02 03:42:59 -04:00
CarterPerez-dev d352a6bce6 chore(monitor): gofmt const-block alignment in phase-2 collectors 2026-05-02 03:39:51 -04:00
CarterPerez-dev 0027de6c72 docs(monitor): add coinbase row to cyber data source matrix 2026-05-02 03:37:56 -04:00
CarterPerez-dev 99f6e41845 feat(monitor/main): wire coinbase BTC/ETH collector under errgroup 2026-05-02 03:36:26 -04:00
CarterPerez-dev 9c35e14ae8 fix(monitor/collectors/coinbase): use envelope timestamp (per-ticker time field is not in real Coinbase frames) 2026-05-02 03:36:26 -04:00
CarterPerez-dev 0a6f4f1e09 feat(monitor/config): coinbase config block (enabled, url, product_ids, throttle) 2026-05-02 03:32:53 -04:00
CarterPerez-dev 89f1c10249 feat(monitor/collectors/coinbase): collector Run loop ties dial+readloop+aggregator+repo+throttled-emit 2026-05-02 03:32:16 -04:00
CarterPerez-dev f6192e9c04 feat(monitor/collectors/coinbase): per-product minute aggregator with rollover-emit 2026-05-02 03:30:24 -04:00
CarterPerez-dev 23a4e2ded4 feat(monitor/collectors/coinbase): gap-aware ReadLoop (snapshot resets sequencer; gap → ErrSequenceGap) 2026-05-02 03:29:43 -04:00
CarterPerez-dev 3be4c9d815 feat(monitor/collectors/coinbase): reconnect-with-backoff helper (1s init, 60s max, ctx-aware) 2026-05-02 03:28:24 -04:00
CarterPerez-dev 3e398da3fb feat(monitor/collectors/coinbase): per-product sequence-gap detector 2026-05-02 03:27:32 -04:00
CarterPerez-dev 10862314f0 feat(monitor/collectors/coinbase): WS dialer + frame decoder for ticker/heartbeats/snapshot 2026-05-02 03:26:58 -04:00
CarterPerez-dev ae5a83c2f8 feat(monitor/collectors/coinbase): ticks + minute OHLC repo with 1h history reads
Adds shopspring/decimal as a require (folded with the first consuming task per
Plan 2 convention). cenkalti/backoff/v4 is also now consumed transitively;
remains indirect until Task 5 (reconnect) imports it.
2026-05-02 03:24:55 -04:00
CarterPerez-dev 687e13389e chore(monitor): tidy headers, drop unused StateDown, gofmt config defaults 2026-05-02 03:01:48 -04:00
CarterPerez-dev b9c5fa1e13 docs(monitor): add cyber data source matrix to README 2026-05-01 22:53:58 -04:00
CarterPerez-dev c64141bc0b feat(monitor/main): wire dshield/cfradar/cve/kev/ransomware collectors under errgroup 2026-05-01 22:51:50 -04:00
CarterPerez-dev cb8cd902f6 feat(monitor/config): collectors config (toggles, intervals, API key bindings) 2026-05-01 22:50:55 -04:00
CarterPerez-dev 6cc3e681de feat(monitor/enrich/greynoise): on-demand IP lookup with 404→ErrUnknownIP via httpx.StatusError 2026-05-01 22:49:48 -04:00
CarterPerez-dev 894ecc8fe5 feat(monitor/collectors/ransomware): 15m collector with hash-diff and victim emit 2026-05-01 22:48:09 -04:00
CarterPerez-dev 3d71fd8c84 feat(monitor/collectors/ransomware): ransomware.live client with stable triple-hash ID
Hand-crafted fixture matches spec §8.5 schema. Live API endpoint may move; client
takes BaseURL config for swap-in once api.ransomware.live publishes a stable JSON
endpoint (current free tier returns HTML landing page on /recentvictims).
2026-05-01 22:47:19 -04:00
CarterPerez-dev 01a22dc7e4 feat(monitor/collectors/ransomware): victims repository with idempotent insert 2026-05-01 22:45:23 -04:00
CarterPerez-dev bdca22ee50 feat(monitor/collectors/kev): 1h KEV collector with set-diff and chime topic 2026-05-01 22:44:37 -04:00
CarterPerez-dev 7395f1064f feat(monitor/collectors/kev): CISA KEV catalog client 2026-05-01 22:43:11 -04:00
CarterPerez-dev 701bc8e790 feat(monitor/collectors/kev): kev_entries repository with diff lookup 2026-05-01 22:42:18 -04:00
CarterPerez-dev 5ad1ff8e14 feat(monitor/collectors/cve): 2h CVE collector (NVD pull + EPSS enrichment per tick) 2026-05-01 22:41:24 -04:00
CarterPerez-dev 35da1f45a6 feat(monitor/collectors/cve): FIRST EPSS client with 100-id batching 2026-05-01 22:40:15 -04:00
CarterPerez-dev f5e8cf5548 feat(monitor/collectors/cve): NVD CVE 2.0 client (apiKey header, last-modified window, NVDTime) 2026-05-01 22:39:29 -04:00
CarterPerez-dev 2a61953cb9 feat(monitor/collectors/cve): cve_events repository with EPSS-only patch 2026-05-01 22:38:08 -04:00
CarterPerez-dev ee358e38f1 feat(monitor/collectors/cfradar): 5m collector with diff-on-ID emitting outages + hijacks 2026-05-01 22:37:10 -04:00
CarterPerez-dev 428610b663 feat(monitor/collectors/cfradar): radar client (outages + bgp hijacks) with bearer auth 2026-05-01 22:35:26 -04:00
CarterPerez-dev ada39ef6d9 feat(monitor/collectors/cfradar): outages + hijacks repository with diff lookups 2026-05-01 22:34:03 -04:00
CarterPerez-dev fd861d587e feat(monitor/collectors/dshield): hourly tick collector emitting scan_firehose 2026-05-01 22:32:10 -04:00
CarterPerez-dev 808eb1b892 feat(monitor/collectors/dshield): http client + frozen fixtures (topports, topips, dailysummary) 2026-05-01 22:31:01 -04:00
CarterPerez-dev a80664b32a feat(monitor/collectors/dshield): snapshot repository (per-kind upsert + read-latest) 2026-05-01 22:28:42 -04:00
CarterPerez-dev 91d855f4ef feat(monitor/collectors/state): collector_state CRUD with healthy/degraded upserts 2026-05-01 22:26:57 -04:00
CarterPerez-dev c5cc7951a3 feat(monitor/httpx): rate-limited HTTP client with 429/5xx retry honoring Retry-After 2026-05-01 22:25:37 -04:00
CarterPerez-dev 33d15a6ab6 docs(monitor): quickstart README 2026-05-01 21:38:59 -04:00
CarterPerez-dev 4bf29da96f fix(monitor/nginx): strip /api prefix on WS proxy_pass so backend route /v1/ws is reached 2026-05-01 21:38:04 -04:00
CarterPerez-dev 92202b1c16 fix(monitor): pick conflict-free host ports (8432/5432/4432/6432/3432); JWT keygen-on-boot, healthcheck path, baseline migration; ignore frontend/.pnpm-store; pre-commit excludes 2026-05-01 20:13:00 -04:00
CarterPerez-dev ecaacc48fa chore(monitor): replace Makefile with Justfile (project convention) 2026-05-01 14:53:28 -04:00
CarterPerez-dev a67ef20680 feat(monitor/main): wire bus + hub + snapshot + heartbeat under errgroup; expose /v1/snapshot, /v1/ws, /v1/healthz 2026-05-01 13:53:16 -04:00
CarterPerez-dev a7865533cd feat(monitor/backend): ws hub, snapshot store, heartbeat collector with TDD 2026-05-01 07:09:28 -04:00
CarterPerez-dev bc9c2d06eb feat(monitor/backend): events, ratelimit, bus core packages with TDD 2026-05-01 07:03:47 -04:00
CarterPerez-dev 73c56333e6 feat(monitor/db): goose migrations 0001-0003 (alerts, panel data, collector state) 2026-05-01 06:28:51 -04:00
CarterPerez-dev 618aaf43e8 deps(monitor/backend): add coder/websocket, gobreaker/v2, errgroup, prometheus, testcontainers 2026-05-01 06:27:23 -04:00
CarterPerez-dev a627161b33 refactor(monitor/backend): rename module to monitor-the-situation/backend 2026-05-01 06:21:01 -04:00
CarterPerez-dev e46728bd80 feat(monitor/infra): three compose files, Makefile, .env.example 2026-05-01 06:17:28 -04:00
CarterPerez-dev a03c060b9f feat(monitor/nginx): base + dev + prod configs with WS upgrade routing 2026-05-01 06:16:12 -04:00
CarterPerez-dev 470533fd66 feat(monitor): infra scaffold — Dockerfiles, conf/ tree, .gitignore 2026-05-01 06:12:25 -04:00
Carter Perez b826b6caed
Merge pull request #203 from CarterPerez-dev/chore/credential-rotation-enforcer-finish
Chore/credential rotation enforcer finish
2026-04-29 03:42:17 -04:00
CarterPerez-dev 91199476e1 feat: comprehensive audit-driven hardening pass
Closes 36 findings from the project audit. The headline win is correctness:
the daemon now actually does what the README claims, all three audit-log
integrity layers verify, and the rotation control loop terminates.

Critical fixes:
- Orchestrator bumps Credential.last_rotated_at on success and writes a
  sealed credential_versions row, so the policy evaluator stops re-firing
  the same rotation every tick
- Envelope encryption (AES-256-GCM, KEK-wrapped DEK, AAD-bound) is now
  wired into the rotation success path; the credential_versions table is
  no longer dead schema
- cre audit verify checks all three layers (hash chain + HMAC ratchet +
  Merkle batches) instead of only the chain
- BatchSealerScheduler runs every 300s and on shutdown so audit_batches
  actually fills with signed Merkle roots
- Compliance bundle exports the real audit_batches list and covers
  public_key.pem under the manifest's checksum
- cre run/watch hard-fail without CRE_HMAC_KEY_HEX + CRE_KEK_HEX rather
  than silently using zero defaults

Quality fixes:
- HTTP::Client wrapper with connect/read timeouts + bounded retry-with-
  jitter on 408/429/5xx; Telegram error logs redact the bot token
- EventBus dispatch isolates Block subscribers behind select+timeout so
  one stuck subscriber can't pin the bus
- RotationWorker checks rotations.in_flight before dispatching to dedupe
  duplicate schedules; PG enforces it at the DB via partial unique index
- Commit-step failures transition rotations to Inconsistent (a terminal
  state) and raise a critical alert instead of pretending success
- env_file rotator uses per-PID pending paths plus an exclusive flock on
  a sibling .lock file so two daemons can't race on the rename
- Versioned migration runner replaces the soup of CREATE IF NOT EXISTS;
  SQLite gains BEFORE UPDATE / BEFORE DELETE triggers on audit_events
- AuditSubscriber publishes a critical alert + panics by default when
  log writes fail, instead of silently dropping
- Engine.stop drains via an ack channel with a 2s deadline rather than
  the previous magic 50ms sleep
- Policy DSL moved into CRE::Policy::Dsl module (consumers `include` it);
  multiple matching policies raise PolicyConflictError
- Evaluator uses credentials.overdue() per-max_age group instead of
  loading every credential each tick
- AuditRepo gains each_in_range streaming and all_batches enumeration;
  bundle export streams instead of buffering
- New cre verify-bundle <zip> CLI re-runs every check the bundle README
  documents; new cre tui-demo for synthetic event preview

Tests: 179 unit (up from 159), all passing. New rotation_worker_spec,
new SigV4 regression vector, new orchestrator success-path coverage
(envelope write + last_rotated_at bump + Inconsistent state).

Docs: README + learn/00..04 aligned with the post-audit behavior;
removed /snooze (was a stub) and the AwsIamKey/Database CredentialKind
variants that had no rotators. Added required-env-var documentation.
2026-04-29 03:35:10 -04:00
dependabot[bot] 1529ad7cdc
chore(deps): bump vite in /PROJECTS/advanced/encrypted-p2p-chat/frontend
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.4.21 to 7.3.2.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v7.3.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v7.3.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 7.3.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-29 07:15:53 +00:00
Carter Perez 4e6084694c
Merge pull request #200 from CarterPerez-dev/worktree-encrypted-p2p-chat-audit-fixes
Worktree encrypted p2p chat audit fixes
2026-04-29 03:13:21 -04:00
CarterPerez-dev 5adb079e76 docs(encrypted-p2p-chat): refresh learn/ for the audit-fixed architecture
Update all five learn documents to match the post-audit codebase:

- Replace references to the deleted server-side X3DH/Double Ratchet (`backend/app/core/encryption/`) with their client-side TypeScript counterparts (`frontend/src/crypto/{x3dh,double-ratchet}.ts`).
- Drop mentions of the removed `RatchetState` and `SkippedMessageKey` PostgreSQL models, the server-generated key paths, and the deprecated `send_encrypted_message` server-encryption method. Add an explicit "why we deleted server-side crypto" rationale in 02-ARCHITECTURE.
- Update API endpoint listings: drop `/encryption/initialize-keys`, `/encryption/rotate-signed-prekey`, `/encryption/opk-count`; add `/auth/me`, `/auth/logout`. Note that all non-auth routes require the session cookie.
- Reflect the new schema: `IdentityKey`, `SignedPrekey`, and `OneTimePrekey` now hold only public material; `User` carries a 64-byte `webauthn_user_handle`.
- Replace the `/ws?user_id=...` example with cookie-authenticated WebSocket usage.
- Remove all line-number references throughout (per the no-line-numbers feedback) — ~300 stripped via regex pass plus targeted prose rewrites.
- Update WebAuthn copy to reflect `UserVerificationRequirement.REQUIRED`, the per-credential-backup-eligibility flag, and challenge-bytes-keyed authentication storage.
- Refresh the file index and project tree to match the trimmed module layout.
2026-04-29 02:57:17 -04:00
Carter Perez 09542d3a83
Merge pull request #199 from CarterPerez-dev/chore/haskell-reverse-proxy-finish
Chore/haskell reverse proxy finish
2026-04-29 02:55:51 -04:00
CarterPerez-dev 6fd5f3d393 fix(encrypted-p2p-chat): apply audit findings — true E2EE, session auth, spec-correct crypto
This commit applies the actionable plan from docs/plans/2026-04-29-encrypted-p2p-chat-audit-fixes.md.

Backend
- Delete server-side X3DH/Double Ratchet (`app/core/encryption/*`); the server is now an opaque relay for client-encrypted messages.
- Drop `private_key` columns from IdentityKey/SignedPrekey/OneTimePrekey; prekey-bundle endpoint serves only public material.
- Remove RatchetState and SkippedMessageKey models and the deprecated server-encryption code path in message_service.
- Fix the broken `not OneTimePrekey.is_used` SQL filter (`Column.is_(False)` instead of Python `not`); repair the silent OPK-replenish amplification by deleting the server-generated path entirely.
- Stop calling `prekey_service.initialize_user_keys` from registration/login. Server never holds private key material again.
- Add session-cookie auth: opaque token issued in Redis on register/auth complete, `current_user` dependency on every protected route, `/auth/me`, `/auth/logout`. WebSocket authenticates via the same cookie.
- Add room membership checks to WS encrypted_message + typing handlers, GET/DELETE /rooms/{id}, GET /rooms/{id}/messages. broadcast_to_room now filters by participants, not presence.
- Add slowapi rate limits to register/auth/search; per-user sliding-window cap on WS messages.
- WebAuthn: replace username with a 64-byte random `webauthn_user_handle` for `user.id`; fix `backup_eligible` to read `credential_backup_eligible`; switch UserVerificationRequirement to REQUIRED; key authentication challenges by challenge bytes (no more "discoverable" collision).
- Implement GET /rooms/{id} and DELETE /rooms/{id} (cascade delete with ownership check) and fix the N+1 in list_rooms by batching User lookups.
- Remove broken/obsolete tests; add regression tests for the OPK filter bug and session-protected endpoints (401 without cookie).
- Sanitize WebSocket error responses; correlation IDs replace stack-trace leaks.

Frontend
- X3DH: prepend the 32-byte 0xff F prefix to the HKDF input per spec section 2.2.
- Double Ratchet: switch KDF_RK to spec form (HKDF salt=root_key, IKM=dh_output) and use 0x01/0x02 byte tags for KDF_CK.
- Replace base64 helpers in `crypto/primitives.ts` with the URL-safe codec from `lib/base64.ts` (single source of truth, no spread-stack hazard).
- Update auth.service to issue a session via cookies (no more dead token store), call `/auth/me` on app boot, clear crypto state on logout. Delete unused `session.store.ts`.
- Update room.service / Chat page to drop client-supplied user_id arguments; the cookie now identifies the caller.
- Better forward-secrecy UX text for messages predating the device.
- Add Vitest with X3DH and Double Ratchet round-trip + tamper-detection tests.

Tooling
- Add `slowapi>=0.1.9` to backend deps; add `vitest` to frontend devDeps with `pnpm test` script.
- justfile: `test-frontend`, `dev-reset`.
- Adjust `.gitignore` so the Python `lib/` rules no longer hide `frontend/src/lib`.

Migration: existing dev volumes hold private-key columns and KDF-incompatible ratchet states; run `just dev-reset` and re-register before hitting the new code path.
2026-04-29 02:30:10 -04:00
CarterPerez-dev 15f795f10c fix: extract Aenebris.Net.IP to dedupe SockAddr rendering (Finding 10)
- New module src/Aenebris/Net/IP.hs exposes
  sockAddrToIPBytes :: SockAddr -> ByteString.
  Single canonical implementation for IPv4 dotted-decimal, IPv6
  colon-hex (8 groups), and unix-socket "unix:<path>" rendering.

- Aenebris.RateLimit.clientIPKey reduces to
  `sockAddrToIPBytes . remoteHost`. Removes the local v6Bytes,
  the hostAddressToTuple/hostAddress6ToTuple/printf/showHex/
  intercalate imports, and the duplicated implementation.

- Aenebris.DDoS.ConnLimit drops its private copy of the same
  function. ipBytesFromSockAddr is kept as a thin alias for
  test backward-compatibility (= sockAddrToIPBytes), so existing
  callers and the connLimitSpec test continue to compile without
  rename churn.

- aenebris.cabal: expose Aenebris.Net.IP in the library stanza
  (now 30 modules total).

- test/Spec.hs: new netIpSpec asserts IPv4 dotted decimal,
  loopback, unix-socket prefix, and IPv6 colon-separated rendering
  (8 groups → 7 colons). Wires netIpSpec into main right after
  geoSpec.

362 examples passing, 0 failures, 0 GHC warnings.
2026-04-29 02:19:56 -04:00
CarterPerez-dev d9dd59db2a fix: close audit-pass-1 remaining MAJOR + quick MINOR (Findings 14, 19, 20, 28, 29)
- Geo.hs (Finding 14): geoAsnCounts is now bounded by
  defaultGeoAsnCountCap = 200_000. capAsnCounts is called inside
  bumpAsnCounter; on overflow, the entry with the oldest
  awWindowStart is evicted via Data.List.minimumBy + Data.Ord.comparing.
  Closes the unbounded-Map memory growth path. Uses minimumBy and
  comparing imports added to the existing module.

- ML/Middleware.hs, WAF/Engine.hs, Honeypot.hs (Finding 19): em
  dashes (\x2014) in user-facing response bodies and the generated
  robots.txt comment replaced with ASCII hyphens, per the project's
  guardrail-safe terminology rule.

- aenebris.cabal (Finding 20): copyright field updated from
  '2025 Carter Perez' to '2026 AngelaMos' to match the file headers.

- ML/IForest.hs (Finding 28): pathLength now respects a hard
  maxIForestDepth = 64 cutoff. Beyond that depth the function
  returns currentDepth + c(1) (= currentDepth) without further
  recursion, so a pathological tree built by a buggy fitter cannot
  blow the stack. Standard iForest depth is ceil(log2(256)) = 8, so
  the cap leaves >>8 generous headroom.

- ML/Model.hs (Finding 29): validateCategoricalNode now also
  rejects categorical thresholds that are not whole numbers. A
  threshold of 1.5 would silently floor to 1 today; with this
  change it is reported as a clear validation error instead.
  Real LightGBM never writes non-integer cat indices, so this is
  defense-in-depth against malformed exporters.

Build clean, 358 examples passing, 0 failures.
2026-04-29 02:15:14 -04:00
CarterPerez-dev 998b5268e0 fix: close audit-pass-1 MAJOR systemic findings (Findings 7-13)
Brings the older pre-ML modules up to the same standard as the
newer ML pipeline. Closes Findings 7 (file headers), 8 (inline
comments), 9 (NIH reimplementations), 11 (selectWeightedRR partial
function + O(n^2)), 12 (weak LoadBalancer tests), 13 (placeholder
Backend / ConnLimit tests), and 32 (locally-redefined HTTP status
constants).

Module rewrites with new file header + zero inline comments:
- src/Aenebris/Proxy.hs
- src/Aenebris/LoadBalancer.hs
- src/Aenebris/HealthCheck.hs
- src/Aenebris/Middleware/Security.hs
- src/Aenebris/Middleware/Redirect.hs
- src/Aenebris/Config.hs
- src/Aenebris/Backend.hs
- app/Main.hs

Algorithmic + structural fixes:
- LoadBalancer.selectWeightedRR no longer uses partial (!!) and a
  separate find/maximum/index pass; replaced with a single STM fold
  that tags each backend with its current weight and picks via
  Data.List.maximumBy + Data.Ord.comparing. O(n) instead of O(n^2),
  no Maybe fallback.
- Proxy.proxyApp deduplicates the regular-HTTP code path that was
  copy-pasted across the WebSocket and `_` connection-type branches
  (Finding 16). Single forwardRegular helper called from both.
- Proxy.logRequest removed; per-request stdout logging was
  inconsistent with the stderr error pattern used elsewhere and a
  perf hazard at scale (Finding 17). Defer real structured logging
  to Phase 4 per docs/status/MASTER_PLAN_2026.md.
- Backend.recordFailure now increments rbTotalFailures uniformly
  across all states (Finding 22), not just Unhealthy.
- Backend.Show instance now includes weight (Finding 21).
- Connection.hs gains microsPerSecond and httpOkStatusCode named
  constants. tcUpstreamReadSeconds added to TimeoutConfig.
- HealthCheck uses the shared microsPerSecond constant instead of
  inline 1000000 (Finding 18). Compares status against the named
  httpOkStatusCode constant.

NIH reimplementations replaced with stdlib:
- LoadBalancer: filterM, forM_ from Control.Monad; minimumBy,
  maximumBy from Data.List; comparing from Data.Ord.
- Proxy: zipWithM from Control.Monad.
- HealthCheck: zipWithM_ from Control.Monad.
- Middleware.Security: catMaybes from Data.Maybe.
- Config: nub from Data.List (replaces local nubText).
- Fingerprint.JA4H, ML.Features: built-in lookup over
  [(CI ByteString, ByteString)] (CI ByteString already has Eq).
- RateLimit, DDoS.ConnLimit: intercalate ":" from Data.List
  (replaces duplicated joinColons in two files).
- Honeypot: status200, status404 from Network.HTTP.Types (drops
  local mkStatus duplications).
- WAF.Engine: status403 from Network.HTTP.Types.

Test strengthening (test/Spec.hs):
- loadBalancerSpec: round-robin now asserts even distribution
  ([3,3,3] across 3 backends in 9 rounds); weighted RR asserts the
  4-weight backend wins >= 35 of 50 picks; least-connections asserts
  the lower-count backend is selected after manually bumping the
  other to 5 active connections.
- backendSpec: "tolerates repeated failures" replaced with a real
  state-transition test ('Healthy after 1 failure, Healthy after 2,
  Unhealthy after 3'). 'records successes without crashing'
  replaced with assertion that recordSuccess on Healthy resets
  rbConsecutiveFailures to 0.
- connLimitSpec: 'release decrements counter' now reads
  currentCount before and after release to assert the count
  actually went 1 -> 0.

Build clean, 358 examples passing, 0 failures, 0 GHC warnings on
any touched module. Audit findings 14, 15, 18 (residual MAJOR) and
19-31 (MINOR) deferred to subsequent passes.
2026-04-29 02:12:00 -04:00
CarterPerez-dev 2c7e743f57 fix: close audit-pass-1 CRITICAL findings (Findings 1-6)
Closes the six CRITICAL findings from the 2026-04-29 audit:

- Tunnel.hs (Findings 1+2+4):
  - connectToBackend now returns Either ConnectError Socket with
    proper handling of resolution failures, connect timeouts, and
    connect errors. Replaces the partial `error` call.
  - parseUpgradeStatus uses safe pattern matching on BS8.lines
    rather than `head $ BS8.lines` (closes the -Wx-partial warning).
  - receiveUpgradeResponse is bounded at 16 KB and uses a tight
    inner loop with strict accumulator (closes the unbounded-read
    DoS). Adds a 30s timeout via System.Timeout.timeout.
  - Adds attemptConnect with bracketOnError to ensure socket is
    closed on partial-failure paths.
  - File header + remove inline comments.

- TLS.hs (Findings 1+15):
  - loadCredentials replaced with credentialsOrDefault that
    returns empty credentials and logs to stderr on failure
    rather than crashing the SNI handler with `error`.
  - Cipher names updated to non-deprecated forms
    (cipher13_AES_128_GCM_SHA256 etc.) to clear -Wdeprecations.
  - File header + remove inline comments.

- Connection.hs (supporting Finding 3):
  - File header + add microsPerSecond and httpOkStatusCode named
    constants. Add tcUpstreamReadSeconds (default 30) to
    TimeoutConfig.

- Proxy.hs (Findings 1+3):
  - All five `error` calls replaced with hPutStrLn stderr +
    exitFailure (matching Main.hs's pattern).
  - forwardRequest wraps withResponse in System.Timeout.timeout
    sized off Connection.tcUpstreamReadSeconds. Returns 504 when
    upstream does not respond within budget.

- Main.hs (supporting Finding 3):
  - HTTP client Manager now configured with managerResponseTimeout
    set to tcUpstreamReadSeconds * microsPerSecond. Belt-and-
    suspenders with the application-layer timeout in Proxy.hs.

- ML/Loader.hs (Finding 5):
  - Added maxNumLeaves (4096) and maxNumTrees (10000) constants.
  - finalizeTree rejects num_leaves <= 0 and num_leaves > maxNumLeaves
    BEFORE allocating SoA arrays. Closes a crafted-model-file DoS.
  - runTrees threads a tree counter and rejects when the parsed
    count would exceed maxNumTrees.

- WAF/Rule.hs (Finding 6):
  - CompiledRegex changed from newtype to data with an extra
    !ByteString field carrying the original pattern.
  - Eq instance now compares by stored pattern bytes; reflexivity
    holds (x == x is True). Show instance now reveals the pattern
    for debugging.

- test/Spec.hs:
  - +2 WAF tests verifying Eq CompiledRegex reflexivity and
    pattern-distinguishing.
  - +2 ML.Loader tests verifying num_leaves bound rejection
    (above maxNumLeaves and at zero).

358 total examples passing, 0 failures, 0 GHC warnings on touched
modules. Audit findings 7-32 (MAJOR systemic + remaining MAJOR +
MINOR) are deferred to subsequent passes per the audit plan.
2026-04-29 01:59:36 -04:00
Carter Perez 3f9bb07096
Update DEMO.md 2026-04-29 01:56:31 -04:00
Carter Perez 21601d4759
Merge pull request #198 from CarterPerez-dev/chore/haskell-reverse-proxy-finish
feat: add Aenebris.ML.Inference and Aenebris.ML.Calibration
2026-04-29 01:54:50 -04:00
CarterPerez-dev 99d89b3050 docs(README): match repo project pattern (#27 intermediate)
Replace MIT placeholder with the standard intermediate-project README
shape used by sbom-generator, secrets-scanner, etc:
- Block-character ASCII banner (CRE)
- Project #27 / Crystal / AGPLv3 / Postgres badges
- One-line description quote -> learn modules pointer
- What It Does (bulleted feature list)
- Quick Start with both clone-and-build and install.sh paths
- just hint with install one-liner
- Demo tiers + daemon usage code blocks
- Flagship Rotators table
- ASCII architecture diagram
- Three-layer audit log diagram (own section)
- Stack section (language + deps + FFI note + testing)
- Configuration pointer (CONFIGURATION.md)
- Learn modules table
- AGPL 3.0 license footer

Also swapped LICENSE from MIT to AGPL v3 to match the rest of the
intermediate projects.
2026-04-29 01:50:30 -04:00