pnpm latest on Node 22 resolves to pnpm 11 which rejects lockfileVersion
9.0 lockfiles with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH. Pinning to pnpm 10
keeps compatibility with existing lockfiles. .npmrc sets
strict-dep-builds=false so build-script warnings don't hard-fail the
frozen install.
Node.js bumped to 22 in lint workflow (pnpm 11.x requires >=22.13).
Extract goconst-flagged string literals in secrets-scanner to named
constants. Carry along monitor-dashboard docker/package fixes.
useDashboardLifecycle() orchestrates the three pieces that make the
dashboard alive:
1. Audio gesture unlock — unlockOnFirstGesture() on mount so playChime()
is ready when Phase 6 wires alert triggers.
2. Snapshot-then-WS — useSnapshot()'s isSuccess gates createDashboardWS
so we never open the WS before initial state is in TanStack cache
(per spec §10.2 race resolution). After connect+setReady the server
stops buffering and starts streaming deltas.
3. Globe ring eviction — every 5min calls useGlobeEvents.evict(now)
so expired pulse rings drop off.
Topic routing (typed switch, no `as any` — every payload narrows to a
local interface that matches the verified backend shape):
cve_new → useCveStore.push
kev_added → useKevStore.push
ransomware_victim → useRansomwareStore.push
coinbase_price → usePrices.pushTick (ISO ts → number, snake_case
volume_24h → camelCase)
earthquake → globe point + ring pulse (4s TTL)
iss_position → globe point (id='iss-current' so each tick replaces
the previous) + setQueryData(iss_position) so the
ISS panel sees the latest
wiki_itn → ticker push (source 'Wikipedia')
gdelt_spike → ticker push (source 'GDELT', headline includes
z-score and count)
space_weather/internet_outage/bgp_hijack/scan_firehose → setQueryData
to merge into snapshot (panels read from there)
heartbeat → no-op (connection liveness only)
Small bug fix in browserDriver: WebSocket.send() throws when readyState
is CONNECTING. createDashboardWS.setReady() can be called before the
socket opens (we call it immediately after snapshot resolves), so the
inner sock.send is now guarded by a readyState===OPEN check. The
onOpen handler in createDashboardWS resends init when the socket
actually opens — belt-and-suspenders.
This is the load-bearing wiring per the plan. Panels stop rendering
single snapshot rows and start growing as WS events arrive.
Native <dialog> opened from the TopStrip ? icon via useUIStore.aboutOpen.
Ref-driven showModal()/close() in useEffect, onClose synced to the store
so Escape closes both the dialog and the store flag in one round trip
(no listener fight). Three short paragraphs: the meme origin, the data
sources, and the keyboard shortcuts. The kbd-styled spans (mono +
1px --fg-4 outline) call out F and Esc.
No animation on open/close beyond the browser's default <dialog>
showModal transition (which is OS UI, not decoration). No backdrop-click
to close — Escape and the close button are enough; backdrop-click adds
a fragile target===dialog comparison for marginal value. ::backdrop
gets a 60% black tint so it reads as a modal, not a layer cake.
prose line-height 1.4 (looser than the 1.25 table tightness because
prose needs readability — operator-density still applies but Tufte
data-ink isn't violated by paragraph leading).
Plan 5 Task 23 Design QA: no illustration, no confetti, no tooltip
arrows, monochrome border, native ::backdrop only.
Per spec §10.5 — three pure functions sitting on top of the existing
useAudioStore (so React surfaces can subscribe to .unlocked if a
"click anywhere to enable audio" toast is needed later):
loadChime(file) fetch / decode arraybuffer into AudioBuffer
unlockOnFirstGesture() attach one-shot pointerdown listener that
resumes a suspended AudioContext, sets
unlocked=true. Idempotent across re-mounts
via module-level listenerAttached guard
(StrictMode double-mount safe).
playChime() → boolean returns false if the audio context isn't
ready / buffer not loaded / not unlocked,
so callers can fall back to a toast.
Phase 6 calls playChime() on alert match; Phase 5 only wires the
unlockOnFirstGesture() call into the dashboard lifecycle (Task 24)
so the audio is ready when Phase 6 lands.
Four-row layout: Lat/Lon (degree symbol, mono), Alt (km, mono tabular),
Vel (km/h with thousands separator, mono tabular), Next Pass (— stub).
Reads snapshot.iss_position directly (no per-panel store — current
position is the only thing the panel cares about, and the wheretheiss.at
collector pushes a complete state every 10s, so latest-event semantics
match what the panel needs). Real shape verified: latitude/longitude/
altitude/velocity/timestamp/fetched_at — all numeric except fetched_at
ISO string. Stale threshold 30s (3x the 10s poll cadence).
Next Pass row stays "—" muted --fg-3 — Phase 6 wires the SGP4-driven
user-observer-location pass prediction per spec §10.4. No "sign in to
add observer" CTA: operator UI doesn't onboard.
Plan 5 Task 21 Design QA: degree symbol + mono coords, alt/vel mono
tabular, no decorative ISS-tracing widget (the centerpiece globe owns
that), next-pass row visible-but-muted (never hidden).
Three-row layout (label | value | secondary or 9-bar). Kp index renders as
a 9-segment grid bar — segments below kp filled --fg-2, at-or-above outlined
--fg-4. When kp >= 7 (G3+ geomagnetic storm) the entire filled portion goes
amber AND the panel's stale dot fires amber — one of the few legitimate
amber-data uses per ethos. X-ray class M or X also flips amber.
Backend snapshot caveat: the SWPC collector currently emits only
{ts, pushed} as the WS event payload — the actual kp / bz_gsm / speed_kms
/ density / xray_class / xray_flux readings live in Redis ring buffers
(per spec §6.2) but aren't yet projected into snapshot.space_weather.
The TS interface defines all fields optimistically; the panel renders
"—" / empty 9-bar for missing fields. When the backend extends the SWPC
event payload (or a separate snapshot enrichment lands), the panel
auto-fills with no frontend changes.
KP_SEGMENT_KEYS is an explicit string array instead of Array.from(...,
i) to avoid biome's noArrayIndexKey warning while preserving the
positional segment semantics.
Plan 5 Task 20 Design QA: Kp bar uses ONLY --fg-2 / --fg-4 / --amber
(no other colors), values mono tabular, X-ray class letter sans, no
sun glyph / aurora photo / weather-forecast iconography.
Mirrors BTCPanel structure 1:1 with SYMBOL='ETH-USD' and the Coinbase
ethereum price page as the raw link. Per Plan 5 Task 19: don't refactor
into a shared <PricePanel symbol="..." /> until Phase 6 — 50 lines of
duplication beats locking ourselves into a shared contract before BTC
and ETH might diverge in display (gas-fee tile for ETH only, etc.).
Same monochrome discipline: --fg-1 hero price, △/▽ glyph + sign for
direction, no green/red, no animated count-up. 24H change deferred for
the same reason as BTC (60-min history cap can't carry 24h yet).
Reads from the existing usePrices store (latest tick + 60-min history).
Seeds latest from snapshot.coinbase_price on mount, converting the ISO ts
string and snake_case volume_24h to the store's number-ts and camelCase
volume24h shapes (verified against live snapshot).
Hero price uses --type-num-xl mono with tabular-nums — the dashboard's
biggest single number by design. Color stays --fg-1 regardless of
direction. The 1H change row uses △ / ▽ glyph + sign for direction
(not green/red) per the ethos no-branded-color-per-metric rule.
Compute is (last - first) / first * 100 over the 60 minute-bar closes.
Stale indicator goes amber when last tick > 60s ago (Coinbase WS expects
sub-second updates; 60s of silence = real staleness).
24H change is deferred — would need backend snapshot to send 24h of
minute bars; the store caps at 60. Plan said both 1H and 24H but 24H
isn't computable from the available data yet. Showing only 1H is
honest — the missing row would be permanent "—" otherwise.
Sparkline is the 60-minute close ladder. Renders null while history is
shorter than 2 points (Sparkline primitive's own contract). Empty
panel surface is fine — operator UI doesn't onboard.
Plan 5 Task 18 Design QA: hero is the largest single number, price
color stays --fg-1, change row uses glyph + sign (no color), sparkline
is single-stroke currentColor with no fill or axes, no animated
count-up on price change.
RansomwareStore (zustand) accumulates RansomwareVictim items dedup-by
composite key (post_title|group_name|discovered) — the JSON payload has
no id field (the backend computes one via Row.ID() but doesn't include
it in the WS payload), so the composite key is what we have. Capped 200.
6-row tight table — victim mono primary, group sans --fg-2, country
sans --fg-3, ago mono muted right-aligned. table-layout fixed +
ellipsis. Hover --bg-row-hover.
Flash on new arrival: a useRef<Set> tracks seen composite keys; new
items not in the set get added to a flashKeys state for 600ms, which
applies a row-flash keyframe animation that fades from --fg-4
background to transparent. Per ethos motion=meaning rule — the only
panel motion allowed because new ransomware victims literally are
"data changed." Initial seed (when seenKeys is empty) does NOT trigger
a flash — the first batch is "what's there now," not "new arrivals."
No skull / lock / "RANSOMWARE!!" iconography (per Plan 5 Task 17 Design
QA). No colored severity. After flash, row visually identical to others.
Real shape verified against live snapshot.ransomware_victim: snake_case
post_title/group_name/discovered/country/activity/website/description.
TS interface matches.
KevStore (zustand) accumulates KevEntry items dedup-by-cveID, capped at
200. Panel seeds the store from snapshot.kev_added on mount.
6-row table — CVE-ID mono / vendor·product sans (--fg-2) / dateAdded mono
muted (--fg-3). table-layout: fixed + per-column widths (38% / 40% / 22%)
+ ellipsis on overflow. Hover --bg-row-hover (subtle, no transition).
No severity column, no fluff per the plan: every row in this catalog is
a known-exploited vulnerability, the existence of the row carries the
weight. No "ransomware-use" badge — that's the separate Ransomware panel's
job. No CTA — operator clicks the catalog via the raw link.
Real KevEntry shape verified against live snapshot.kev_added: camelCase
keys cveID/vendorProject/product/vulnerabilityName/dateAdded/dueDate/
knownRansomwareCampaignUse/shortDescription/requiredAction. Mirrored
exactly in the TS interface (no transform layer where bugs hide).
Plan 5 Task 16 Design QA: six rows max, CVE-ID mono primary text, vendor
· product body sans secondary, date mono muted, no severity badge.
CveStore (zustand) accumulates CveEvent items dedup-by-CveID, capped at
500. The panel seeds the store from snapshot.cve_new on mount via
useEffect — Task 24's WS routing will extend this with live deltas.
Three KPIs derived in-component from the store (1h/6h/24h counts, mono
tabular --type-num-l). 24h sparkline computed from per-hour bucketing
of items[].Published (one of the few places a chart is justified —
shape-of-CVE-velocity IS the insight per ethos rule on charts). Bottom
table shows the 5 most recent: CVE-ID mono, severity sans plain (NOT a
colored badge — severity stays monochrome per ethos), EPSS percentile
mono right-aligned, relative-time mono muted right-aligned. table-layout:
fixed + per-column widths + ellipsis to keep rows on a single line in
the 320px column.
Real CveEvent shape matches the backend Row struct (verified against
live snapshot): PascalCase keys CveID/Published/LastModified/Severity/
CVSS/EPSSScore/EPSSPercentile/InKEV. Earlier plan-author assumption of
{recent_24h:[]} was wrong — snapshot delivers a single latest event,
the running list builds in the store over time.
Plan 5 Task 15 Design QA: KPIs mono tabular, sparkline pure stroke
(currentColor, no fill/axes), severity NOT a colored badge, EPSS mono %,
ago muted, no "view all" CTA — operator clicks NVD via the raw link.
Plan 5 said "two compact tables side-by-side" but didn't account for the
320px right-column width — tables overflowed and the source-IP table's
Reports/Tgt columns scrolled off-screen. Replaces grid 1fr 1.2fr with a
flex column so each table gets the full panel width: top ports on top,
top sources below.
The panel crashed at runtime with "(ds.topports ?? []).slice is not a
function" because Plan 5 assumed shapes that didn't match what the Phase 2
DShield collector actually emits. Reality:
topports: Record<string, Port> // dict keyed "0".."N", not Array
port field: targetport // not "port"
topips[].source // the IP — not "ip"
topips[].reports // the count — not "records"
topips // has no country/CC field
dailysummary[].date // ISO date string, ascending order
toArray() helper accepts both Record<string,T> and T[] defensively (so
either shape works if the backend changes its mind). Both lists sort by
rank before slicing. The CC column is dropped (data doesn't carry it);
its slot becomes the per-source target count which the data does carry.
Daily summary picks the latest entry by date instead of index 0 (the
order is ascending, so [0] would be the oldest day).
Bigger discovery: most snapshot keys (cve_new / kev_added /
ransomware_victim / coinbase_price / etc.) are LATEST-event payloads,
not the recent-list arrays Plan 5 anticipated. Tasks 15-21 will need
per-topic Zustand stores that accumulate over time, populated by Task 24
from snapshot bootstrap + WS messages. Will discuss with Carter before
committing to that architecture.
Two side-by-side tables fed from snapshot.scan_firehose: Top Ports (port +
24h hits) and Top Sources (IP + CC + hits). Below them a one-line muted
mono summary "Xk records · Yk sources · Z targets — last 24h." Tables, not
bar charts — operator wants to read the port number 22 / 3389, not stare
at a horizontal bar.
Port and IP and hit counts are mono with tabular-nums (digit alignment
matters). Country code stays sans (text label, not a number). Hover row
background var(--bg-row-hover) — subtle, no transform / no transition /
no theatre. Defensive empty-state — when snapshot has no scan_firehose
yet, tables render as empty (per ethos: no "no data" message).
PORT_ROW_LIMIT / SOURCE_ROW_LIMIT / THOUSAND / MILLION constants at module
scope (no inline magic). Renders in the right column as the bottom panel
per spec §11.1 column ordering.
The shared shell every operator panel uses. 28px header strip with
UPPERCASE letter-spaced title (--type-label, --fg-2), optional muted
subtitle (--fg-3), StaleIndicator dot, and an external-link "raw"
icon → upstream source. Body fills remaining space and scrolls
internally. Border-bottom 1px --fg-4 separates panels in the column.
Primitives in shared/:
KPI — value (mono, --type-num-l, tabular-nums, --fg-1) + label
(UPPERCASE, --type-label, --fg-3). No card wrapper, no
rounded corner, no background. The number IS the unit.
Sparkline — d3-scale + d3-array driven inline SVG. Single stroke
in currentColor, no axes / no fill / no animation. Used
only when the shape IS the insight (per ethos). Returns
null with <2 points so empty panels don't ship a flat line
as a placeholder.
StaleIndicator — 6px square dot. --ok green (fresh), --amber
(stale, breached threshold), --fg-4 muted (unknown / no
data yet). Title attribute carries last update UTC time
for hover context.
RawLink not extracted yet — Panel inlines the icon since one raw
link per panel is the rule. Extract when a panel needs more.
Adds d3-scale 4.0.2 + d3-array 3.2.4 + their @types — small d3
modules (not the full umbrella) per spec §10.3 to keep bundle lean.
Plan 5 Task 13 Design QA: panel header 28px, title UPPERCASE
letter-spaced, sparkline pure stroke (no axes / fill), stale dot
6px, no border-radius anywhere, no hover scale.
react-globe.gl 2.37 + three 0.184 added. Globe component is React.memo'd,
absolute-fills the .center grid cell, sizes itself via ResizeObserver on
the wrapper div (the library defaults to window.innerWidth/Height which
would blow past the 1247px cell). pointsMerge: true so hundreds of
points cost one ThreeJS Mesh draw call.
Layer dataset built by useGlobePoints / useGlobeRings hooks in
globeLayers.ts, projecting each store point onto color/altitude/radius
lookup tables keyed on the 6 GlobePointTypes:
earthquake/ransomware/scan = monochrome (--fg-2/--fg-1/--fg-4)
iss = --ok (live position is functional, not decorative)
outage/hijack = --amber (the only globe surfaces with alarm color —
earthquakes and ransomware stay monochrome because an M2.5 quake or
daily victim is not user-actionable)
Rings are simple white pulses (no per-ring color decoration), single
pulse (ringRepeatPeriod=0). Camera panTo() in globeCamera.ts is the
only motion API; the dashboard lifecycle (Task 24) will trigger it on
high-mag quakes etc. Auto-rotate disabled on globe ready — motion is
meaning, a globe that spins for no reason violates the rule.
atmosphereAltitude=0.12, atmosphereColor=#1f2937 per spec §10.4 — a thin
glow that gives the sphere silhouette without becoming "3D space
ambiance." backgroundColor=rgba(0,0,0,0) so the panel cell --bg shows
through. No starfield, no nebulas. No globeImageUrl — keeps the build
self-contained (no CDN dep); we add a local texture later if geographic
context needs to be richer.
Plan 5 Task 12 Design QA: no starfield/nebula/extra glow, monochrome
quake dots, amber confined to outage/hijack, no hover tooltip on globe.
Press F → hide TopStrip + BottomTicker, drop the panel column borders, and
shrink panel columns from 320 to 280 so the globe gets more room. Esc →
exit. Mode persists to localStorage at STORAGE_KEYS.PRESENTATION_MODE so
refresh maintains it (use case: second-monitor wall).
Implementation: each chrome component (TopStrip, BottomTicker) reads
presentationMode from the ui store and early-returns null. Dashboard
applies a .presentation modifier to .root that re-templates the inner
.grid columns and zeroes the aside borders. CSS Modules-scoped — no
:global escape, no clsx dep.
isTypingInForm guards F so it doesn't fire when typing in inputs / textareas
/ contentEditable. Escape always exits — universal "get me out" semantics.
useUIStore.getState() inside the keydown handler avoids stale closure.
Plan 5 Task 11 Design QA: F hides chrome cleanly without animation flash,
globe area expands, panels stay readable, Esc restores, state persists.
32px strip in the bottomticker grid-area that renders the ticker Zustand
store as a horizontally scrolling marquee — one of the few places motion
is allowed because new entries pushing older ones along IS data changing.
Source label uppercase + letter-spaced --fg-3, headline body --fg-2,
relative timestamp mono tabular --fg-3.
Empty state is a bare strip (no "no events" message, no illustration) —
operator UI doesn't onboard. The track only renders when items.length > 0
so the marquee animation isn't running over an empty list.
formatRel uses module-scope SECONDS_PER_MINUTE / SECONDS_PER_HOUR /
MS_PER_SECOND constants (no inline magic numbers). Animation uses kebab
keyframe name (stylelint keyframes-name-pattern).
The only surface in the dashboard that uses amber as a fill color — the
one-alarm-color rule reserves this paint for "look here." 48px strip in
the alertbanner grid-area, dark text (--amber-fg #1a1a1a) on the amber
fill for cross-room legibility, ellipsis on long messages, dismiss button
at right. Returns null when the ui store has no currentAlert so the row
collapses to zero and the center grid sits flush with the top strip.
Auto-dismiss after 30s via setTimeout in useEffect; cleanup on unmount
or alert change. role="alert" for screen readers. focus-visible outline
1px --amber-fg.
Adds --amber-fg to ethos tokens — pairs explicitly with --amber so any
future amber-fill surface uses the same dark text. Also cleans up an
empty `//` line in the ethos comment block (stylelint scss/comment-no-empty).
Thin 36px strip in the topstrip grid-area: left = MONITORING THE SITUATION
label + about-trigger icon, center = HH:MM:SS UTC mono clock with tabular-nums
ticking once per second, right = settings gear (auth) or user (anonymous)
that navigates via ROUTES.{SETTINGS,LOGIN}. About icon dispatches openAbout
on the ui store; the modal itself is Task 23.
Pure monochrome — title --fg-2 letter-spaced uppercase --type-label, clock
--fg-2 mono, icons --fg-3 with --fg-1 hover color shift (no transition,
no scale, no rounded surface). Reads tmux-status-line, not SaaS app header.
focus-visible outline 1px --fg-2 for keyboard a11y. Dashboard.tsx renders
<TopStrip /> as a direct child of .root so it auto-places into the
topstrip named area.
Plan 5 Task 8 Design QA: strip height, label tier, mono tabular clock,
zero rounding, tmux feel — all verified.
Replaces the demolition scaffold with the structural grid that Tasks 8-21
fill in. Root is a 4-row grid (topstrip / alertbanner / main / bottomticker)
with named grid-areas; the main row contains a 3-col sub-grid (left aside /
center section / right aside) at fixed 320px / 1fr / 320px. 100dvh, no body
scroll, ethos custom properties only — no template SCSS variables touched.
Adds --col-panel-width: 320px to the ethos token block (no magic number in
the layout module). index.tsx becomes a one-line re-export of Dashboard so
the lazy-route contract in routers.tsx stays unchanged.
No placeholder components. The empty <aside>/<section> elements are the
column containers themselves; subsequent tasks add real children inside
them rather than swap stub components.
Verified in browser at 1920x1080: rgb(10,10,10) bg, 320/1247/320 col rects,
1px var(--fg-4) rules between columns, body has no scroll. All 5 Plan 5
Task 7 Design QA boxes checked.
Two related issues caught in live verification:
1. Coinbase's sequence_num is connection-global (not per-product). Previous
per-product tracking flagged every cross-product frame as a gap.
2. Even after switching to global tracking, gaps still occur regularly under
normal operation (Coinbase's exact sequencing pattern across channels is
not strictly seq+1 frame-to-frame). Treating gaps as fatal kills the
ReadLoop in seconds, never giving the minute aggregator a chance to cross
a boundary, so btc_eth_minute stays empty.
Resolution: rewrite Sequencer for a single global counter and downgrade
gap-detection to a non-fatal Warn. ReadLoop continues; (symbol, ts) PK on
btc_eth_ticks is the safety net for any duplicate replay. Includes a
diagnostic log on loop exit (handleConn) for ops visibility.
Coinbase Advanced Trade WS leaks Go's time.Time.String() format in
heartbeat events (e.g. "2026-05-02 07:55:50.784... +0000 UTC m=+102632...")
which is not RFC 3339 — UnmarshalJSON into time.Time fails and the entire
envelope errors. Decode current_time as a string and parse it with a
fallback that handles both RFC 3339 and the Go-string variant (with the
m=+monotonic suffix stripped).
Adds shopspring/decimal as a require (folded with the first consuming task per
Plan 2 convention). cenkalti/backoff/v4 is also now consumed transitively;
remains indirect until Task 5 (reconnect) imports it.
Hand-crafted fixture matches spec §8.5 schema. Live API endpoint may move; client
takes BaseURL config for swap-in once api.ransomware.live publishes a stable JSON
endpoint (current free tier returns HTML landing page on /recentvictims).
Closes 36 findings from the project audit. The headline win is correctness:
the daemon now actually does what the README claims, all three audit-log
integrity layers verify, and the rotation control loop terminates.
Critical fixes:
- Orchestrator bumps Credential.last_rotated_at on success and writes a
sealed credential_versions row, so the policy evaluator stops re-firing
the same rotation every tick
- Envelope encryption (AES-256-GCM, KEK-wrapped DEK, AAD-bound) is now
wired into the rotation success path; the credential_versions table is
no longer dead schema
- cre audit verify checks all three layers (hash chain + HMAC ratchet +
Merkle batches) instead of only the chain
- BatchSealerScheduler runs every 300s and on shutdown so audit_batches
actually fills with signed Merkle roots
- Compliance bundle exports the real audit_batches list and covers
public_key.pem under the manifest's checksum
- cre run/watch hard-fail without CRE_HMAC_KEY_HEX + CRE_KEK_HEX rather
than silently using zero defaults
Quality fixes:
- HTTP::Client wrapper with connect/read timeouts + bounded retry-with-
jitter on 408/429/5xx; Telegram error logs redact the bot token
- EventBus dispatch isolates Block subscribers behind select+timeout so
one stuck subscriber can't pin the bus
- RotationWorker checks rotations.in_flight before dispatching to dedupe
duplicate schedules; PG enforces it at the DB via partial unique index
- Commit-step failures transition rotations to Inconsistent (a terminal
state) and raise a critical alert instead of pretending success
- env_file rotator uses per-PID pending paths plus an exclusive flock on
a sibling .lock file so two daemons can't race on the rename
- Versioned migration runner replaces the soup of CREATE IF NOT EXISTS;
SQLite gains BEFORE UPDATE / BEFORE DELETE triggers on audit_events
- AuditSubscriber publishes a critical alert + panics by default when
log writes fail, instead of silently dropping
- Engine.stop drains via an ack channel with a 2s deadline rather than
the previous magic 50ms sleep
- Policy DSL moved into CRE::Policy::Dsl module (consumers `include` it);
multiple matching policies raise PolicyConflictError
- Evaluator uses credentials.overdue() per-max_age group instead of
loading every credential each tick
- AuditRepo gains each_in_range streaming and all_batches enumeration;
bundle export streams instead of buffering
- New cre verify-bundle <zip> CLI re-runs every check the bundle README
documents; new cre tui-demo for synthetic event preview
Tests: 179 unit (up from 159), all passing. New rotation_worker_spec,
new SigV4 regression vector, new orchestrator success-path coverage
(envelope write + last_rotated_at bump + Inconsistent state).
Docs: README + learn/00..04 aligned with the post-audit behavior;
removed /snooze (was a stub) and the AwsIamKey/Database CredentialKind
variants that had no rotators. Added required-env-var documentation.
Update all five learn documents to match the post-audit codebase:
- Replace references to the deleted server-side X3DH/Double Ratchet (`backend/app/core/encryption/`) with their client-side TypeScript counterparts (`frontend/src/crypto/{x3dh,double-ratchet}.ts`).
- Drop mentions of the removed `RatchetState` and `SkippedMessageKey` PostgreSQL models, the server-generated key paths, and the deprecated `send_encrypted_message` server-encryption method. Add an explicit "why we deleted server-side crypto" rationale in 02-ARCHITECTURE.
- Update API endpoint listings: drop `/encryption/initialize-keys`, `/encryption/rotate-signed-prekey`, `/encryption/opk-count`; add `/auth/me`, `/auth/logout`. Note that all non-auth routes require the session cookie.
- Reflect the new schema: `IdentityKey`, `SignedPrekey`, and `OneTimePrekey` now hold only public material; `User` carries a 64-byte `webauthn_user_handle`.
- Replace the `/ws?user_id=...` example with cookie-authenticated WebSocket usage.
- Remove all line-number references throughout (per the no-line-numbers feedback) — ~300 stripped via regex pass plus targeted prose rewrites.
- Update WebAuthn copy to reflect `UserVerificationRequirement.REQUIRED`, the per-credential-backup-eligibility flag, and challenge-bytes-keyed authentication storage.
- Refresh the file index and project tree to match the trimmed module layout.
This commit applies the actionable plan from docs/plans/2026-04-29-encrypted-p2p-chat-audit-fixes.md.
Backend
- Delete server-side X3DH/Double Ratchet (`app/core/encryption/*`); the server is now an opaque relay for client-encrypted messages.
- Drop `private_key` columns from IdentityKey/SignedPrekey/OneTimePrekey; prekey-bundle endpoint serves only public material.
- Remove RatchetState and SkippedMessageKey models and the deprecated server-encryption code path in message_service.
- Fix the broken `not OneTimePrekey.is_used` SQL filter (`Column.is_(False)` instead of Python `not`); repair the silent OPK-replenish amplification by deleting the server-generated path entirely.
- Stop calling `prekey_service.initialize_user_keys` from registration/login. Server never holds private key material again.
- Add session-cookie auth: opaque token issued in Redis on register/auth complete, `current_user` dependency on every protected route, `/auth/me`, `/auth/logout`. WebSocket authenticates via the same cookie.
- Add room membership checks to WS encrypted_message + typing handlers, GET/DELETE /rooms/{id}, GET /rooms/{id}/messages. broadcast_to_room now filters by participants, not presence.
- Add slowapi rate limits to register/auth/search; per-user sliding-window cap on WS messages.
- WebAuthn: replace username with a 64-byte random `webauthn_user_handle` for `user.id`; fix `backup_eligible` to read `credential_backup_eligible`; switch UserVerificationRequirement to REQUIRED; key authentication challenges by challenge bytes (no more "discoverable" collision).
- Implement GET /rooms/{id} and DELETE /rooms/{id} (cascade delete with ownership check) and fix the N+1 in list_rooms by batching User lookups.
- Remove broken/obsolete tests; add regression tests for the OPK filter bug and session-protected endpoints (401 without cookie).
- Sanitize WebSocket error responses; correlation IDs replace stack-trace leaks.
Frontend
- X3DH: prepend the 32-byte 0xff F prefix to the HKDF input per spec section 2.2.
- Double Ratchet: switch KDF_RK to spec form (HKDF salt=root_key, IKM=dh_output) and use 0x01/0x02 byte tags for KDF_CK.
- Replace base64 helpers in `crypto/primitives.ts` with the URL-safe codec from `lib/base64.ts` (single source of truth, no spread-stack hazard).
- Update auth.service to issue a session via cookies (no more dead token store), call `/auth/me` on app boot, clear crypto state on logout. Delete unused `session.store.ts`.
- Update room.service / Chat page to drop client-supplied user_id arguments; the cookie now identifies the caller.
- Better forward-secrecy UX text for messages predating the device.
- Add Vitest with X3DH and Double Ratchet round-trip + tamper-detection tests.
Tooling
- Add `slowapi>=0.1.9` to backend deps; add `vitest` to frontend devDeps with `pnpm test` script.
- justfile: `test-frontend`, `dev-reset`.
- Adjust `.gitignore` so the Python `lib/` rules no longer hide `frontend/src/lib`.
Migration: existing dev volumes hold private-key columns and KDF-incompatible ratchet states; run `just dev-reset` and re-register before hitting the new code path.
- New module src/Aenebris/Net/IP.hs exposes
sockAddrToIPBytes :: SockAddr -> ByteString.
Single canonical implementation for IPv4 dotted-decimal, IPv6
colon-hex (8 groups), and unix-socket "unix:<path>" rendering.
- Aenebris.RateLimit.clientIPKey reduces to
`sockAddrToIPBytes . remoteHost`. Removes the local v6Bytes,
the hostAddressToTuple/hostAddress6ToTuple/printf/showHex/
intercalate imports, and the duplicated implementation.
- Aenebris.DDoS.ConnLimit drops its private copy of the same
function. ipBytesFromSockAddr is kept as a thin alias for
test backward-compatibility (= sockAddrToIPBytes), so existing
callers and the connLimitSpec test continue to compile without
rename churn.
- aenebris.cabal: expose Aenebris.Net.IP in the library stanza
(now 30 modules total).
- test/Spec.hs: new netIpSpec asserts IPv4 dotted decimal,
loopback, unix-socket prefix, and IPv6 colon-separated rendering
(8 groups → 7 colons). Wires netIpSpec into main right after
geoSpec.
362 examples passing, 0 failures, 0 GHC warnings.
- Geo.hs (Finding 14): geoAsnCounts is now bounded by
defaultGeoAsnCountCap = 200_000. capAsnCounts is called inside
bumpAsnCounter; on overflow, the entry with the oldest
awWindowStart is evicted via Data.List.minimumBy + Data.Ord.comparing.
Closes the unbounded-Map memory growth path. Uses minimumBy and
comparing imports added to the existing module.
- ML/Middleware.hs, WAF/Engine.hs, Honeypot.hs (Finding 19): em
dashes (\x2014) in user-facing response bodies and the generated
robots.txt comment replaced with ASCII hyphens, per the project's
guardrail-safe terminology rule.
- aenebris.cabal (Finding 20): copyright field updated from
'2025 Carter Perez' to '2026 AngelaMos' to match the file headers.
- ML/IForest.hs (Finding 28): pathLength now respects a hard
maxIForestDepth = 64 cutoff. Beyond that depth the function
returns currentDepth + c(1) (= currentDepth) without further
recursion, so a pathological tree built by a buggy fitter cannot
blow the stack. Standard iForest depth is ceil(log2(256)) = 8, so
the cap leaves >>8 generous headroom.
- ML/Model.hs (Finding 29): validateCategoricalNode now also
rejects categorical thresholds that are not whole numbers. A
threshold of 1.5 would silently floor to 1 today; with this
change it is reported as a clear validation error instead.
Real LightGBM never writes non-integer cat indices, so this is
defense-in-depth against malformed exporters.
Build clean, 358 examples passing, 0 failures.
Brings the older pre-ML modules up to the same standard as the
newer ML pipeline. Closes Findings 7 (file headers), 8 (inline
comments), 9 (NIH reimplementations), 11 (selectWeightedRR partial
function + O(n^2)), 12 (weak LoadBalancer tests), 13 (placeholder
Backend / ConnLimit tests), and 32 (locally-redefined HTTP status
constants).
Module rewrites with new file header + zero inline comments:
- src/Aenebris/Proxy.hs
- src/Aenebris/LoadBalancer.hs
- src/Aenebris/HealthCheck.hs
- src/Aenebris/Middleware/Security.hs
- src/Aenebris/Middleware/Redirect.hs
- src/Aenebris/Config.hs
- src/Aenebris/Backend.hs
- app/Main.hs
Algorithmic + structural fixes:
- LoadBalancer.selectWeightedRR no longer uses partial (!!) and a
separate find/maximum/index pass; replaced with a single STM fold
that tags each backend with its current weight and picks via
Data.List.maximumBy + Data.Ord.comparing. O(n) instead of O(n^2),
no Maybe fallback.
- Proxy.proxyApp deduplicates the regular-HTTP code path that was
copy-pasted across the WebSocket and `_` connection-type branches
(Finding 16). Single forwardRegular helper called from both.
- Proxy.logRequest removed; per-request stdout logging was
inconsistent with the stderr error pattern used elsewhere and a
perf hazard at scale (Finding 17). Defer real structured logging
to Phase 4 per docs/status/MASTER_PLAN_2026.md.
- Backend.recordFailure now increments rbTotalFailures uniformly
across all states (Finding 22), not just Unhealthy.
- Backend.Show instance now includes weight (Finding 21).
- Connection.hs gains microsPerSecond and httpOkStatusCode named
constants. tcUpstreamReadSeconds added to TimeoutConfig.
- HealthCheck uses the shared microsPerSecond constant instead of
inline 1000000 (Finding 18). Compares status against the named
httpOkStatusCode constant.
NIH reimplementations replaced with stdlib:
- LoadBalancer: filterM, forM_ from Control.Monad; minimumBy,
maximumBy from Data.List; comparing from Data.Ord.
- Proxy: zipWithM from Control.Monad.
- HealthCheck: zipWithM_ from Control.Monad.
- Middleware.Security: catMaybes from Data.Maybe.
- Config: nub from Data.List (replaces local nubText).
- Fingerprint.JA4H, ML.Features: built-in lookup over
[(CI ByteString, ByteString)] (CI ByteString already has Eq).
- RateLimit, DDoS.ConnLimit: intercalate ":" from Data.List
(replaces duplicated joinColons in two files).
- Honeypot: status200, status404 from Network.HTTP.Types (drops
local mkStatus duplications).
- WAF.Engine: status403 from Network.HTTP.Types.
Test strengthening (test/Spec.hs):
- loadBalancerSpec: round-robin now asserts even distribution
([3,3,3] across 3 backends in 9 rounds); weighted RR asserts the
4-weight backend wins >= 35 of 50 picks; least-connections asserts
the lower-count backend is selected after manually bumping the
other to 5 active connections.
- backendSpec: "tolerates repeated failures" replaced with a real
state-transition test ('Healthy after 1 failure, Healthy after 2,
Unhealthy after 3'). 'records successes without crashing'
replaced with assertion that recordSuccess on Healthy resets
rbConsecutiveFailures to 0.
- connLimitSpec: 'release decrements counter' now reads
currentCount before and after release to assert the count
actually went 1 -> 0.
Build clean, 358 examples passing, 0 failures, 0 GHC warnings on
any touched module. Audit findings 14, 15, 18 (residual MAJOR) and
19-31 (MINOR) deferred to subsequent passes.
Closes the six CRITICAL findings from the 2026-04-29 audit:
- Tunnel.hs (Findings 1+2+4):
- connectToBackend now returns Either ConnectError Socket with
proper handling of resolution failures, connect timeouts, and
connect errors. Replaces the partial `error` call.
- parseUpgradeStatus uses safe pattern matching on BS8.lines
rather than `head $ BS8.lines` (closes the -Wx-partial warning).
- receiveUpgradeResponse is bounded at 16 KB and uses a tight
inner loop with strict accumulator (closes the unbounded-read
DoS). Adds a 30s timeout via System.Timeout.timeout.
- Adds attemptConnect with bracketOnError to ensure socket is
closed on partial-failure paths.
- File header + remove inline comments.
- TLS.hs (Findings 1+15):
- loadCredentials replaced with credentialsOrDefault that
returns empty credentials and logs to stderr on failure
rather than crashing the SNI handler with `error`.
- Cipher names updated to non-deprecated forms
(cipher13_AES_128_GCM_SHA256 etc.) to clear -Wdeprecations.
- File header + remove inline comments.
- Connection.hs (supporting Finding 3):
- File header + add microsPerSecond and httpOkStatusCode named
constants. Add tcUpstreamReadSeconds (default 30) to
TimeoutConfig.
- Proxy.hs (Findings 1+3):
- All five `error` calls replaced with hPutStrLn stderr +
exitFailure (matching Main.hs's pattern).
- forwardRequest wraps withResponse in System.Timeout.timeout
sized off Connection.tcUpstreamReadSeconds. Returns 504 when
upstream does not respond within budget.
- Main.hs (supporting Finding 3):
- HTTP client Manager now configured with managerResponseTimeout
set to tcUpstreamReadSeconds * microsPerSecond. Belt-and-
suspenders with the application-layer timeout in Proxy.hs.
- ML/Loader.hs (Finding 5):
- Added maxNumLeaves (4096) and maxNumTrees (10000) constants.
- finalizeTree rejects num_leaves <= 0 and num_leaves > maxNumLeaves
BEFORE allocating SoA arrays. Closes a crafted-model-file DoS.
- runTrees threads a tree counter and rejects when the parsed
count would exceed maxNumTrees.
- WAF/Rule.hs (Finding 6):
- CompiledRegex changed from newtype to data with an extra
!ByteString field carrying the original pattern.
- Eq instance now compares by stored pattern bytes; reflexivity
holds (x == x is True). Show instance now reveals the pattern
for debugging.
- test/Spec.hs:
- +2 WAF tests verifying Eq CompiledRegex reflexivity and
pattern-distinguishing.
- +2 ML.Loader tests verifying num_leaves bound rejection
(above maxNumLeaves and at zero).
358 total examples passing, 0 failures, 0 GHC warnings on touched
modules. Audit findings 7-32 (MAJOR systemic + remaining MAJOR +
MINOR) are deferred to subsequent passes per the audit plan.
- Switch frontend Dockerfiles to pnpm via corepack (project uses pnpm-lock.yaml)
- Pin SurrealDB to v3.0.5 and switch datastore URL from `file:` to `rocksdb:`
- Add idempotent SurrealDB schema bootstrap on connect to prevent NotFoundError on first SELECT against schemaless tables in 3.x
Crystal's compiler warns when an overriding method's parameter name
differs from the parent's, since named-argument callers would pass
the wrong slot. Underscore-prefix was meant as 'unused' but it changed
the parameter name, which Crystal correctly flagged.
Match the parent signature (s : Domain::NewSecret) and use the explicit
'_ = s' idiom inside the body to mark it intentionally unused. Now
'shards build cre' compiles with zero warnings.
Two changes that go together:
1. RotationWorker (src/cre/engine/rotation_worker.cr) is the missing
subscriber that turns RotationScheduled events into actual 4-step
rotations. Without it, the daemon ticked, evaluated, fired
RotationScheduled, and... nothing happened. Now it dispatches via
the orchestrator using a kind -> Rotator dispatch table populated
at boot.
2. cre run now wires:
- All 4 rotators based on env vars (AWS_ACCESS_KEY_ID, VAULT_ADDR/TOKEN,
GITHUB_TOKEN). EnvFile is always available.
- Telegram subscriber + bot if TELEGRAM_TOKEN + chat-id env vars set.
Boot output reports which rotators wired and whether telegram is on.
3. CONFIGURATION.md: full operator setup guide. 13 sections covering
required env vars, Postgres setup with role hardening, key generation,
policy authoring, inventory seeding (per-rotator tag schemas), AWS IAM
policy, Vault token policy.hcl, GitHub admin PAT acquisition, Telegram
bot creation + chat ID discovery, systemd unit with hardening
directives, audit chain verification, KEK rotation procedure, and a
13-item production security checklist.
Walks through the full flow — passkey registration, passkey login,
new conversation lookup, empty thread, first message, end-to-end
chat, and mobile view — matching the format used by the bug-bounty
and SIEM project demos.
cre tui-demo (8 seconds default) renders the live TUI with synthetic
events: 4 fake credentials going through rotation states, a policy
violation, drift detection, an apply-step failure, and a critical alert.
Lets you see what watch mode looks like without seeding any real data.
demo-full: dropped the bogus --config=... flag (cre run never
implemented config-file loading; uses env vars + --db). Now exports
the four service URLs (DATABASE_URL, AWS_ENDPOINT, VAULT_ADDR,
GITHUB_API_BASE) at the user's manually-set ports (6022/45666/18201/7115)
and runs the daemon against the real Postgres. Added demo-full-status
to peek at container health.
Justfile mirrors the haskell-reverse-proxy pattern (grouped recipes:
build, run, demo, test, quality, db, util). 33 recipes covering full
lifecycle - including parameterized recipes (cre rotate ID, cre policy
show NAME) that read more naturally as 'just rotate ...' than as
make targets.
User-edited docker-compose.yml port mappings preserved
(postgres:18 on 6022, localstack on 45666, vault on 18201,
fake-github on 7115).
Nim package names cannot contain hyphens, so `nimble install -y nph`
in the lint workflow aborted on `validatePackageName` before nph could
be installed, causing the Nim lint job to fail with `nph: command not
found`. Renaming the manifest to match the binary name resolves it
and lets contributors run nimble inside the project.
Tier 1 demo was creating an AuditLog instance but never subscribing it
to the bus, so the verification step always reported '0 events'.
Now we explicitly start the AuditSubscriber and the verifier reports
the actual chain length (4 step events + RotationCompleted = 5).
Tier 1 (cre demo): in-memory SQLite + tempfile .env rotator. Walks
through inventory -> before -> 4-step rotation -> after -> audit chain
verify. Narrates every step event live with green/red glyphs. Runs in
under 1 second on commodity hardware, no external deps.
Tier 2 (docker/docker-compose.yml): postgres:16, localstack
(secretsmanager), hashicorp/vault dev mode, plus a fake-GitHub Flask
app that mocks /user, POST/DELETE /user/personal-access-tokens.
config/demo-full.cr.example documents env vars for the full stack.
Also fixed env_file rotator newline preservation (lines(chomp: true)
+ explicit \n join) so the rotated file is properly delimited.
Stack:
- Ansi: stdlib-only escape helpers (move, colorize, strip)
- State: rolling view of active rotations + recent events; trims to N
- Renderer: paints header / status / active / recent panels; works against
any IO so tests assert against IO::Memory
- Tui: ties State + Renderer to event bus; coalesces repaints to a tick
interval (default 200ms) to avoid flicker under burst events
Active rotation rows show step progress as a filled-bar (▰▰▰▱). Recent
events list latest 8 with timestamp + severity glyph (✓ / ⚠ / !). The
renderer's pad/center helpers correctly account for ANSI escape widths
when computing visible column width.
9 specs cover ANSI helpers (color codes, cursor move, strip), State
event-to-row mapping with completion bookkeeping and ring-buffer trim,
and Renderer output assertions across empty + active + populated states.
LogNotifier subscribes (Drop overflow) and emits stdlib Log lines with
structured kwargs (credential_id, rotation_id, severity, etc.) suitable
for downstream ingestion by journald/vector/fluentd.
Telegram is a thin HTTP::Client wrapper for sendMessage and getUpdates.
TelegramSubscriber sends emoji-prefixed alerts on RotationFailed,
DriftDetected, PolicyViolation, AlertRaised; success notifications gated
by the notify_on_success flag. Errors are swallowed so a flaky bot never
blocks the engine.
TelegramBot does long-polling getUpdates and dispatches commands:
- viewer tier: /status /queue /history /alerts /help
- operator tier: viewer + /rotate + /snooze
ACL is by chat_id allowlist (bot token + chat IDs in env vars).
/rotate publishes RotationScheduled to the bus.
11 unit specs cover send_message + error handling + getUpdates parsing,
subscriber dispatch, success-suppression flag, and bot ACL enforcement
(unauthorized / viewer-blocked-from-mutations / operator-can-rotate).
Initial tick fires immediately on start so first policy evaluation
doesn't wait for the full interval on boot. The PolicyEvaluator
subscribes to SchedulerTick events and invokes evaluate_all -> overdue
discovery -> RotationScheduled fan-out. 3 specs verify boot-tick,
periodic cadence, and clean stop().
Github::Client wraps the bearer-auth REST API with required
'X-GitHub-Api-Version: 2022-11-28' header. Implements me, create_pat
(POST /user/personal-access-tokens), delete_pat (DELETE by id).
GithubPatRotator's 4-step contract:
- generate: create_pat returns new id + token value (fresh PAT alongside old)
- apply: no-op (create_pat already active)
- verify: probe-client uses the NEW PAT to GET /user; success means it works
- commit: delete OLD PAT by id (tracked via old_pat_id tag)
- rollback_apply: delete NEW PAT
7 unit specs verify create/delete/me round-trips, full rotation path,
verify-on-401, and rollback deletion.
- New Aenebris.ML.Middleware module: Wai middleware that wires the
Aenebris.ML.Engine into the per-request pipeline. Per request:
1. Run extractFeatures (FeatureContext + Request) -> FeatureVector
2. Convert via featureVectorToVector -> VU.Vector Double
3. runEngine eng fv -> DecisionDetails
4. Route by ddDecision:
DecisionHuman -> pass through to inner Application,
optionally attach X-Aenebris-ML-Decision
and X-Aenebris-ML-Score response headers
DecisionBot -> 403 + text/plain block body
DecisionChallenge -> 403 + text/html challenge page
All response builders are operator-overridable via the
MLMiddlewareConfig record. Optional logging callback fires once
per request for structured logs / metrics integration.
- defaultBotResponse, defaultChallengeResponse: sensible defaults
matching the existing WAF middleware pattern (403 + descriptive
body). decisionResponseHeader / scoreResponseHeader are exported
so operators can build custom responses that still surface the
signal headers.
- 12 tests via Network.Wai.Test.runSession covering: pass-through
on Human, 403 on Bot, 403 on Challenge with text/html body,
signal-header attachment toggle, custom response builder override,
logging callback invocation count.
- aenebris.cabal: expose Aenebris.ML.Middleware in the library stanza.
- test/Spec.hs: add modifyTVar' to the Control.Concurrent.STM import
list for the logging-callback test.
- 354 total examples passing, 0 GHC warnings on the new module.
This closes the core Phase 2.5 ML pipeline. The full sequence
(Features -> Model -> Loader -> Inference -> Calibration -> IForest
-> Engine -> Middleware) is now fully implemented and integration-
tested. A hosted Aenebris instance can now be configured with a
trained LightGBM model + optional calibrator + optional IForest
and will block bots, challenge ambiguous traffic, and pass humans
on every HTTP request.
Rotator base exposes the four lifecycle methods as abstract; subclasses
register at compile time via 'register_as :kind' macro. REGISTRY is
populated as soon as the rotator file is required - drop a new file in
src/cre/rotators/ and the orchestrator can dispatch to it.
EnvFileRotator implements the simplest rotation:
- generate: 32 random bytes -> base64-urlsafe (no padding)
- apply: write to PATH.pending atomically (in-place line replace, 0600)
- verify: parse pending file, confirm key=value present and non-empty
- commit: rename PATH.pending -> PATH (atomic on POSIX)
- rollback_apply: unlink PATH.pending
RotationOrchestrator runs the 4-step contract with full event publishing,
state machine transitions in DB (generating -> applying -> verifying ->
committing -> completed | failed), and rollback_apply on apply/verify
failure. 8 specs cover happy path + raised-during-apply + rollback verification.
- Policy struct: matcher Proc, max_age, warn_at, enforce_action, notify_channels, triggers
- Builder validates all required fields at .build, raises BuilderError with the policy name
- DSL exposes top-level 'policy' method via 'with builder yield' so all
builder methods (description, match, max_age, enforce, notify_via, etc.)
are receiver-less inside the block
- Symbol literals autocast to Action/Channel/Trigger enums on direct calls;
Symbol overload for splat parameters (notify_via :telegram, :email)
- Evaluator subscribes to bus, fires PolicyViolation + RotationScheduled
(rotate_immediately) or AlertRaised (notify_only/quarantine) when overdue
- 15 unit specs cover Policy.matches?/overdue?/in_warning_window?,
Builder validation, DSL syntax with closures, evaluator action dispatch
- New Aenebris.ML.Engine module: pure decision pipeline composing
the previously-built Loader, Inference, Calibration, and IForest
modules into a single per-request scoring path.
Engine record holds (Ensemble, Calibrator, Maybe IForest,
EngineConfig); runEngine takes a feature vector and produces
DecisionDetails (Decision, raw proba, calibrated proba, optional
IForest score). The Decision is one of DecisionHuman, DecisionBot,
or DecisionChallenge.
- Implements the escalation-gate semantics per the 2026 research
correction (over the deprecated 0.8/0.2 weighted blend):
calibrated <= humanThreshold -> DecisionHuman
calibrated >= botThreshold -> DecisionBot
otherwise (ambiguous band):
if IForest configured and ifScore >= escalation threshold
-> DecisionBot (escalation)
else if challenges enabled
-> DecisionChallenge
else
-> DecisionHuman (fallthrough)
Defaults: 0.3 / 0.7 thresholds, 0.6 IForest escalation, challenges
on by default.
- 17 tests covering: defaultEngineConfig values, decision boundaries
with NoCalibrator and no IForest (low/mid/high leaf ensembles),
ambiguous-band escalation via low-vs-high anomaly IForests,
ambiguous-band with challenges disabled (fallthrough to Human or
escalation to Bot), Platt calibrator pulling decisions across
thresholds, and DecisionDetails field correctness.
- aenebris.cabal: expose Aenebris.ML.Engine in the library stanza.
- 342 total examples passing, 0 GHC warnings on the new module.
The Wai middleware wiring (extract features -> runEngine -> route by
Decision) is the next module (ML.Middleware); Engine intentionally
stays pure / independent of HTTP machinery so it can be unit-tested
without request fixtures.
Events form a Crystal class hierarchy so subscribers pattern-match
exhaustively. EventBus delivers to subscribers via Crystal channels with
per-subscriber overflow policy (Block for audit, Drop for best-effort).
AuditSubscriber listens for all rotation/policy/drift/alert events and
serializes them into the hash-chained audit log via AuditLog.append.
Engine wires Persistence + AuditLog + AuditSubscriber + EventBus together
so callers just .start/.stop. 7 unit specs verify boot, fanout to N
subscribers, drop-on-overflow semantics, and end-to-end event -> audit row.
Three layers of integrity:
- Hash chain (SHA-256 chained) catches silent edits
- HMAC ratchet rotates keys every N entries with HKDF-style derivation
and zeroizes old key bytes from memory
- Merkle batch sealing builds tree over content_hashes, signs root
with Ed25519 (FFI to libcrypto EVP_PKEY_ED25519 directly since stdlib
lacks the high-level OpenSSL::PKey wrapper)
AuditLog.append is mutex-guarded for serial writes; verify_chain detects
both single-row tampering and structural inconsistency. BatchSealer's
pack_message format (BE start_seq || BE end_seq || merkle_root) is shared
between signer and verifier so external auditors can validate offline.
- Random with constant-time-equal helper
- KEK loader from env (Tier 1) with InvalidKekError on bad/missing var
- AEAD via direct LibCrypto FFI bindings (Crystal 1.20 stdlib OpenSSL::Cipher
doesn't expose GCM auth_data/auth_tag, so we extend LibCrypto with
EVP_CIPHER_CTX_ctrl + EVP_aes_256_gcm and call them directly)
- Envelope: per-row DEK wrapped by KEK, AAD-bound, algorithm_id reserved
for crypto agility (0x01 = AES-256-GCM today, 0x02/0x03 reserved)
- 17 unit specs including a 100-iteration random property test
- Tag-tampered ciphertext / wrong-AAD / wrong-key all raise Aead::Error
- New Aenebris.ML.IForest module: pure Isolation Forest scorer
implementing the Liu et al. 2008 ICDM formula
anomaly_score = 2^(-E[h(x)] / c(n))
where E[h(x)] is the average path length across iTrees and
c(n) = 2*H(n-1) - 2(n-1)/n (expected unsuccessful BST search depth).
ITree ADT (ITreeLeaf size | ITreeSplit featIdx threshold left right)
with leaf-size c(n) correction added to traversal depth at every
leaf, matching the original paper. Score-only mode (accept pre-
trained forests from Python sklearn or similar); fitting is deferred
to a later phase. Default constants from Liu et al.: 100 trees,
256 subsample size, max depth ceil(log2(256)) = 8.
- 24 tests covering: harmonicNumber edge cases (H(0), H(1), H(2),
H(3), large-n asymptotic), normalizationConstant for n in {0, 1,
2, 256}, single-split and deep-tree path length traversal,
scoreIForest edge cases (empty forest, subsample 0, subsample 1),
shorter-path-equals-higher-score invariant, default constants,
Euler-Mascheroni precision.
- aenebris.cabal: expose Aenebris.ML.IForest in the library stanza.
- test/Spec.hs: import Expectation explicitly from Test.Hspec for the
shouldBeApprox helper.
- 325 total examples passing, 0 GHC warnings on the new module.
The escalation-gate composition with the calibrated GBDT score
(per docs/research/phase-2.5-ml-synthesis.md correction over the
0.8/0.2 folk-wisdom blend) is intentionally NOT in this module --
it belongs in the downstream ML.Engine module that wires Loader +
Inference + Calibration + IForest into a single decision pipeline.
- All 4 repos mirror the SQLite contracts using PG-native types
(UUID, BYTEA, JSONB, BIGSERIAL, TIMESTAMPTZ, SMALLINT)
- audit_no_modify() trigger refuses UPDATE/DELETE/TRUNCATE on audit_events
- pg_advisory_xact_lock for cross-process row locking
- Integration tests verify trigger fires correctly on tamper attempt
- Test cleanup helper temporarily disables trigger to reset between specs
- 4 integration specs all green against PostgreSQL 16
All four repos implement their abstract contracts. Migrations create
tables idempotently. Bytes round-trip through SQLite BLOB columns
correctly. RotatorKind and RotationState enums replace Symbol fields
in record types so they round-trip safely through DB string columns.
8 unit specs verify CRUD, find_by_external, in_flight filtering,
revocation, and audit append/range/latest_hash genesis behavior.
- New Aenebris.ML.Inference module: pure tree-walk over the unified Tree
SoA produced by Aenebris.ML.Loader. Mirrors LightGBM C++ NumericalDecision
exactly: NaN remapped to 0 unless MissingType=NaN; MissingType=Zero/NaN
routes via default-left flag; '<=' predicate (not '<'); categorical
bitmap test using cat_boundaries slice; average_output divisor for RF
mode; sigmoid link for binary logistic objective. kZeroThreshold = 1e-35
matches LightGBM's IsZero exactly. Defensive against NaN/Inf/negative
on categorical paths. 35 tests covering leaf-only, numerical splits,
missing-value semantics, categorical routing, multi-tree sums, sigmoid
saturation, average_output, regression vs binary objectives, and
end-to-end against the disk fixture.
- New Aenebris.ML.Calibration module: pure calibration of binary
classifier outputs. Calibrator ADT (NoCalibrator | PlattCalibrator a b
| IsotonicCalibrator bp). Platt scaling uses Newton's method with
damped line search on Platt-1999 smoothed targets (N+ + 1)/(N+ + 2)
and 1/(N- + 2); numerically stable softplus-based log-loss. Isotonic
regression uses Pool Adjacent Violators on a stack with tie pre-
grouping, sorted (raw, calibrated) breakpoints, linear interpolation,
out-of-range clamping. Sub-2-sample input returns NoCalibrator. 28
tests covering all three calibrator constructors, fit edge cases,
Platt convergence on logistic data, isotonic correction of non-
monotone data, tie handling, all-same-label degenerate cases, and
lookup behavior including clamping and interpolation.
- aenebris.cabal: expose both new modules in the library stanza.
- 301 total examples passing, 0 GHC warnings on either new module.
- shard.yml with db, pg, sqlite3, tourmaline (deps), webmock (dev)
- Makefile with build/test/format/lint/demo targets
- .gitignore, .editorconfig, LICENSE (MIT), README skeleton
- ameba lint deferred until libpcre3-dev is available on this env
- New Aenebris.ML.Loader module: pure ByteString -> Either ParseError Ensemble
parser for LightGBM v4 plain-text models. Header / tree-blocks / trailing-
section state machine; converts split-arrays + leaf-arrays into the unified
Tree SoA via unifyChild + complement; rejects non-v4, multi-class, and
is_linear=1; extracts embedded sigmoid scale and bare average_output flag;
runs validateEnsemble before returning.
- Frozen test fixtures under test/fixtures/ml/ (tiny_lgbm_v4.txt,
stump_lgbm_v4.txt) hand-crafted from the v4 spec annotated example. CI runs
against these with zero Python dependency.
- scripts/regen_lgbm_fixture.py: uv-runnable PEP 723 script that retrains a
tiny binary GBDT and writes to a separate
test/fixtures/ml/regen_sample_v4.txt so the frozen fixtures stay pristine.
- 30 new mlLoaderSpec tests covering happy path, header rejection, sigmoid
extraction, average_output, tree-level rejection, feature_names with '=',
and ParseError reporting. 238 total examples passing, 0 GHC warnings on
Loader.
- Add DEMO.md and screenshots for bug-bounty-platform, hash-cracker,
linux-cis-hardening-auditor, simple-port-scanner, simple-vulnerability-scanner,
systemd-persistence-scanner, base64-tool, caesar-cipher, dns-lookup,
metadata-scrubber-tool, network-traffic-analyzer, siem-dashboard
- Link DEMO.md from project READMEs
- Add .gitignore entries for DEMO-TRACKER and simple-port-scanner build dir
- Restructure haskell-reverse-proxy with DDoS, Fingerprint, ML, RateLimit, WAF,
Geo, and Honeypot modules; drop superseded research docs and old Makefile
- Refresh siem-dashboard dashboard.png and rename alerts.png to alert-detail.png
Replace broken jiro4989/setup-nim-action@v2 with direct choosenim
install. Add ebpf tracer and dlp scanner to lint matrix. Update SBOM
generator badge to Project #24 and add source/docs links in main
README. Bump project count to 24/67.
Add ASCII art, project number badges, justfile tip, and learn module
links. Add missing justfile to dlp-scanner. Update main README with
source code links and bump project count to 23/67.
eBPF-based security monitoring tool with process, file, network,
privilege escalation, and system call tracing via BCC. Includes
threat detection engine, Rich TUI renderer, and learn docs.
Add bomber CLI tool (Go) — scans dependencies across Go/Node/Python
ecosystems, generates SPDX 2.3 and CycloneDX 1.5 SBOMs, and matches
against OSV/NVD vulnerability databases with policy engine for CI/CD.
Add file-level docstrings to ai-threat-detection, firewall-rule-engine,
hash-cracker, linux-cis-hardening-auditor, binary-analysis-tool, and
credential-enumeration.
chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.39.0 to 1.43.0 in /PROJECTS/intermediate/docker-security-audit
Carter Perez
dependabot[bot]