Add a --report flag to the pcap command that prints one forensic summary of
a whole capture (endpoint inventory, fingerprint distribution, intel verdicts,
detection alerts, and a coverage section with the TLS miss rate and throughput)
instead of the per-event stream, folding in intel and detection automatically
when a seeded database is present. Add a tls_handshakes_fingerprinted counter
and Counters::tls_miss_rate so truncated and multi-segment ClientHellos the
capture clipped are reported rather than mistaken for absence. Add criterion
benches over the vendored captures and the parse/hash hot path. Plus frontend
landing and asset polish.
materializeDefaults now sets CKA_ENCRYPT/DECRYPT/SIGN/VERIFY/SIGN_RECOVER/VERIFY_RECOVER/WRAP/UNWRAP/DERIVE (plus ALWAYS_AUTHENTICATE for private keys) to CK_FALSE when absent, per CKO_PUBLIC_KEY/PRIVATE_KEY/SECRET_KEY; smoke sets the RSA wrap and recover usage flags.
Passive TLS client and server fingerprinting from packet captures. Two-crate
workspace: tlsfp-core (parsing and fingerprint logic, no I/O, unsafe forbidden)
and tlsfp (clap CLI). Edition 2024, MSRV 1.85.
Core (M0-M4): a hand-rolled bounds-checked TLS parser for ClientHello,
ServerHello, and the TLS 1.2 Certificate chain, plus the full fingerprint
family computed byte-exact against published vectors: JA3/JA3S (MD5), JA4/JA4_r,
JA4S, JA4H, JA4X, and JA4T. GREASE is stripped, wire order and unknown cipher
and extension values are preserved.
Pipeline (M5): a pcap and pcapng file reader behind a PacketSource trait,
etherparse L2-L4 decode (Ethernet, raw IP, BSD loopback, Linux SLL and SLL2,
VLAN and QinQ), and a hand-rolled per-flow TCP reassembler with wrapping serial
arithmetic, out-of-order parking, first-write-wins overlap resolution, and
bounded memory. The tlsfp pcap subcommand fingerprints every handshake in a
capture, as readable lines or NDJSON.
Verified three ways: 94 tests including unit KATs, nine real-pcap full-pipeline
KATs against the FoxIO sample corpus, and property tests that reassemble
arbitrarily segmented and reordered streams. Builds clean on stable and MSRV
1.85; clippy -D warnings and rustfmt clean.
Test captures under testdata/pcap are vendored unmodified from the FoxIO ja4
repository with per-file SHA-256 provenance.
build_module printed (ReleaseSafe) but ran plain 'zig build', which is Debug here (preferred_optimize_mode only ships ReleaseSafe with --release). It left a 20M Debug .so labeled as the shipped build, and 'zig build smoke' would reinstall Debug over any ReleaseSafe artifact via its install-step dependency.
Run build + test + smoke with --release=safe so the script produces and validates the real 5.9M shipped artifact end to end. Kept check-only (no dependency auto-install): missing deps still hard-fail/warn with the exact apt command.
Verified: ./install.sh exits 0 on a clean tree, final zig-out/lib/libhsm.so is 5.9M ReleaseSafe, tests + smoke pass, pkcs11-tool loads it.
learn/: add 00-OVERVIEW, 01-CONCEPTS, 02-ARCHITECTURE, 03-IMPLEMENTATION, 04-CHALLENGES, and MECHANICS (the byte-by-byte crypto deep-dive). CONFORMANCE shipped in M11. Real-breach grounding, function/file code refs (no line numbers), no AI voice.
Project README: flip M0-stale to real M0-M11 (21 mechanisms, std.Io.randomSecure, roadmap done through M12, learn/ link table).
Root README: graduate HSM to a completed project (Full Source Code 32->33, C badge -> Zig, title + footer now link to source and learn/).
Verified docs-only: zig build test and zig build smoke both exit 0, smoke mechanisms=21, objdump -T shows only C_GetFunctionList. No source changed.
go build ./cmd/healthcheck (run for verification, not via the bin/ recipe)
drops the binary in the working dir. Untracked previously; landed in the
last commit by accident. Gitignore now lists each cmd binary explicitly so
the same mistake can't recur for canary, healthcheck, or the two pdf/docx
template builders.
Three things that all blocked tunnel-start:
1. justfile: dropped set dotenv-load / set export — they pulled
.env.development into every recipe shell, where empty values
beat docker compose --env-file .env per shell-env precedence
(POSTGRES_PASSWORD was the visible casualty). Dev recipes now
pass --env-file .env.development explicitly so they're not
affected by the change.
2. canary image is distroless/static, so the wget-based healthcheck
had no binary to run and every check failed → nginx/tunnel never
started. Added cmd/healthcheck (tiny Go HTTP probe of /healthz),
built alongside canary, swapped compose healthcheck to /healthcheck.
3. Added tunnel-build and tunnel-rebuild recipes so the next person
doesn't have to remember which compose files the tunnel overlay
needs.
Operator request: give the webbug some personality. Trigger response is now
a small JPEG embedded into the binary via //go:embed, served from
backend/internal/token/generators/webbug/asset/pixel.jpg. Default is a
545-byte 32x32 yellow placeholder I generated with imagemagick — replace
the file with any image (same path, same filename, doesn't matter what
size or content) and the next backend rebuild picks it up. No other code
needs to change.
Scope: ONLY webbug. The docx, pdf, and envfile generators keep their own
Trigger functions that still call pixel.Clone() for the canonical
invisible 1x1 GIF — those types embed the trigger URL inside their
respective artifact bytes (a Word footer, a PDF /AA action, a fake .env
INTERNAL_METRICS_ENDPOINT line) and need the response to stay invisible
so the attacker doesn't notice they pinged us. Stealth preserved where
it matters.
The pixel package itself is untouched (still has the 43-byte GIF
constant); only webbug stops importing it.
Tradeoff baked in: a webbug whose response is a visible image is
*technically* less stealthy than the canonical 1x1 — the attacker can
see "huh there's a small image here." But the visible mechanism is real
too (think Mailchimp / SendGrid email-open tracking, which use sized
"social proof" images, not just 1x1s), so this is a legitimate variant.
For an operator who wants strict 1x1-invisible behavior in prod, the
revert is a 3-line change in webbug/generator.go (swap embed for
pixel.Clone() again).
Tests updated: webbug/generator_test.go no longer asserts gifByteLength
or pixel.Clone() equality; it now asserts the body starts with the JPEG
SOI marker (0xff 0xd8) + ContentType is image/jpeg. The
"independent-copy-per-call" invariant test still passes (bytes.Clone in
the Trigger ensures each Body slice is a fresh allocation).
Cache headers unchanged (no-store / no-cache / must-revalidate) so every
fetch hits the trigger handler.
Upstream 8d2599c0 added a mysqlEnabled parameter to token.NewHandler and
took a long line over the limit in handler_test. Pass the parameter from
the e2e stack (true: integration test exercises mysql token creation) and
reflow the long handler_test return so golines is happy.
CarterPerez-dev
dependabot[bot]