From 51287d3f7d48ccfc231bb3ba1d6606ae1c4301f2 Mon Sep 17 00:00:00 2001 From: CarterPerez-dev Date: Wed, 29 Apr 2026 01:21:17 -0400 Subject: [PATCH] docs: install.sh + learn/ walkthrough series + full README install.sh: cross-platform Crystal version check, libpcre2 dependency hint, shards install + release build, optional install to PATH via INSTALL_TO_PATH=1. learn/: 00-OVERVIEW.md - quick start + three-tier demo path 01-CONCEPTS.md - rotation theory, real breaches (SolarWinds, Codecov, LastPass, Storm-0558, Snowflake/UNC5537), framework controls 02-ARCHITECTURE.md - bus + plugin diagrams, three-layer audit integrity, AEAD envelope encryption layer breakdown 03-IMPLEMENTATION.md - code walkthrough pointing at each key file 04-CHALLENGES.md - 10 ranked extension challenges (PG ALTER USER rotator, Slack notifier, ML-KEM hybrid wrap, OpenTimestamps anchoring, SPIFFE workload identity, JIT credential broker, etc.) README.md: full hero + badges + quick start + subcommand cheatsheet + flagship rotator table + ASCII architecture diagram + project layout + links to learn/ + test suite instructions. --- .../credential-rotation-enforcer/README.md | 158 +++++++++++++++- .../learn/00-OVERVIEW.md | 83 +++++++++ .../learn/01-CONCEPTS.md | 96 ++++++++++ .../learn/02-ARCHITECTURE.md | 174 ++++++++++++++++++ .../learn/03-IMPLEMENTATION.md | 121 ++++++++++++ .../learn/04-CHALLENGES.md | 53 ++++++ .../scripts/install.sh | 73 ++++++++ 7 files changed, 752 insertions(+), 6 deletions(-) create mode 100644 PROJECTS/intermediate/credential-rotation-enforcer/learn/00-OVERVIEW.md create mode 100644 PROJECTS/intermediate/credential-rotation-enforcer/learn/01-CONCEPTS.md create mode 100644 PROJECTS/intermediate/credential-rotation-enforcer/learn/02-ARCHITECTURE.md create mode 100644 PROJECTS/intermediate/credential-rotation-enforcer/learn/03-IMPLEMENTATION.md create mode 100644 PROJECTS/intermediate/credential-rotation-enforcer/learn/04-CHALLENGES.md create mode 100755 PROJECTS/intermediate/credential-rotation-enforcer/scripts/install.sh diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/README.md b/PROJECTS/intermediate/credential-rotation-enforcer/README.md index 9160b09b..af6d4d57 100644 --- a/PROJECTS/intermediate/credential-rotation-enforcer/README.md +++ b/PROJECTS/intermediate/credential-rotation-enforcer/README.md @@ -3,11 +3,157 @@ README.md --> -# Credential Rotation Enforcer (cre) +# Credential Rotation Enforcer (`cre`) -A Crystal-based daemon that tracks and enforces credential rotation -policies across AWS Secrets Manager, HashiCorp Vault, GitHub fine-grained -PATs, and local `.env` files. +> A Crystal daemon that tracks credentials, enforces rotation policies as code, and executes the four-step rotation contract against AWS Secrets Manager, HashiCorp Vault, GitHub fine-grained PATs, and local `.env` files. Single binary. Live TUI. Tamper-evident audit log. Bidirectional Telegram bot. Signed compliance evidence export. -> Full README, asciinema demos, and walkthrough live in `learn/`. -> This README will be expanded in Phase 16 of the build. +[![Crystal](https://img.shields.io/badge/crystal-1.20+-black?logo=crystal)](https://crystal-lang.org) +[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE) +[![Tests](https://img.shields.io/badge/tests-200+-brightgreen)](spec/) + +--- + +## What this is + +A senior+ portfolio implementation of an enterprise credential rotation enforcer, built end-to-end in Crystal. The code is the lesson - every architectural choice (event bus, plugin macros, AEAD envelope, hash-chained audit log, three demo tiers) is intentional and explained in `learn/`. + +This is **not** a wrapper around HashiCorp Vault or AWS Secrets Manager. It is its own coherent enforcer that talks *to* those systems via their HTTP APIs (real SigV4, real bearer auth, real lease tokens). + +## What it does + +1. **Tracks** credentials in an inventory (PostgreSQL or SQLite). +2. **Evaluates** Crystal-DSL policies that compile-time-check for typos, missing fields, and bad enum values. +3. **Rotates** credentials using AWS Secrets Manager's four-step contract (`generate -> apply -> verify -> commit`) with dual-version safety so concurrent consumers never crash mid-rotation. +4. **Records** every event in a tamper-evident audit log: SHA-256 hash chain + ratcheting HMAC-SHA256 + Ed25519-signed Merkle batches. +5. **Encrypts** stored credentials at rest with AES-256-GCM AEAD, per-row DEKs wrapped by a KEK, AAD-bound to credential identity. +6. **Notifies** via structured logs and a bidirectional Telegram bot supporting `/status`, `/rotate `, `/snooze`, `/history`, `/queue`. +7. **Exports** signed compliance evidence bundles mapping audit events to SOC 2 / PCI-DSS / ISO 27001 / HIPAA controls. +8. **Renders** a hand-rolled live TUI (no `crysterm` dependency) showing active rotations, recent events, and KEK version, repainted at most every 200ms. + +## Quick start + +### Tier 1 - Zero-deps demo (under 30 seconds) + +```bash +git clone && cd PROJECTS/intermediate/credential-rotation-enforcer +shards install && shards build cre +./bin/cre demo +``` + +You'll see live narration of an in-memory SQLite + tempfile rotation, with audit-chain verification at the end. + +### Tier 2 - Full mocked stack (under 2 minutes) + +```bash +make demo-full +``` + +Brings up Docker Compose with PostgreSQL 16, LocalStack (AWS Secrets Manager), HashiCorp Vault dev mode, and a fake-GitHub Flask service. CRE talks to all four with real network calls. + +### Tier 3 - Real cloud + +Edit `config/demo-full.cr.example` and set env vars to point at your real AWS account / Vault server / GitHub Apps token. Then `cre run --db=postgres://...`. + +## Subcommands + +| Command | Purpose | +|---|---| +| `cre run` | Headless daemon (production / systemd) | +| `cre watch` | Engine + live TUI in same process | +| `cre check` | One-shot policy evaluation; exit code reflects violations | +| `cre rotate ` | Manually rotate a single credential | +| `cre policy list / show ` | Inspect compiled-in policies | +| `cre export --framework=soc2` | Generate signed compliance evidence ZIP | +| `cre audit verify` | Verify hash chain + HMAC ratchet + Merkle batch signatures | +| `cre demo` | Tier 1 zero-deps demo | +| `cre version` | Print version | + +## The flagship rotators + +| Rotator | What it talks to | Auth | +|---|---|---| +| AWS Secrets Manager | `secretsmanager.us-east-1.amazonaws.com` | SigV4 (rolled from scratch in `src/cre/aws/signer.cr`) | +| HashiCorp Vault | `vault read database/creds/` + lease revoke | `X-Vault-Token` | +| GitHub fine-grained PATs | `POST/DELETE /user/personal-access-tokens` | `Bearer ghp_...` | +| Local `.env` file | atomic temp+rename | n/a | + +Adding a fifth rotator means dropping a single file in `src/cre/rotators/`. The `register_as :kind` macro hooks it into the registry at compile time. + +## Architecture at a glance + +``` + ┌──────────────────────────────────────┐ + │ cre (single Crystal binary) │ + │ │ + ┌────────────┐ │ ┌──────────────────────────────┐ │ + │ Scheduler │─────►│ │ Typed Event Bus │ │ + │ (fiber) │ │ └──┬─────┬─────┬─────┬─────┬───┘ │ + └────────────┘ │ │ │ │ │ │ │ + │ ┌──▼──┐ ┌▼────┐ ┌▼──┐ ┌▼──┐ ┌▼───┐ │ + │ │Rot. │ │Audit│ │TUI│ │Tg │ │Pol.│ │ + │ │Reg. │ │Log │ │ │ │Bot│ │Eval│ │ + │ └──┬──┘ └──┬──┘ └───┘ └───┘ └────┘ │ + │ └──────┴──────────────┐ │ + │ │ │ + │ ┌────────────▼────────┐ │ + │ │ Persistence │ │ + │ │ (PG / SQLite) │ │ + │ └─────────────────────┘ │ + └──────────────────────────────────────┘ +``` + +All components are fibers in one OS process. The bus is in-process - Crystal channels are nanosecond-scale. + +## Project layout + +``` +credential-rotation-enforcer/ +├── shard.yml Crystal manifest (1.20+) +├── Makefile build / test / demo / lint targets +├── policies/ USER policy files (compiled in) +├── src/cre/ +│ ├── cli/ subcommand dispatch + 9 commands +│ ├── tui/ ANSI primitives + 4-panel live monitor +│ ├── engine/ scheduler, event bus, lifecycle, orchestrator +│ ├── events/ typed event hierarchy +│ ├── rotators/ registry + 4 flagship rotators +│ ├── policy/ macro DSL + evaluation engine +│ ├── audit/ hash chain + HMAC ratchet + Merkle + Ed25519 +│ ├── crypto/ AES-256-GCM envelope, KEK/DEK +│ ├── persistence/ PG + SQLite adapters (same interface) +│ ├── notifiers/ structured log + Telegram bidirectional bot +│ ├── compliance/ SOC2/PCI/ISO/HIPAA control mapping + bundle export +│ ├── aws/ SigV4 signer + Secrets Manager client +│ ├── vault/ dynamic-secrets HTTP client +│ ├── github/ fine-grained PAT API client +│ └── demo/ Tier 1 demo +├── docker/ Tier 2 docker-compose stack (PG + LocalStack + Vault + fake-GH) +├── spec/ 200+ unit + integration tests +└── learn/ walkthrough docs (this is the teaching folder) +``` + +## Read the walkthrough + +- [`learn/00-OVERVIEW.md`](learn/00-OVERVIEW.md) - quick start, prerequisites, three-tier demo path +- [`learn/01-CONCEPTS.md`](learn/01-CONCEPTS.md) - rotation theory, real breaches that motivated the design, framework controls +- [`learn/02-ARCHITECTURE.md`](learn/02-ARCHITECTURE.md) - bus + plugin design, persistence layers, crypto stack +- [`learn/03-IMPLEMENTATION.md`](learn/03-IMPLEMENTATION.md) - code-level walkthrough; where to look in the source for each concept +- [`learn/04-CHALLENGES.md`](learn/04-CHALLENGES.md) - 10 extension challenges (beginner -> advanced) + +## Running the test suite + +```bash +crystal spec # all 200+ tests +crystal spec spec/unit # unit only (no DB required) +DATABASE_URL=postgres://cre_test:cre_test@localhost:5432/cre_test \ + crystal spec spec/integration # integration with real PG +make check # format + lint + unit +``` + +## License + +MIT - see [LICENSE](LICENSE). + +## Credits + +Built as part of the [Cybersecurity Projects](https://github.com/CarterPerez-dev/Cybersecurity-Projects) portfolio - 60+ enterprise-grade cybersecurity projects designed as senior-level learning resources. diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/learn/00-OVERVIEW.md b/PROJECTS/intermediate/credential-rotation-enforcer/learn/00-OVERVIEW.md new file mode 100644 index 00000000..9ba6a9cf --- /dev/null +++ b/PROJECTS/intermediate/credential-rotation-enforcer/learn/00-OVERVIEW.md @@ -0,0 +1,83 @@ + + +# Credential Rotation Enforcer - Overview + +A Crystal daemon that **tracks** credentials, **enforces** rotation policies as code, and **executes** the four-step rotation contract against AWS Secrets Manager, HashiCorp Vault, GitHub fine-grained PATs, and local `.env` files. Single binary. Live TUI. Bidirectional Telegram bot. Tamper-evident audit log. Signed compliance evidence export. + +## What This Project Demonstrates + +| Concept | What you'll see in the code | +|---|---| +| Compile-time-checked policy DSL | `policies/*.cr` evaluated by the Crystal compiler; typo'd action symbols, missing fields, or bad credential property references all fail `crystal build` | +| Bus + plugin architecture | Typed events fan out across Crystal channels; subscribers (audit, TUI, Telegram, log) react independently; rotators register at compile time via `register_as :kind` macro | +| 4-step rotation contract | `generate -> apply -> verify -> commit`, dual-version safe (AWSCURRENT / AWSPENDING analog), with rollback on failure between apply and commit | +| Tamper-evident audit log | SHA-256 hash chain + ratcheting HMAC-SHA256 + Ed25519-signed Merkle batches (3 independent layers of integrity) | +| AEAD envelope encryption | AES-256-GCM with per-row DEKs wrapped by a KEK; AAD-bound to `tenant_id || credential_id || version_id`; reserved `algorithm_id` byte for crypto agility | +| Hand-rolled live TUI | ANSI escapes only (no `crysterm` dependency); event-driven repaints coalesced to a tick interval; works against any IO so it's testable | + +## Prerequisites + +- Crystal **1.20.0+** (see `https://crystal-lang.org/install/`) +- For Tier 1 demo: nothing else +- For Tier 2 demo: Docker + Docker Compose +- For Tier 3 (real cloud): AWS account, Vault server, GitHub Apps token + +## Three-Tier Demo Path + +Each tier teaches a different lesson. Run them in order to see the system evolve. + +### Tier 1 - Zero Deps + +``` +$ git clone && cd credential-rotation-enforcer +$ shards install && shards build cre +$ ./bin/cre demo +``` + +What you'll see: +- A temp `.env` file with `API_KEY=oldvalue-aaa` +- A simulated 60-day-old credential triggering policy violation +- Live narration of all 4 rotation steps +- The same `.env` file with a fresh random `API_KEY=...` value +- Audit chain verification confirming integrity + +Runtime: under 1 second. + +### Tier 2 - Docker Compose + +``` +$ make demo-full +``` + +Brings up: PostgreSQL 16, LocalStack (AWS Secrets Manager), HashiCorp Vault dev mode, a fake-GitHub Flask service. CRE connects to all four and rotates one credential through each rotator. Demonstrates real network calls, real auth (SigV4 to LocalStack, token to Vault, bearer to fake-GitHub), real persistence to Postgres, real append-only audit triggers. + +Setup time: ~2 minutes (mostly image pulls). + +### Tier 3 - Real Cloud + +Copy `config/demo-full.cr.example` and edit env vars to point at your real AWS account / Vault server / GitHub. Run `cre run` headless or `cre watch` for the live TUI. + +## Subcommand Cheat Sheet + +| Command | Purpose | +|---|---| +| `cre run` | Headless daemon (production / systemd) | +| `cre watch` | Engine + live TUI in one process | +| `cre check` | One-shot policy eval, exit code by violations (CI-friendly) | +| `cre rotate ` | Manual rotation of a single credential | +| `cre policy list` | List compiled-in policies | +| `cre policy show ` | Inspect one policy in detail | +| `cre export --framework=soc2` | Generate signed compliance evidence ZIP | +| `cre audit verify` | Verify hash chain + HMAC + Merkle batch signatures | +| `cre demo` | Tier 1 zero-deps demo | +| `cre version` | Print version | + +## Where to Read Next + +- **Architecture** -> `02-ARCHITECTURE.md` (event bus, persistence, crypto layers) +- **Concepts** -> `01-CONCEPTS.md` (rotation theory, NIST/SOC2 framework controls, real breaches) +- **Code walkthrough** -> `03-IMPLEMENTATION.md` (key functions, where to make changes) +- **Extension ideas** -> `04-CHALLENGES.md` (add a 5th rotator, ML-KEM hybrid wrap, web UI) diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/learn/01-CONCEPTS.md b/PROJECTS/intermediate/credential-rotation-enforcer/learn/01-CONCEPTS.md new file mode 100644 index 00000000..ca24b177 --- /dev/null +++ b/PROJECTS/intermediate/credential-rotation-enforcer/learn/01-CONCEPTS.md @@ -0,0 +1,96 @@ + + +# Concepts - Credential Rotation, in Practice and in History + +## Why credential rotation is its own discipline + +A credential is a fact in shared state: "this token authorizes this principal." The longer that fact stays in shared state, the more places it can leak from. Compromise has a half-life: every commit, every CI log, every laptop, every backup tape extends the radius. Rotation is the discipline of forcing the fact to expire on a schedule **shorter than the time-to-discover** of the leakage paths you can't see. + +The post-2020 industry consensus has shifted significantly: + +- **NIST SP 800-63B-4** (final July 2025): *prohibits* periodic password rotation for human credentials, BUT continues to require rotation for service accounts, API keys, machine identities. +- **NIST SP 800-57 Pt1 Rev5**: cryptoperiod-driven rotation per key type (DEKs, KEKs, signing keys, MACs). +- **PCI DSS v4.0.1** (mandatory March 2025): the 90-day rule still defaults; the "Customized Approach" off-ramp lets you replace it with continuous monitoring + risk-based rotation. +- **CA/Browser Forum SC-081v3**: TLS certificate lifetimes shrink to 47 days by 2029. + +So the practical answer in 2026 is not "rotate everything every 90 days" - it's **risk-based rotation** for human credentials, **cryptoperiod-driven rotation** for keys, and **JIT / ephemeral / workload-identity** for the highest-risk service-to-service paths. + +## Real breaches that motivated this design + +| Year | Incident | Root credential failure | +|---|---|---| +| 2020 | SolarWinds Sunburst | Service account token reused for ~9 months across attacker dwell | +| 2021 | Codecov bash uploader | One leaked GCS upload key, no rotation, 2 months of supply-chain access | +| 2022 | Heroku/Travis OAuth | OAuth tokens for npm reuse - no rotation, no scope reduction | +| 2023 | Storm-0558 (Microsoft) | MSA signing key generated 2016, never rotated, leaked via crash dump | +| 2023 | Okta support breach | HAR files containing session tokens - no exfil-detection, no token binding | +| 2024 | Snowflake / UNC5537 | Customer credentials leaked years prior, never rotated, no MFA | +| 2024 | Microsoft Midnight Blizzard | Legacy non-prod tenant test creds, never rotated | +| 2025 | tj-actions CVE-2025-30066 | GitHub Actions token theft via supply-chain action | + +The pattern repeats: **time** is the attacker's ally. Each row above had at least one credential that lived too long in shared state. + +## What this enforcer prevents (and doesn't) + +**Prevents:** +- Stale credentials living past their expected lifetime (policy violation -> notify or auto-rotate) +- Silent rotation drift (current-vs-DB-fingerprint detection) +- Forgetting to log a rotation (audit-log automatic via bus subscriber - the orchestrator can't bypass it) +- Audit-log tampering (3-layer integrity: chain + HMAC ratchet + Ed25519 batch signing) +- Unauthorized rotation triggers (Telegram bot ACL, CLI requires explicit credential ID) + +**Does NOT prevent:** +- Compromise of the KEK itself (use AWS KMS / HSM in production) +- Attacker who already exfiltrated a valid credential before rotation (rotate ASAP, but the fact already escaped) +- Insider running `cre rotate ` legitimately but with malicious intent (audit log captures it; doesn't stop it) +- DNS hijack of api.telegram.org (use webhooks with mTLS for hardened deployments) + +## The Four-Step Rotation Contract, Explained + +Borrowed verbatim from AWS Secrets Manager's Rotation Lambda template. The rule is: **between step 2 and step 4, both old AND new credentials are valid.** This is the dual-version safety guarantee. Concurrent consumers using either credential succeed; nobody crashes mid-rotation. + +``` + time --> + step 1 (generate) step 2 (apply) step 3 (verify) step 4 (commit) + | | | | + old: usable old: usable old: usable old: REVOKED + new: pending new: usable new: usable new: current + | | + +-- rollback OK --+ +--+--+ v + upstream pending verify failed? | irreversible + artifact deletable rollback_apply | + revokes new | + v + if verify passes, + proceed to commit +``` + +Step 4 is the only **irreversible** step. By design. If step 4 fails partially (commit succeeded for new but old wasn't actually revoked), the orchestrator marks the rotation `inconsistent` and surfaces an alert - we honestly tell you "this needs a human" rather than silently corrupting state. + +## Audit-log integrity, three layers deep + +| Layer | Mechanism | Defends against | +|---|---|---| +| **Hash chain** | Each row's `content_hash = SHA256(prev_hash || canonical_payload)` | Silent edits - any change breaks the forward chain | +| **HMAC ratchet** | Each row HMAC'd with key K_v; every 1024 rows derive K_{v+1} = HKDF(K_v, "ratchet") and zeroize K_v | An attacker who later gets DB write + current key still cannot rewrite past entries (the old key is gone) | +| **Ed25519 Merkle batches** | Hourly: build Merkle root over `content_hash` leaves; sign root with Ed25519 | Auditor can verify with **only the public key + the batches**, no DB access required | + +Verification is exposed as a CLI command: +``` +$ cre audit verify +✓ chain valid: 14,892 entries +✓ hmac ratchet: 14 generations traversed; all valid +✓ merkle batches: 24 sealed, all signatures verify against pubkey v1 +``` + +## Compliance Framework Coverage + +The export bundle (`cre export --framework=soc2`) maps audit events to specific framework controls. Per `src/cre/compliance/control_mapping.cr`: + +- **SOC 2** - CC6.1 (logical access mgmt), CC6.6 (vulnerability mgmt), CC6.7 (access review), CC4.1 (monitoring), CC7.1 / CC7.2 (incident detection) +- **PCI DSS v4.0.1** - 8.3.9 / 8.6.3 (auth & rotation), 10.5.x (audit log integrity), 3.7.4 (key management) +- **ISO 27001:2022** - A.5.16 / A.5.17 / A.5.18 (identity & access), A.8.5 (secure auth), A.8.15 / A.8.16 (logging & monitoring), A.8.24 (cryptography) +- **HIPAA Security Rule** - 164.308(a)(5)(ii)(D) (password mgmt), 164.312(b) (audit controls) diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/learn/02-ARCHITECTURE.md b/PROJECTS/intermediate/credential-rotation-enforcer/learn/02-ARCHITECTURE.md new file mode 100644 index 00000000..81f68a01 --- /dev/null +++ b/PROJECTS/intermediate/credential-rotation-enforcer/learn/02-ARCHITECTURE.md @@ -0,0 +1,174 @@ + + +# Architecture + +## System overview + +``` +┌─────────────────────────────────────────────────────────────┐ +│ cre (single Crystal binary) │ +│ │ +│ ┌──────────────────────────────────────────────────────┐ │ +│ │ Event Bus │ │ +│ │ (typed Crystal channels) │ │ +│ └────┬─────┬─────┬─────┬─────┬─────┬─────┬────────────┘ │ +│ │ │ │ │ │ │ │ │ +│ ┌────▼─┐ ┌─▼──┐ ┌▼───┐ ┌▼──┐ ┌▼────┐ ┌──▼──┐ ┌──────┐ │ +│ │Sched │ │Rot.│ │Pol.│ │TUI│ │Tele.│ │Audit│ │Notify│ │ +│ │ulers │ │Reg │ │Eval│ │ │ │Bot │ │Log │ │ │ │ +│ └──┬───┘ └─┬──┘ └─┬──┘ └───┘ └─────┘ └──┬──┘ └──────┘ │ +│ │ │ │ │ │ +│ └───────┴──────┴─────────────────────┘ │ +│ │ │ +│ ┌────────▼─────────┐ │ +│ │ Persistence │ ◄── SQLite (Tier 1) │ +│ │ (PG / SQLite) │ ◄── PostgreSQL (T2/T3) │ +│ └──────────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +All long-lived components are **fibers in one OS process**. The bus is in-process - Crystal channels are nanosecond-scale, so the architectural overhead is essentially free. + +## Components + +### Event Bus (`src/cre/engine/event_bus.cr`) + +Fanout dispatch via Crystal channels. Each subscriber gets its own bounded channel and chooses an overflow policy: + +| Subscriber | Overflow | Reason | +|---|---|---| +| `AuditSubscriber` | `Block` | Never drop audit events; compliance requirement | +| `TuiSubscriber` | `Drop` | Stale UI is fine; can't block engine | +| `MetricsSubscriber` | `Drop` | Best-effort metrics | +| `TelegramSubscriber` | `Drop` (large buffer) | Network-flaky anyway | +| `RotationOrchestrator` | `Block` | Must process scheduled rotations | + +The dispatcher is a single fiber reading from the inbox channel and writing to all subscriber channels in order. A slow subscriber configured `Block` causes the dispatcher to block on that subscriber's `send` - which is exactly what you want for audit (better to backpressure than to lose). + +### Rotators (`src/cre/rotators/`) + +The abstract `Rotator` class exposes the four lifecycle methods plus `rollback_apply`. Concrete rotators self-register at compile time: + +```crystal +class AwsSecretsRotator < Rotator + register_as :aws_secretsmgr + ... +end +``` + +Adding a fifth rotator means dropping a single file in `src/cre/rotators/`. The macro hooks it into the registry at compile time. No central wiring to update. + +Rotators receive their cloud client through their constructor (DI). The CLI `run` command wires the right client based on env vars / config. + +### Persistence (`src/cre/persistence/`) + +Two adapters behind one interface: +- `Sqlite::SqlitePersistence` - WAL mode, single connection (avoids `:memory:` per-connection split), application-level mutex for advisory lock simulation. Used for Tier 1 demo. +- `Postgres::PostgresPersistence` - JSONB tags, BIGSERIAL audit, append-only triggers refusing UPDATE/DELETE/TRUNCATE on `audit_events`, `pg_advisory_xact_lock` for cross-process row locking. Used for Tier 2/3. + +Same repo contracts (`CredentialsRepo`, `VersionsRepo`, `RotationsRepo`, `AuditRepo`) for both. The `Persistence` superclass exposes `transaction(&)` and `with_advisory_lock(key, &)` so the rest of the system is backend-agnostic. + +### Crypto layers (`src/cre/crypto/`, `src/cre/audit/`) + +``` ++--------------------+ +| Plaintext | ++--------------------+ + | + | AES-256-GCM(plain, DEK, AAD = tenant||cred||version, nonce 96b) + v ++--------------------+ +| ciphertext + tag | -> credential_versions.ciphertext ++--------------------+ + ++----+ +| DEK| (32 random bytes per row) ++----+ + | + | KEK_v.wrap(DEK) (envelope encryption) + v ++--------------------+ +| wrapped DEK | -> credential_versions.dek_wrapped + kek_version ++--------------------+ + +KEK source (per tier): + Tier 1: env var CRE_KEK_HEX (64-hex chars = 32 bytes) + Tier 2: AWS KMS via LocalStack + Tier 3: real AWS KMS or HSM +``` + +Per-row DEKs collapse the AES-GCM nonce-reuse birthday concern (each row's DEK encrypts exactly one message). AAD-binding prevents ciphertext-swap attacks where an attacker with DB write tries to swap a low-privilege row's ciphertext into a high-privilege row. + +`algorithm_id` is reserved for crypto agility: +- `0x01` = AES-256-GCM (today) +- `0x02` = XChaCha20-Poly1305 (long nonce, simpler) +- `0x03` = ML-KEM hybrid wrap (post-quantum forward secrecy) + +### Audit log integrity stack + +Three layers, increasingly hard to bypass: + +``` ++-------------------------------------------------+ +| Layer 3: Ed25519-signed Merkle batches | +| audit_batches table, hourly seal, signed | +| over (start_seq, end_seq, root) | +| Auditor verifies with public key only. | ++-------------------------------------------------+ + | + | leaves: content_hash[] + v ++-------------------------------------------------+ +| Layer 2: HMAC ratchet | +| K_v signs each row's content_hash; every | +| 1024 rows -> K_{v+1} = HKDF(K_v, "ratchet"); | +| K_v zeroized in memory | ++-------------------------------------------------+ + | + v ++-------------------------------------------------+ +| Layer 1: Hash chain | +| content_hash = SHA256(prev_hash || | +| canonical_payload) | +| rendering single-row tampering visible | ++-------------------------------------------------+ + | + v ++-------------------------------------------------+ +| PostgreSQL audit_events table | +| - INSERT-only role grant | +| - UPDATE/DELETE/TRUNCATE trigger refuses | ++-------------------------------------------------+ +``` + +The PG triggers are not strictly necessary (the chain catches tampering anyway), but they fail loud at write-time which is much friendlier for operators. SQLite tier 1 documents the relaxed guarantee. + +## Concurrency + +| Scope | Bound | Mechanism | +|---|---|---| +| Per-credential | 1 active rotation | PG advisory lock keyed on `credential_id` (or per-process Mutex on SQLite) | +| Per-rotator-kind | configurable | `Channel(Nil).new(capacity: N)` semaphore | +| Global | 20 (default) | Global rotation worker pool | + +Crystal fibers + bounded channels = clean rate limiting without threads or locks. + +## Lifecycle (cre run) + +``` +1. Load config (env + flags) +2. Open persistence (PG or SQLite); migrate! +3. Initialize crypto (load KEK from env or KMS) +4. Load + validate compiled-in policies (REGISTRY) +5. Start EventBus.run (dispatcher fiber) +6. Start subscribers: audit, log, telegram, metrics +7. Start Scheduler (publishes SchedulerTick on tick) +8. Start PolicyEvaluator (subscribes to ticks + credential events) +9. Optionally start TUI (cre watch) +10. Block on signal: SIGTERM/SIGINT triggers graceful drain +``` + +Graceful shutdown: `engine.stop` publishes `ShutdownRequested`, gives subscribers ~50ms to flush, then closes the bus inbox and joins each subscriber fiber. diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/learn/03-IMPLEMENTATION.md b/PROJECTS/intermediate/credential-rotation-enforcer/learn/03-IMPLEMENTATION.md new file mode 100644 index 00000000..31d8c9a5 --- /dev/null +++ b/PROJECTS/intermediate/credential-rotation-enforcer/learn/03-IMPLEMENTATION.md @@ -0,0 +1,121 @@ + + +# Implementation Walkthrough + +This document points you at the most important code paths. Read it with `tree src/` open in another window. + +## The Policy DSL (compile-time validation) + +`src/cre/policy/dsl.cr` defines a top-level `policy` method that uses Crystal's `with builder yield` so every Builder method is callable receiver-less inside the block: + +```crystal +def policy(name : String, &block) + builder = CRE::Policy::Builder.new(name) + with builder yield + CRE::Policy::REGISTRY << builder.build +end +``` + +The Builder methods (in `src/cre/policy/builder.cr`) take typed enum parameters - so `enforce :rotate_immediately` autocasts the symbol to `Action::RotateImmediately` at compile time. Typo'd `:rotate_immediatly` fails the build with `expected Action, got :rotate_immediatly`. The `match {}` block is a real `Proc(Credential, Bool)` so `c.kund` (typo) breaks compilation pointing at the policy file. + +`Builder#build` validates required fields (`matcher`, `max_age`, `enforce_action`) and raises `BuilderError` if any are missing. This makes a policy literally unable to ship to production in a misformed state. + +## The Event Bus (fanout via Crystal channels) + +`src/cre/engine/event_bus.cr` exposes `subscribe(buffer:, overflow:)` returning a `Channel(Event)`. The `run` method spawns a single dispatcher fiber that reads from `@inbox` and forwards to each subscriber's channel. Per-subscriber overflow policy (`Block` or `Drop`) drives whether a slow consumer pauses the dispatcher or quietly loses events. + +The `dispatch` method uses Crystal's `select` to attempt a non-blocking send for `Drop` subscribers and logs a warning when full. `Block` subscribers get `send` directly. + +## The Rotator Plugin Registration + +`src/cre/rotators/rotator.cr` declares an abstract base with a class-level `REGISTRY = {} of Symbol => Rotator.class`. The macro `register_as` populates this at compile time: + +```crystal +abstract class Rotator + REGISTRY = {} of Symbol => Rotator.class + + macro register_as(kind) + ::CRE::Rotators::Rotator::REGISTRY[{{ kind }}] = self + end +end +``` + +When a file like `src/cre/rotators/aws_secrets.cr` is required, the `register_as :aws_secretsmgr` line runs at *compile time* and the class shows up in `Rotator::REGISTRY[:aws_secretsmgr]`. No central list to maintain. + +## The 4-step Orchestrator + +`src/cre/engine/rotation_orchestrator.cr` runs the contract: + +``` +generate -> persist pending version +apply -> rotator-specific (often no-op for cloud rotators where generate already exposed) +verify -> read back, byte-equal check +commit -> promote new -> AWSCURRENT, demote old -> AWSPREVIOUS +``` + +Each step publishes `RotationStepStarted` and either `RotationStepCompleted` or `RotationStepFailed` to the bus. On any exception during apply/verify, `rollback_apply` is invoked and `RotationFailed` is published. `RotationCompleted` is the success terminal. + +The orchestrator never directly calls audit. Audit happens automatically because `AuditSubscriber` is on the bus listening for these exact event types - the orchestrator can't forget to log. + +## SigV4 Signer (the AWS-flavored work) + +`src/cre/aws/signer.cr` implements RFC-style AWS SigV4: + +``` +canonical_request = method + canonical_uri + canonical_query + + canonical_headers + signed_headers + payload_hash +string_to_sign = "AWS4-HMAC-SHA256\n" + amz_date + "\n" + + credential_scope + "\n" + sha256(canonical_request) +signing_key = HMAC chain (kSecret -> kDate -> kRegion -> kService -> kSigning) +signature = HMAC(signing_key, string_to_sign) +``` + +The `Authorization` header is built from `algorithm + Credential=... + SignedHeaders=... + Signature=...`. Includes `X-Amz-Security-Token` when an STS session token is supplied. + +Tested against AWS canonical examples in `spec/unit/aws/signer_spec.cr` for idempotence and format conformance. + +## Audit Log Integrity (three-layer) + +`src/cre/audit/audit_log.cr` orchestrates Layer 1 + 2: +- `latest_hash` from the DB (genesis = 32 zero bytes for an empty log) +- `content_hash = HashChain.next_hash(prev_hash, canonical_payload)` +- `hmac = HmacRatchet#sign(content_hash)`; ratchet rolls every 1024 rows + +`src/cre/audit/batch_sealer.cr` builds Layer 3: +- Walk new audit_events since `last_sealed_seq` +- Build a Merkle tree (`Merkle.root`) over each row's `content_hash` +- Sign `(start_seq, end_seq, root)` with Ed25519 via `Signing::Ed25519Signer` +- Store the signed batch in `audit_batches` + +Crystal's stdlib OpenSSL doesn't expose Ed25519 high-level wrappers, so `src/cre/audit/signing.cr` reaches into LibCrypto via FFI: `EVP_PKEY_new_raw_private_key`, `EVP_DigestSign`, etc. Public-key verification is symmetrical: `Ed25519Verifier#verify(message, signature)`. + +## AEAD Envelope Encryption + +`src/cre/crypto/aead.cr` does AES-256-GCM via LibCrypto FFI (stdlib `OpenSSL::Cipher` doesn't expose `auth_data=` / `auth_tag` for GCM). The envelope (`src/cre/crypto/envelope.cr`) generates a 32-byte DEK per row, encrypts plaintext with AES-256-GCM(plaintext, DEK, AAD), then wraps the DEK with KEK using a separate AEAD (with its own AAD `kek-wrap|v`). Both ciphertexts are `nonce(12) || tag(16) || body`. + +Decrypting requires the KEK to unwrap the DEK, then the DEK + AAD to decrypt the payload. AAD mismatch fails tag verification at the inner layer; KEK version mismatch fails at unwrap. + +## TUI + +`src/cre/tui/state.cr` holds a rolling view of active rotations + recent events. `apply(ev)` is the single entry point that mutates state; pure update logic, easy to test. + +`src/cre/tui/renderer.cr` paints the four panels to any IO. ANSI escapes via `src/cre/tui/ansi.cr` (stdlib only). The renderer's `pad` helper accounts for ANSI escape widths so column alignment is correct under colors. + +`src/cre/tui/tui.cr` ties it together: subscribes to the bus (Drop overflow), spawns a tick fiber + an event fiber, both calling `maybe_render` which throttles to `refresh_interval`. + +## Telegram Bot + +`src/cre/notifiers/telegram.cr` is a thin HTTP::Client wrapper for the Telegram Bot API (no tourmaline dependency for the notification path). + +`src/cre/notifiers/telegram_bot.cr` does long-poll `getUpdates` and dispatches commands. Auth is by chat-ID allowlist; viewer tier vs operator tier separates `/status` from `/rotate`. `/rotate` publishes `RotationScheduled` to the bus, where the orchestrator picks it up. + +## Persistence Layer Shape + +`src/cre/persistence/repos.cr` declares the abstract repos (`CredentialsRepo`, `VersionsRepo`, `RotationsRepo`, `AuditRepo`) and the record types (`RotationRecord`, `AuditEntry`, `AuditBatch`, plus the `RotatorKind` and `RotationState` enums). + +`src/cre/persistence/sqlite/` and `src/cre/persistence/postgres/` mirror each other under the same interface. PG uses `$1, $2` placeholders, BYTEA + JSONB native types, BIGSERIAL audit; SQLite uses `?` placeholders, BLOB + TEXT (with JSON helpers). + +`audit_events` is the most carefully guarded table in the schema - PG triggers refuse `UPDATE`, `DELETE`, `TRUNCATE` and the application role doesn't have those grants either. Two independent locks; both must be subverted to forge history. diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/learn/04-CHALLENGES.md b/PROJECTS/intermediate/credential-rotation-enforcer/learn/04-CHALLENGES.md new file mode 100644 index 00000000..ad62acc6 --- /dev/null +++ b/PROJECTS/intermediate/credential-rotation-enforcer/learn/04-CHALLENGES.md @@ -0,0 +1,53 @@ + + +# Extension Challenges + +Pick one and ship it as a PR. + +## Beginner + +### 1. Add a fifth rotator: PostgreSQL `ALTER USER` +A new file `src/cre/rotators/postgres_user.cr` that rotates a Postgres role's password directly via `ALTER USER ... PASSWORD ...`. Use the existing 4-step contract; verify by opening a fresh connection with the new password. Drop the file in - the macro registers it automatically. Add unit specs in `spec/unit/rotators/postgres_user_spec.cr`. + +### 2. Slack notifier subscriber +Mirror `TelegramSubscriber` against Slack's `chat.postMessage`. Add chat ID allowlist, channel-id parameterization, and message formatting. Single new file in `src/cre/notifiers/`. + +### 3. Add `notify_via :slack` to the Channel enum +Once you have a Slack notifier, add `Slack` to `Channel` in `src/cre/policy/policy.cr`, dispatch on it in the evaluator, and write a policy in `policies/` that uses it. + +## Intermediate + +### 4. Web dashboard via SSE +Add `src/cre/web/` with a Lucky/Kemal HTTP server that subscribes to the bus and pushes events as Server-Sent Events to an HTMX dashboard. Reuse `Tui::State` as the data model - it already has the right shape. The point of the bus + plugin architecture is that this is a *new subscriber*, not a rewrite. + +### 5. ML-KEM hybrid wrap for KEK +Add `algorithm_id = 0x03` to envelope encryption: hybrid Curve25519 + ML-KEM-768 (Kyber) for the DEK wrap. Provides forward secrecy against future quantum-attack on captured ciphertexts. Note: ML-KEM is in OpenSSL 3.x; you may need to update LibCrypto FFI bindings. + +### 6. OpenTimestamps anchoring +Anchor each `audit_batches` Merkle root to the Bitcoin blockchain via OpenTimestamps. Adds a fourth integrity layer: even if the entire DB and signing key are compromised, an offline auditor with a Bitcoin full node can verify when each batch existed. Update `src/cre/audit/batch_sealer.cr` to publish OTS proofs alongside Ed25519 signatures. + +## Advanced + +### 7. SPIFFE/SPIRE workload identity rotator +Add a rotator that *replaces* static credentials with SPIFFE SVIDs (X.509 + JWT). Demonstrates the post-2024 industry shift away from rotation entirely toward attestation-based ephemeral identity. Touch points: new client in `src/cre/spiffe/`, new rotator in `src/cre/rotators/spiffe.cr`, new credential kind in `src/cre/domain/credential.cr`. + +### 8. Crash recovery state machine +Implement the recovery protocol described in the spec: on boot, scan `rotations` table for non-terminal states; for each, decide whether to rollback, retry, or mark `inconsistent`. The current orchestrator publishes events but doesn't recover from a daemon crash mid-rotation. Add `src/cre/engine/recovery.cr` with explicit state-machine semantics for each `(rotator_kind, last_step)` pair. + +### 9. Multi-tenant support +Wire `tenant_id` through the schema (already reserved as a column placeholder), the AAD construction, the policy matchers (allow `c.tenant == "tenant-x"`), and the API surface. Postgres row-level security policies enforce isolation at the DB level. The biggest design decision: per-tenant KEKs vs shared KEK with per-tenant DEKs. + +### 10. JIT credential broker +Replace the rotation contract entirely for some credential types: instead of rotating, *issue* a fresh ephemeral credential on each access (5-15 minute TTL). Requires a new `Broker` abstraction alongside `Rotator`, integration with AWS STS / GCP service account impersonation / Vault dynamic, and consumer-side token refresh logic in the example apps. + +## Evaluation rubric (for self-review) + +A great extension PR: +- Has unit tests in `spec/unit//` mirroring the source layout +- Has a focused commit message explaining the *why*, not just the *what* +- Doesn't break existing tests (`crystal spec spec/` should be green) +- Adds 1-3 file-level changes; if it touches more than 5 files, the abstraction is probably wrong +- Surfaces failure modes honestly (e.g., the JIT broker should clearly document the consumer-side complexity it shifts) diff --git a/PROJECTS/intermediate/credential-rotation-enforcer/scripts/install.sh b/PROJECTS/intermediate/credential-rotation-enforcer/scripts/install.sh new file mode 100755 index 00000000..74c74822 --- /dev/null +++ b/PROJECTS/intermediate/credential-rotation-enforcer/scripts/install.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# ©AngelaMos | 2026 +# install.sh + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +INSTALL_PREFIX="${INSTALL_PREFIX:-/usr/local}" + +bold() { printf "\033[1m%s\033[0m\n" "$*"; } +green() { printf "\033[32m%s\033[0m\n" "$*"; } +yellow() { printf "\033[33m%s\033[0m\n" "$*"; } +red() { printf "\033[31m%s\033[0m\n" "$*"; } + +bold "Credential Rotation Enforcer - install.sh" + +# 1) Crystal +if ! command -v crystal >/dev/null 2>&1; then + yellow "Crystal not found. Install instructions: https://crystal-lang.org/install/" + yellow "Or, on Linux:" + yellow " curl -fsSL https://crystal-lang.org/install.sh | sudo bash" + yellow "Or, on macOS:" + yellow " brew install crystal" + exit 1 +fi +crystal_version=$(crystal --version | head -1 | awk '{print $2}') +green "Crystal $crystal_version found" + +# 2) System deps (libpcre2 for regex, libssl for crypto) +case "$(uname -s)" in + Linux) + if ! ldconfig -p 2>/dev/null | grep -q libpcre2-8; then + yellow "libpcre2 not detected. On Debian/Ubuntu: sudo apt-get install -y libpcre2-dev" + yellow "On Alpine: apk add pcre2-dev" + fi + ;; + Darwin) + : # macOS bundles what's needed + ;; +esac + +cd "$PROJECT_ROOT" + +# 3) Shards +bold "Resolving shard dependencies..." +shards install +green "shards installed" + +# 4) Build +bold "Building cre (release mode)..." +shards build cre --release +green "cre binary built at bin/cre ($(stat -c %s bin/cre 2>/dev/null || stat -f %z bin/cre) bytes)" + +# 5) Optional install to PATH +if [ "${INSTALL_TO_PATH:-0}" = "1" ]; then + bold "Installing to ${INSTALL_PREFIX}/bin/cre" + if [ -w "${INSTALL_PREFIX}/bin" ]; then + cp bin/cre "${INSTALL_PREFIX}/bin/cre" + else + sudo cp bin/cre "${INSTALL_PREFIX}/bin/cre" + fi + green "Installed: $(which cre)" +fi + +bold "" +bold "Try the zero-deps demo:" +echo " ./bin/cre demo" +bold "" +bold "Or start the daemon:" +echo " ./bin/cre run --db=sqlite:cre.db" +bold "" +bold "See learn/00-OVERVIEW.md for the full walkthrough."