Cybersecurity-Projects/PROJECTS/advanced/hsm-emulator/learn/04-CHALLENGES.md

Extension Challenges

You have a working software HSM. Now make it yours. These challenges are ordered by difficulty, and each one names the real files and functions you will touch so you are not hunting. Start easy to learn the codebase's shape, then go deeper.

The golden rule of this project applies to every challenge: nothing is done until an external tool exercises it. Add a unit test against a published vector, then prove it through pkcs11-tool or the smoke harness.

Easy Challenges

Challenge 1: Add SHA-224 as a digest mechanism

What to build: Support CKM_SHA224 alongside the existing SHA-256/384/512 digests.

Why it's useful: SHA-224 is still required by some compliance profiles, and adding it teaches you the exact path a new mechanism takes through the layers.

What you'll learn:

• The four-step "add a mechanism" path described in 02-ARCHITECTURE.md
• How the ABI test cross-checks a new constant for free

Hints:

• Add CKM_SHA224 to ck.zig. The "every hand-coded constant equals its OASIS value" test in tests/abi_test.zig will check the value against the header automatically.
• Add a sha224 arm to the Hasher union in crypto/digest.zig (Zig's std has sha2.Sha224), and give it a new state tag for operation-state serialization.
• Add it to config.supported_mechanisms and to the digest arm of C_GetMechanismInfo in slot_token.zig.

Test it works:

pkcs11-tool --module zig-out/lib/libhsm.so --hash --mechanism SHA224 --input-file msg.bin

Compare against sha224sum msg.bin. Add a unit test in digest.zig against the SHA-224("abc") vector.

Challenge 2: Expose a token-info detail you control

What to build: Make the token model string or serial number configurable through an environment variable instead of a fixed constant.

Why it's useful: Real deployments label tokens. It is a gentle introduction to the config and slot-token layers.

What you'll learn:

• How config.zig centralizes every constant
• How env.zig reads the environment at the C boundary (there is no std.process arena here)

Hints:

• C_GetTokenInfo in slot_token.zig fills the model and serialNumber fields from config.token_model and config.token_serial through util.padded.
• Add an env lookup in env.zig and fall back to the constant when it is unset, mirroring how the storage paths work.
• Remember the fields are space-padded fixed arrays, not NUL-terminated strings.

Test it works: pkcs11-tool -T should show your value, and an unset variable should show the default.

Challenge 3: Add a just recipe that signs and then verifies with OpenSSL

What to build: A recipe that generates an EC key in the module, signs a file, exports the public key, and verifies the signature with the openssl command line.

Why it's useful: Cross-verifying against an independent implementation is exactly how this project proves correctness. You are turning that into a one-liner.

What you'll learn:

• How pkcs11-tool and openssl interoperate through the module
• Why an independent oracle is stronger proof than a self-test

Hints:

• Look at the existing justfile recipes and the smoke flow for the command sequence.
• pkcs11-tool --read-object --type pubkey exports the public key; openssl dgst -verify checks the signature.

Test it works: The recipe exits 0 and prints Verified OK.

Intermediate Challenges

Challenge 4: AES-192

What to build: Support 192-bit AES keys for CKM_AES_KEY_GEN, CKM_AES_CBC, CKM_AES_CBC_PAD, and CKM_AES_GCM.

Why it's useful: It is currently unsupported because Zig's standard library exposes no 192-bit AES (see CONFORMANCE.md section 3.3). Solving it teaches you the same libcrypto-bridge pattern RSA uses.

Real world application: Some FIPS profiles and legacy systems require AES-192 specifically.

Implementation approach:

1. Decide the backend. Zig std has no Aes192, so route 192-bit keys through libcrypto's EVP AES, the way rsa.zig routes RSA.
2. Add the extern declarations you need to openssl.zig (the EVP cipher functions).
3. Branch on key length in cipher.zig's block and GCM functions, calling libcrypto for 24-byte keys and keeping std for 16 and 32.
4. Update cipher.validKeyLen and the CKR_KEY_SIZE_RANGE checks to accept 24.

Hints:

• The encBlockRaw / decBlockRaw functions currently switch (key.len) over 16 and 32 with unreachable otherwise. That switch is where the 24-byte arm goes.
• Keep the constant-time and zeroization patterns identical to the existing arms.

Test it works: A NIST AES-192-CBC vector in cipher.zig, plus a pkcs11-tool round-trip with a 24-byte key.

Challenge 5: A per-object file backend

What to build: Replace the single object file with a directory where each token object is its own file, the way SoftHSM2 does it.

Why it's useful: A single file means rewriting everything on every change. Per-object files let you write only what changed and make backup a per-object copy.

What you'll learn:

• Storage design trade-offs (one file versus many)
• How the persistence codec in object_store.zig separates serialization from I/O

Implementation approach:

1. Keep the existing serialize / parse for a single object's attributes; you are changing only how they are grouped on disk.
2. Give each object a stable on-disk id (the handle is monotonic and never reused, which helps).
3. Write the sealing logic unchanged; each object file still seals its sensitive values under the master key.

Hints:

• The seal-on-write and mark-sealed-on-read logic in object_store.zig does not care how many files there are. Keep it.
• env.zig resolves a path today; you will resolve a directory.

Test it works: Create several objects, restart the process, and confirm they all reload. Corrupt one object file and confirm only that object is lost, not the whole store.

Advanced Challenges

Challenge 6: Implement the PKCS#11 v3.0 interface discovery

What to build: Export C_GetInterface alongside C_GetFunctionList, returning a CK_FUNCTION_LIST_3_0 for hosts that ask for v3.0.

Why this is hard: You are extending the ABI itself, the part that must match the host byte for byte. A mistake here means hosts crash, not just misbehave.

What you'll learn:

• How v3.0 interface discovery works and why hosts fall back to C_GetFunctionList when C_GetInterface is absent (the OpenJDK fallback path)
• How CK_FUNCTION_LIST_3_0 shares a prefix with CK_FUNCTION_LIST and appends the new pointers

Architecture changes needed:

   C_GetInterface(name, version, &iface, flags)
        │
        ▼
   CK_INTERFACE { pInterfaceName, pFunctionList -> CK_FUNCTION_LIST_3_0, flags }
        │
   (C_GetFunctionList still returns the v2.40 CK_FUNCTION_LIST prefix)

Implementation steps:

1. Add CK_FUNCTION_LIST_3_0 and CK_INTERFACE to ck.zig, with the 24 new v3.0 function-pointer typedefs.
2. Build one larger table and hand out a pointer typed either way, setting the version field correctly per caller.
3. Export C_GetInterface in main.zig and add it to the version script in pkcs11.map.
4. Extend tests/abi_test.zig to cross-check the new structs against the v3.0 OASIS headers (vendor them alongside the v2.40 ones).

Gotchas:

• The first N members of CK_FUNCTION_LIST_3_0 must be byte-identical to CK_FUNCTION_LIST. The cross-check test is your safety net.
• New v3.0 functions you do not implement still need real, correctly-typed stub functions in the table. Never null.

Resources: The PKCS#11 v3.1 specification, and the 08-pkcs11.md research note in docs/zig/reference/ which covers the v3.0 surface.

Challenge 7: The v3.0 message-based AEAD API

What to build: Implement C_EncryptMessage / C_DecryptMessage (and the begin/next/final variants) for AES-GCM.

Why this is hard: The message API is how the OpenSSL 3.x provider increasingly prefers to do GCM, and it has a different nonce-management contract than the classic C_Encrypt.

What you'll learn:

• The v3.0 message API shape and how it differs from the v2.40 single-shot and update/final flows
• How to keep your RUP-safe stance under the new API

Implementation approach:

1. Requires Challenge 6 first (the v3.0 table).
2. The message API passes the nonce and AAD through CK_MESSAGE-family parameters rather than the mechanism parameter.
3. Keep the buffered, verify-before-release behavior for decrypt that the classic path already has in GcmStream.

Hints: Your existing GCM accumulator and the strict parameter validation in buildCipher are most of the work; the new part is the message-API parameter parsing.

Challenge 8: Fork safety

What to build: Detect a fork() and refuse to operate in the child with stale state inherited from the parent, or re-initialize cleanly.

Why this is hard: A child process inherits the parent's initialized state, including an unwrapped master key, which is a real hazard. The PKCS#11 answer is the CKF_INTERFACE_FORK_SAFE flag, but only if you actually handle it.

What you'll learn:

• Why inherited crypto state across a fork is dangerous
• How to track the process id and invalidate state on change

Implementation approach:

1. Record the pid in the Instance at C_Initialize.
2. In state.acquire(), compare the current pid; on a mismatch, treat the library as not initialized in the child (force a re-init) rather than operating on the parent's secrets.
3. Only then advertise CKF_INTERFACE_FORK_SAFE (a v3.0 interface flag, so this pairs with Challenge 6).

Gotchas: The master key inherited by a child must be wiped, not used. Be careful that the detection happens before any operation can touch inst.mk.

Expert Challenges

Challenge 9: A SQLite storage backend with multi-token support

What to build: Replace the file backend with SQLite, and lift the one-token limit so the module can host several tokens across several slots.

Estimated time: A week or more.

Prerequisites: You should have done Challenge 5 (per-object storage) first, because this builds on separating the codec from the I/O.

What you'll learn:

• How a real software HSM (SoftHSM2 has exactly this) structures multi-token storage
• How to shard the global state per slot without losing the single-lock simplicity, or how to move to per-slot locks

Planning this feature:

Before you code, think through:

• How does multi-slot change the global Instance? Today it is one token, one session table, one object store. You need a collection keyed by slot.
• Does the single global mutex still serve, or do you want per-slot locks? Per-slot locks add real concurrency but also real deadlock risk.
• How do you migrate an existing single-file token into the database on first run?

High level architecture:

   slots: [N] of Token
     each Token: its own PIN state, master key, object store
   storage: SQLite (one table for tokens, one for objects, sealed values as BLOBs)
   locking: either the one global mutex (simple) or per-slot (concurrent)

Implementation phases:

Phase 1: Multi-token in memory. Turn the single token/store/sessions in Instance into slot-indexed collections. Keep the file backend. Get C_GetSlotList returning several slots and the per-slot operations routing correctly.

Phase 2: The SQLite backend. Link SQLite (the libcrypto bridge in openssl.zig is your template for binding a C library cleanly). Move serialize/parse to read and write rows. Keep the sealing logic untouched; values are still GCM-sealed BLOBs.

Phase 3: Migration and concurrency. Migrate an existing file token on first open. Decide and implement the locking model.

Phase 4: Polish. Error handling for a locked or corrupt database, and the full test pass through pkcs11-tool against multiple tokens.

Success criteria:

• C_GetSlotList returns more than one slot
• Two tokens hold independent keys and PINs
• Keys survive a restart, loaded from SQLite
• A corrupt or locked database fails closed, never returns a wrong key
• The ABI test and smoke harness still pass unchanged

Mix and Match

Combine challenges into bigger projects:

A v3.0-native module. Challenge 6 (interface) plus Challenge 7 (message API) plus Challenge 8 (fork safety) gives you a module a modern OpenSSL 3.x provider talks to natively.

A production-shaped store. Challenge 5 (per-object) plus Challenge 9 (SQLite, multi-token) gives you something with SoftHSM2's storage capabilities.

Real World Integration Challenges

Drive the module from the OpenSSL pkcs11 provider

The goal: Use this module as the key store behind OpenSSL itself, so openssl commands sign and decrypt through it.

What you'll need:

• The pkcs11-provider (the modern OpenSSL 3.x provider) configured to point at libhsm.so
• An OpenSSL config that loads the provider

Watch out for: The provider exercises corners pkcs11-tool does not, especially around RSA-PSS parameters and GCM. This is a great way to find conformance gaps. Anything it trips is a real bug or a documented narrowing in CONFORMANCE.md.

Use the module from Java via SunPKCS11

The goal: Have a Java program generate a key and sign through the module using java.security and the SunPKCS11 provider.

What you'll learn: SunPKCS11 is the classic C_GetInterface-fallback consumer, so this also validates that your C_GetFunctionList path is solid (and motivates Challenge 6 if you want the v3.0 path).

Performance Challenges

Reduce lock contention under concurrent sessions

The goal: Let many sessions do crypto in parallel instead of serializing on the one global mutex.

Current bottleneck: Every entry point takes state.mutex. Two sessions signing at once serialize, even though their operations are independent.

Approaches:

• Per-session operation state, lock only for shared state. The op-state already lives in the Session; you could do the crypto outside the lock and only lock to fetch and commit, the way C_Login already does for Argon2id.
• Per-slot locks. Pairs with the multi-slot work in Challenge 9.

Benchmark it: Drive N threads each signing in a loop and measure throughput before and after. Watch for races; the generation-counter pattern in state.zig is your model for safe lock-release-relock.

Security Challenges

Prove constant-time behavior with Valgrind

The goal: Use Valgrind's memcheck and Zig's classify / declassify to detect any branch or memory access that depends on a secret.

What you'll learn: The closest thing to a constant-time verifier Zig has. classify marks secret memory as uninitialized to Valgrind, which then flags any conditional that depends on it.

Implementation: Add a debug build that classifies the PIN hash and key material before the crypto, run it under valgrind --tool=memcheck --track-origins=yes, and chase down any "conditional jump depends on uninitialised value" reports. The 16-constant-time-security.md research note in docs/zig/reference/ walks through the API.

Testing the security: A clean run with no conditional-on-secret warnings through a full sign and decrypt is the goal.

Fuzz the persistence codec

The goal: Throw malformed token and object files at parse and confirm it always fails closed (clears to empty) and never crashes or reads out of bounds.

Threat model: This protects against a malicious or corrupted store file. The codec already has length and bounds checks; fuzzing proves they are complete.

Implementation: Generate random and mutated record bytes, feed them to object_store.parse and token.loadFrom, and assert the result is either a valid parse or a clean empty store, never a panic. The existing "rejects a bad magic" and "fails safe on a truncated record" tests are your starting points.

Contribution Ideas

Finished something? Share it back:

1. Fork the repo.
2. Implement your extension in a branch, with unit tests against a published vector and a smoke-harness or pkcs11-tool proof.
3. Document it: update CONFORMANCE.md if you changed a boundary, and add a note to the relevant learn doc.
4. Open a PR with the implementation, the tests, and the external-tool proof.

Good extensions (a new mechanism with vectors, a real storage backend, the v3.0 surface) are exactly the kind of thing that makes the project more useful to the next person.

Challenge Yourself Further

Build something new

Use what you learned here to build:

• A PKCS#11 provider for a different key type, like Ed25519 or ML-DSA (both are in Zig's std), following the ECDSA path.
• A minimal host that loads any PKCS#11 module and exercises it, the consumer side of the interface you just implemented.

Study real implementations

Read these and steal their good ideas:

• SoftHSMv2 for the per-token directory store and the OpenSSL/Botan crypto abstraction.
• OpenSC pkcs11-spy for how a transparent shim logs every call, which is a clean piece of ABI engineering in its own right.

Getting Help

Stuck on a challenge?

1. Debug systematically. What did you expect, what happened, what is the smallest input that reproduces it? Run the failing case through just spy and read the exact call and return code.
2. Read the existing code. The mechanism you are adding almost certainly has a sibling already implemented. The path is the same.
3. Lean on the cross-check. If you touched ck.zig, run zig build test; the ABI test will tell you precisely which field or constant diverged from the OASIS headers.

Challenge Completion

Track your progress:

• Easy 1: SHA-224 digest
• Easy 2: configurable token info
• Easy 3: OpenSSL cross-verify recipe
• Intermediate 4: AES-192 via libcrypto
• Intermediate 5: per-object file backend
• Advanced 6: v3.0 C_GetInterface
• Advanced 7: v3.0 message API
• Advanced 8: fork safety
• Expert 9: SQLite, multi-token

Finished them all? You have built a multi-token, v3.0-capable software HSM with a real database backend and proven constant-time behavior. At that point you understand key custody at a level most working engineers never reach. Go read a real HSM vendor's PKCS#11 docs; you will recognize every design decision they made and why.