RomanNum3ral
/
Cybersecurity-Projects
mirror of https://github.com/CarterPerez-dev/Cybersecurity-Projects.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Cybersecurity-Projects/PROJECTS/advanced/encrypted-p2p-chat/learn/00-OVERVIEW.md

46 KiB
Raw Permalink Blame History

Encrypted P2P Chat

What This Is

This is a production-grade end-to-end encrypted peer-to-peer messaging application built on the Signal Protocol. It implements X3DH (Extended Triple Diffie-Hellman) for asynchronous key exchange, the Double Ratchet algorithm for forward-secret message encryption, and WebAuthn/Passkeys for phishing-resistant authentication. The server orchestrates message delivery and stores encrypted blobs, but it never sees plaintext messages or private keys. If someone compromises the server, they get ciphertext and public keys. That is it.

Why This Matters

On January 22, 2020, the United Nations confirmed that Jeff Bezos's phone had been compromised through a WhatsApp message. The forensic analysis by FTI Consulting traced the infection to a video file sent from the personal WhatsApp account of Saudi Crown Prince Mohammed bin Salman. Within hours of receiving that message, data exfiltration from the phone increased by approximately 29,000 percent. The attackers used a zero-day exploit in the WhatsApp video rendering stack to deliver spyware, likely NSO Group's Pegasus. What made this attack especially instructive is what it was not: it was not a failure of end-to-end encryption. The E2E encryption in WhatsApp (which uses the Signal Protocol) worked correctly. The message content was encrypted in transit. But the exploit happened on the device itself, after decryption. This distinction matters because it shows exactly where E2E encryption draws its line. It protects data in transit and at rest on the server. It does not protect against compromised endpoints. Understanding this boundary is critical to building and reasoning about encrypted messaging systems.

Facebook Messenger, by contrast, showed what happens when encryption is not the default. In September 2019, a server containing phone numbers linked to hundreds of millions of Facebook accounts was found exposed online without password protection. The dataset included 419 million records spanning users in the US, UK, and Vietnam. While this particular leak was about phone numbers rather than message content, the core problem was structural: Facebook Messenger did not enable end-to-end encryption by default. "Secret Conversations" existed as an opt-in feature, but the vast majority of messages flowed through Facebook's servers in a form the company could read. This meant that any server compromise, government subpoena, or rogue employee could access message content directly. Facebook did not roll out default E2E encryption for Messenger until late 2023. For years, billions of messages were protected only by transport-layer encryption (TLS), which encrypts data between the client and the server but leaves the server itself as a trusted party with full access to plaintext.

The 2020 Zoom encryption controversy demonstrated how the gap between marketing claims and cryptographic reality can erode user trust. Zoom's marketing materials and security white paper stated that the platform supported "end-to-end encryption." The Citizen Lab at the University of Toronto investigated and found that Zoom was actually using TLS for transport encryption, with AES-128 keys generated by Zoom's own servers. This meant Zoom had the ability to decrypt any meeting. The keys were sometimes routed through servers in China, even for meetings where all participants were in North America. Zoom eventually settled an FTC complaint and committed to implementing actual E2E encryption, but the incident highlighted a critical point: the phrase "end-to-end encrypted" is meaningless without verifiable implementation. In a real E2E system, key generation happens on client devices, the server relays ciphertext, and at no point does the server have access to decryption keys. That is exactly what this project implements.

The Cambridge Analytica scandal of 2018, while primarily about data harvesting through Facebook's Graph API rather than message interception, drives home a broader principle: any data a server can access, a server can leak, sell, or be compelled to hand over. Cambridge Analytica harvested personal data from approximately 87 million Facebook profiles through a personality quiz app that exploited Facebook's permissive data-sharing policies. The data was used for political advertising targeting during the 2016 US presidential election and the Brexit referendum. The lesson for messaging applications is straightforward. If your server can read messages, your server is a liability. Every employee with database access, every government with a court order, every attacker who breaches your infrastructure becomes a threat to user privacy. A zero-knowledge server design, where the server handles routing and storage of ciphertext but never possesses decryption keys, eliminates this entire category of risk. That is the architecture this project uses.

The Signal Protocol is now the gold standard for encrypted messaging. Signal (the application) pioneered it. WhatsApp adopted it in 2016, bringing end-to-end encryption to over 2 billion users. Google Messages uses it for RCS chats. Meta's Messenger migration to default E2E encryption in December 2023 also drew on Signal Protocol concepts. The protocol's core innovations, X3DH for asynchronous key establishment and the Double Ratchet for per-message forward secrecy, solve problems that earlier encrypted messaging systems (like PGP-encrypted email) left open for decades. This project implements both protocols from scratch, giving you direct experience with the same cryptographic machinery that protects billions of real conversations. And by pairing it with WebAuthn/Passkey authentication, it eliminates the number one attack vector for account compromise: password phishing. Phishing accounts for approximately 36% of all data breaches according to the 2023 Verizon Data Breach Investigations Report. WebAuthn makes phishing structurally impossible because the authenticator is bound to the origin domain. A fake login page on a different domain simply cannot trigger the correct credential.

What You'll Learn

Cryptographic Protocols

  • X3DH key exchange for asynchronous session establishment. The fundamental problem with Diffie-Hellman key exchange is that both parties need to be online at the same time. If Alice wants to send a message to Bob, but Bob is offline, classic DH cannot work. X3DH (Extended Triple Diffie-Hellman) solves this by having Bob pre-publish a bundle of public keys to the server: a long-term identity key, a medium-term signed prekey, and a set of single-use one-time prekeys. When Alice wants to start a conversation, she downloads Bob's bundle and performs three (or four, if a one-time prekey is available) DH computations to derive a shared secret. Bob can complete his half of the exchange when he comes back online. All X3DH math runs in the browser in frontend/src/crypto/x3dh.ts (initiateX3DH and receiveX3DH); the backend only stores and serves the public bundle in backend/app/services/prekey_service.py. X25519 is used for Diffie-Hellman, Ed25519 for prekey signatures.

  • Double Ratchet algorithm for forward secrecy and post-compromise security. Once X3DH establishes an initial shared secret, the Double Ratchet takes over for ongoing message encryption. It combines two ratcheting mechanisms: a symmetric-key ratchet (HMAC-based) that advances with every message, and a Diffie-Hellman ratchet that advances every time the conversation "turn" changes. The symmetric ratchet provides forward secrecy: if an attacker steals your current keys, they cannot decrypt past messages because each message key is derived from and then discarded after the previous chain key. The DH ratchet provides post-compromise security (also called "future secrecy" or "self-healing"): even if an attacker compromises your current state completely, once you send a new message with a fresh DH key pair, the attacker loses access because they do not know the new private key. The Double Ratchet runs entirely in the browser in frontend/src/crypto/double-ratchet.ts (encryptMessage / decryptMessage); the backend only relays the resulting ciphertext.

  • AES-256-GCM authenticated encryption. Each message is encrypted using AES-256-GCM (Galois/Counter Mode), which provides both confidentiality and integrity. GCM is an AEAD (Authenticated Encryption with Associated Data) cipher, meaning it produces a ciphertext and an authentication tag. If anyone tampers with the ciphertext, the tag verification fails and decryption is rejected. The "associated data" in this project is the serialized message header (DH public key, message number, previous chain length), binding each ciphertext to the ratchet state that produced it. The implementation uses 12-byte random nonces and 32-byte keys derived from the Double Ratchet chain. You'll see it in aesGcmEncrypt / aesGcmDecrypt in frontend/src/crypto/primitives.ts, called from inside encryptMessage / decryptMessage.

  • HKDF-SHA256 key derivation. Raw Diffie-Hellman outputs are not suitable for direct use as encryption keys. They have non-uniform distribution and may contain structural biases. HKDF (HMAC-based Key Derivation Function) extracts entropy from DH outputs and expands it into keys of the required length. This project uses HKDF-SHA256 throughout: in X3DH to derive the initial shared secret from concatenated DH outputs (with the spec-mandated 0xFF * 32 F prefix and b'X3DH' info), and in the Double Ratchet's KDF_RK (root-key derivation) which uses the previous root key as HKDF salt and the new DH output as input keying material. The chain-key step (KDF_CK) uses HMAC-SHA256 with single-byte tags 0x01 (message key) and 0x02 (next chain key) per the Signal spec.

  • Ed25519 digital signatures and X25519 Diffie-Hellman. The project uses two related but distinct elliptic curve operations on Curve25519. X25519 is used for Diffie-Hellman key exchange: two parties each generate a key pair, exchange public keys, and compute a shared secret. Ed25519 is used for digital signatures: the identity key owner signs their signed prekey so that other users can verify it was genuinely published by that identity, not substituted by a malicious server. The mathematical operations are different (X25519 operates on the Montgomery form of the curve, Ed25519 on the Edwards form), but they share the same underlying curve. Each user generates separate X25519 and Ed25519 identity key pairs in generateIdentityKeyPair (frontend/src/crypto/x3dh.ts); only the public halves ever leave the device.

Authentication and Identity

  • WebAuthn/FIDO2 passkey registration and authentication. WebAuthn is a W3C standard that allows users to authenticate using public key cryptography instead of passwords. During registration, the browser creates a new key pair on a hardware or platform authenticator (a YubiKey, a fingerprint reader, a phone's secure enclave). The public key is sent to the server. The private key never leaves the authenticator. During login, the server sends a random challenge, the authenticator signs it with the private key, and the server verifies the signature against the stored public key. No shared secret ever crosses the network. The implementation uses the py_webauthn library on the server (backend/app/core/passkey/passkey_manager.py) and the browser's native navigator.credentials API on the client. Registration options require resident keys (enabling usernameless login) and require user verification (PIN/biometric), since this is an E2EE chat app and unattended-device access shouldn't unlock the conversation. The WebAuthn user.id is a 64-byte random webauthn_user_handle per spec, stored on the User row and never exposed to clients.

  • Public key cryptography for identity verification. Each user in the system has two types of public keys: WebAuthn credentials (for authentication with the server) and Signal Protocol identity keys (for end-to-end encryption with other users). The WebAuthn credential proves to the server that you are who you claim to be. The Signal Protocol identity key lets other users verify your encryption keys. These are separate key pairs serving separate purposes. The identity key is an Ed25519 key pair that signs the user's medium-term signed prekeys, letting recipients verify that the prekey bundle they downloaded was genuinely published by that user and not substituted by a compromised server.

  • Authenticator clone detection via signature counters. Every WebAuthn authenticator maintains a signature counter that increments with each authentication ceremony. The server stores the last-seen counter value. If the counter ever goes backward or stays the same (when both the server and authenticator track non-zero counters), it indicates that the authenticator may have been cloned. This is a known FIDO2 security mechanism. The verify_authentication method in passkey_manager.py explicitly checks for this: if new_sign_count <= credential_current_sign_count, it raises a ValueError with a message about potential cloned authenticator detection. This catches scenarios where an attacker has physically duplicated a hardware security key.

Fullstack Architecture

  • FastAPI async backend with SQLModel ORM. The backend is built with FastAPI, using its native async support with asyncio and async/await throughout. Database models use SQLModel, which merges SQLAlchemy's ORM capabilities with Pydantic's data validation. The app factory pattern in factory.py creates the FastAPI instance with a lifespan context manager that handles startup (initializing PostgreSQL, connecting to Redis, connecting to SurrealDB) and shutdown (disconnecting cleanly). API routes are organized into four routers: auth (WebAuthn registration / login / /auth/me / /auth/logout), encryption (public prekey bundle upload and lookup), rooms (chat room CRUD with membership-gated access), and websocket (real-time message transport, authenticated via the same session cookie). All non-auth endpoints depend on current_user from app/core/dependencies.py, which resolves the session cookie against Redis. Rate limits are applied via slowapi. The backend uses ORJSONResponse as the default response class for faster JSON serialization.

  • SolidJS reactive frontend with TypeScript. The frontend uses SolidJS, a reactive UI framework with fine-grained reactivity (no virtual DOM). State management uses Nanostores with the @nanostores/solid binding and @nanostores/persistent for localStorage persistence. Routing uses @solidjs/router. Server state management uses @tanstack/solid-query for data fetching, caching, and synchronization. The component structure is organized into domain folders: Auth (AuthCard, AuthForm, PasskeyButton), Chat (ChatInput, ChatHeader, ConversationList, MessageBubble, MessageList, EncryptionBadge, OnlineStatus, TypingIndicator, UserSearch), Layout (AppShell, Header, Sidebar, ProtectedRoute), and UI (Avatar, Badge, Button, Dropdown, Input, Modal, Spinner, Toast, Tooltip). Styling uses Tailwind CSS v4 via the Vite plugin.

  • WebSocket real-time messaging. Messages are delivered in real time over WebSocket connections. The backend's websocket_manager.py maintains a connection pool and handles broadcasting. The frontend's websocket/websocket-manager.ts manages connection lifecycle, reconnection, and heartbeats. Message types include encrypted messages, typing indicators, presence updates, read receipts, and error notifications. The WebSocket protocol carries ciphertext between clients; the server routes messages without decrypting them. Heartbeat interval is configurable (default 30 seconds), and each user can have up to 5 concurrent connections (for multiple devices or tabs).

  • Multi-database architecture: PostgreSQL, SurrealDB, Redis. The project uses three databases, each chosen for a specific access pattern. PostgreSQL (via asyncpg and SQLModel) stores authentication data, user profiles, WebAuthn credentials, and public prekey material (identity public keys, signed prekey publics + signatures, unused one-time prekey publics). No private key ever lives server-side. SurrealDB stores real-time chat data: messages (ciphertext only), room membership, and presence. Its live query feature enables real-time push notifications without polling. Redis stores ephemeral state: WebAuthn registration context (challenge + user handle + display name), authentication challenges (keyed by challenge bytes for replay-safety), session tokens (session:<token> -> user_id with 24h TTL), and slowapi rate-limit counters. Each database has its own manager class (Base.py for PostgreSQL, surreal_manager.py for SurrealDB, redis_manager.py for Redis).

  • Docker containerization with Nginx reverse proxy. The entire application runs in Docker containers orchestrated by Docker Compose. The development compose file (dev.compose.yml) defines six services: PostgreSQL 16 Alpine, SurrealDB (latest), Redis 8 Alpine, the FastAPI backend, the Vite dev server for the frontend, and Nginx Alpine as a reverse proxy. Each database has health checks to ensure the backend only starts after its dependencies are ready. The Nginx configuration routes HTTP traffic and proxies WebSocket connections. There are separate Dockerfiles for development (with hot-reload via volume mounts) and production (with optimized builds) in conf/docker/dev/ and conf/docker/prod/.

Security Engineering

  • Zero-knowledge server design. The server in this application is designed so that it cannot read messages even if it wanted to. Private keys are generated on client devices (generateIdentityKeyPair in frontend/src/crypto/x3dh.ts) and stored in the browser's IndexedDB (frontend/src/crypto/key-store.ts). The server stores only public keys (backend/app/services/prekey_service.py:store_client_keys writes only the public halves; the schema columns for private keys were dropped) and ciphertext blobs in SurrealDB. When Alice sends a message to Bob, crypto-service.ts:encrypt runs the Double Ratchet locally, hands the ciphertext to the WebSocket, and the server's message_service.store_encrypted_message does a pure pass-through write — there is no decrypt code path on the backend. The header serialization that becomes the AES-GCM associated data binds each ciphertext to the ratchet state that produced it, so a compromised server cannot redirect ciphertext between sessions without breaking authentication.

  • Key rotation and lifecycle management. Cryptographic keys in this system have defined lifecycles, all driven by the client. One-time prekeys are single-use: when a sender fetches Bob's bundle, the chosen OPK is marked is_used=True server-side (so it can't be handed out twice), but the private OPK lives only in Bob's IndexedDB. The client maintains a pool of 100 OPKs and replenishes (and re-uploads the new public OPKs) when the unused count drops below half (crypto-service.ts:replenishOneTimePreKeys). Signed prekeys rotate every 48 hours; the client generates a new SPK locally and re-runs uploadKeys to publish the new public + signature. Identity keys are long-term and intentionally don't rotate — they're the trust anchor that signs the SPK. Ratchet state is per-conversation, advances with every message, and is serialized into the IndexedDB ratchet store after every encrypt/decrypt. Skipped message keys (for out-of-order delivery) live in the in-memory ratchet state, capped at MAX_SKIP_MESSAGE_KEYS.

  • Threat modeling for messaging applications. Building this project exposes you to the specific threat model that the Signal Protocol is designed to address. The primary threats include: a compromised server attempting to read or modify messages (mitigated by E2E encryption and associated data binding), an attacker who steals a user's current session keys (mitigated by forward secrecy from the symmetric ratchet), an attacker who fully compromises a device and then loses access (mitigated by post-compromise security from the DH ratchet), a man-in-the-middle attempting to substitute public keys (mitigated by identity key signing and optional safety number verification), and an attacker who intercepts one-time prekeys (mitigated by the three base DH computations that provide security even without the one-time prekey). Understanding which threats each mechanism addresses is the core of cryptographic protocol design.

  • Client-side cryptography with WebCrypto API and IndexedDB key storage. All encryption and decryption happens in the browser using the WebCrypto API (window.crypto.subtle). The primitives.ts file wraps the WebCrypto API into functions for X25519 key generation and DH, Ed25519 signing and verification, HKDF key derivation, AES-256-GCM encryption/decryption, HMAC-SHA256, and utility functions like constant-time comparison. Private keys are stored in IndexedDB (key-store.ts) in a database called encrypted-chat-keys with separate object stores for identity keys, signed prekeys, one-time prekeys, and ratchet states. IndexedDB was chosen over localStorage because it can store structured data (including CryptoKey objects in browsers that support it) and has no size limits comparable to localStorage's typical 5-10MB cap.

Prerequisites

Required Knowledge

  • Python async programming (asyncio, async/await). The entire backend is async. Route handlers are async def, database queries use await, and the application lifespan is managed by an asynccontextmanager. You need to understand coroutines, event loops, and how await yields control.
  • TypeScript and reactive UI frameworks. The frontend is written in TypeScript with SolidJS. You should be comfortable with TypeScript's type system (generics, discriminated unions, type guards) and the concept of reactive state (signals, effects, derived state). Prior experience with React or Vue is transferable.
  • Basic understanding of public key cryptography. You need to know what a key pair is (private key stays secret, public key is shared), what signing and verification mean, and the general concept of Diffie-Hellman (two parties derive a shared secret from their public/private key pairs). You do not need to know the math behind elliptic curves.
  • Docker and Docker Compose. The development environment runs six containers. You need to know how to read a compose file, run docker compose up, check logs, and debug container startup issues.
  • SQL and database fundamentals. The PostgreSQL models use SQLModel (SQLAlchemy under the hood). You should be able to read an ORM model definition and understand concepts like foreign keys, indexes, and migrations.

Required Tools

  • Docker and Docker Compose. For running the multi-service development environment. Docker Desktop or Docker Engine with the Compose plugin.
  • Python 3.13+. The project requires Python 3.13 or higher as specified in pyproject.toml with requires-python = ">=3.13". The type annotations and language features used throughout the codebase depend on this version.
  • Node.js 20+. For the SolidJS/Vite frontend build toolchain.
  • uv (Python package manager). This project uses uv for Python dependency management. Never use pip directly. Install with curl -LsSf https://astral.sh/uv/install.sh | sh or brew install uv.
  • pnpm (Node package manager). This project uses pnpm for Node dependency management. Never use npm directly. Install with corepack enable && corepack prepare pnpm@latest --activate or npm install -g pnpm.
  • just (command runner). The project uses a justfile for build automation, linting, testing, database migrations, and Docker orchestration. Install with cargo install just, brew install just, or your system's package manager.

Helpful But Optional

  • Prior exposure to WebSocket programming. The real-time messaging layer uses WebSockets. Familiarity with the WebSocket lifecycle (connect, send, receive, close) and concepts like heartbeats and reconnection will help, but the code is self-documenting.
  • Understanding of Diffie-Hellman key exchange. Knowing the basic DH protocol (Alice and Bob each generate key pairs, exchange public keys, compute the same shared secret) will make the X3DH implementation much easier to follow. The X3DH protocol is essentially four DH computations combined.
  • Familiarity with FastAPI or Flask. The backend uses FastAPI patterns: route decorators, dependency injection, Pydantic schemas for request/response validation, and middleware. If you have used any Python web framework, the structure will be familiar.
  • Knowledge of browser IndexedDB API. The client-side key storage uses IndexedDB directly (no wrapper library). IndexedDB is an asynchronous, transactional, key-value database built into every modern browser. Understanding its callback-based API (which this project wraps in Promises) will help you follow the key-store.ts implementation.

Quick Start

Docker (Recommended)

# Clone and enter the project
cd PROJECTS/advanced/encrypted-p2p-chat

# Copy environment file and update passwords
cp .env.example .env

# Start all services (PostgreSQL, SurrealDB, Redis, backend, frontend, nginx)
docker compose -f dev.compose.yml up

# Backend API:      http://localhost:8000
# Frontend dev:     http://localhost:5173
# API docs:         http://localhost:8000/docs
# Nginx proxy:      http://localhost:80

Manual Setup (Without Docker)

# Backend
cd backend
uv sync
uv run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Frontend (separate terminal)
cd frontend
pnpm install
pnpm dev

For manual setup, you need PostgreSQL, SurrealDB, and Redis running locally. Update the connection strings in .env to point to localhost instead of the Docker service names.

Using the justfile

# See all available commands
just

# Full setup (backend deps, frontend deps, env file)
just setup

# Start dev environment
just dev-up

# Run linters
just lint

# Run tests
just test

# Database migrations
just migrate-local head

Project Structure

encrypted-p2p-chat/
|
|-- backend/
|   |-- app/
|   |   |-- api/                              # HTTP and WebSocket route handlers
|   |   |   |-- auth.py                       # WebAuthn registration and login endpoints
|   |   |   |-- encryption.py                 # X3DH prekey bundle upload and retrieval
|   |   |   |-- rooms.py                      # Chat room creation, listing, membership
|   |   |   +-- websocket.py                  # WebSocket connection and message routing
|   |   |
|   |   |-- core/
|   |   |   |-- passkey/
|   |   |   |   +-- passkey_manager.py        # WebAuthn credential registration and verification
|   |   |   |-- dependencies.py               # current_user, issue_session, revoke_session
|   |   |   |-- websocket_manager.py          # Connection pool, broadcasting, heartbeat management
|   |   |   |-- surreal_manager.py            # SurrealDB client: connect, query, live subscriptions
|   |   |   |-- redis_manager.py              # Redis client: sessions, challenges, ephemeral cache
|   |   |   |-- enums.py                      # Shared enumerations (message types, status codes)
|   |   |   |-- exceptions.py                 # Domain-specific exception classes
|   |   |   +-- exception_handlers.py         # FastAPI exception handler registration
|   |   |
|   |   |-- models/                           # SQLModel database models (PostgreSQL)
|   |   |   |-- Base.py                       # SQLAlchemy engine, session factory, init_db
|   |   |   |-- User.py                       # User account + 64-byte WebAuthn user_handle
|   |   |   |-- Credential.py                 # WebAuthn credential storage (public key, counter, BE/BS flags)
|   |   |   |-- IdentityKey.py                # Long-term X25519 + Ed25519 identity public keys (no privates)
|   |   |   |-- SignedPrekey.py               # Client-uploaded signed prekey publics + signatures
|   |   |   +-- OneTimePrekey.py              # Single-use prekey publics (consumed via is_used flag)
|   |   |
|   |   |-- services/                         # Business logic layer
|   |   |   |-- auth_service.py               # Registration / authentication / search (no key gen)
|   |   |   |-- prekey_service.py             # Public-only prekey bundle storage + lookup
|   |   |   |-- message_service.py            # Pass-through ciphertext writer (server never decrypts)
|   |   |   |-- presence_service.py           # Online/offline status tracking via SurrealDB
|   |   |   +-- websocket_service.py          # WebSocket message routing + membership check + per-user rate cap
|   |   |
|   |   |-- schemas/                          # Pydantic request/response models
|   |   |   |-- auth.py                       # Registration/login options and verification schemas
|   |   |   |-- rooms.py                      # Room creation and listing schemas
|   |   |   |-- websocket.py                  # WebSocket message payload schemas
|   |   |   |-- surreal.py                    # SurrealDB record schemas
|   |   |   +-- common.py                     # Shared schemas (health, root, pagination)
|   |   |
|   |   |-- config.py                         # Centralized settings, constants, environment loading
|   |   |-- factory.py                        # FastAPI app factory with lifespan context manager
|   |   +-- main.py                           # Uvicorn entry point
|   |
|   |-- tests/                                # pytest test suite
|   |   |-- conftest.py                       # Shared fixtures (in-memory async SQLite, test users)
|   |   |-- test_auth_service.py              # User CRUD and lookup
|   |   |-- test_prekey_service.py            # Bundle/upload regression tests (incl. broken-filter regression)
|   |   +-- test_session_auth.py              # Protected endpoints return 401 without session cookie
|   |
|   |-- alembic/                              # Database migration scripts
|   |   +-- env.py                            # Alembic environment configuration
|   +-- pyproject.toml                        # uv dependencies and tool configuration
|
|-- frontend/
|   |-- src/
|   |   |-- crypto/                           # Client-side Signal Protocol implementation
|   |   |   |-- primitives.ts                 # WebCrypto API wrappers (X25519, Ed25519, AES-GCM, HKDF)
|   |   |   |-- x3dh.ts                       # X3DH key exchange (client-side sender/receiver)
|   |   |   |-- double-ratchet.ts             # Double Ratchet state machine (client-side)
|   |   |   |-- crypto-service.ts             # Encryption orchestrator (ties X3DH + Ratchet together)
|   |   |   |-- key-store.ts                  # IndexedDB private key storage (identity, prekeys, ratchet)
|   |   |   +-- message-store.ts              # Encrypted message serialization and deserialization
|   |   |
|   |   |-- pages/                            # Route-level page components
|   |   |   |-- Home.tsx                      # Landing page
|   |   |   |-- Login.tsx                     # WebAuthn login flow
|   |   |   |-- Register.tsx                  # WebAuthn registration flow
|   |   |   |-- Chat.tsx                      # Main chat interface
|   |   |   +-- NotFound.tsx                  # 404 page
|   |   |
|   |   |-- components/
|   |   |   |-- Auth/                         # Authentication components
|   |   |   |   |-- AuthCard.tsx              # Card container for auth forms
|   |   |   |   |-- AuthForm.tsx              # Username/display name input form
|   |   |   |   +-- PasskeyButton.tsx         # WebAuthn credential creation trigger
|   |   |   |-- Chat/                         # Chat interface components
|   |   |   |   |-- ChatHeader.tsx            # Conversation header with user info
|   |   |   |   |-- ChatInput.tsx             # Message composition and send
|   |   |   |   |-- ConversationList.tsx      # Sidebar list of active conversations
|   |   |   |   |-- ConversationItem.tsx      # Single conversation preview
|   |   |   |   |-- MessageBubble.tsx         # Individual message display
|   |   |   |   |-- MessageList.tsx           # Scrollable message history
|   |   |   |   |-- EncryptionBadge.tsx       # Visual indicator of encryption status
|   |   |   |   |-- OnlineStatus.tsx          # User online/offline indicator
|   |   |   |   |-- TypingIndicator.tsx       # "User is typing..." display
|   |   |   |   |-- UserSearch.tsx            # Search for users to start conversations
|   |   |   |   +-- NewConversation.tsx       # New conversation creation flow
|   |   |   |-- Layout/                       # Page layout components
|   |   |   |   |-- AppShell.tsx              # Main app layout wrapper
|   |   |   |   |-- Header.tsx               # Top navigation bar
|   |   |   |   |-- Sidebar.tsx              # Side navigation panel
|   |   |   |   +-- ProtectedRoute.tsx       # Auth-gated route wrapper
|   |   |   +-- UI/                           # Reusable UI primitives
|   |   |       |-- Avatar.tsx               # User avatar display
|   |   |       |-- Badge.tsx                # Status badge component
|   |   |       |-- Button.tsx               # Button with variants
|   |   |       |-- Dropdown.tsx             # Dropdown menu
|   |   |       |-- IconButton.tsx           # Icon-only button
|   |   |       |-- Input.tsx               # Text input field
|   |   |       |-- Modal.tsx               # Dialog overlay
|   |   |       |-- Skeleton.tsx            # Loading placeholder
|   |   |       |-- Spinner.tsx             # Loading spinner
|   |   |       |-- TextArea.tsx            # Multiline text input
|   |   |       |-- Toast.tsx              # Notification toast
|   |   |       +-- Tooltip.tsx            # Hover tooltip
|   |   |
|   |   |-- services/                         # API client layer
|   |   |   |-- auth.service.ts               # Authentication API calls
|   |   |   +-- room.service.ts               # Room and message API calls
|   |   |
|   |   |-- stores/                           # Nanostores state management
|   |   |   |-- auth.store.ts                 # Current user + persistent userId hint
|   |   |   |-- rooms.store.ts                # Room list and active room state
|   |   |   |-- messages.store.ts             # Message history per conversation
|   |   |   |-- presence.store.ts             # User online status tracking
|   |   |   |-- typing.store.ts               # Typing indicator state
|   |   |   |-- settings.store.ts             # User preferences
|   |   |   +-- ui.store.ts                   # UI state (modals, sidebars, loading)
|   |   |
|   |   |-- websocket/                        # WebSocket client
|   |   |   |-- websocket-manager.ts          # Connection lifecycle and reconnection
|   |   |   +-- message-handlers.ts           # Incoming message type dispatch
|   |   |
|   |   |-- lib/                              # Shared utilities
|   |   |   |-- api-client.ts                 # HTTP client wrapper (fetch-based)
|   |   |   |-- base64.ts                     # Base64/Base64URL encoding helpers
|   |   |   |-- validators.ts                 # Input validation functions
|   |   |   +-- date.ts                       # Date formatting utilities
|   |   |
|   |   |-- types/                            # TypeScript type definitions
|   |   |   |-- encryption.ts                 # Crypto-related types and interfaces
|   |   |   |-- auth.ts                       # Authentication types
|   |   |   |-- chat.ts                       # Chat message and room types
|   |   |   |-- websocket.ts                  # WebSocket message types
|   |   |   |-- api.ts                        # API response types
|   |   |   |-- components.ts                 # Component prop types
|   |   |   +-- guards.ts                     # Type guard functions
|   |   |
|   |   |-- App.tsx                           # Root component with router
|   |   |-- config.ts                         # Environment configuration
|   |   |-- index.tsx                         # Application entry point
|   |   +-- index.css                         # Global styles and Tailwind imports
|   |
|   |-- vite.config.ts                        # Vite build configuration
|   |-- tsconfig.json                         # TypeScript compiler options
|   |-- eslint.config.js                      # ESLint configuration
|   +-- package.json                          # pnpm dependencies
|
|-- conf/
|   |-- docker/
|   |   |-- dev/
|   |   |   |-- fastapi.docker                # Dev Dockerfile for backend (hot reload)
|   |   |   +-- vite.docker                   # Dev Dockerfile for frontend (HMR)
|   |   +-- prod/
|   |       |-- fastapi.docker                # Production Dockerfile for backend
|   |       +-- vite.docker                   # Production Dockerfile for frontend (static build)
|   +-- nginx/
|       |-- dev.nginx                         # Development Nginx config
|       |-- prod.nginx                        # Production Nginx config
|       +-- http.conf                         # Shared HTTP settings
|
|-- learn/                                    # Educational documentation (you are here)
|-- .env.example                              # Template environment variables
|-- compose.yml                               # Production Docker Compose
|-- dev.compose.yml                           # Development Docker Compose
|-- justfile                                  # Command runner recipes
+-- LICENSE

Architecture Overview

The following diagram shows how data flows through the system during an encrypted message exchange:

+------------------+                                    +------------------+
|    Alice's       |                                    |     Bob's        |
|    Browser       |                                    |     Browser      |
|                  |                                    |                  |
|  +------------+  |                                    |  +------------+  |
|  | IndexedDB  |  |                                    |  | IndexedDB  |  |
|  | (private   |  |                                    |  | (private   |  |
|  |  keys)     |  |                                    |  |  keys)     |  |
|  +-----+------+  |                                    |  +-----+------+  |
|        |         |                                    |        |         |
|  +-----v------+  |                                    |  +-----v------+  |
|  | WebCrypto  |  |                                    |  | WebCrypto  |  |
|  | X3DH +     |  |                                    |  | X3DH +     |  |
|  | Ratchet    |  |                                    |  | Ratchet    |  |
|  +-----+------+  |                                    |  +-----+------+  |
|        |         |                                    |        ^         |
|  plaintext       |                                    |  plaintext       |
|  encrypted to    |                                    |  decrypted from  |
|  ciphertext      |                                    |  ciphertext      |
|        |         |                                    |        |         |
|  +-----v------+  |         +----------------+         |  +-----+------+  |
|  | WebSocket  +--+-------->|    Server      +-------->+--+ WebSocket  |  |
|  | Client     |  |         |                |         |  | Client     |  |
|  +------------+  |         | +------------+ |         |  +------------+  |
|                  |         | | FastAPI    | |         |                  |
+------------------+         | | WebSocket  | |         +------------------+
                             | | Router     | |
                             | +-----+------+ |
                             |       |        |
                             | +-----v------+ |
                             | | SurrealDB  | |
                             | | (encrypted | |
                             | |  blobs)    | |
                             | +------------+ |
                             |                |
                             | +------------+ |
                             | | PostgreSQL | |
                             | | (auth,     | |
                             | |  keys)     | |
                             | +------------+ |
                             |                |
                             | +------------+ |
                             | | Redis      | |
                             | | (challenges| |
                             | |  sessions) | |
                             | +------------+ |
                             +----------------+

Key points:

  1. Plaintext never leaves the browser. Encryption happens in the WebCrypto API before the message hits the WebSocket.
  2. The server sees only ciphertext. It stores and forwards encrypted blobs. It cannot decrypt them.
  3. Private keys live in IndexedDB. They are generated by the WebCrypto API and stored client-side. The server stores only public keys.
  4. Three databases serve distinct purposes. PostgreSQL handles relational auth data, SurrealDB handles real-time message storage, and Redis handles ephemeral state like challenge tokens.

Next Steps

The remaining documents in this learn folder walk through the project layer by layer:

  • 01-CONCEPTS.md -- Signal Protocol theory, X3DH mathematics, Double Ratchet mechanics, WebAuthn internals, and the real-world breaches that motivated these protocols. This is where you build the mental model for why each cryptographic decision was made.

  • 02-ARCHITECTURE.md -- System design, database schemas, data flow diagrams, security architecture, and the reasoning behind each technology choice. Covers the three-database split, the WebSocket message protocol, and the zero-knowledge server design.

  • 03-IMPLEMENTATION.md -- Line-by-line walkthrough of the cryptographic implementations, authentication flows, WebSocket messaging, and key lifecycle management. Follows the code through a complete message exchange from registration through key upload, session establishment, message encryption, delivery, and decryption.

  • 04-CHALLENGES.md -- Extension projects ranging from adding read receipts and typing indicators to implementing group encryption (Sender Keys), multi-device support, disappearing messages, and post-quantum key exchange using the liboqs-python library already included in the project dependencies.

Common Issues

Docker services will not start: Check that ports 5432, 8001, 6379, 8000, 5173, and 80 are not already in use. Note that SurrealDB maps to host port 8001 (not 8000, which is used by the FastAPI backend). Run docker compose -f dev.compose.yml down -v to remove stale containers and volumes, then restart. If a specific service fails, check its logs with just dev-logs postgres (or whichever service name).

WebAuthn registration fails in development: WebAuthn requires either localhost or a valid HTTPS origin. The default configuration uses RP_ID=localhost and RP_ORIGIN=http://localhost in .env.example. Make sure your .env file has these values set correctly. If you are accessing the frontend at http://localhost:5173, then RP_ORIGIN must match that exact origin (scheme + host + port). If you are testing from a LAN IP address or a non-localhost hostname, you will need a TLS certificate because WebAuthn will refuse to create credentials on non-secure origins other than localhost.

"No identity key" error when starting a chat: After WebAuthn registration, the client must generate and upload public encryption keys (identity key pair publics, signed prekey + signature, one-time prekey publics) to the server. This happens in crypto-service.ts:initialize, which then calls uploadPublicKeys → POST /encryption/upload-keys/{user_id}. If this step fails silently, the user will have an account but no published prekey bundle, and other users will get a 400 "User has not uploaded keys yet" when trying to initiate an X3DH session. Check the browser console for errors during the key generation step. Common causes include IndexedDB access being blocked (private browsing mode in some browsers restricts IndexedDB), WebCrypto API not being available (requires a secure context), or the session cookie not being sent (verify the request has credentials: 'include', which the shared api-client.ts already sets).

Database migration errors: If you see schema mismatch errors on startup, run migrations first. For Docker: just migrate head. For local development: just migrate-local head. If you need to start fresh, just dev-down followed by removing the Docker volumes will give you a clean database.

SurrealDB connection refused: SurrealDB runs on port 8000 inside the container but is mapped to port 8001 on the host. The backend connects to ws://surrealdb:8000 (the internal Docker network address). If you are running the backend outside Docker, update SURREAL_URL in .env to ws://localhost:8001.

Frontend WebSocket disconnects: The WebSocket client has automatic reconnection built in, but if the backend is not running or the connection URL is wrong, you will see repeated disconnection/reconnection attempts in the browser console. Verify that VITE_WS_URL in the frontend .env matches the actual backend WebSocket endpoint. The default is ws://localhost:8000.

Related Projects

  • API Rate Limiter (advanced) -- Implements Redis-based sliding window and token bucket rate limiting. The same Redis patterns (TTL-based key expiration, atomic increment operations) are used in this project for WebAuthn challenge storage and message rate limiting at 60 messages per minute per user.

  • Bug Bounty Platform (advanced) -- Another fullstack project combining authentication, real-time features, and multi-database architecture. Shares architectural patterns like the FastAPI app factory, service layer abstraction, and Docker Compose orchestration.

  • Simple Port Scanner (beginner) -- Introduces TCP networking fundamentals including socket programming, port states, and connection handshakes. These network-layer concepts underpin the WebSocket transport used for real-time message delivery in this project.