13 KiB

Raw Blame History

Building a Production-Grade Haskell Reverse Proxy for WebSocket and HTTP Streaming

The root cause of nginx's WebSocket/streaming conflict lies in mutually exclusive Connection header requirements combined with fundamentally different connection state semantics. Modern proxies like HAProxy, Caddy, and Envoy avoid this through unified state machines that treat protocol upgrades as first-class citizens. A Haskell implementation using WAI's responseRaw with the splice package for zero-copy forwarding can achieve superior architecture without these conflicts.

The nginx conflict stems from header and buffer incompatibilities

The nginx WebSocket/SSE conflict is architectural, not a bug. WebSocket requires Connection: upgrade to signal protocol switch, while SSE requires Connection: "" (empty) for persistent HTTP streaming. When using nginx's common map block for WebSocket handling, non-WebSocket requests (including SSE) receive Connection: close, which terminates the persistent connection needed for streaming.

The secondary conflict involves buffering semantics. SSE requires proxy_buffering off to pass each event immediately as it arrives. WebSocket operates in a special tunnel mode that bypasses normal response buffering entirely, using exactly two memory buffers after the 101 handshake. While proxy_buffering off doesn't directly break WebSocket, these configurations target fundamentally different connection state models.

nginx strips hop-by-hop headers (Upgrade, Connection) by default, requiring explicit re-addition for WebSocket. This manual header passthrough conflicts with SSE's empty Connection header requirement. The documentation confirms: "Hop-by-hop headers including 'Upgrade' and 'Connection' are not passed from a client to proxied server" by default.

Timeout semantics compound the problem. nginx's proxy_read_timeout applies uniformly to all traffic, including upgraded WebSockets. Since nginx treats WebSocket as an opaque TCP tunnel after handshake, it cannot parse WebSocket frames—meaning ping/pong frames don't reset the timer. Production deployments often set year-long timeouts (31536000s) as workaround, risking resource leaks from silent connection failures.

Modern proxies solve this through unified connection state machines

HAProxy elegantly handles this through a two-phase connection model. During the HTTP setup phase, it works in Layer 7 mode processing requests normally. Upon detecting successful upgrade (HTTP 101), HAProxy automatically switches to tunnel mode where no data is analyzed—WebSocket frames pass through as raw TCP. The key innovation is timeout tunnel, a separate timeout that takes precedence after upgrade, allowing short HTTP timeouts (25s) alongside long WebSocket timeouts (3600s).

Caddy takes the simplest approach: WebSocket proxying requires zero configuration in version 2. Caddy performs HTTP upgrade requests automatically, transitions the connection to a bidirectional tunnel, and maintains a registry for graceful shutdown. For streaming, Caddy uses intelligent automatic flush detection—when it sees Content-Type: text/event-stream or unknown Content-Length, it flushes immediately without buffering.

Envoy provides the most explicit configuration through upgrade_configs that explicitly enable protocol upgrades per-route or globally. After upgrade, Envoy treats WebSocket connections as plain TCP connections, independent of wire format. Slack successfully migrated millions of concurrent WebSockets from HAProxy to Envoy, citing hot restart without dropping connections and dynamic endpoint updates without configuration reload.

The common pattern across all successful implementations: detect upgrade headers → validate 101 response → transition to tunnel mode → apply separate timeout domain. Buffering filters are automatically excluded from upgraded connections—Envoy explicitly documents that "buffering is generally not compatible with upgrades."

Haskell's WAI ecosystem supports transparent proxy architecture

For a Haskell reverse proxy, the recommended approach uses WAI's responseRaw combined with raw byte forwarding rather than full WebSocket frame parsing. The http-reverse-proxy package demonstrates this pattern: after detecting WebSocket upgrade via isWebSocketsReq, it creates bidirectional conduit streams between client and backend sockets.

handleWebSocketProxy :: Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
handleWebSocketProxy req respond = respond $
  responseRaw (rawProxy targetHost targetPort) backupResponse

The websockets library (version 0.13.0.0) is production-quality and actively maintained since 2011, integrating with WAI via wai-websockets. It exposes multiple abstraction levels: high-level message-oriented APIs, mid-level Message types including control frames, and low-level Stream module for custom connection sources. However, for a transparent proxy, full frame parsing introduces unnecessary overhead.

Warp does not have built-in WebSocket handling—it relies on external libraries. The key mechanism is responseRaw, introduced specifically to enable reverse proxying WebSockets. Michael Snoyman noted: "Now with responseRaw support, all users of http-reverse-proxy automatically get WebSockets support."

Memory considerations matter for high-connection scenarios. The websockets library's per-connection overhead comes from GHC runtime stack space, per-message buffering (configurable via SizeLimit), and compression state (~64KB with defaults). ByteString uses pinned memory that cannot be moved by GC, potentially causing heap fragmentation with many small strings. For stored metadata, prefer ShortByteString (unpinned memory).

Zero-copy forwarding matches production proxy behavior

Production proxies universally use opaque/tunnel-mode forwarding for WebSocket traffic after HTTP upgrade. HAProxy documentation states: "If everything goes well, the websocket is established, then HAProxy fails over to tunnel mode, no data is analyzed anymore." Envoy similarly "does not interpret the websocket frames."

The Haskell splice package provides production-ready socket-to-socket data transfer. On GNU/Linux, it uses the zero-copy splice() system call; on other platforms, it falls back to a portable implementation with single memory buffer:

-- Bidirectional proxy loop
void . forkIO . try_ $ splice 1024 (clientSocket, _) (backendSocket, _)
void . forkIO . try_ $ splice 1024 (backendSocket, _) (clientSocket, _)

Protocol requirements that force frame parsing include: client-to-server frame masking validation, per-message compression (permessage-deflate extension), frame validation for security, and control frame handling at proxy level. However, for transparent proxying, these requirements don't apply—masking happens at endpoints, and control frames pass through unchanged.

A critical performance insight from kernel developers: splice() can actually be slower than naive read/write for small transfers due to abstraction overhead. Cloudflare's analysis shows that splice "copies data by assigning physical memory page pointers" rather than byte copying, but the syscall overhead matters. Recommended chunk size is 64KB (matching kernel pipe size) for optimal splice performance.

Strategy	CPU Usage	Memory	Visibility	Production Use
Zero-copy (opaque)	Minimal	Buffer only	Connection-level	HAProxy, Envoy, Traefik
Parsed frames	Higher	Buffer + structures	Full messages	Security proxies

Ping/pong handling should be transparent with proxy-initiated health checks

RFC 6455 permits intermediaries to "coalesce or split" fragmented messages, but does not specify how proxies should handle ping/pong frames. The protocol expects pings to come from actual endpoints for true end-to-end liveness verification.

Transparent forwarding preserves end-to-end semantics, provides accurate RTT measurements, and requires no WebSocket parsing after handshake. This is what nginx and HAProxy use in tunnel mode. The downside: proxy cannot independently detect if backend dies silently.

Proxy-level handling (intercepting pings, responding with pongs) breaks end-to-end semantics—the server thinks a connection is alive when the client may be dead. Traefik received criticism for this behavior (GitHub issue #3683): "Traefik will answer all Ping messages and drop all Pong messages instead of forwarding them."

The recommended hybrid approach: forward all ping/pong frames transparently while implementing proxy-level connection monitoring. Track last activity timestamp per connection; after configurable idle period, proxy sends its own independent pings to both sides; terminate connections that don't respond within timeout; reset activity timer on ANY frame (data or control).

Per-connection state:
  last_activity: timestamp
  pong_expected: bool

Periodic check (every 10s):
  if pong_expected and (now - ping_sent_at) > pong_timeout:
    terminate_connection("pong timeout")
  elif (now - last_activity) > idle_threshold:
    send_ping(); pong_expected = true

Timeout architecture requires protocol-aware separation

The fundamental problem: different protocols have vastly different lifetime expectations. HTTP requests complete in 30-300 seconds. WebSocket connections may live for hours to days, possibly legitimately idle. SSE/streaming connections are long-lived but unidirectional.

nginx's timeout problem stems from proxy_read_timeout applying uniformly to all traffic. Combined with WebSocket frame opacity in tunnel mode, ping frames cannot reset the timer. Common workaround of year-long timeouts creates resource leak risk—"you won't detect silent backend netsplits."

HAProxy's model provides the blueprint for proper timeout architecture:

defaults
  timeout client    25s     # HTTP inactivity
  timeout server    25s     # HTTP inactivity
  timeout connect   5s      # TCP establishment
  timeout tunnel    3600s   # WebSocket/upgraded connections

The key insight: timeout tunnel automatically applies after successful protocol upgrade, allowing short HTTP timeouts but long WebSocket timeouts without configuration conflicts.

Recommended timeout architecture for Haskell proxy:

HTTP idle timeout: 60 seconds
WebSocket tunnel timeout: 3600 seconds (1 hour)
Proxy ping interval: 30 seconds (must be shorter than tunnel timeout)
Pong response timeout: 10 seconds
SSE/streaming timeout: 3600 seconds
Per-route overrides: Essential for diverse application requirements

Auto-detect connection type via Connection: Upgrade + Upgrade: websocket headers, response code 101, or Content-Type: text/event-stream for SSE.

Architectural design principles for the Haskell proxy

The connection state machine should explicitly model different phases:

data ConnectionState
    = HttpRequest          -- Initial HTTP parsing
    | HttpResponse         -- Response from upstream
    | ProtocolUpgrade      -- Detected upgrade, awaiting 101
    | TunnelMode           -- Bidirectional byte stream (WebSocket)
    | StreamingResponse    -- Chunked/SSE, flush immediately

transitionState :: ConnectionState -> Event -> ConnectionState

Buffer management strategy:

Default: Small buffers with periodic flushing for wire efficiency
Streaming detection: Zero-copy pass-through when text/event-stream detected
Upgrade connections: No buffering, direct pipe via splice
Backpressure: Watermark-based flow control (Envoy model)

Key implementation decisions to avoid nginx's problems:

Preserve Upgrade headers: Don't strip hop-by-hop headers for reverse proxy use case
Detect Content-Type early: Check for text/event-stream before buffering decisions
Separate timeout pools: Long-lived connections get different timeout handling automatically
Connection registry: Track upgraded connections for graceful shutdown
WebSocket frame awareness: Even in tunnel mode, optionally parse control frames to reset idle timer and enable proxy-level health checks

Recommended Haskell stack:

wai + warp: HTTP server foundation
streaming-commons: TCP client connections
splice: Zero-copy socket forwarding on Linux
async: Concurrent bidirectional copy with race_
websockets + wai-websockets: Only if frame-level access needed

Architecture flow:

Listener → HTTP Parser → Route Match →
  ├─ Regular HTTP → Buffer Pool → Upstream → Response Buffer → Client
  ├─ WebSocket → Upgrade Handler → Bidirectional Tunnel (splice)
  └─ SSE/Stream → Detect Header → Immediate Flush → Client

The fundamental architectural advantage over nginx: a single, unified routing path that auto-detects protocol type and transitions connection state appropriately, rather than requiring separate location blocks with conflicting header/buffering configurations. By treating streaming and WebSocket as natural variations of connection state rather than special cases requiring manual configuration, the Haskell proxy can achieve the elegant behavior of modern proxies like Caddy while maintaining the performance characteristics of HAProxy's tunnel mode.

13 KiB Raw Blame History