wse-client 1.4.3 → 2.0.0

package/README.md

@@ -1,469 +1,539 @@
-# WSE -- WebSocket Engine
+# WSE - WebSocket Engine
 
-**A complete, out-of-the-box solution for real-time communication between React, Python, and backend services.**
-
-Three packages. Four lines of code. Your frontend and backend talk in real time.
-
-[![CI](https://github.com/silvermpx/wse/actions/workflows/ci.yml/badge.svg)](https://github.com/silvermpx/wse/actions/workflows/ci.yml)
 [![PyPI - Server](https://img.shields.io/pypi/v/wse-server)](https://pypi.org/project/wse-server/)
 [![PyPI - Client](https://img.shields.io/pypi/v/wse-client)](https://pypi.org/project/wse-client/)
 [![npm](https://img.shields.io/npm/v/wse-client)](https://www.npmjs.com/package/wse-client)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
----
-
-## Why WSE?
+High-performance WebSocket server built in Rust with native clustering, message recovery, presence tracking, and real-time fan-out. Exposed to Python via PyO3 with zero GIL overhead on the data path.
+
+## Features
+
+### Server
+
+| Feature | Details |
+|---------|---------|
+| **Rust core** | tokio async runtime, tungstenite WebSocket transport, dedicated thread pool, zero GIL on the data path |
+| **JWT authentication** | Rust-native HS256 validation during handshake (0.01ms), cookie + Authorization header extraction |
+| **Protocol negotiation** | `client_hello`/`server_hello` handshake with feature discovery, capability advertisement, version agreement |
+| **Topic subscriptions** | Per-connection topic subscriptions with automatic cleanup on disconnect |
+| **Pre-framed broadcast** | WebSocket frame built once, shared via Arc across all connections, single allocation per broadcast |
+| **Vectored writes** | `write_vectored` (writev syscall) batches multiple frames per connection in a single kernel call |
+| **Write coalescing** | Write task drains up to 256 pending frames per iteration via `recv_many` |
+| **DashMap state** | Lock-free sharded concurrent hash maps for topics, rates, formats, activity tracking |
+| **mimalloc allocator** | Global allocator optimized for multi-threaded workloads with frequent small allocations |
+| **Deduplication** | 50,000-entry AHashSet with FIFO eviction per `send_event()` call |
+| **Rate limiting** | Per-connection token bucket: 100K capacity, 10K/s refill, client warning at 20% remaining |
+| **Zombie detection** | Server pings every 25s, force-closes connections with no activity for 60s |
+| **Drain mode** | Lock-free crossbeam bounded channel, Python acquires GIL once per batch (not per event) |
+| **Compression** | zlib for client-facing messages above threshold (default 1024 bytes) |
+| **MessagePack** | Opt-in binary transport via `?format=msgpack`, roughly 2x faster serialization, 30% smaller |
+| **Message signing** | Selective HMAC-SHA256 signing for critical operations, nonce-based replay prevention |
 
-Building real-time features between React and Python is painful. You need WebSocket handling, reconnection logic, message ordering, authentication, encryption, offline support, health monitoring. That's weeks of work before you ship a single feature.
-
-**WSE gives you all of this out of the box.**
-
-Install `wse-server` on your backend, `wse-client` on your frontend (React or Python). Everything works immediately: auto-reconnection, message encryption, sequence ordering, offline queues, health monitoring. No configuration required for the defaults. Override what you need.
+### End-to-End Encryption
 
-The engine is Rust-accelerated via PyO3. **14M msg/s** JSON, **30M msg/s** binary, **2.1M deliveries/s** fan-out, **500K concurrent connections** with zero message loss -- benchmarked on AMD EPYC 7502P. Multi-instance horizontal scaling via Redis pub/sub. Sub-millisecond latency (0.38 ms) with Rust JWT authentication.
+| Feature | Details |
+|---------|---------|
+| **Key exchange** | ECDH P-256 (per-connection keypair, automatic during handshake) |
+| **Encryption** | AES-GCM-256 with unique 12-byte IV per message |
+| **Key derivation** | HKDF-SHA256 (salt: `wse-encryption`, info: `aes-gcm-key`) |
+| **Wire format** | `E:` prefix + 12-byte IV + AES-GCM ciphertext + 16-byte auth tag |
+| **Key rotation** | Configurable rotation interval (default 1 hour), automatic renegotiation |
+| **Replay prevention** | Nonce cache (10K entries, 5-minute TTL) on the client side |
+
+### Cluster Protocol
+
+| Feature | Details |
+|---------|---------|
+| **Topology** | Full TCP mesh, direct peer-to-peer connections |
+| **Wire format** | Custom binary frames: 8-byte header + topic + payload, 12 message types |
+| **Interest routing** | SUB/UNSUB/RESYNC frames, messages forwarded only to peers with matching subscribers |
+| **Gossip discovery** | PeerAnnounce/PeerList frames, new nodes need one seed address to join |
+| **mTLS** | rustls + tokio-rustls, P-256 certificates, WebPkiClientVerifier for both sides |
+| **Compression** | zstd level 1 for payloads above 256 bytes, capability-negotiated, output capped at 1 MB |
+| **Heartbeat** | 5s ping interval, 15s timeout, dead peer detection |
+| **Circuit breaker** | 10 failures to open, 60s reset, 3 half-open probe calls |
+| **Dead letter queue** | 1000-entry ring buffer for failed cluster sends |
+| **Presence sync** | PresenceUpdate/PresenceFull frames, CRDT last-write-wins conflict resolution |
+
+### Presence Tracking
+
+| Feature | Details |
+|---------|---------|
+| **Per-topic tracking** | Which users are active in each topic, with custom metadata (status, avatar, etc.) |
+| **User-level grouping** | Multiple connections from same JWT `sub` share one presence entry |
+| **Join/leave lifecycle** | `presence_join` on first connection, `presence_leave` on last disconnect |
+| **O(1) stats** | `presence_stats()` returns member/connection counts without iteration |
+| **Data updates** | `update_presence()` broadcasts to all topics where the user is present |
+| **Cluster sync** | Synchronized across all nodes, CRDT last-write-wins resolution |
+| **TTL sweep** | Background task every 30s removes entries from dead connections |
+
+### Message Recovery
+
+| Feature | Details |
+|---------|---------|
+| **Ring buffers** | Per-topic, power-of-2 capacity, bitmask indexing (single AND instruction) |
+| **Epoch+offset tracking** | Precise recovery positioning, epoch changes on buffer recreation |
+| **Memory management** | Global budget (default 256 MB), TTL eviction, LRU eviction when over budget |
+| **Zero-copy storage** | Recovery entries share `Bytes` (Arc) with the broadcast path |
+| **Recovery on reconnect** | `subscribe_with_recovery()` replays missed messages automatically |
+
+### Client SDKs (Python + TypeScript/React)
+
+| Feature | Details |
+|---------|---------|
+| **Auto-reconnection** | 4 strategies: exponential, linear, fibonacci, adaptive backoff with jitter |
+| **Connection pool** | Multi-endpoint with health scoring, 3 load balancing strategies, automatic failover |
+| **Circuit breaker** | CLOSED/OPEN/HALF_OPEN state machine, prevents connection storms |
+| **Rate limiting** | Client-side token bucket, coordinates with server feedback |
+| **E2E encryption** | Wire-compatible AES-GCM-256 + ECDH P-256 (both clients speak the same protocol) |
+| **Event sequencing** | Duplicate detection (sliding window) + out-of-order buffering |
+| **Network monitor** | Real-time latency, jitter, packet loss measurement, quality scoring |
+| **Priority queues** | 5 levels from CRITICAL to BACKGROUND |
+| **Offline queue** | IndexedDB persistence (TypeScript), replayed on reconnect |
+| **Compression** | Automatic zlib for messages above threshold |
+| **MessagePack** | Binary encoding for smaller payloads and faster serialization |
+| **Message signing** | HMAC-SHA256 integrity verification |
+
+### Transport Security
+
+| Feature | Details |
+|---------|---------|
+| **Origin validation** | `ALLOWED_ORIGINS` env var, rejects unlisted origins with close code 4403 |
+| **Cookie auth** | `access_token` HTTP-only cookie with `Secure + SameSite=Lax` (OWASP recommended for browsers) |
+| **Frame protection** | 1 MB max frame size, serde_json parsing (no eval), escaped user IDs in server_ready |
+| **Cluster frame protection** | zstd decompression output capped at 1 MB (MAX_FRAME_SIZE), protocol version validation |
 
 ---
 
 ## Quick Start
 
-### Server (Python) -- Router Mode
-
-Embed WSE into your existing FastAPI app on the same port:
+```bash
+pip install wse-server
+```
 
 ```python
-from fastapi import FastAPI
-from wse_server import create_wse_router, WSEConfig
-import redis.asyncio as redis
+from wse_server import RustWSEServer, rust_jwt_encode
+import time, threading
 
-app = FastAPI()
+server = RustWSEServer(
+    "0.0.0.0", 5007,
+    max_connections=10_000,
+    jwt_secret=b"replace-with-a-strong-secret-key!",
+    jwt_issuer="my-app",
+    jwt_audience="my-api",
+)
+server.enable_drain_mode()
+server.start()
 
-redis_client = redis.Redis(host="localhost", port=6379, decode_responses=False)
+def handle_events(srv):
+    while True:
+        for ev in srv.drain_inbound(256, 50):
+            if ev[0] == "auth_connect":
+                srv.subscribe_connection(ev[1], ["updates"])
+            elif ev[0] == "msg":
+                print(f"Message from {ev[1]}: {ev[2]}")
+            elif ev[0] == "disconnect":
+                print(f"Disconnected: {ev[1]}")
 
-wse = create_wse_router(WSEConfig(
-    redis_client=redis_client,
-))
+threading.Thread(target=handle_events, args=(server,), daemon=True).start()
 
-app.include_router(wse, prefix="/wse")
+while server.is_running():
+    time.sleep(1)
 ```
 
-Publish events from anywhere in your app via the PubSub bus:
+Generate a test token:
 
 ```python
-bus = app.state.pubsub_bus
-
-await bus.publish(
-    topic="notifications",
-    event={"event_type": "order_shipped", "order_id": 42, "text": "Order shipped!"},
+token = rust_jwt_encode(
+    {"sub": "user-1", "iss": "my-app", "aud": "my-api",
+     "exp": int(time.time()) + 3600, "iat": int(time.time())},
+    b"replace-with-a-strong-secret-key!",
 )
 ```
 
-### Server (Python) -- Standalone Mode
+---
 
-Run the Rust WebSocket server on a dedicated port for maximum throughput:
+## Server Configuration
+
+`RustWSEServer` constructor parameters:
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `host` | required | Bind address |
+| `port` | required | Bind port |
+| `max_connections` | 1000 | Maximum concurrent WebSocket connections |
+| `jwt_secret` | None | HS256 secret for JWT validation (bytes, min 32 bytes). `None` disables authentication |
+| `jwt_issuer` | None | Expected `iss` claim. Skipped if `None` |
+| `jwt_audience` | None | Expected `aud` claim. Skipped if `None` |
+| `max_inbound_queue_size` | 131072 | Drain mode bounded queue capacity |
+| `recovery_enabled` | False | Enable per-topic message recovery buffers |
+| `recovery_buffer_size` | 128 | Ring buffer slots per topic (rounded to power-of-2) |
+| `recovery_ttl` | 300 | Buffer TTL in seconds before eviction |
+| `recovery_max_messages` | 500 | Max messages returned per recovery response |
+| `recovery_memory_budget` | 268435456 | Global memory limit for all recovery buffers (bytes, default 256 MB) |
+| `presence_enabled` | False | Enable per-topic presence tracking |
+| `presence_max_data_size` | 4096 | Max bytes for a user's presence metadata |
+| `presence_max_members` | 0 | Max tracked members per topic (0 = unlimited) |
 
-```python
-from wse_server._wse_accel import RustWSEServer
+---
 
-server = RustWSEServer(
-    "0.0.0.0", 5006,
-    max_connections=10000,
-    jwt_secret=b"your-secret-key",     # Rust JWT validation in handshake
-    jwt_issuer="your-app",
-    jwt_audience="your-api",
-)
-server.start()
+## API Reference
+
+### Lifecycle
 
-# Drain inbound events in a background thread
-while True:
-    events = server.drain_inbound(256, 50)  # batch size, timeout ms
-    for event in events:
-        handle(event)
+```python
+server.start()                          # Start the server
+server.stop()                           # Graceful shutdown
+server.is_running()                     # Check server status (bool)
 ```
 
-Standalone mode gives you a dedicated Rust tokio runtime on its own port -- no FastAPI overhead, no GIL on the hot path. This is how WSE achieves 14M msg/s on JSON.
+### Event Handling
 
-### Client (React)
+**Drain mode** (recommended) - events are queued in a lock-free crossbeam channel. Python polls in batches, acquiring the GIL once per batch.
 
-```tsx
-import { useWSE } from 'wse-client';
+```python
+server.enable_drain_mode()              # Switch to batch-polling mode (call before start)
+events = server.drain_inbound(256, 50)  # Poll up to 256 events, wait up to 50ms
+```
 
-function Dashboard() {
-  const { isConnected, connectionHealth } = useWSE({
-    topics: ['notifications', 'live_data'],
-    endpoints: ['ws://localhost:8000/wse'],
-  });
+Each event is a tuple: `(event_type, conn_id, payload)`
 
-  useEffect(() => {
-    const handler = (e: CustomEvent) => {
-      console.log('New notification:', e.detail);
-    };
-    window.addEventListener('notifications', handler);
-    return () => window.removeEventListener('notifications', handler);
-  }, []);
+| Event Type | Trigger | Payload |
+|------------|---------|---------|
+| `"auth_connect"` | JWT-validated connection | user_id (string) |
+| `"connect"` | Connection without JWT | cookies (string) |
+| `"msg"` | Client sent WSE-prefixed JSON | parsed dict |
+| `"raw"` | Client sent plain text | raw string |
+| `"bin"` | Client sent binary frame | bytes |
+| `"disconnect"` | Connection closed | None |
+| `"presence_join"` | User's first connection joined a topic | dict with user_id, topic, data |
+| `"presence_leave"` | User's last connection left a topic | dict with user_id, topic, data |
 
-  return <div>Status: {connectionHealth}</div>;
-}
-```
+**Callback mode** - alternative to drain mode. Callbacks are invoked via `spawn_blocking` per event.
 
-That's it. Your React app receives real-time updates from your Python backend.
+```python
+server.set_callbacks(on_connect, on_message, on_disconnect)
+```
 
-### Client (Python)
+### Sending Messages
 
 ```python
-from wse_client import connect
-
-async with connect("ws://localhost:5006/wse", token="your-jwt") as client:
-    await client.subscribe(["notifications", "live_data"])
-    async for event in client:
-        print(event.type, event.payload)
+server.send(conn_id, text)                      # Send text to one connection
+server.send_bytes(conn_id, data)                 # Send binary to one connection
+server.send_event(conn_id, event_dict)           # Send structured event (auto-serialized, deduped, rate-checked)
+
+server.broadcast_all(text)                       # Send to every connected client (text)
+server.broadcast_all_bytes(data)                 # Send to every connected client (binary)
+server.broadcast_local(topic, text)              # Fan-out to topic subscribers on this instance
+server.broadcast(topic, text)                    # Fan-out to topic subscribers across all cluster nodes
 ```
 
-Same wire protocol, same features. Use the Python client for backend-to-backend communication, microservices, CLI tools, integration tests, or any non-browser use case.
+| Method | Scope | Notes |
+|--------|-------|-------|
+| `send` | Single connection | Raw text frame |
+| `send_bytes` | Single connection | Raw binary frame |
+| `send_event` | Single connection | JSON-serialized, compressed if above threshold, deduplication via 50K-entry FIFO window |
+| `broadcast_all` | All connections | Pre-framed, single frame build shared via Arc |
+| `broadcast_local` | Topic (local) | Pre-framed, DashMap subscriber lookup, stored in recovery buffer if enabled |
+| `broadcast` | Topic (all nodes) | Local fan-out + forwarded to cluster peers with matching interest |
 
----
+### Topic Subscriptions
 
-## What You Get Out of the Box
+```python
+server.subscribe_connection(conn_id, ["prices", "news"])              # Subscribe to topics
+server.subscribe_connection(conn_id, ["chat"], {"status": "online"})  # Subscribe with presence data
+server.unsubscribe_connection(conn_id, ["news"])                      # Unsubscribe from specific topics
+server.unsubscribe_connection(conn_id, None)                          # Unsubscribe from all topics
+server.get_topic_subscriber_count("prices")                           # Subscriber count for a topic
+```
 
-Everything listed below works the moment you install. No extra setup.
+Subscriptions are cleaned up automatically on disconnect. In cluster mode, interest changes are propagated to peers via SUB/UNSUB frames.
 
-### Reactive Interface
+### Presence Tracking
 
-Real-time data flow from Python to React. One hook (`useWSE`) on the client, one `publish()` call on the server. Events appear in your components instantly.
+Requires `presence_enabled=True` in the constructor.
 
-### Auto-Reconnection
+```python
+# Query members in a topic
+members = server.presence("chat-room")
+# {"alice": {"data": {"status": "online"}, "connections": 2},
+#  "bob":   {"data": {"status": "away"},   "connections": 1}}
 
-Exponential backoff with jitter. Connection drops? The client reconnects automatically. No lost messages -- offline queue with IndexedDB persistence stores messages while disconnected and replays them on reconnect.
+# Lightweight counts (O(1), no iteration)
+stats = server.presence_stats("chat-room")
+# {"num_users": 2, "num_connections": 3}
 
-### End-to-End Encryption
+# Update a user's presence data across all their subscribed topics
+server.update_presence(conn_id, {"status": "away"})
+```
 
-ECDH P-256 key exchange with AES-GCM-256 per connection. Each connection negotiates a unique session key during the handshake (server_ready / client_hello), then all sensitive messages are encrypted end-to-end. HMAC-SHA256 message signing for integrity. Pluggable encryption and token providers via Python protocols.
+Presence is tracked at the user level (JWT `sub` claim). Multiple connections from the same user share a single presence entry. `presence_join` fires on first connection, `presence_leave` on last disconnect. In cluster mode, presence state is synchronized across all nodes using CRDT last-write-wins resolution.
 
-### Message Ordering
+### Message Recovery
 
-Sequence numbers with gap detection and reordering buffer. Messages arrive in order even under high load or network instability. Out-of-order messages are buffered and delivered once the gap fills.
+Requires `recovery_enabled=True` in the constructor.
 
-### Authentication
+```python
+result = server.subscribe_with_recovery(
+    conn_id, ["prices"],
+    recover=True,
+    epoch=client_epoch,       # From previous session
+    offset=client_offset,     # From previous session
+)
+# {"topics": {"prices": {"epoch": 123, "offset": 456, "recovered": True, "count": 12}}}
+```
 
-JWT-based with HS256, validated in Rust during the WebSocket handshake (zero GIL, 0.01ms decode). Per-connection, per-topic access control. Cookie-based token extraction for seamless browser auth. Fallback to Python auth handler when Rust JWT is not configured.
+The server maintains per-topic ring buffers (power-of-2 capacity, bitmask indexing). Clients store the `epoch` and `offset` from their last received message. On reconnect, the server replays missed messages from the ring buffer. If the gap is too large or the epoch has changed, the client receives a `NotRecovered` status and should re-subscribe from scratch.
 
-### Health Monitoring
+Memory is managed with a global budget (default 256 MB), TTL eviction for idle buffers, and LRU eviction when over budget.
 
-Connection quality scoring (excellent / good / fair / poor), latency tracking, jitter analysis, packet loss detection. Your UI knows when the connection is degraded and can react accordingly.
+### Cluster
 
-### Scaling
+```python
+# Join a cluster mesh with mTLS
+server.connect_cluster(
+    peers=["10.0.0.2:9999", "10.0.0.3:9999"],
+    tls_ca="/etc/wse/ca.pem",
+    tls_cert="/etc/wse/node.pem",
+    tls_key="/etc/wse/node.key",
+    cluster_port=9999,
+)
 
-Redis pub/sub for multi-instance fan-out. Run N server instances behind a load balancer -- publish on any instance, all subscribers receive the message regardless of which instance they're connected to. Pipelined PUBLISH (up to 64 per round-trip), circuit breaker, exponential backoff with jitter, dead letter queue for failed messages. Capacity scales linearly with instances: tested up to 1.04M deliveries/s per instance.
+# With gossip discovery (only seed addresses needed)
+server.connect_cluster(
+    peers=[],
+    seeds=["10.0.0.2:9999"],
+    cluster_addr="10.0.0.1:9999",
+    cluster_port=9999,
+)
 
-### Rust Performance
+server.cluster_connected()       # True if connected to at least one peer
+server.cluster_peers_count()     # Number of active peer connections
+```
 
-Compression, sequencing, filtering, rate limiting, and the WebSocket server itself are implemented in Rust via PyO3. Python API stays the same. Rust accelerates transparently.
+Nodes form a full TCP mesh automatically. The cluster protocol uses a custom binary frame format with an 8-byte header, 12 message types, and capability negotiation during handshake. Features:
 
----
+- **Interest-based routing** - SUB/UNSUB/RESYNC frames. Messages are only forwarded to peers with matching subscribers.
+- **Gossip discovery** - PeerAnnounce/PeerList frames. New nodes need one seed address to join.
+- **mTLS** - mutual TLS via rustls with P-256 certificates and WebPkiClientVerifier.
+- **zstd compression** - payloads above 256 bytes compressed at level 1, capability-negotiated.
+- **Circuit breaker** - 10 failures to open, 60s reset, 3 half-open probe calls.
+- **Heartbeat** - 5s interval, 15s timeout, dead peer detection.
+- **Dead letter queue** - 1000-entry ring buffer for failed cluster sends.
+- **Presence sync** - PresenceUpdate/PresenceFull frames with CRDT conflict resolution.
 
-## Full Feature List
+### Health Monitoring
 
-### Server (Python + Rust)
+```python
+health = server.health_snapshot()
+# {
+#     "connections": 150,
+#     "inbound_queue_depth": 0,
+#     "inbound_dropped": 0,
+#     "uptime_secs": 3600.5,
+#     "recovery_enabled": True,
+#     "recovery_topic_count": 5,
+#     "recovery_total_bytes": 1048576,
+#     "cluster_connected": True,
+#     "cluster_peer_count": 2,
+#     "cluster_messages_sent": 50000,
+#     "cluster_messages_delivered": 49950,
+#     "cluster_messages_dropped": 0,
+#     "cluster_bytes_sent": 1048576,
+#     "cluster_bytes_received": 1024000,
+#     "cluster_reconnect_count": 0,
+#     "cluster_unknown_message_types": 0,
+#     "cluster_dlq_size": 0,
+#     "presence_enabled": True,
+#     "presence_topics": 3,
+#     "presence_total_users": 25,
+# }
+```
 
-| Feature | Description |
-|---------|-------------|
-| **Drain Mode** | Batch-polling inbound events from Rust. One GIL acquisition per batch (up to 256 messages) instead of per-message Python callbacks. Condvar-based wakeup for zero busy-wait. |
-| **Write Coalescing** | Outbound pipeline: `feed()` + batch `try_recv()` + single `flush()`. Reduces syscalls under load by coalescing multiple messages into one write. |
-| **Ping/Pong in Rust** | Heartbeat handled entirely in Rust with zero Python round-trips. Configurable intervals. TCP_NODELAY on accept for minimal latency. |
-| **5-Level Priority Queue** | CRITICAL(10), HIGH(8), NORMAL(5), LOW(3), BACKGROUND(1). Smart dropping under backpressure: lower-priority messages are dropped first. Batch dequeue ordered by priority. |
-| **Dead Letter Queue** | Redis-backed DLQ for failed messages. 7-day TTL, 1000-message cap per channel. Manual replay via `replay_dlq_message()`. Prometheus metrics for DLQ size and replay count. |
-| **MongoDB-like Filters** | 14 operators: `$eq`, `$ne`, `$gt`, `$lt`, `$gte`, `$lte`, `$in`, `$nin`, `$regex`, `$exists`, `$contains`, `$startswith`, `$endswith`. Logical: `$and`, `$or`. Dot-notation for nested fields (`payload.price`). Compiled regex cache. |
-| **Event Sequencer** | Monotonic sequence numbers with AHashSet dedup. Size-based and age-based eviction. Gap detection on both server and client. |
-| **Compression** | Flate2 zlib with configurable levels (1-9). Adaptive threshold -- only compress when payload exceeds size limit. Binary mode via msgpack (rmp-serde) for 30% smaller payloads. |
-| **Rate Limiter** | Atomic token-bucket rate limiter in Rust. Per-connection rate enforcement. 100K tokens capacity, 10K tokens/sec refill. |
-| **Message Deduplication** | AHashSet-backed dedup with bounded queue. Prevents duplicate delivery across reconnections and Redis fan-out. |
-| **Wire Envelope** | Protocol v1: `{t, id, ts, seq, p, v}`. Generic payload extraction with automatic type conversion (UUID, datetime, Enum, bytes to JSON-safe primitives). Latency tracking (`latency_ms` field). |
-| **Snapshot Provider** | Protocol for initial state delivery. Implement `get_snapshot(user_id, topics)` and clients receive current state immediately on subscribe -- no waiting for the next publish cycle. |
-| **Circuit Breaker** | Three-state machine (CLOSED / OPEN / HALF_OPEN). Sliding-window failure tracking. Automatic recovery probes. Prevents cascade failures when downstream services are unhealthy. |
-| **Message Categories** | `S` (snapshot), `U` (update), `WSE` (system). Category prefixing for client-side routing and filtering. |
-| **Multi-Instance Orchestration** | Horizontal scaling via Redis pub/sub. Publish on any instance, all subscribers receive the message. Pipelined PUBLISH (64 commands/batch, 3 retries), circuit breaker (10-fail threshold, 60s reset), exponential backoff with jitter, dead letter queue (1000-entry ring buffer). 1.04M deliveries/s per instance, linear scaling with N instances. |
-| **PubSub Bus** | Redis pub/sub with PSUBSCRIBE pattern matching. Glob wildcard topic routing (`user:*:events`). orjson fast-path serialization. Non-blocking handler invocation. |
-| **Pluggable Security** | `EncryptionProvider` and `TokenProvider` protocols. Built-in: AES-GCM-256 with ECDH P-256 key exchange, HMAC-SHA256 signing, selective message signing. Rust-accelerated crypto (SHA-256, HMAC, AES-GCM, ECDH). |
-| **Rust JWT Authentication** | HS256 JWT validation in Rust during the WebSocket handshake. Zero GIL acquisition on the connection critical path. 0.01ms decode (85x faster than Python). Cookie extraction and `server_ready` sent from Rust before Python runs. |
-| **Lock-Free Server Queries** | `get_connection_count()` uses `AtomicUsize` — zero GIL, zero blocking, safe to call from async Python handlers. No channel round-trip to the tokio runtime. |
-| **Inbound MsgPack** | Binary frames from msgpack clients are parsed in Rust via rmpv. Python receives pre-parsed dicts regardless of wire format. Zero Python overhead for msgpack connections. |
-| **Connection Metrics** | Prometheus-compatible stubs for: messages sent/received, publish latency, DLQ size, handler errors, circuit breaker state. Drop-in Prometheus integration or use the built-in stubs. |
-
-### Client (React + TypeScript)
-
-| Feature | Description |
-|---------|-------------|
-| **useWSE Hook** | Single React hook for the entire WebSocket lifecycle. Accepts topics, endpoints, auth tokens. Returns `isConnected`, `connectionHealth`, connection controls. |
-| **Connection Pool** | Multi-endpoint support with health-scored failover. Three load-balancing strategies: weighted-random, least-connections, round-robin. Automatic health checks with latency tracking. |
-| **Adaptive Quality Manager** | Adjusts React Query defaults based on connection quality. Excellent: `staleTime: Infinity` (pure WebSocket). Poor: aggressive polling fallback. Dispatches `wse:quality-change` events. Optional QueryClient integration. |
-| **Offline Queue** | IndexedDB-backed persistent queue. Messages are stored when disconnected and replayed on reconnect, ordered by priority. Configurable max size and TTL. |
-| **Network Monitor** | Real-time latency, jitter, and packet-loss analysis. Determines connection quality (excellent / good / fair / poor). Generates diagnostic suggestions. |
-| **Event Sequencer** | Client-side sequence validation with gap detection. Out-of-order buffer for reordering. Duplicate detection via seen-ID window with age-based eviction. |
-| **Circuit Breaker** | Client-side circuit breaker for connection attempts. Prevents reconnection storms when the server is down. Configurable failure threshold and recovery timeout. |
-| **Compression + msgpack** | Client-side decompression (pako zlib) and msgpack decoding. Automatic detection of binary vs JSON frames. |
-| **Zustand Stores** | `useWSEStore` for connection state, latency history, diagnostics. `useMessageQueueStore` for message buffering with priority. Lightweight, no boilerplate. |
-| **Rate Limiter** | Client-side token-bucket rate limiter for outbound messages. Prevents flooding the server. |
-| **Security Manager** | Client-side HMAC verification and optional decryption. Validates message integrity before dispatching to handlers. |
-
-### Client (Python)
-
-| Feature | Description |
-|---------|-------------|
-| **Async + Sync API** | `AsyncWSEClient` with async context manager and async iterator. `SyncWSEClient` wrapper for threaded/synchronous code. |
-| **Connection Manager** | Auto-reconnection with 4 strategies (exponential, linear, fibonacci, adaptive). Jitter, configurable max attempts. Heartbeat with PING/PONG. |
-| **Connection Pool** | Multi-endpoint support with health scoring. Weighted-random, least-connections, round-robin load balancing. |
-| **Circuit Breaker** | Three-state machine (CLOSED / OPEN / HALF_OPEN). Prevents reconnection storms. |
-| **Rate Limiter** | Client-side token-bucket rate limiter for outbound messages. |
-| **Event Sequencer** | Duplicate detection (10K ID window) and out-of-order reordering buffer. |
-| **Network Monitor** | Latency, jitter, packet loss analysis. Connection quality scoring. |
-| **Security** | ECDH P-256 key exchange, AES-GCM-256 encryption, HMAC-SHA256 signing. Wire-compatible with server and TypeScript client. |
-| **Compression + msgpack** | Zlib decompression and msgpack decoding. Automatic binary frame detection. |
+### Connection Management
+
+```python
+server.get_connection_count()        # Lock-free AtomicUsize read
+server.get_connections()             # List all connection IDs (snapshot)
+server.disconnect(conn_id)           # Force-disconnect a connection
+server.inbound_queue_depth()         # Events waiting to be drained
+server.inbound_dropped_count()       # Events dropped due to full queue
+server.get_cluster_dlq_entries()     # Retrieve failed cluster messages from dead letter queue
+```
 
 ---
 
-## Performance
+## Security
 
-Rust-accelerated engine via PyO3. AMD EPYC 7502P (32 cores, 128 GB), Ubuntu 24.04, localhost.
+### JWT Authentication
 
-### Highlights
+Rust-native HS256 validation during the WebSocket handshake. Zero GIL acquisition, 0.01ms per decode.
 
-| Metric | Value | Details |
-|--------|-------|---------|
-| **Peak throughput** | **14.2M msg/s** | JSON, Rust client, 500 connections |
-| **Peak binary** | **30M msg/s** | MsgPack/compressed, Rust client |
-| **Fan-out broadcast** | **2.1M deliveries/s** | Single-instance, zero message loss |
-| **Max connections** | **500,000** | Zero errors, zero gaps at every tier |
-| **Connection latency** | **0.38 ms** median | Rust JWT auth in handshake |
-| **Accept rate** | **15,020 conn/s** | Sustained connection establishment |
-| **Memory per conn** | **4.4 KB** | Rust core static overhead |
+Token delivery:
+- **Browser clients**: `access_token` HTTP-only cookie (set by your login endpoint, attached automatically by the browser)
+- **Backend clients**: `Authorization: Bearer <token>` header and/or `access_token` cookie
+- **API clients**: `Authorization: Bearer <token>` header
 
-### Point-to-Point (Rust Native Client)
+Required claims: `sub` (user ID), `exp` (expiration), `iat` (issued at). Optional: `iss`, `aud` (validated if configured).
 
-| Payload | Throughput | Bandwidth |
-|---------|-----------|-----------|
-| 64 bytes | **19.4M msg/s** | 1.2 GB/s |
-| 256 bytes | **12.5M msg/s** | 3.1 GB/s |
-| 1 KB | **10.3M msg/s** | 10.0 GB/s |
-| 16 KB | **1.2M msg/s** | **19.9 GB/s** |
-| 64 KB | **284K msg/s** | **18.2 GB/s** |
+### End-to-End Encryption
 
-### Point-to-Point (Python Multi-Process, 64 Workers)
+Per-connection session keys via ECDH P-256 key exchange, AES-GCM-256 encryption, HKDF-SHA256 key derivation.
 
-| Metric | JSON | MsgPack |
-|--------|------|---------|
-| **Sustained** | **2,045,000 msg/s** | **2,072,000 msg/s** |
-| **Burst** | 1,557,000 msg/s | 1,836,000 msg/s |
-| **64KB messages** | 256K msg/s (16.0 GB/s) | -- |
-| **Ping RTT** | 0.26 ms median | -- |
+Wire format: `E:` prefix + 12-byte IV + AES-GCM ciphertext + 16-byte auth tag.
 
-### Point-to-Point (TypeScript/Node.js, 64 Processes)
+Enable on the client side - the server handles key exchange automatically during the handshake.
 
-| Format | Throughput | Scaling |
-|--------|-----------|---------|
-| JSON | **7.0M msg/s** | 97% linear |
-| MsgPack | **7.9M msg/s** | +13% over JSON |
+### Rate Limiting
 
-### Fan-out Broadcast (Single-Instance)
+Per-connection token bucket: 100,000 token capacity, 10,000 tokens/second refill. Clients receive a `rate_limit_warning` at 20% remaining capacity, and `RATE_LIMITED` error when exceeded.
 
-Server broadcasts to N subscribers. **Zero message loss at every tier.**
+### Deduplication
 
-| Subscribers | Deliveries/s | Bandwidth | p50 Latency |
-|-------------|-------------|-----------|-------------|
-| 10 | **2.1M** | 295 MB/s | 0.005 ms |
-| 1,000 | 1.4M | 185 MB/s | -- |
-| 10,000 | 1.2M | 163 MB/s | -- |
-| 100,000 | 1.7M | 234 MB/s | -- |
-| **500,000** | **1.4M** | 128 MB/s | -- |
+`send_event()` maintains a 50,000-entry AHashSet with FIFO eviction. Duplicate message IDs are dropped before serialization.
 
-### Rust Acceleration vs Python
+### Zombie Detection
 
-| Component | Speedup | Rust | Python |
-|-----------|---------|------|--------|
-| JWT decode (HS256) | **85x** | 10 us | 850 us |
-| Msgpack parsing | **~50x** | -- | -- |
-| Rate limiter | **40x** | -- | -- |
-| HMAC-SHA256 | **22x** | -- | -- |
-| Compression (zlib) | **6.7x** | -- | -- |
+Server pings every connected client every 25 seconds. Connections with no activity for 60 seconds are force-closed.
 
-See benchmark docs for full results: [Overview](docs/BENCHMARKS.md) | [Rust Client](docs/BENCHMARKS_RUST_CLIENT.md) | [TypeScript Client](docs/BENCHMARKS_TS_CLIENT.md) | [Python Client](docs/BENCHMARKS_PYTHON_CLIENT.md) | [Fan-out](docs/BENCHMARKS_FANOUT.md)
+Full security documentation: [docs/SECURITY.md](docs/SECURITY.md)
 
 ---
 
-## Use Cases
+## Wire Protocol
 
-### Live Dashboards
+WSE uses a custom wire protocol with category-prefixed messages:
 
-Push price updates, sensor data, or analytics to the browser in real time.
+**Text frames:** `WSE{...}` (system), `S{...}` (snapshot), `U{...}` (update) + JSON envelope
 
-```python
-# Server: push price updates
-await bus.publish(
-    topic="prices",
-    event={"event_type": "price_update", "symbol": "AAPL", "price": 187.42},
-)
-```
+**Binary frames:** `C:` (zlib compressed), `M:` (MessagePack), `E:` (AES-GCM encrypted), raw zlib (0x78 magic byte)
 
-```tsx
-// React: consume them
-window.addEventListener('price_update', (e: CustomEvent) => {
-  updateChart(e.detail.symbol, e.detail.price);
-});
-```
+**MessagePack transport:** opt-in per connection via `?format=msgpack` query parameter. Roughly 2x faster serialization and 30% smaller payloads.
 
-### Notifications
+**Protocol negotiation:** `client_hello`/`server_hello` handshake with feature discovery, capability advertisement, and version agreement.
 
-Push order updates, alerts, or system events to specific users.
+Full protocol specification: [docs/PROTOCOL.md](docs/PROTOCOL.md)
 
-```python
-# Server: notify a specific user
-await bus.publish(
-    topic=f"user:{user_id}:events",
-    event={"event_type": "order_shipped", "order_id": 42, "text": "Your order shipped!"},
-)
-```
+---
 
-```tsx
-// React: show a toast
-window.addEventListener('order_shipped', (e: CustomEvent) => {
-  showToast(e.detail.text);
-});
-```
+## Compression
 
-### Chat and Messaging
+Two compression layers:
 
-Group chats, DMs, typing indicators, read receipts. Enable encryption for privacy.
+- **Client-facing:** zlib for messages above the configurable threshold (default 1024 bytes). Applied automatically by `send_event()`.
+- **Inter-peer (cluster):** zstd level 1 for payloads above 256 bytes. Capability-negotiated during handshake. Decompression output capped at 1 MB (MAX_FRAME_SIZE).
 
-```python
-# Server: broadcast a chat message
-await bus.publish(
-    topic=f"chat:{channel_id}",
-    event={"event_type": "message_sent", "text": body.text, "author": user.name},
-)
+---
+
+## Client SDKs
+
+### Python
+
+```bash
+pip install wse-client
 ```
 
+Full-featured async and sync client with connection pool, circuit breaker, auto-reconnect, E2E encryption, and msgpack binary transport.
+
 ```python
-# Python client: listen for messages
-async with connect("ws://localhost:5006/wse", token=jwt) as client:
-    await client.subscribe([f"chat:{channel_id}"])
+from wse_client import connect
+
+async with connect("ws://localhost:5007/wse", token="<jwt>") as client:
+    await client.subscribe(["updates"])
     async for event in client:
-        print(f"{event.payload['author']}: {event.payload['text']}")
+        print(event.type, event.payload)
 ```
 
-### Other Use Cases
+**Sync interface:**
 
-- **Collaborative editing** -- shared cursors, document changes, conflict detection via sequence numbers
-- **IoT and telemetry** -- device status, sensor metrics, command and control. Use LOW priority for telemetry, HIGH for alerts
-- **Gaming** -- game state sync, leaderboards, matchmaking. Use msgpack binary protocol for lower latency
-
----
+```python
+from wse_client import SyncWSEClient
 
-## Installation
+client = SyncWSEClient("ws://localhost:5007/wse", token="<jwt>")
+client.connect()
+client.subscribe(["updates"])
 
-```bash
-# Server (Python) -- includes prebuilt Rust engine
-pip install wse-server
-
-# Client (React/TypeScript)
-npm install wse-client
+@client.on("updates")
+def handle(event):
+    print(event.payload)
 
-# Client (Python) -- pure Python, no Rust required
-pip install wse-client
+client.run_forever()
 ```
 
-Server: prebuilt wheels for Linux (x86_64, aarch64), macOS (Intel, Apple Silicon), and Windows. Python 3.12+ (ABI3 stable -- one wheel per platform).
-
-Python client: pure Python, Python 3.11+. Optional extras: `pip install wse-client[crypto]` for encryption, `pip install wse-client[all]` for everything.
+Key features: 4 reconnect strategies (exponential, linear, fibonacci, adaptive), connection pool with health scoring and 3 load balancing strategies, circuit breaker, token bucket rate limiter, event sequencer with dedup and reorder buffering, network quality monitoring (latency/jitter/packet loss).
 
----
+See [python-client/README.md](python-client/README.md) for full API reference.
 
-## Architecture
+### TypeScript / React
 
+```bash
+npm install wse-client
 ```
-React Client (TypeScript)       Python Client              Server (Python + Rust)
-========================        ========================   ========================
-
-useWSE hook                     AsyncWSEClient             FastAPI Router (/wse)
-    |                               |                        -- OR --
-    v                               v                      RustWSEServer (:5006)
-ConnectionPool                  ConnectionPool                  |
-    |  (multi-endpoint,             |  (multi-endpoint,         v
-    |   health scoring)             |   health scoring)   Rust Engine (PyO3)
-    v                               v                          |
-ConnectionManager               ConnectionManager              v
-    |  (auto-reconnect,             |  (auto-reconnect,   EventTransformer
-    |   circuit breaker)            |   circuit breaker)       |
-    v                               v                          v
-MessageProcessor                MessageCodec              PriorityQueue
-    |  (decompress, verify,         |  (decompress,            |
-    |   sequence, dispatch)         |   sequence, dedup)       v
-    v                               v                     Sequencer + Dedup
-AdaptiveQualityManager          NetworkMonitor                 |
-    |                               |  (quality scoring,       v
-    v                               |   latency, jitter)  Compression + Rate Limiter
-Zustand Store                       v                          |
-    |                           Event handlers /               v
-    v                           async iterator            PubSub Bus (Redis)
-React Components                                               |
-                                                               v
-                                                          Dead Letter Queue
-```
 
-**Wire format (v1):**
+Single React hook (`useWSE`) for connection lifecycle, subscriptions, and message dispatch.
+
+```tsx
+import { useWSE } from 'wse-client';
 
-```json
-{
-  "v": 1,
-  "id": "019503a1-...",
-  "t": "price_update",
-  "ts": "2026-01-15T10:30:00Z",
-  "seq": 42,
-  "p": { "symbol": "AAPL", "price": 187.42 }
+function App() {
+  const { sendMessage, connectionHealth } = useWSE(
+    '<jwt-token>',
+    ['updates'],
+    { endpoints: ['ws://localhost:5007/wse'] },
+  );
+
+  return <div>Status: {connectionHealth}</div>;
 }
 ```
 
+Key features: offline queue with IndexedDB persistence, adaptive quality management, connection pool with health scoring, E2E encryption (Web Crypto API), message batching, 5 priority levels, Zustand store for external state access.
+
+See [client/README.md](client/README.md) for full API reference.
+
 ---
 
-## Packages
+## Performance
+
+Benchmarked on AMD EPYC 7502P (64 cores, 128 GB RAM), Ubuntu 24.04.
+
+| Mode | Peak Throughput | Connections | Message Loss |
+|------|----------------|-------------|--------------|
+| Standalone (fan-out) | 5.0M deliveries/s | 500K | 0% |
+| Standalone (inbound JSON) | 14.7M msg/s | 500K | 0% |
+| Standalone (inbound msgpack) | 30M msg/s | 500K | 0% |
+| Cluster (2 nodes) | 9.5M deliveries/s | 20K per node | 0% |
 
-| Package | Registry | Language | Install |
-|---------|----------|----------|---------|
-| `wse-server` | [PyPI](https://pypi.org/project/wse-server/) | Python + Rust | `pip install wse-server` |
-| `wse-client` | [npm](https://www.npmjs.com/package/wse-client) | TypeScript + React | `npm install wse-client` |
-| `wse-client` | [PyPI](https://pypi.org/project/wse-client/) | Python | `pip install wse-client` |
+Sub-millisecond latency. Median 0.38ms with JWT authentication. Connection handshake: 0.53ms median (Rust JWT path).
 
-All packages are standalone. No shared dependencies between server and clients.
+Detailed results: [Benchmarks](docs/BENCHMARKS.md) | [Fan-out](docs/BENCHMARKS_FANOUT.md) | [Rust Client](docs/BENCHMARKS_RUST_CLIENT.md) | [Python Client](docs/BENCHMARKS_PYTHON_CLIENT.md) | [TypeScript Client](docs/BENCHMARKS_TS_CLIENT.md)
 
 ---
 
-## Documentation
+## Examples
 
-| Document | Description |
-|----------|-------------|
-| [Protocol Spec](docs/PROTOCOL.md) | Wire format, versioning, encryption |
-| [Architecture](docs/ARCHITECTURE.md) | System design, data flow |
-| [Benchmarks](docs/BENCHMARKS.md) | Methodology, results, comparisons |
-| [Security Model](docs/SECURITY.md) | Encryption, auth, threat model |
-| [Integration Guide](docs/INTEGRATION.md) | FastAPI setup, Redis, deployment |
+Working examples in the [`examples/`](examples/) directory:
+
+| Example | Description |
+|---------|-------------|
+| [`standalone_basic.py`](examples/standalone_basic.py) | Basic server with JWT auth and echo |
+| [`standalone_broadcast.py`](examples/standalone_broadcast.py) | Topic-based pub/sub with broadcasting |
+| [`standalone_presence.py`](examples/standalone_presence.py) | Per-topic presence tracking with join/leave events |
+| [`standalone_recovery.py`](examples/standalone_recovery.py) | Message recovery on reconnect with epoch+offset |
 
 ---
 
-## Technology Stack
-
-| Component | Technology | Purpose |
-|-----------|-----------|---------|
-| Rust engine | PyO3 + maturin | Compression, sequencing, filtering, rate limiting, WebSocket server |
-| Server framework | FastAPI + Starlette | ASGI WebSocket handling |
-| Serialization | orjson (Rust) | Zero-copy JSON |
-| Binary protocol | msgpack (rmp-serde) | 30% smaller payloads |
-| Encryption | AES-GCM-256 + ECDH P-256 (Rust) | Per-connection E2E encryption with key exchange |
-| Message signing | HMAC-SHA256 (Rust) | Per-message integrity verification |
-| Authentication | Rust JWT (HS256) | Zero-GIL token validation in handshake |
-| Pub/Sub backbone | Redis Pub/Sub | Multi-process fan-out |
-| Dead Letter Queue | Redis Lists | Failed message recovery |
-| Client state | Zustand | Lightweight React store |
-| Client hooks | React 18+ | useWSE hook with TypeScript |
-| Offline storage | IndexedDB | Persistent offline queue |
-| Python client | websockets + cryptography | Async/sync WebSocket client |
-| Build system | maturin | Rust+Python hybrid wheels |
+## Documentation
+
+- [Architecture](docs/ARCHITECTURE.md) - server internals, PyO3 bridge, concurrency model
+- [Integration Guide](docs/INTEGRATION.md) - complete setup and API reference
+- [Wire Protocol](docs/PROTOCOL.md) - message format specification
+- [Cluster Protocol](docs/CLUSTER_PROTOCOL.md) - TCP mesh, frame format, gossip, interest routing
+- [Security](docs/SECURITY.md) - JWT, encryption, mTLS, rate limiting, circuit breaker
+- [Deployment](docs/DEPLOYMENT.md) - production setup, Docker, Kubernetes, cluster configuration
+- [Migration Guide](docs/MIGRATION.md) - upgrading from v1.x to v2.0
+- [Contributing](CONTRIBUTING.md) - development setup and coding standards
+- [Changelog](CHANGELOG.md) - version history
 
 ---