castrum 0.7.0 → 0.9.1

package/docs/BENCHMARKS.md

@@ -0,0 +1,343 @@
+# Benchmarks Guide
+
+This document describes the benchmark framework, how to run benchmarks, and how to interpret results.
+
+---
+
+## Overview
+
+The benchmark system compares the Rust implementations against pure JavaScript/Bun baseline implementations. It measures throughput (ops/sec), latency (mean/percentiles), and memory allocation. Under Bun the `rust:*` side runs through the PRIMARY `bun:ffi` transport (NAPI is the fallback); the CPU bench warns — or hard-fails with `CASTRUM_BENCH_FFI=1` — if the ffi transport is not live.
+
+### Benchmark Types
+
+| Type | Description |
+|------|-------------|
+| **Unit benchmarks** | Individual function comparisons (hashing, validation, JSON, etc.) |
+| **HTTP benchmarks** | Full HTTP server comparisons (Bun vs Elysia vs Ingress) |
+| **Batch benchmarks** | Throughput for batch operations |
+| **Stress benchmarks** | Performance under load |
+| **Concurrency benchmarks** | Performance with multiple concurrent requests |
+
+---
+
+## Running Benchmarks
+
+### Unit Benchmarks
+
+```bash
+# Run all unit benchmarks
+bun bench.ts
+```
+
+### Bun built-ins diagnostic set ("don't reinvent the wheel")
+
+`bun run check` also races castrum ops against **Bun's native built-ins**
+(`Bun.hash`, `Bun.password`, `Bun.CryptoHasher`, `Bun.gzipSync`,
+`Bun.randomUUIDv7`) as a diagnostic set:
+
+- Task names use the **`diag:`** prefix (not `native:`/`rust:`) so they stay out
+  of the shipped-op `native:`/`rust:` comparisons — they are informational.
+- Sources: `src/baseline/tasks/bun-builtins.ts` (Bun-only baselines, guarded
+  with `typeof Bun` so the module still loads under Node),
+  `src/bench/tasks/bun-builtins.ts`, comparisons in `src/bench/comparisons.ts`.
+- **Runtime honesty guard**: a Bun API must be verified to actually execute real
+  work before its result is trusted. The first run produced false "wins"
+  (brotli 156x, validators 3-5x) because Bun 1.4 has **no**
+  `Bun.brotliCompressSync` and **no** `Bun.validators` — those pairs were
+  removed, not reported.
+- Results and the keep/delegate/add decisions live in
+  `docs/bun-builtins-decision-matrix.md`.
+
+### HTTP Benchmarks
+
+```bash
+# Run all HTTP scenarios
+bun run bench:http
+
+# Run specific scenario
+bun run bench:http:smoke     # Light load, quick health check
+bun run bench:http:stress    # High concurrency, short duration
+bun run bench:http:heavy     # Heavy JSON payloads
+bun run bench:http:crud      # CRUD operations
+bun run bench:http:spike     # Burst traffic patterns
+bun run bench:http:soak      # Long duration (endurance)
+bun run bench:http:storm     # Very high concurrency burst
+bun run bench:http:boundary  # Edge case payloads
+
+# Compare specific frameworks
+bun run bench:http:elysia    # Run only Elysia vs baseline
+bun run bench:http:ingress   # Run only Ingress vs baseline
+bun run bench:http:bun-only  # Run only raw Bun vs baseline
+
+# Run all heavy scenarios
+bun run bench:http:all-heavy
+
+# Max-throughput cross-check (autocannon — no Bun fetch cap)
+bun run bench:http:ac          # all servers, 03-stress weighted mix
+bun run bench:http:stress:ac   # 03-stress mix only
+
+# Single-path static comparison (fastest, most stable signal):
+#   node bench/autocannon-stress.mjs            # dynamic 03-stress mix
+#   AC_PATH=/api/users node bench/autocannon-stress.mjs
+#   AC_PATH=/api/users AC_METHOD=POST AC_BODY='{"id":1,"name":"x"}' \
+#     AC_CONTENT_TYPE=application/json node bench/autocannon-stress.mjs
+#   AC_RUNS=3 node bench/autocannon-stress.mjs  # median-of-N (see §Single-core)
+#     — run-major interleaving: every server is measured once per round, so
+#       the median compares servers over the same load windows on a noisy host.
+#
+# Multi-core scaling (SO_REUSEPORT — the way to exceed the single-core ceiling):
+#   AC_INSTANCES=4 AC_PIPELINING=10 node bench/autocannon-stress.mjs
+#   (ingress/router only — they read INGRESS_REUSE_PORT=1)
+```
+
+### Bun `fetch()` concurrency cap (read this before interpreting HTTP results)
+
+Bun's `fetch()` limits the number of **simultaneous** requests to **256 by
+default** (`BUN_CONFIG_MAX_HTTP_REQUESTS`, max 65,336). When a scenario's
+`maxConcurrent` exceeds that cap, the excess requests queue **FIFO client-side**
+and the run measures the *generator*, not the server — the classic signature is
+a sub-ms `p50` with a multi-second tail and a flat RPS wall across every server.
+
+- All `bench:http:*` scripts set `BUN_CONFIG_MAX_HTTP_REQUESTS=65536` for you.
+- `bench/run-bench.ts` warns if a selected scenario exceeds the effective cap.
+- For **max-throughput** server comparisons use **autocannon**
+  (`bench/autocannon-stress.mjs`, run with `node`): it has no such cap, uses a
+  real socket pool, scales across worker threads, and reports RPS + latency
+  percentiles natively. It replicates the `03-stress` weighted flow mix
+  (50% GET /api/users?q=…&page=…, 30% POST /api/users, 20% GET /health).
+- **Two autocannon gotchas** (both fixed in the runner):
+  1. `requestGenerator` is CLI-ONLY — the library silently ignores it (every
+     request then goes to `GET /` → 404 → "0 2xx"). The runner uses the
+     library's `requests: [{ setupRequest }]` API instead.
+  2. `result.workers` is a count (`opts.workers`), not an array — the report no
+     longer treats it as a per-worker array.
+
+### Single-core ceiling & multi-core scaling (measured 2026-08-16)
+
+All four servers are **single-process `Bun.serve`** (one event loop), so they
+pin at a per-core ceiling on this machine. The host is load-noisy, so use the
+**median-of-N** runner (`AC_RUNS=N`, default 1) with **run-major interleaving** —
+every server is visited once per round, so a single slow window affects all
+servers' run-`i` equally and the median compares servers over the same load
+windows (the old server-major order biased later servers into noisier windows).
+> **Dated snapshot (2026-08-16 ingress-RPS pass).** The host is load-noisy;
+> the newest persisted autocannon medians under `bench/results/autocannon/*/final-*.median.md`
+> supersede these numbers run-to-run. Use the median-of-N runner below for fresh
+> comparisons.
+
+Current medians (`AC_DURATION=8 AC_RUNS=3`, 500 connections, single instance):
+
+- `GET /api/users?q=…` (query + cookies + CORS + metadata envelope):
+  **bun 45.2k, elysia 41.7k, ingress 51.5k, router 49.4k RPS** — ingress ~**+14%**
+  vs bun, with a tighter tail (p50 9ms / p99 16ms vs bun p50 10ms / p99 28ms).
+- `POST /api/users` (JSON body, fast-path schema validation):
+  **bun 40.8k, elysia 35.1k, ingress 45.4k, router 35.1k RPS** — ingress ~**+11%**
+  vs bun. The pre-optimization ingress POST gap (≈ −24% vs bun) is closed by
+  the changes in `CHANGELOG [Unreleased]` (JS hot-path elimination + single-pass
+  body validation).
+- Per-request cost breakdown (`bun bench/ingress-cost.ts` / `ingress-cost-post.ts`,
+  min-of-5): GET `run` ~826ns (native 168ns + JS ~446ns packing + decode 60ns);
+  POST `run` ~1154ns (native ~740ns — was 808ns before the single-pass — + JS).
+  The POST body read is now **synchronous for buffered bodies**: `Bun.peek` on
+  `req.bytes()` skips the deadline race + watchdog for declared-length bodies
+  (was ~600ns of race machinery per POST).
+- **To exceed the single-core ceiling**, run multiple server processes on the
+  same port via SO_REUSEPORT: `AC_INSTANCES=4` (with `AC_PIPELINING` so the
+  client can drive enough in-flight requests) scaled ingress from ~45k to
+  ~69k RPS on this machine. Only ingress/router bind with `reusePort`
+  (`INGRESS_REUSE_PORT=1`); bun/elysia do not set it. For cleanest numbers run
+  autocannon on a separate host.
+
+---
+
+## HTTP Benchmark Scenarios
+
+Each scenario targets a specific performance aspect:
+
+### 01-smoke
+- **Goal**: Quick health check (health, GET, POST mix)
+- **Duration**: ~10 seconds
+- **Concurrency**: up to 50
+
+### 02-load
+- **Goal**: Sustained mixed load (warm-up → sustain → cool-down)
+- **Duration**: ~120 seconds
+- **Concurrency**: up to 1,000
+- **Traffic**: GET/POST mix with cookies, health
+
+### 03-stress
+- **Goal**: Ramp to maximum throughput
+- **Duration**: ~100 seconds (5 × 20s ramp phases)
+- **Concurrency**: up to 2,000 (higher on a single machine only deepens the
+  client-side fetch queue — the generator can't drive 10k usefully; see the
+  Bun fetch-cap note above. For true max-throughput use `bench:http:ac`.)
+
+### 04-spike
+- **Goal**: Burst resilience (baseline → spike → recovery)
+- **Duration**: ~100 seconds
+- **Concurrency**: up to 8,000 (spikes up to 5,000 req/s)
+
+### 05-soak
+- **Goal**: Endurance/stability
+- **Duration**: ~600 seconds (10 minutes)
+- **Concurrency**: up to 1,000
+
+### 06-edge-cases
+- **Goal**: Malformed / edge-case payloads
+- **Duration**: ~30 seconds
+- **Concurrency**: up to 200
+- **Traffic**: invalid JSON, schema failures, 404/405/415/422, Unicode, oversize query
+
+### 07-cors-preflight
+- **Goal**: CORS preflight storm (allowed + disallowed origins)
+- **Duration**: ~30 seconds
+- **Concurrency**: up to 500
+
+### 08-rate-limit
+- **Goal**: Rate limiting (under limit → over limit → recovery)
+- **Duration**: ~40 seconds
+- **Concurrency**: up to 1,000
+
+### 09-large-payload
+- **Goal**: Large payload handling
+- **Duration**: ~30 seconds
+- **Concurrency**: up to 100
+- **Payload**: large binary + large JSON arrays
+
+### 10-mixed-realistic
+- **Goal**: Realistic mixed traffic (origins, cookies, auth)
+- **Duration**: ~60 seconds (ramp)
+- **Concurrency**: up to 5,000
+
+### 11-concurrent-burst
+- **Goal**: High-concurrency bursts with pauses
+- **Duration**: ~35 seconds
+- **Concurrency**: up to 8,000
+
+### 12-slowloris
+- **Goal**: Slow clients
+- **Duration**: ~60 seconds
+- **Concurrency**: up to 500
+
+### Heavy JSON scenarios (`13`–`20`)
+
+`13-heavy-json-nested` … `20-validation-storm` stress schema validation with nested, array-heavy, wide, CRUD-mix, spike, soak, large-body, and storm payloads. Run them with `bun run bench:http:heavy`.
+
+---
+
+## Interpreting Results
+
+### Report Format
+
+Results are saved to `bench/results/<framework>/<scenario>.bench.md` and include:
+
+```md
+# Benchmark Report: bun / 01-smoke
+
+| Metric | Value |
+|--------|-------|
+| Duration | 10.0s |
+| Requests | 125,432 |
+| Throughput | 12,543 req/s |
+| Mean Latency | 0.79ms |
+| p50 Latency | 0.72ms |
+| p95 Latency | 1.13ms |
+| p99 Latency | 2.45ms |
+| Max Latency | 15.67ms |
+| Errors | 0 |
+```
+
+### Comparison Reports
+
+When comparing multiple frameworks, differences are expressed as:
+
+- **Speedup**: `2.45x` (Rust is 2.45x faster than JS)
+- **Slowdown**: `0.82x` (Rust is 0.82x the speed of JS, i.e., 18% slower)
+- **% Improvement**: `+145%` throughput increase
+
+### Key Metrics
+
+| Metric | What It Measures |
+|--------|-----------------|
+| **Throughput** | Requests per second — higher is better |
+| **Mean Latency** | Average response time — lower is better |
+| **p50/p95/p99** | Percentile latencies — lower is better |
+| **Max Latency** | Worst-case response time — should be bounded |
+| **Error Rate** | Percentage of failed requests — should be 0% |
+
+---
+
+## Benchmark Architecture
+
+```
+bench.ts
+  └─ Loads benchmark tasks from src/bench/tasks/ (33 tasks)
+       ├── hashing.ts       Compare Rust fnv1a64 vs JS TextEncoder + hash
+       ├── validation.ts    Compare Rust validation vs validator library
+       ├── json.ts          Compare Rust sonic-rs vs JSON.parse
+       ├── http.ts          Compare Rust httparse vs manual
+       ├── complex.ts       Combined operations
+       ├── stress.ts        Concurrent load tests
+       └── ...              (aead, accept, compress, cookie-sign, csrf, encoding, etag,
+                            form, json-schema, jwt, media-type, multipart, password,
+                            streaming, template, url-join, ... — full list in docs/REPO_MAP.md)
+```
+
+Each task runs:
+1. **Warmup**: 100ms to JIT-compile JavaScript
+2. **Measurement**: Fixed duration (default 2000ms) in a tight loop
+3. **Operations counted**: Number of function calls completed
+4. **Result**: Ops/sec calculated from wall-clock time
+
+### Benchmark controls
+
+| Env / flag | Effect |
+|------------|--------|
+| `CASTRUM_BENCH_BATCH_SIZE` | Batch size for sub-µs operations (default `64`). Batching amortizes the timer/measurement overhead for very fast ops. |
+| `HTTP_NO_SHAPE=1` | The HTTP load generator (`bench/load.ts`) skips response-shape `JSON.parse` for pure-throughput runs. |
+
+---
+
+## HTTP Server Architecture
+
+```
+bench/servers/
+├── bun-server.ts       # Raw Bun.serve with manual routing
+├── elysia-server.ts    # Elysia framework
+├── ingress-server.ts   # Ingress (this project) optimized handler
+├── router-server.ts    # Ingress per-route compiled-router variant (createIngressRouter)
+└── shared.ts           # Shared route handlers, PORTS, env helpers
+```
+
+Each server implements the same routes for fair comparison:
+
+- `GET /` — Plain text hello
+- `GET /json` — JSON response
+- `POST /json` — Echo JSON body
+- `GET /id/:id` — Path parameter
+- `POST /users` — User creation
+
+---
+
+## Environment
+
+All benchmarks should be run on a dedicated machine for reproducible results:
+
+- **CPU**: Modern x86_64 (Intel/AMD) or Apple Silicon
+- **Memory**: 8GB+ RAM
+- **OS**: Linux (recommended) or macOS
+- **Bun**: Latest stable
+- **Rust**: Latest stable (via rustup)
+
+### Turbo Mode (Linux)
+
+For consistent results on Linux, disable CPU frequency scaling:
+
+```bash
+sudo cpupower frequency-set --governor performance
+```
+
+Run benchmarks, then restore:
+
+```bash
+sudo cpupower frequency-set --governor powersave

package/docs/CASE_STUDY.md

@@ -0,0 +1,240 @@
+# Case study — building a Rust-accelerated HTTP pipeline for Bun
+
+**castrum** is a hybrid TypeScript + Rust package that gives Bun servers native
+speed for the work that actually costs CPU — request validation, parsing,
+hashing, crypto, and a full HTTP ingress pipeline — behind a small, flat API.
+This document tells the story of how it was built and measured, what the
+benchmarks actually showed, and the lessons that generalise to any
+"should I write this in Rust?" decision.
+
+All figures below come from the reports in `bench/results/` (generated on a
+release addon, baseline CPU, Bun 1.4.0). They're **representative**, not
+advertised as universal — re-run `bun run check` / `bun run bench:http` on your
+own hardware before trusting them for your numbers.
+
+---
+
+## 1. Background: the problem
+
+Bun is already very fast. The gap that motivated castrum was not "Bun is slow"
+but three specific frictions:
+
+1. **CPU-bound request work runs in JavaScript.** JSON validity checking, email
+   / UUID / IP validation, query & cookie parsing, hashing, and schema
+   validation are pure CPU. In JS they compete with the event loop; the moment
+   a request needs several of them, tail latency climbs.
+2. **Production HTTP servers need a lot of "plumbing" middleware.** CORS,
+   rate limiting, body-size guards, schema validation, security headers,
+   request IDs, metrics. Hand-wiring these into `Bun.serve` is repetitive and
+   error-prone; pulling in a framework drags in a dependency graph.
+3. **Bun and Node want different integration stories.** The same logic should
+   run on Bun (the primary target, zero startup cost) and Node ≥ 20.3 (a
+   compiled ESM entry) without two implementations.
+
+The hypothesis: a **hybrid** design — ergonomic TypeScript on top of one Rust
+cdylib — could remove the CPU cost from the event loop, package the middleware
+as a pre-baked pipeline, and still be usable from both runtimes.
+
+---
+
+## 2. The approach
+
+### 2.1 One cdylib, two transports
+
+The addon is built with napi-rs (so Node can load it), but it *also* exports
+`extern "C"` symbols (`rust/ffi.rs`, **79 `castrum_*` symbols** — 71 direct + 4
+`validator_c_abi!` + 4 `compress_to_out!`) so Bun can
+`dlopen` it directly:
+
+- **`bun:ffi` (primary on Bun)** — Bun JIT-calls the C-ABI exports at
+  ~10–20 ns per crossing, roughly an order of magnitude cheaper than a NAPI
+  call. A **bind-time self-test** verifies every symbol and the ingress layout
+  blob; on *any* failure the layer disables itself and calls fall back to NAPI.
+- **NAPI (fallback)** — used under Node, when `CASTRUM_FFI_MODE=napi`, or when
+  the self-test fails. It's the safety net that keeps the same code working on
+  both runtimes.
+
+```mermaid
+flowchart LR
+    TS["TypeScript (index.ts)"] -->|"bun:ffi · 75 C-ABI symbols · ~10–20 ns"| RUST["Rust cdylib (rust/ffi.rs)"]
+    TS -.->|"NAPI fallback (Node / self-test failure)"| RUST
+    RUST --> PIPELINE["8-stage ingress pipeline"]
+    TS --> PACKED["Packed, length-prefixed buffers (zero-copy)"]
+    PACKED --> RUST
+```
+
+### 2.2 Zero-copy, zero-alloc request handling
+
+Instead of marshalling objects, the hot path **packs** the request frame
+(method kind, URL, client IP, request ID, headers) into a shared `Uint8Array`,
+Rust parses it and writes a decision into a **single output buffer**, and TS
+decodes only the fields it needs. Hot paths allocate nothing per request;
+pooled output buffers (with an adaptive-estimate size heuristic) are reused.
+
+### 2.3 The ingress pipeline as a product, not a library
+
+The 8-stage pipeline (trust/IP → HTTPS → CORS → rate limit → body guard →
+JSON/schema → serialize cookies/query → write output) ships twice:
+`createIngressServer` wraps it in a `Bun.serve` route table, and the same
+handlers run over `node:http` via `createIngressServerNode`.
+
+> **Two wire formats on purpose.** The low-level fast path emits
+> `{"error":{...}}` with `x-ratelimit-*`; the pre-baked server emits
+> `{"ok":true,...,"requestId":...}` with `ratelimit-*`. They are a **contract**
+> the load generator validates — unifying them would break the benchmark
+> baseline. (Recorded in `docs/adr/0001-two-wire-formats.md`.)
+
+---
+
+## 3. Results
+
+### 3.1 Startup & first call (the "is it instant?" test)
+
+From `bench/results/startup/latest.json` (Bun 1.4.0, 15 iterations):
+
+| Metric | p50 | p95 |
+|--------|-----|-----|
+| Import the package (raw `index.ts`, no build step) | ~90 ms | ~112 ms |
+| First `rust.crc32` call (ffi transport already warm) | ~0.04 ms | ~0.08 ms |
+| First ingress pipeline call (addon loaded) | ~2.6 ms | ~5.3 ms |
+
+The important number is the first-*call* cost: the cdylib is lazy-loaded, so
+the import itself doesn't pay a native-init penalty.
+
+### 3.2 CPU: where Rust wins, and where it doesn't
+
+The CPU benchmark races every `rust.*` op against a pure-JS baseline **and**
+Bun's built-ins (`docs/bun-builtins-decision-matrix.md`, release addon,
+2026-08-11). Ratio > 1 = Bun is faster.
+
+| Op | Bun built-in | Ratio | Verdict |
+|----|--------------|-------|---------|
+| `passwordHash` (argon2id) | `Bun.password.hashSync` | **0.55** (rust 1.83×) | **keep rust** |
+| `passwordVerify` (argon2id) | `Bun.password.verifySync` | **0.53** (rust 1.88×) | **keep rust** |
+| `passwordVerifyBcrypt` | `Bun.password.verifySync` | **0.67** (rust 1.49×) | **keep rust** |
+| `pbkdf2Sha256` | `node:crypto.pbkdf2Sync` | 0.93 (parity) | keep rust |
+| `fnv1a64` | `Bun.hash` (wyhash) | 1.06 | parity |
+| `hmacSha256` | `Bun.CryptoHasher` | 1.1–1.4 | mild Bun win |
+| `randomToken` | `Bun.randomUUIDv7` | 1.62 | **Bun wins** |
+| `gzipDecompress` | `Bun.gunzipSync` | 1.38 | Bun wins* |
+| `gzipCompress` | `Bun.gzipSync` | 2.02 | **Bun wins** |
+| `crc32` | `Bun.hash.crc32` | 2.8–8.4 | **Bun wins** |
+| `xxh3` | `Bun.hash.xxHash3` | 4.15 | **Bun wins** |
+| brotli / zero-DOM JSON / parsers | *no sync Bun equivalent* | — | keep rust |
+
+\* `gzipDecompress` deliberately stays native: the Rust path enforces a **64 MiB
+decompression-bomb cap** that `Bun.gunzipSync` lacks.
+
+**The honest takeaway:** "native is faster" is *not* automatic. The runtime
+adapter's `builtins` registry (`src/runtime/builtins.ts`) delegates the ops
+where a Bun built-in wins (e.g. `rust.xxh3` loses 4× to `Bun.hash.xxHash3`),
+while argon2id and the zero-DOM JSON path stay native. A consumer-facing selection
+surface (`opImpl` / `isNativeOp` / `opDecision`) lets frameworks bind each op to
+native or JS **once at load time** instead of per call.
+
+### 3.3 HTTP: the pipeline holds up under load
+
+From `bench/results/*/03-stress.bench.md` (ramp to 10k concurrent, ~104 s):
+
+| Server | Achieved RPS | Total requests | Unexpected errors | Shape failures |
+|--------|-------------|----------------|-------------------|----------------|
+| raw `Bun.serve` | 862.5 | 92,174 | 0 | 0 |
+| **castrum ingress** | **904.2** | **94,060** | **0** | **0** |
+
+Under extreme concurrency both saturate and show long tail latencies (p99 ~18 s
+on this machine) — that's the load generator over-driving the box, not a bug.
+The meaningful result is that adding the full pipeline (CORS + rate limit +
+body guard + schema + security headers + request IDs) costs **no correctness
+and no measurable throughput** versus a hand-rolled raw server.
+
+### 3.4 The bug that almost killed the server: C-ABI panics
+
+The most instructive moment: under `11-concurrent-burst`, the ingress server
+**died at ~2.5 s and stayed down** (91% network errors). Root cause: the `bun:ffi`
+`extern "C"` exports had no panic guard. A panic in a napi call becomes a JS
+exception, but a panic unwinding through raw `extern "C"` is undefined behaviour
+that takes the whole process down.
+
+The fix (in `rust/ffi.rs`): every fallible / allocating C-ABI export now routes
+its core through `panic_guard` (`catch_unwind`) and reports a sentinel instead
+of panicking. After the fix, the same scenario completes cleanly:
+
+| Scenario (post-fix) | Requests | Network errors | Unexpected errors |
+|---------------------|----------|----------------|-------------------|
+| `11-concurrent-burst` | 30,211 | 0 | 0 |
+
+**Lesson:** if you expose a raw C ABI to a host runtime, you own the unwind
+boundary. Never assume a panic can't escape — contain it explicitly, and make
+your bind-time self-test exercise the fallible paths.
+
+### 3.5 Correctness as a first-class gate
+
+The HTTP smoke scenario (`01-smoke`) is a **wire-format guard**, not a
+throughput test: it verifies `ok === true` + `requestId` on success and
+`error.code` / `error.message` on failure for every response. Across the
+scenarios, `shape_failures` and `unexpected_statuses` are 0 — the two wire
+formats are validated every CI run, so a change to the response shape breaks
+the benchmark, not prod.
+
+---
+
+## 4. Engineering decisions & lessons
+
+1. **Two transports, one binary.** Build with napi-rs (portability), export
+   `extern "C"` (speed on Bun), and let a **bind-time self-test** pick the safe
+   path automatically. Never let a transport failure be silent.
+2. **Benchmark against the built-in, not just the baseline.** Bun's `Bun.hash.*`,
+   `Bun.gzipSync`, `Bun.CryptoHasher`, and `Bun.randomUUIDv7` beat an FFI
+   crossing on several ops. The rule that emerged: **keep Rust where it wins or
+   where Bun has no synchronous equivalent; delegate where Bun wins.** Don't
+   reinvent the wheel — `docs/adr/0003-bun-builtins-delegation.md`.
+3. **"Native" is not a synonym for "fast".** The `builtins` registry delegates
+   where Bun wins. Let data, not enthusiasm, decide.
+4. **A wire format is a contract.** Two deliberately-different formats are
+   pinned by tests and the load generator. If you change the shape, you change
+   the benchmark.
+5. **Contain the C ABI.** A panic across `extern "C"` is a crash. Route every
+   fallible export through a guard and cover it in the self-test.
+6. **Zero-copy is a discipline.** Packed buffers, pooled outputs, lazy decode —
+   every allocation you remove from the hot path is tail latency you never pay.
+7. **Be honest about scope.** The rate limiter is **per-process, not
+   distributed**; proxy trust is **off by default** (spoof-proof); the
+   decompression bomb cap is **64 MiB**. Documenting the boundaries is as
+   important as the speed.
+
+---
+
+## 5. Reproduce it yourself
+
+```bash
+bun install
+bun run build              # release addon (baseline CPU — what ships)
+
+bun run check              # CPU benchmark → bench/results/cpu/latest.json
+bun run bench:startup      # import + first-call timing
+bun run bench:http:smoke   # HTTP wire-format guard
+bun run bench:http         # all HTTP scenarios (bun / elysia / ingress)
+```
+
+See [`BENCHMARKS.md`](./BENCHMARKS.md) for scenario descriptions and how to read
+the reports.
+
+---
+
+## 6. When this pattern applies (and when it doesn't)
+
+The hybrid TS + Rust + FFI pattern paid off here because the workload was
+**CPU-bound, synchronous, and high-volume**. Reach for it when:
+
+- Request handling spends measurable time in validation/parsing/hashing.
+- You want a pre-baked HTTP pipeline (CORS, rate limit, schema, security
+  headers) without a heavy middleware graph.
+- You must support both Bun and Node from one codebase.
+
+Don't reach for it when:
+
+- The bottleneck is I/O or the database, not CPU — Rust won't help.
+- You only need a handful of ops that Bun's built-ins already win (use
+  `Bun.hash.crc32`, `Bun.gzipSync`, `Bun.randomUUIDv7` directly).
+- You can't ship/ build a native `.node` in your deployment (no Rust toolchain
+  in CI, or a locked-down runtime that forbids native addons).

package/docs/ENVIRONMENT.md

@@ -0,0 +1,81 @@
+# Environment Variables Reference
+
+This document lists every environment variable read by the package and its
+example servers. All values are read once at module load; invalid values are
+warned about and replaced with the documented default.
+
+## Ingress server (`bench/servers/ingress-server.ts`)
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `INGRESS_TRUST_PROXY` | bool | `false` | **Opt-in.** When `true`, honors `X-Forwarded-For` / `X-Real-IP` for client-IP resolution. Keep **off** unless the server sits behind a trusted edge that strips these headers, otherwise clients can spoof arbitrary IPs for IP-based rate limiting. |
+| `INGRESS_REQUEST_ID_HEADER` | bool | `false` | Emit an `x-request-id` response header per request. The request ID is always generated internally; this only controls whether it is echoed to clients. |
+| `INGRESS_ZERO_COPY` | bool | `false` | Enable zero-copy response bodies (legacy alias `INGRESS_UNSAFE_ZERO_COPY` still honored). **Not recommended** unless per-response output buffers are implemented; leave off for safety (bodies are copied by default). |
+| `INGRESS_ZERO_COPY_MAX_IN_FLIGHT` | int | `128` | Max pooled output buffers kept in flight for zero-copy responses (min `1`). |
+| `INGRESS_ZERO_COPY_TIMEOUT_MS` | int | `1000` | Timeout before a zero-copy pooled buffer is force-released (min `0`). |
+| `INGRESS_OUTPUT_BUF_BYTES` | int | `131072` | Size of the Rust output buffer (min `65536`). If cookies + query + metadata JSON exceed this, `FLAG_BODY_TRUNCATED` is set and the request fails closed. |
+| `INGRESS_SECURITY_HEADERS` | bool | `true` | Emit the precomputed security headers (CSP, HSTS, X-Frame-Options, etc.). Disable only behind an edge that already sets them. |
+| `INGRESS_REUSE_PORT` | bool | `false` | Enable `SO_REUSEPORT` for multi-process deployments (spawn N processes, each with a distinct `INGRESS_WORKER_ID`). |
+| `INGRESS_LOG_REQUESTS` | bool | `false` | Write one JSON line per request to **stderr** (`{ts, kind, requestId, method, path, status, errorCode?, rateLimited?, ...}`). |
+| `INGRESS_METRICS` | bool | `false` | Write a periodic aggregate summary to **stderr** every 10s (`{ts, kind, total, rps, status, rateLimited, internalErrors}`). |
+| `INGRESS_HOST` | string | `0.0.0.0` | Bind address. |
+| `INGRESS_IDLE_TIMEOUT` | int | `30` | Idle keep-alive timeout in seconds (min `1`). |
+| `INGRESS_WORKER_ID` | int | `0` | Worker identifier (0–65535), mixed into request-ID generation to reduce collision risk across processes. |
+
+Boolean flags accept `1`/`true`/`yes`/`on` for true and `0`/`false`/`no`/`off`
+for false; anything else is warned about and falls back to the default.
+
+## Rust FFI thread pool
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `CASTRUM_RAYON_THREADS` | int | `max(1, cores-1)` | Rayon thread pool size. The pool is process-wide and initialized **once** — the first use wins. With lazy init, call `rust.configure({ rayonThreads })` (or set this env) before the first batch operation. |
+| `CASTRUM_MAX_RAYON_THREADS` | int | `cores` | Hard cap on rayon threads (native). |
+| `CASTRUM_PIN_CORES` | (presence) | — | When set (Linux), pin rayon worker threads to distinct cores. |
+
+## Native transport selection (`src/native/ffi.ts`)
+
+`bun:ffi` is the PRIMARY transport under Bun; NAPI is the fallback (Node,
+`CASTRUM_FFI_MODE=napi`, or a failed bind-time self-test). `rust.transport()`
+reports the resolved transport (`"ffi"` / `"napi"`); `rust.ffiActive()` reports
+whether the ffi transport is live.
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `CASTRUM_FFI_MODE` | string | `auto` | Native transport selection: `auto` (default) — use `bun:ffi` on Bun, silently fall back to napi on bind/self-test failure (and always on Node); `ffi` — force `bun:ffi` on Bun and THROW if it cannot bind or the self-test fails (for benches/CI that must run the primary transport; invalid on Node); `napi` — never bind, every call goes through the napi addon (exercises the fallback on Bun). |
+
+## Native addon loader (`src/native/loader.ts`)
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `CASTRUM_DEBUG` | (presence) | — | Log the resolved addon path and exported keys at load time. |
+| `CASTRUM_NATIVE_LIBRARY_PATH` / `NAPI_RS_NATIVE_LIBRARY_PATH` | path | — | Explicit path (file or directory) to the native `.node` artifact, overriding normal resolution. Useful for bundled/electron layouts. |
+
+## Library logging (`src/shared/log.ts`)
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `CASTRUM_LOG_LEVEL` | string | `info` | Threshold for the built-in structured logger (`debug`/`info`/`warn`/`error`/`silent`). Enabled per handler via the `structuredLog: true` runtime option; emits one JSON line per request/error to stderr (`{ts, level, event, requestId, method, path, status, durationMs, ...}`). |
+
+## Security guidance
+
+- **Proxy trust**: `trustProxy: true` is deprecated — it trusts EVERY hop of
+  `X-Forwarded-For`/`X-Real-IP`, so clients can forge their IP and bypass IP-based
+  rate limiting. Prefer `trustedProxies: { enabled: true, networks: [...] }` and only
+  enable proxy trust behind a trusted edge that strips these headers.
+- **Rate limiting is per-process, not distributed**: with N worker processes
+  (`reusePort`), an IP's effective limit is multiplied by N. The shared-limiter config
+  registry is bounded (LRU, max 16 distinct configs) to prevent unbounded memory growth.
+- **Request bodies**: `createIngressServer` applies a server-level `maxRequestBodySize`
+  default of 16 MiB, and JSON-write routes stream-read with `maxBodyBytes` enforced
+  while reading (an oversized body is rejected as soon as the limit is crossed, never
+  fully buffered). Route/async handlers apply a non-zero body-read deadline by default
+  (`bodyTimeoutMs`, default **30 s** — set `bodyTimeoutMs: 0` to disable) that guards
+  against slowloris/trickling bodies.
+- **Secrets (JWT/AEAD)**: keys and secrets are caller-owned buffers; treat them as
+  sensitive (never log them, rotate regularly). The AEAD **batch** APIs derive a unique
+  nonce per item, but callers MUST use a fresh base nonce for every batch call — never
+  reuse a base nonce with the same key.
+- **Multipart**: `multipart_parse` accepts an optional `limits` object
+  (`max_parts`, `max_field_count`, `max_part_bytes`, `max_total_bytes`) with generous
+  defaults to bound worst-case parse cost.