castrum 0.9.0 → 0.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +153 -630
- package/castrum.linux-x64-gnu.node +0 -0
- package/dist/index.d.ts +1374 -1772
- package/dist/index.js +5570 -4169
- package/docs/API.md +337 -0
- package/docs/ARCHITECTURE.md +106 -14
- package/docs/BENCHMARKS.md +83 -47
- package/docs/CASE_STUDY.md +240 -0
- package/docs/ENVIRONMENT.md +8 -6
- package/docs/FFI_BUN_GUIDE.md +327 -0
- package/docs/GETTING_STARTED.md +15 -32
- package/docs/INGRESS-ROUTER.md +81 -0
- package/docs/INGRESS.md +5 -4
- package/docs/NATIVE-ROUTE.md +53 -0
- package/docs/REPO_MAP.md +46 -26
- package/docs/adr/0003-bun-builtins-delegation.md +6 -5
- package/docs/bun-builtins-decision-matrix.md +25 -18
- package/index.ts +9 -16
- package/package.json +36 -24
- package/src/ingress/body.ts +167 -27
- package/src/ingress/handlers.ts +91 -15
- package/src/ingress/health.ts +1 -0
- package/src/ingress/index.ts +39 -27
- package/src/ingress/packing/gather-raw-headers.ts +85 -21
- package/src/ingress/packing/header-packing.ts +5 -5
- package/src/ingress/packing/input-packer.ts +17 -3
- package/src/ingress/packing/select-headers.ts +12 -6
- package/src/ingress/router.ts +237 -0
- package/src/ingress/routes/common.ts +42 -1
- package/src/ingress/routes/fallback.ts +2 -2
- package/src/ingress/routes/head.ts +3 -13
- package/src/ingress/routes/index.ts +5 -4
- package/src/ingress/routes/json-write.ts +7 -31
- package/src/ingress/routes/options.ts +2 -12
- package/src/ingress/routes/read.ts +3 -13
- package/src/ingress/routes/responder.ts +159 -0
- package/src/ingress/server-node.ts +6 -6
- package/src/ingress/server.ts +45 -13
- package/src/ingress/types.ts +35 -1
- package/src/integration/pipeline.ts +3 -0
- package/src/integration/streaming.ts +2 -3
- package/src/integration/websocket.ts +4 -3
- package/src/loader/batch.ts +4 -0
- package/src/loader/cost.ts +1 -0
- package/src/loader/index.ts +43 -228
- package/src/loader/state.ts +84 -0
- package/src/loader/types.ts +145 -0
- package/src/native/ffi/constants.ts +58 -0
- package/src/native/ffi/selftest.ts +729 -0
- package/src/native/ffi/types.ts +474 -0
- package/src/native/ffi.ts +1063 -829
- package/src/native/loader.ts +52 -6
- package/src/native/types/instances.ts +274 -0
- package/src/native/types.ts +41 -238
- package/src/runtime/builtins.ts +131 -0
- package/src/runtime/codec.ts +147 -0
- package/src/runtime/detect.ts +36 -0
- package/src/runtime/index.ts +28 -0
- package/src/runtime/native.ts +35 -0
- package/src/runtime/server.ts +31 -0
- package/src/runtime/transport.ts +64 -0
- package/src/runtime/types.ts +173 -0
- package/src/runtime/uuid.ts +45 -0
- package/src/runtime/websocket.ts +30 -0
- package/src/rust-ffi/addon.ts +1 -1
- package/src/rust-ffi/batch/build.ts +155 -0
- package/src/rust-ffi/batch/index.ts +8 -0
- package/src/rust-ffi/batch/types.ts +92 -0
- package/src/rust-ffi/client.ts +5 -6
- package/src/rust-ffi/context.ts +39 -10
- package/src/rust-ffi/index.ts +28 -17
- package/src/rust-ffi/into.ts +21 -0
- package/src/rust-ffi/options.ts +1 -1
- package/src/rust-ffi/packed.ts +2 -2
- package/src/rust-ffi/proven.ts +16 -30
- package/src/rust-ffi/scalar/crypto.ts +188 -107
- package/src/rust-ffi/scalar/factories.ts +253 -27
- package/src/rust-ffi/scalar/hashing.ts +43 -41
- package/src/rust-ffi/scalar/http.ts +208 -77
- package/src/rust-ffi/scalar/index.ts +3 -3
- package/src/rust-ffi/scalar/interface.ts +88 -213
- package/src/rust-ffi/scalar/json-packed.ts +175 -0
- package/src/rust-ffi/scalar/json.ts +28 -12
- package/src/rust-ffi/scalar/payload.ts +159 -42
- package/src/rust-ffi/text.ts +31 -5
- package/src/selection.ts +27 -31
- package/src/shared/adaptive.ts +1 -2
- package/src/shared/buffer-pool.ts +72 -17
- package/src/shared/bytes.ts +75 -6
- package/src/shared/codec.ts +78 -0
- package/src/shared/env.ts +4 -4
- package/src/shared/log.ts +1 -0
- package/src/shared/packed/index.ts +10 -0
- package/src/shared/packed/parsers.ts +31 -0
- package/src/shared/packed/schema.ts +23 -0
- package/src/shared/{packed.ts → packed/wire.ts} +31 -39
- package/src/shared/proven.ts +189 -496
- package/src/shared/runtime.ts +20 -25
- package/src/shared/uuid.ts +12 -28
- package/src/data/json-rows.ts +0 -33
- package/src/rust-ffi/batch.ts +0 -431
- package/src/shared/bench-classify.ts +0 -209
package/docs/BENCHMARKS.md
CHANGED
|
@@ -29,55 +29,14 @@ The benchmark system compares the Rust implementations against pure JavaScript/B
|
|
|
29
29
|
bun bench.ts
|
|
30
30
|
```
|
|
31
31
|
|
|
32
|
-
### Performance-proven surface & auto-annotated JSDoc
|
|
33
|
-
|
|
34
|
-
The CPU benchmark persists a machine-readable report to `bench/results/cpu/latest.json`
|
|
35
|
-
(+ timestamped snapshots). Three commands consume it; `bun run check:docs` runs all
|
|
36
|
-
three as the one-stop "informed decision" pipeline:
|
|
37
|
-
|
|
38
|
-
| Command | What it does |
|
|
39
|
-
|---------|--------------|
|
|
40
|
-
| `bun run check` | Runs the CPU benchmark and writes `bench/results/cpu/latest.json` (report-only — **never mutates source**). |
|
|
41
|
-
| `bun run check:annotate` | Rewrites the `@performance` / `@deprecated` / `@remarks` JSDoc on the public `rust.*` declarations (`src/rust-ffi/scalar/interface.ts`, `batch.ts`, `client.ts`) from the report. Idempotent — generated lines carry the `[check:annotate]` marker; hand-written JSDoc is preserved. `--dry-run` previews changes. |
|
|
42
|
-
| `bun run check:proven` | Audits the `PROVEN_SURFACE` registry against the report and flags `REGRESSION`, `PARITY-DRIFT`, and `PROMOTABLE`. `--fail` exits 1 on regressions. |
|
|
43
|
-
| `bun run check:docs` | `check` + `check:annotate` + `check:proven` — the full pipeline. |
|
|
44
|
-
|
|
45
|
-
**Automatic annotation (opt-in):** `CASTRUM_BENCH_ANNOTATE=1 bun run check`
|
|
46
|
-
auto-runs the annotator right after the benchmark, so the JSDoc always reflects
|
|
47
|
-
the latest run. It is opt-in because it mutates tracked source files.
|
|
48
|
-
|
|
49
|
-
**How the annotations work (data-driven):**
|
|
50
|
-
|
|
51
|
-
- Each function's **score** is embedded in the `@performance` line, e.g.
|
|
52
|
-
`JSON valid: ~3.2x faster than the JS baseline (8/8 tasks won)` or
|
|
53
|
-
`CRC32: ~4.1x slower than the JS baseline (2/3 tasks failed)`. A function's
|
|
54
|
-
benchmark set is the canonical comparison plus all its `_`-suffixed variants
|
|
55
|
-
(`_large`, `_batch`, `_concurrent_*`, `_stress`, pooled `_into` overloads)
|
|
56
|
-
that no other registry entry claims.
|
|
57
|
-
- Task bands (see `src/shared/bench-classify.ts`): **won** = `ratio >= 1`
|
|
58
|
-
(Rust at least as fast), **failed** = `ratio < 1 / 1.35` (clear loss), the
|
|
59
|
-
middle is a **tie** that counts toward neither.
|
|
60
|
-
- `@deprecated` is **hybrid**:
|
|
61
|
-
- A clear **majority** of a function's benchmarks failing in the latest run
|
|
62
|
-
auto-deprecates it even when the static registry says `proven`/`parity` — a
|
|
63
|
-
`@remarks Auto-deprecated: N/M benchmarks failed ...` note records the drift.
|
|
64
|
-
- A single good run **never** removes an existing `@deprecated` (static
|
|
65
|
-
classifications reflect the shipped release build, not one local run). A
|
|
66
|
-
majority win on a static `not-competitive` function only adds a
|
|
67
|
-
`@remarks Promotion candidate ...` note.
|
|
68
|
-
- Classifications are **build-dependent** (debug/SIMD builds distort ratios). The
|
|
69
|
-
annotator warns when the report carries no `buildFlavor`; confirm on a release
|
|
70
|
-
build (`bun run build`) or local perf build (`bun run build:perf`).
|
|
71
|
-
|
|
72
32
|
### Bun built-ins diagnostic set ("don't reinvent the wheel")
|
|
73
33
|
|
|
74
34
|
`bun run check` also races castrum ops against **Bun's native built-ins**
|
|
75
35
|
(`Bun.hash`, `Bun.password`, `Bun.CryptoHasher`, `Bun.gzipSync`,
|
|
76
36
|
`Bun.randomUUIDv7`) as a diagnostic set:
|
|
77
37
|
|
|
78
|
-
- Task names use the **`diag:`** prefix (not `native:`/`rust:`) so
|
|
79
|
-
|
|
80
|
-
and their `_`-suffixed variants) never picks them up — they are informational.
|
|
38
|
+
- Task names use the **`diag:`** prefix (not `native:`/`rust:`) so they stay out
|
|
39
|
+
of the shipped-op `native:`/`rust:` comparisons — they are informational.
|
|
81
40
|
- Sources: `src/baseline/tasks/bun-builtins.ts` (Bun-only baselines, guarded
|
|
82
41
|
with `typeof Bun` so the module still loads under Node),
|
|
83
42
|
`src/bench/tasks/bun-builtins.ts`, comparisons in `src/bench/comparisons.ts`.
|
|
@@ -112,8 +71,83 @@ bun run bench:http:bun-only # Run only raw Bun vs baseline
|
|
|
112
71
|
|
|
113
72
|
# Run all heavy scenarios
|
|
114
73
|
bun run bench:http:all-heavy
|
|
74
|
+
|
|
75
|
+
# Max-throughput cross-check (autocannon — no Bun fetch cap)
|
|
76
|
+
bun run bench:http:ac # all servers, 03-stress weighted mix
|
|
77
|
+
bun run bench:http:stress:ac # 03-stress mix only
|
|
78
|
+
|
|
79
|
+
# Single-path static comparison (fastest, most stable signal):
|
|
80
|
+
# node bench/autocannon-stress.mjs # dynamic 03-stress mix
|
|
81
|
+
# AC_PATH=/api/users node bench/autocannon-stress.mjs
|
|
82
|
+
# AC_PATH=/api/users AC_METHOD=POST AC_BODY='{"id":1,"name":"x"}' \
|
|
83
|
+
# AC_CONTENT_TYPE=application/json node bench/autocannon-stress.mjs
|
|
84
|
+
# AC_RUNS=3 node bench/autocannon-stress.mjs # median-of-N (see §Single-core)
|
|
85
|
+
# — run-major interleaving: every server is measured once per round, so
|
|
86
|
+
# the median compares servers over the same load windows on a noisy host.
|
|
87
|
+
#
|
|
88
|
+
# Multi-core scaling (SO_REUSEPORT — the way to exceed the single-core ceiling):
|
|
89
|
+
# AC_INSTANCES=4 AC_PIPELINING=10 node bench/autocannon-stress.mjs
|
|
90
|
+
# (ingress/router only — they read INGRESS_REUSE_PORT=1)
|
|
115
91
|
```
|
|
116
92
|
|
|
93
|
+
### Bun `fetch()` concurrency cap (read this before interpreting HTTP results)
|
|
94
|
+
|
|
95
|
+
Bun's `fetch()` limits the number of **simultaneous** requests to **256 by
|
|
96
|
+
default** (`BUN_CONFIG_MAX_HTTP_REQUESTS`, max 65,336). When a scenario's
|
|
97
|
+
`maxConcurrent` exceeds that cap, the excess requests queue **FIFO client-side**
|
|
98
|
+
and the run measures the *generator*, not the server — the classic signature is
|
|
99
|
+
a sub-ms `p50` with a multi-second tail and a flat RPS wall across every server.
|
|
100
|
+
|
|
101
|
+
- All `bench:http:*` scripts set `BUN_CONFIG_MAX_HTTP_REQUESTS=65536` for you.
|
|
102
|
+
- `bench/run-bench.ts` warns if a selected scenario exceeds the effective cap.
|
|
103
|
+
- For **max-throughput** server comparisons use **autocannon**
|
|
104
|
+
(`bench/autocannon-stress.mjs`, run with `node`): it has no such cap, uses a
|
|
105
|
+
real socket pool, scales across worker threads, and reports RPS + latency
|
|
106
|
+
percentiles natively. It replicates the `03-stress` weighted flow mix
|
|
107
|
+
(50% GET /api/users?q=…&page=…, 30% POST /api/users, 20% GET /health).
|
|
108
|
+
- **Two autocannon gotchas** (both fixed in the runner):
|
|
109
|
+
1. `requestGenerator` is CLI-ONLY — the library silently ignores it (every
|
|
110
|
+
request then goes to `GET /` → 404 → "0 2xx"). The runner uses the
|
|
111
|
+
library's `requests: [{ setupRequest }]` API instead.
|
|
112
|
+
2. `result.workers` is a count (`opts.workers`), not an array — the report no
|
|
113
|
+
longer treats it as a per-worker array.
|
|
114
|
+
|
|
115
|
+
### Single-core ceiling & multi-core scaling (measured 2026-08-16)
|
|
116
|
+
|
|
117
|
+
All four servers are **single-process `Bun.serve`** (one event loop), so they
|
|
118
|
+
pin at a per-core ceiling on this machine. The host is load-noisy, so use the
|
|
119
|
+
**median-of-N** runner (`AC_RUNS=N`, default 1) with **run-major interleaving** —
|
|
120
|
+
every server is visited once per round, so a single slow window affects all
|
|
121
|
+
servers' run-`i` equally and the median compares servers over the same load
|
|
122
|
+
windows (the old server-major order biased later servers into noisier windows).
|
|
123
|
+
> **Dated snapshot (2026-08-16 ingress-RPS pass).** The host is load-noisy;
|
|
124
|
+
> the newest persisted autocannon medians under `bench/results/autocannon/*/final-*.median.md`
|
|
125
|
+
> supersede these numbers run-to-run. Use the median-of-N runner below for fresh
|
|
126
|
+
> comparisons.
|
|
127
|
+
|
|
128
|
+
Current medians (`AC_DURATION=8 AC_RUNS=3`, 500 connections, single instance):
|
|
129
|
+
|
|
130
|
+
- `GET /api/users?q=…` (query + cookies + CORS + metadata envelope):
|
|
131
|
+
**bun 45.2k, elysia 41.7k, ingress 51.5k, router 49.4k RPS** — ingress ~**+14%**
|
|
132
|
+
vs bun, with a tighter tail (p50 9ms / p99 16ms vs bun p50 10ms / p99 28ms).
|
|
133
|
+
- `POST /api/users` (JSON body, fast-path schema validation):
|
|
134
|
+
**bun 40.8k, elysia 35.1k, ingress 45.4k, router 35.1k RPS** — ingress ~**+11%**
|
|
135
|
+
vs bun. The pre-optimization ingress POST gap (≈ −24% vs bun) is closed by
|
|
136
|
+
the changes in `CHANGELOG [Unreleased]` (JS hot-path elimination + single-pass
|
|
137
|
+
body validation).
|
|
138
|
+
- Per-request cost breakdown (`bun bench/ingress-cost.ts` / `ingress-cost-post.ts`,
|
|
139
|
+
min-of-5): GET `run` ~826ns (native 168ns + JS ~446ns packing + decode 60ns);
|
|
140
|
+
POST `run` ~1154ns (native ~740ns — was 808ns before the single-pass — + JS).
|
|
141
|
+
The POST body read is now **synchronous for buffered bodies**: `Bun.peek` on
|
|
142
|
+
`req.bytes()` skips the deadline race + watchdog for declared-length bodies
|
|
143
|
+
(was ~600ns of race machinery per POST).
|
|
144
|
+
- **To exceed the single-core ceiling**, run multiple server processes on the
|
|
145
|
+
same port via SO_REUSEPORT: `AC_INSTANCES=4` (with `AC_PIPELINING` so the
|
|
146
|
+
client can drive enough in-flight requests) scaled ingress from ~45k to
|
|
147
|
+
~69k RPS on this machine. Only ingress/router bind with `reusePort`
|
|
148
|
+
(`INGRESS_REUSE_PORT=1`); bun/elysia do not set it. For cleanest numbers run
|
|
149
|
+
autocannon on a separate host.
|
|
150
|
+
|
|
117
151
|
---
|
|
118
152
|
|
|
119
153
|
## HTTP Benchmark Scenarios
|
|
@@ -134,7 +168,9 @@ Each scenario targets a specific performance aspect:
|
|
|
134
168
|
### 03-stress
|
|
135
169
|
- **Goal**: Ramp to maximum throughput
|
|
136
170
|
- **Duration**: ~100 seconds (5 × 20s ramp phases)
|
|
137
|
-
- **Concurrency**: up to
|
|
171
|
+
- **Concurrency**: up to 2,000 (higher on a single machine only deepens the
|
|
172
|
+
client-side fetch queue — the generator can't drive 10k usefully; see the
|
|
173
|
+
Bun fetch-cap note above. For true max-throughput use `bench:http:ac`.)
|
|
138
174
|
|
|
139
175
|
### 04-spike
|
|
140
176
|
- **Goal**: Burst resilience (baseline → spike → recovery)
|
|
@@ -235,7 +271,7 @@ When comparing multiple frameworks, differences are expressed as:
|
|
|
235
271
|
|
|
236
272
|
```
|
|
237
273
|
bench.ts
|
|
238
|
-
└─ Loads benchmark tasks from src/bench/tasks/ (
|
|
274
|
+
└─ Loads benchmark tasks from src/bench/tasks/ (33 tasks)
|
|
239
275
|
├── hashing.ts Compare Rust fnv1a64 vs JS TextEncoder + hash
|
|
240
276
|
├── validation.ts Compare Rust validation vs validator library
|
|
241
277
|
├── json.ts Compare Rust sonic-rs vs JSON.parse
|
|
@@ -258,7 +294,6 @@ Each task runs:
|
|
|
258
294
|
| Env / flag | Effect |
|
|
259
295
|
|------------|--------|
|
|
260
296
|
| `CASTRUM_BENCH_BATCH_SIZE` | Batch size for sub-µs operations (default `64`). Batching amortizes the timer/measurement overhead for very fast ops. |
|
|
261
|
-
| `CASTRUM_BENCH_ANNOTATE=1` | After `bun run check`, auto-rewrite the `rust.*` JSDoc annotations from the fresh report (see "Performance-proven surface & auto-annotated JSDoc"). |
|
|
262
297
|
| `HTTP_NO_SHAPE=1` | The HTTP load generator (`bench/load.ts`) skips response-shape `JSON.parse` for pure-throughput runs. |
|
|
263
298
|
|
|
264
299
|
---
|
|
@@ -270,7 +305,8 @@ bench/servers/
|
|
|
270
305
|
├── bun-server.ts # Raw Bun.serve with manual routing
|
|
271
306
|
├── elysia-server.ts # Elysia framework
|
|
272
307
|
├── ingress-server.ts # Ingress (this project) optimized handler
|
|
273
|
-
|
|
308
|
+
├── router-server.ts # Ingress per-route compiled-router variant (createIngressRouter)
|
|
309
|
+
└── shared.ts # Shared route handlers, PORTS, env helpers
|
|
274
310
|
```
|
|
275
311
|
|
|
276
312
|
Each server implements the same routes for fair comparison:
|
|
@@ -0,0 +1,240 @@
|
|
|
1
|
+
# Case study — building a Rust-accelerated HTTP pipeline for Bun
|
|
2
|
+
|
|
3
|
+
**castrum** is a hybrid TypeScript + Rust package that gives Bun servers native
|
|
4
|
+
speed for the work that actually costs CPU — request validation, parsing,
|
|
5
|
+
hashing, crypto, and a full HTTP ingress pipeline — behind a small, flat API.
|
|
6
|
+
This document tells the story of how it was built and measured, what the
|
|
7
|
+
benchmarks actually showed, and the lessons that generalise to any
|
|
8
|
+
"should I write this in Rust?" decision.
|
|
9
|
+
|
|
10
|
+
All figures below come from the reports in `bench/results/` (generated on a
|
|
11
|
+
release addon, baseline CPU, Bun 1.4.0). They're **representative**, not
|
|
12
|
+
advertised as universal — re-run `bun run check` / `bun run bench:http` on your
|
|
13
|
+
own hardware before trusting them for your numbers.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Background: the problem
|
|
18
|
+
|
|
19
|
+
Bun is already very fast. The gap that motivated castrum was not "Bun is slow"
|
|
20
|
+
but three specific frictions:
|
|
21
|
+
|
|
22
|
+
1. **CPU-bound request work runs in JavaScript.** JSON validity checking, email
|
|
23
|
+
/ UUID / IP validation, query & cookie parsing, hashing, and schema
|
|
24
|
+
validation are pure CPU. In JS they compete with the event loop; the moment
|
|
25
|
+
a request needs several of them, tail latency climbs.
|
|
26
|
+
2. **Production HTTP servers need a lot of "plumbing" middleware.** CORS,
|
|
27
|
+
rate limiting, body-size guards, schema validation, security headers,
|
|
28
|
+
request IDs, metrics. Hand-wiring these into `Bun.serve` is repetitive and
|
|
29
|
+
error-prone; pulling in a framework drags in a dependency graph.
|
|
30
|
+
3. **Bun and Node want different integration stories.** The same logic should
|
|
31
|
+
run on Bun (the primary target, zero startup cost) and Node ≥ 20.3 (a
|
|
32
|
+
compiled ESM entry) without two implementations.
|
|
33
|
+
|
|
34
|
+
The hypothesis: a **hybrid** design — ergonomic TypeScript on top of one Rust
|
|
35
|
+
cdylib — could remove the CPU cost from the event loop, package the middleware
|
|
36
|
+
as a pre-baked pipeline, and still be usable from both runtimes.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## 2. The approach
|
|
41
|
+
|
|
42
|
+
### 2.1 One cdylib, two transports
|
|
43
|
+
|
|
44
|
+
The addon is built with napi-rs (so Node can load it), but it *also* exports
|
|
45
|
+
`extern "C"` symbols (`rust/ffi.rs`, **79 `castrum_*` symbols** — 71 direct + 4
|
|
46
|
+
`validator_c_abi!` + 4 `compress_to_out!`) so Bun can
|
|
47
|
+
`dlopen` it directly:
|
|
48
|
+
|
|
49
|
+
- **`bun:ffi` (primary on Bun)** — Bun JIT-calls the C-ABI exports at
|
|
50
|
+
~10–20 ns per crossing, roughly an order of magnitude cheaper than a NAPI
|
|
51
|
+
call. A **bind-time self-test** verifies every symbol and the ingress layout
|
|
52
|
+
blob; on *any* failure the layer disables itself and calls fall back to NAPI.
|
|
53
|
+
- **NAPI (fallback)** — used under Node, when `CASTRUM_FFI_MODE=napi`, or when
|
|
54
|
+
the self-test fails. It's the safety net that keeps the same code working on
|
|
55
|
+
both runtimes.
|
|
56
|
+
|
|
57
|
+
```mermaid
|
|
58
|
+
flowchart LR
|
|
59
|
+
TS["TypeScript (index.ts)"] -->|"bun:ffi · 75 C-ABI symbols · ~10–20 ns"| RUST["Rust cdylib (rust/ffi.rs)"]
|
|
60
|
+
TS -.->|"NAPI fallback (Node / self-test failure)"| RUST
|
|
61
|
+
RUST --> PIPELINE["8-stage ingress pipeline"]
|
|
62
|
+
TS --> PACKED["Packed, length-prefixed buffers (zero-copy)"]
|
|
63
|
+
PACKED --> RUST
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### 2.2 Zero-copy, zero-alloc request handling
|
|
67
|
+
|
|
68
|
+
Instead of marshalling objects, the hot path **packs** the request frame
|
|
69
|
+
(method kind, URL, client IP, request ID, headers) into a shared `Uint8Array`,
|
|
70
|
+
Rust parses it and writes a decision into a **single output buffer**, and TS
|
|
71
|
+
decodes only the fields it needs. Hot paths allocate nothing per request;
|
|
72
|
+
pooled output buffers (with an adaptive-estimate size heuristic) are reused.
|
|
73
|
+
|
|
74
|
+
### 2.3 The ingress pipeline as a product, not a library
|
|
75
|
+
|
|
76
|
+
The 8-stage pipeline (trust/IP → HTTPS → CORS → rate limit → body guard →
|
|
77
|
+
JSON/schema → serialize cookies/query → write output) ships twice:
|
|
78
|
+
`createIngressServer` wraps it in a `Bun.serve` route table, and the same
|
|
79
|
+
handlers run over `node:http` via `createIngressServerNode`.
|
|
80
|
+
|
|
81
|
+
> **Two wire formats on purpose.** The low-level fast path emits
|
|
82
|
+
> `{"error":{...}}` with `x-ratelimit-*`; the pre-baked server emits
|
|
83
|
+
> `{"ok":true,...,"requestId":...}` with `ratelimit-*`. They are a **contract**
|
|
84
|
+
> the load generator validates — unifying them would break the benchmark
|
|
85
|
+
> baseline. (Recorded in `docs/adr/0001-two-wire-formats.md`.)
|
|
86
|
+
|
|
87
|
+
---
|
|
88
|
+
|
|
89
|
+
## 3. Results
|
|
90
|
+
|
|
91
|
+
### 3.1 Startup & first call (the "is it instant?" test)
|
|
92
|
+
|
|
93
|
+
From `bench/results/startup/latest.json` (Bun 1.4.0, 15 iterations):
|
|
94
|
+
|
|
95
|
+
| Metric | p50 | p95 |
|
|
96
|
+
|--------|-----|-----|
|
|
97
|
+
| Import the package (raw `index.ts`, no build step) | ~90 ms | ~112 ms |
|
|
98
|
+
| First `rust.crc32` call (ffi transport already warm) | ~0.04 ms | ~0.08 ms |
|
|
99
|
+
| First ingress pipeline call (addon loaded) | ~2.6 ms | ~5.3 ms |
|
|
100
|
+
|
|
101
|
+
The important number is the first-*call* cost: the cdylib is lazy-loaded, so
|
|
102
|
+
the import itself doesn't pay a native-init penalty.
|
|
103
|
+
|
|
104
|
+
### 3.2 CPU: where Rust wins, and where it doesn't
|
|
105
|
+
|
|
106
|
+
The CPU benchmark races every `rust.*` op against a pure-JS baseline **and**
|
|
107
|
+
Bun's built-ins (`docs/bun-builtins-decision-matrix.md`, release addon,
|
|
108
|
+
2026-08-11). Ratio > 1 = Bun is faster.
|
|
109
|
+
|
|
110
|
+
| Op | Bun built-in | Ratio | Verdict |
|
|
111
|
+
|----|--------------|-------|---------|
|
|
112
|
+
| `passwordHash` (argon2id) | `Bun.password.hashSync` | **0.55** (rust 1.83×) | **keep rust** |
|
|
113
|
+
| `passwordVerify` (argon2id) | `Bun.password.verifySync` | **0.53** (rust 1.88×) | **keep rust** |
|
|
114
|
+
| `passwordVerifyBcrypt` | `Bun.password.verifySync` | **0.67** (rust 1.49×) | **keep rust** |
|
|
115
|
+
| `pbkdf2Sha256` | `node:crypto.pbkdf2Sync` | 0.93 (parity) | keep rust |
|
|
116
|
+
| `fnv1a64` | `Bun.hash` (wyhash) | 1.06 | parity |
|
|
117
|
+
| `hmacSha256` | `Bun.CryptoHasher` | 1.1–1.4 | mild Bun win |
|
|
118
|
+
| `randomToken` | `Bun.randomUUIDv7` | 1.62 | **Bun wins** |
|
|
119
|
+
| `gzipDecompress` | `Bun.gunzipSync` | 1.38 | Bun wins* |
|
|
120
|
+
| `gzipCompress` | `Bun.gzipSync` | 2.02 | **Bun wins** |
|
|
121
|
+
| `crc32` | `Bun.hash.crc32` | 2.8–8.4 | **Bun wins** |
|
|
122
|
+
| `xxh3` | `Bun.hash.xxHash3` | 4.15 | **Bun wins** |
|
|
123
|
+
| brotli / zero-DOM JSON / parsers | *no sync Bun equivalent* | — | keep rust |
|
|
124
|
+
|
|
125
|
+
\* `gzipDecompress` deliberately stays native: the Rust path enforces a **64 MiB
|
|
126
|
+
decompression-bomb cap** that `Bun.gunzipSync` lacks.
|
|
127
|
+
|
|
128
|
+
**The honest takeaway:** "native is faster" is *not* automatic. The runtime
|
|
129
|
+
adapter's `builtins` registry (`src/runtime/builtins.ts`) delegates the ops
|
|
130
|
+
where a Bun built-in wins (e.g. `rust.xxh3` loses 4× to `Bun.hash.xxHash3`),
|
|
131
|
+
while argon2id and the zero-DOM JSON path stay native. A consumer-facing selection
|
|
132
|
+
surface (`opImpl` / `isNativeOp` / `opDecision`) lets frameworks bind each op to
|
|
133
|
+
native or JS **once at load time** instead of per call.
|
|
134
|
+
|
|
135
|
+
### 3.3 HTTP: the pipeline holds up under load
|
|
136
|
+
|
|
137
|
+
From `bench/results/*/03-stress.bench.md` (ramp to 10k concurrent, ~104 s):
|
|
138
|
+
|
|
139
|
+
| Server | Achieved RPS | Total requests | Unexpected errors | Shape failures |
|
|
140
|
+
|--------|-------------|----------------|-------------------|----------------|
|
|
141
|
+
| raw `Bun.serve` | 862.5 | 92,174 | 0 | 0 |
|
|
142
|
+
| **castrum ingress** | **904.2** | **94,060** | **0** | **0** |
|
|
143
|
+
|
|
144
|
+
Under extreme concurrency both saturate and show long tail latencies (p99 ~18 s
|
|
145
|
+
on this machine) — that's the load generator over-driving the box, not a bug.
|
|
146
|
+
The meaningful result is that adding the full pipeline (CORS + rate limit +
|
|
147
|
+
body guard + schema + security headers + request IDs) costs **no correctness
|
|
148
|
+
and no measurable throughput** versus a hand-rolled raw server.
|
|
149
|
+
|
|
150
|
+
### 3.4 The bug that almost killed the server: C-ABI panics
|
|
151
|
+
|
|
152
|
+
The most instructive moment: under `11-concurrent-burst`, the ingress server
|
|
153
|
+
**died at ~2.5 s and stayed down** (91% network errors). Root cause: the `bun:ffi`
|
|
154
|
+
`extern "C"` exports had no panic guard. A panic in a napi call becomes a JS
|
|
155
|
+
exception, but a panic unwinding through raw `extern "C"` is undefined behaviour
|
|
156
|
+
that takes the whole process down.
|
|
157
|
+
|
|
158
|
+
The fix (in `rust/ffi.rs`): every fallible / allocating C-ABI export now routes
|
|
159
|
+
its core through `panic_guard` (`catch_unwind`) and reports a sentinel instead
|
|
160
|
+
of panicking. After the fix, the same scenario completes cleanly:
|
|
161
|
+
|
|
162
|
+
| Scenario (post-fix) | Requests | Network errors | Unexpected errors |
|
|
163
|
+
|---------------------|----------|----------------|-------------------|
|
|
164
|
+
| `11-concurrent-burst` | 30,211 | 0 | 0 |
|
|
165
|
+
|
|
166
|
+
**Lesson:** if you expose a raw C ABI to a host runtime, you own the unwind
|
|
167
|
+
boundary. Never assume a panic can't escape — contain it explicitly, and make
|
|
168
|
+
your bind-time self-test exercise the fallible paths.
|
|
169
|
+
|
|
170
|
+
### 3.5 Correctness as a first-class gate
|
|
171
|
+
|
|
172
|
+
The HTTP smoke scenario (`01-smoke`) is a **wire-format guard**, not a
|
|
173
|
+
throughput test: it verifies `ok === true` + `requestId` on success and
|
|
174
|
+
`error.code` / `error.message` on failure for every response. Across the
|
|
175
|
+
scenarios, `shape_failures` and `unexpected_statuses` are 0 — the two wire
|
|
176
|
+
formats are validated every CI run, so a change to the response shape breaks
|
|
177
|
+
the benchmark, not prod.
|
|
178
|
+
|
|
179
|
+
---
|
|
180
|
+
|
|
181
|
+
## 4. Engineering decisions & lessons
|
|
182
|
+
|
|
183
|
+
1. **Two transports, one binary.** Build with napi-rs (portability), export
|
|
184
|
+
`extern "C"` (speed on Bun), and let a **bind-time self-test** pick the safe
|
|
185
|
+
path automatically. Never let a transport failure be silent.
|
|
186
|
+
2. **Benchmark against the built-in, not just the baseline.** Bun's `Bun.hash.*`,
|
|
187
|
+
`Bun.gzipSync`, `Bun.CryptoHasher`, and `Bun.randomUUIDv7` beat an FFI
|
|
188
|
+
crossing on several ops. The rule that emerged: **keep Rust where it wins or
|
|
189
|
+
where Bun has no synchronous equivalent; delegate where Bun wins.** Don't
|
|
190
|
+
reinvent the wheel — `docs/adr/0003-bun-builtins-delegation.md`.
|
|
191
|
+
3. **"Native" is not a synonym for "fast".** The `builtins` registry delegates
|
|
192
|
+
where Bun wins. Let data, not enthusiasm, decide.
|
|
193
|
+
4. **A wire format is a contract.** Two deliberately-different formats are
|
|
194
|
+
pinned by tests and the load generator. If you change the shape, you change
|
|
195
|
+
the benchmark.
|
|
196
|
+
5. **Contain the C ABI.** A panic across `extern "C"` is a crash. Route every
|
|
197
|
+
fallible export through a guard and cover it in the self-test.
|
|
198
|
+
6. **Zero-copy is a discipline.** Packed buffers, pooled outputs, lazy decode —
|
|
199
|
+
every allocation you remove from the hot path is tail latency you never pay.
|
|
200
|
+
7. **Be honest about scope.** The rate limiter is **per-process, not
|
|
201
|
+
distributed**; proxy trust is **off by default** (spoof-proof); the
|
|
202
|
+
decompression bomb cap is **64 MiB**. Documenting the boundaries is as
|
|
203
|
+
important as the speed.
|
|
204
|
+
|
|
205
|
+
---
|
|
206
|
+
|
|
207
|
+
## 5. Reproduce it yourself
|
|
208
|
+
|
|
209
|
+
```bash
|
|
210
|
+
bun install
|
|
211
|
+
bun run build # release addon (baseline CPU — what ships)
|
|
212
|
+
|
|
213
|
+
bun run check # CPU benchmark → bench/results/cpu/latest.json
|
|
214
|
+
bun run bench:startup # import + first-call timing
|
|
215
|
+
bun run bench:http:smoke # HTTP wire-format guard
|
|
216
|
+
bun run bench:http # all HTTP scenarios (bun / elysia / ingress)
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
See [`BENCHMARKS.md`](./BENCHMARKS.md) for scenario descriptions and how to read
|
|
220
|
+
the reports.
|
|
221
|
+
|
|
222
|
+
---
|
|
223
|
+
|
|
224
|
+
## 6. When this pattern applies (and when it doesn't)
|
|
225
|
+
|
|
226
|
+
The hybrid TS + Rust + FFI pattern paid off here because the workload was
|
|
227
|
+
**CPU-bound, synchronous, and high-volume**. Reach for it when:
|
|
228
|
+
|
|
229
|
+
- Request handling spends measurable time in validation/parsing/hashing.
|
|
230
|
+
- You want a pre-baked HTTP pipeline (CORS, rate limit, schema, security
|
|
231
|
+
headers) without a heavy middleware graph.
|
|
232
|
+
- You must support both Bun and Node from one codebase.
|
|
233
|
+
|
|
234
|
+
Don't reach for it when:
|
|
235
|
+
|
|
236
|
+
- The bottleneck is I/O or the database, not CPU — Rust won't help.
|
|
237
|
+
- You only need a handful of ops that Bun's built-ins already win (use
|
|
238
|
+
`Bun.hash.crc32`, `Bun.gzipSync`, `Bun.randomUUIDv7` directly).
|
|
239
|
+
- You can't ship/ build a native `.node` in your deployment (no Rust toolchain
|
|
240
|
+
in CI, or a locked-down runtime that forbids native addons).
|
package/docs/ENVIRONMENT.md
CHANGED
|
@@ -10,7 +10,9 @@ warned about and replaced with the documented default.
|
|
|
10
10
|
|----------|------|---------|-------------|
|
|
11
11
|
| `INGRESS_TRUST_PROXY` | bool | `false` | **Opt-in.** When `true`, honors `X-Forwarded-For` / `X-Real-IP` for client-IP resolution. Keep **off** unless the server sits behind a trusted edge that strips these headers, otherwise clients can spoof arbitrary IPs for IP-based rate limiting. |
|
|
12
12
|
| `INGRESS_REQUEST_ID_HEADER` | bool | `false` | Emit an `x-request-id` response header per request. The request ID is always generated internally; this only controls whether it is echoed to clients. |
|
|
13
|
-
| `
|
|
13
|
+
| `INGRESS_ZERO_COPY` | bool | `false` | Enable zero-copy response bodies (legacy alias `INGRESS_UNSAFE_ZERO_COPY` still honored). **Not recommended** unless per-response output buffers are implemented; leave off for safety (bodies are copied by default). |
|
|
14
|
+
| `INGRESS_ZERO_COPY_MAX_IN_FLIGHT` | int | `128` | Max pooled output buffers kept in flight for zero-copy responses (min `1`). |
|
|
15
|
+
| `INGRESS_ZERO_COPY_TIMEOUT_MS` | int | `1000` | Timeout before a zero-copy pooled buffer is force-released (min `0`). |
|
|
14
16
|
| `INGRESS_OUTPUT_BUF_BYTES` | int | `131072` | Size of the Rust output buffer (min `65536`). If cookies + query + metadata JSON exceed this, `FLAG_BODY_TRUNCATED` is set and the request fails closed. |
|
|
15
17
|
| `INGRESS_SECURITY_HEADERS` | bool | `true` | Emit the precomputed security headers (CSP, HSTS, X-Frame-Options, etc.). Disable only behind an edge that already sets them. |
|
|
16
18
|
| `INGRESS_REUSE_PORT` | bool | `false` | Enable `SO_REUSEPORT` for multi-process deployments (spawn N processes, each with a distinct `INGRESS_WORKER_ID`). |
|
|
@@ -27,9 +29,9 @@ for false; anything else is warned about and falls back to the default.
|
|
|
27
29
|
|
|
28
30
|
| Variable | Type | Default | Description |
|
|
29
31
|
|----------|------|---------|-------------|
|
|
30
|
-
| `CASTRUM_RAYON_THREADS`
|
|
31
|
-
| `CASTRUM_MAX_RAYON_THREADS`
|
|
32
|
-
| `CASTRUM_PIN_CORES`
|
|
32
|
+
| `CASTRUM_RAYON_THREADS` | int | `max(1, cores-1)` | Rayon thread pool size. The pool is process-wide and initialized **once** — the first use wins. With lazy init, call `rust.configure({ rayonThreads })` (or set this env) before the first batch operation. |
|
|
33
|
+
| `CASTRUM_MAX_RAYON_THREADS` | int | `cores` | Hard cap on rayon threads (native). |
|
|
34
|
+
| `CASTRUM_PIN_CORES` | (presence) | — | When set (Linux), pin rayon worker threads to distinct cores. |
|
|
33
35
|
|
|
34
36
|
## Native transport selection (`src/native/ffi.ts`)
|
|
35
37
|
|
|
@@ -40,13 +42,13 @@ whether the ffi transport is live.
|
|
|
40
42
|
|
|
41
43
|
| Variable | Type | Default | Description |
|
|
42
44
|
|----------|------|---------|-------------|
|
|
43
|
-
| `CASTRUM_FFI_MODE`
|
|
45
|
+
| `CASTRUM_FFI_MODE` | string | `auto` | Native transport selection: `auto` (default) — use `bun:ffi` on Bun, silently fall back to napi on bind/self-test failure (and always on Node); `ffi` — force `bun:ffi` on Bun and THROW if it cannot bind or the self-test fails (for benches/CI that must run the primary transport; invalid on Node); `napi` — never bind, every call goes through the napi addon (exercises the fallback on Bun). |
|
|
44
46
|
|
|
45
47
|
## Native addon loader (`src/native/loader.ts`)
|
|
46
48
|
|
|
47
49
|
| Variable | Type | Default | Description |
|
|
48
50
|
|----------|------|---------|-------------|
|
|
49
|
-
| `CASTRUM_DEBUG`
|
|
51
|
+
| `CASTRUM_DEBUG` | (presence) | — | Log the resolved addon path and exported keys at load time. |
|
|
50
52
|
| `CASTRUM_NATIVE_LIBRARY_PATH` / `NAPI_RS_NATIVE_LIBRARY_PATH` | path | — | Explicit path (file or directory) to the native `.node` artifact, overriding normal resolution. Useful for bundled/electron layouts. |
|
|
51
53
|
|
|
52
54
|
## Library logging (`src/shared/log.ts`)
|