castrum 0.9.8 → 0.9.10

package/docs/ENVIRONMENT.md

@@ -38,8 +38,15 @@ for false; anything else is warned about and falls back to the default.
 | Variable | Type | Default | Description |
 |----------|------|---------|-------------|
 | `CASTRUM_TASK_THREADS` | int | `max(1, cores-1)` | Worker threads for the off-thread task runtime (castrum Tasks). Read when the pool first starts — the pool is process-wide and cannot be resized afterwards (`tasks.stats().threads` reports the live count). |
+| `CASTRUM_TASK_QUEUE_MAX` | int | `4096` | Admission bound on the off-thread task work queue. Read when the pool first starts (minimum 1). A submit that would exceed it is rejected rather than growing memory: the native submit returns `2` and the task promise rejects with a typed `OVERLOADED` error. |
 | `CASTRUM_TASK_PIN_CORES` | (presence) | — | When set (Linux), pin each task worker to its own core (core 0 is skipped so the JS event loop keeps it). `CASTRUM_PIN_CORES` enables it too. Off by default: pinning competes with the JS/rayon threads for the same box. |
 
+## Compiled-schema cache (`rust/ingress/schema_cache.rs`)
+
+| Variable | Type | Default | Description |
+|----------|------|---------|-------------|
+| `CASTRUM_SCHEMA_CACHE_MAX` | int | `128` | Max distinct request-body schemas retained process-wide. `Ingress`/`NativeRoute` share one compiled `IngressSchema` per distinct schema byte string; this bounds that LRU. Read at first schema compile (the cache is process-wide and cannot be resized afterwards). Values below 1 (unset, unparseable, `0`) fall back to the default; values are clamped to `4096`. Call `flushMemory()` to drop the cache explicitly. |
+
 ## Native transport selection (`src/native/ffi.ts`)
 
 `bun:ffi` is the PRIMARY transport under Bun; NAPI is the fallback (Node,

package/docs/RND-CONCURRENCY.md

@@ -69,7 +69,8 @@ documented `JSTypedArrayBytesDeallocator` hook crashes the process (fault at
 
 ```
 rust/task/runtime.rs     condvar pool, N = cores−1 (CASTRUM_TASK_THREADS), auto-start on submit,
-                         batch dequeue + bounded spin-then-park, optional core pinning
+                         batch dequeue + bounded spin-then-park, optional core pinning,
+                         HARD admission bound on the queue (CASTRUM_TASK_QUEUE_MAX)
 rust/task/completion.rs  result ring + ARMED coalescing doorbell + needed-size drain layout
 rust/task/ops.rs         op dispatch (header + payload split), cancellation set, catch_unwind
                          containment, `_into` (zero-copy output) and slice (zero-copy input) forms
@@ -121,6 +122,17 @@ right one is picked per op:
    copied and the payload is read in place, so a large argument never crosses the
    boundary. This one was forced by a measurement, not symmetry (§7b).
 
+**Bounded admission (backpressure).** The work queue is HARD-bounded by
+`CASTRUM_TASK_QUEUE_MAX` (default 4096, minimum 1; read once when the pool is
+created). A submit that would exceed the bound is rejected instead of growing
+memory: `castrum_task_submit{,_out,_slice}` return `2 = overloaded`, and the JS
+runtime rejects the task promise with a typed `code: 'OVERLOADED'` error. The
+existing return meanings are unchanged — `1` accepted, `0` invalid args / panic.
+A rejected zero-copy submit drops its `output`/`keep` reference exactly like any
+other rejection, so a full queue cannot leak buffers. (The richer M2 design — a
+`WouldBlock` + re-queue-behind-drain policy and priority levels — remains a
+later milestone; v1 is a hard cap with a distinct overload outcome.)
+
 **Measured comparison** (`bun run bench:task`, Bun 1.4.2, 12 cores):
 
 | variant | time | loop stall | note |
@@ -466,9 +478,12 @@ rust/task/
   as in rayon/Go. Deliberately **separate from the rayon global pool**: rayon
   stays the batch executor for `par_iter`; the task runtime owns offloaded ops.
   `CASTRUM_TASK_THREADS` sizes it (default `cores − 1`).
-- **Admission/backpressure**: bounded queue (`CASTRUM_TASK_QUEUE_MAX`, default
-  4× threads). `submit` returns `WouldBlock` when full; JS resolves by queueing
-  behind a `drain` promise — no unbounded growth (fixes F6).
+- **Admission/backpressure** (v1 shipped): the queue is hard-bounded by
+  `CASTRUM_TASK_QUEUE_MAX` (default 4096, min 1, read once at pool creation).
+  `submit` returns `false` when full; the C ABI reports `2 = overloaded` and JS
+  rejects that promise with a typed `OVERLOADED` error — no unbounded growth
+  (fixes F6). M2 replaces this hard cap with `WouldBlock` + requeue-behind-drain
+  and priority levels; v1 has neither.
 - **Cancellation**: each task carries an `Arc<CancelToken>` (atomic + reason).
   Ops check it at safepoints (between chunks: gzip blocks, schema subtrees,
   Argon2 is atomic and reports "not cancellable"). Scope cancellation cascades
@@ -618,6 +633,7 @@ Each of these removes syscalls, lock traffic, or reallocations from the hot path
 | **Zero-copy INPUT** (`castrum_task_submit_slice` + the header/payload split in `ops.rs`) | Forced by a measurement: packing the payload into the args blob copied the whole input on the JS thread, making offloaded `gzipCompress` of 24 MiB **18.6 ms / 11.7 ms stall against 5.6 ms synchronous** — a net loss on both axes. Reading the payload in place: **7.3 ms / 2.1 ms**. The split also simplified every op arm (no more `args[4..]` offset arithmetic). |
 | **No BigInt, no per-completion DataView** | The drain loop reads ids and lengths as two `u32` reads at an offset — `getBigUint64` boxes a BigInt and `readU64` allocated a DataView *per completion*. Exact for any value under 2^53 (ids are a JS counter, sizes are byte counts). A zero-copy completion's body is never materialized at all. |
 | **Reusable args scratch** (copy path only) | Rust copies args on submit, so one growable buffer serves every synchronous submission. Deliberately NOT used for secret-bearing ops (pbkdf2/argon2) — a shared buffer would keep plaintext passwords alive far longer than a per-call allocation — nor for zero-copy retries, which re-submit after an `await` and must own their memory. |
+| **Bounded admission** (`CASTRUM_TASK_QUEUE_MAX`, default 4096) | A hard cap turns sustained overload into an explicit rejection (`2 = overloaded` → typed `OVERLOADED`) instead of unbounded queue/`inflight` growth. The bound is a length check inside the same lock that pushes, so it is one comparison on a path that already locks. |
 
 ---
 

package/docs/bun-builtins-decision-matrix.md

@@ -31,7 +31,7 @@
 |---|---|---|---|---|
 | `rust.fnv1a64` | `Bun.hash` (wyhash) | 42 B | Bun (parity) | 1.06× |
 | `rust.crc32` | `Bun.hash.crc32` | 42 B | **Bun** | 2.8–8.4× |
-| `rust.xxh3` (new) | `Bun.hash.xxHash3` | 42 B | **Bun** | 4.15× |
+| `rust.xxh3` (new) | `Bun.hash.xxHash3` | 42 B–1 MB | **Bun** (≥1 KB) | 0.97× (≤64 B); ~2.0× (≥1 KB) |
 | `rust.hmacSha256` | `Bun.CryptoHasher("sha256", key)` | 32 B | Bun (mild) | 1.1–1.4× |
 | `rust.passwordHash` (argon2id) | `Bun.password.hashSync` (argon2id) | 12 B | **rust** | 0.55× (rust 1.83×) |
 | `rust.passwordVerify` | `Bun.password.verifySync` | 12 B | **rust** | 0.53× (rust 1.88×) |
@@ -62,6 +62,14 @@
 These rust ops lose to Bun's native implementation on the SAME workload — the
 FFI crossing cannot beat Bun's in-process C++:
 
+> Re-swept 2026-09-19 (64 B → 1 MB, interleaved medians + per-call input
+> rotation): the hex and url rows in this table no longer lose — rust hex
+> encode/decode is now 4.7–18.9× / 1.4–6.7× over Buffer, the url codec
+> ~2–3× over `encodeURIComponent`, and base64 decode wins 1.4–6.0×. Only the
+> rows still marked "Prefer Bun" below remain genuinely faster in Bun. See
+> the rows themselves and `src/runtime/builtins.ts` for what the adapter
+> currently delegates.
+
 This table mirrors `BUILTIN_OPS` in `src/runtime/builtins.ts` (the 11 ops that
 delegate to Bun built-ins under Bun; `src/selection.ts` derives from it). `fnv1a64` is parity/either but is NOT delegated
 (`Bun.hash` is wyhash — a different algorithm — and rust `fnv1a64` stays the
@@ -70,13 +78,13 @@ benchmarked impl).
 | Op | Recommendation |
 |---|---|
 | `crc32` | Prefer `Bun.hash.crc32`; keep rust as fallback (pure-TS path already exists). |
-| `xxh3` | Prefer `Bun.hash.xxHash3` (~4×); keep the rust export as the Node/non-Bun fast path. The runtime adapter delegates the public `rust.xxh3` to Bun under Bun. |
+| `xxh3` | Prefer `Bun.hash.xxHash3` (~2× at ≥1 KB via AVX2; parity ≤64 B). The rust core is FLAT ~30 GB/s at every size (re-swept 2026-09-19 — the gap is Bun's AVX2 width, not a size pathology); keep it as the Node/non-Bun fast path. The adapter delegates the public `rust.xxh3` to Bun under Bun. |
 | `gzipCompress` | Prefer `Bun.gzipSync` (~2×). **`gzipDecompress` is deliberately NOT delegated** — `Bun.gunzipSync` has no decompression-bomb cap; the rust surface keeps its native 64 MiB-capped path under Bun. |
 | `randomToken` | Prefer `Bun.randomUUIDv7()` (or `crypto.getRandomValues`) for token-sized output; keep rust for byte-precise control. |
 | `hmacSha256` | Prefer `Bun.CryptoHasher("sha256", key)` (1.17× — mild; keep rust batch path, which wins on larger inputs). |
-| `urlEncode` / `urlDecode` | Prefer `encodeURIComponent` / `decodeURIComponent` (JSC string builtins — ~11.5×, zero alloc). |
-| `base64Encode` / `base64UrlEncode` | Prefer `Buffer.toString('base64'|'base64url')`. |
-| `hexEncode` | Prefer `Buffer.toString('hex')`. |
+| `urlEncode` / `urlDecode` | Fan-out. Re-swept 2026-09-19 on escape-heavy input the rust LUT+run-copy codec is ~2.8–3.2× (encode) / ~2× (decode) FASTER than `encodeURIComponent` / `decodeURIComponent` — the earlier ~11.5× reading was input/profile-dependent. The current adapter still delegates via `BUILTIN_OPS`. |
+| `base64Encode` / `base64UrlEncode` | rust wins ≤1 KB (encode 1.8–2.3×); parity ≥16 KiB (0.88–1.07× — both AVX2 and memory-bound). Base64 **decode** is the strong rust play (1.4–6.0×). Fine to prefer rust for both; the adapter currently delegates encode to Buffer per `BUILTIN_OPS`. |
+| `hexEncode` / `hexDecode` | Prefer rust: encode 4.7–18.9×, decode 1.4–6.7× over Buffer (faster-hex SIMD + single-pass AVX2 decode kernel; re-swept 2026-09-19 — previously Buffer won). The adapter still delegates to Buffer per `BUILTIN_OPS`. |
 | `httpDate` | Prefer `Date.toUTCString()`. |
 
 **Delegation rule of thumb**: rust stays the default only where it beats its
@@ -96,7 +104,7 @@ genuinely wins or Bun has no equivalent.
 ### → Implemented primitives (gaps closed)
 | Primitive | Outcome |
 |---|---|
-| `rust.xxh3` (XXH3-64) | Exposed; measured `Bun.hash.xxHash3` 4.15× faster → the runtime adapter delegates the public `rust.xxh3` to Bun under Bun (Node keeps the addon). |
+| `rust.xxh3` (XXH3-64) | Exposed; re-swept 2026-09-19: parity ≤64 B, Bun ~2× at ≥1 KB (its AVX2 width; rust is flat 30 GB/s) → the adapter still delegates under Bun (Node keeps the addon path). |
 | `rust.passwordHashBcrypt` / `rust.passwordVerifyBcrypt` | bcrypt `$2b$` PHC; hash parity, verify rust 1.49×. |
 | `rust.pbkdf2Sha256` | PBKDF2-HMAC-SHA256; parity with `node:crypto`, the only sync option in Bun. |
 | UUIDv7 (`uuidv7()`) | `Bun.randomUUIDv7` already wins — delegated to Bun (`crypto.randomUUID` on Node), not built in Rust. |
@@ -108,7 +116,9 @@ random tokens, and gzip, Bun 1.4's built-ins are already faster than any
 FFI-crossing Rust op — the selection layer should prefer them. Rust retains a clear,
 defensible moat exactly where it matters: **argon2id password hashing
 (≈1.9× vs Bun)**, the **zero-DOM / zero-copy parsers and JSON paths** (no Bun
-sync equivalent), and **brotli** (no Bun sync API).
+sync equivalent), **brotli** (no Bun sync API), and — since the 2026-09-19
+re-sweep — the **hex and URL byte codecs and base64 decode** (1.4–18.9× over
+the corresponding Buffer / URI built-ins at every size).
 
 ## Re-run