castrum 0.7.0 → 0.9.1

package/docs/NATIVE-ROUTE.md

@@ -0,0 +1,53 @@
+# Native route stack — the `@ignex/native` wire (route-wire v3)
+
+`rust/ingress/native_route.rs` exposes the **per-route native stack**: a route
+descriptor compiles once into a pre-baked `NativeRoute`, then each request runs
+ONE native call. It is the LIVE external wire consumed by `@ignex/native`'s
+`createNativeRoute` — a `castrum` consumer that compiles routes natively.
+
+> **History**: the old `rust/route.rs` was dead external-project wire (a THIRD
+> wire format, never bound in `src/native/ffi.ts`) and was removed in v0.9.0.
+> `native_route.rs` implements the now-live ignex contract and supersedes it.
+
+## Route-wire v3
+
+- Magic `ROUT` (0x524f5554), version 3.
+- Stage tags: `parseQuery = 0 … requireJsonBody = 5`.
+- Result layout: `[flags u32][errorCode u32]` + optional query/cookie pair
+  sections.
+
+Descriptor/stage/part tags + result layout must match `route-wire.ts` EXACTLY —
+`ROUTE_DESC_VERSION` bumps on any wire change (a mismatched compiler/addon must
+be a hard reject, never a silent misparse).
+
+## Parse semantics (LENIENT)
+
+Byte-parity with ignex's JS `queryPairs` / `cookiePairs`: malformed `%ZZ` /
+invalid-UTF-8 `%FF` pass through raw, `+` → space, `%2B` → `+`, cookies trim +
+DQUOTE-unwrap the VALUE but not the name, no cookie URL-decoding. Do NOT reuse
+the strict scalar `query_parser` for this wire.
+
+## Validation
+
+The stack validates the BODY only (via `IngressSchema`); a non-body schema in
+the descriptor is an unsupported feature → fail compile so the caller falls
+back to JS. `requireJsonBody` → 400; `validateBody` schema fail → 422;
+first-failure-wins in stage order.
+
+## Surfaces
+
+- `castrum_route_compile` / `castrum_route_run` / `castrum_route_destroy`
+  (C-ABI, needed-size convention, panic-guarded, immutable `&self` run).
+- napi `Route` class (Node / fallback path).
+
+## Tests / parity
+
+- `rust/ingress/native_route.rs` unit tests + `rust/ffi.rs` C-ABI tests.
+- `test/unit/ingress/native-route.test.ts`.
+- `scripts/verify-native-route.ts` + ignex's `route-wire.test.ts` /
+  `packages/native/test/route.test.ts`.
+
+> **Naming note (flux → ignex)**: the external consumer was historically called
+> `@flux/native` (castrum v0.8.0, guarded by `test/compat/flux-contract.test.ts`).
+> The current live wire is `@ignex/native`. Both names appear in changelogs;
+> `@ignex/native` is the active contract.

package/docs/REPO_MAP.md

@@ -0,0 +1,279 @@
+# castrum — Repo Map (what is where, and why it works this way)
+
+This is the **navigation doc**. It tells you where everything lives, how the
+pieces connect, and how to run/build/test/bench/publish. If you are brand new,
+read [`GETTING_STARTED.md`](./GETTING_STARTED.md) first; if you want deep-dive
+architecture, read [`ARCHITECTURE.md`](./ARCHITECTURE.md). If you are an AI
+agent editing this repo, [`AGENTS.md`](../AGENTS.md) is the agent-facing map.
+
+---
+
+## 1. What this project is
+
+`castrum` is a hybrid **Bun (TypeScript) + Rust** package:
+
+- A **Rust cdylib** (`castrum.<platform>-<arch>.node`, built with napi-rs,
+  ALSO exporting `extern "C"` symbols for Bun's `bun:ffi` — see
+  `rust/ffi.rs`) provides the performance-critical primitives (parsers,
+  crypto, hashing, JSON, compression, and the HTTP **ingress pipeline**).
+  Under Bun, `bun:ffi` is the PRIMARY transport; NAPI is the fallback
+  (Node, `CASTRUM_FFI_MODE=napi`, or a failed ffi self-test).
+- **TypeScript** (`src/`) provides the ergonomic public API: the flat `rust.*`
+  FFI client and the ingress layer.
+- It ships benchmarks: a **CPU benchmark** (`bench.ts` → `src/bench`) and an
+  **HTTP benchmark** (`bench/run-bench.ts` + `bench/servers/*`).
+
+Bun is the primary runtime; Node.js ≥20.3 is supported via a compiled ESM entry
+(`dist/`).
+
+---
+
+## 2. Command cheat-sheet
+
+| Task | Command | Notes |
+|------|---------|-------|
+| Install deps | `bun install` | |
+| Build Rust addon (release) | `bun run build` | `napi build --release --platform` |
+| Build Rust addon (LOCAL max perf) | `bun run build:perf` | x86-64-v3 + AVX2 — **never for publish** |
+| Build Rust addon (debug) | `bun run build:debug` | |
+| Build compiled JS entry (Node) | `bun run build:js` | bundle + types → `dist/` (gitignored) |
+| TS unit tests | `bun test` | ~540 tests, `test/unit/**` |
+| Rust unit tests | `bun run test:rust` | `cargo test`, ~440 tests |
+| Node smoke tests | `bun run test:node` | explicit `.mjs` paths (Node 24 rejects dir arg) |
+| Installed-tarball e2e | `bun run verify:install` | pack → install → import from `node_modules` |
+| Typecheck | `bun run typecheck` | `bunx tsc --noEmit` (only `index.ts`, `bench.ts`, `src/`) |
+| CPU benchmark + correctness | `bun run check` | == `bun bench.ts`; writes `bench/results/cpu/` |
+| HTTP bench (all servers) | `bun run bench:http` | |
+| HTTP smoke (fast sanity) | `bun run bench:http:smoke` | **the wire-format guard** (CI-gated) |
+| Startup bench | `bun run bench:startup` | import + first-call timing |
+| JS lint (Biome) | `bun run lint` / `lint:fix` | `bunx biome check [--write] .` |
+| JS format (Biome) | `bun run format` | `bunx biome format --write .` |
+| Version consistency | `bun run check:version` | package.json ↔ Cargo.toml ↔ CHANGELOG |
+| JS dependency audit | `bun run audit` | `bun audit` (reports advisories) |
+| Cargo deny audit | `bun run deny` | `cargo deny check` |
+| Publish (manual, single-platform) | `bun run publish:manual[:dry]` | see §6 |
+
+---
+
+## 3. Directory map
+
+### Source (what you edit)
+
+```
+index.ts                  Package entry (Bun). Re-exports src/rust-ffi, shared helpers, src/ingress.
+bench.ts                  CPU benchmark entry point → src/bench/run.ts.
+
+examples/                 Runnable sample app (basic-server.ts + README) — see also README Quick Start.
+  basic-server.ts         Minimal pre-baked ingress server + rust.* primitives.
+
+src/
+  native/                 The native transport layer: bun:ffi (primary on Bun) + NAPI fallback.
+    types.ts              NativeAddon module-surface interface (mirrors index.d.ts).
+    types/instances.ts    Per-class INSTANCE types (Ingress, SchemaValidator, JwtSigner, ...) + result types.
+    loader.ts             Addon path resolution (multi-root) + getAddon()/lazyAddon().
+    ffi.ts                bun:ffi C-ABI bindings (Bun-only): lazy bind + bind-time self-test,
+                          CASTRUM_FFI_MODE gating, castrum_ingress_layout blob for constants.ts.
+    ffi/                  ffi/types.ts (BunFFI interface) + constants.ts + selftest.ts (bind-time self-test).
+    index.ts              Barrel.
+  runtime/                THE RUNTIME ADAPTER seam (Bun/Node) — selected once at load.
+    detect.ts             Cached runtime detection — the ONLY place `typeof Bun` is checked.
+    types.ts              RuntimeAdapter contract (codec/uuid/env/builtins/transport/server/websocket).
+    codec.ts              Runtime-native UTF-8 codec (Bun transfer vs TextEncoder/Decoder).
+    uuid.ts               uuidv7 (Bun.randomUUIDv7 vs crypto.randomUUID).
+    builtins.ts           BUN_WINS delegation registry (single source; `has(op)`).
+    transport.ts          bun:ffi-first `resolve(op)` + ffi/napi surfaces (lazy bind).
+    native.ts             FFI-facing seam (codec/uuid/env/builtins/transport) — imported by rust-ffi.
+    server.ts             createIngressServer: Bun.serve on Bun, node:http on Node.
+    websocket.ts          101 upgrade on Bun; clear Bun-only error on Node.
+    index.ts              FULL adapter `runtime` (native + server + websocket) — for the public API.
+  rust-ffi/               The flat `rust.*` FFI client (TS wrappers over the addon).
+    client.ts             createRust() factory + the default `rust` instance + configure().
+    scalar/ / text.ts / packed.ts   namespaces (buildScalar, buildText, ...).
+    batch/                Array-of-bytes batch namespace: types.ts (RustBatch interface) + build.ts (impl) + index.ts.
+    context.ts            Per-instance state (caches, rayon-pool bookkeeping, runtime seam).
+    options.ts            RustOptions + rayon-thread resolution + coercion helpers.
+    addon.ts              The single shared lazy addon proxy.
+    index.ts              Barrel (exports rust, createRust, types).
+  shared/                 Cross-cutting helpers.
+    runtime.ts            Facade over src/runtime/detect.ts (isBun/isNode — cached).
+    codec.ts              Facade over src/runtime/codec.ts.
+    uuid.ts               Facade over src/runtime/uuid.ts.
+    bytes.ts              encoder/decoder (Uint8Array|string union normalization) + toPlainBuffer.
+    packed/               Packed wire-format helpers (split from the old packed.ts monolith).
+      wire.ts             PURE byte encode/decode (u32/bitset/i64/byte-results/multipart, pairs, pack-scratch) — no addon.
+      schema.ts           SchemaValidator alias + schemaValidateBatch/Count.
+      parsers.ts          lazy-addon string parsers (parseQueryString/parseCookieHeader/parseFormBody).
+      index.ts            Barrel (existing `../shared/packed` imports unchanged).
+    request-id.ts         Zero-alloc request-id generator (both ingress paths).
+    buffer-pool.ts        Generic reusable byte-buffer pool (pooled ingress output).
+    response.ts           pooledBodyResponse (releases the pool on body consume).
+    log.ts                createStructuredLogger (CASTRUM_LOG_LEVEL-gated JSON lines).
+  ingress/                The HTTP ingress pipeline (two paths — see §4.2).
+    fast.ts               PATH 1: createIngressFast (packed-input via handleRequestPacked).
+    handlers.ts           PATH 2: createIngressHandler (JS-packs the frame via IngressInputPacker
+                          + gatherRawHeadersPacked → handleRequestPacked).
+    index.ts              Barrel + createIngress / createIngressSync convenience factories.
+    shared.ts             JS constants + buildHeaderPlan + assertSyncCallback (shared by both paths).
+    constants.ts          Binary-layout constants read from Rust at runtime (SINGLE SOURCE).
+    status.ts / errors.ts / options.ts / types.ts / body.ts / context.ts
+    packing/              header-packing.ts, input-packer.ts, gather-raw-headers.ts
+    headers/              cors.ts, hsts.ts, fast-templates.ts, baked-templates.ts
+    decode/               fast-result.ts, baked-result.ts   (TWO decoders — see §4.2)
+    response/             terminal.ts, error-bodies.ts
+    routes/               read/head/json-write/echo/fallback factories + common.ts
+    server.ts             createIngressServer (Bun.serve) + buildRouteHandlers + gracefulShutdown.
+    server-node.ts        createIngressServerNode (node:http adapter, same route handlers).
+  integration/            Framework integration (wraps path 2 — createIngressHandler).
+    pipeline.ts           createPipeline (framework-agnostic request stage for Hono/Elysia/Bun.serve).
+    websocket.ts          createWebSocketUpgrade (RFC 6455 101 handshake + subprotocol).
+    streaming.ts          sseResponse (SSE frames via rust.sseEncodeEvent).
+  baseline/               Pure-JS baseline implementations (benchmark reference). Internal.
+  bench/                  CPU benchmark framework (tasks, measure, report, checks, comparisons).
+  data/                   json-rows.ts fixture generator (benchmark-only).
+```
+
+```
+rust/                     ONE cdylib crate (Cargo [lib] → lib.rs).
+  lib.rs                  Declaration hub + module map + crate docs.
+  util/                   Shared infrastructure: bytes, packed, batch (+batch_core), threadpool, validation.
+  http/                   HTTP wire formats & parsing: headers, method, http/cookie/query parsers,
+                          form, media_type, url_codec, url_join, etag, accept, mime_lookup, multipart.
+  crypto/                 Auth & hashing: hmac_sha256, cookie_sign, csrf, jwt, aead, argon2,
+                          base64, hashing (fnv/crc32/xxh3), random_token.
+  json/                   JSON & schema: json_ops, json_ser, json_patch_ops, json_schema (napi),
+                          fast_schema/ (zero-DOM engine — pure, no napi).
+  payload/                Output & streaming: compress, sse, ws_frames, websocket, template.
+  ingress/                THE ingress pipeline: mod.rs (napi boundary), pipeline.rs (core 8-stage),
+                          options/time/packed, cors, proxy, ip_trust, rate_limit, terminal,
+                          output.rs (single numeric layout source), ingress_constants.rs (napi projection).
+  ffi.rs                  #[no_mangle] extern "C" exports (79 castrum_* symbols — 71 direct + 4
+                          validator_c_abi! + 4 compress_to_out!; parity guarded by
+                          test/unit/features/ffi-symbol-parity.test.ts) for Bun's bun:ffi
+                          primary transport, incl. castrum_ingress_layout (the layout blob).
+  unit_tests.rs / test_support.rs   Cross-module Rust test suites.
+```
+
+### Generated / ignored (do NOT edit)
+
+| Path | What it is | Why ignored |
+|------|-----------|-------------|
+| `dist/` | Compiled ESM entry + types for Node | Built by `build:js`; gitignored |
+| `index.js` / `index.d.ts` | NAPI-RS generated loader/declarations | Regenerated by `napi build`; gitignored |
+| `castrum.<platform>-<arch>.node` | The native addon binary | Built locally; gitignored |
+| `bench/results/` | Bench reports (cpu/, http/) | Generated; gitignored |
+| `target/` | Cargo build output | gitignored |
+| `node_modules/` | Deps | gitignored |
+| `artifacts/` | CI publish staging dir | gitignored |
+
+---
+
+## 4. How the pieces connect (the 5 cross-cutting flows)
+
+### 4.1 Constants never drift (single numeric source)
+`rust/ingress/output.rs` defines every ingress layout constant once → projected
+to JS two ways: `rust/ingress/ingress_constants.rs` via `#[napi] const` (NAPI)
+AND `rust/ffi.rs` `IngressLayout` via `castrum_ingress_layout` (C ABI, primary
+on Bun) → `src/ingress/constants.ts` reads them at runtime (ffi blob on Bun,
+napi on Node). Drift is pinned by the Rust unit test
+`ingress_layout_c_abi_matches_output_source` and the bun:ffi bind-time
+self-test. **Never hardcode a layout value** — if you change the layout, change
+`output.rs` and nothing else.
+
+### 4.2 The TWO ingress paths (do not conflate)
+1. **`src/ingress/fast.ts`** — `createIngressFast(options)`. JS packs request
+   headers into a binary buffer (`IngressInputPacker`), Rust parses via
+   `handleRequestPacked`. Responses: `{"error":{code,status,message,requestId}}`
+   with `x-ratelimit-*` headers.
+2. **`src/ingress/handlers.ts`** — `createIngressHandler(options, runtime)`.
+   The request frame is packed in JS (`IngressInputPacker` +
+   `gatherRawHeadersPacked`) and driven through the SAME native core as path 1
+   (`handleRequestPacked`), writing into a **pooled** output buffer. Responses:
+   `{"ok":true,...,"requestId":...}` (success) / `{"ok":false,"error":{code,
+   message}}` (errors) with `ratelimit-*` headers.
+
+These wire formats are a **benchmark contract** (`bench/load.ts` validates
+them) — they are intentionally NOT unified. The two decoders
+(`decode/fast-result.ts` vs `decode/baked-result.ts`) and the two header-template
+builders (`headers/fast-templates.ts` vs `headers/baked-templates.ts`) exist for
+this reason. **Shared between both paths**: `generateRequestId`
+(`src/shared/request-id.ts`) and `buildHeaderPlan` / `assertSyncCallback`
+(`src/ingress/shared.ts`).
+
+### 4.3 Addon loading + runtime seam
+- `src/native/loader.ts` resolves the `.node` from multiple roots so it works
+  from both the source layout (`src/…`) and the bundled layout (`dist/…`).
+  Honors `CASTRUM_NATIVE_LIBRARY_PATH` / `NAPI_RS_NATIVE_LIBRARY_PATH`.
+- Runtime detection lives **only** in `src/runtime/detect.ts` (resolved once at
+  load; `src/shared/runtime.ts` is a thin facade). The public
+  `createIngressServer` (via the adapter's `server`) picks `Bun.serve` on Bun
+  and the `node:http` backend on Node; `createIngressServerNode` stays for
+  pinned Node users (same route handlers via `buildRouteHandlers`).
+- ⚠️ `src/ingress/constants.ts` is the **one module that dlopens the addon at
+  import time** (its constants are needed as soon as ingress is imported).
+  Everything else is lazy.
+
+### 4.4 The runtime adapter
+`src/runtime/` is the single seam for every Bun-vs-Node and transport decision
+(detection, codec, uuid, env, the BUN_WINS built-in delegation, the bun:ffi-vs-
+napi transport, the ingress server backend, websocket) — selected ONCE at
+module load. The FFI layer imports the seam (`src/runtime/native.ts`); the
+public API imports the full `runtime` (`src/runtime/index.ts`). No
+`isBun()` / `getBunFFI()` branches in builders.
+
+### 4.5 Pooled output buffers (path 2)
+`createIngressHandler` owns a `BufferPool` (`src/shared/buffer-pool.ts`) sized
+by `runtime.outputBufferSize`. `run()` packs the frame in JS (`IngressInputPacker`
++ `gatherRawHeadersPacked`) and writes via `handleRequestPacked`
+into the pooled buffer, passing the exact written subarray to
+`BakedIngressResult.refresh`. In copy
+mode the buffer is released at the end of `run()`; in zero-copy mode
+`ingress.zeroCopyResponse()` serves a `pooledBodyResponse`
+(`src/shared/response.ts`) that keeps the buffer in flight until the body is
+consumed.
+
+---
+
+## 5. Testing & benchmarks
+
+- **TS**: `bun test` → `test/unit/**` (`ingress/`, `shared/`, `features/`).
+- **Rust**: `cargo test` → per-module `#[cfg(test)] mod tests` + cross-module
+  `rust/unit_tests.rs`.
+- **Node**: `test/integration/node-smoke.test.mjs` + `node-enterprise.test.mjs`
+  via `node --test` (run with `bun run test:node`).
+- **CPU bench**: `bun run check` (correctness checks + comparisons). Sub-µs ops
+  are batched (`CASTRUM_BENCH_BATCH_SIZE`). Report → `bench/results/cpu/`.
+- **HTTP bench**: `bench/run-bench.ts` + `bench/load.ts` (load generator that
+  validates response SHAPE) + `bench/servers/{bun,elysia,ingress}-server.ts`.
+  `bench:http:smoke` is the fast CI wire-format gate.
+
+---
+
+## 6. Publishing (npm)
+
+- Package name `castrum` (unscoped, public). Native binary name `castrum`
+  (declared in package.json `napi.binaryName` + Cargo.toml — keep in sync).
+- **Multi-platform (recommended)**: push a `v*` tag → `.github/workflows/ci.yml`
+  builds each platform addon, the `publish` job downloads them into
+  `./artifacts`, stages them, and runs `npm publish` (needs `NPM_TOKEN`).
+- **Manual single-platform**: `bun run publish:manual --increment minor`
+  (syncs package.json ↔ Cargo.toml ↔ CHANGELOG, tags, builds, publishes with
+  `CASTRUM_PUBLISH_ALLOW_PARTIAL=1`). `--dry-run` plans only.
+- `prepublishOnly` runs `build:js` + `scripts/prepublish.mjs` (stages
+  `artifacts/*.node` and verifies every `napi.targets` maps to a `.node`).
+- Run `bun run check:version` before tagging.
+
+---
+
+## 7. Cross-links
+
+| Doc | Purpose |
+|-----|---------|
+| [`GETTING_STARTED.md`](./GETTING_STARTED.md) | Beginner tutorial (intern-friendly) |
+| [`API.md`](./API.md) | The complete public API reference |
+| [`CASE_STUDY.md`](./CASE_STUDY.md) | Data-driven case study: how castrum was built and measured |
+| [`ARCHITECTURE.md`](./ARCHITECTURE.md) | Deep-dive: modules, data flow, output layout, memory, concurrency |
+| [`INGRESS.md`](./INGRESS.md) | The pre-baked ingress API + route factories + servers |
+| [`BENCHMARKS.md`](./BENCHMARKS.md) | Benchmark scenarios, report format, how to run |
+| [`ENVIRONMENT.md`](./ENVIRONMENT.md) | All `CASTRUM_*` env vars (+ legacy aliases) |
+| [`AGENTS.md`](../AGENTS.md) | AI-agent editing guidance (commands, constraints, gotchas) |
+| `README.md` | Public-facing overview + quick start |

package/docs/adr/0001-two-wire-formats.md

@@ -0,0 +1,39 @@
+# ADR-0001: The two ingress wire formats are a contract, not a bug
+
+- **Status**: Accepted (2026-08-06)
+- **Deciders**: maintainers (via benchmark + hardening passes)
+
+## Context
+
+The ingress layer ships two paths that consume the SAME native core
+(`handleRequestPacked`):
+
+1. `src/ingress/fast.ts` — `createIngressFast`; JS packs headers via
+   `IngressInputPacker`; errors are `{"error":{code,status,message,requestId}}`
+   with `x-ratelimit-*` headers.
+2. `src/ingress/handlers.ts` — `createIngressHandler`; JS packs the full frame;
+   success is Rust-generated `{"ok":true,...,"requestId":...}`, errors are
+   `{"ok":false,"error":{"code","message"}}`, with `ratelimit-*` headers.
+
+They also use separate result decoders (`FastIngressResult` vs
+`BakedIngressResult`) and separate header-template builders.
+
+## Decision
+
+**Do not unify them.** The HTTP benchmark load generator (`bench/load.ts`)
+requires `ok === true` + `requestId: string` on success and `error.code` /
+`error.message` on errors — i.e. path 2's format. The two formats are a
+**benchmark contract**: the benchmark server uses path 2, and changing its
+wire shape invalidates baselines.
+
+## Consequences
+
+- Both paths stay hot and zero-alloc.
+- Duplicated decoders/templates are intentional; sharing them is
+  **forbidden** by AGENTS.md (it would couple two differently-shaped outputs).
+- New contributors must not "helpfully" unify the formats.
+
+## See also
+
+- AGENTS.md §"Ingress: the two paths (do NOT conflate)"
+- docs/INGRESS.md §"Wire format"

package/docs/adr/0002-zero-dom-fast-schema.md

@@ -0,0 +1,38 @@
+# ADR-0002: Zero-DOM fast path for JSON Schema validation
+
+- **Status**: Accepted (2026-08-07), extended 2026-08-11
+- **Deciders**: maintainers (measured via `bun run check`)
+
+## Context
+
+`jsonschema::Validator::is_valid` requires a `serde_json::Value` — a full DOM
+per document, measured at ~95% of validation cost. That made the Rust schema
+path parity with `ajv` at best, so castrum couldn't match its own zero-DOM
+wins (json_valid / json_sum / packed parsers).
+
+## Decision
+
+Implement a **zero-DOM draft-07 fast path** (`rust/json/fast_schema/`) that
+compiles the common keyword subset into a `FastNode` AST and validates raw
+bytes with a zero-alloc cursor. Unsupported keywords → `compile()` returns
+`Err` → the caller falls back to the authoritative `jsonschema` crate (DOM
+path). Parity is **byte-for-byte gated** against the crate
+(`rust/json/fast_schema/tests.rs`).
+
+Draft-07 is pinned because `jsonschema` 0.48's default draft is 2020-12 whose
+`items` / `$ref`-sibling / boolean-exclusive semantics differ; both
+`SchemaValidator` and `IngressSchema` build the reference with
+`.with_draft(Draft7)` + `should_validate_formats(true)`.
+
+## Consequences
+
+- Fast path: scalar ~1.2–2.4x, batch ~1.0–1.56x vs `ajv` (release build).
+- A `$schema` declaring non-draft-07 (or unsupported keywords) still falls
+  back — versatility preserved.
+- The fast path MUST stay byte-parity with the crate for the supported
+  keyword surface; any divergence is a bug.
+
+## See also
+
+- rust/json/fast_schema/tests.rs (parity corpus)
+- docs/ARCHITECTURE.md §JSON & schema

package/docs/adr/0003-bun-builtins-delegation.md

@@ -0,0 +1,43 @@
+# ADR-0003: Don't reinvent the wheel — benchmark, then delegate to Bun built-ins
+
+- **Status**: Accepted (2026-08-11)
+- **Deciders**: maintainers (measured on Bun 1.4.0 / release addon)
+
+## Context
+
+castrum ships Rust ops that overlap with Bun's native built-ins
+(`Bun.hash`, `Bun.password`, `Bun.CryptoHasher`, `Bun.gzipSync`,
+`Bun.randomUUIDv7`, `crypto.subtle`, `JSON.parse`, `URLSearchParams`). For the
+same workload, the in-process Bun implementation can beat an FFI-crossing Rust
+op — spending effort building or keeping such ops is reinventing the wheel.
+
+## Decision
+
+Every overlapping op is **benchmarked against its Bun built-in** before it is
+built, kept, or delegated. Results live in
+`docs/bun-builtins-decision-matrix.md`. Rules:
+
+1. **Delegate to Bun** at the selection/delegation layer where Bun wins
+   clearly: non-crypto hashing (`Bun.hash.crc32` ~3x, `xxHash3` ~4x),
+   gzip (`Bun.gzipSync`/`gunzipSync`), `Bun.randomUUIDv7` (~2x), HMAC
+   (`Bun.CryptoHasher` ~1.2x).
+2. **Keep rust** where it wins or Bun has no synchronous equivalent:
+   argon2id password hashing (~1.8–2x vs `Bun.password`), bcrypt verify
+   (~1.5x), brotli, PBKDF2, zero-DOM JSON/parsers.
+3. **Don't build** what Bun already wins at (UUIDv7 → `uuidv7()` delegates to
+   `Bun.randomUUIDv7`, falls back to `crypto.randomUUID`).
+
+## Consequences
+
+- New `rust.*` primitives ship with a Bun-built-in baseline + decision-matrix
+  entry (`xxh3`, `passwordHashBcrypt`/`verifyBcrypt`, `pbkdf2Sha256`).
+- The Bun/JS baselines are recorded honestly in the decision matrix
+  (e.g. `xxh3` is measured `Bun.hash.xxHash3` ~4x faster — the public
+  `rust.xxh3` delegates to it under Bun via the runtime adapter's `builtins`).
+- Diagnostic benchmark tasks use `diag:` names so they are never mistaken for
+  shipped-op `native:`/`rust:` comparisons.
+
+## See also
+
+- docs/bun-builtins-decision-matrix.md
+- src/bench/tasks/bun-builtins.ts

package/docs/bun-builtins-decision-matrix.md

@@ -0,0 +1,115 @@
+# Bun Built-ins vs castrum — Decision Matrix
+
+> **Question this answers**: for the same workload, does a Bun native built-in
+> beat the castrum Rust op? Where it does, castrum should stop reinventing the
+> wheel and delegate to Bun; where castrum wins (or Bun
+> has no sync equivalent), the Rust op stands.
+
+## Methodology
+
+- **Runtime**: Bun **1.4.0** (the primary target).
+- **Addon**: castrum **release** build (`bun run build`, baseline CPU — the
+  shipped profile).
+- **Driver**: `bun run check` (CPU benchmark → `bench/results/cpu/latest.json`).
+  Each pair runs on the SAME input (fixture payloads, see table).
+- **Naming**: diagnostic comparisons use `diag:` task names so they are kept out
+  of the shipped-op `native:`/`rust:` comparisons (see `src/bench/comparisons.ts`).
+- **Honesty guard**: every Bun API is verified to actually execute real work at
+  runtime before a result is trusted. This is important — the first run
+  produced false "wins" (brotli 156×, validators 3–5×) because this Bun
+  version has **no** `Bun.brotliCompressSync` and **no** `Bun.validators`, and
+  the feature-detect fell through to an empty return. Those pairs were removed,
+  not reported.
+- Ratio is **nativeAvgMs / rustAvgMs** (>1 = Bun faster; <1 = rust faster).
+
+## Results (Bun 1.4.0, release addon, 2026-08-11)
+
+| castrum op | Bun built-in | Input | Winner | Ratio |
+|---|---|---|---|---|
+| `rust.fnv1a64` | `Bun.hash` (wyhash) | 42 B | Bun (parity) | 1.06× |
+| `rust.crc32` | `Bun.hash.crc32` | 42 B | **Bun** | 2.8–8.4× |
+| `rust.xxh3` (new) | `Bun.hash.xxHash3` | 42 B | **Bun** | 4.15× |
+| `rust.hmacSha256` | `Bun.CryptoHasher("sha256", key)` | 32 B | Bun (mild) | 1.1–1.4× |
+| `rust.passwordHash` (argon2id) | `Bun.password.hashSync` (argon2id) | 12 B | **rust** | 0.55× (rust 1.83×) |
+| `rust.passwordVerify` | `Bun.password.verifySync` | 12 B | **rust** | 0.53× (rust 1.88×) |
+| `rust.passwordHashBcrypt` (new) | `Bun.password.hashSync` (bcrypt) | 12 B | Bun (mild) | 1.24× |
+| `rust.passwordVerifyBcrypt` (new) | `Bun.password.verifySync` (bcrypt) | 12 B | **rust** | 0.67× (rust 1.49×) |
+| `rust.pbkdf2Sha256` (new) | `node:crypto.pbkdf2Sync` | 12 B | rust (parity) | 0.93× (rust 1.08×) |
+| `rust.randomToken` (16 B) | `Bun.randomUUIDv7` | — | **Bun** | 1.62× |
+| `rust.gzipCompress` | `Bun.gzipSync` | 11 179 B | **Bun** | 2.02× |
+| `rust.gzipDecompress` | `Bun.gunzipSync` | 1 045 B | **Bun** | 1.38× |
+| `rust.brotliCompress` | *no sync Bun API* | — | N/A | — |
+| `rust.brotliDecompress` | *no sync Bun API* | — | N/A | — |
+| `rust.validateEmail/Uuid/Ipv4/Ipv6` | `Bun.validators` (*absent in 1.4*) | — | N/A | — |
+
+> Same-cost note: `Bun.password.hashSync` ran at `memoryCost 4096 / timeCost 1`
+> (4 MiB, 1 iter) matching the rust bench cost (`mCost 4096 / tCost 1`) — the
+> rust argon2id win is cost-for-cost, not a cost cheat.
+
+> Transport note: "FFI-crossing Rust op" below means any Rust call over the
+> native bridge. Under Bun that bridge is now `bun:ffi` (the PRIMARY transport);
+> under Node it is the napi fallback. This Bun-builtin-vs-Rust axis is
+> independent of the ffi-vs-napi transport axis — these decisions recommend
+> WHICH implementation to call (`rust.*` vs a Bun built-in), not which transport
+> carries the call.
+
+## Decisions
+
+### → Delegate to Bun at the selection layer (Bun wins)
+These rust ops lose to Bun's native implementation on the SAME workload — the
+FFI crossing cannot beat Bun's in-process C++:
+
+This table mirrors `BUILTIN_OPS` in `src/runtime/builtins.ts` (the 11 ops that
+delegate to Bun built-ins under Bun; `src/selection.ts` derives from it). `fnv1a64` is parity/either but is NOT delegated
+(`Bun.hash` is wyhash — a different algorithm — and rust `fnv1a64` stays the
+benchmarked impl).
+
+| Op | Recommendation |
+|---|---|
+| `crc32` | Prefer `Bun.hash.crc32`; keep rust as fallback (pure-TS path already exists). |
+| `xxh3` | Prefer `Bun.hash.xxHash3` (~4×); keep the rust export as the Node/non-Bun fast path. The runtime adapter delegates the public `rust.xxh3` to Bun under Bun. |
+| `gzipCompress` | Prefer `Bun.gzipSync` (~2×). **`gzipDecompress` is deliberately NOT delegated** — `Bun.gunzipSync` has no decompression-bomb cap; the rust surface keeps its native 64 MiB-capped path under Bun. |
+| `randomToken` | Prefer `Bun.randomUUIDv7()` (or `crypto.getRandomValues`) for token-sized output; keep rust for byte-precise control. |
+| `hmacSha256` | Prefer `Bun.CryptoHasher("sha256", key)` (1.17× — mild; keep rust batch path, which wins on larger inputs). |
+| `urlEncode` / `urlDecode` | Prefer `encodeURIComponent` / `decodeURIComponent` (JSC string builtins — ~11.5×, zero alloc). |
+| `base64Encode` / `base64UrlEncode` | Prefer `Buffer.toString('base64'|'base64url')`. |
+| `hexEncode` | Prefer `Buffer.toString('hex')`. |
+| `httpDate` | Prefer `Date.toUTCString()`. |
+
+**Delegation rule of thumb**: rust stays the default only where it beats its
+**Bun** baseline (not just its JS baseline). These are the ops where rust
+genuinely wins or Bun has no equivalent.
+
+### → Keep rust (rust wins or Bun has no equivalent)
+| Op | Rationale |
+|---|---|
+| `passwordHash` / `passwordVerify` (argon2id) | rust 1.83–1.88× faster than `Bun.password` at equal cost. **Do not delegate.** |
+| `passwordHashBcrypt` / `passwordVerifyBcrypt` (new) | Hash ≈ parity (Bun 1.24×), verify rust 1.49×. Keep rust — self-contained and works under Node. |
+| `pbkdf2Sha256` (new) | Bun has no sync PBKDF2; rust ≈ parity with node:crypto (1.08×). Keep rust. |
+| `brotliCompress` / `brotliDecompress` | Bun 1.4 has no synchronous brotli; keep rust (async `CompressionStream` is not drop-in). |
+| `validateEmail` / `validateUuid` / `validateIpv4` / `validateIpv6` | `Bun.validators` absent in 1.4; keep rust. Re-check when Bun adds it. |
+| `jsonValid` / `jsonSumIds` / packed parsers / ingress | No Bun sync equivalent for the zero-DOM / zero-copy semantics; keep rust. |
+
+### → Implemented primitives (gaps closed)
+| Primitive | Outcome |
+|---|---|
+| `rust.xxh3` (XXH3-64) | Exposed; measured `Bun.hash.xxHash3` 4.15× faster → the runtime adapter delegates the public `rust.xxh3` to Bun under Bun (Node keeps the addon). |
+| `rust.passwordHashBcrypt` / `rust.passwordVerifyBcrypt` | bcrypt `$2b$` PHC; hash parity, verify rust 1.49×. |
+| `rust.pbkdf2Sha256` | PBKDF2-HMAC-SHA256; parity with `node:crypto`, the only sync option in Bun. |
+| UUIDv7 (`uuidv7()`) | `Bun.randomUUIDv7` already wins — delegated to Bun (`crypto.randomUUID` on Node), not built in Rust. |
+
+## Summary
+
+**The "don't reinvent the wheel" takeaway:** for non-cryptographic hashing,
+random tokens, and gzip, Bun 1.4's built-ins are already faster than any
+FFI-crossing Rust op — the selection layer should prefer them. Rust retains a clear,
+defensible moat exactly where it matters: **argon2id password hashing
+(≈1.9× vs Bun)**, the **zero-DOM / zero-copy parsers and JSON paths** (no Bun
+sync equivalent), and **brotli** (no Bun sync API).
+
+## Re-run
+
+```sh
+bun run build   # release addon
+bun run check   # writes latest.json; grep the "vs Bun" lines
+```

package/index.ts

@@ -1,8 +1,36 @@
-export * from "./src/rust-ffi";
+// index.ts — castrum package entry (barrel).
+//
+// Re-exports the full public API: the flat `rust.*` FFI surface
+// (src/rust-ffi), the runtime selection hints (src/selection), and the
+// higher-order loader (src/loader). See AGENTS.md for the layout map.
+//
+// EAGER-DLOPEN CONTRACT: this barrel unconditionally re-exports `src/ingress`
+// → `src/ingress/constants.ts`, which is the ONE module that touches native at
+// import time (it reads the Rust binary layout and runs the bind-time
+// self-test). This is intentional — even `import { rust }` pays the eager
+// load. Do NOT add more eager-native modules; keep everything else lazy (see
+// test/unit/features/import-contract.test.ts).
 
-export * as native from "./src/baseline";
+export * from './src/rust-ffi'
 
-export { encoder, decoder } from "./src/shared/bytes";
+// Native-vs-JS selection (OWNED HERE, not by consumers): `opImpl(op)` is the
+// benchmark-driven recommendation for which operations should use the Rust
+// addon vs a pure-JS implementation. Consumers read it once at load time and
+// bind each op to a fixed impl. Source: rust/selection.rs (audited by
+// `bun run check:selection`). See src/selection.ts.
+export { opImpl, isNativeOp, opDecision } from './src/selection'
+export type { OpImpl, OpDecision } from './src/selection'
+
+export { encoder, decoder } from './src/shared/bytes'
+// Bounded EWMA adaptive-estimate utility (drives BufferPool adaptive sizing
+// and reusable runtime self-optimization decisions).
+export { AdaptiveEstimate } from './src/shared/adaptive'
+export type { AdaptiveEstimateOptions } from './src/shared/adaptive'
+// UUIDv7 generation — delegates to Bun.randomUUIDv7, crypto.randomUUID on Node.
+export { uuidv7 } from './src/shared/uuid'
+// Zero-dep metrics registry (counters/gauges/histograms + Prometheus render).
+export { createMetrics, DEFAULT_BUCKETS } from './src/shared/metrics'
+export type { MetricsRegistry, Counter, Gauge, Histogram } from './src/shared/metrics'
 
 export {
   packBatch,
@@ -12,13 +40,21 @@ export {
   pairsToObject,
   parseQueryString,
   parseCookieHeader,
+  parseFormBody,
   unpackBitset,
   unpackByteResults,
   unpackI64ArrayAsBigInt,
   unpackU32Array,
-} from "./src/shared/packed";
-export type { Pair, ParsedHttpRequestPacked } from "./src/shared/packed";
+} from './src/shared/packed'
+export type { Pair, ParsedHttpRequestPacked } from './src/shared/packed'
+
+export * from './src/ingress'
+
+// Framework-agnostic integration helpers: run the ingress pipeline as a
+// request stage (createPipeline), RFC 6455 upgrade (createWebSocketUpgrade),
+// and SSE framing (sseResponse). See docs/INGRESS.md.
+export * from './src/integration'
 
-export { jsonRowsBytes, createJsonRows } from "./src/data/json-rows";
-export type { JsonRow } from "./src/data/json-rows";
-export * from "./src/ingress";
+// Higher-order loader (HFC) over the curated op set: createLoader / loader
+// dispatch single items, packed batches, and load() coalescing. See README.
+export * from './src/loader'