castrum 0.7.0 → 0.9.0

package/docs/REPO_MAP.md

@@ -0,0 +1,259 @@
+# castrum — Repo Map (what is where, and why it works this way)
+
+This is the **navigation doc**. It tells you where everything lives, how the
+pieces connect, and how to run/build/test/bench/publish. If you are brand new,
+read [`GETTING_STARTED.md`](./GETTING_STARTED.md) first; if you want deep-dive
+architecture, read [`ARCHITECTURE.md`](./ARCHITECTURE.md). If you are an AI
+agent editing this repo, [`AGENTS.md`](../AGENTS.md) is the agent-facing map.
+
+---
+
+## 1. What this project is
+
+`castrum` is a hybrid **Bun (TypeScript) + Rust** package:
+
+- A **Rust cdylib** (`castrum.<platform>-<arch>.node`, built with napi-rs,
+  ALSO exporting `extern "C"` symbols for Bun's `bun:ffi` — see
+  `rust/ffi.rs`) provides the performance-critical primitives (parsers,
+  crypto, hashing, JSON, compression, and the HTTP **ingress pipeline**).
+  Under Bun, `bun:ffi` is the PRIMARY transport; NAPI is the fallback
+  (Node, `CASTRUM_FFI_MODE=napi`, or a failed ffi self-test).
+- **TypeScript** (`src/`) provides the ergonomic public API: the flat `rust.*`
+  FFI client, the `proven` performance surface, and the ingress layer.
+- It ships benchmarks: a **CPU benchmark** (`bench.ts` → `src/bench`) and an
+  **HTTP benchmark** (`bench/run-bench.ts` + `bench/servers/*`).
+
+Bun is the primary runtime; Node.js ≥20.3 is supported via a compiled ESM entry
+(`dist/`).
+
+---
+
+## 2. Command cheat-sheet
+
+| Task | Command | Notes |
+|------|---------|-------|
+| Install deps | `bun install` | |
+| Build Rust addon (release) | `bun run build` | `napi build --release --platform` |
+| Build Rust addon (LOCAL max perf) | `bun run build:perf` | x86-64-v3 + AVX2 — **never for publish** |
+| Build Rust addon (debug) | `bun run build:debug` | |
+| Build compiled JS entry (Node) | `bun run build:js` | bundle + types → `dist/` (gitignored) |
+| TS unit tests | `bun test` | ~540 tests, `test/unit/**` |
+| Rust unit tests | `bun run test:rust` | `cargo test`, ~440 tests |
+| Node smoke tests | `bun run test:node` | explicit `.mjs` paths (Node 24 rejects dir arg) |
+| Installed-tarball e2e | `bun run verify:install` | pack → install → import from `node_modules` |
+| Typecheck | `bun run typecheck` | `bunx tsc --noEmit` (only `index.ts`, `bench.ts`, `src/`) |
+| CPU benchmark + correctness | `bun run check` | == `bun bench.ts`; writes `bench/results/cpu/` |
+| Proven-surface audit | `bun run check:proven[:fail]` | `:fail` gates CI on regressions |
+| Performance JSDoc | `bun run check:annotate` | rewrites perf JSDoc from CPU report; `--dry-run` previews |
+| HTTP bench (all servers) | `bun run bench:http` | |
+| HTTP smoke (fast sanity) | `bun run bench:http:smoke` | **the wire-format guard** (CI-gated) |
+| Startup bench | `bun run bench:startup` | import + first-call timing |
+| JS lint (Biome) | `bun run lint` / `lint:fix` | `bunx biome check [--write] .` |
+| JS format (Biome) | `bun run format` | `bunx biome format --write .` |
+| Version consistency | `bun run check:version` | package.json ↔ Cargo.toml ↔ CHANGELOG |
+| JS dependency audit | `bun run audit` | `bun audit` (reports advisories) |
+| Cargo deny audit | `bun run deny` | `cargo deny check` |
+| Publish (manual, single-platform) | `bun run publish:manual[:dry]` | see §6 |
+
+---
+
+## 3. Directory map
+
+### Source (what you edit)
+
+```
+index.ts                  Package entry (Bun). Re-exports src/rust-ffi, proven, shared helpers, src/ingress.
+bench.ts                  CPU benchmark entry point → src/bench/run.ts.
+
+examples/                 Runnable sample app (basic-server.ts + README) — see also README Quick Start.
+  basic-server.ts         Minimal pre-baked ingress server + rust.* primitives.
+
+src/
+  native/                 The native transport layer: bun:ffi (primary on Bun) + NAPI fallback.
+    types.ts              NativeAddon interface + every instance type (mirrors index.d.ts).
+    loader.ts             Addon path resolution (multi-root) + getAddon()/lazyAddon().
+    ffi.ts                bun:ffi C-ABI bindings (Bun-only): lazy bind + bind-time self-test,
+                          CASTRUM_FFI_MODE gating, castrum_ingress_layout blob for constants.ts.
+    index.ts              Barrel.
+  rust-ffi/               The flat `rust.*` FFI client (TS wrappers over the addon).
+    client.ts             createRust() factory + the default `rust` instance + configure().
+    scalar/ / text.ts / batch.ts / packed.ts   namespaces (buildScalar, buildText, ...).
+    context.ts            Per-instance state (caches, rayon-pool bookkeeping).
+    options.ts            RustOptions + rayon-thread resolution + coercion helpers.
+    addon.ts              The single shared lazy addon proxy.
+    proven.ts             `proven` client — the FULL rust.* surface, annotated (same object as `rust`).
+    index.ts              Barrel (exports rust, createRust, types).
+  shared/                 Cross-cutting helpers.
+    runtime.ts            THE ONLY place `typeof Bun` is checked (isBun/isNode).
+    bytes.ts              Shared TextEncoder/Decoder singletons + toPlainBuffer.
+    packed.ts             Packed-wire unpackers (unpackU32Array, packBatch, readPairsPacked, ...).
+    request-id.ts         Zero-alloc request-id generator (both ingress paths).
+    buffer-pool.ts        Generic reusable byte-buffer pool (pooled ingress output).
+    response.ts           pooledBodyResponse (releases the pool on body consume).
+    log.ts                createStructuredLogger (CASTRUM_LOG_LEVEL-gated JSON lines).
+    proven.ts             PROVEN_SURFACE registry (PURE DATA — no addon imports).
+  ingress/                The HTTP ingress pipeline (two paths — see §4.2).
+    fast.ts               PATH 1: createIngressFast (packed-input via handleRequestPacked).
+    handlers.ts           PATH 2: createIngressHandler (JS-packs the frame via IngressInputPacker
+                          + gatherRawHeadersPacked → handleRequestPacked).
+    index.ts              Barrel + createIngress / createIngressSync convenience factories.
+    shared.ts             JS constants + buildHeaderPlan + assertSyncCallback (shared by both paths).
+    constants.ts          Binary-layout constants read from Rust at runtime (SINGLE SOURCE).
+    status.ts / errors.ts / options.ts / types.ts / body.ts / context.ts
+    packing/              header-packing.ts, input-packer.ts, gather-raw-headers.ts
+    headers/              cors.ts, hsts.ts, fast-templates.ts, baked-templates.ts
+    decode/               fast-result.ts, baked-result.ts   (TWO decoders — see §4.2)
+    response/             terminal.ts, error-bodies.ts
+    routes/               read/head/json-write/echo/fallback factories + common.ts
+    server.ts             createIngressServer (Bun.serve) + buildRouteHandlers + gracefulShutdown.
+    server-node.ts        createIngressServerNode (node:http adapter, same route handlers).
+  integration/            Framework integration (wraps path 2 — createIngressHandler).
+    pipeline.ts           createPipeline (framework-agnostic request stage for Hono/Elysia/Bun.serve).
+    websocket.ts          createWebSocketUpgrade (RFC 6455 101 handshake + subprotocol).
+    streaming.ts          sseResponse (SSE frames via rust.sseEncodeEvent).
+  baseline/               Pure-JS baseline implementations (benchmark reference). Internal.
+  bench/                  CPU benchmark framework (tasks, measure, report, checks, comparisons).
+  data/                   json-rows.ts fixture generator (benchmark-only).
+```
+
+```
+rust/                     ONE cdylib crate (Cargo [lib] → lib.rs).
+  lib.rs                  Declaration hub + module map + crate docs.
+  util/                   Shared infrastructure: bytes, packed, batch (+batch_core), threadpool, validation.
+  http/                   HTTP wire formats & parsing: headers, method, http/cookie/query parsers,
+                          form, media_type, url_codec, url_join, etag, accept, mime_lookup, multipart.
+  crypto/                 Auth & hashing: hmac_sha256, cookie_sign, csrf, jwt, aead, argon2,
+                          base64, hashing (fnv/crc32/xxh3), random_token.
+  json/                   JSON & schema: json_ops, json_ser, json_patch_ops, json_schema (napi),
+                          fast_schema/ (zero-DOM engine — pure, no napi).
+  payload/                Output & streaming: compress, sse, ws_frames, websocket, template.
+  ingress/                THE ingress pipeline: mod.rs (napi boundary), pipeline.rs (core 8-stage),
+                          options/time/packed, cors, proxy, ip_trust, rate_limit, terminal,
+                          output.rs (single numeric layout source), ingress_constants.rs (napi projection).
+  ffi.rs                  #[no_mangle] extern "C" exports (45 castrum_* symbols) for Bun's bun:ffi
+                          primary transport, incl. castrum_ingress_layout (the layout blob).
+  unit_tests.rs / test_support.rs   Cross-module Rust test suites.
+```
+
+### Generated / ignored (do NOT edit)
+
+| Path | What it is | Why ignored |
+|------|-----------|-------------|
+| `dist/` | Compiled ESM entry + types for Node | Built by `build:js`; gitignored |
+| `index.js` / `index.d.ts` | NAPI-RS generated loader/declarations | Regenerated by `napi build`; gitignored |
+| `castrum.<platform>-<arch>.node` | The native addon binary | Built locally; gitignored |
+| `bench/results/` | Bench reports (cpu/, http/) | Generated; gitignored |
+| `target/` | Cargo build output | gitignored |
+| `node_modules/` | Deps | gitignored |
+| `artifacts/` | CI publish staging dir | gitignored |
+
+---
+
+## 4. How the pieces connect (the 5 cross-cutting flows)
+
+### 4.1 Constants never drift (single numeric source)
+`rust/ingress/output.rs` defines every ingress layout constant once → projected
+to JS two ways: `rust/ingress/ingress_constants.rs` via `#[napi] const` (NAPI)
+AND `rust/ffi.rs` `IngressLayout` via `castrum_ingress_layout` (C ABI, primary
+on Bun) → `src/ingress/constants.ts` reads them at runtime (ffi blob on Bun,
+napi on Node). Drift is pinned by the Rust unit test
+`ingress_layout_c_abi_matches_output_source` and the bun:ffi bind-time
+self-test. **Never hardcode a layout value** — if you change the layout, change
+`output.rs` and nothing else.
+
+### 4.2 The TWO ingress paths (do not conflate)
+1. **`src/ingress/fast.ts`** — `createIngressFast(options)`. JS packs request
+   headers into a binary buffer (`IngressInputPacker`), Rust parses via
+   `handleRequestPacked`. Responses: `{"error":{code,status,message,requestId}}`
+   with `x-ratelimit-*` headers.
+2. **`src/ingress/handlers.ts`** — `createIngressHandler(options, runtime)`.
+   The request frame is packed in JS (`IngressInputPacker` +
+   `gatherRawHeadersPacked`) and driven through the SAME native core as path 1
+   (`handleRequestPacked`), writing into a **pooled** output buffer. Responses:
+   `{"ok":true,...,"requestId":...}` (success) / `{"ok":false,"error":{code,
+   message}}` (errors) with `ratelimit-*` headers.
+
+These wire formats are a **benchmark contract** (`bench/load.ts` validates
+them) — they are intentionally NOT unified. The two decoders
+(`decode/fast-result.ts` vs `decode/baked-result.ts`) and the two header-template
+builders (`headers/fast-templates.ts` vs `headers/baked-templates.ts`) exist for
+this reason. **Shared between both paths**: `generateRequestId`
+(`src/shared/request-id.ts`) and `buildHeaderPlan` / `assertSyncCallback`
+(`src/ingress/shared.ts`).
+
+### 4.3 Addon loading + runtime seam
+- `src/native/loader.ts` resolves the `.node` from multiple roots so it works
+  from both the source layout (`src/…`) and the bundled layout (`dist/…`).
+  Honors `CASTRUM_NATIVE_LIBRARY_PATH` / `NAPI_RS_NATIVE_LIBRARY_PATH`.
+- Runtime detection lives **only** in `src/shared/runtime.ts`
+  (`isBun`/`isNode`). `createIngressServer` is Bun-only; Node users use
+  `createIngressServerNode` (same route handlers via `buildRouteHandlers`).
+- ⚠️ `src/ingress/constants.ts` is the **one module that dlopens the addon at
+  import time** (its constants are needed as soon as ingress is imported).
+  Everything else is lazy.
+
+### 4.4 The `proven` performance surface
+`src/shared/proven.ts` (`PROVEN_SURFACE`) is the single source of truth for the
+per-function performance classifications. It is **pure data** (no addon
+imports) so `scripts/check-proven.ts` can audit it without dlopening.
+`src/rust-ffi/proven.ts` exposes the FULL `rust.*` surface (nothing is
+filtered); each function's JSDoc carries its measured performance vs the JS
+baseline, written by `scripts/annotate-performance.ts` (`bun run check:annotate`)
+and marked `@deprecated` when slower. `bun run check:proven:fail` (CI) audits
+the registry against the CPU benchmark report on a **release** build.
+
+### 4.5 Pooled output buffers (path 2)
+`createIngressHandler` owns a `BufferPool` (`src/shared/buffer-pool.ts`) sized
+by `runtime.outputBufferSize`. `run()` packs the frame in JS (`IngressInputPacker`
++ `gatherRawHeadersPacked`) and writes via `handleRequestPacked`
+into the pooled buffer, passing the exact written subarray to
+`BakedIngressResult.refresh`. In copy
+mode the buffer is released at the end of `run()`; in zero-copy mode
+`ingress.zeroCopyResponse()` serves a `pooledBodyResponse`
+(`src/shared/response.ts`) that keeps the buffer in flight until the body is
+consumed.
+
+---
+
+## 5. Testing & benchmarks
+
+- **TS**: `bun test` → `test/unit/**` (`ingress/`, `shared/`, `features/`).
+- **Rust**: `cargo test` → per-module `#[cfg(test)] mod tests` + cross-module
+  `rust/unit_tests.rs`.
+- **Node**: `test/integration/node-smoke.test.mjs` + `node-enterprise.test.mjs`
+  via `node --test` (run with `bun run test:node`).
+- **CPU bench**: `bun run check` (correctness checks + comparisons). Sub-µs ops
+  are batched (`CASTRUM_BENCH_BATCH_SIZE`). Report → `bench/results/cpu/`.
+- **HTTP bench**: `bench/run-bench.ts` + `bench/load.ts` (load generator that
+  validates response SHAPE) + `bench/servers/{bun,elysia,ingress}-server.ts`.
+  `bench:http:smoke` is the fast CI wire-format gate.
+
+---
+
+## 6. Publishing (npm)
+
+- Package name `castrum` (unscoped, public). Native binary name `castrum`
+  (declared in package.json `napi.binaryName` + Cargo.toml — keep in sync).
+- **Multi-platform (recommended)**: push a `v*` tag → `.github/workflows/ci.yml`
+  builds each platform addon, the `publish` job downloads them into
+  `./artifacts`, stages them, and runs `npm publish` (needs `NPM_TOKEN`).
+- **Manual single-platform**: `bun run publish:manual --increment minor`
+  (syncs package.json ↔ Cargo.toml ↔ CHANGELOG, tags, builds, publishes with
+  `CASTRUM_PUBLISH_ALLOW_PARTIAL=1`). `--dry-run` plans only.
+- `prepublishOnly` runs `build:js` + `scripts/prepublish.mjs` (stages
+  `artifacts/*.node` and verifies every `napi.targets` maps to a `.node`).
+- Run `bun run check:version` before tagging.
+
+---
+
+## 7. Cross-links
+
+| Doc | Purpose |
+|-----|---------|
+| [`GETTING_STARTED.md`](./GETTING_STARTED.md) | Beginner tutorial (intern-friendly) |
+| [`ARCHITECTURE.md`](./ARCHITECTURE.md) | Deep-dive: modules, data flow, output layout, memory, concurrency |
+| [`INGRESS.md`](./INGRESS.md) | The pre-baked ingress API + route factories + servers |
+| [`BENCHMARKS.md`](./BENCHMARKS.md) | Benchmark scenarios, report format, how to run |
+| [`ENVIRONMENT.md`](./ENVIRONMENT.md) | All `CASTRUM_*` env vars (+ legacy aliases) |
+| [`AGENTS.md`](../AGENTS.md) | AI-agent editing guidance (commands, constraints, gotchas) |
+| `README.md` | Public-facing quick start + API reference |

package/docs/adr/0001-two-wire-formats.md

@@ -0,0 +1,39 @@
+# ADR-0001: The two ingress wire formats are a contract, not a bug
+
+- **Status**: Accepted (2026-08-06)
+- **Deciders**: maintainers (via benchmark + hardening passes)
+
+## Context
+
+The ingress layer ships two paths that consume the SAME native core
+(`handleRequestPacked`):
+
+1. `src/ingress/fast.ts` — `createIngressFast`; JS packs headers via
+   `IngressInputPacker`; errors are `{"error":{code,status,message,requestId}}`
+   with `x-ratelimit-*` headers.
+2. `src/ingress/handlers.ts` — `createIngressHandler`; JS packs the full frame;
+   success is Rust-generated `{"ok":true,...,"requestId":...}`, errors are
+   `{"ok":false,"error":{"code","message"}}`, with `ratelimit-*` headers.
+
+They also use separate result decoders (`FastIngressResult` vs
+`BakedIngressResult`) and separate header-template builders.
+
+## Decision
+
+**Do not unify them.** The HTTP benchmark load generator (`bench/load.ts`)
+requires `ok === true` + `requestId: string` on success and `error.code` /
+`error.message` on errors — i.e. path 2's format. The two formats are a
+**benchmark contract**: the benchmark server uses path 2, and changing its
+wire shape invalidates baselines.
+
+## Consequences
+
+- Both paths stay hot and zero-alloc.
+- Duplicated decoders/templates are intentional; sharing them is
+  **forbidden** by AGENTS.md (it would couple two differently-shaped outputs).
+- New contributors must not "helpfully" unify the formats.
+
+## See also
+
+- AGENTS.md §"Ingress: the two paths (do NOT conflate)"
+- docs/INGRESS.md §"Wire format"

package/docs/adr/0002-zero-dom-fast-schema.md

@@ -0,0 +1,38 @@
+# ADR-0002: Zero-DOM fast path for JSON Schema validation
+
+- **Status**: Accepted (2026-08-07), extended 2026-08-11
+- **Deciders**: maintainers (measured via `bun run check`)
+
+## Context
+
+`jsonschema::Validator::is_valid` requires a `serde_json::Value` — a full DOM
+per document, measured at ~95% of validation cost. That made the Rust schema
+path parity with `ajv` at best, so castrum couldn't match its own zero-DOM
+wins (json_valid / json_sum / packed parsers).
+
+## Decision
+
+Implement a **zero-DOM draft-07 fast path** (`rust/json/fast_schema/`) that
+compiles the common keyword subset into a `FastNode` AST and validates raw
+bytes with a zero-alloc cursor. Unsupported keywords → `compile()` returns
+`Err` → the caller falls back to the authoritative `jsonschema` crate (DOM
+path). Parity is **byte-for-byte gated** against the crate
+(`rust/json/fast_schema/tests.rs`).
+
+Draft-07 is pinned because `jsonschema` 0.48's default draft is 2020-12 whose
+`items` / `$ref`-sibling / boolean-exclusive semantics differ; both
+`SchemaValidator` and `IngressSchema` build the reference with
+`.with_draft(Draft7)` + `should_validate_formats(true)`.
+
+## Consequences
+
+- Fast path: scalar ~1.2–2.4x, batch ~1.0–1.56x vs `ajv` (release build).
+- A `$schema` declaring non-draft-07 (or unsupported keywords) still falls
+  back — versatility preserved.
+- The fast path MUST stay byte-parity with the crate for the supported
+  keyword surface; any divergence is a bug.
+
+## See also
+
+- rust/json/fast_schema/tests.rs (parity corpus)
+- docs/ARCHITECTURE.md §JSON & schema

package/docs/adr/0003-bun-builtins-delegation.md

@@ -0,0 +1,42 @@
+# ADR-0003: Don't reinvent the wheel — benchmark, then delegate to Bun built-ins
+
+- **Status**: Accepted (2026-08-11)
+- **Deciders**: maintainers (measured on Bun 1.4.0 / release addon)
+
+## Context
+
+castrum ships Rust ops that overlap with Bun's native built-ins
+(`Bun.hash`, `Bun.password`, `Bun.CryptoHasher`, `Bun.gzipSync`,
+`Bun.randomUUIDv7`, `crypto.subtle`, `JSON.parse`, `URLSearchParams`). For the
+same workload, the in-process Bun implementation can beat an FFI-crossing Rust
+op — spending effort building or keeping such ops is reinventing the wheel.
+
+## Decision
+
+Every overlapping op is **benchmarked against its Bun built-in** before it is
+built, kept, or delegated. Results live in
+`docs/bun-builtins-decision-matrix.md`. Rules:
+
+1. **Delegate to Bun** at the `@flux/native` wrapper layer where Bun wins
+   clearly: non-crypto hashing (`Bun.hash.crc32` ~3x, `xxHash3` ~4x),
+   gzip (`Bun.gzipSync`/`gunzipSync`), `Bun.randomUUIDv7` (~2x), HMAC
+   (`Bun.CryptoHasher` ~1.2x).
+2. **Keep rust** where it wins or Bun has no synchronous equivalent:
+   argon2id password hashing (~1.8–2x vs `Bun.password`), bcrypt verify
+   (~1.5x), brotli, PBKDF2, zero-DOM JSON/parsers.
+3. **Don't build** what Bun already wins at (UUIDv7 → `uuidv7()` delegates to
+   `Bun.randomUUIDv7`, falls back to `crypto.randomUUID`).
+
+## Consequences
+
+- New `rust.*` primitives ship with a Bun-built-in baseline + decision-matrix
+  entry (`xxh3`, `passwordHashBcrypt`/`verifyBcrypt`, `pbkdf2Sha256`).
+- `PROVEN_SURFACE` classifications reflect the Bun/JS baseline honestly
+  (e.g. `xxh3` is `not-competitive` vs `Bun.hash.xxHash3`).
+- Diagnostic benchmark tasks use `diag:` names so they never perturb the
+  `check:proven` audit.
+
+## See also
+
+- docs/bun-builtins-decision-matrix.md
+- src/bench/tasks/bun-builtins.ts

package/docs/bun-builtins-decision-matrix.md

@@ -0,0 +1,108 @@
+# Bun Built-ins vs castrum — Decision Matrix
+
+> **Question this answers**: for the same workload, does a Bun native built-in
+> beat the castrum Rust op? Where it does, castrum should stop reinventing the
+> wheel and `@flux/native` should delegate to Bun; where castrum wins (or Bun
+> has no sync equivalent), the Rust op stands.
+
+## Methodology
+
+- **Runtime**: Bun **1.4.0** (the primary target).
+- **Addon**: castrum **release** build (`bun run build`, baseline CPU — the
+  shipped profile).
+- **Driver**: `bun run check` (CPU benchmark → `bench/results/cpu/latest.json`).
+  Each pair runs on the SAME input (fixture payloads, see table).
+- **Naming**: diagnostic comparisons use `diag:` task names so
+  `scripts/check-proven.ts` never aggregates them into a `PROVEN_SURFACE`
+  entry (see `src/bench/comparisons.ts`).
+- **Honesty guard**: every Bun API is verified to actually execute real work at
+  runtime before a result is trusted. This is important — the first run
+  produced false "wins" (brotli 156×, validators 3–5×) because this Bun
+  version has **no** `Bun.brotliCompressSync` and **no** `Bun.validators`, and
+  the feature-detect fell through to an empty return. Those pairs were removed,
+  not reported.
+- Ratio is **nativeAvgMs / rustAvgMs** (>1 = Bun faster; <1 = rust faster).
+
+## Results (Bun 1.4.0, release addon, 2026-08-11)
+
+| castrum op | Bun built-in | Input | Winner | Ratio |
+|---|---|---|---|---|
+| `rust.fnv1a64` | `Bun.hash` (wyhash) | 42 B | Bun (parity) | 1.06× |
+| `rust.crc32` | `Bun.hash.crc32` | 42 B | **Bun** | 2.8–8.4× |
+| `rust.xxh3` (new) | `Bun.hash.xxHash3` | 42 B | **Bun** | 4.15× |
+| `rust.hmacSha256` | `Bun.CryptoHasher("sha256", key)` | 32 B | Bun (mild) | 1.1–1.4× |
+| `rust.passwordHash` (argon2id) | `Bun.password.hashSync` (argon2id) | 12 B | **rust** | 0.55× (rust 1.83×) |
+| `rust.passwordVerify` | `Bun.password.verifySync` | 12 B | **rust** | 0.53× (rust 1.88×) |
+| `rust.passwordHashBcrypt` (new) | `Bun.password.hashSync` (bcrypt) | 12 B | Bun (mild) | 1.24× |
+| `rust.passwordVerifyBcrypt` (new) | `Bun.password.verifySync` (bcrypt) | 12 B | **rust** | 0.67× (rust 1.49×) |
+| `rust.pbkdf2Sha256` (new) | `node:crypto.pbkdf2Sync` | 12 B | rust (parity) | 0.93× (rust 1.08×) |
+| `rust.randomToken` (16 B) | `Bun.randomUUIDv7` | — | **Bun** | 1.62× |
+| `rust.gzipCompress` | `Bun.gzipSync` | 11 179 B | **Bun** | 2.02× |
+| `rust.gzipDecompress` | `Bun.gunzipSync` | 1 045 B | **Bun** | 1.38× |
+| `rust.brotliCompress` | *no sync Bun API* | — | N/A | — |
+| `rust.brotliDecompress` | *no sync Bun API* | — | N/A | — |
+| `rust.validateEmail/Uuid/Ipv4/Ipv6` | `Bun.validators` (*absent in 1.4*) | — | N/A | — |
+
+> Same-cost note: `Bun.password.hashSync` ran at `memoryCost 4096 / timeCost 1`
+> (4 MiB, 1 iter) matching the rust bench cost (`mCost 4096 / tCost 1`) — the
+> rust argon2id win is cost-for-cost, not a cost cheat.
+
+> Transport note: "FFI-crossing Rust op" below means any Rust call over the
+> native bridge. Under Bun that bridge is now `bun:ffi` (the PRIMARY transport);
+> under Node it is the napi fallback. This Bun-builtin-vs-Rust axis is
+> independent of the ffi-vs-napi transport axis — these decisions recommend
+> WHICH implementation to call (`rust.*` vs a Bun built-in), not which transport
+> carries the call.
+
+## Decisions
+
+### → Delegate to Bun at the `@flux/native` wrapper layer (Bun wins)
+These rust ops lose to Bun's native implementation on the SAME workload — the
+FFI crossing cannot beat Bun's in-process C++:
+
+| Op | Recommendation |
+|---|---|
+| `crc32` | Prefer `Bun.hash.crc32`; keep rust as fallback (pure-TS path already exists). |
+| `xxh3` | Prefer `Bun.hash.xxHash3` (~4×); keep the rust export as the Node/non-Bun fast path. Classified `not-competitive` in `PROVEN_SURFACE`. |
+| `gzipCompress` / `gzipDecompress` | Prefer `Bun.gzipSync` / `Bun.gunzipSync`. |
+| `randomToken` | Prefer `Bun.randomUUIDv7()` (or `crypto.getRandomValues`) for token-sized output; keep rust for byte-precise control. |
+| `hmacSha256` | Prefer `Bun.CryptoHasher("sha256", key)` (1.17× — mild; keep rust batch path, which wins on larger inputs). |
+| `fnv1a64` | Parity — either is fine; `Bun.hash` (wyhash) is a stronger algorithm, prefer it for NEW non-crypto hashing needs. |
+
+**Delegation rule of thumb**: rust stays the default only where it is
+`proven` against its **Bun** baseline (not just its JS baseline). These are the
+ops where rust genuinely wins or Bun has no equivalent.
+
+### → Keep rust (rust wins or Bun has no equivalent)
+| Op | Rationale |
+|---|---|
+| `passwordHash` / `passwordVerify` (argon2id) | rust 1.83–1.88× faster than `Bun.password` at equal cost. **Do not delegate.** |
+| `passwordHashBcrypt` / `passwordVerifyBcrypt` (new) | Hash ≈ parity (Bun 1.24×), verify rust 1.49×. Keep rust — self-contained and works under Node. |
+| `pbkdf2Sha256` (new) | Bun has no sync PBKDF2; rust ≈ parity with node:crypto (1.08×). Keep rust. |
+| `brotliCompress` / `brotliDecompress` | Bun 1.4 has no synchronous brotli; keep rust (async `CompressionStream` is not drop-in). |
+| `validateEmail` / `validateUuid` / `validateIpv4` / `validateIpv6` | `Bun.validators` absent in 1.4; keep rust. Re-check when Bun adds it. |
+| `jsonValid` / `jsonSumIds` / packed parsers / ingress | No Bun sync equivalent for the zero-DOM / zero-copy semantics; keep rust. |
+
+### → Add primitives that close gaps (Phase 2)
+| Gap | Plan |
+|---|---|
+| Public XXH3-64 (`rust.xxh3`) | **Done** — exposed; measured Bun.hash.xxHash3 4.15× faster → classify `not-competitive`, prefer Bun under Bun. |
+| bcrypt | **Done** — `rust.passwordHashBcrypt`/`passwordVerifyBcrypt`; hash parity, verify rust 1.49×. |
+| PBKDF2 | **Done** — `rust.pbkdf2Sha256`; parity with node:crypto, only sync option in Bun. |
+| UUIDv7 | `Bun.randomUUIDv7` already wins — **do not** build a rust UUIDv7; use Bun. |
+
+## Summary
+
+**The "don't reinvent the wheel" takeaway:** for non-cryptographic hashing,
+random tokens, and gzip, Bun 1.4's built-ins are already faster than any
+FFI-crossing Rust op — `@flux/native` should prefer them. Rust retains a clear,
+defensible moat exactly where it matters: **argon2id password hashing
+(≈1.9× vs Bun)**, the **zero-DOM / zero-copy parsers and JSON paths** (no Bun
+sync equivalent), and **brotli** (no Bun sync API).
+
+## Re-run
+
+```sh
+bun run build   # release addon
+bun run check   # writes latest.json; grep the "vs Bun" lines
+```

package/index.ts

@@ -1,8 +1,43 @@
-export * from "./src/rust-ffi";
+// index.ts — castrum package entry (barrel).
+//
+// Re-exports the full public API: the flat `rust.*` FFI surface
+// (src/rust-ffi), the runtime selection hints (src/selection), the
+// performance-proven registry (src/shared/proven), and the higher-order
+// loader (src/loader). See AGENTS.md for the layout map.
 
-export * as native from "./src/baseline";
+export * from './src/rust-ffi'
 
-export { encoder, decoder } from "./src/shared/bytes";
+// Native-vs-JS selection (OWNED HERE, not by consumers): `opImpl(op)` is the
+// benchmark-driven recommendation for which operations should use the Rust
+// addon vs a pure-JS implementation. Consumers read it once at load time and
+// bind each op to a fixed impl. Source: rust/selection.rs (audited by
+// `bun run check:selection`). See src/selection.ts.
+export { opImpl, isNativeOp, opDecision } from './src/selection'
+export type { OpImpl, OpDecision } from './src/selection'
+
+// Performance-proven surface: only `rust.*` functions that beat their JS
+// baseline in benchmarks. See src/shared/proven.ts for the registry.
+export {
+  proven,
+  PROVEN_SURFACE,
+  provenStatus,
+  isProven,
+  provenSurface,
+  provenSummary,
+} from './src/rust-ffi/proven'
+export type { ProvenClient, ProvenKey } from './src/rust-ffi/proven'
+export type { PerformanceStatus, ProvenEntry } from './src/shared/proven'
+
+export { encoder, decoder } from './src/shared/bytes'
+// Bounded EWMA adaptive-estimate utility (drives BufferPool adaptive sizing
+// and reusable runtime self-optimization decisions).
+export { AdaptiveEstimate } from './src/shared/adaptive'
+export type { AdaptiveEstimateOptions } from './src/shared/adaptive'
+// UUIDv7 generation — delegates to Bun.randomUUIDv7, crypto.randomUUID on Node.
+export { uuidv7 } from './src/shared/uuid'
+// Zero-dep metrics registry (counters/gauges/histograms + Prometheus render).
+export { createMetrics, DEFAULT_BUCKETS } from './src/shared/metrics'
+export type { MetricsRegistry, Counter, Gauge, Histogram } from './src/shared/metrics'
 
 export {
   packBatch,
@@ -12,13 +47,21 @@ export {
   pairsToObject,
   parseQueryString,
   parseCookieHeader,
+  parseFormBody,
   unpackBitset,
   unpackByteResults,
   unpackI64ArrayAsBigInt,
   unpackU32Array,
-} from "./src/shared/packed";
-export type { Pair, ParsedHttpRequestPacked } from "./src/shared/packed";
+} from './src/shared/packed'
+export type { Pair, ParsedHttpRequestPacked } from './src/shared/packed'
+
+export * from './src/ingress'
+
+// Framework-agnostic integration helpers: run the ingress pipeline as a
+// request stage (createPipeline), RFC 6455 upgrade (createWebSocketUpgrade),
+// and SSE framing (sseResponse). See docs/INGRESS.md.
+export * from './src/integration'
 
-export { jsonRowsBytes, createJsonRows } from "./src/data/json-rows";
-export type { JsonRow } from "./src/data/json-rows";
-export * from "./src/ingress";
+// Higher-order loader (HFC) over the curated op set: createLoader / loader
+// dispatch single items, packed batches, and load() coalescing. See README.
+export * from './src/loader'