castrum 0.9.4 → 0.9.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. package/Cargo.lock +25 -42
  2. package/Cargo.toml +21 -4
  3. package/README.md +8 -6
  4. package/castrum.darwin-arm64.node +0 -0
  5. package/castrum.darwin-x64.node +0 -0
  6. package/castrum.linux-arm64-gnu.node +0 -0
  7. package/castrum.linux-arm64-musl.node +0 -0
  8. package/castrum.linux-x64-gnu.node +0 -0
  9. package/castrum.linux-x64-musl.node +0 -0
  10. package/castrum.linux-x64-v3-gnu.node +0 -0
  11. package/castrum.win32-x64-msvc.node +0 -0
  12. package/dist/index.d.ts +164 -0
  13. package/dist/index.js +4432 -3641
  14. package/docs/API.md +1 -0
  15. package/docs/ARCHITECTURE.md +1 -1
  16. package/docs/CASE_STUDY.md +4 -2
  17. package/docs/ENVIRONMENT.md +9 -2
  18. package/docs/FFI_BUN_GUIDE.md +154 -9
  19. package/docs/HOW_TO_ADD_AN_OP.md +10 -0
  20. package/docs/REPO_MAP.md +16 -5
  21. package/docs/RND-CONCURRENCY.md +772 -0
  22. package/docs/bun-builtins-decision-matrix.md +4 -1
  23. package/index.ts +5 -0
  24. package/package.json +172 -163
  25. package/rust/crypto/pbkdf2.rs +9 -0
  26. package/rust/ffi/mod.rs +1 -0
  27. package/rust/ffi/task.rs +172 -0
  28. package/rust/http/form.rs +6 -2
  29. package/rust/http/query_parser.rs +53 -22
  30. package/rust/ingress/native_route.rs +131 -140
  31. package/rust/ingress/pipeline.rs +3 -3
  32. package/rust/ingress/tests.rs +5 -4
  33. package/rust/json/json_ser.rs +13 -11
  34. package/rust/lib.rs +1 -0
  35. package/rust/task/completion.rs +119 -0
  36. package/rust/task/mod.rs +65 -0
  37. package/rust/task/ops.rs +503 -0
  38. package/rust/task/runtime.rs +220 -0
  39. package/rust/task/tests.rs +231 -0
  40. package/rust/util/bytes.rs +238 -51
  41. package/src/ingress/health.ts +1 -2
  42. package/src/native/ffi/build/codecs.ts +29 -7
  43. package/src/native/ffi/build/compress.ts +8 -0
  44. package/src/native/ffi/build/instances.ts +7 -2
  45. package/src/native/ffi/build/metrics.ts +23 -0
  46. package/src/native/ffi/build/task.ts +79 -0
  47. package/src/native/ffi/build/util.ts +18 -0
  48. package/src/native/ffi/build.ts +2 -0
  49. package/src/native/ffi/selftest.ts +3 -1
  50. package/src/native/ffi/types.ts +40 -0
  51. package/src/native/ffi.ts +73 -14
  52. package/src/native/loader.ts +53 -3
  53. package/src/rust-ffi/scalar/http.ts +54 -16
  54. package/src/rust-ffi/text.ts +25 -3
  55. package/src/shared/bytes.ts +19 -0
  56. package/src/task/index.ts +24 -0
  57. package/src/task/op.ts +171 -0
  58. package/src/task/runtime.ts +568 -0
package/docs/API.md CHANGED
@@ -41,6 +41,7 @@ runtimes, and binary ops (`gzip*`, `brotli*`, `aead*`, `pbkdf2Sha256`,
41
41
  | `encoder`, `decoder` | Codec-backed UTF-8 helpers (shared native `TextEncoder` / Bun: `CString`; Node: `TextEncoder`/`TextDecoder`). `encode`/`decode` accept the `Uint8Array | string` union and normalize. |
42
42
  | `uuidv7()` | UUIDv7 — `Bun.randomUUIDv7` on Bun, `crypto.randomUUID` on Node. |
43
43
  | `AdaptiveEstimate`, `AdaptiveEstimateOptions` | Bounded EWMA adaptive-estimate utility. |
44
+ | `createTaskRuntime`, `TASK_OP`, `TASK_STATUS_OK` / `_ERROR` / `_CANCELLED`, `TaskRuntime`, `TaskStats`, `TaskRunOptions`, `Pbkdf2RunOptions`, `encodeGzipDecompressArgs`, `encodeGzipCompressHeader`, `encodeBrotliDecompressArgs`, `encodeArgon2VerifyArgs`, `encodePbkdf2Args`, `gzipCompressUpperBound`, `brotliCapacityGuess` | Off-thread task runtime ("castrum Tasks"): `await` native work on a Rust pool without blocking the event loop — `gzipDecompress`, `brotliDecompress`, `gzipCompress`, `pbkdf2Sha256`, `argon2Verify`, `AbortSignal`, `stats()`, zero-copy results *and* inputs. See [`RND-CONCURRENCY.md`](./RND-CONCURRENCY.md) §1b. |
44
45
  | `createMetrics`, `DEFAULT_BUCKETS` | Zero-dependency metrics registry (counters/gauges/histograms + Prometheus text). |
45
46
  | `packBatch`, `packPairs`, `readPairsPacked`, `readHttpPacked`, `pairsToObject`, `parseQueryString`, `parseCookieHeader`, `parseFormBody`, `unpackBitset`, `unpackByteResults`, `unpackI64ArrayAsBigInt`, `unpackU32Array` | Packed-wire + high-level parsing utilities. |
46
47
  | Ingress: `createIngress`, `createIngressSync`, `createIngressFast`, `createIngressHandler`, `createIngressServer`, `createIngressServerNode`, `createIngressRouter`, route factories, metrics/health/trace helpers | See [`INGRESS.md`](./INGRESS.md) and [`INGRESS-ROUTER.md`](./INGRESS-ROUTER.md). |
@@ -31,7 +31,7 @@ The bridge between TypeScript and Rust has **two transports**, both loading the
31
31
  cdylib (built with napi-rs):
32
32
 
33
33
  - **`bun:ffi` (PRIMARY on Bun)** — `src/native/ffi.ts` `dlopen`s the addon's
34
- `extern "C"` exports (`rust/ffi/`, 109 `castrum_*` symbols — 97 direct + 4
34
+ `extern "C"` exports (`rust/ffi/`, 119 `castrum_*` symbols — 107 direct + 4
35
35
  `validator_c_abi!` + 4 `validator_bytes_c_abi!` + 4 `compress_to_out!`,
36
36
  parity guarded by
37
37
  `test/unit/native/ffi-symbol-parity.test.ts`) and Bun JIT-calls them
@@ -10,7 +10,9 @@ benchmarks actually showed, and the lessons that generalise to any
10
10
  All figures below come from the reports in `bench/results/` (generated on a
11
11
  release addon, baseline CPU, Bun 1.4.0). They're **representative**, not
12
12
  advertised as universal — re-run `bun run check` / `bun run bench:http` on your
13
- own hardware before trusting them for your numbers.
13
+ own hardware before trusting them for your numbers. (The local toolchain is now
14
+ Bun 1.4.2; the figures below still carry their 1.4.0 measurement date, and the
15
+ newer `bench:task` / `bench:ffi:margin` rows name 1.4.2.)
14
16
 
15
17
  ---
16
18
 
@@ -42,7 +44,7 @@ as a pre-baked pipeline, and still be usable from both runtimes.
42
44
  ### 2.1 One cdylib, two transports
43
45
 
44
46
  The addon is built with napi-rs (so Node can load it), but it *also* exports
45
- `extern "C"` symbols (`rust/ffi/`, **109 `castrum_*` symbols** — 97 direct + 4
47
+ `extern "C"` symbols (`rust/ffi/`, **119 `castrum_*` symbols** — 107 direct + 4
46
48
  `validator_c_abi!` + 4 `validator_bytes_c_abi!` + 4 `compress_to_out!`) so Bun can
47
49
  `dlopen` it directly:
48
50
 
@@ -33,6 +33,13 @@ for false; anything else is warned about and falls back to the default.
33
33
  | `CASTRUM_MAX_RAYON_THREADS` | int | `cores` | Hard cap on rayon threads (native). |
34
34
  | `CASTRUM_PIN_CORES` | (presence) | — | When set (Linux), pin rayon worker threads to distinct cores. |
35
35
 
36
+ ## Off-thread task pool (`rust/task/`)
37
+
38
+ | Variable | Type | Default | Description |
39
+ |----------|------|---------|-------------|
40
+ | `CASTRUM_TASK_THREADS` | int | `max(1, cores-1)` | Worker threads for the off-thread task runtime (castrum Tasks). Read when the pool first starts — the pool is process-wide and cannot be resized afterwards (`tasks.stats().threads` reports the live count). |
41
+ | `CASTRUM_TASK_PIN_CORES` | (presence) | — | When set (Linux), pin each task worker to its own core (core 0 is skipped so the JS event loop keeps it). `CASTRUM_PIN_CORES` enables it too. Off by default: pinning competes with the JS/rayon threads for the same box. |
42
+
36
43
  ## Native transport selection (`src/native/ffi.ts`)
37
44
 
38
45
  `bun:ffi` is the PRIMARY transport under Bun; NAPI is the fallback (Node,
@@ -66,11 +73,11 @@ whether the ffi transport is live.
66
73
  | `CASTRUM_FFI_LOAD_MS` | int | `400` | Per-window duration (ms) of the `ffi-all` load-run (`bench/ffi/ffi-all.ts`). |
67
74
  | `CASTRUM_FFI_LOAD_WINDOWS` | int | `4` | Number of load windows in `ffi-all` (`bench/ffi/ffi-all.ts`). |
68
75
 
69
- ## Publishing (`scripts/prepublish.mjs`, `publish-manual.mjs`)
76
+ ## Publishing (`scripts/prepublish.mjs`, canonical `scripts/release.ts`)
70
77
 
71
78
  | Variable | Type | Default | Description |
72
79
  |----------|------|---------|-------------|
73
- | `CASTRUM_PUBLISH_ALLOW_PARTIAL` | bool | — | When `1`, `prepublishOnly` ships a tarball containing only the locally-built platforms instead of failing on the full `napi.targets` set. Set automatically by `bun run publish:manual`; **never** set it for a normal `npm publish`.
80
+ | `CASTRUM_PUBLISH_ALLOW_PARTIAL` | bool | — | When `1`, `prepublishOnly` ships a tarball containing only the locally-built platforms instead of failing on the full `napi.targets` set. Set automatically by `bun run release:manual` (the local-publish path of the canonical release flow); **never** set it for a normal `npm publish`.
74
81
 
75
82
  ## Postinstall fallback build (`scripts/postinstall.mjs`)
76
83
 
@@ -7,10 +7,12 @@
7
7
  > code is **optimized for Bun** out of the box — correct on the first `dlopen`,
8
8
  > fast on the first call, and safe under burst load.
9
9
  >
10
- > **Version anchor**: behavior claims are verified against the **Rust-based Bun
11
- > 1.3.14** source (`/home/adeel/poc/bun`). `bun:ffi` is experimental upstream;
12
- > the bind-time self-test + napi fallback are what make it safe to use, and
13
- > they are mandatory — not optional.
10
+ > **Version anchor**: behavior claims were distilled from the **Rust-based Bun
11
+ > 1.3.14** source (`/home/adeel/poc/bun`) and RE-VERIFIED against the installed
12
+ > **Bun 1.4.2** runtime (the local dev toolchain, matching the CI matrix). Where
13
+ > a claim is runtime-specific the number is stated inline. `bun:ffi` is
14
+ > experimental upstream; the bind-time self-test + napi fallback are what make
15
+ > it safe to use, and they are mandatory — not optional.
14
16
 
15
17
  ---
16
18
 
@@ -191,6 +193,66 @@ returns.
191
193
  **Rule**: use per-thread reused buffers for cstring returns; never return a
192
194
  pointer into JS-owned memory; treat `null` as the failure sentinel.
193
195
 
196
+ ### 6.1 `cstring` ARGS: the engine transcodes — and truncates at NUL
197
+
198
+ The same doc section defines the **argument** side, which is a different trade
199
+ from returns:
200
+
201
+ > "When used in `args`, `FFIType.cstring` accepts everything `ptr` does and
202
+ > additionally accepts a JavaScript string directly. The engine transcodes the
203
+ > string to a **null-terminated** UTF-8 buffer that lives for the duration of
204
+ > the call, so you don't need to encode it into a `Buffer` yourself."
205
+
206
+ This is a real win — it removes a `TextEncoder` round trip per call, and it is
207
+ why castrum binds the string surfaces (`validate*`, `metrics_*`,
208
+ `regex_escape_str`, `ws_accept_key`, `mime_from_extension`, `session_*`, the JWT
209
+ /route id strings) as `cstring` ARGs, with byte-input callers crossing as
210
+ `(buffer, buffer_length)` instead. But "null-terminated" is load-bearing:
211
+
212
+ **An embedded `U+0000` silently truncates the value native-side.** Where the
213
+ callee answers with a VERDICT, that is a correctness hole, and two shipped bugs
214
+ came from it (found 2026-09-11, both now fixed and pinned by
215
+ `test/unit/native/cstring-nul.test.ts`):
216
+
217
+ | call | was | now |
218
+ |---|---|---|
219
+ | `rust.validateEmail(bytes('a@b.com\0<script>'))` | `true` | `false` |
220
+ | `rust.regexEscape('abc\0def')` | `'abc'` | `'abc\0def'` |
221
+
222
+ The validator case was the worst shape: bytes were decoded to a JS string and
223
+ handed to the `cstring` ARG, so a NUL **byte** made a non-address validate. The
224
+ fix routes byte-input APIs through the `*_bytes` symbols — which is also
225
+ **faster**, because the bytes then cross as an atomic `(buffer, buffer_length)`
226
+ pair with no transcode. Measured through the PUBLIC API (`rust.validateEmail`,
227
+ 400k calls, Bun 1.4.2): email 236 → 110 ns, uuid 153 → 50 ns, ipv4 118 → 37 ns.
228
+ Correctness and speed pointed the same way; the old comment claiming the decode
229
+ route was faster predated the byte siblings.
230
+
231
+ Two further byte-input sites were closed the same way, with one caveat worth
232
+ recording. `mimeFromExtension` no longer resolves a NUL-suffixed extension as its
233
+ prefix (`'.js\0evil'` resolved as `.js` — a Content-Type chosen from truncated
234
+ input; it now takes the unknown sentinel). `wsAcceptKey` no longer computes the
235
+ accept value for a prefix of the key. Both KEEP the `cstring` fast path and divert
236
+ only the NUL case, because measuring the real call path reversed the raw-binding
237
+ result: `rust.wsAcceptKey` is 367 ns via `cstring` versus 381 ns via the byte
238
+ sibling (the extra `subarray` + decode outweighs the transcode saved), whereas
239
+ measuring the bare binding suggested the opposite. **Measure through the function
240
+ users call, not the binding** — the guard itself costs ~4 ns, so guarding is
241
+ cheap and routing everything through bytes is not always a win.
242
+
243
+ **Rule**: a `cstring` ARG is only correct when truncation cannot change the
244
+ answer — i.e. the string is developer-supplied config or NUL-free by
245
+ construction (a MIME extension, a base64 key, a header name). Anything
246
+ reachable from user input must either
247
+
248
+ 1. cross as bytes (`buffer` + `buffer_length`) when the API is byte-shaped, or
249
+ 2. be guarded with `hasNul` (`src/native/ffi/build/util.ts`): short-circuit the
250
+ verdict (`false`), throw at declare time (metric names / label keys), or
251
+ route to the `(ptr, usize)` sibling for a byte-exact result (`regexEscape`,
252
+ batch hex validation).
253
+
254
+ Never silently return a value derived from a truncated prefix.
255
+
194
256
  ---
195
257
 
196
258
  ## 7. `dlopen` lifetime — never `close()` while bound symbols live
@@ -295,6 +357,75 @@ napi-style unwind guard exists for raw `extern "C"`).
295
357
  the scalar builder, and `ffi-symbol-parity.test.ts`; a Rust unit test +
296
358
  self-test; cross-check FFI vs napi byte-for-byte in `ffi.test.ts`.
297
359
 
360
+ ### 11.1 Conformance with Bun's published FFI guidance (audit)
361
+
362
+ The question "are we using Bun's FFI the way Bun recommends?" is answerable from
363
+ this table rather than from memory. Verified 2026-09-11 against
364
+ `https://bun.sh/docs/api/ffi` and the live binding on Bun 1.4.2
365
+ (`ffiBufferMode() === 'buffer-pair'`).
366
+
367
+ | Bun's guidance (docs) | castrum | Where |
368
+ |---|---|---|
369
+ | `buffer` + `buffer_length`: pass the **same view twice** for atomic ptr+len | **Followed and LIVE** — `abi()` rewrites every `(ptr,usize)` pair; `probeBufferLength()` verifies support at bind time with a `(ptr,usize)` fallback | §4, `src/native/ffi.ts` |
370
+ | `cstring` **returns** are cloned at call time (NULL → null) | Followed — per-thread `CSTR_BUF`, clone is synchronous | §6, `rust/ffi/` |
371
+ | `cstring` **args** accept a JS string; the engine transcodes it | Followed for the string surface; **byte-input APIs use `(buffer,buffer_length)` instead — measured 2.2–3.2× faster AND immune to NUL truncation** | §6.1 |
372
+ | `u64_fast`/`i64_fast` for byte counts (avoid BigInt boxing) | Followed — `U64_FAST` on every size/count return | §5 |
373
+ | `JSCallback`: pass `.ptr`, not the object ("slight performance boost") | Followed — `taskSetDoorbell(Number(cb.ptr))` | §10 |
374
+ | `read.u8/u32/...` avoids a `DataView`/`ArrayBuffer` for pointer reads | Applied where it matters (hot paths use pooled JS scratch, not pointer reads); bind-time blob reads keep `DataView` | `instances.ts` |
375
+ | `toArrayBuffer` with a deallocator for C-owned memory | **Deliberately NOT used** — Bun 1.4.2's deallocator path segfaults (`arity 1`, crash at `0x0`) while the docs describe 4/5-arg forms. Zero-copy instead keeps ownership in JS | `docs/RND-CONCURRENCY.md` §7a |
376
+ | `dlopen` is never `close()`d while symbols live | Followed — bound once, held forever | §7 |
377
+ | Async functions are not supported in `JSCallback` | Followed — the doorbell is a `void()` trampoline | §10 |
378
+
379
+ Reading the table honestly: the `buffer`/`buffer_length` and `cstring` return
380
+ practices were already conformant. The real gaps were on the **argument** side
381
+ (§6.1) — and those turned out to be correctness bugs, not just style.
382
+
383
+ ### 11.2 Audit log — 2026-09-11 (Bun 1.4.2)
384
+
385
+ The audit above was run because the practices were *questioned*, not because a
386
+ failure was observed. Recording what it found, including the parts that were
387
+ already right, so nobody has to re-derive it:
388
+
389
+ **Already conformant (verified, do not "fix"):**
390
+
391
+ - `buffer`/`buffer_length` is live — `ffiBufferMode() === 'buffer-pair'`, and
392
+ `abi()` rewrites every `(ptr, usize)` pair.
393
+ - `cstring` for returns and for the string surface; `u64_fast` for byte counts;
394
+ `JSCallback.prototype.ptr`; `dlopen` never closed. All per the docs.
395
+
396
+ **Found and fixed (all correctness, two of them security-grade):**
397
+
398
+ | finding | evidence | fix |
399
+ |---|---|---|
400
+ | Validator verdicts could be fooled | `validateEmail(bytes('a@b.com\0<script>'))` → `true` | byte sibling (`buffer`/`buffer_length`), 2.2–3.2× faster too |
401
+ | `regexEscape` truncated output | `'abc\0def'` → `'abc'` | route the NUL case to the `(ptr,len)` sibling |
402
+ | Batch hex validation could pass a prefix | joined ids truncated at a NUL | same |
403
+ | Query/cookie gates could validate a prefix | `cstring` ARG + user-supplied wire text | short-circuit `false` on NUL |
404
+ | Metric label values silently merged series | truncation collapsed two label values into one | throw at declare time / reject at record |
405
+ | `mimeFromExtension` resolved a NUL-suffixed extension as its prefix | `.js\0evil` → `text/javascript` | unknown sentinel |
406
+ | `wsAcceptKey` hashed a prefix of the key | suffix silently ignored | byte path for the NUL case |
407
+
408
+ **Methodological corrections (both were nearly written into the docs as fact):**
409
+
410
+ 1. **Raw-binding probes mislead.** `getBunFFI()!.op(...)` omits the memoized
411
+ `ffi()` hop and the caller's own decode/subarray. For `wsAcceptKey` it made the
412
+ byte path look 11% faster when through `rust.wsAcceptKey` it is 4% *slower*
413
+ (381 vs 367 ns) — so that call keeps `cstring` and diverts only the NUL case.
414
+ 2. **Closure-wrapped micro-benchmarks inflate small costs.** A guard that
415
+ measured 90 ns inside a benchmark closure is 3.6 ns isolated (`indexOf`). That
416
+ ~25× error would have justified adding a C-ABI symbol that earns nothing.
417
+
418
+ **Also closed:** the sweep found no other non-conformance. `toArrayBuffer`
419
+ appears once in `src/` (a comment explaining why it is avoided), the only
420
+ `bun:ffi` import outside the native layer is a type import, `read.*` has no
421
+ hot-path use (our pointer reads happen once at bind), and `rust.text.*` was
422
+ already FFI-first with `cstring` (zero encode).
423
+
424
+ **Follow-ups, deliberately not done:** `mimeFromExtension` still decodes for the
425
+ `cstring` ARG because no byte sibling exists (a symbol that would earn ~40 ns on
426
+ a non-hot path); `@types/bun` remains 1.3.14 (see §13 — the cache has no newer
427
+ version, so the casts stay).
428
+
298
429
  ---
299
430
 
300
431
  ## 12. Bun's own Rust hygiene (borrowed rules)
@@ -318,11 +449,23 @@ that transfer directly to castrum's cdylib:
318
449
 
319
450
  ## 13. Version anchors & verification
320
451
 
321
- The behavioral claims above are verified against the Rust-based Bun source
452
+ The behavioral claims above were distilled from the Rust-based Bun source
322
453
  checkout at `/home/adeel/poc/bun` (LATEST file: 1.3.14; `package.json` tracks
323
- the 1.4 development line). The decision matrix and ADR-0003 measure against
324
- the installed **Bun 1.4.0** runtime — both anchors describe the same
325
- Rust-based runtime family. Castrum's own measurements live in `bench/`.
454
+ the 1.4 development line) and are **re-verified against the installed Bun 1.4.2
455
+ runtime** that the code actually runs on (and that the CI matrix pins). The
456
+ decision matrix and ADR-0003 measured on the installed **Bun 1.4.0** runtime —
457
+ all anchors describe the same Rust-based runtime family. Castrum's own
458
+ measurements live in `bench/`.
459
+
460
+ **Known gap (2026-09-11): the type package lags the runtime.** `package.json`
461
+ pins `@types/bun@^1.3.14` / `bun-types@1.3.14` while the runtime is 1.4.2, and the
462
+ local package cache holds only 1.3.14 (so bumping it needs network access). The
463
+ visible consequence is that 1.4-era `bun:ffi` declarations are absent from the
464
+ types — `buffer_length` is not a valid `FFITypeOrString` literal there, so
465
+ `src/native/ffi.ts` casts it (`as unknown as readonly FFITypeOrString[]`). Any
466
+ new `bun:ffi` surface added in 1.4.x will need the same treatment until the
467
+ types are upgraded. The casts are load-bearing, not sloppiness: dropping them is
468
+ a type error, not a behavior change.
326
469
 
327
470
  | Claim | Source (Bun checkout `/home/adeel/poc/bun`) |
328
471
  |---|---|
@@ -378,7 +521,9 @@ to bun:ffi itself:
378
521
  The minimal C-dylib control experiment bound every suspect signature shape
379
522
  (`cstring` leading/middle/trailing, mixed with `buffer`/`buffer_length`,
380
523
  `u64_fast` returns) against trivial exports: all marshaled correctly on Bun
381
- 1.4.0/1.4.1 canaries. **No engine bug exists to report.**
524
+ 1.4.0/1.4.1 canaries. **No engine bug exists to report.** (Re-run on the 1.4.2
525
+ binding: `ffiBufferMode() === 'buffer-pair'`, i.e. the `buffer`/`buffer_length`
526
+ pair is still live, and the full symbol set passes the bind-time self-test.)
382
527
 
383
528
  ### Rules for any ABI change (signature, arg order, or type)
384
529
 
@@ -72,6 +72,16 @@ Rules (see `docs/FFI_BUN_GUIDE.md` for the full contract):
72
72
  pass as `usize`/number.
73
73
  - String ARGS come in as `*const c_char` (bun:ffi `cstring`); use
74
74
  `CStr::from_ptr(header).to_bytes()`.
75
+
76
+ **NUL caveat**: the engine's transcode is NUL-terminated, so an embedded
77
+ `U+0000` TRUNCATES the value before Rust sees it. Only bind a `cstring` ARG
78
+ for developer-supplied config or input that cannot contain a NUL. If the op is
79
+ reachable from user input and answers with a verdict (validate/gate) or a
80
+ byte-exact value, provide a `*_bytes` sibling taking `(ptr,len)` (preferred —
81
+ exact length, no transcode, and measured faster for byte-shaped callers) or
82
+ guard the string form with `hasNul` (`src/shared/bytes.ts`).
83
+ See `docs/FFI_BUN_GUIDE.md` §6.1; regressions are caught by
84
+ `test/unit/native/cstring-nul.test.ts`.
75
85
  - String RETURNS use `cstring_return` (per-thread `CSTR_BUF`, cloned at call
76
86
  time — never return a pointer into JS-owned memory).
77
87
  - **Variable-size outputs**: return the EXACT required byte count on a
package/docs/REPO_MAP.md CHANGED
@@ -53,7 +53,7 @@ Bun is the primary runtime; Node.js ≥20.3 is supported via a compiled ESM entr
53
53
  | Version consistency | `bun run check:version` | package.json ↔ Cargo.toml ↔ CHANGELOG |
54
54
  | JS dependency audit | `bun run audit` | `bun audit` (reports advisories) |
55
55
  | Cargo deny audit | `bun run deny` | `cargo deny check` |
56
- | Publish (manual, single-platform) | `bun run publish:manual[:dry]` | see §6 |
56
+ | Publish (canonical) | `bun run release[:dry]` / `release:manual` | see §6 |
57
57
 
58
58
  ---
59
59
 
@@ -144,6 +144,11 @@ src/
144
144
  websocket.ts createWebSocketUpgrade (RFC 6455 101 handshake + subprotocol).
145
145
  streaming.ts sseResponse (SSE frames via rust.sseEncodeEvent).
146
146
  baseline/ Pure-JS baseline implementations (benchmark reference). Internal.
147
+ task/ Off-thread task runtime ("castrum Tasks", Bun-first). See §RND-CONCURRENCY.
148
+ op.ts PURE op ids + packed-arg encoders (no addon import).
149
+ runtime.ts createTaskRuntime() — process-wide singleton; submit → pool → batched
150
+ doorbell → drained completion → promise; AbortSignal; zero-copy
151
+ submitInto for ops with a size hint.
147
152
  bench/ CPU benchmark framework (tasks, measure, report, checks, comparisons).
148
153
  data/ json-rows.ts fixture generator (benchmark-only).
149
154
  ```
@@ -169,13 +174,17 @@ rust/ ONE cdylib crate (Cargo [lib] → lib.rs).
169
174
  native stack), options/time/packed, cors, proxy, ip_trust, rate_limit,
170
175
  terminal, output.rs (single numeric layout source),
171
176
  ingress_constants.rs (napi projection).
172
- ffi/ #[no_mangle] extern "C" exports (109 castrum_* symbols — 97 direct + 4
177
+ task/ Off-thread task runtime (\"castrum Tasks\"): condvar pool (N = cores−1,
178
+ CASTRUM_TASK_THREADS, batch dequeue + spin-then-park + optional
179
+ CASTRUM_TASK_PIN_CORES), result ring + batched thread-safe doorbell,
180
+ op dispatch + cancel + catch_unwind containment + `_into` ops.
181
+ ffi/ #[no_mangle] extern "C" exports (119 castrum_* symbols — 107 direct + 4
173
182
  validator_c_abi! + 4 validator_bytes_c_abi! + 4 compress_to_out!; parity guarded by
174
183
  test/unit/native/ffi-symbol-parity.test.ts) for Bun's bun:ffi
175
184
  primary transport, incl. castrum_ingress_layout (the layout blob).
176
185
  mod.rs (module map) + util.rs (panic_guard / HMAC cache / cstring helpers)
177
186
  + per-domain wrappers (hashing/validators/crypto/jwt/http/payload/json/
178
- rate_limit/ingress/route/probe) + tests.rs.
187
+ rate_limit/ingress/route/task/probe) + tests.rs.
179
188
  panic_safety.rs / test_support.rs Cross-module fuzz + shared #[cfg(test)] helpers.
180
189
  proptest_suite.rs Property-based adversarial-parser tests (dev-dep proptest).
181
190
  ```
@@ -284,8 +293,9 @@ consumed.
284
293
  (declared in package.json `napi.binaryName` + Cargo.toml — keep in sync).
285
294
  - **Multi-platform (recommended)**: push a `v*` tag → `.github/workflows/ci.yml`
286
295
  builds each platform addon, the `publish` job downloads them into
287
- `./artifacts`, stages them, and runs `npm publish` (needs `NPM_TOKEN`).
288
- - **Manual single-platform**: `bun run publish:manual --increment minor`
296
+ `./artifacts`, stages them, and runs `npm publish` (npm Trusted Publishing /
297
+ OIDC — no token secret; needs the Trusted Publisher configured on npmjs.com).
298
+ - **Manual single-platform**: `bun run release:manual`
289
299
  (syncs package.json ↔ Cargo.toml ↔ CHANGELOG, tags, builds, publishes with
290
300
  `CASTRUM_PUBLISH_ALLOW_PARTIAL=1`). `--dry-run` plans only.
291
301
  - `prepublishOnly` runs `build:js` + `scripts/prepublish.mjs` (stages
@@ -302,6 +312,7 @@ consumed.
302
312
  | [`API.md`](./API.md) | The complete public API reference |
303
313
  | [`CASE_STUDY.md`](./CASE_STUDY.md) | Data-driven case study: how castrum was built and measured |
304
314
  | [`ARCHITECTURE.md`](./ARCHITECTURE.md) | Deep-dive: modules, data flow, output layout, memory, concurrency |
315
+ | [`RND-CONCURRENCY.md`](./RND-CONCURRENCY.md) | R&D + design plan: native (Rust) task offload for Bun — the "goroutine" gap, measured Bun concurrency costs, proposed `rust.tasks` runtime |
305
316
  | [`INGRESS.md`](./INGRESS.md) | The pre-baked ingress API + route factories + servers |
306
317
  | [`BENCHMARKS.md`](./BENCHMARKS.md) | Benchmark scenarios, report format, how to run |
307
318
  | [`ENVIRONMENT.md`](./ENVIRONMENT.md) | All `CASTRUM_*` env vars (+ legacy aliases) |