knitting 0.1.70 → 0.1.73

package/README.md

@@ -104,6 +104,9 @@ if (isMain) {
 }
 ```
 
+The examples use `using`, which Node.js 22 cannot parse: there, write
+`const pool = ...` and call `await pool.shutdown()` when you are done.
+
 Use the `isMain` guard when a module can be loaded by both the host and its
 workers. Export tasks at module scope so Knitting can find them, then create and
 use the pool only from the main program.
@@ -153,8 +156,8 @@ if (isMain) {
 ```
 
 `using` starts pool shutdown when the scope exits and does not wait for it. Use
-`await pool.shutdown()` when you need to wait for shutdown or pass a shutdown
-delay.
+`await using pool = ...` or `await pool.shutdown()` when you need to wait for
+shutdown; only `shutdown()` takes a shutdown delay.
 
 Deno 2+, Bun 1+, and Node.js 24+ parse `using` natively. Node.js 22 does not: it
 has `Symbol.dispose`, but the declaration itself is a `SyntaxError`, and Node's
@@ -474,7 +477,11 @@ Common options you might tweak:
 | `permission`                      | Runtime permission policy for workers.                                                                                            |
 | `host.dispatcher`                 | Experimental host dispatcher topology: `"per-thread"` or `"serial-channel"`.                                                      |
 | `host.steal`                      | Shared-submit work stealing for compatible multi-worker thread/process pools; enabled by default. Set `false` to use private submit lanes. |
-| `debug`                           | Enable diagnostics (`host`, `globals`, `signals`, `imports`, `lifecycle`) or use `KNITTING_DEBUG`.                                |
+| `host.stealRegionLanes`           | Submit lanes claimed per stealing handshake (a power of two). Smaller regions are fairer for expensive tasks; wider regions amortise arbitration for cheap ones. |
+| `host.stealClaim`                 | Claim discipline: `"ticket"` (default) or `"dekker"`. Also settable with `KNITTING_STEAL_CLAIM`; an unrecognised value is rejected.  |
+| `host.doorbell`                   | Wait for completion notifications instead of polling an empty return mailbox; enabled by default where supported. Set `false` to force polling. |
+| `host.nativeDoorbell`             | Opt into Node's native `uv_async_t` completion bridge for thread workers. Off by default; ignored when `host.doorbell` is `false`. |
+| `debug`                           | Enable diagnostics (`host`, `globals`, `signals`, `imports`, `lifecycle`, `steal`) or use `KNITTING_DEBUG`.                       |
 | `source`                          | Worker source override for advanced runtimes.                                                                                     |
 
 ### Worker bootstrap
@@ -530,6 +537,65 @@ protocol limit also fall back. Set `host: { steal: false }` or
 where arbitration has nothing to rebalance. `host: { steal: true }` or
 `KNITTING_STEAL=1` forces it for an otherwise compatible pool.
 
+Two options tune the arbitration itself, and both only apply to a stealing pool:
+
+- `host.stealRegionLanes` is how many submit lanes one handshake claims (a power
+  of two). **A region is a batch**: a wide region amortises arbitration best for
+  cheap tasks, but it also lets one worker claim work the others could have run
+  in parallel. Set it to `1` (or a small value) when per-task cost dominates
+  arbitration cost. The default is the widest region the lane budget allows.
+- `host.stealClaim` selects the claim discipline: `"ticket"` (the default)
+  claims in publication order from one monotonic counter; `"dekker"` gives each
+  consumer its own intent slot and requires at least one region per consumer.
+  Set `KNITTING_STEAL_CLAIM=ticket` or `KNITTING_STEAL_CLAIM=dekker` to select
+  it from the environment; the explicit option wins. **An unrecognised value is
+  an error, not a fallback**: a typo, or a `cas-mask` setting left over from
+  when that discipline existed, fails at pool creation rather than quietly
+  running a discipline you did not choose.
+
+The ticket discipline uses a 64-bit claim head and a wrapping 32-bit
+publication tail. The head CAS validates the tail snapshot: at most 32 tickets
+can be pending, so unsigned subtraction recovers the distance across a tail wrap
+without ever recycling a claim identity. `stealRegionLanes` caps the number of
+tickets one claim takes, and its default width is unchanged from Dekker's.
+Claims are ordered, but workers may decode or complete later claims first.
+Ticket identities never wrap: the queue fails closed if a claim would exhaust
+the signed 64-bit sequence.
+
+Ticket is the default while it is under evaluation, so ordinary runs exercise
+it; `"dekker"` remains explicitly selectable and is unchanged. The two differ in
+how they behave after a fatal fault. Dekker releases the region and lets a peer
+finish the rest. **Ticket fails closed**: a fatal decoder or worker failure
+closes the shared queue, rejects outstanding calls, and rejects every future
+call, so that pool has to be shut down and replaced. Claimed work is not
+replayed, because a failed decoder may already have consumed payload state.
+Ordinary task promise rejections stay task-local under both.
+
+### Completion doorbells
+
+By default the host waits to be told a result is ready instead of repeatedly
+polling an empty return mailbox. `host.doorbell` is enabled wherever a wake path
+exists, and each runtime uses the one it has:
+
+- Node and Bun thread workers use `Atomics.waitAsync`.
+- Deno uses a thread-safe FFI callback, because its `waitAsync` does not wake an
+  idle event loop. It is skipped when FFI permission is unavailable.
+- Process workers use a process-local completion transport, since Atomics
+  waiters are per-isolate and cannot be rung from another process.
+- Anything unsupported or denied falls back to the portable polling path.
+
+`host.nativeDoorbell: true` additionally opts Node thread workers into the
+native `uv_async_t` bridge from the `knitting_doorbell` addon. It is off by
+default, does not apply to process workers, and is ignored entirely when
+`host.doorbell` is `false`.
+Node thread permissions must allow native addons for this bridge; otherwise
+Knitting uses the portable wake path.
+
+Set `host: { doorbell: false }` to force polling — useful for controlled
+comparisons, and for pools that oversubscribe the machine. A doorbell only makes
+progress when the host gets scheduled, so once workers occupy every core a wake
+has to preempt one.
+
 ### Useful tuning options
 
 - Increase `threads` for parallel CPU-heavy work.
@@ -795,7 +861,8 @@ const pool = createPool({
 
 ## Permissions
 
-Knitting defaults to a strict worker permission policy:
+Knitting defaults to a strict worker permission policy where the selected
+runtime supports it:
 
 ```ts
 permission: { mode: "strict", allowImport: true }
@@ -853,16 +920,33 @@ backward compatible and produce a once-per-runtime warning.
 cannot be represented. When a wrapper or cross-runtime host hides the target
 Node version, Knitting uses the conservative Node 22/24 capability set.
 
-These compatibility checks currently cover process workers. Thread workers use
-the host runtime's worker behavior and should not be treated as a sandbox.
-Runtime permissions are guardrails, not the only security boundary for hostile
-code.
+These compatibility checks currently cover process workers. Node thread
+workers receive their resolved Node permission flags, and Knitting fails pool
+creation if Node cannot apply them. Deno thread workers inherit the creator's
+permissions because Knitting does not yet set Deno's worker-specific permission
+options, which are unstable and gated by `--unstable-worker-options`; Bun
+thread workers do not have a matching permission mechanism here.
+For cross-runtime permission enforcement, use a process worker with a runtime
+that supports the restrictions you need; Deno has the broadest coverage in the
+table above. Runtime permissions are guardrails, not the only security boundary
+for hostile code.
 
 The top-level `ffi` permission is the explicit cross-runtime native-code
 capability. On Node it enables both native addons and `node:ffi`; Node's
 `--allow-ffi` permission is currently unrestricted. The legacy/runtime-specific
 `node.allowAddons` and `node.allowFfi` switches are independent—enabling addons
-does not silently enable FFI.
+does not silently enable FFI. Node thread workers deny addon loading by default
+under the strict policy. Set `permission.node.allowAddons: true` when a task
+needs an addon-backed feature on addon-backed Node versions: `SharedArrayBuffer`
+arguments, returns and `Envelope` bodies, `ProcessSharedBuffer`, and
+`BufferReference`. Node 26 maps the same pointers through `node:ffi`, so there
+the switch is `permission.node.allowFfi`; top-level `ffi: true` grants both.
+Without it a Node thread worker cannot map those pointers: the worker crashes
+and later calls on it fail. Either switch lets task code load native code
+generally, so use it only for trusted tasks. Two paths degrade instead of
+failing when addon loading is denied: large returns are copied rather than
+moved, and the optional native completion doorbell falls back to the portable
+wake path.
 
 Node process workers are a transport exception: Knitting needs `--allow-addons`
 on Node 22/24 or `--allow-ffi` on Node 26 to map their shared memory. Deno
@@ -883,6 +967,13 @@ Knitting aims to make the safer path the default:
 - Shutdown can stop immediately or wait for submitted work with
   `worker.resolveAfterFinishingAll`.
 
+When Knitting itself rejects a call, the reason is a `KnittingError` with a
+`code`: `KNT_ERROR_0`–`KNT_ERROR_3` for an argument the host cannot encode, and
+`WORKER_STARTUP_FAILED`, `WORKER_CRASHED`, `WORKER_EXITED` or `THREAD_CLOSED`
+when the worker behind the call is gone. Calls to a dead worker reject at once
+instead of staying pending. A return value the worker cannot encode still
+rejects with the bare `KNT_ERROR_n` string.
+
 That said, workers still run code. If you treat tasks like plugins, keep
 permissions tight, keep named shared-memory names hard to guess, and avoid
 passing broad capabilities into worker code.
@@ -1248,7 +1339,7 @@ ownership move on thread workers:
 
 | Runtime | Result ownership |
 | --- | --- |
-| Node 22/24 with the addon | The host co-owns the V8 backing store: zero byte copies. |
+| Node 22/24 with the addon | The host co-owns the V8 backing store: zero byte copies. Under the strict permission policy this needs `permission.node.allowAddons`; otherwise it takes the one-private-copy fallback. |
 | Deno and Bun | The host makes one private copy before the worker releases its pin. |
 | Older Node backend | The same one-private-copy fallback. |
 

package/knitting.browser.d.ts

@@ -1,6 +1,8 @@
 import { workerMainLoop } from "./src/worker/loop.ts";
 import { createPool, importTask, isMain, setModuleUrl, task } from "./src/api.ts";
 import { Envelope } from "./src/common/envelope.ts";
+import { KnittingError } from "./src/error.ts";
 import { isNumericArray, NumericArray } from "./src/connections/numeric-array.ts";
-export { createPool as createPool, Envelope as Envelope, importTask as importTask, isMain as isMain, isNumericArray as isNumericArray, NumericArray as NumericArray, setModuleUrl as setModuleUrl, task as task, workerMainLoop as workerMainLoop, };
+export { createPool as createPool, Envelope as Envelope, importTask as importTask, isMain as isMain, isNumericArray as isNumericArray, KnittingError as KnittingError, NumericArray as NumericArray, setModuleUrl as setModuleUrl, task as task, workerMainLoop as workerMainLoop, };
 export type { EnvelopeBody as EnvelopeBody, EnvelopeHeader as EnvelopeHeader, } from "./src/common/envelope.ts";
+export type { KnittingErrorCode } from "./src/error.ts";