knitting 0.1.51 → 0.1.52

package/README.md

@@ -11,44 +11,55 @@
 [![Deno](https://img.shields.io/badge/deno-2%2B-000000?logo=deno&logoColor=white)](https://deno.com/)
 [![Bun](https://img.shields.io/badge/bun-1%2B-f472b6?logo=bun&logoColor=white)](https://bun.sh/)
 
-Knitting is a worker pool over a shared-memory IPC runtime for Node.js, Deno, and Bun. Our mission is to make JavaScript a multicore language with real inter-runtime communication. 
+Knitting is a worker pool over a shared-memory IPC runtime for Node.js, Deno,
+and Bun. Our mission is to make JavaScript a multicore language with real
+inter-runtime communication.
 
-Thanks to its memory design, it can be 5x to 25x faster than using `postMessages` , bypassing OS socket communication entirely with a novel protocol written from scratch.
+Thanks to its memory design, it can be 5x to 25x faster than using
+`postMessages` , bypassing OS socket communication entirely with a novel
+protocol written from scratch.
 
-Use it for parts of your program that should run in different environments, such as CPU-intensive tasks, small jobs, runtime-isolated tasks, custom isolation for workers in Docker or bwrap environments, long-running tools, or any processes that require speed and type flexibility.
+Use it for parts of your program that should run in different environments, such
+as CPU-intensive tasks, small jobs, runtime-isolated tasks, custom isolation for
+workers in Docker or bwrap environments, long-running tools, or any processes
+that require speed and type flexibility.
 
 You export a function or task, spin up a pool, and call it like a normal async
 function:
 
 ```ts
-
 const result = await pool.call.resizeImage(file);
-
 ```
 
 So you only have to take care of 4 things:
- - Export a function or task
- - Create a pool
- - Call it
- - Let `using` or `shutdown()` close the pool
-
 
-Under the hood, we take care of scheduling and orchestration across worker threads or separate processes, also handling signals, timeouts, life cycles, memory allocation, garbage collection, and cross-runtime memory over the processes.
+- Export a function or task
+- Create a pool
+- Call it
+- Let `using` or `shutdown()` close the pool
 
+Under the hood, we take care of scheduling and orchestration across worker
+threads or separate processes, also handling signals, timeouts, life cycles,
+memory allocation, garbage collection, and cross-runtime memory over the
+processes.
 
 ## Why use it?
 
-- Easy to use: Have a multithreaded environment or process with few lines of code.
-- Great type support: pass primitives, JSON, Promise of these, and special types (typed arrays, `Node Buffer`, `Envelope`, and `ProcessSharedBuffer`).
+- Easy to use: Have a multithreaded environment or process with few lines of
+  code.
+- Great type support: pass primitives, JSON, Promise of these, and special types
+  (typed arrays, `Node Buffer`, `Envelope`, and `ProcessSharedBuffer`).
 - Runtime flexibility: the same API across Node.js, Deno, and Bun.
-- Worker choices: use threads for fast pools or processes for stronger isolation.
-- All out-of-the-box experiences: strict-by-default permissions, payload-size limits, task timeouts, abort-aware tasks, and worker hard timeouts.
+- Worker choices: use threads for fast pools or processes for stronger
+  isolation.
+- All out-of-the-box experiences: strict-by-default permissions, payload-size
+  limits, task timeouts, abort-aware tasks, and worker hard timeouts.
 
 ## Requirements
 
-- Node.js 22+ 
-- Deno 2+ 
-- Bun 1+ 
+- Node.js 22+
+- Deno 2+
+- Bun 1+
 
 ## Install
 
@@ -86,8 +97,8 @@ if (isMain) {
 ```
 
 The `isMain` guard when the same module is loaded by workers or process. Export
-exposes the tasks or functions at module scope, so knitting maps down the imports,
-then use and use the pool only from the main program.
+exposes the tasks or functions at module scope, so knitting maps down the
+imports, then use and use the pool only from the main program.
 
 ## The Mental Model
 
@@ -100,7 +111,7 @@ import { createPool, isMain, task } from "knitting";
 - `task(...)` describes a callable worker function (types + implementation).
 
 - `createPool(options)({ tasks })` starts workers and gives you a typed `call`
-object for invoking tasks.
+  object for invoking tasks.
 
 - `pool.shutdown()` stops workers when you're done.
 
@@ -182,8 +193,8 @@ export const pixels = task<ResizeInput, number>({
 ```
 
 Supported payloads are listed below. For large binary data, prefer
-`ArrayBuffer`, typed arrays, or `ProcessSharedBuffer` instead of serializing
-big objects.
+`ArrayBuffer`, typed arrays, or `ProcessSharedBuffer` instead of serializing big
+objects.
 
 ### Task timeouts
 
@@ -312,20 +323,20 @@ const pool = createPool({
 
 Common options you might tweak:
 
-| Option | What it does |
-| --- | --- |
-| `threads` | Number of workers to start. |
-| `balancer` | Scheduling strategy: `"roundRobin"`, `"firstIdle"`, `"randomLane"`, `"firstIdleOrRandom"`, or the legacy alias `"robinRound"`. |
-| `payload` | Shared payload-buffer settings: `mode`, `payloadInitialBytes`, `payloadMaxByteLength`, and `maxPayloadBytes`. |
-| `abortSignalCapacity` | Number of shared abort slots available to abort-aware calls. |
-| `worker.resolveAfterFinishingAll` | Let submitted calls finish before shutdown resolves. |
-| `worker.bootstrap` | Privileged async hook imported and awaited before task modules load. |
-| `worker.hardTimeoutMs` | Force pool shutdown when a task exceeds this many milliseconds. |
-| `worker.runtime` | Choose `"thread"` or `"process"` workers. |
-| `worker.processSharedMemory` | Process-worker memory discovery: `"inherit"` by default on POSIX, or `"named"` for wrappers/containers that cannot preserve fd 0. |
-| `permission` | Runtime permission policy for workers. |
-| `debug` | Enable extra diagnostics. |
-| `source` | Worker source override for advanced runtimes. |
+| Option                            | What it does                                                                                                                      |
+| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
+| `threads`                         | Number of workers to start.                                                                                                       |
+| `balancer`                        | Scheduling strategy: `"roundRobin"`, `"firstIdle"`, `"randomLane"`, `"firstIdleOrRandom"`, or the legacy alias `"robinRound"`.    |
+| `payload`                         | Shared payload-buffer settings: `mode`, `payloadInitialBytes`, `payloadMaxByteLength`, and `maxPayloadBytes`.                     |
+| `abortSignalCapacity`             | Number of shared abort slots available to abort-aware calls.                                                                      |
+| `worker.resolveAfterFinishingAll` | Let submitted calls finish before shutdown resolves.                                                                              |
+| `worker.bootstrap`                | Privileged async hook imported and awaited before task modules load.                                                              |
+| `worker.hardTimeoutMs`            | Force pool shutdown when a task exceeds this many milliseconds.                                                                   |
+| `worker.runtime`                  | Choose `"thread"` or `"process"` workers.                                                                                         |
+| `worker.processSharedMemory`      | Process-worker memory discovery: `"inherit"` by default on POSIX, or `"named"` for wrappers/containers that cannot preserve fd 0. |
+| `permission`                      | Runtime permission policy for workers.                                                                                            |
+| `debug`                           | Enable extra diagnostics.                                                                                                         |
+| `source`                          | Worker source override for advanced runtimes.                                                                                     |
 
 ### Worker bootstrap
 
@@ -374,10 +385,9 @@ const pool = createPool({
 })({ add });
 ```
 
-`processRuntime` can be `"node"`, `"deno"`, or `"bun"` and defaults to
-`"deno"`. You can also provide a `processCommandPrefix` when workers need to be
-launched through a wrapper such as a package manager, container command, or
-runtime shim.
+`processRuntime` can be `"node"`, `"deno"`, or `"bun"` and defaults to `"deno"`.
+You can also provide a `processCommandPrefix` when workers need to be launched
+through a wrapper such as a package manager, container command, or runtime shim.
 
 That prefix is also useful for sandbox and resource-control tools. The one
 important detail is that process workers receive their shared-memory handle on
@@ -428,8 +438,8 @@ const pool = createPool({
 
 On Windows, Knitting automatically uses named shared memory for the
 process-worker control channel. You do not need to set
-`processSharedMemory: "named"` yourself — the runtime detects Windows and
-forces it.
+`processSharedMemory: "named"` yourself — the runtime detects Windows and forces
+it.
 
 ```ts
 // Works on Windows without extra options.
@@ -525,7 +535,10 @@ Worker calls can carry the following values across the shared-memory transport:
 - Plain objects and arrays made from supported values.
 - `ArrayBuffer`, Node `Buffer`, `DataView`, and supported typed arrays.
 - `ProcessSharedBuffer`.
-- `Envelope` for metadata plus binary payloads.
+- `BufferReference` from `knitting/unsafe` for experimental zero-copy buffers to
+  thread workers (same process only; see below).
+- `Envelope` for a JSON header plus a binary body (`ArrayBuffer`,
+  `SharedArrayBuffer`, `ProcessSharedBuffer`, or `BufferReference`).
 - `Error`, `Date`, and global symbols created with `Symbol.for(...)`.
 - Native `Promise<supported-value>` inputs. The promise is awaited before
   dispatch.
@@ -544,12 +557,13 @@ shouldn't) cross the boundary:
 
 ### Envelope
 
-`Envelope` pairs a JSON-serializable header with a binary `ArrayBuffer`
-payload. Use it when a call needs both structured metadata and raw bytes in a
-single argument.
+`Envelope` pairs a JSON-serializable header with a binary body. Use it when a
+call needs both structured metadata and raw bytes in a single argument — the
+transport carries one special binary value per call, so an envelope is the way
+to attach a header to one.
 
 ```ts
-import { Envelope, createPool, isMain, task } from "knitting";
+import { createPool, Envelope, isMain, task } from "knitting";
 
 export const processImage = task<
   Envelope<{ format: string }>,
@@ -577,6 +591,58 @@ if (isMain) {
 }
 ```
 
+#### Body types
+
+The body is generic — `Envelope<Header, Body>` — and accepts any of the binary
+shapes the transport understands:
+
+| Body                 | Copy?                | Workers          | Notes                                            |
+| -------------------- | -------------------- | ---------------- | ------------------------------------------------ |
+| `ArrayBuffer`        | copied               | thread + process | The default body; works everywhere.              |
+| `SharedArrayBuffer`  | zero-copy, shared    | thread only      | Shared by reference; process workers reject it.  |
+| `ProcessSharedBuffer`| zero-copy, shared    | thread + process | Cross-process shared memory.                     |
+| `BufferReference`    | zero-copy, moved     | thread only      | From `knitting/unsafe`; same constraints as bare `BufferReference`. |
+
+The header keeps its fast paths regardless of the body: a small header is
+written inline, and only large headers spill to the dynamic payload region. A
+zero-copy body keeps its own semantics — a `SharedArrayBuffer` stays shared by
+reference, and a `BufferReference` body is still moved (its source is detached)
+and joins the same borrow/copy/release flow it follows on its own.
+
+```ts
+import { createPool, Envelope, isMain, task } from "knitting";
+import { BufferReference } from "knitting/unsafe";
+
+export const invert = task<
+  Envelope<{ op: string }, BufferReference>,
+  Envelope<{ op: string }, BufferReference>
+>({
+  f: (envelope) => {
+    const pixels = envelope.payload.toUint8Array();
+    const out = new Uint8Array(pixels.length);
+    for (let i = 0; i < pixels.length; i++) out[i] = 255 - pixels[i];
+    return new Envelope({ op: "inverted" }, new BufferReference(out));
+  },
+});
+
+if (isMain) {
+  using pool = createPool({ threads: 1 })({ invert });
+  const pixels = new Uint8Array([0, 64, 128, 192, 255]);
+
+  using result = await pool.call.invert(
+    new Envelope({ op: "invert" }, new BufferReference(pixels)),
+  );
+  console.log(result.header, [...result.payload.toUint8Array()]);
+}
+```
+
+`Envelope` is disposable: disposing it (via `using` or `Symbol.dispose`)
+disposes a disposable body such as a `BufferReference`, and is a harmless no-op
+for `ArrayBuffer` / `SharedArrayBuffer` bodies. See
+[Experimental zero-copy buffers for thread workers](#experimental-zero-copy-buffers-for-thread-workers)
+for the full `BufferReference` constraints, which apply unchanged to a
+`BufferReference` body.
+
 If a payload is large, set `payload.maxPayloadBytes` deliberately and prefer
 binary/shared-memory shapes over deeply nested objects.
 
@@ -743,8 +809,8 @@ There are three moving parts:
 
 - `processSharedMemory: "named"` lets the Docker worker find Knitting's control
   channel.
-- `ProcessSharedBuffer.create({ mode: "create", name, size })` makes the
-  payload buffer reopenable by name.
+- `ProcessSharedBuffer.create({ mode: "create", name, size })` makes the payload
+  buffer reopenable by name.
 - `--ipc=host` lets the container see the same POSIX shared-memory namespace.
 
 This is same-host communication. It is fast because both sides map the same
@@ -752,10 +818,103 @@ bytes, but it is not a network transport and it deliberately shares IPC with the
 container. Use names like capabilities: generate them, keep them private, and
 unlink them when the shared memory is no longer needed.
 
+### Experimental zero-copy buffers for thread workers
+
+`BufferReference` lives in `knitting/unsafe`. It is experimental and may be
+changed or removed if its safety tradeoffs are not acceptable. It **moves** a
+buffer's ownership to a **thread** worker: constructing one detaches the source,
+so the bytes travel to the worker without being serialized through the
+transport. Send a result back the same way — return a `BufferReference` from the
+worker. It is the same-process counterpart to `ProcessSharedBuffer`: reach for
+it when you hold a large `ArrayBuffer` or typed array and the copy cost to a
+thread worker actually matters.
+
+```ts
+import { createPool, isMain, task } from "knitting";
+import { BufferReference } from "knitting/unsafe";
+
+export const invert = task<BufferReference, BufferReference>({
+  f: (ref) => {
+    const pixels = ref.toUint8Array(); // the moved bytes, no copy
+    const out = new Uint8Array(pixels.length);
+    for (let i = 0; i < pixels.length; i++) out[i] = 255 - pixels[i];
+    return new BufferReference(out); // move the result back to the host
+  },
+});
+
+if (isMain) {
+  const pixels = new Uint8Array([0, 64, 128, 192, 255]);
+  using pool = createPool({ threads: 1 })({ invert });
+
+  // `pixels` is detached by the move; the result comes back as a BufferReference.
+  const result = await pool.call.invert(new BufferReference(pixels));
+  console.log([...result.toUint8Array()]); // [255, 191, 127, 63, 0]
+}
+```
+
+Read these constraints before reaching for it:
+
+- **Thread workers only.** The handle is a process-local pointer. Process
+  workers do not share it, so a `BufferReference` sent to a process worker
+  throws. For cross-process sharing use `ProcessSharedBuffer`.
+- **ArrayBuffer only.** `SharedArrayBuffer` is already shareable and cannot be
+  detached, so `BufferReference` rejects SAB sources and SAB-backed typed-array
+  views.
+- **Move semantics.** Constructing a `BufferReference` detaches its source — the
+  original buffer is empty afterward, and reads/writes through it are gone. The
+  bytes now belong to the reference; to get a result back, the worker returns
+  its own `BufferReference`. Each handle is one-shot. Forward inputs the worker
+  materializes with `.toArrayBuffer()`/`.toUint8Array()` are borrowed for the
+  duration of the call and detached once it settles; do not keep using them from
+  fire-and-forget work after the task returns.
+- **Forward is zero-copy everywhere; the return is zero-copy on Node.** Sending
+  a buffer to the worker never copies. On Node the returned buffer is also
+  handed back with no copy (the engine co-owns the backing store across
+  threads); on Deno and Bun the host takes a single copy of the returned bytes,
+  because their FFI cannot co-own a worker-thread backing store. Both are far
+  cheaper than serializing a large buffer through the transport.
+- **Borrowed Deno/Bun returns are opt-in.** The default is
+  `unsafe: { BufferReferenceReturn: "copy" }` — the safe single copy described
+  above. Set it to `"borrow"` on `createPool` to skip that copy on Deno/Bun by
+  borrowing the worker's backing store until the returned `BufferReference` is
+  released. Call `ref.release()` or use `using`, and do it before shutting down
+  the producing worker. **After `release()` the borrowed bytes are gone —
+  reading the reference, or any view you took from it, is a use-after-free.**
+  If the bytes escape into HTTP responses, streams, timers, callbacks, or caches,
+  copy them before the borrowed reference is released.
+- **Unsafe escape hatch.** This is not a security boundary. Forged metadata or
+  unsynchronized host/worker mutation can still be unsafe.
+- **Node uses a native addon.** Bun and Deno go through their FFI; Node uses the
+  `knitting_buffer_pointer` prebuild shipped with the package (or
+  `bun run build:native` when developing on a new ABI). Without it, constructing
+  a `BufferReference` on Node throws.
+
+Borrowed returns, end to end (Node is always zero-copy; this opts Deno/Bun in):
+
+```ts
+using pool = createPool({
+  threads: 1,
+  unsafe: {
+    BufferReferenceReturn: "borrow",
+  },
+})({ invert });
+
+{
+  using result = await pool.call.invert(new BufferReference(pixels));
+  const out = result.toUint8Array(); // borrowed — valid only while `result` lives
+  console.log([...out]);
+} // `using` releases the borrow here; do not read `out` after this point
+```
+
+When in doubt, a plain `ArrayBuffer` or typed-array payload — which knitting
+copies through the shared transport — is simpler and works for both thread and
+process workers. Reach for `BufferReference` only when the copy cost of a large
+buffer to a thread worker actually matters: below roughly 256 KiB the per-call
+pointer setup tends to cost more than just copying, so the plain transport wins.
+
 ### Current support
 
-Knitting supports Node.js 22+, Deno 2+, and Bun 1+ on Linux, macOS, and
-Windows.
+Knitting supports Node.js 22+, Deno 2+, and Bun 1+ on Linux, macOS, and Windows.
 
 Thread workers work without native pieces. Process workers and
 `ProcessSharedBuffer` use the platform's shared-memory APIs. Release packages
@@ -766,8 +925,8 @@ FFI path; if you are developing locally on a new Node ABI or architecture, run:
 bun run build:native
 ```
 
-For Deno projects with permissions enabled, allow FFI when using process
-workers or `ProcessSharedBuffer`.
+For Deno projects with permissions enabled, allow FFI when using process workers
+or `ProcessSharedBuffer`.
 
 ## Runtime Safety
 

package/knitting.d.ts

@@ -2,3 +2,4 @@ import { workerMainLoop } from "./src/worker/loop.js";
 import { createPool, importTask, isMain, task } from "./src/api.js";
 import { Envelope } from "./src/common/envelope.js";
 export { createPool as createPool, Envelope as Envelope, importTask as importTask, isMain as isMain, task as task, workerMainLoop as workerMainLoop, };
+export type { EnvelopeBody as EnvelopeBody, EnvelopeHeader as EnvelopeHeader, } from "./src/common/envelope.js";

package/map.md

@@ -20,6 +20,9 @@ The core flow is:
   `importTask`, `isMain`, `Envelope`, and `workerMainLoop`.
 - `process-shared-buffer.ts`: Secondary public export for process-shared-buffer
   helpers.
+- `unsafe.ts`: Public export for experimental lower-level capabilities such as
+  `BufferReference`.
+- `utils.ts`: Public export for request/response buffer conversion helpers.
 - `README.md`: User-facing documentation, examples, configuration, and command
   reference.
 - `package.json`: Package metadata, export map, shipped files, scripts, runtime
@@ -54,8 +57,15 @@ The core flow is:
 - `.github/workflows/publish.yml`: Manual native-prebuild workflow. Verifies
   Node and Windows FFI prebuilds, checks the JSR package contents, and commits
   updated `prebuilds/` artifacts back to the branch.
-- `docs/windows-process-worker-hang-fix.md`: Investigation notes for the
-  Windows process-worker shared-memory and parked-worker hang fixes.
+
+## Project Guidance And Docs
+
+- `docs/AGENTS.md`: Agent/contributor orientation, invariants, and workflow
+  notes.
+- `docs/CLAUDE.md`: Pointer to the shared agent guidance.
+- `docs/buffer-reference-ownership-move.md`: Design record for
+  `BufferReference` ownership transfer.
+- `docs/knitting.pdf`: Longer-form project design/theory document.
 
 ## Public API Layer
 
@@ -88,6 +98,8 @@ The core flow is:
 - `src/common/with-resolvers.ts`: `Promise.withResolvers` compatibility helper.
 - `src/common/worker-runtime.ts`: Runtime-neutral worker/thread/process-worker
   detection, parent-port access, and message-channel creation.
+- `src/utils/http.ts`: Buffer conversion helpers for HTTP/request-response
+  boundaries, including raw bytes, strings, JSON, and numeric arrays.
 
 ## Runtime Host Side
 
@@ -286,6 +298,7 @@ The core flow is:
 - `test/task-abort-context-api.test.ts`: Worker abort toolkit/context behavior.
 - `test/tx-queue.test.ts`: Host transmit queue behavior and late-result safety.
 - `test/type-inference.test.ts`: Public type inference guarantees.
+- `test/utils.test.ts`: Public utility conversion helpers.
 - `test/worker-bootstrap.test.ts`: Worker bootstrap hook behavior, shared-buffer
   metadata revival, startup failure propagation, and inliner incompatibility.
 - `test/fixtures/*.ts`: Task modules used by tests.
@@ -305,4 +318,3 @@ The core flow is:
 - `log/`, `logs`, and `*.log`: Local runtime/log output.
 - `node_modules/`: Installed dependencies. Not part of the source map.
 
-

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "knitting",
-  "version": "0.1.51",
+  "version": "0.1.52",
   "description": "Shared-memory IPC runtime for Node.js, Deno, and Bun.",
   "license": "Apache-2.0",
   "repository": {
@@ -23,6 +23,14 @@
     "./process-shared-buffer": {
       "types": "./process-shared-buffer.d.ts",
       "default": "./process-shared-buffer.js"
+    },
+    "./unsafe": {
+      "types": "./unsafe.d.ts",
+      "default": "./unsafe.js"
+    },
+    "./utils": {
+      "types": "./utils.d.ts",
+      "default": "./utils.js"
     }
   },
   "files": [
@@ -38,7 +46,11 @@
     "knitting.d.ts",
     "knitting.js",
     "process-shared-buffer.d.ts",
-    "process-shared-buffer.js"
+    "process-shared-buffer.js",
+    "unsafe.d.ts",
+    "unsafe.js",
+    "utils.d.ts",
+    "utils.js"
   ],
   "scripts": {
     "build": "bun run build.ts",

package/scripts/build-native-addons.ts

@@ -232,6 +232,11 @@ const addons = [
     source: "src/knitting_shm.cc",
     output: "build/Release/knitting_shm.node",
   },
+  {
+    name: "knitting_buffer_pointer",
+    source: "src/knitting_buffer_pointer.cc",
+    output: "build/Release/knitting_buffer_pointer.node",
+  },
 ];
 const ffiLibraries = isWindows
   ? [

package/src/api.d.ts

@@ -1,5 +1,5 @@
 import { endpointSymbol } from "./common/task-symbol.js";
-import type { Args, AbortSignalConfig, AbortSignalOption, AbortSignalToolkit, ComposedWithKey, CreatePool, FixPoint, MaybePromise, Pool, TaskInput, ReturnFixed, ImportTaskOptions, TaskTimeout, tasks } from "./types.js";
+import type { AbortSignalConfig, AbortSignalOption, AbortSignalToolkit, Args, ComposedWithKey, CreatePool, FixPoint, ImportTaskOptions, MaybePromise, Pool, ReturnFixed, TaskInput, tasks, TaskTimeout } from "./types.js";
 type ToListAndIds = {
     list: string[];
     ids: number[];
@@ -11,9 +11,7 @@ type CreatePoolFactory = (options: CreatePool) => <T extends tasks>(tasks: T) =>
 type InferredTaskFunction = (...args: any[]) => MaybePromise<Args>;
 type InferredTaskInput<F extends InferredTaskFunction, AS extends AbortSignalOption> = Parameters<F> extends [] ? void : AS extends undefined ? Parameters<F> extends [infer A] ? A extends TaskInput ? A : never : never : Parameters<F> extends [infer A] ? A extends TaskInput ? A : never : Parameters<F> extends [infer A, AbortSignalToolkit<AS>] ? A extends TaskInput ? A : never : never;
 type InferredTaskOutput<F extends InferredTaskFunction> = Awaited<ReturnType<F>> extends infer R ? R extends Blob ? never : R extends Args ? R : never : never;
-type InferredTaskShape<F extends InferredTaskFunction, AS extends AbortSignalOption> = [
-    InferredTaskInput<F, AS>
-] extends [never] ? never : [InferredTaskOutput<F>] extends [never] ? never : {
+type InferredTaskShape<F extends InferredTaskFunction, AS extends AbortSignalOption> = [InferredTaskInput<F, AS>] extends [never] ? never : [InferredTaskOutput<F>] extends [never] ? never : {
     readonly f: F;
     readonly timeout?: TaskTimeout;
 } & (AS extends undefined ? {
@@ -24,12 +22,7 @@ type InferredTaskShape<F extends InferredTaskFunction, AS extends AbortSignalOpt
 export declare const isMain: boolean;
 export { endpointSymbol as endpointSymbol };
 /**
- *  With this information we can recreate the logical order of
- *  relevant exported functions from a file, also it helps to
- *  track a task before naming, ` export ` elements have to be declared
- *  at top level and without branching, we take advantage of this to
- *  correctly map them.
- *
+ * Reconstructs stable task order from top-level exports before names are bound.
  */
 export declare const toListAndIds: ToListAndIdsFn;
 export declare const createPool: CreatePoolFactory;

package/src/api.js

@@ -11,7 +11,7 @@ import { genTaskID } from "./common/task-source.js";
 import { toModuleUrl } from "./common/module-url.js";
 import { endpointSymbol } from "./common/task-symbol.js";
 import { spawnWorkerContext } from "./runtime/pool.js";
-import { RUNTIME_IS_MAIN_THREAD, RUNTIME_WORKER_DATA, } from "./common/worker-runtime.js";
+import { RUNTIME_IS_MAIN_THREAD, RUNTIME_POOL_DEPTH, RUNTIME_WORKER_DATA, } from "./common/worker-runtime.js";
 import { resolvePermissionProtocol, toRuntimePermissionFlags, } from "./permission/index.js";
 import { getNodeProcess } from "./common/node-compat.js";
 import { managerMethod } from "./runtime/balancer.js";
@@ -22,12 +22,7 @@ const DEFAULT_IMPORT_EXPORT_NAME = "default";
 export const isMain = RUNTIME_IS_MAIN_THREAD;
 export { endpointSymbol as endpointSymbol };
 /**
- *  With this information we can recreate the logical order of
- *  relevant exported functions from a file, also it helps to
- *  track a task before naming, ` export ` elements have to be declared
- *  at top level and without branching, we take advantage of this to
- *  correctly map them.
- *
+ * Reconstructs stable task order from top-level exports before names are bound.
  */
 export const toListAndIds = (args) => {
     const result = args
@@ -98,7 +93,8 @@ const toPoolTaskEntries = (input, callerHref) => Object.entries(input).map(([nam
     }
     throw new TypeError(`createPool task "${name}" must be a task definition or exported function`);
 });
-export const createPool = ({ threads, debug, inliner, balancer, payload, payloadInitialBytes, payloadMaxBytes, bufferMode, maxPayloadBytes, abortSignalCapacity, source, worker, workerExecArgv, permission, dispatcher, host, }) => (tasks) => {
+export const createPool = ({ threads, debug, inliner, balancer, payload, unsafe, payloadInitialBytes, payloadMaxBytes, bufferMode, maxPayloadBytes, abortSignalCapacity, source, worker, workerExecArgv, permission, dispatcher, host, }) => (tasks) => {
+    const bufferReferenceReturn = unsafe?.BufferReferenceReturn;
     /**
      *  This functions is only available in the main thread.
      *  Also triggers when debug extra is enabled.
@@ -209,6 +205,15 @@ export const createPool = ({ threads, debug, inliner, balancer, payload, payload
     const hardTimeoutMs = Number.isFinite(resolvedWorker?.hardTimeoutMs)
         ? Math.max(1, Math.floor(resolvedWorker?.hardTimeoutMs))
         : undefined;
+    if (RUNTIME_POOL_DEPTH >= 1) {
+        throw new Error(`createPool() tried to spawn workers from inside a worker process ` +
+            `(pool depth ${RUNTIME_POOL_DEPTH}). This usually means a pool is ` +
+            `created at module scope in a module your workers import, so every ` +
+            `worker spawns its own pool recursively. Is your createPool protected ` +
+            `by isMain? Guard pool creation behind \`if (isMain) { ... }\` ` +
+            `(import { isMain } from "knitting") so only the main program starts ` +
+            `the pool.`);
+    }
     let workers = Array.from({
         length: threads ?? 1,
     }).map((_, thread) => spawnWorkerContext({
@@ -224,6 +229,7 @@ export const createPool = ({ threads, debug, inliner, balancer, payload, payload
         workerExecArgv: execArgv,
         host: hostDispatcher,
         payload,
+        bufferReferenceReturn,
         payloadInitialBytes,
         payloadMaxBytes,
         bufferMode,
@@ -377,19 +383,17 @@ export const createPool = ({ threads, debug, inliner, balancer, payload, payload
         if (imported && usingInliner) {
             return buildImportedInvoker(handlers);
         }
-        return useDirectHandler
-            ? handlers[0]
-            : managerMethod({
-                contexts: workers,
-                balancer,
-                handlers,
-                inlinerGate: usingInliner
-                    ? {
-                        index: inlinerIndex,
-                        threshold: inlinerDispatchThreshold,
-                    }
-                    : undefined,
-            });
+        return useDirectHandler ? handlers[0] : managerMethod({
+            contexts: workers,
+            balancer,
+            handlers,
+            inlinerGate: usingInliner
+                ? {
+                    index: inlinerIndex,
+                    threshold: inlinerDispatchThreshold,
+                }
+                : undefined,
+        });
     };
     let callEntries;
     try {