@ai-ecoverse/slicc-kernel 1.160.1 → 1.160.3

package/README.md

@@ -207,7 +207,7 @@ kernel.terminate();
 
 `createNodeKernel({ root, modules, commands, env, network, worker, facades, processMounts, fstabRetries, cdp, hostfs, hostfsOrigin, hostfsFetch, users, onDenial, approve, compileCache, warmWorker, warmWorkers })` takes the options of `createKernel` except `metadata`, and `fstabRetries`, the delays in ms between tries of a failing `/etc/fstab` line (default `[1000, 4000, 16000]`). `root` is an in-memory directory by default (`memoryRoot()` makes another), which keeps each file in 1 MiB chunks, and POSIX metadata stays in memory. Processes and threads run on `worker_threads`. The kernel has `run`, `openTerminal`, `users`, `setRoutes`, `updateTransportTraits` and `terminate` as above, plus `integrity.revoke(path?)`, `root`, `writeFile(path, data, { mode }?)` (creating the parent directories; `mode` sets the permission bits, `0` to `0o7777`, before the data is written), `chmod(path, mode)` and `readFile(path)` to put files in place and read results; like the `fs` of an attached client, they go through the mounts, as the processes see them. `nodeTransport()` is `fetchTransport()` with Node's `fetch`, which no CORS binds (`crossOrigin: 'any'`). `connect({ user })` and `attachKernel` work as in the browser: the port is a `worker_threads` `MessagePort`, which a worker thread can attach with, and `terminate()` ends every attached client.
 
-`compileCache` (opt-in, off by default; #1285, slicc-jsh#566) is an absolute host directory where each process worker keeps the compiled code of the JS programs it imports (`module.enableCompileCache`), so the next spawn of the same program skips V8's compile. It stores V8's code cache, keyed and checksummed by the module's source, one file per program (about 7 MB for a 2.4 MB bundle), shared by every kernel given the same directory; programs can't reach or write it (the worker hands it to Node before the lockdown, and `node:worker_threads` stays refused). Measured on a 2.4 MB bundle, a spawn went from about 54 to 39 ms; a trivial program costs about 1 ms more. **The directory must be private to the embedder:** V8's code cache is trusted bytecode that Node only checksums, so whoever can write it can plant code that runs inside a process worker past the lockdown. Never make it reachable from inside the kernel (no `hostfs` mount of it or of a parent), and don't share one between kernels of different trust. A host's own `NODE_COMPILE_CACHE` is never handed to the process workers, with or without the option, so programs never land in a host-wide cache. If a host preload (`--import` or `--require`) has already enabled another compile cache in the workers, a kernel given `compileCache` refuses to run programs there instead of caching them in the host's directory.
+`compileCache` (#1285, slicc-jsh#566) is where each process worker keeps the compiled code of the JS programs it imports (`module.enableCompileCache`), so the next spawn of the same program skips V8's compile. **By default a Node kernel writes a private directory of its own** (#1597): `slicc-kernel-cc-<pid>-<random>` under the host's temp directory (`os.tmpdir()`), mode 0700, removed by `terminate()`. The first kernel in a host process also removes any such directory whose process is gone, so a crashed host leaves none behind for good. Measured on Node 26 with a 2 MB module, the default takes a spawn from about 21 to 8 ms and adds no measurable RSS per process. `compileCache: false` turns it off, and an absolute host directory uses that one instead, shared by every kernel given it. It stores V8's code cache, keyed and checksummed by the module's source, one file per program (about 7 MB for a 2.4 MB bundle), shared by every kernel given the same directory; programs can't reach or write it (the worker hands it to Node before the lockdown, and `node:worker_threads` stays refused). Measured on a 2.4 MB bundle, a spawn went from about 54 to 39 ms; a trivial program costs about 1 ms more. **The directory must be private to the embedder:** V8's code cache is trusted bytecode that Node only checksums, so whoever can write it can plant code that runs inside a process worker past the lockdown. Never make it reachable from inside the kernel (no `hostfs` mount of it or of a parent), and don't share one between kernels of different trust. A host's own `NODE_COMPILE_CACHE` is never handed to the process workers, with or without the option, so programs never land in a host-wide cache. If a host preload (`--import` or `--require`) has already enabled another compile cache in the workers, a kernel given a `compileCache` directory refuses to run programs there instead of caching them in the host's directory; a kernel on its default private directory leaves it to the host's, as it did before the default.
 
 `warmWorker` (on by default; `false` turns it off; #1341, slicc-jsh#566) keeps one process worker booted ahead of the next spawn, fork or thread on the Node entry, so the spawn skips the worker's boot: it has loaded the process worker module and nothing else, no pid, credentials, environment, cwd, descriptors, lockdown or program, all of which arrive with its init message, and every worker still serves exactly one process or thread and is then terminated. Its own `process.env` is a snapshot taken when the spare is created rather than at the spawn, but a program never sees it (`process` is withheld, and a program's environment is the one its spawn gives it). The spare never keeps the host alive, is replaced after each spawn off the critical path, is dropped if it errors or exits while idle, and is released after 30 s without a spawn; `terminate()` ends it. It costs about 11 MB of host RSS while it waits. `warmWorkers: n` (opt-in; #1361, slicc-jsh#566) keeps up to `n` spares instead of one (an integer from 0 to 64; `warmWorker: false` still means none), so a burst of starts, as worker pools make, finds a spare for each: all are refilled after each take off the critical path, and all are released together after 30 s without a spawn and ended by `terminate()`. Each costs about 7–11 MB of host RSS while it waits. It pays off for bursts of `Worker` threads (jsh measured about 13 ms less per start in a burst of 8); for bursts of child processes that boot a large runtime of their own, the refills compete for CPU with the children and the gain is small or none.
 
@@ -359,7 +359,7 @@ A WASI command can name an ES module of its package that provides imports of its
 
 ### JS programs
 
-A command can be an ES module instead of a wasm program: `"abi": "js"` and `module` name the module in the package, which must be one self-contained file (no relative or bare imports), and `args`, `argv0` and `env` work as for other commands. An entry with `"abi": "js"` but no `module` (or one outside the package) is not a command, like a wasm entry without its module, so running it fails with 127. Every run gets its own process worker, so each process evaluates the module afresh: module-level state never carries over from one run to the next. The kernel keeps an importable URL per module source (a `blob:` URL in the browser, a base64 `data:` URL under Node) for the 16 sources it ran last, so a command run in a loop doesn't encode its module again each time ([#846](https://github.com/ai-ecoverse/slicc-kernel/issues/846)). A changed file is a new source and gets a new URL. An evicted URL is revoked once every run it was handed to has exited, so a run that is still loading its module never loses it.
+A command can be an ES module instead of a wasm program: `"abi": "js"` and `module` name the module in the package, which must be one self-contained file (no relative or bare imports), and `args`, `argv0` and `env` work as for other commands. An entry with `"abi": "js"` but no `module` (or one outside the package) is not a command, like a wasm entry without its module, so running it fails with 127. Every run gets its own process worker, so each process evaluates the module afresh: module-level state never carries over from one run to the next. The kernel keeps an importable URL per module source (a `blob:` URL in the browser; under Node a short `slicc-module:<sha-256>` id that the process worker is handed together with the source, so no base64 is made or decoded, [#1596](https://github.com/ai-ecoverse/slicc-kernel/issues/1596)) for the 16 sources it ran last, so a command run in a loop doesn't encode its module again each time ([#846](https://github.com/ai-ecoverse/slicc-kernel/issues/846)). A changed file is a new source and gets a new URL. An evicted URL is revoked once every run it was handed to has exited, so a run that is still loading its module never loses it.
 
 ```json
 { "slicc": { "commands": { "jstool": { "abi": "js", "module": "dist/jstool.mjs" } } } }
@@ -591,9 +591,9 @@ The module's default export (or its `main` export) is called with a context and
 
 A program that throws, or leaves an error or a rejected promise unhandled with no listener for it, ends with status 1 and the message on stderr; one whose module cannot be loaded, or exports no function, ends with 126. A write to a pipe whose reader is gone ends the program with 141, unless it ignores or catches `SIGPIPE` (an ignore inherited from its parent counts), in which case the write fails with `EPIPE`. Errors carry a POSIX `code`. Reads and writes of pipes, terminals and sockets go through the kernel's asynchronous operations, so a program blocked reading stdin can still write and handle signals; other calls of one process are made one at a time. A JS program cannot `fork`, and threads it starts itself are not the kernel's.
 
-The module has the trust of an Emscripten program's glue or a WASI program's imports module: it comes from a package that was installed, and it runs in the worker of its own process. The kernel withholds the worker's own ways out before it loads the module: `XMLHttpRequest`, `WebSocketStream`, `WebTransport`, `EventSource`, `indexedDB`, `caches`, `cookieStore`, `SharedWorker`, `close`, Chrome's old file system API (`webkitRequestFileSystem`, `webkitRequestFileSystemSync`, `webkitResolveLocalFileSystemURL` and `webkitResolveLocalFileSystemSyncURL`, which reach the origin's OPFS), and `navigator.storage`, `navigator.storageBuckets`, `navigator.locks`, `navigator.serviceWorker`, `navigator.usb`, `navigator.hid` and `navigator.serial` are `undefined`, also through their prototypes, and a worker where one of them cannot be removed refuses to run the program (126). `fetch`, `WebSocket`, `BroadcastChannel`, `importScripts`, `Worker` and `postMessage` are `undefined` too, but stay writable and configurable, so a program can install its own over `ctx.fetch` and `ctx.websocket`, a `BroadcastChannel` within the program (as slicc-jsh does), or the globals napi-rs's wasm32-wasi worker template assigns (`Object.assign(globalThis, { self, require, Worker, importScripts, postMessage })`); the worker's own are gone before the program runs, so deleting the program's brings back `undefined`. `self` stays the global, but as an own writable slot of the scope, so a program can assign its own (the template does) without reaching the worker's. The platform's `BroadcastChannel`, which would reach the origin's other contexts around the kernel, is gone from the scope and its prototypes before the program runs, so deleting the program's own brings back `undefined`, not it; a worker where it can't be removed refuses to run the program (126), as for the others. A program's files are the kernel's, through the context; it reaches the kernel only through its own process's system calls, as the context makes them, and no other process's. This is a guard against programs going around the kernel, not a sandbox: `import()` of a URL is a request the kernel cannot withhold. In a browser, every process worker gets the same lockdown before any program code runs, so an Emscripten program's glue and a WASI program's imports module (any user can write either) see the same globals `undefined`, and a worker where one cannot be removed ends the process with 126. Under the Node entry, every process worker gets the same lockdown before any program code runs: an Emscripten program's glue, a WASI program's imports module and a package driver all see `require`, `process`, `module`, `__dirname` and every Node builtin `undefined`. An Emscripten program's worker is also shaped like a dedicated worker's, as it is in a browser, with `WorkerGlobalScope`, `self` (the global) and `location.href` (the glue's own path in the kernel as a `file:` URL, never the host's install path) as writable slots, so a glue built with `-sENVIRONMENT=web,worker,node` and assertions (`emcc -O0`) reads it as a worker, not a JS shell (#1238); JS and WASI programs don't get them (`require`, `module`, `exports`, `__filename` and `__dirname` writable and configurable, as `process` is, so a program can define its own, never Node's), and `import` of a builtin or a non-`data:` URL fails with `ERR_ACCESS_DENIED`.
+The module has the trust of an Emscripten program's glue or a WASI program's imports module: it comes from a package that was installed, and it runs in the worker of its own process. The kernel withholds the worker's own ways out before it loads the module: `XMLHttpRequest`, `WebSocketStream`, `WebTransport`, `EventSource`, `indexedDB`, `caches`, `cookieStore`, `SharedWorker`, `close`, Chrome's old file system API (`webkitRequestFileSystem`, `webkitRequestFileSystemSync`, `webkitResolveLocalFileSystemURL` and `webkitResolveLocalFileSystemSyncURL`, which reach the origin's OPFS), and `navigator.storage`, `navigator.storageBuckets`, `navigator.locks`, `navigator.serviceWorker`, `navigator.usb`, `navigator.hid` and `navigator.serial` are `undefined`, also through their prototypes, and a worker where one of them cannot be removed refuses to run the program (126). `fetch`, `WebSocket`, `BroadcastChannel`, `importScripts`, `Worker` and `postMessage` are `undefined` too, but stay writable and configurable, so a program can install its own over `ctx.fetch` and `ctx.websocket`, a `BroadcastChannel` within the program (as slicc-jsh does), or the globals napi-rs's wasm32-wasi worker template assigns (`Object.assign(globalThis, { self, require, Worker, importScripts, postMessage })`); the worker's own are gone before the program runs, so deleting the program's brings back `undefined`. `self` stays the global, but as an own writable slot of the scope, so a program can assign its own (the template does) without reaching the worker's. The platform's `BroadcastChannel`, which would reach the origin's other contexts around the kernel, is gone from the scope and its prototypes before the program runs, so deleting the program's own brings back `undefined`, not it; a worker where it can't be removed refuses to run the program (126), as for the others. A program's files are the kernel's, through the context; it reaches the kernel only through its own process's system calls, as the context makes them, and no other process's. This is a guard against programs going around the kernel, not a sandbox: `import()` of a URL is a request the kernel cannot withhold. In a browser, every process worker gets the same lockdown before any program code runs, so an Emscripten program's glue and a WASI program's imports module (any user can write either) see the same globals `undefined`, and a worker where one cannot be removed ends the process with 126. Under the Node entry, every process worker gets the same lockdown before any program code runs: an Emscripten program's glue, a WASI program's imports module and a package driver all see `require`, `process`, `module`, `__dirname` and every Node builtin `undefined`. An Emscripten program's worker is also shaped like a dedicated worker's, as it is in a browser, with `WorkerGlobalScope`, `self` (the global) and `location.href` (the glue's own path in the kernel as a `file:` URL, never the host's install path) as writable slots, so a glue built with `-sENVIRONMENT=web,worker,node` and assertions (`emcc -O0`) reads it as a worker, not a JS shell (#1238); JS and WASI programs don't get them (`require`, `module`, `exports`, `__filename` and `__dirname` writable and configurable, as `process` is, so a program can define its own, never Node's), and `import` of a builtin or a non-`data:` URL fails with `ERR_ACCESS_DENIED`, as does any `slicc-module:` id other than the one the kernel handed that worker.
 
-Under the Node entry, the kernel also withholds the host's Node from the program before its module loads ([#297](https://github.com/ai-ecoverse/slicc-kernel/issues/297)). `localStorage` and `sessionStorage` are `undefined` for good. `require`, `module`, `exports`, `__filename`, `__dirname`, `process`, `global`, `setImmediate` and `clearImmediate` are `undefined` too, but stay writable and configurable, so a program can install its own (as slicc-jsh and napi-rs's wasi worker template do); Node's are gone before the program runs, so deleting the program's brings back `undefined`. `Buffer` stays Node's own, since Node's `Response`, `Request`, `Blob` and stream bodies read it when they run, and a program can replace it. Process workers share Node's memory allocator, so its `allocUnsafe` and `allocUnsafeSlow` zero-fill and its pool is off (`Buffer.poolSize` is 0): a program never sees memory another process freed. A host can also start Node with `--zero-fill-buffers`. Imports of Node's builtin modules (`node:fs`, `fs`, `node:worker_threads`, …) and of anything but a `data:` URL, `file:` included, fail with `ERR_ACCESS_DENIED`, whether static, dynamic or through `import.meta.resolve`. A process worker that exits or crashes on its own ends its process with 70. This is the same guard, not a sandbox: a host that runs code it does not trust, such as an agent's, needs its own isolation around the Node process as well.
+Under the Node entry, the kernel also withholds the host's Node from the program before its module loads ([#297](https://github.com/ai-ecoverse/slicc-kernel/issues/297)). `localStorage` and `sessionStorage` are `undefined` for good. `require`, `module`, `exports`, `__filename`, `__dirname`, `process`, `global`, `setImmediate` and `clearImmediate` are `undefined` too, but stay writable and configurable, so a program can install its own (as slicc-jsh and napi-rs's wasi worker template do); Node's are gone before the program runs, so deleting the program's brings back `undefined`. `Buffer` stays Node's own, since Node's `Response`, `Request`, `Blob` and stream bodies read it when they run, and a program can replace it. Process workers share Node's memory allocator, so its `allocUnsafe` and `allocUnsafeSlow` zero-fill and its pool is off (`Buffer.poolSize` is 0): a program never sees memory another process freed. A host can also start Node with `--zero-fill-buffers`. Imports of Node's builtin modules (`node:fs`, `fs`, `node:worker_threads`, …) and of anything but a `data:` URL or the worker's own `slicc-module:` id, `file:` included, fail with `ERR_ACCESS_DENIED`, whether static, dynamic or through `import.meta.resolve`. A process worker that exits or crashes on its own ends its process with 70. This is the same guard, not a sandbox: a host that runs code it does not trust, such as an agent's, needs its own isolation around the Node process as well.
 
 ## Network
 

package/dist/kernel-worker.js

@@ -8236,7 +8236,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.160.1" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.160.3" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -31822,7 +31822,7 @@ var Launcher = class {
       this.moduleUrls.delete(source);
       this.moduleUrls.set(source, known);
       known.users++;
-      return known.made.url;
+      return known.made;
     }
     const held = {
       made: (this.moduleUrl ?? dataModuleUrl)(source),
@@ -31837,7 +31837,7 @@ var Launcher = class {
       entry.evicted = true;
       this.releaseUrl(entry);
     }
-    return held.made.url;
+    return held.made;
   }
   heldUrl(program) {
     const held = program.abi === "js" && program.url ? this.urlsInUse.get(program.url) : void 0;
@@ -31918,8 +31918,8 @@ var Launcher = class {
     }
     const { asked } = target;
     if (target.abi === "js") {
-      const url = this.urlFor(await this.text(target.wasm, caller, asked));
-      return { abi: "js", glue: "", url, path: target.wasm };
+      const { url, source } = this.urlFor(await this.text(target.wasm, caller, asked));
+      return { abi: "js", glue: "", url, path: target.wasm, ...source ? { source } : {} };
     }
     if (target.abi !== "wasi") {
       const [glue, { module: module2 }] = await Promise.all([