@ai-ecoverse/slicc-kernel 1.143.3 → 1.143.4

package/README.md

@@ -202,11 +202,11 @@ const { status, stdout } = await kernel.run(['bash', '-c', 'echo hi'], { cwd: '/
 kernel.terminate();
 ```
 
-`createNodeKernel({ root, modules, commands, env, network, worker, facades, processMounts, fstabRetries, cdp, hostfs, hostfsOrigin, hostfsFetch, users, onDenial, approve, compileCache, warmWorker })` takes the options of `createKernel` except `metadata`, and `fstabRetries`, the delays in ms between tries of a failing `/etc/fstab` line (default `[1000, 4000, 16000]`). `root` is an in-memory directory by default (`memoryRoot()` makes another), which keeps each file in 1 MiB chunks, and POSIX metadata stays in memory. Processes and threads run on `worker_threads`. The kernel has `run`, `openTerminal`, `users`, `setRoutes`, `updateTransportTraits` and `terminate` as above, plus `integrity.revoke(path?)`, `root`, `writeFile(path, data, { mode }?)` (creating the parent directories; `mode` sets the permission bits, `0` to `0o7777`, before the data is written), `chmod(path, mode)` and `readFile(path)` to put files in place and read results; like the `fs` of an attached client, they go through the mounts, as the processes see them. `nodeTransport()` is `fetchTransport()` with Node's `fetch`, which no CORS binds (`crossOrigin: 'any'`). `connect({ user })` and `attachKernel` work as in the browser: the port is a `worker_threads` `MessagePort`, which a worker thread can attach with, and `terminate()` ends every attached client.
+`createNodeKernel({ root, modules, commands, env, network, worker, facades, processMounts, fstabRetries, cdp, hostfs, hostfsOrigin, hostfsFetch, users, onDenial, approve, compileCache, warmWorker, warmWorkers })` takes the options of `createKernel` except `metadata`, and `fstabRetries`, the delays in ms between tries of a failing `/etc/fstab` line (default `[1000, 4000, 16000]`). `root` is an in-memory directory by default (`memoryRoot()` makes another), which keeps each file in 1 MiB chunks, and POSIX metadata stays in memory. Processes and threads run on `worker_threads`. The kernel has `run`, `openTerminal`, `users`, `setRoutes`, `updateTransportTraits` and `terminate` as above, plus `integrity.revoke(path?)`, `root`, `writeFile(path, data, { mode }?)` (creating the parent directories; `mode` sets the permission bits, `0` to `0o7777`, before the data is written), `chmod(path, mode)` and `readFile(path)` to put files in place and read results; like the `fs` of an attached client, they go through the mounts, as the processes see them. `nodeTransport()` is `fetchTransport()` with Node's `fetch`, which no CORS binds (`crossOrigin: 'any'`). `connect({ user })` and `attachKernel` work as in the browser: the port is a `worker_threads` `MessagePort`, which a worker thread can attach with, and `terminate()` ends every attached client.
 
 `compileCache` (opt-in, off by default; #1285, slicc-jsh#566) is an absolute host directory where each process worker keeps the compiled code of the JS programs it imports (`module.enableCompileCache`), so the next spawn of the same program skips V8's compile. It stores V8's code cache, keyed and checksummed by the module's source, one file per program (about 7 MB for a 2.4 MB bundle), shared by every kernel given the same directory; programs can't reach or write it (the worker hands it to Node before the lockdown, and `node:worker_threads` stays refused). Measured on a 2.4 MB bundle, a spawn went from about 54 to 39 ms; a trivial program costs about 1 ms more. **The directory must be private to the embedder:** V8's code cache is trusted bytecode that Node only checksums, so whoever can write it can plant code that runs inside a process worker past the lockdown. Never make it reachable from inside the kernel (no `hostfs` mount of it or of a parent), and don't share one between kernels of different trust. A host's own `NODE_COMPILE_CACHE` is never handed to the process workers, with or without the option, so programs never land in a host-wide cache. If a host preload (`--import` or `--require`) has already enabled another compile cache in the workers, a kernel given `compileCache` refuses to run programs there instead of caching them in the host's directory.
 
-`warmWorker` (on by default; `false` turns it off; #1341, slicc-jsh#566) keeps one process worker booted ahead of the next spawn, fork or thread on the Node entry, so the spawn skips the worker's boot: it has loaded the process worker module and nothing else, no pid, credentials, environment, cwd, descriptors, lockdown or program, all of which arrive with its init message, and every worker still serves exactly one process or thread and is then terminated. Its own `process.env` is a snapshot taken when the spare is created rather than at the spawn, but a program never sees it (`process` is withheld, and a program's environment is the one its spawn gives it). The spare never keeps the host alive, is replaced after each spawn off the critical path, is dropped if it errors or exits while idle, and is released after 30 s without a spawn; `terminate()` ends it. It costs about 11 MB of host RSS while it waits.
+`warmWorker` (on by default; `false` turns it off; #1341, slicc-jsh#566) keeps one process worker booted ahead of the next spawn, fork or thread on the Node entry, so the spawn skips the worker's boot: it has loaded the process worker module and nothing else, no pid, credentials, environment, cwd, descriptors, lockdown or program, all of which arrive with its init message, and every worker still serves exactly one process or thread and is then terminated. Its own `process.env` is a snapshot taken when the spare is created rather than at the spawn, but a program never sees it (`process` is withheld, and a program's environment is the one its spawn gives it). The spare never keeps the host alive, is replaced after each spawn off the critical path, is dropped if it errors or exits while idle, and is released after 30 s without a spawn; `terminate()` ends it. It costs about 11 MB of host RSS while it waits. `warmWorkers: n` (opt-in; #1361, slicc-jsh#566) keeps up to `n` spares instead of one (an integer from 0 to 64; `warmWorker: false` still means none), so a burst of starts, as worker pools make, finds a spare for each: all are refilled after each take off the critical path, and all are released together after 30 s without a spawn and ended by `terminate()`. Each costs about 7–11 MB of host RSS while it waits. It pays off for bursts of `Worker` threads (jsh measured about 13 ms less per start in a burst of 8); for bursts of child processes that boot a large runtime of their own, the refills compete for CPU with the children and the gain is small or none.
 
 By default `nodeTransport()` refuses destinations on the host itself, because Node can reach them and a page can't: local DevTools on 9222–9229, dev servers, local admin APIs, and the cloud metadata address.
 - **What it refuses:** loopback (`127.0.0.0/8`, `::1`, and their IPv4-mapped forms), unspecified addresses (`0.0.0.0/8`, `::`), `localhost` and `*.localhost`, and link-local addresses (`169.254.0.0/16`, `fe80::/10`), plus AWS's IPv6 metadata address `fd00:ec2::254`.
@@ -370,9 +370,10 @@ The module's default export (or its `main` export) is called with a context and
 - `fs`: `stat`, `lstat`, `exists`, `readdir`, `mkdir` (with parents, or, with `{ recursive: false }`, as Node's `fs.mkdir` without `recursive`: `EEXIST` for an existing entry and `ENOENT` for a missing parent, atomically among the processes of one kernel, so it can serve as a lock there, but not between two kernels on one origin, such as two tabs ([#519](https://github.com/ai-ecoverse/slicc-kernel/issues/519)); `sync.fs.mkdir` takes the same option), `rm` (recursive), `unlink`, `rename`, `symlink`, `readlink`, `statfs`, `chmod(path, mode)`, and `readFile`/`writeFile` for small files. `chmod` takes the permission bits (`0` to `0o7777`, `EINVAL` otherwise) and follows a symlink; as `chmod(2)`, only the file's owner or root may (`EPERM`), so a program can make a script it wrote runnable by path. `sync.fs.chmod` is the same, synchronously. `statfs(path)` answers `{ type, bsize, blocks, bfree, bavail, files, ffree, quota, usage }`: the fields of Node's `StatFs`, plus `quota` and `usage` in bytes (what jsh 0.17 reads). The values come from the same source as an Emscripten program's `statfs`/`df`: the driver of the mount the path is on, or, on the root file system, the origin's storage estimate (`navigator.storage.estimate()`). `bsize` is 4096, `bfree` and `bavail` are what's left of the quota, and `type`, `files` and `ffree` are 0, since the kernel doesn't know them. Where there's no estimate (the Node entry's root), every count is 0. A missing path is `ENOENT`. `sync.fs` has it too. `rm` removes a whole tree in one kernel call (on the OPFS root, one recursive removal and one metadata delete: 50,000 files take about 3 s in Chrome, where `rm -r` takes about 25 s) with the result of a file-by-file removal: symlinks inside the tree are removed, never followed; a tree holding a protected path is refused with `EROFS` and nothing is removed; a mount inside the tree is emptied and its mount point answers `EBUSY`, while the rest is removed; under `users`, what the user may remove is removed and the call then fails with the first `EACCES`/`EPERM`; files a process holds open stay readable to it;
 - **`fs.watch(path, options?, onEvent)`** resolves with `{ close() }` ([#1031](https://github.com/ai-ecoverse/slicc-kernel/issues/1031)). `sync.fs.watch` returns the watcher directly and is in place when it returns, so it sees every change made after the call. This is the shape for Node's synchronous `fs.watch`.
   - **What it reports:** a file reports its own changes. A directory reports its direct entries, or with `{ recursive: true }`, everything below it.
-  - **Events:** each one is `{ type, filename }`, in the order the operations happened.
+  - **Events:** each one is `{ type, filename, seq }`, in the order the operations happened.
     - `'rename'`: an entry appeared or went away (create, `mkdir`, `unlink`, `rm`, both names of a `rename`, `link`, `symlink`).
-    - `'change'`: an existing entry's data or metadata changed (a write, `truncate`, `chmod`, `chown`, `utimes`). A write that creates a file gives `'rename'` and then `'change'`, as on Linux.
+    - `'change'`: an existing entry's data or metadata changed (a write, `truncate`, `chmod`, `chown`, `utimes`). A write that creates a file gives `'rename'` and then `'change'`, as on Linux. Writing a whole file over an existing one is a truncate and a write, so it gives two `'change'`s, or one if the new content is empty ([#1357](https://github.com/ai-ecoverse/slicc-kernel/issues/1357)).
+    - `seq` numbers the operations across the whole kernel, increasing. One operation heard by several watches (a directory's and a file's in it) has the same `seq` in each, so a program with several watches can merge their events back into one queue in operation order and, as inotify does, merge only events that are adjacent in it. `{ overflow: true }` and `{ ended: true }` carry the `seq` of the operation that caused them.
     - `filename` is the entry's name relative to the watched directory (`sub/a.txt` when recursive), or the file's basename for a file watch.
   - **Overflow:** a watch holds 1,000 events between reads. Beyond that, the rest are dropped, and `{ overflow: true }` follows the ones kept.
   - **The watched path itself:** like inotify, a watch follows the file or directory it was opened on, not its name. When a directory above it is renamed, the watch stays on the moved path with no event, and later events keep the name it was opened with. When the path itself is renamed, the watch gets one `{ type: 'rename', filename, moved: true }` and stays on the new path. Only a removal ends it: `rm` of the path or a directory above it, or a rename of something else over it gives one `'rename'` with its own name, then `{ ended: true }`. Renames are followed as this kernel makes them ([#1356](https://github.com/ai-ecoverse/slicc-kernel/issues/1356)). `close()` is safe at any time and idempotent, and nothing is delivered after it returns.

package/dist/kernel-worker.js

@@ -7598,7 +7598,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.143.3" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.143.4" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -10500,22 +10500,22 @@ function noted(watch, note) {
   if (watch.overflowed) return;
   if (watch.notes.length >= WATCH_NOTES_LIMIT) {
     watch.overflowed = true;
-    watch.notes.push({ overflow: true });
+    watch.notes.push({ overflow: true, seq: note.seq });
   } else watch.notes.push(note);
   watch.wake?.();
 }
-function route(watch, type, path) {
-  for (const root of watch.roots) if (routed(watch, root, type, path)) return;
+function route(watch, type, path, seq2) {
+  for (const root of watch.roots) if (routed(watch, root, type, path, seq2)) return;
 }
-function routed(watch, root, type, path) {
+function routed(watch, root, type, path, seq2) {
   if (path === root || type === "rename" && root !== "/" && under3(root, path)) {
-    noted(watch, { type, filename: watch.names.get(root) ?? baseOf3(root) });
-    if (type === "rename") noted(watch, { ended: true });
+    noted(watch, { type, filename: watch.names.get(root) ?? baseOf3(root), seq: seq2 });
+    if (type === "rename") noted(watch, { ended: true, seq: seq2 });
     return true;
   }
-  if (parentOf7(path) === root) noted(watch, { type, filename: baseOf3(path) });
+  if (parentOf7(path) === root) noted(watch, { type, filename: baseOf3(path), seq: seq2 });
   else if (watch.recursive && under3(path, root)) {
-    noted(watch, { type, filename: path.slice(root === "/" ? 1 : root.length + 1) });
+    noted(watch, { type, filename: path.slice(root === "/" ? 1 : root.length + 1), seq: seq2 });
   } else return false;
   return true;
 }
@@ -10529,6 +10529,7 @@ var FsWatchers = class {
   typed = /* @__PURE__ */ new Set();
   taps = [];
   reals = /* @__PURE__ */ new Map();
+  seq = 0;
   reshapes = [];
   onReshape(listener) {
     this.reshapes.push(listener);
@@ -10588,7 +10589,7 @@ var FsWatchers = class {
         const notes = watch.notes.splice(0);
         watch.overflowed = false;
         if (watch.ended) this.typed.delete(watch);
-        return notes.length === 0 && watch.ended ? [{ ended: true }] : notes;
+        return notes.length === 0 && watch.ended ? [{ ended: true, seq: this.seq }] : notes;
       }
     };
   }
@@ -10605,10 +10606,11 @@ var FsWatchers = class {
     if (this.typed.size === 0) return;
     for (const path of paths) {
       const at = normalizePath(path);
-      for (const watch of [...this.typed]) route(watch, type, at);
+      const seq2 = ++this.seq;
+      for (const watch of [...this.typed]) route(watch, type, at, seq2);
     }
   }
-  moved(from, to) {
+  moved(from, to, seq2 = ++this.seq) {
     const source = normalizePath(from);
     const target = normalizePath(to);
     const followed2 = /* @__PURE__ */ new Set();
@@ -10624,17 +10626,25 @@ var FsWatchers = class {
       watch.names = names2;
       watch.roots = [...new Set(roots)];
       if (main === source) {
-        noted(watch, { type: "rename", filename: names2.get(target), moved: true });
+        noted(watch, {
+          type: "rename",
+          filename: names2.get(target),
+          moved: true,
+          seq: seq2
+        });
       }
     }
     return followed2;
   }
   renamed(from, to) {
-    const followed2 = this.moved(from, to);
+    const first = ++this.seq;
+    const followed2 = this.moved(from, to, first);
     if (this.typed.size === 0) return;
-    for (const path of [from, to]) {
+    for (const [i, path] of [from, to].entries()) {
       const at = normalizePath(path);
-      for (const watch of [...this.typed]) if (!followed2.has(watch)) route(watch, "rename", at);
+      const seq2 = i === 0 ? first : ++this.seq;
+      for (const watch of [...this.typed])
+        if (!followed2.has(watch)) route(watch, "rename", at, seq2);
     }
   }
   changed(...paths) {
@@ -10699,10 +10709,11 @@ var FsWatchers = class {
         const real = await realOf(at);
         const paths = real === at ? [at] : [at, real];
         this.report(paths);
-        if (done5) for (const type of types(fresh2)) this.emit(type, paths);
+        if (done5) for (const type of types(fresh2, args)) this.emit(type, paths);
       }
     };
     const written2 = (fresh2) => fresh2 ? ["rename", "change"] : ["change"];
+    const overwritten = (fresh2, args) => fresh2 || args[1].length === 0 ? written2(fresh2) : ["change", "change"];
     const reshaping = (run2) => async (...args) => {
       this.reshaped();
       try {
@@ -10712,7 +10723,7 @@ var FsWatchers = class {
       }
     };
     const overrides = {
-      writeFile: content(fs.writeFile.bind(fs), written2),
+      writeFile: content(fs.writeFile.bind(fs), overwritten),
       ...fs.createExclusive ? { createExclusive: content(fs.createExclusive.bind(fs), () => ["rename"]) } : {},
       ...fs.copyFile ? {
         copyFile: (from, to, options) => content(