@ai-ecoverse/slicc-kernel 1.133.3 → 1.133.4

package/README.md

@@ -709,7 +709,7 @@ The proxy answers `200` with `Content-Type: application/vnd.slicc.raw-fetch`: a
 
 `/` is the OPFS root. Each process mounts the top-level directories that exist when it starts (plus `/usr` and `/bin`), so files in them are shared by all processes and visible through the OPFS API as soon as the process that wrote them has closed them; `run` resolves only after that. `createKernel` creates `/tmp` and `/home` in OPFS, so they are shared too. Files directly in `/`, and directories created there after a process started, are OPFS as well, the same for Emscripten and WASI programs; only `/dev` and `/proc` live in each process's memory. A program's `mkdir` (WASI `path_create_directory`, Emscripten's `mkdir`) is exclusive, as on Linux: it fails with `EEXIST` when the name exists, and of several processes or threads making one name at once exactly one succeeds, so a directory works as a lock. Under a missing directory it fails with `ENOENT`, as on Linux (`mkdir -p` and `os.makedirs` make the parents in the program); a JS program's `ctx.fs.mkdir` makes parents itself. The kernel makes the system `PNPM_HOME` (`/usr/local/share/pnpm`, or the embedder's) when it starts, since pnpm makes only its `bin` directory there. A full origin storage (`QuotaExceededError`, which also comes early in a browser's private or off-the-record contexts, whatever `navigator.storage.estimate()` says) is `ENOSPC`, never retried. A read that meets `NotFoundError` while a writer outside the kernel swaps the file in with `createWritable` (and a `stat`, `lstat` or `utimes` that meets it) looks the name up again, up to three times, before it fails ([#1075](https://github.com/ai-ecoverse/slicc-kernel/issues/1075)), so a program whose package is overwritten in place keeps starting. An OPFS or IndexedDB failure that has no POSIX counterpart is `EIO`; its message names the DOM error and the operation (`lookup`, `read`, `write`, `commit` or `sidecar`), and the kernel worker logs the first of each kind to its console. File contents are buffered per open file and written back on close, `fsync` and exit, so a refused write (`ENOSPC`, `EIO`) surfaces at that close or `fsync`, and a program that ignores close errors doesn't see it (`echo x > f` can exit 0 with `f` empty, as with any buffered write-back). A write-back error a close has already reported to a WASI program isn't raised again when it exits, so the program keeps its own exit status. Metadata operations (`mkdir`, `rename`, `rm`, …) go straight to OPFS. The kernel worker is the only writer.
 
-**Renaming a directory on the OPFS root** ([#1122](https://github.com/ai-ecoverse/slicc-kernel/issues/1122)): Chrome's OPFS can move a file but not a directory. So the kernel renames a directory by creating the new tree and moving each file into it (copying a file only if it can't move, e.g. while it's open elsewhere), then removing the old tree. Within the kernel the rename is atomic to readers: a lookup or listing under the old or the new name, or of either parent, waits until the whole tree has moved. So a stage-then-rename updater (pnpm's, an atomic config write) is never seen half there. Another tab sharing the root reads OPFS directly and can see the tree mid-move. A root whose directories can move (the Node memory root) renames them in one step. **If the page reloads or crashes mid-rename**, some files are already in the new tree and the rest are still in the old one. So before the first move, the kernel writes the rename's intent (`{ from, to }`) to its metadata store, and clears it after the last. At the next boot, when no other tab shares the root (the same rule as for the boot's other repair of stale metadata), the kernel finishes every recorded rename: it moves what's left into the new tree, removes the old name, and clears the record. While another tab shares the root, the split tree stays as it is (both names visible, each with part of the files) until a boot that's alone.
+**Renaming a directory on the OPFS root** ([#1122](https://github.com/ai-ecoverse/slicc-kernel/issues/1122)): Chrome's OPFS can move a file but not a directory. So the kernel renames a directory by creating the new tree and moving each file into it (copying a file only if it can't move, e.g. while it's open elsewhere), then removing the old tree. Within the kernel the rename is atomic to readers: a lookup or listing under the old or the new name, or of either parent, waits until the whole tree has moved. So a stage-then-rename updater (pnpm's, an atomic config write) is never seen half there. Another tab sharing the root reads OPFS directly and can see the tree mid-move. A root whose directories can move (the Node memory root) renames them in one step. **If the page reloads or crashes mid-rename**, some files are already in the new tree and the rest are still in the old one. So before the first move, the kernel writes the rename's intent (`{ from, to }`) to its metadata store, and clears it after the last. At the next boot, when no other tab shares the root (the same rule as for the boot's other repair of stale metadata), the kernel finishes every recorded rename: it moves what's left into the new tree, removes the old name, and clears the record. The moved directory and its files keep their owner, mode and inode, as after a rename that wasn't stopped. While another tab shares the root, a kernel finishes only renames whose kernel is gone ([#1141](https://github.com/ai-ecoverse/slicc-kernel/issues/1141)): the renaming kernel holds a Web Lock (`slicc-rename:<id>`, named in the record) for the whole rename, and the browser releases it when that tab closes or its worker ends. A kernel that boots beside a peer, and every kernel that sees a peer go, finishes each recorded rename whose lock is free, taking that lock while it does, so two survivors never replay the same rename and a live kernel's rename is never touched. A record without a lock (written by an older kernel, or where Web Locks are missing) waits for a boot that's alone, as before. While another tab finishes a rename, this kernel's readers wait as for its own; the other tabs can see the tree mid-move, as during any rename.
 
 A lookup follows at most 40 symlinks, as Linux does: the 41st is `ELOOP`, for WASI programs, JS programs and the kernel API alike, with or without `..` in a link's target. Emscripten programs resolve paths in their glue, which stops one link earlier, at 40.
 

package/dist/kernel-worker.js

@@ -900,6 +900,7 @@ var RemoteWatch = class {
   settled = false;
   quiet;
   crowd;
+  gone;
   told;
   counted = 0;
   recount() {
@@ -989,6 +990,7 @@ var RemoteWatch = class {
       this.peers.delete(peer);
       this.rebuild();
       this.recount();
+      this.gone?.();
       for (const wait2 of this.waiting.values()) {
         if (wait2.left.delete(peer) && wait2.left.size === 0) wait2.done();
       }
@@ -1024,14 +1026,16 @@ var RemoteWatch = class {
   async prune() {
     if (!this.locks) return;
     const held = new Set(((await this.locks.query()).held ?? []).map((lock) => lock.name));
+    let left = false;
     for (const tab of /* @__PURE__ */ new Set([...this.others.keys(), ...this.peers])) {
       if (!held.has(this.lockOf(tab))) {
+        if (this.peers.delete(tab)) left = true;
         this.others.delete(tab);
-        this.peers.delete(tab);
       }
     }
     this.rebuild();
     this.recount();
+    if (left) this.gone?.();
   }
   watched() {
     return this.union.size > 0;
@@ -1121,6 +1125,7 @@ var REPLACED_ATTEMPTS = 4;
 var RECONCILE_LISTINGS = 8;
 var EXECUTABLE = /\/node_modules\/(?:@[^/]+\/)?[^/]+\/bin\//;
 var RENAMES = "/.slicc-renames";
+var RENAME_LOCK = "slicc-rename:";
 function movable(handle) {
   return typeof handle.move === "function";
 }
@@ -1287,8 +1292,11 @@ var OpfsFs = class {
     this.lockPrefix = `slicc-excl:${channel ?? ""}:`;
     this.rootName = channel;
     this.channel = channel === void 0 ? void 0 : new BroadcastChannel(channel);
-    this.remote = channel === void 0 ? void 0 : new RemoteWatch(`${channel}:watch`);
-    if (this.remote) this.remote.quiet = () => this.quiet().then(() => this.settle?.());
+    this.remote = channel === void 0 ? void 0 : new RemoteWatch(`${channel}:watch`, { locks: this.locks ?? null });
+    if (this.remote) {
+      this.remote.quiet = () => this.quiet().then(() => this.settle?.());
+      this.remote.gone = () => void this.orphans();
+    }
     if (this.channel) {
       const remote = this.remote;
       meta.listen?.((c) => {
@@ -2361,14 +2369,17 @@ var OpfsFs = class {
     const dir = parent.handle;
     const name2 = baseOf(destination);
     const swap = targetStat?.isDirectory === true;
+    let lock;
     const staging = swap ? `.${name2}.${crypto.randomUUID()}` : name2;
     const copying = sourceStat.isDirectory && !movable(source.handle);
     const staged = resolvePath(parent.path, staging);
     const place = async () => {
       this.forget(destination);
       const timed2 = await this.explicitTimes(source.path);
-      const first = copying ? await this.intend({ from: source.path, to: staged }) : void 0;
-      const second = copying && swap ? await this.intend({ from: staged, to: destination, replace: true }) : void 0;
+      const root = { ino: inodeOf(source.path), ...moving };
+      const owned4 = lock ? { lock } : {};
+      const first = copying ? await this.intend({ from: source.path, to: staged, root, ...owned4 }) : void 0;
+      const second = copying && swap ? await this.intend({ from: staged, to: destination, replace: true, root, ...owned4 }) : void 0;
       const moved2 = await this.replacingTarget(
         targetStat !== null,
         () => this.place(source.handle, dir, staging)
@@ -2377,7 +2388,7 @@ var OpfsFs = class {
         throw err;
       });
       if (swap) await this.rm(destination, { recursive: true });
-      if (second) await this.intend({ from: staged, to: destination }, second);
+      if (second) await this.intend({ from: staged, to: destination, root, ...owned4 }, second);
       await this.meta.move(source.path, destination, carry, left);
       this.movePending(source.path, destination);
       if (!moved2) {
@@ -2402,7 +2413,11 @@ var OpfsFs = class {
     const entry = { from: source.path, to: destination, done: promise };
     this.renamingDirs.add(entry);
     try {
-      await place();
+      const locks = this.locks;
+      if (locks) {
+        lock = `${RENAME_LOCK}${crypto.randomUUID()}`;
+        await locks.request(lock, place);
+      } else await place();
     } finally {
       this.renamingDirs.delete(entry);
       resolve2();
@@ -2434,12 +2449,40 @@ var OpfsFs = class {
     } catch {
     }
   }
-  async finishRenames() {
-    const intents = (await this.meta.under(RENAMES)).filter((entry) => entry.rename !== void 0).sort(
+  replaying = Promise.resolve();
+  orphans() {
+    this.replaying = this.replaying.then(() => this.finishRenames(true)).catch(() => 0);
+    return this.replaying;
+  }
+  async finishRenames(orphaned = false) {
+    const locks = this.locks;
+    const held = /* @__PURE__ */ new Set();
+    if (orphaned && locks) {
+      const { held: h = [], pending = [] } = await locks.query();
+      for (const lock of [...h, ...pending]) held.add(lock.name);
+    }
+    const intents = (await this.meta.under(RENAMES)).filter(
+      (entry) => entry.rename !== void 0 && (!orphaned || locks !== void 0 && entry.rename.lock !== void 0 && !held.has(entry.rename.lock))
+    ).sort(
       (a, b) => a.rename.at - b.rename.at
     );
     for (const entry of intents) {
-      const { from, to, replace } = entry.rename;
+      const lock = entry.rename.lock;
+      if (!orphaned || !locks || !lock) await this.finishRename(entry);
+      else await locks.request(lock, () => this.finishRename(entry));
+    }
+    return intents.length;
+  }
+  async finishRename(entry) {
+    const [current] = await this.meta.under(entry.path);
+    if (current?.rename === void 0) return;
+    const { from, to, replace, root } = current.rename;
+    const { promise, resolve: resolve2 } = Promise.withResolvers();
+    const barrier = { from, to, done: promise };
+    this.renamingDirs.add(barrier);
+    this.forget(from);
+    this.forget(to);
+    try {
       const left = await this.real(from);
       if (left && replace && await this.real(to)) await this.rm(to, { recursive: true });
       if (left) {
@@ -2456,11 +2499,23 @@ var OpfsFs = class {
         this.forget(to);
       }
       if ((await this.meta.under(from)).length > 0) {
-        await this.meta.move(from, to, (moved2) => ({ ...occupied(moved2), path: to }));
+        await this.meta.move(
+          from,
+          to,
+          (moved2) => ({
+            ...occupied(moved2),
+            ...root,
+            ino: occupied(moved2).ino ?? root?.ino ?? inodeOf(from),
+            path: to
+          }),
+          void 0
+        );
       }
       await this.meta.update(entry.path, () => void 0);
+    } finally {
+      this.renamingDirs.delete(barrier);
+      resolve2();
     }
-    return intents.length;
   }
   async settledRename(path) {
     if (this.renamingDirs.size === 0 && this.copying.size === 0 || typeof path !== "string")
@@ -2744,7 +2799,10 @@ var OpfsFs = class {
     };
   }
   async reconcile() {
-    if (this.remote && await this.remote.peered()) return 0;
+    if (this.remote && await this.remote.peered()) {
+      await this.orphans();
+      return 0;
+    }
     await this.finishRenames();
     let removed = 0;
     const entries = await this.meta.all();
@@ -7034,7 +7092,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.133.3" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.133.4" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC