@ai-ecoverse/slicc-kernel 1.146.3 → 1.146.5

package/README.md

@@ -754,6 +754,10 @@ The proxy answers `200` with `Content-Type: application/vnd.slicc.raw-fetch`: a
 
 **Renaming a directory on the OPFS root** ([#1122](https://github.com/ai-ecoverse/slicc-kernel/issues/1122)): Chrome's OPFS can move a file but not a directory. So the kernel renames a directory by creating the new tree and moving each file into it (copying a file only if it can't move, e.g. while it's open elsewhere), then removing the old tree. Within the kernel the rename is atomic to readers: a lookup or listing under the old or the new name, or of either parent, waits until the whole tree has moved. So a stage-then-rename updater (pnpm's, an atomic config write) is never seen half there. Another tab sharing the root sees it all at once too (see below). A root whose directories can move (the Node memory root) renames them in one step. **If the page reloads or crashes mid-rename**, some files are already in the new tree and the rest are still in the old one. So before the first move, the kernel writes the rename's intent (`{ from, to }`) to its metadata store, and clears it after the last. At the next boot, when no other tab shares the root (the same rule as for the boot's other repair of stale metadata), the kernel finishes every recorded rename: it moves what's left into the new tree, removes the old name, and clears the record. The moved directory and its files keep their owner, mode and inode, as after a rename that wasn't stopped. While another tab shares the root, a kernel finishes only renames whose kernel is gone ([#1141](https://github.com/ai-ecoverse/slicc-kernel/issues/1141)): the renaming kernel holds a Web Lock (`slicc-rename:<id>`, named in the record) for the whole rename, and the browser releases it when that tab closes or its worker ends. A kernel that boots beside a peer, and every kernel that sees a peer go, finishes each recorded rename whose lock is free, taking that lock while it does, so two survivors never replay the same rename and a live kernel's rename is never touched. A record without a lock (written by an older kernel, or where Web Locks are missing) waits for a boot that's alone, as before. While another tab finishes a rename, this kernel's readers wait as for its own, and so do the other tabs'. A kernel whose wait for another tab's rename ends with the lock free and the record still there finishes the rename itself, under that lock, and a boot that's alone takes a record's lock too, so a third tab that boots meanwhile waits for it.
 
+**An open file stays bound to the file it opened** ([#1394](https://github.com/ai-ecoverse/slicc-kernel/issues/1394)): as on Linux, a program's open file keeps reaching the file it opened when its name is later renamed over, by a file or a symlink, by the program itself or by another process.
+- **The kernel keeps it:** the program keeps the file's contents before its own rename over the name, as it already did before its own unlink, and a rename by another process sets the held file aside, as an unlink does. Reads and writes through the descriptor reach the old file, and `fstat` keeps its inode.
+- **Another tab:** a tab doesn't know which files another tab holds open, so a rename over the name there can't keep the file. The descriptor then fails with `ESTALE`; it never reads or writes the file the name now names. `/proc/sys/kernel/slicc_stale_fds` counts these.
+
 **A rename by another process** ([#1257](https://github.com/ai-ecoverse/slicc-kernel/issues/1257)): as on Linux, where a descriptor names the file and not its path, a program's open file follows a rename of that file, or of a directory above it, by another process.
 - **The journal:** the kernel keeps the last 4096 renames it applied, each from and to a path, and counts them in the change buffer it shares with its processes. A rename in another tab on the same root reaches it through the sidecar's move notice.
 - **Catching up:** before a WASI or Emscripten program uses an open file's path (a read, a write, a seek, an `fsync`, a close, an open), it reads that count once. If another process renamed something since, and the program holds files open, it asks the kernel once for the renames of the paths it holds and moves its open files to their new names. The kernel answers only about the paths the program sent, and, under `users: 'enforce'`, drops every rename whose old or new name the program couldn't look up (search permission on each directory above it), so no process learns names in a directory it can't search. A program's own rename moves the count without asking, when nothing else came in between.

package/dist/kernel-worker.js

@@ -1376,6 +1376,28 @@ function prefixes(path) {
 function within2(path, root) {
   return path === root || path.startsWith(`${root}/`);
 }
+var unstored = () => Object.assign(new Error("the file system kept fewer bytes than were written"), {
+  code: "ENOSPC"
+});
+var CHECKED_BYTES = 16 * 1024 * 1024;
+var Extent = class {
+  floor = 0;
+  exact;
+  wrote(end) {
+    this.floor = Math.max(this.floor, end);
+    if (this.exact !== void 0) this.exact = Math.max(this.exact, end);
+  }
+  cut(size) {
+    this.exact = size;
+    this.floor = Math.min(this.floor, size);
+  }
+  big() {
+    return Math.max(this.floor, this.exact ?? 0) >= CHECKED_BYTES;
+  }
+  check(size) {
+    if (size < this.floor || this.exact !== void 0 && size < this.exact) throw unstored();
+  }
+};
 var LOCKED = /* @__PURE__ */ new Set(["NotReadableError", "NoModificationAllowedError", "InvalidStateError"]);
 var SYNC_BACKOFF_MS = [5, 10, 20, 40, 80];
 async function openSync(handle) {
@@ -1401,14 +1423,22 @@ function writeAll(access2, at, bytes) {
   }
 }
 async function positioned(handle, op) {
+  const extent = new Extent();
   const access2 = await openSync(handle);
   if (access2) {
     try {
       op({
-        write: (at, bytes) => writeAll(access2, at, bytes),
-        truncate: (n) => access2.truncate(n)
+        write: (at, bytes) => {
+          writeAll(access2, at, bytes);
+          extent.wrote(at + bytes.length);
+        },
+        truncate: (n) => {
+          access2.truncate(n);
+          extent.cut(n);
+        }
       });
       access2.flush();
+      if (access2.getSize) extent.check(access2.getSize());
     } finally {
       access2.close();
     }
@@ -1417,14 +1447,25 @@ async function positioned(handle, op) {
   const writable2 = await handle.createWritable({ keepExistingData: true });
   try {
     await op({
-      write: (at, bytes) => writable2.write({ type: "write", position: at, data: bytes }),
-      truncate: (n) => writable2.truncate(n)
+      write: async (at, bytes) => {
+        await writable2.write({
+          type: "write",
+          position: at,
+          data: bytes
+        });
+        extent.wrote(at + bytes.length);
+      },
+      truncate: async (n) => {
+        await writable2.truncate(n);
+        extent.cut(n);
+      }
     });
   } catch (err) {
     await writable2.abort();
     throw err;
   }
   await writable2.close();
+  if (extent.big()) extent.check((await handle.getFile()).size);
 }
 function syncAccessHandles(scope = globalThis) {
   const handle = scope.FileSystemFileHandle;
@@ -2403,6 +2444,11 @@ var OpfsFs = class {
       writable2 = await this.io(file.createWritable());
       await writable2.write(content);
       await writable2.close();
+      if (typeof content !== "string" && content.length >= CHECKED_BYTES) {
+        const extent = new Extent();
+        extent.cut(content.length);
+        extent.check((await file.getFile()).size);
+      }
     } catch (err) {
       await writable2?.abort().catch(() => void 0);
       throw translate(err, path, "commit", false);
@@ -7728,7 +7774,7 @@ function satisfies(version, range2) {
 }
 
 // src/version.ts
-var KERNEL_VERSION = true ? "1.146.3" : "0.0.0-dev";
+var KERNEL_VERSION = true ? "1.146.5" : "0.0.0-dev";
 var OS_RELEASE = "6.0.0-slicc";
 function procVersion() {
   return `Linux version ${OS_RELEASE} (slicc-kernel ${KERNEL_VERSION}) #1 SMP PREEMPT_DYNAMIC
@@ -10631,16 +10677,16 @@ var FreshModules = class {
         if (size < MAGIC.length && this.fresh.has(normalizePath(path))) await decide(path);
       };
     }
-    const bound2 = /* @__PURE__ */ new WeakMap();
+    const bound3 = /* @__PURE__ */ new WeakMap();
     return new Proxy(fs, {
       get(target, key, receiver) {
         if (Object.hasOwn(overrides, key)) return overrides[key];
         const value2 = Reflect.get(target, key, receiver);
         if (typeof value2 !== "function") return value2;
-        let method = bound2.get(value2);
+        let method = bound3.get(value2);
         if (method === void 0) {
           method = value2.bind(target);
-          bound2.set(value2, method);
+          bound3.set(value2, method);
         }
         return method;
       }
@@ -11880,6 +11926,12 @@ var HeldSet = class extends Set {
     holds2.onIdle = this.idle;
     return super.add(holds2);
   }
+  doubt(paths, over) {
+    for (const holds2 of this) {
+      for (const path of paths) holds2.doubt(path, true);
+      if (over !== void 0) holds2.doubt(over);
+    }
+  }
   delete(holds2) {
     const removed = super.delete(holds2);
     for (const path of /* @__PURE__ */ new Set([...holds2.keys(), ...holds2.opens.keys()])) this.idle(path);
@@ -11901,6 +11953,9 @@ var HeldPaths = class extends Map {
   uses = /* @__PURE__ */ new Map();
   lastHandle = 0;
   owning;
+  bound = /* @__PURE__ */ new Map();
+  doubted = /* @__PURE__ */ new Map();
+  doubts = 0;
   onIdle;
   keyOf(path) {
     return this.aliases.get(path) ?? path;
@@ -11921,6 +11976,21 @@ var HeldPaths = class extends Map {
   handlesAt(path) {
     return [...this.handles].filter(([, key]) => key === path).map(([id2]) => id2);
   }
+  doubt(over, exact = false) {
+    if (this.bound.size === 0) return;
+    for (const [handle, key] of this.handles) {
+      if (!this.bound.has(handle) || this.keptHandles.has(handle)) continue;
+      if (exact ? key === over : within6(key, over)) this.doubted.set(handle, ++this.doubts);
+    }
+  }
+  boundIno(handle) {
+    const doubt = this.doubted.get(handle);
+    const ino = this.bound.get(handle);
+    return doubt === void 0 || ino === void 0 ? void 0 : { ino, doubt };
+  }
+  trust(handle, doubt) {
+    if (this.doubted.get(handle) === doubt) this.doubted.delete(handle);
+  }
   keptFor(handle) {
     return this.keptHandles.get(handle);
   }
@@ -11957,8 +12027,16 @@ var HeldPaths = class extends Map {
     for (const file of [...this.uses.keys()]) this.letGo(file, true);
     this.keptHandles.clear();
   }
-  hold(path, on, open2 = false, real, handle) {
-    if (open2) return this.holdOpen(path, on, real, handle);
+  hold(path, on, open2 = false, real, handle, ino) {
+    if (open2) {
+      const made2 = this.holdOpen(path, on, real, handle);
+      if (handle !== void 0) {
+        this.bound.delete(handle);
+        this.doubted.delete(handle);
+      }
+      if (made2 !== void 0 && ino !== void 0) this.bound.set(made2, ino);
+      return made2;
+    }
     this.holdWrite(path, on);
     return void 0;
   }
@@ -13637,7 +13715,7 @@ async function holdOp(entry, path, req) {
       await stillThere(entry, path, req);
       if (req.nofollow && !await ownEntry(entry, path, real ?? path)) continue;
       if (!r || !r.missed(path) && !r.changedSince(path, gen) && !r.inFlight?.(path))
-        return json2(entry.hold?.(path, on, open2, real) ?? null);
+        return json2(entry.hold?.(path, on, open2, real, void 0, req.ino) ?? null);
     }
   }
   entry.hold?.(path, on, open2, void 0, req.handle);
@@ -13787,10 +13865,26 @@ async function linkedElsewhere(fs, path, from) {
   return !(source && source.ino === st.ino && source.dev === st.dev);
 }
 async function keptOr(entry, path, handle) {
-  if (handle !== void 0) return entry.keptFor?.(handle)?.hidden ?? path;
+  if (handle !== void 0) {
+    const hidden = entry.keptFor?.(handle)?.hidden;
+    if (hidden !== void 0) return hidden;
+    await bound(entry, path, handle);
+    return path;
+  }
   const kept = entry.unlinked?.(path);
   return kept && !await entry.fs.exists(path) ? kept.hidden : path;
 }
+async function bound(entry, path, handle) {
+  const bound3 = entry.boundIno?.(handle);
+  if (bound3 === void 0) return;
+  const now = await entry.fs.stat(path).catch(() => void 0);
+  if (now?.ino === bound3.ino) {
+    entry.trust?.(handle, bound3.doubt);
+    return;
+  }
+  staleFds++;
+  throw syncError("ESTALE", path);
+}
 async function keptStat(entry, req) {
   const hidden = entry.keptFor?.(req.handle ?? -1)?.hidden;
   if (hidden === void 0)
@@ -13817,6 +13911,10 @@ var served = 0;
 function servedFsCalls() {
   return served;
 }
+var staleFds = 0;
+function staleFdCount() {
+  return staleFds;
+}
 async function dispatchSyncFs(req) {
   served++;
   const entry = resolveSyncFsToken(req.token);
@@ -14592,6 +14690,8 @@ var SYS_TREE = {
     slicc_cwd_checks: (listing) => `${listing.cwdChecks ?? 0}
 `,
     slicc_rename_ack_busy: (listing) => `${listing.renameAcks?.busy ?? 0}
+`,
+    slicc_stale_fds: (listing) => `${listing.staleFds ?? 0}
 `,
     slicc_rename_acks: (listing) => {
       const samples = [...listing.renameAcks?.ackMs ?? []].sort((a, b) => a - b);
@@ -19264,7 +19364,7 @@ async function writeAll3(file, bytes) {
   const write = file.write;
   for (let at = 0; at < bytes.length; ) at += await write.call(file, bytes.subarray(at));
 }
-function bound(file, key) {
+function bound2(file, key) {
   const method = file[key];
   return typeof method === "function" ? { [key]: method.bind(file) } : {};
 }
@@ -19356,9 +19456,9 @@ function scrubbed(inner, scrubber, idleMs = SCRUB_IDLE_MS) {
     later(stream.idle());
   };
   return new OpenFile({
-    ...bound(file, "read"),
-    ...bound(file, "peek"),
-    ...bound(file, "stat"),
+    ...bound2(file, "read"),
+    ...bound2(file, "peek"),
+    ...bound2(file, "stat"),
     poll: () => {
       const state = pollFile(file);
       return quiet2() ? state : { ...state, writable: false };
@@ -25077,7 +25177,9 @@ function holdingToken(opts, holds2, caller) {
     ...opts.statfs ? { statfs: opts.statfs } : {},
     ...opts.renames ? { renames: opts.renames } : {},
     ...opts.renameSeq ? { renameSeq: opts.renameSeq } : {},
-    hold: (path, on, open2, real, handle) => holds2.hold(path, on, open2, real, handle),
+    hold: (path, on, open2, real, handle, ino) => holds2.hold(path, on, open2, real, handle, ino),
+    boundIno: holds2.boundIno.bind(holds2),
+    trust: holds2.trust.bind(holds2),
     keptFor: (handle) => holds2.keptFor(handle),
     unlinked: (path) => holds2.keptAt(path),
     gate: (path, op) => opts.unlinkGate && holds2.holding(path) ? opts.unlinkGate.run(holds2.keyOf(path), op) : op(),
@@ -29128,6 +29230,7 @@ var Launcher = class {
     this.openFiles.add(this.nodes);
     options.fs.onPeer?.(() => this.flushOpen());
     options.fs.remoteChanges?.((change) => {
+      this.held.doubt(change.paths, change.moved?.to);
       if (change.moved) this.moved(change.moved.from, change.moved.to);
       this.changes.flush();
       this.cwds.touched(change.paths, change.trees);
@@ -30140,7 +30243,8 @@ var Launcher = class {
         opfsCalls: this.rootFs.opfsCalls?.() ?? 0,
         opfsOps: this.rootFs.opfsOps?.() ?? 0,
         cwdChecks: this.cwds.checks,
-        renameAcks: this.rootFs.renameWaits?.()
+        renameAcks: this.rootFs.renameWaits?.(),
+        staleFds: staleFdCount()
       }),
       sudo: (call, signal) => this.sudo(pid, call, signal),
       crontab: (call) => this.crontab(pid, call),
@@ -31118,6 +31222,7 @@ var Launcher = class {
     this.renameLog.add(from, to);
     if (source) return;
     for (const nodes of this.openFiles) nodes.renamed(from, to);
+    for (const holds2 of this.held) holds2.renamed(from, to);
     this.watchers.moved(from, to);
   }
   processChain(fs, nodes, unlinked) {