@ai-ecoverse/slicc-kernel 1.167.4 → 1.168.0

package/README.md

@@ -1250,7 +1250,15 @@ OPFS stays authoritative: an entry is consulted only for a path that exists in O
 
 **Stamping files written to OPFS:** `import { writeMetadata } from '@ai-ecoverse/slicc-kernel/meta'` and `await writeMetadata(root, [{ path, mode?, uid?, gid? }], { metadata?, indexedDB? })` is the supported way for an embedder to give files it wrote straight to OPFS a mode and an owner, also while no kernel runs (an installer before the first boot). `root` is the directory handle that becomes `/`, each `path` is absolute (`EINVAL` otherwise) and must exist there (`ENOENT` otherwise), `mode` is the permission bits (`0` to `0o7777`), and `uid` and `gid` come together or not at all (`EINVAL` otherwise). Entries for the same path (after normalization) are merged in order, a later field winning. All entries are written in one IndexedDB transaction, to the database a kernel on that root reads (`metadata`, default `slicc-kernel`), keeping what else a record holds and setting its change time (`ctime`) to now, and a running kernel on the same origin drops what it cached of them. The record format is the kernel's own and changes with it, so use this rather than writing the database. The kernel gives its layout paths (see [Permissions](#permissions)) an owner and mode only when they have no owner yet, so a path stamped here with `uid` and `gid` keeps them.
 
-**Telling running kernels about files written to OPFS:** a writer outside the kernel (an installer worker, a service worker, the page) that changes files on a root a kernel may be running on calls `import { postFsNotice } from '@ai-ecoverse/slicc-kernel/meta'` and `postFsNotice({ paths?, trees? }, { metadata? })` **after** each write has committed: after `close()` of its writable, `removeEntry` or `move()` has resolved. `paths` are absolute paths written, created or removed. List a created or removed name's parent directory too, so listings refresh. `trees` are absolute directories whose whole subtree changed, for a recursive replace or a sweep. The notice goes on the BroadcastChannel `slicc-kernel-dirs:<metadata>` (default `slicc-kernel-dirs:slicc-kernel`), the one kernels on that root already use between themselves, as `{ paths, trees }`. One notice per batch of writes is enough, if it's posted after the last of them. A kernel that receives it drops its cached lookups, handles and misses for those paths (for a tree, everything under it), tells its open files to reread, and publishes the paths to its programs' caches. A tree marks every path changed for them, so a running program's next look at any path goes to the kernel. **The residual window:** a reader can still see the old state between the write's commit and the notice's delivery (a message, usually well under a millisecond, but not bounded). A writer that runs while no kernel is up needs no notice: a kernel's caches start empty when it boots. A notice for a path that exists as a regular file, listed in `paths` (a `trees` entry refreshes caches but stamps no times, since a tree can hold thousands of files), also gives it a new modification time, before running programs are told of the change: the notice's arrival, with the change time along, unless its modification time is already later (an explicit future one is kept, with its change time), as a write does on Linux, recorded once per burst of notices for the same file ([#1275](https://github.com/ai-ecoverse/slicc-kernel/issues/1275)). That matters because Chrome can keep a file's OPFS `lastModified` across an append through `createWritable({ keepExistingData: true })`, and the kernel reports `lastModified` for files it didn't write, so `-nt` and make wouldn't see the change. A notice without a real change still moves the time, so a make may rebuild once. An outside write that posts no notice keeps OPFS's time. Notices between kernels (the array form they post for their own writes) don't stamp: those kernels record their times themselves. That's part of the channel's contract: kernels post their own change notices on the dirs channel as a plain array of paths, and only the `{ paths, trees }` object form that `postFsNotice` sends counts as an outside writer's. A kernel must keep posting arrays there, which `fs-notice-page.test.mjs`'s peer row checks. **Without a notice** a kernel that is alone on its root keeps what it cached: a running program's stats of a path stay as they were for as long as the program runs, and the kernel's own directory handles for up to a second. So a writer that doesn't post notices must not change files a running kernel uses ([#1156](https://github.com/ai-ecoverse/slicc-kernel/issues/1156)).
+**Telling running kernels about files written to OPFS:** a writer outside the kernel (an installer worker, a service worker, the page) that changes files on a root a kernel may be running on calls `import { postFsNotice } from '@ai-ecoverse/slicc-kernel/meta'` and `postFsNotice({ paths?, trees? }, { metadata? })` **after** each write has committed: after `close()` of its writable, `removeEntry` or `move()` has resolved. `paths` are absolute paths written, created or removed. List a created or removed name's parent directory too, so listings refresh. `trees` are absolute directories whose whole subtree changed, for a recursive replace or a sweep. The notice goes on the BroadcastChannel `slicc-kernel-dirs:<metadata>` (default `slicc-kernel-dirs:slicc-kernel`), the one kernels on that root already use between themselves, as `{ paths, trees }`. One notice per batch of writes is enough, if it's posted after the last of them. A kernel that receives it drops its cached lookups, handles and misses for those paths (for a tree, everything under it), tells its open files to reread, and publishes the paths to its programs' caches. If one of them is at or under a command root (`/node_modules`, `PNPM_HOME`, a `commands.append` root) or names a `node_modules` directory, the kernel also reads its command catalog (the packages' `package.json` files) again before its next exec, so a running shell's next command sees a command a manifest added or changed ([#1703](https://github.com/ai-ecoverse/slicc-kernel/issues/1703)). A tree marks every path changed for them, so a running program's next look at any path goes to the kernel. **The residual window:** a reader can still see the old state between the write's commit and the notice's delivery (a message, usually well under a millisecond, but not bounded). A writer that runs while no kernel is up needs no notice: a kernel's caches start empty when it boots. A notice for a path that exists as a regular file, listed in `paths` (a `trees` entry refreshes caches but stamps no times, since a tree can hold thousands of files), also gives it a new modification time, before running programs are told of the change: the notice's arrival, with the change time along, unless its modification time is already later (an explicit future one is kept, with its change time), as a write does on Linux, recorded once per burst of notices for the same file ([#1275](https://github.com/ai-ecoverse/slicc-kernel/issues/1275)). That matters because Chrome can keep a file's OPFS `lastModified` across an append through `createWritable({ keepExistingData: true })`, and the kernel reports `lastModified` for files it didn't write, so `-nt` and make wouldn't see the change. A notice without a real change still moves the time, so a make may rebuild once. An outside write that posts no notice keeps OPFS's time. Notices between kernels (the array form they post for their own writes) don't stamp: those kernels record their times themselves. That's part of the channel's contract: kernels post their own change notices on the dirs channel as a plain array of paths, and only the `{ paths, trees }` object form that `postFsNotice` sends counts as an outside writer's. A kernel must keep posting arrays there, which `fs-notice-page.test.mjs`'s peer row checks. **Without a notice** a kernel that is alone on its root keeps what it cached: a running program's stats of a path stay as they were for as long as the program runs, and the kernel's own directory handles for up to a second. So a writer that doesn't post notices must not change files a running kernel uses ([#1156](https://github.com/ai-ecoverse/slicc-kernel/issues/1156)).
+
+**Replacing a package in place** ([#1703](https://github.com/ai-ecoverse/slicc-kernel/issues/1703)): an installer that rewrites a package's files where they are, as the BIOS installer does, wraps those writes in `import { replacing } from '@ai-ecoverse/slicc-kernel/meta'` and `await replacing(paths, task, { metadata? })`, so that no exec loads a mix of old and new files (a new wasm with the old glue fails to link, exit 70). `paths` are the absolute package directories being replaced, the ones holding each `package.json` (`EINVAL` for a relative one); they are normalized, merged and sorted. For each one, `replacing` takes the exclusive Web Lock `slicc-kernel-replacing:v1:<metadata>:<directory>` (the directory verbatim, `metadata` defaulting to `slicc-kernel`, so for example `slicc-kernel-replacing:v1:slicc-kernel:/node_modules/@ai-ecoverse/wasm-bash`), runs `task`, posts `postFsNotice({ trees: paths })`, waits 150 ms and then releases, also when the task throws. It resolves to what the task returns or rejects with its error. The wait outlasts the longest a kernel's lookups serve a file's earlier state (100 ms, above), so no exec after the release reads a stat from before the last write. Every exec of a program that lives in a package takes the same lock in shared mode, from the moment its files are known until they are read and compiled, never while it runs: one Web Lock request per package its files are in, which is one for a program whose glue, module, variants and imports module share a package (about 60 µs in Chrome, under 0.2% of a fork and exec from bash). It makes no other OPFS request for it. An exec that comes while a replacement runs waits for it, then plans again from fresh lookups and a fresh command catalog and loads the new files, and a replacement waits for the execs that are already reading. The new plan goes through the same checks as the first (the caller's credentials, scope and exec permissions). A spawn that redeems a sealed exec ticket (see [`/dev/sealed`](#devsealed)) is also held to what the vault approved: if the plan after the wait has other files, another interpreter chain or another `trusted` than the ones `materializeConsume` was asked about, the spawn fails with `EACCES` and never starts, so the values never reach a process. The lock names keep their `v1` across kernel versions, so an installer can use the `replacing` of whichever version is installed while kernels of other versions run. Feature-test it with `typeof replacing === 'function'`; a kernel without it takes no lock. Without `navigator.locks`, `replacing` runs the task and posts the notice. The contract:
+
+- **Write each file whole,** with `createWritable()` without `keepExistingData`, then `close()`: the kernel keys what it compiled on a file's size and modification time, and Chrome can keep `lastModified` across a `keepExistingData` write (see the notices above).
+- **Name package directories:** an exec locks only the directory of its program's package (`<…>/node_modules/<name>` or `<…>/node_modules/@scope/<name>`, or a package directly under `modules` or a `commands.append` root), so a path above a package, such as `/node_modules`, doesn't hold its programs. Packages nested in one another are separate locks and may be replaced at the same time.
+- **The task must not run a program under a directory it replaces,** through any kernel: that exec waits for the task, which waits for the exec. A `replacing()` of a directory its own realm (the page or worker that calls it) is already replacing throws `EDEADLK` at once instead. A realm can't tell a nested call from a concurrent one, so two concurrent calls of one realm for the same directory get `EDEADLK` too.
+- **A holder that ends** (its tab closes or its worker dies) releases its locks, as Web Locks do, so execs go on with the files as it left them, and the next installer run repairs them.
+- **Under the Node entry,** Node's `navigator.locks` (26.11) is per thread, although its documentation says it spans worker threads: a lock taken in one thread neither holds nor shows in another. So call `replacing` on the thread that runs the kernel. A Node kernel names its locks with the default `slicc-kernel`.
 
 The database is `slicc-kernel` (or the `metadata` option), schema version 1, with one object store, `entries`, keyed by absolute path (`keyPath: 'path'`) and an index `dir` on the parent directory of symlinks. A record looks like `{ path, mode?, atimeMs?, mtimeMs?, mtimeFor?, ctimeMs?, ino?, link?, dir?, uid?, gid?, heir?, writes?, epoch? }`, where `mtimeFor` is the OPFS `lastModified` an explicit `mtimeMs` belongs to, `uid`, `gid` and `heir` (`[uid, gid]`) are the owners of [File owners](#file-owners), `writes` counts the writes kernels with `integrity` made to a protected file (see [Protected paths](#protected-paths)), whose `lastModified` OPFS leaves alone for a write through a sync access handle, and `epoch`, on the entry of `/`, counts their renames and symlinks at or above a protected path. A future schema bumps the version and migrates in `onupgradeneeded`; an older kernel then closes its connection. The sidecar is keyed by path from the `root` handle, so use a different `metadata` name for each distinct root.