@objectstack/metadata-fs 17.1.0 → 17.3.0

package/CHANGELOG.md

@@ -1,5 +1,191 @@
 # @objectstack/metadata-fs
 
+## 17.3.0
+
+### Patch Changes
+
+- e7191ce: fix(build): give each `exports` condition its own `types` target in the 28 dual-build packages (#13112)
+  
+  **Published-surface change, zero runtime change.** No emitted byte moves; what
+  moves is which declaration file a resolver READS. Maintainer ruling 2026-08-29
+  (decision batch #3, verbatim 「同意」) chose declaring the files over deleting
+  them.
+  
+  ## What was wrong
+  
+  These 28 packages are `"type": "module"` and dual-built, and each spelled one
+  `types` condition as a **sibling** of `import`/`require`:
+  
+  ```json
+  "exports": { ".": {
+    "types": "./dist/index.d.ts", "import": "./dist/index.js", "require": "./dist/index.cjs"
+  } }
+  ```
+  
+  A sibling `types` answers for **both** conditions, so a CommonJS consumer was
+  handed `dist/index.d.ts` — an ES-module declaration, because the package is
+  `"type": "module"` — for an entry point it reaches with `require`. Measured with
+  `tsc --traceResolution` on a `"type": "commonjs"` fixture at `moduleResolution:
+  node16`:
+  
+  ```
+  error TS1479: The current file is a CommonJS module whose imports will produce
+  'require' calls; however, the referenced file is an ECMAScript module and cannot
+  be imported with 'require'.
+  ```
+  
+  The JavaScript at `dist/index.cjs` loads perfectly (`check:dual-build-cjs-loads`
+  has asserted that for months). It is the **types** that told the consumer the
+  supported `require` entry point could not be required. The `dist/index.d.cts`
+  twin tsup emits beside it — 36 files, 5,517,701 B on this build — was named by
+  no condition at all and shipped in every tarball unreachable.
+  
+  ## What changed
+  
+  Each condition now names its own declaration, the shape TypeScript documents:
+  
+  ```json
+  "exports": { ".": {
+    "import":  { "types": "./dist/index.d.ts",  "default": "./dist/index.js" },
+    "require": { "types": "./dist/index.d.cts", "default": "./dist/index.cjs" }
+  } }
+  ```
+  
+  33 entry points across 27 packages, subpaths included. The root `types` field is
+  untouched, so `node10` resolvers are unaffected; the `import` condition resolves
+  exactly what it resolved before, measured as an unchanged control in the same
+  run.
+  
+  ## `@objectstack/core` is deliberately NOT changed
+  
+  Splitting a declaration in two makes TypeScript compare it nominally, and
+  `ObjectKernel` carries a `private plugins` member that reaches every plugin
+  through `PluginContext.getKernel()`. With core split, whole-repo `pnpm build`
+  fails in `@objectstack/verify` with 5 × TS2345 ("Types have separate
+  declarations of a private property 'plugins'"); with core held back and the
+  other 27 split, 71/71 tasks pass. So core keeps the sibling-`types` shape and
+  its two `.d.cts` files (220,854 B) stay unreachable, declared as such in
+  `check:dual-build-cjs-loads`. Splitting it needs a decision about core's public
+  types, not about an exports map.
+  
+  ## For consumers
+  
+  - **ESM consumers: nothing changes.** Same declaration file, byte for byte.
+  - **CJS consumers under `node16`/`nodenext`: TS1479 goes away** and the
+    declarations they get are the ones built for CommonJS.
+  - **`node10` / `moduleResolution: node` consumers: nothing changes** — they never
+    read `exports`.
+  - Nothing is removed: every path that resolved before still resolves.
+  
+  Packages that are CJS-first (`require` → `./dist/index.js`, no `"type": "module"`)
+  were already correct and are untouched — their `dist/index.d.ts` really is the
+  CommonJS declaration. Their ESM mirror (an unreachable `.d.mts` under the
+  `import` condition) is a separate, larger population and is filed separately per
+  the ruling, not fixed here.
+  
+  `check:dual-build-cjs-loads` grew a fourth invariant (TYPED) that reds on the old
+  shape, so the drift cannot return silently.
+- a07a831: fix(metadata-fs): declare `startWatcher()`'s chokidar `atomic` option explicitly (#12696)
+  
+  `FileSystemRepository.startWatcher()` constructed its chokidar watcher with
+  `usePolling: true` but never passed `atomic`, leaving it to inherit chokidar's
+  default. That default is unconditionally `true` in the installed version
+  (chokidar 5.0.0): the defaults literal assigns `atomic: true` *before* the
+  caller's options are spread in, so chokidar's own default-correction
+  (`if (opts.atomic === undefined) opts.atomic = !opts.usePolling`) can never
+  fire — it only runs when `atomic` is literally `undefined` after the merge,
+  which it never is. The comment beside that correction ("Editor atomic write
+  normalization enabled by default with fs.watch") reads as "off under
+  polling"; the actual resolved behaviour was on regardless.
+  
+  This change passes `atomic: true` explicitly at the call site, with a comment
+  explaining why. **Patch, not a behaviour change**: verified at runtime
+  (constructing a watcher the way `startWatcher()` does and reading back
+  `watcher.options.atomic`) that the resolved value is identical before and
+  after — `true` either way, today. The only thing that changes is that the
+  value is now DECLARED rather than inherited from an upstream branch that
+  cannot execute, so a future chokidar release that fixes the ordering (making
+  the correction real) cannot silently flip this repository's watcher to
+  `atomic: false` under polling and change behaviour with no diff to review.
+  
+  Not addressed here (see #12696): whether `atomic: true` (the 100ms
+  unlink-coalescing deferral and the `DOT_RE` editor-temp-file matcher it turns
+  on) is actually the right value. No evidence surfaced that either has ever
+  affected a run; flipping it to `false` is a deliberate behaviour change to a
+  live delivery path that needs its own reverse verification, and is out of
+  scope for this card.
+- 3e8f5b0: The file watcher no longer publishes a `delete` for an item that is still on disk.
+  
+  A watcher `unlink` is a claim of absence, not absence: chokidar reaches its removal path from failed stats as well as from real removals, so under filesystem pressure it can retire a file that is still there. `FileSystemRepository` published those claims straight through as `delete` events — appended to the change log and broadcast to every subscriber, which drops the item from the metadata registry and the `list()` cache — and the reconciliation sweep then republished the untouched file as a `create`. A failed stat therefore produced a durable delete/create pair for an item nobody removed, with a window in between where live metadata had disappeared.
+  
+  The removal face now confirms the absence against the disk under the same per-key lock the reconciliation sweep already used for this, and an `unlink` for a path that still exists falls through to the content comparison — so a spurious unlink that arrived alongside a real external edit surfaces as the `update` it always was. Genuine external removals are unaffected and are still published on the first delivery.
+- Updated dependencies [54e2d36]
+- Updated dependencies [b745157]
+- Updated dependencies [d23ebb9]
+- Updated dependencies [fa5d137]
+- Updated dependencies [e7191ce]
+- Updated dependencies [0fd4899]
+- Updated dependencies [00d8f65]
+- Updated dependencies [200d255]
+- Updated dependencies [2852acc]
+- Updated dependencies [15d55fb]
+- Updated dependencies [15eb2c9]
+- Updated dependencies [1272f0a]
+- Updated dependencies [d41d166]
+- Updated dependencies [5d16379]
+  - @objectstack/metadata-core@17.3.0
+
+## 17.2.0
+
+### Patch Changes
+
+- 46644e2: `FileSystemRepository.close()` now terminates every live `watch()` iterator
+  instead of leaving it parked (#11127). A consumer holding a `for await` over
+  `watch()` at shutdown never saw its loop end — on a repository that was already
+  gone.
+  
+  `close()` retired the chokidar watcher and the resync sweep and stopped there.
+  It never reached the event broker, and the broker had no teardown of its own:
+  `subscribe`/`unsubscribe` add to and delete from a plain `Set`, and nothing else
+  emptied it. Each iterator parks its pending `next()` on a `waiter` that only two
+  things can settle — a broker `push`, or the iterator's own terminator, which ran
+  from `iterator.return()`/`throw()` and from nowhere else. After `close()` the
+  chokidar source was gone so no `push` could arrive, and the subscriber was still
+  registered with nothing left to run its terminator.
+  
+  Unlike the sibling defect in `SysMetadataRepository` (#11021) this was not
+  filter-dependent: there was no drain attempt at all, so every subscription shape
+  hung, `watch({})` included. Measured before the fix: nine cases —
+  `watch({org}, seq)`, `watch({org})`, `watch({})`, a ref-exact filter, a watcher
+  over the real chokidar watcher, a watcher with no pull outstanding, four
+  concurrent watchers, and the `return()`-symmetry comparison — were all still
+  unsettled 2s after `close()`. `MetadataManager.startRepositoryWatch()`, which
+  awaits `iter.next()` in a loop, is exactly the shape that hung.
+  
+  The broker now holds each subscription's terminator next to its event sink, and
+  `close()` runs every terminator — the same routine the consumer's own
+  `iterator.return()` runs, so a parked `next()` settles with `{ done: true }` and
+  no value, and so does every later one. Shutdown is deliberately **not** delivered
+  as an event: a synthetic drain event is subject to the very filters `watch()`
+  applies to real ones, and delivering an event has never ended an iterator
+  (invariant 8, `@objectstack/metadata-core`'s `repository.ts`).
+  
+  One narrower path is closed with it. `watch()` returns a deferred iterable whose
+  subscriber registers only once the eager log read resolves, so a `close()`
+  landing inside that window swept a broker the subscription had not yet joined —
+  the same forever-parked shape by a different route. `watch()` now carries the
+  close generation it was opened under, and a subscription that arrives after a
+  shutdown terminates on arrival.
+  
+  Invariant 8 named `FileSystemRepository` as its one known non-conformance. With
+  this change the invariant has no declared exceptions, and its text says so.
+- Updated dependencies [8cc8401]
+- Updated dependencies [26f3588]
+- Updated dependencies [05bc692]
+- Updated dependencies [f334d66]
+- Updated dependencies [2810695]
+  - @objectstack/metadata-core@17.2.0
+
 ## 17.1.0
 
 ### Patch Changes

package/dist/index.cjs

@@ -144,6 +144,16 @@ function createBroker(matches) {
         if (!matches(evt, s.filter)) continue;
         s.push(evt);
       }
+    },
+    terminateAll: () => {
+      const snapshot = Array.from(subs);
+      subs.clear();
+      for (const s of snapshot) {
+        try {
+          s.terminate();
+        } catch {
+        }
+      }
     }
   };
 }
@@ -158,6 +168,10 @@ function createWatchIterable(args) {
   const subscriber = {
     filter: args.filter,
     closed: false,
+    // Assigned below, once `close` exists. Termination and the consumer's own
+    // `return()` are ONE routine, deliberately: invariant 8 requires shutdown
+    // to be indistinguishable from `iterator.return()`.
+    terminate: () => void 0,
     push: (evt) => {
       if (subscriber.closed) return;
       const k = evtKey(evt);
@@ -206,6 +220,8 @@ function createWatchIterable(args) {
     }
     return { value: void 0, done: true };
   };
+  subscriber.terminate = close;
+  if (args.arrivesClosed?.()) close();
   const iterator = {
     next: () => {
       if (closed) return Promise.resolve({ value: void 0, done: true });
@@ -257,6 +273,15 @@ var FileSystemRepository = class {
      * the first degradation). An entry is cleared when that path reads again.
      */
     this.resyncFaults = /* @__PURE__ */ new Set();
+    /**
+     * Bumped by every `close()`. `watch()` reads it before its deferred log
+     * replay starts and hands the comparison to `createWatchIterable`, so a
+     * subscription that registers AFTER the shutdown sweep terminates on
+     * arrival instead of parking forever (#11127). A counter rather than a
+     * boolean because `start()` may follow `close()`: a repository restart must
+     * not poison the watchers opened after it.
+     */
+    this.closeGeneration = 0;
     this.org = opts.org;
     this.fsActor = opts.fsActor ?? "fs";
     this.disableWatch = opts.disableWatch ?? false;
@@ -302,8 +327,35 @@ var FileSystemRepository = class {
     await import_promises2.default.mkdir(this.layout.root, { recursive: true });
     if (this.started && !this.disableWatch && !this.watcher) this.startWatcher();
   }
+  /**
+   * Shut the repository down, ending every live `watch()` iterator.
+   *
+   * **Shutdown terminates; it does not emit** — invariant 8 in
+   * `@objectstack/metadata-core`'s `repository.ts`, and the reason this method
+   * reaches the broker at all. It used to retire the chokidar watcher and the
+   * resync sweep and stop there. The broker has no teardown of its own
+   * (`subscribe`/`unsubscribe` add to and delete from a plain `Set`), and each
+   * iterator parks its pending `next()` on a `waiter` that only a broker
+   * `push` or the iterator's own terminator can settle. After `close()` the
+   * chokidar source was gone, so no `push` could arrive; the subscriber was
+   * still registered, and nothing ran its terminator. A consumer holding a
+   * `for await` at shutdown — `MetadataManager.startRepositoryWatch()` is
+   * exactly that shape — therefore never saw its loop end, for EVERY
+   * subscription shape including `watch({})`.
+   *
+   * Termination is expressed as termination: each subscription's
+   * `terminate()`, which is the same routine the consumer's own
+   * `iterator.return()` runs, so no consumer has to tell "the repository shut
+   * down under me" apart from "I broke my own loop". A synthetic drain event
+   * would be the wrong shape and was measured to be so (#11021): the
+   * subscriptions most in need of draining are exactly the ones whose filter
+   * or numeric `since` drops it, and delivering an event has never ended an
+   * iterator.
+   */
   async close() {
     this.stopResync();
+    this.closeGeneration++;
+    this.broker.terminateAll();
     if (this.watcher) {
       await this.watcher.close();
       this.watcher = null;
@@ -378,6 +430,7 @@ var FileSystemRepository = class {
         if (matchEvent(evt, filter)) replay.push(evt);
       }
     })();
+    const generation = this.closeGeneration;
     return deferredIterable(promise.then(
       () => createWatchIterable({
         filter,
@@ -385,7 +438,8 @@ var FileSystemRepository = class {
         replay,
         broker: this.broker,
         matches: matchEvent,
-        branchKeyOf: (e) => e.ref.org
+        branchKeyOf: (e) => e.ref.org,
+        arrivesClosed: () => this.closeGeneration !== generation
       })
     ));
   }
@@ -598,7 +652,24 @@ var FileSystemRepository = class {
       // the entire customization tree.
       usePolling: true,
       interval: 1e3,
-      binaryInterval: 2e3
+      binaryInterval: 2e3,
+      // Declared explicitly, not inherited. chokidar's own default-correction
+      // (`if (opts.atomic === undefined) opts.atomic = !opts.usePolling`) can
+      // only fire when the caller omits `atomic`, but its defaults literal
+      // already assigns `atomic: true` *before* the caller's options are
+      // spread in — so leaving `atomic` unset here does not mean "off under
+      // polling" the way the correction's own comment claims, it silently
+      // resolves to `true` regardless of `usePolling`. That has been this
+      // repository's actual runtime behaviour all along (verified by reading
+      // back the resolved option from a real watcher instance, #12696): every
+      // `unlink` gets chokidar's 100ms editor-atomic-write deferral, and
+      // `DOT_RE` (vim swap files, `~`, sublime tmp) is folded into
+      // `_isIgnored` on top of this repository's own `isIgnoredWatchPath`
+      // (#7150). `atomic: true` here keeps that behaviour byte-for-byte —
+      // this is a declaration, not a change. Flipping it to `false` would
+      // remove both behaviours from a live delivery path and needs its own
+      // reverse verification; see #12696 for the analysis.
+      atomic: true
     });
     w.on("add", (p) => void this.handleFsChange(p, "add"));
     w.on("change", (p) => void this.handleFsChange(p, "change"));
@@ -864,9 +935,36 @@ var FileSystemRepository = class {
    *   - `unlink` — `!currentHead` drops the event when the index already
    *     agrees the item is gone. `delete()` retires the head *before* it
    *     unlinks, precisely because this face gets no `awaitWriteFinish` delay.
+   *     This is the whole of the *self-write* answer on this face, and it is
+   *     not the whole of the face — see the section below it.
    *
    * Both faces are pinned together in `test/self-write-suppression.test.ts`.
    *
+   * ## A removal is confirmed against the disk before it is published (#7369)
+   *
+   * Those two checks answer "is this event OURS". Neither answers "did this
+   * happen at all", and the `unlink` face needs that second question asked
+   * because its input is a third party's inference: chokidar decides a file is
+   * gone from a *stat that failed*, not only from a file that went away, and
+   * `!currentHead` cannot tell the two apart because a spurious unlink leaves
+   * the index exactly as valid as it was.
+   *
+   * The cost of getting it wrong is not a dropped notification, which the
+   * sweep would repair. A `delete` is appended to the change log and broadcast
+   * to every subscriber, and `MetadataManager` drops the item from the
+   * registry and the `list()` cache on receipt. The sweep then finds the file
+   * still on disk and republishes it as a `create` — so a failed stat becomes
+   * a durable, permanently recorded delete/create pair for an item that never
+   * changed, and every consumer sees the item disappear in between. That is
+   * the shape ADR-0008's log is least able to walk back.
+   *
+   * So `existsSync` under the same per-key lock the sweep uses, and the same
+   * decision it makes: absent ⇒ publish the removal; present ⇒ this was a
+   * change, answered by the content path below. Genuine removals pay nothing —
+   * `delete()` is still suppressed by `!currentHead`, and an external `rm` is
+   * still published on the first delivery, because for those the file really
+   * is gone. Pinned in `test/external-delete-requires-absence.test.ts`.
+   *
    * Note the deliberate limit: identity is judged on what round-trips through
    * the file, so a spec whose in-memory form does not (a `Date`, which
    * canonicalises to `{}` in memory but to an ISO string once written and
@@ -885,7 +983,7 @@ var FileSystemRepository = class {
     };
     const key = (0, import_metadata_core.refKey)(ref);
     await this.mutex.run(key, async () => {
-      if (kind === "unlink") {
+      if (kind === "unlink" && !(0, import_node_fs2.existsSync)(absPath)) {
         await this.publishExternalDelete(ref, key);
         return;
       }