@objectstack/metadata-fs 17.0.0-rc.5 → 17.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,328 @@
1
1
  # @objectstack/metadata-fs
2
2
 
3
+ ## 17.0.0
4
+
5
+ ### Patch Changes
6
+
7
+ - 2e836de: chore(packaging): CHANGELOG.md ships in every npm tarball (#4261)
8
+
9
+ The AGENTS.md post-task checklist requires breaking changesets to carry their
10
+ FROM → TO migration because "this text ships to consumers as `CHANGELOG.md`
11
+ inside the npm package and is what an upgrading agent greps after the tombstone
12
+ error." That delivery path was severed for 68 of the 69 publishable packages:
13
+ npm packs `package.json` / `README*` / `LICENSE*` unconditionally but — unlike
14
+ older npm versions — not `CHANGELOG.md`, and the canonical
15
+ `"files": ["dist", "README.md"]` whitelist never named it. Measured on npm
16
+ 10.9.7: `npm pack --dry-run` on `@objectstack/types` shipped 3 files while its
17
+ 70KB `CHANGELOG.md` stayed behind. Only `@objectstack/spec` listed it
18
+ explicitly.
19
+
20
+ The tombstone-error scenario is precisely the one where the repo is out of
21
+ reach — the upgrading agent has `node_modules` and nothing else — so the
22
+ migration text has to ride in the tarball. Every publishable package now
23
+ declares `CHANGELOG.md` in `files`, and the canonical whitelist is
24
+ `["dist", "README.md", "CHANGELOG.md"]`.
25
+
26
+ The other half is the gate: `check:published-files` gains a fifth invariant,
27
+ COMPLETE — a whitelist that fails to cover `CHANGELOG.md` fails the
28
+ always-required lint job, so the next package cannot silently sever the path
29
+ again. `@objectstack/spec`'s per-package EXTRA_ENTRIES exemption dissolves
30
+ into the canonical set.
31
+
32
+ Consumer-visible change: one more file per install (the package's changelog,
33
+ e.g. 70.8KB for `@objectstack/types`), and `grep -r "removed key"
34
+ node_modules/@objectstack/*/CHANGELOG.md` now finds the migration it was
35
+ promised.
36
+
37
+ - a1b66ef: `FileSystemRepository` no longer creates its root directory when it is attached — only when it first writes.
38
+
39
+ `start()` used to `mkdir` both `<root>` and `<root>/.objectstack/.log` unconditionally, so merely attaching a repository was a write. Because `MetadataPlugin` attaches one at `<project>/.objectstack/metadata` during every boot, a command that never writes metadata still brought a directory skeleton into existence. The loudest case is `os migrate plan`, a declared dry run: on a project that had never been started it left
40
+
41
+ ```
42
+ .objectstack/metadata/.objectstack/.log
43
+ ```
44
+
45
+ behind, which also destroyed the one signal — does `.objectstack/` exist? — by which the next command can tell a fresh project from a started one. This is the filesystem half of the same property the database half already covers: a dry run leaves nothing behind.
46
+
47
+ Attaching and reading a repository whose root does not exist is now explicitly supported and answers as an empty repository (`get`, `getByHash`, `list`, `history`, `watch`). The root, the type directories and the JSONL change log all appear on the first `put` / `delete`, and nothing about the boot's read-only character changes: no metadata is written that was not written before.
48
+
49
+ One behavioural note for direct users of the package: when the root is absent at `start()`, the chokidar watcher is armed by the first write instead, because chokidar cannot watch a path that does not yet exist. A root brought into existence by a third party while the process runs — with this repository never writing — is therefore not picked up until the next `start()`.
50
+
51
+ - c7e7900: fix(metadata-core,metadata-fs): hash the serialized form, so `put().version` identifies the bytes actually stored (#7856)
52
+
53
+ `hashSpec` canonicalised a `Date` to `{}`, because `canonicalize` walked a
54
+ value's own enumerable keys and a `Date` has none. `JSON.stringify` — what every
55
+ repository actually writes — turns the same `Date` into an ISO string. So the
56
+ hash of the in-memory spec and the hash of the bytes on disk were **different
57
+ hashes for the same item**, and the version handed back to a caller did not
58
+ identify what had been stored.
59
+
60
+ Measured on `main`, one spec carrying one `Date`:
61
+
62
+ ```
63
+ canonicalize(in-memory) : {"createdAt":{},"label":"Home"}
64
+ JSON.stringify (bytes) : {"label":"Home","createdAt":"2024-01-01T00:00:00.000Z"}
65
+ ```
66
+
67
+ `canonicalize` now honours `toJSON` exactly as `JSON.stringify` does —
68
+ consulted once per position, its result serialised as-is and never
69
+ re-consulted — which makes a new guarantee true by construction:
70
+
71
+ ```
72
+ canonicalize(x) === canonicalize(JSON.parse(JSON.stringify(x)))
73
+ ```
74
+
75
+ **Both repository implementations were wrong, in different places**, which is
76
+ why the fix is one function rather than two patches. `FileSystemRepository`
77
+ broke `put().version === get().hash`: it hashed the spec it was handed, wrote
78
+ `JSON.stringify` of it, and re-hashed the parse on the way back out.
79
+ `InMemoryRepository` broke the repository contract's invariant 4
80
+ (`item.hash === hashSpec(item.body)`): it stores `body` already serialised
81
+ (`clonePlain`) while hashing the in-memory spec, so the item it returns
82
+ disagreed with its own hash. `SysMetadataRepository` inherits the fix through
83
+ the same function.
84
+
85
+ Downstream, an incoherent version meant a repository could report an
86
+ `{op:'update', actor:'fs'}` for a file nothing outside the process had touched:
87
+ the head index held a hash the disk could never reproduce, so re-reading one's
88
+ own write looked like somebody else's edit. That surfaces without any watcher —
89
+ a restart rebuilds the index from disk and the version the caller was handed no
90
+ longer matches it.
91
+
92
+ **Ordinary specs hash exactly as before, and this is not a migration.** The new
93
+ path diverges only at a position carrying a callable `toJSON`; a graph without
94
+ one is byte-identical through `canonicalize`. Verified against this repository's
95
+ entire checked-in JSON corpus — 1973 files hashed under both the old and the new
96
+ implementation, **0 hashes changed** — and the `hashSpec({})` regression guard
97
+ in `metadata-core` is unmoved. Stored versions for ordinary specs keep their
98
+ meaning. Versions for `toJSON`-carrying specs do change, and those are exactly
99
+ the versions that never identified their stored bytes in the first place.
100
+
101
+ Also supported as a consequence: a class instance with a `toJSON` now hashes as
102
+ whatever it serialises to, rather than as its private fields. One without a
103
+ `toJSON` still hashes as its own enumerable keys — which is what
104
+ `JSON.stringify` writes for it.
105
+
106
+ The pin is table-driven and lives in the shared repository contract suite, so
107
+ every `MetadataRepository` implementation is held to it: `Date` at a key, `Date`
108
+ under an array index, a class whose `toJSON` yields a string, an object literal
109
+ carrying its own `toJSON`, a nested case, and a plain-JSON control row that
110
+ proves the fix did not simply change every hash.
111
+
112
+ - a1686f9: fix(metadata-fs): stop suppressing self-writes on a wall clock, so a poll tick can no longer swallow an external edit (#7335)
113
+
114
+ `FileSystemRepository` suppressed the watcher event its own `put()`/`delete()`
115
+ was about to produce by adding the path to a `selfWrites` Set and clearing it on
116
+ a fixed `setTimeout(…, 200)`. `handleFsChange` then dropped **any** event for a
117
+ path in that Set, without ever reading what the watcher had observed.
118
+
119
+ Under `usePolling: true, interval: 1000` chokidar compares state once per tick,
120
+ so our own write and an external edit landing between two ticks are delivered as
121
+ a **single** event carrying the _external_ content. Dropping that on a timer
122
+ destroyed the only notification the external edit would ever produce — the edit
123
+ was silently lost, and nothing later recovered it. The realistic trigger is a
124
+ `git checkout` or an editor save arriving while the process writes the same item:
125
+ the dev-mode authoring loop.
126
+
127
+ **Measured.** The filing recorded 0/360 instrumented iterations reaching the
128
+ window and called it derived rather than observed. That was a sampling artefact:
129
+ the delivery lag of a self-write event is
130
+ `(interval - (writeTime mod interval)) + awaitWriteFinish`, so a _fixed_
131
+ pre-edit sleep phase-locks the poll and pins the lag outside the window
132
+ (measured: 519–585 ms across 25 runs). Randomising the sleep so the lag samples
133
+ `[0, interval)` uniformly, 40 runs:
134
+
135
+ | delivery lag | runs | external edit |
136
+ | ------------ | ---- | ------------- |
137
+ | < 200 ms | 7 | **swallowed** |
138
+ | > 200 ms | 33 | delivered |
139
+
140
+ A perfect split on the wall-clock boundary — the mechanism, observed.
141
+
142
+ **The fix removes the pre-check rather than re-keying it**, because the
143
+ content-keyed suppression it was shadowing already existed one step further
144
+ down and needs no timer:
145
+
146
+ - `add`/`change` — `currentHead === hash` drops the event when the bytes on disk
147
+ are the bytes we last published. `put()` sets that head in the same
148
+ continuation as its `rename`, and `awaitWriteFinish` holds any event for a
149
+ further `stabilityThreshold`, so the index is never late.
150
+ - `unlink` — `!currentHead` drops the event when the index already agrees the
151
+ item is gone.
152
+
153
+ `delete()` additionally now retires the head **before** it unlinks rather than
154
+ after. `awaitWriteFinish` debounces only `add`/`change`, so that face gets no
155
+ stability cushion between the disk mutation and the event it produces; ordering
156
+ the index update first makes the downstream check a total suppression rather
157
+ than a race against the poll callback. A failed `unlink` restores the head
158
+ before rethrowing, so the error path is unchanged.
159
+
160
+ No API or configuration change; the repository publishes strictly more of the
161
+ external edits it was always meant to report.
162
+
163
+ One pre-existing limit is now documented rather than altered: identity is judged
164
+ on what round-trips through the file, so a spec whose in-memory form does not
165
+ (a `Date`, which canonicalises to `{}` in memory but to an ISO string once
166
+ written and re-read) is republished as an external `update`. Such a spec already
167
+ fails `put().version === get().hash` independently of the watcher, and the
168
+ 200 ms window never covered it either — it expired some 360 ms before the event
169
+ it would have had to catch.
170
+
171
+ - ab07b53: fix(metadata-fs): register every written path with the watcher, so an item created while chokidar is still scanning is not invisible forever (#7282)
172
+
173
+ `FileSystemRepository`'s watcher could go **permanently blind to a single
174
+ item** — external edits to that file produced no `MetadataEvent` for the whole
175
+ life of the process, and nothing recovered short of a restart. The window is a
176
+ race between chokidar's asynchronous initial scan and the repository's own
177
+ first write, and both `start()` (which arms the watcher, after which the caller
178
+ may `put()` on the next tick) and `ensureRoot()` (which arms it in the middle
179
+ of the very first write, #7000) can open it.
180
+
181
+ Measured on chokidar 5 with this repository's options (`usePolling`,
182
+ `interval: 1000`):
183
+
184
+ 1. chokidar reads `<root>/<type>/` and finds it EMPTY — the atomic `rename` in
185
+ `writeJsonAtomic` has not landed yet;
186
+ 2. the rename lands, changing the directory's mtime;
187
+ 3. chokidar calls `watchFile()` on that directory and libuv takes its polling
188
+ baseline stat, which already reflects step 2.
189
+
190
+ The directory's stat then never changes again, so no poll ever fires for it,
191
+ the directory is never re-read, the item file is never added to the watched
192
+ set, and no per-file watcher is created. `getWatched()` reports the type
193
+ directory as `[]` while the file sits in it, and neither `add` nor `change` is
194
+ ever emitted for that path.
195
+
196
+ The fix does not widen any timer. The only writer that can be inside that
197
+ window is the repository itself, so `put()` now tells the watcher explicitly
198
+ about the path it created instead of depending on a directory scan that may
199
+ never notice it. Registration is idempotent and emits nothing.
200
+
201
+ User-visible effect: `MetadataManager.subscribe()` (and every consumer of
202
+ `repo.watch()`) now reliably sees out-of-process edits — a hand edit, or a
203
+ `git checkout` bringing metadata JSON in — to items written earlier in the same
204
+ process. This was also the cause of four merge-queue ejections across three
205
+ PRs; the two time-based mitigations tried before it (a 20s/25s event deadline
206
+ and a wider pre-edit sleep) could not have worked, because the event was never
207
+ delivered rather than late.
208
+
209
+ - 684ab22: fix(metadata-fs): the `FileSystemRepository` watcher now sees external edits in the production layout
210
+
211
+ `MetadataPlugin` attaches the repository at `<project>/.objectstack/metadata`, and the
212
+ watcher's `ignored` matcher was a bare dotfile regex. chokidar applies that matcher to the
213
+ watched root path itself, not only to entries found underneath it, so the `.objectstack`
214
+ segment of the root matched and the entire watch was inert — `getWatched()` returned `{}`
215
+ and no event ever fired. Hand edits, a `git checkout` that brings metadata JSON in, and any
216
+ other out-of-process writer under `.objectstack/metadata/` were invisible until the next
217
+ `start()`, even though `MetadataManager.setRepository()` is wired to those events and uses
218
+ them to invalidate the registry and the `list()` cache.
219
+
220
+ The matcher is now evaluated against the path _relative_ to the watch root, so dot segments
221
+ belonging to the root itself are never considered while dotfiles under the root — including
222
+ the repository's own `.objectstack/` bookkeeping subtree — stay ignored as before.
223
+
224
+ - Updated dependencies [f5a4ef0]
225
+ - Updated dependencies [2e836de]
226
+ - Updated dependencies [121852d]
227
+ - Updated dependencies [db0d53c]
228
+ - Updated dependencies [c7e7900]
229
+ - Updated dependencies [72c3c86]
230
+ - Updated dependencies [3670cf9]
231
+ - Updated dependencies [2d8dba3]
232
+ - Updated dependencies [7372d46]
233
+ - Updated dependencies [5e247fd]
234
+ - Updated dependencies [1a53a02]
235
+ - Updated dependencies [a954634]
236
+ - Updated dependencies [fda61e4]
237
+ - Updated dependencies [db48ad5]
238
+ - Updated dependencies [65f184b]
239
+ - Updated dependencies [51a587d]
240
+ - Updated dependencies [c073b8c]
241
+ - Updated dependencies [946a131]
242
+ - Updated dependencies [ce92674]
243
+ - Updated dependencies [3d4c545]
244
+ - Updated dependencies [bb7cb41]
245
+ - @objectstack/metadata-core@17.0.0
246
+
247
+ ## 17.0.0-rc.6
248
+
249
+ ### Patch Changes
250
+
251
+ - a1b66ef: `FileSystemRepository` no longer creates its root directory when it is attached — only when it first writes.
252
+
253
+ `start()` used to `mkdir` both `<root>` and `<root>/.objectstack/.log` unconditionally, so merely attaching a repository was a write. Because `MetadataPlugin` attaches one at `<project>/.objectstack/metadata` during every boot, a command that never writes metadata still brought a directory skeleton into existence. The loudest case is `os migrate plan`, a declared dry run: on a project that had never been started it left
254
+
255
+ ```
256
+ .objectstack/metadata/.objectstack/.log
257
+ ```
258
+
259
+ behind, which also destroyed the one signal — does `.objectstack/` exist? — by which the next command can tell a fresh project from a started one. This is the filesystem half of the same property the database half already covers: a dry run leaves nothing behind.
260
+
261
+ Attaching and reading a repository whose root does not exist is now explicitly supported and answers as an empty repository (`get`, `getByHash`, `list`, `history`, `watch`). The root, the type directories and the JSONL change log all appear on the first `put` / `delete`, and nothing about the boot's read-only character changes: no metadata is written that was not written before.
262
+
263
+ One behavioural note for direct users of the package: when the root is absent at `start()`, the chokidar watcher is armed by the first write instead, because chokidar cannot watch a path that does not yet exist. A root brought into existence by a third party while the process runs — with this repository never writing — is therefore not picked up until the next `start()`.
264
+
265
+ - ab07b53: fix(metadata-fs): register every written path with the watcher, so an item created while chokidar is still scanning is not invisible forever (#7282)
266
+
267
+ `FileSystemRepository`'s watcher could go **permanently blind to a single
268
+ item** — external edits to that file produced no `MetadataEvent` for the whole
269
+ life of the process, and nothing recovered short of a restart. The window is a
270
+ race between chokidar's asynchronous initial scan and the repository's own
271
+ first write, and both `start()` (which arms the watcher, after which the caller
272
+ may `put()` on the next tick) and `ensureRoot()` (which arms it in the middle
273
+ of the very first write, #7000) can open it.
274
+
275
+ Measured on chokidar 5 with this repository's options (`usePolling`,
276
+ `interval: 1000`):
277
+
278
+ 1. chokidar reads `<root>/<type>/` and finds it EMPTY — the atomic `rename` in
279
+ `writeJsonAtomic` has not landed yet;
280
+ 2. the rename lands, changing the directory's mtime;
281
+ 3. chokidar calls `watchFile()` on that directory and libuv takes its polling
282
+ baseline stat, which already reflects step 2.
283
+
284
+ The directory's stat then never changes again, so no poll ever fires for it,
285
+ the directory is never re-read, the item file is never added to the watched
286
+ set, and no per-file watcher is created. `getWatched()` reports the type
287
+ directory as `[]` while the file sits in it, and neither `add` nor `change` is
288
+ ever emitted for that path.
289
+
290
+ The fix does not widen any timer. The only writer that can be inside that
291
+ window is the repository itself, so `put()` now tells the watcher explicitly
292
+ about the path it created instead of depending on a directory scan that may
293
+ never notice it. Registration is idempotent and emits nothing.
294
+
295
+ User-visible effect: `MetadataManager.subscribe()` (and every consumer of
296
+ `repo.watch()`) now reliably sees out-of-process edits — a hand edit, or a
297
+ `git checkout` bringing metadata JSON in — to items written earlier in the same
298
+ process. This was also the cause of four merge-queue ejections across three
299
+ PRs; the two time-based mitigations tried before it (a 20s/25s event deadline
300
+ and a wider pre-edit sleep) could not have worked, because the event was never
301
+ delivered rather than late.
302
+
303
+ - 684ab22: fix(metadata-fs): the `FileSystemRepository` watcher now sees external edits in the production layout
304
+
305
+ `MetadataPlugin` attaches the repository at `<project>/.objectstack/metadata`, and the
306
+ watcher's `ignored` matcher was a bare dotfile regex. chokidar applies that matcher to the
307
+ watched root path itself, not only to entries found underneath it, so the `.objectstack`
308
+ segment of the root matched and the entire watch was inert — `getWatched()` returned `{}`
309
+ and no event ever fired. Hand edits, a `git checkout` that brings metadata JSON in, and any
310
+ other out-of-process writer under `.objectstack/metadata/` were invisible until the next
311
+ `start()`, even though `MetadataManager.setRepository()` is wired to those events and uses
312
+ them to invalidate the registry and the `list()` cache.
313
+
314
+ The matcher is now evaluated against the path _relative_ to the watch root, so dot segments
315
+ belonging to the root itself are never considered while dotfiles under the root — including
316
+ the repository's own `.objectstack/` bookkeeping subtree — stay ignored as before.
317
+
318
+ - Updated dependencies [121852d]
319
+ - Updated dependencies [5e247fd]
320
+ - Updated dependencies [1a53a02]
321
+ - Updated dependencies [a954634]
322
+ - Updated dependencies [3d4c545]
323
+ - Updated dependencies [bb7cb41]
324
+ - @objectstack/metadata-core@17.0.0-rc.6
325
+
3
326
  ## 17.0.0-rc.5
4
327
 
5
328
  ### Patch Changes
package/README.md CHANGED
@@ -35,7 +35,7 @@ const repo = new FileSystemRepository({
35
35
  root: './metadata',
36
36
  org: 'system',
37
37
  });
38
- await repo.start(); // scan + open watcher
38
+ await repo.start(); // scan + open watcher — creates nothing on disk
39
39
 
40
40
  const view = await repo.get({
41
41
  org: 'system',
@@ -47,4 +47,19 @@ for await (const evt of repo.watch({})) {
47
47
  }
48
48
  ```
49
49
 
50
+ ## Root creation is a write, not an attach
51
+
52
+ `start()` never creates `<root>`. Attaching a repository whose root does not
53
+ exist is legal: reads answer as if the repository were empty, and the root —
54
+ together with `<root>/.objectstack/.log/` — appears on the **first write**
55
+ (`put` / `delete`). This is what keeps a read-only boot, such as the dry run
56
+ `os migrate plan` performs on a project that has never been started, from
57
+ leaving a directory skeleton behind (#7000, the filesystem half of #6743).
58
+
59
+ One consequence worth knowing: when the root is absent at `start()`, the
60
+ chokidar watcher is armed by the first write instead, because chokidar cannot
61
+ watch a path that does not exist yet. A root brought into existence by a third
62
+ party while the process runs, without this repository ever writing, is
63
+ therefore not picked up until the next `start()`.
64
+
50
65
  See ADR-0008 (incl. §0 amendment) and the `metadata-branch-removal` changeset.
package/dist/index.cjs CHANGED
@@ -243,8 +243,6 @@ var FileSystemRepository = class {
243
243
  this.heads = /* @__PURE__ */ new Map();
244
244
  /** Next seq counter, hydrated from the log on `start()`. */
245
245
  this.nextSeq = 1;
246
- /** Paths we wrote ourselves; suppress the resulting chokidar event. */
247
- this.selfWrites = /* @__PURE__ */ new Set();
248
246
  this.watcher = null;
249
247
  this.started = false;
250
248
  this.org = opts.org;
@@ -255,15 +253,42 @@ var FileSystemRepository = class {
255
253
  this.log = new JsonlLog(logFile(this.layout));
256
254
  }
257
255
  // ── Lifecycle ───────────────────────────────────────────────────────
256
+ /**
257
+ * Attach the repository. **Creates nothing on disk** (#7000).
258
+ *
259
+ * Attaching is not a write. `start()` used to `mkdir` both the root and
260
+ * `<root>/.objectstack/.log` unconditionally, which meant every read-only
261
+ * boot that merely attaches a repository left a skeleton behind — most
262
+ * visibly `os migrate plan`, a declared dry run, on a project that has
263
+ * never been started. That is the same property #6743 ruled on for
264
+ * `.objectstack/data/`: a dry run leaves nothing behind, and the existence
265
+ * of `.objectstack/` has to stay a usable "this project has been started"
266
+ * signal.
267
+ *
268
+ * Every read path below already treats a missing root as an empty
269
+ * repository (`scanHeads` swallows ENOENT, `JsonlLog` guards on
270
+ * `existsSync`, `get` guards on `existsSync`), so the root is materialized
271
+ * by `ensureRoot()` on the first write instead.
272
+ */
258
273
  async start() {
259
274
  if (this.started) return;
260
275
  this.started = true;
261
- await import_promises2.default.mkdir(this.layout.root, { recursive: true });
262
- await import_promises2.default.mkdir(logDir(this.layout), { recursive: true });
263
276
  await this.scanHeads();
264
277
  const highest = await this.log.highestSeq();
265
278
  this.nextSeq = highest + 1;
266
- if (!this.disableWatch) this.startWatcher();
279
+ if (!this.disableWatch && (0, import_node_fs2.existsSync)(this.layout.root)) this.startWatcher();
280
+ }
281
+ /**
282
+ * Bring the repository root into existence. Called by every write path
283
+ * immediately before it touches the disk — `start()` deliberately does not
284
+ * create it (#7000), so this is the single seam where the root appears.
285
+ *
286
+ * It is also where a watcher that `start()` could not arm (missing root)
287
+ * gets armed, so "external edits are detected" survives the change.
288
+ */
289
+ async ensureRoot() {
290
+ await import_promises2.default.mkdir(this.layout.root, { recursive: true });
291
+ if (this.started && !this.disableWatch && !this.watcher) this.startWatcher();
267
292
  }
268
293
  async close() {
269
294
  if (this.watcher) {
@@ -381,13 +406,10 @@ var FileSystemRepository = class {
381
406
  const seq = this.nextSeq++;
382
407
  const ts = this.now().toISOString();
383
408
  const file = itemPath(this.layout, ref.type, ref.name);
409
+ await this.ensureRoot();
384
410
  await import_promises2.default.mkdir(typeDir(this.layout, ref.type), { recursive: true });
385
- this.selfWrites.add(file);
386
- try {
387
- await writeJsonAtomic(file, spec);
388
- } finally {
389
- setTimeout(() => this.selfWrites.delete(file), 200);
390
- }
411
+ await writeJsonAtomic(file, spec);
412
+ this.trackWrittenPath(file);
391
413
  this.heads.set(key, hash);
392
414
  const evt = {
393
415
  seq,
@@ -427,13 +449,14 @@ var FileSystemRepository = class {
427
449
  throw new import_metadata_core.ConflictError(ref, opts.parentVersion, currentHead);
428
450
  }
429
451
  const file = itemPath(this.layout, ref.type, ref.name);
430
- this.selfWrites.add(file);
452
+ await this.ensureRoot();
453
+ this.heads.delete(key);
431
454
  try {
432
455
  if ((0, import_node_fs2.existsSync)(file)) await import_promises2.default.unlink(file);
433
- } finally {
434
- setTimeout(() => this.selfWrites.delete(file), 200);
456
+ } catch (err) {
457
+ if (currentHead !== null) this.heads.set(key, currentHead);
458
+ throw err;
435
459
  }
436
- this.heads.delete(key);
437
460
  const seq = this.nextSeq++;
438
461
  const ts = this.now().toISOString();
439
462
  const evt = {
@@ -502,10 +525,58 @@ var FileSystemRepository = class {
502
525
  }
503
526
  return last;
504
527
  }
528
+ /**
529
+ * Register a path this repository just wrote with the watcher (#7282).
530
+ *
531
+ * chokidar's initial scan is asynchronous, and every write path here can be
532
+ * running **while it is still walking the tree** — `start()` arms the watcher
533
+ * and the caller may `put()` on the next tick, and `ensureRoot()` arms it in
534
+ * the middle of the very first write. With `usePolling` that combination has
535
+ * a permanently-blinding interleaving, measured on chokidar 5 with this
536
+ * repository's own options:
537
+ *
538
+ * 1. chokidar reads `<root>/<type>/` and finds it EMPTY — the atomic
539
+ * `rename` in `writeJsonAtomic` has not landed yet.
540
+ * 2. the rename lands; the directory's mtime changes.
541
+ * 3. chokidar calls `watchFile()` on that directory, and libuv takes its
542
+ * polling baseline stat — which already reflects step 2.
543
+ *
544
+ * From then on the directory's stat never changes again, so no poll ever
545
+ * fires for it, `_handleRead` never re-runs, the item file is never added to
546
+ * the watched set, and no per-file watcher is ever created. chokidar emits
547
+ * neither `add` nor `change` for that path **for the life of the process** —
548
+ * `getWatched()` reports the type directory as `[]` forever while the file
549
+ * sits in it. That is the whole of #7282: the four merge-queue ejections all
550
+ * waited out their deadlines (20s, then 25541ms against 25s) on an event that
551
+ * was never going to be delivered, which is why widening the deadline and
552
+ * widening the pre-edit sleep both changed nothing, and why lowering
553
+ * `interval` would change nothing either — a shorter poll re-compares against
554
+ * the same unchanged directory stat.
555
+ *
556
+ * The window is exactly "files that exist at baseline time but were absent
557
+ * from the snapshot read a moment earlier", and the only writer that can be
558
+ * inside it is us. So we close it at the source: tell the watcher explicitly
559
+ * about every path we create, instead of hoping its scan happened to see it.
560
+ *
561
+ * `add()` is idempotent here — `_handleFile` returns early when the parent
562
+ * directory already tracks the basename — and it emits nothing, because
563
+ * chokidar treats an explicit `add()` as an initial add and `ignoreInitial`
564
+ * is set. Its effect is the one we need: `_watchWithNodeFs` registers the
565
+ * basename with the parent directory (without which chokidar drops `change`
566
+ * events for the file) and starts the per-file poll.
567
+ */
568
+ trackWrittenPath(file) {
569
+ const w = this.watcher;
570
+ if (!w || w.closed) return;
571
+ w.add(file);
572
+ }
505
573
  startWatcher() {
506
- const w = import_chokidar.default.watch(this.layout.root, {
507
- ignored: [/(^|[\\/])\../],
508
- // skip dotfiles incl. .objectstack
574
+ const root = this.layout.root;
575
+ const w = import_chokidar.default.watch(root, {
576
+ // Skip dotfiles under the root — including the repository's own
577
+ // `.objectstack/` bookkeeping subtree — matched on the path RELATIVE
578
+ // to the watch root (#7150). See `isIgnoredWatchPath`.
579
+ ignored: (p) => isIgnoredWatchPath(root, p),
509
580
  ignoreInitial: true,
510
581
  depth: 2,
511
582
  awaitWriteFinish: { stabilityThreshold: 50, pollInterval: 20 },
@@ -521,8 +592,55 @@ var FileSystemRepository = class {
521
592
  w.on("unlink", (p) => void this.handleFsChange(p, "unlink"));
522
593
  this.watcher = w;
523
594
  }
595
+ /**
596
+ * Translate a watcher event into a `MetadataEvent`, or drop it.
597
+ *
598
+ * ## Self-writes are suppressed by content identity, never by a clock (#7335)
599
+ *
600
+ * This used to open with `if (this.selfWrites.has(absPath)) return;` — a
601
+ * `Set` that `put()`/`delete()` added the path to and a `setTimeout(…, 200)`
602
+ * cleared. That check discarded **every** event for a recently-written path
603
+ * without ever looking at what the watcher had actually observed, which is
604
+ * the whole defect: with `usePolling`, chokidar compares state once per
605
+ * `interval`, so our write and an external edit landing between two ticks
606
+ * are delivered as **one** event carrying the *external* content. Dropping
607
+ * it on a wall clock destroyed the only notification that edit would ever
608
+ * produce.
609
+ *
610
+ * Measured on `origin/main` @ `69fde55`, 40 iterations, poll phase
611
+ * randomised so the delivery lag samples `[0, interval)` uniformly:
612
+ *
613
+ * delivery lag < 200ms → 7 runs → external edit SWALLOWED, every time
614
+ * delivery lag > 200ms → 33 runs → external edit delivered, every time
615
+ *
616
+ * A perfect split on the wall-clock boundary, and the reason earlier
617
+ * instrumentation saw 0/360: a *fixed* pre-edit sleep phase-locks the poll,
618
+ * pinning the lag (measured: 519–585ms across 25 runs) safely outside the
619
+ * window. Nothing about the window was rare — it was unsampled.
620
+ *
621
+ * What remains is the check that was already doing the real work one step
622
+ * down, and it needs no timer because it compares the content the watcher
623
+ * **read** against the index:
624
+ *
625
+ * - `add`/`change` — `currentHead === hash` drops the event when the bytes
626
+ * on disk are the bytes we last published. `put()` sets that head in the
627
+ * same continuation as its `rename`, and `awaitWriteFinish` holds the
628
+ * event for a further `stabilityThreshold`, so it is never late.
629
+ * - `unlink` — `!currentHead` drops the event when the index already
630
+ * agrees the item is gone. `delete()` retires the head *before* it
631
+ * unlinks, precisely because this face gets no `awaitWriteFinish` delay.
632
+ *
633
+ * Both faces are pinned together in `test/self-write-suppression.test.ts`.
634
+ *
635
+ * Note the deliberate limit: identity is judged on what round-trips through
636
+ * the file, so a spec whose in-memory form does not (a `Date`, which
637
+ * canonicalises to `{}` in memory but to an ISO string once written and
638
+ * re-read) is republished as an external `update`. That predates this change
639
+ * and is independent of it — such a spec already fails `put().version ===
640
+ * get().hash`, and the 200ms window never covered it either, expiring some
641
+ * 360ms before the event it would have had to catch.
642
+ */
524
643
  async handleFsChange(absPath, kind) {
525
- if (this.selfWrites.has(absPath)) return;
526
644
  const parsed = parseItemPath(this.layout, absPath);
527
645
  if (!parsed) return;
528
646
  const ref = {
@@ -573,6 +691,11 @@ var FileSystemRepository = class {
573
691
  });
574
692
  }
575
693
  };
694
+ function isIgnoredWatchPath(root, absPath) {
695
+ const rel = import_node_path3.default.relative(root, absPath);
696
+ if (rel === "" || rel.startsWith("..")) return false;
697
+ return rel.split(/[\\/]/).some((segment) => segment.startsWith("."));
698
+ }
576
699
  async function readJson(file) {
577
700
  try {
578
701
  const text = await import_promises2.default.readFile(file, "utf8");