@abelxiaoxing/cadence 2.0.14 → 2.0.15

package/README.md

@@ -332,6 +332,11 @@ Use their current declarations rather than copying DTO parsing, reducers, a seco
   Ordinary-user Git trust behavior remains unchanged.
   System services require uid0, bypass user managers/logind entirely and require confirmed native Stop before manager stop or definition replacement; changed state-home inputs, old user units and detached running owners refuse implicit adoption.
   Registry reservation atomically rejects canonical ancestor/descendant overlaps, including nested Git roots and aliases; a different state-home does not coordinate arbitrary other owners.
+- The short global registry lock never expires or auto-reclaims from PID absence.
+  `recover inspect|resolve` handles a valid dead same-host lock explicitly before project recovery, with fresh local-terminal acknowledgments, exact registry/unreserved-project checks and a flushed exclusive evidence receipt.
+  Concurrent or interrupted resolvers cannot remove a successor lock; all project reservations, locks and history remain intact.
+  Unreserved partial release, unsafe/corrupt registry or receipt bytes, live/remote/ambiguous holders and changed resources stay fenced.
+  Readonly diagnosis and project Open name `registry-locked`; see [global registry recovery](docs/service-management.md#residual-global-registry-lock).
 - Native entry requires fresh raw Enter within1500ms, trusted/idle TUI, no pending modal/messages and no child guard.
   Browser authentication grants access, not task consent; normal mutation ingress also requires independently built compatible client/service/assets identities.
 - Boot model choice is absent until explicitly selected.
@@ -450,6 +455,7 @@ Byte values are UTF-8/encoded bounds where specified; they are not targets to si
 | Navigation            | 8192 encoded ASCII bytes; encrypted plaintext6000 bytes; no execution authority                                      |
 | Live/export           | 16 viewers,256 messages/1MiB each; two search/two export readers;120s download idle, not total archive timeout       |
 | Storage               | 10GiB checkout quota;512MiB reserved per episode;256MiB free-space floor; no TTL cleanup                             |
+| Registry recovery     | 1024 namespace entries; 128 immutable attempts per lock incarnation; no evidence eviction                            |
 | Unified workbench     | 128 associated projects; 1024 durable opening receipts; 64 connections; bounded proxy frames and bodies; no eviction |
 | Assets                | 16MiB per member/64MiB closure; fixed complete manifest and all font bodies                                          |
 
@@ -505,7 +511,7 @@ Preserve failed attempts; correct a demonstrated cause before rerunning, and nev
 Regression owners:
 
 - `service-contract`, `service-current`, `service-model-choice`, `service-auth`: closed DTOs, exact source/choice/current authority, security and compatibility.
-- `service-owner`, `service-accepted-work`, `service-stop-continuation`, `service-recovery`: original operations, overlap, Stop, unknown fences and informed recovery.
+- `service-owner`, `service-accepted-work`, `service-stop-continuation`, `service-recovery`, `service-registry-recovery`: original operations, overlap, Stop, unknown fences and informed recovery.
 - `service-scheduler`, `service-checks`, `service-human-review`, `service-accepting-host`: real SDK overlap/exclusion, native checks, review and publication.
 - `worker-*`, `lightweight-sdk`, `lightweight-process-diff`: genuine SDK/tool capture, split redaction, no-spill bounds and original native facts.
 - `service-history-*`: durable ordering, cold/no-index/restarted proof/search/export, corruption/torn/quota and expected-byte publication.
@@ -536,6 +542,21 @@ Review the independently enumerated package member list and every emitted body,
 Keep public Pi peers external for the extension, the exact Undici/ws runtime edges, the exact optional SDK pins for the standalone owner and all upstream/font/vendor notices.
 Preserve original license bytes through their precise Git attributes; line-ending normalization must not invalidate audited source/notice hashes.
 
+### Release 2.0.15 acceptance snapshot
+
+The release candidate passed the complete root suite: 2034 tests in 85 files, with 27 platform skips on Linux.
+The final browser-only frame-readiness adjustment then passed all 9 real-browser scenarios; 130 web tests and the 135-member real-pack audit also passed.
+This is separate-lane evidence, not a claim that one final `verify` invocation was entirely green.
+Observed environment: Linux x86_64, Node26.7.0, Bun1.3.14, Pi SDK0.84.3 and existing Chrome for Testing151.0.7922.34; root runs used `VITEST_MAX_WORKERS=1` without changing test deadlines.
+New coverage includes actual registry-writer and resolver exits, concurrent resolvers, unchanged project fences/history, changed/linked/remote/live resources, a removed stopped workspace, piped/pasted/canceled refusal, genuine native-TTY confirmation and browser Open after explicit recovery without replay.
+
+The first integration command was cut off by its 25-minute outer command budget while tests were still running.
+The next completed root run failed one omitted browser-test provenance pin (updated) and three owner/proxy settlement or timeout cases, which passed separately; their timing cause was not established.
+A later complete root/web pass encountered Chromium's first screenshot-capture failure; the browser fixture now waits for painted frames before that capture, without retrying operations or swallowing capture errors, and all browser scenarios passed afterward.
+The last full attempt hit six history-test deadlines in two files; those unchanged files passed all 45 tests in a subsequent diagnostic run, and the earlier complete root pass remains distinct from that later failure.
+Failed logs and isolated fixture evidence were preserved; no test budget was increased, dependency/browser downloaded, consumer lock deleted or service installed/restarted.
+No paid-provider use or new native Windows/macOS qualification is claimed.
+
 ### Release 2.0.14 acceptance snapshot
 
 Full `verify` passed on the release candidate before this evidence-only note, with `VITEST_MAX_WORKERS=1`: 2000 root tests in 84 files (27 skipped on Linux), 130 web tests, 9 real-browser scenarios and the 135-member real-pack audit.

package/docs/service-management.md

@@ -258,7 +258,7 @@ At sign-out or shutdown Windows ends the owner without orderly Stop, which can l
   Ctrl-C, SIGTERM, machine crash or terminal destruction is **not** the authenticated orderly Stop route and can leave recovery fences.
   A recovered `stopped` registration means acknowledged fence release, not retroactive native closure certification.
 
-`diagnose` reports fixed pending/recording facts and `proxyAuth: direct|validated-proxy`.
+`diagnose` reports fixed pending/recording facts and `proxyAuth: direct|validated-proxy` when the owner is reachable; a blocked global registry instead reports its fixed lock reason and local recovery hint.
 No normal status/diagnosis prints access secrets, provider credentials or outbound proxy URLs.
 Keep the explicitly revealed token link private.
 Browser entry redirects to a token-free URL and uses an HttpOnly cookie; that does not retract a link already copied into history, a clipboard or external logs.
@@ -300,8 +300,46 @@ cadence recover inspect --workspace /absolute/project
 cadence recover resolve --workspace /absolute/project
 ```
 
-Inspection is readonly.
-It displays the epoch, exact coordination/history digests, owner PID/host observation and independent process/credential/storage uncertainty.
+Inspection is readonly and handles the global registry lock before project-owner recovery.
+
+### Residual global registry lock
+
+`status` and `diagnose` report `reason: "registry-locked"` and the lock holder observation (`live`, `absent`, `ambiguous` or `unreadable`).
+Project Open reports `registry-locked` instead of an opaque `unavailable`; the folder picker explains the local recovery command.
+This lock is shared by every project in the selected state home, so trying a different project does not bypass it.
+Use the same account and `--state-home` as the owner; omit that flag only when it uses the default state home:
+
+```sh
+cadence recover inspect --workspace /absolute/project --state-home /absolute/state-home
+cadence recover resolve --workspace /absolute/project --state-home /absolute/state-home
+```
+
+With a registry lock present, inspection reports `scope: "global-registry"`, the exact lock-incarnation digest, holder PID/host, reservation count and any previous resolver's liveness.
+A project with no selected owner can be used to address this global operation; it still must be an existing suitable directory.
+Resolve requires the same four fresh local-terminal acknowledgments described below, with the displayed lock-incarnation digest instead of a service epoch.
+The holder must be on this host and its PID probe must specifically report absence; age, a remote host, permission denial or a reused live PID never authorizes release.
+
+Recovery validates the registry and checks that unreserved project selections have no remaining owner/checkout fences.
+A normally stopped project whose working directory was later removed is checked through its retained private metadata; recovery neither requires nor reconstructs that directory.
+An interrupted reservation remains reserved even when its project initialization is incomplete: unrelated projects can reopen, but that project's uncertain resources are not repaired.
+An older release interrupted after removing a reservation but before releasing its project locks is unsafe and remains blocked, as do corrupt/partial registry bytes, linked files and changed resources.
+Inspection reports `blocked-preserve-state` when it cannot establish those conditions; preserve that state for manual diagnosis rather than deleting locks.
+
+Before releasing only `registry.lock`, the resolver exclusively creates, flushes and closes a private receipt under `service-registry-v1/recovery-v1/<lock-incarnation>/<attempt>.json`, retaining the original lock bytes and acknowledged digest.
+It then rechecks the lock incarnation, registry, unreserved project selections, receipt and terminal authority.
+Concurrent resolvers cannot release a successor lock.
+If a resolver dies after the receipt but before release, a new inspection and fresh terminal confirmation may append the next attempt after that resolver's same-host PID is absent; neither receipt nor lock is silently reclaimed.
+Torn or unsafe receipts remain preserved and refused.
+Inspection is bounded to 1024 namespace entries and 128 attempts for one lock incarnation; it never evicts evidence to make space.
+
+All project reservations, owner locks, checkout locks, capabilities, history and unknown outcomes remain unchanged.
+No Open/Start is replayed, no process is signaled, and no service is installed or started by recovery.
+After success, explicitly open the desired project again.
+If its own owner is still fenced, run `recover inspect` again and follow project-owner recovery separately; restoring global registry access does not certify or release that owner's work.
+
+### Project-owner recovery
+
+With no registry lock present, inspection displays the epoch, exact coordination/history digests, owner PID/host observation and independent process/credential/storage uncertainty.
 Those digests identify preserved resources, not successful work or credential validity.
 A journal with no unmatched intent does not prove every escaped descendant or external credential writer stopped.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@abelxiaoxing/cadence",
-  "version": "2.0.14",
+  "version": "2.0.15",
   "description": "Shared Pi task workspace: native terminal and WebUI, persistent service, checks and human review.",
   "license": "MIT",
   "type": "module",

package/src/native-build.ts

@@ -2,5 +2,5 @@
 export const nativeBuild = {
   kind: "native",
   contractHash:
-    "9c917c085d034e08764de359cee49aea748c395a63ed629fe67d149b97850fa1",
+    "944f3b53bd5b7f1d3f30868e0947d900e356ae9a2bcfb9f591edd988b3d5eb3a",
 } as const;

package/src/service/coordination.ts

@@ -150,6 +150,42 @@ function names(path: string, allowed: readonly string[]): void {
     dir.closeSync();
   }
 }
+interface RegistryLock {
+  schema: "cadence-service-registry-lock/1";
+  token: string;
+  pid: number;
+  host: string;
+}
+function decodeRegistryLock(snapshot: FileSnapshot): RegistryLock {
+  const o = closed(json(snapshot), ["schema", "token", "pid", "host"]);
+  return {
+    schema: literal(o.schema, "cadence-service-registry-lock/1"),
+    token: decodeId(o.token),
+    // kill(0)/negative process-group probes must never be possible.
+    pid: integer(o.pid, 1, 2147483647),
+    host: text(o.host, 255, true),
+  };
+}
+export interface RegistryLockDiagnostic {
+  reason: "registry-locked";
+  owner: "live" | "absent" | "ambiguous" | "unreadable";
+}
+export interface RegistryRecoveryInspection {
+  scope: "global-registry";
+  epoch: string; // exact lock incarnation, not a service epoch
+  coordinationDigest: string;
+  owner: "live" | "absent" | "ambiguous";
+  process: { pid: number; host: string };
+  reservations: number;
+  previousResolver: "live" | "absent" | "ambiguous" | null;
+  resolution: "local-interactive-only";
+}
+const registryNames = [
+  "format.json",
+  "roots.json",
+  "registry.lock",
+  "recovery-v1",
+];
 interface RegistryView {
   format: FileSnapshot;
   snapshot: FileSnapshot;
@@ -240,11 +276,7 @@ export class ServiceCoordination {
   }
   private registry(): RegistryView | undefined {
     if (!this.files.directory(this.registryDirectory)) return undefined;
-    names(this.registryDirectory, [
-      "format.json",
-      "roots.json",
-      "registry.lock",
-    ]);
+    names(this.registryDirectory, registryNames);
     const format = this.files.read(this.registryPath("format.json"));
     const snapshot = this.files.read(
       this.registryPath("roots.json"),
@@ -283,9 +315,9 @@ export class ServiceCoordination {
       },
     };
   }
-  assertNamespace(walks?: DirectoryWalks): void {
-    if (!this.files.directory(this.directory, false, walks)) refused();
-    names(this.directory, [
+  assertNamespace(walks?: DirectoryWalks, directory = this.directory): void {
+    if (!this.files.directory(directory, false, walks)) refused();
+    names(directory, [
       "format.json",
       "config.json",
       "registration.json",
@@ -297,18 +329,19 @@ export class ServiceCoordination {
       "history-v2",
       "conversations-v1",
     ]);
-    this.files.directory(join(this.directory, "history-v2"), false, walks);
+    this.files.directory(join(directory, "history-v2"), false, walks);
   }
-  private selected(): Selected | undefined {
+  private selected(
+    directory = this.directory,
+    root = this.store.root,
+    stoppedOnly = false,
+  ): Selected | undefined {
     const walks = new DirectoryWalks();
-    if (!this.files.directory(this.directory, false, walks)) return undefined;
-    this.assertNamespace(walks);
-    const manifest = this.files.read(
-      this.path("format.json"),
-      undefined,
-      walks,
-    );
-    const config = this.files.read(this.path("config.json"), undefined, walks);
+    if (!this.files.directory(directory, false, walks)) return undefined;
+    this.assertNamespace(walks, directory);
+    const path = (name: string) => join(directory, name);
+    const manifest = this.files.read(path("format.json"), undefined, walks);
+    const config = this.files.read(path("config.json"), undefined, walks);
     if (!manifest || !config) refused();
     const f = closed(json(manifest), [
       "schema",
@@ -317,39 +350,45 @@ export class ServiceCoordination {
       "configIdentity",
     ]);
     literal(f.schema, "cadence-service-format/1");
-    literal(f.root, this.store.root);
+    literal(f.root, root);
     literal(f.stateHome, this.stateHome);
     literal(f.configIdentity, config.identity);
     const c = decodeOwnerConfig(json(config));
     const hostConfig = this.files.read(
-      this.path("host-config.json"),
+      path("host-config.json"),
       undefined,
       walks,
     );
     if (hostConfig && hostConfig.identity !== c.policyIdentity) refused();
     const registration = this.files.read(
-      this.path("registration.json"),
+      path("registration.json"),
       undefined,
       walks,
     );
-    const owner = this.files.read(this.path("owner.lock"), undefined, walks);
+    const owner = this.files.read(path("owner.lock"), undefined, walks);
     const capability = this.files.read(
-      this.path("local-capability.json"),
+      path("local-capability.json"),
       undefined,
       walks,
     );
-    const checkout = this.files.read(this.store.lockPath, 8192, walks);
+    const checkout = this.files.read(
+      join(dirname(directory), "checkout.lock"),
+      8192,
+      walks,
+    );
     const value = registration
       ? decodeRegistration(json(registration))
       : undefined;
     if (
       value &&
-      (value.root !== this.store.root ||
+      (value.root !== root ||
         value.stateHome !== this.stateHome ||
         value.configIdentity !== config.identity ||
         !endpointMatches(c.endpoint, value.endpoint))
     )
       refused();
+    if (stoppedOnly && (checkout || (value && value.phase !== "stopped")))
+      throw new ContractError("quarantine");
     if (value && value.phase !== "stopped") {
       if (!owner || !checkout || !capability) refused();
       const lock = closed(json(owner), [
@@ -448,21 +487,24 @@ export class ServiceCoordination {
     ) => T,
   ): T {
     this.files.directory(this.registryDirectory, true);
-    names(this.registryDirectory, [
-      "format.json",
-      "roots.json",
-      "registry.lock",
-    ]);
+    names(this.registryDirectory, registryNames);
     const lockPath = this.registryPath("registry.lock");
-    const lock = this.files.create(
-      lockPath,
-      encoded({
-        schema: "cadence-service-registry-lock/1",
-        token: randomUUID(),
-        pid: process.pid,
-        host: hostname(),
-      }),
-    );
+    let lock: FileSnapshot;
+    try {
+      lock = this.files.create(
+        lockPath,
+        encoded({
+          schema: "cadence-service-registry-lock/1",
+          token: randomUUID(),
+          pid: process.pid,
+          host: hostname(),
+        }),
+      );
+    } catch (error) {
+      if (error instanceof Error && "code" in error && error.code === "EEXIST")
+        throw new ContractError("registry-locked");
+      throw error;
+    }
     let dirty = false;
     try {
       let format = this.files.read(this.registryPath("format.json"));
@@ -499,11 +541,7 @@ export class ServiceCoordination {
         () => {
           this.files.assert(lockPath, lock);
           this.files.assert(this.registryPath("format.json"), format);
-          names(this.registryDirectory, [
-            "format.json",
-            "roots.json",
-            "registry.lock",
-          ]);
+          names(this.registryDirectory, registryNames);
         },
       );
       this.files.remove(lockPath, lock);
@@ -696,6 +734,199 @@ export class ServiceCoordination {
       });
     });
   }
+  /** Readonly and secret-free; never probes another host or reclaims on age. */
+  registryLockDiagnostic(): RegistryLockDiagnostic | undefined {
+    try {
+      const lock = this.files.read(this.registryPath("registry.lock"));
+      if (!lock) return undefined;
+      const value = decodeRegistryLock(lock);
+      return {
+        reason: "registry-locked",
+        owner: recoveryOwnerStatus(value.pid, value.host),
+      };
+    } catch {
+      return { reason: "registry-locked", owner: "unreadable" };
+    }
+  }
+  private registryRecoveryState() {
+    const lock = this.files.read(this.registryPath("registry.lock"));
+    if (!lock) throw new ContractError("conflict");
+    const value = decodeRegistryLock(lock);
+    names(this.registryDirectory, registryNames);
+    const format = this.files.read(this.registryPath("format.json"));
+    const roots = this.files.read(
+      this.registryPath("roots.json"),
+      REGISTRY_BYTES,
+    );
+    // A crash immediately after exclusive creation may precede initialization.
+    const registry = !format && !roots ? undefined : this.registry();
+    const entries = registry?.value.entries ?? [];
+    const protectedRoots = new Set(entries.map((row) => hash(row.root)));
+    const proofs: unknown[] = [];
+    // Older releases removed a reservation BEFORE the checkout fence. A dead
+    // short-lock PID cannot make those unreserved, partially released projects
+    // safe. Verify every unreserved service selection; never repair it here.
+    const directory = opendirSync(this.files.namespace);
+    try {
+      let count = 0;
+      for (
+        let entry = directory.readSync();
+        entry;
+        entry = directory.readSync()
+      ) {
+        if (++count > 1024) throw new ContractError("capacity");
+        if (
+          !/^[a-f0-9]{64}$/.test(entry.name) ||
+          protectedRoots.has(entry.name)
+        )
+          continue;
+        const service = join(this.files.namespace, entry.name, "service-v1");
+        if (!this.files.directory(service)) continue;
+        const manifest = this.files.read(join(service, "format.json"));
+        if (!manifest) refused();
+        const f = closed(json(manifest), [
+          "schema",
+          "root",
+          "stateHome",
+          "configIdentity",
+        ]);
+        const root = pathValue(f.root);
+        if (hash(root) !== entry.name) refused();
+        // Retained stopped metadata remains checkable even when the consumer
+        // deleted/moved that old working directory. Never resolve/recreate it.
+        const selected = this.selected(service, root, true);
+        if (!selected) throw new ContractError("quarantine");
+        proofs.push([
+          entry.name,
+          ...[
+            selected.manifest,
+            selected.config,
+            selected.hostConfig,
+            selected.registration,
+          ].map((file) => (file ? [file.identity, file.stamp] : null)),
+        ]);
+      }
+    } finally {
+      directory.closeSync();
+    }
+    proofs.sort((a, b) =>
+      JSON.stringify(a).localeCompare(JSON.stringify(b), "en"),
+    );
+    const stateDigest = hash(
+      encoded([
+        lock.identity,
+        lock.stamp,
+        format ? [format.identity, format.stamp] : null,
+        roots ? [roots.identity, roots.stamp] : null,
+        proofs,
+      ]),
+    );
+    const epoch = hash(encoded([lock.identity, lock.stamp]));
+    const archive = this.registryPath(join("recovery-v1", epoch));
+    const attempts: FileSnapshot[] = [];
+    let previousResolver: RegistryRecoveryInspection["previousResolver"] = null;
+    if (this.files.directory(archive)) {
+      // Immutable, exclusive attempts serialize resolvers. If one crashes before
+      // release, a fresh confirmation may append the next attempt ONLY after its
+      // PID is absent. No stale secondary lock needs deletion or takeover.
+      names(
+        archive,
+        Array.from({ length: 128 }, (_, n) => `${n}.json`),
+      );
+      for (let n = 0; n < 128; n++) {
+        const attempt = this.files.read(join(archive, `${n}.json`), 32768);
+        if (!attempt) {
+          names(
+            archive,
+            attempts.map((_, i) => `${i}.json`),
+          );
+          break;
+        }
+        const o = closed(json(attempt, 32768), [
+          "schema",
+          "lock",
+          "pid",
+          "host",
+          "digest",
+        ]);
+        literal(o.schema, "cadence-registry-recovery/1");
+        literal(o.lock, lock.bytes.toString("utf8"));
+        decodeHash(o.digest);
+        previousResolver = recoveryOwnerStatus(
+          integer(o.pid, 1, 2147483647),
+          text(o.host, 255, true),
+        );
+        attempts.push(attempt);
+      }
+    }
+    const inspection: RegistryRecoveryInspection = {
+      scope: "global-registry",
+      epoch,
+      coordinationDigest: hash(
+        encoded([stateDigest, attempts.map((a) => [a.identity, a.stamp])]),
+      ),
+      owner: recoveryOwnerStatus(value.pid, value.host),
+      process: { pid: value.pid, host: value.host },
+      reservations: entries.length,
+      previousResolver,
+      resolution: "local-interactive-only",
+    };
+    return { lock, archive, attempts, stateDigest, inspection };
+  }
+  inspectRegistryRecovery(): RegistryRecoveryInspection {
+    return this.registryRecoveryState().inspection;
+  }
+  /** Local native management only. Retire just the proven dead short lock after
+   * durable evidence; reservations, owner/checkout locks and history stay intact. */
+  resolveRegistryRecovery(
+    expected: RegistryRecoveryInspection,
+    assertAuthority: () => void,
+  ) {
+    assertAuthority();
+    const state = this.registryRecoveryState();
+    if (encoded(state.inspection) !== encoded(expected))
+      throw new ContractError("conflict");
+    if (
+      expected.owner !== "absent" ||
+      (expected.previousResolver !== null &&
+        expected.previousResolver !== "absent")
+    )
+      throw new ContractError("quarantine");
+    if (state.attempts.length >= 128) throw new ContractError("capacity");
+    assertAuthority();
+    const receiptPath = join(state.archive, `${state.attempts.length}.json`);
+    const receipt = this.files.create(
+      receiptPath,
+      encoded({
+        schema: "cadence-registry-recovery/1",
+        lock: state.lock.bytes.toString("utf8"),
+        pid: process.pid,
+        host: hostname(),
+        digest: expected.coordinationDigest,
+      }),
+      32768,
+    );
+    // create fsyncs/closes the original receipt before any release effect.
+    assertAuthority();
+    const current = this.registryRecoveryState();
+    if (
+      current.stateDigest !== state.stateDigest ||
+      current.inspection.owner !== "absent" ||
+      current.attempts.length !== state.attempts.length + 1
+    )
+      throw new ContractError("conflict");
+    for (const [n, attempt] of state.attempts.entries())
+      this.files.assert(join(state.archive, `${n}.json`), attempt, 32768);
+    this.files.assert(receiptPath, receipt, 32768);
+    assertAuthority();
+    this.files.remove(this.registryPath("registry.lock"), state.lock);
+    return {
+      status: "resolved",
+      scope: "global-registry",
+      recoveryId: expected.epoch,
+      notice: "registry-only-project-fences-preserved",
+    } as const;
+  }
   /** Capture exact private resources BEFORE the append/confirmation awaits. The
    * closure is internal local management authority, never a wire capability.
    * A durable bridge must be verified by the caller before EVERY release effect. */