mellos-mapping 0.23.0 → 0.25.0

package/docs/locking.md

@@ -0,0 +1,115 @@
+# Project locking
+
+Map writes have three separate guarantees: atomic file replacement keeps a
+reader from seeing half a saved map, a project lock serializes cooperating
+writers, and an optional revision check rejects edits computed from stale
+data. None of these replaces the others.
+
+## One stable lock for every page
+
+The default page and all named pages use the same fixed regular file,
+`.mellos/.write-lock`, for their project.
+`withStoreLock` acquires an OS exclusive lock without waiting before reading
+the current graph. MCP graph mutations, HTTP viewer mutations and watcher
+mutations all use this boundary. A competing writer receives `BUSY` and can
+retry after the active operation completes; acquisition does not start a
+background poll or queue.
+
+The lock file is independent of the map JSON files. A successful map save
+replaces its target through a private sibling temporary file, while the lock
+file keeps the same identity. Do not delete, rename or replace the lock file:
+doing so can leave processes locking different files for the same project.
+Keeping the file after an operation is deliberate; its presence is not proof
+that a writer is active.
+
+Normal completion and errors release the lock. When the owning process exits,
+including a crash, the OS releases its lock. The new protocol does not use
+directory creation, an `owner.json` PID, a `.reap` directory, or elapsed time
+to decide ownership or reclaim a lock. Failure to acquire the required OS lock
+must not cause a write to continue without it.
+
+## A lock does not validate an earlier read
+
+Each operation loads the current map, checks a supplied `expectedRevision`,
+applies its changes and saves while holding the project lock. This prevents
+two cooperating writers from both saving independently loaded copies at once.
+The lock does not remain held across separate MCP read and write calls.
+
+When an edit depends on data returned by `mmap_read`, include that page's
+revision in the write. If another operation changed the map in between, the
+write returns `CONFLICT` without applying the edit. Read again, reconsider the
+change, and submit it with the new revision. Use `expectedRevision: "absent"`
+when declaring a page that must not already exist.
+
+Omitting `expectedRevision` preserves compatibility, but skips that comparison.
+For example, if two sessions read the same `context`, revise different parts,
+and then write their complete objects without a revision, the later write
+replaces the earlier one. The project lock serializes both writes; it cannot
+infer which fields the caller intended to preserve.
+
+## Scope
+
+The lock is a contract among participating writers. Hand edits, direct calls
+to low-level `saveMapFile`, and older running processes do not acquire this
+new lock. A filesystem must support the OS lock used by the runtime; file
+replacement alone is not an alternative locking protocol.
+
+The bundled native bindings require Node.js `^18.17.0 || >=20.3.0` (N-API 9).
+Releases include bindings for Windows 10+ (Server 2016+ on x64), macOS 13.0+
+and Linux with glibc 2.28+, each on x64 and arm64; users do not compile them
+during installation. A missing binding or unsupported platform returns
+`LOCK_UNAVAILABLE` instead of falling back to directory locks or continuing
+without a lock.
+
+macOS 13.0 is the deployment target recorded in both shipped Mach-O addons.
+The OS must also meet the selected Node.js version's requirements: for example,
+[Node.js 24 requires macOS 13.5+](https://github.com/nodejs/node/blob/v24.0.0/BUILDING.md#platform-list).
+The Linux addons require only `libc.so.6`, with maximum symbol versions
+`GLIBC_2.14` on x64 and `GLIBC_2.17` on arm64; they have no `GLIBCXX` or `CXXABI`
+dependency. The higher glibc 2.28 baseline comes from the supported official
+Node.js binaries, which also require kernel 4.18+ and libstdc++ providing
+`GLIBCXX_3.4.25`; see the [Node.js 18.17 platform requirements](https://github.com/nodejs/node/blob/v18.17.0/BUILDING.md#platform-list).
+The Windows addons import system `KERNEL32.dll` and `ntdll.dll`; the supported
+Windows floor follows Node.js, rather than the older PE header version.
+
+Pages organize separate efforts, but share the project lock. This does not
+turn legacy `mmap_remove {pages:[...]}` into a multi-page transaction: that API
+still reports partial deletion and does not accept `expectedRevision`. Use
+per-page `deletePage` for revision-checked deletion. See the
+[persistent-map API guide](map-api.md) for the data contract.
+
+## Upgrade from directory locks
+
+Earlier runtimes used a directory named `.mellos/.write-lock`, containing
+`owner.json` and sometimes `.reap`. New writers use a regular file at that same
+path. If they find a directory there, they return `LOCK_MIGRATION_REQUIRED`.
+They do not reclaim it automatically, even when the recorded PID appears dead
+or its timestamp is old.
+
+1. Stop every old MCP server, HTTP viewer service and native watcher using this
+   project. Updating files or opening a new browser tab does not stop those
+   processes. Stop and restart existing HTTP services explicitly; there is no
+   hot upgrade of the locking protocol.
+   New clients check the viewer's health response for `lockProtocol:
+   "os-file-v1"`; an older service returns a migration error on reuse rather
+   than silently providing a viewer with different write semantics. Use
+   `mellos-mapping-web <project-directory> --stop` before reopening it.
+2. Inspect `.mellos/.write-lock`. If it is a **directory**, and all old writers
+   have stopped, move that directory to a separate backup location. Preserve
+   it for diagnosis. If it is already a **regular file**, leave it in place:
+   it belongs to the new protocol and must not be deleted or renamed. If the
+   path is absent, no old directory needs moving.
+3. Start the updated runtime and retry the operation. It creates the permanent
+   regular file if absent, acquires the OS lock and then performs the write.
+   Reconnect viewer tabs using the URL returned by the restarted service.
+
+Using the same path prevents old and new writers from silently choosing
+different locks. The new runtime's atomic file creation competes with the old
+runtime's directory creation. If the old directory wins, the new runtime
+requires migration. If the regular file wins, an old writer cannot create its
+directory or read `owner.json` beneath the file and returns `BUSY` instead of
+writing. Do not remove the new file to make an old client proceed; upgrade and
+restart that client.
+
+Do not automate this migration by deleting everything named `.write-lock`.
+The old directory and the new permanent file require different treatment.

package/docs/map-api.md

@@ -64,20 +64,35 @@ a group does not silently move members: include their intended node moves in
 the same update or transaction. Normal single-field changes keep the old graph
 constraints; use a batch for changes that require a coordinated final graph.
 
-Every graph writer in the current MCP, HTTP viewer and watcher uses a cooperative
-cross-process project lock. MCP expectedRevision is compared inside that lock
-before loading the proposed changes into the saved map. `CONFLICT` requires a
-fresh read and reconsideration of the edit. `BUSY` means another transaction is
-active; retry after it completes. There is no background lock polling. Locks are
-released on normal completion/error; a confirmed dead PID can be recovered.
-An incomplete owner file after an abrupt crash is refused for manual inspection,
-not guessed stale from its age.
+Every graph writer in the current MCP, HTTP viewer and watcher uses the same
+`withStoreLock` boundary. It takes a non-blocking OS exclusive lock on one fixed
+regular file, `.mellos/.write-lock`; named pages and the default page share it.
+That file is separate from the map JSON files and is never removed or renamed
+as part of locking. The read, revision check, change and save happen while the
+lock is held.
+Contention returns `BUSY`; retry after the other operation completes. There is
+no background lock polling. `LOCK_MIGRATION_REQUIRED` means the lock path is
+still a directory from the old protocol; `LOCK_UNAVAILABLE` means the required
+native binding or supported platform is unavailable. Neither error permits an
+unlocked write. Normal completion/error releases the lock, and the OS also
+releases it when the owning process exits or crashes. A leftover lock
+file is expected and does not indicate a live or stale owner.
+
+MCP `expectedRevision` is optional. When supplied, it is compared inside the
+lock; `CONFLICT` requires a fresh read and reconsideration of the edit. Omitting
+it still serializes the operation against other writers, but does not check
+whether its inputs came from an older read. A later update can overwrite the
+same field or replace an entire `context` or `sources` value. Pass the revision
+from the page read whenever an edit depends on that read.
 
 Low-level library saveMapFile, hand edits, and older running processes do not
-participate in this contract. Restart MCP processes and native watchers after
-upgrading. Opening a Web surface upgrades services that lack format-2 support;
-existing tabs must reconnect with the newly returned URL. Atomic file
-replacement alone does not make a caller's stale read/modify/write safe.
+participate in this contract. Atomic file replacement alone does not make a
+caller's stale read/modify/write safe. Stop all old MCP, HTTP viewer and watcher
+processes for the project before upgrading; restart the HTTP service as well
+as reopening its tabs. A leftover old lock directory requires the manual
+[migration steps](locking.md#upgrade-from-directory-locks). There is no hot
+upgrade of the lock protocol, and supporting map format 2 alone does not
+establish that all writers use the new lock.
 
 Legacy `mmap_remove {pages:[...]}` remains a separately documented batch of file
 deletions, with explicit partial success and historical reference behavior. It
@@ -106,7 +121,8 @@ and the page is saved once; a refusal preserves the original file bytes.
 ```
 
 Machine-readable error codes include NOT_FOUND, INVALID_STORE, REFUSED,
-CONFLICT, BUSY, INVALID_CURSOR, INVALID_ARGUMENT, REFERENCED and SAVE_FAILED.
+CONFLICT, BUSY, LOCK_MIGRATION_REQUIRED, LOCK_UNAVAILABLE, INVALID_CURSOR,
+INVALID_ARGUMENT, REFERENCED and SAVE_FAILED.
 Malformed schema inputs remain MCP invalid-argument errors. Read calls do not
 open panels or create page files.
 

package/lib/store/migration.js

@@ -1,7 +1,8 @@
-import { existsSync, mkdirSync, renameSync } from 'node:fs';
+import { existsSync, renameSync } from 'node:fs';
 import { dirname, join } from 'node:path';
 import { PAGES_DIR_NAME } from './pages.js';
 import { configFilePath } from './policy.js';
+import { withStoreLock } from './transaction.js';
 // ---------------------------------------------------------------------------
 // legacy migration — stores written under the old host-coupled location
 // ---------------------------------------------------------------------------
@@ -29,18 +30,24 @@ export function migrateLegacyStore(defaultFile) {
     const legacyDefault = join(projectRoot, LEGACY_STATE_FILE_RELATIVE_PATH);
     const legacyPages = join(dirname(legacyDefault), LEGACY_PAGES_DIR_NAME);
     const legacyConfig = join(dirname(legacyDefault), LEGACY_CONFIG_FILE_NAME);
-    const hasLegacy = existsSync(legacyDefault) || existsSync(legacyPages) || existsSync(legacyConfig);
-    const hasCurrent = existsSync(defaultFile)
+    const hasLegacy = () => existsSync(legacyDefault) || existsSync(legacyPages) || existsSync(legacyConfig);
+    const hasCurrent = () => existsSync(defaultFile)
         || existsSync(join(dirname(defaultFile), PAGES_DIR_NAME))
         || existsSync(configFilePath(defaultFile));
-    if (!hasLegacy || hasCurrent)
+    if (!hasLegacy() || hasCurrent())
         return false;
-    mkdirSync(dirname(defaultFile), { recursive: true });
-    if (existsSync(legacyDefault))
-        renameSync(legacyDefault, defaultFile);
-    if (existsSync(legacyPages))
-        renameSync(legacyPages, join(dirname(defaultFile), PAGES_DIR_NAME));
-    if (existsSync(legacyConfig))
-        renameSync(legacyConfig, configFilePath(defaultFile));
-    return true;
+    return withStoreLock(defaultFile, () => {
+        // A writer or another startup may have committed after the fast checks.
+        // Recheck under the same lock used by all current project mutations;
+        // rename is allowed to replace its destination on every supported OS.
+        if (!hasLegacy() || hasCurrent())
+            return false;
+        if (existsSync(legacyDefault))
+            renameSync(legacyDefault, defaultFile);
+        if (existsSync(legacyPages))
+            renameSync(legacyPages, join(dirname(defaultFile), PAGES_DIR_NAME));
+        if (existsSync(legacyConfig))
+            renameSync(legacyConfig, configFilePath(defaultFile));
+        return true;
+    });
 }

package/lib/store/native-lock.d.ts

@@ -0,0 +1,5 @@
+export interface NativeLock {
+    tryLock(fd: number): boolean;
+    unlock(fd: number): void;
+}
+export declare function nativeLock(): NativeLock;

package/lib/store/native-lock.js

@@ -0,0 +1,27 @@
+/** Load the shipped N-API implementation without a runtime npm dependency. */
+import { existsSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { fileURLToPath } from 'node:url';
+let loaded;
+export function nativeLock() {
+    if (loaded)
+        return loaded;
+    if (Number(process.versions.napi ?? 0) < 9) {
+        throw new Error('Project locks require Node-API 9 (Node 18.17+ or 20.3+).');
+    }
+    // Bundled entry points live in dist/; source and npm library modules live
+    // two directories below the package root. Never resolve from the user's cwd.
+    const candidates = [
+        new URL('./native-lock.cjs', import.meta.url),
+        new URL('../../dist/native-lock.cjs', import.meta.url),
+    ];
+    const entry = candidates.find(candidate => existsSync(candidate));
+    if (!entry)
+        throw new Error('The installed package is missing dist/native-lock.cjs; reinstall the complete package.');
+    const backend = createRequire(import.meta.url)(fileURLToPath(entry));
+    if (typeof backend.tryLock !== 'function' || typeof backend.unlock !== 'function') {
+        throw new Error('The installed native lock backend is invalid.');
+    }
+    loaded = backend;
+    return backend;
+}

package/lib/store/transaction.d.ts

@@ -1,4 +1,6 @@
 import type { MellosMap } from '../domain/types.js';
+/** Viewer health must identify the writer protocol before a new client reuses it. */
+export declare const STORE_LOCK_PROTOCOL = "os-file-v1";
 export declare class LedgerError extends Error {
     readonly code: string;
     readonly details: Record<string, unknown>;
@@ -8,5 +10,10 @@ export declare const revisionOf: (map: MellosMap) => string;
 export declare function assertRevision(actual: string, expected?: string): void;
 /** A named page and the default page share the same project lock. */
 export declare function storeDirectory(file: string): string;
-/** No timed polling: contention is an explicit retryable BUSY result. Never steal a live lock. */
-export declare function withStoreLock<T>(file: string, action: () => T): T;
+/**
+ * A synchronous transaction on a permanent, independent OS lock file.
+ * The action must finish synchronously; all cooperating writers use this entry.
+ * Never unlink/rename this file: another opener must reach the same lock object.
+ * Contention is BUSY immediately, with no timer, stale lease or PID recovery.
+ */
+export declare function withStoreLock<T>(file: string, action: () => T & (T extends PromiseLike<unknown> ? never : unknown)): T;

package/lib/store/transaction.js

@@ -1,8 +1,11 @@
 /** Cooperative, cross-process transactions for MCP and HTTP writers. */
-import { mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
-import { dirname, join } from 'node:path';
-import { randomUUID, createHash } from 'node:crypto';
+import { closeSync, fstatSync, lstatSync, mkdirSync, openSync, realpathSync } from 'node:fs';
+import { basename, dirname, join } from 'node:path';
+import { createHash } from 'node:crypto';
 import { serializeMap } from './format.js';
+import { nativeLock } from './native-lock.js';
+/** Viewer health must identify the writer protocol before a new client reuses it. */
+export const STORE_LOCK_PROTOCOL = 'os-file-v1';
 export class LedgerError extends Error {
     code;
     details;
@@ -20,72 +23,98 @@ export function assertRevision(actual, expected) {
 /** A named page and the default page share the same project lock. */
 export function storeDirectory(file) {
     const dir = dirname(file);
-    return dir.endsWith('/pages') || dir.endsWith('\\pages') ? dirname(dir) : dir;
-}
-function deadOwner(lock) {
+    // Keep a configured pages directory tied to its project even when that
+    // directory itself is a symlink. Case variants must share the lock on
+    // Windows and case-insensitive macOS volumes (and are harmless elsewhere).
+    if (basename(dir).toLowerCase() === 'pages')
+        return dirname(dir);
+    let canonical = dir;
     try {
-        const owner = JSON.parse(readFileSync(join(lock, 'owner.json'), 'utf8'));
-        if (!Number.isSafeInteger(owner.pid) || owner.pid <= 0)
-            return false;
-        try {
-            process.kill(owner.pid, 0);
-            return false;
-        }
-        catch (e) {
-            return e.code === 'ESRCH';
-        }
+        canonical = realpathSync.native(dir);
+    }
+    catch (error) {
+        if (error.code !== 'ENOENT')
+            throw error;
     }
-    catch {
-        return false;
+    // An explicit --file may reach pages through a differently named alias.
+    return basename(canonical).toLowerCase() === 'pages' ? dirname(canonical) : canonical;
+}
+function checkLockPath(lock) {
+    const entry = lstatSync(lock, { throwIfNoEntry: false });
+    if (entry?.isDirectory()) {
+        throw new LedgerError('LOCK_MIGRATION_REQUIRED', `Legacy directory lock at ${lock}. Stop all old MCP servers, viewers and watchers for this project, then move that directory aside for inspection and retry. Never remove the new regular lock file.`, { path: lock });
+    }
+    if (entry && !entry.isFile()) {
+        throw new LedgerError('LOCK_UNAVAILABLE', `Project lock must be a regular file, not a symlink or special file: ${lock}`, { path: lock });
     }
 }
-/** No timed polling: contention is an explicit retryable BUSY result. Never steal a live lock. */
+/**
+ * A synchronous transaction on a permanent, independent OS lock file.
+ * The action must finish synchronously; all cooperating writers use this entry.
+ * Never unlink/rename this file: another opener must reach the same lock object.
+ * Contention is BUSY immediately, with no timer, stale lease or PID recovery.
+ */
 export function withStoreLock(file, action) {
+    let backend;
+    try {
+        backend = nativeLock();
+    }
+    catch (error) {
+        throw new LedgerError('LOCK_UNAVAILABLE', `Cannot load the operating-system lock backend: ${error instanceof Error ? error.message : String(error)}`);
+    }
     const dir = storeDirectory(file);
     mkdirSync(dir, { recursive: true });
     const lock = join(dir, '.write-lock');
-    const owner = join(lock, 'owner.json');
-    const token = randomUUID();
+    checkLockPath(lock);
+    let fd;
+    // a+ atomically opens/creates without truncation. At a legacy/new startup
+    // race, either old mkdir wins (we refuse its directory), or our file wins
+    // (old mkdir gets EEXIST and cannot find owner.json, so refuses to write).
     try {
-        mkdirSync(lock);
+        fd = openSync(lock, 'a+', 0o600);
     }
     catch (error) {
-        if (error.code !== 'EEXIST')
-            throw error;
-        // Reapers serialize inside the old directory and recheck its owner after acquiring.
-        // Missing/invalid owners are deliberately not reclaimed on an age heuristic.
-        if (!deadOwner(lock))
-            throw new LedgerError('BUSY', `Another writer owns ${lock}; retry after it completes. An orphan without owner metadata needs manual inspection.`);
+        checkLockPath(lock);
+        throw new LedgerError('LOCK_UNAVAILABLE', `Cannot open project lock ${lock}: ${error instanceof Error ? error.message : String(error)}`, { path: lock });
+    }
+    let acquired = false;
+    try {
+        checkLockPath(lock);
+        if (!fstatSync(fd).isFile())
+            throw new LedgerError('LOCK_UNAVAILABLE', `Project lock is not a regular file: ${lock}`);
         try {
-            mkdirSync(join(lock, '.reap'));
-        }
-        catch {
-            throw new LedgerError('BUSY', 'Another process is recovering the writer lock.');
+            acquired = backend.tryLock(fd);
         }
-        if (!deadOwner(lock)) {
-            rmSync(join(lock, '.reap'), { recursive: true, force: true });
-            throw new LedgerError('BUSY', 'Writer ownership changed.');
+        catch (error) {
+            throw new LedgerError('LOCK_UNAVAILABLE', `Cannot acquire project lock ${lock}: ${error instanceof Error ? error.message : String(error)}`, { path: lock });
         }
-        rmSync(lock, { recursive: true });
+        if (!acquired)
+            throw new LedgerError('BUSY', `Another writer owns ${lock}; retry after it completes.`);
+        return action();
+    }
+    finally {
+        // Closing also releases this descriptor's lock if explicit unlock fails.
+        // Cleanup must not misreport an already committed map as unsaved.
         try {
-            mkdirSync(lock);
+            if (acquired)
+                backend.unlock(fd);
         }
         catch {
-            throw new LedgerError('BUSY', 'Another writer acquired the recovered lock.');
+            warnLockCleanup(`Could not explicitly unlock ${lock}; closing its file handle.`);
+        }
+        finally {
+            try {
+                closeSync(fd);
+            }
+            catch {
+                warnLockCleanup(`Could not close the project lock handle for ${lock}; restart this process before retrying writes.`);
+            }
         }
     }
-    let initialized = false;
+}
+function warnLockCleanup(message) {
     try {
-        writeFileSync(owner, JSON.stringify({ pid: process.pid, token }), { flag: 'wx' });
-        initialized = true;
-        return action();
-    }
-    finally {
-        // Only remove the directory still owned by this invocation.
-        try {
-            if (!initialized || JSON.parse(readFileSync(owner, 'utf8')).token === token)
-                rmSync(lock, { recursive: true });
-        }
-        catch { /* A cleanup error must not misreport a successfully committed map as unsaved. */ }
+        process.stderr.write(`mellos-mapping: ${message}\n`);
     }
+    catch { /* Preserve the action's result. */ }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mellos-mapping",
-  "version": "0.23.0",
+  "version": "0.25.0",
   "mcpName": "io.github.GuangminJu/mellos-mapping",
   "description": "A live layered dependency map for bottom-up development — MCP server + terminal pane. Ghost the design first, then light nodes up from the bottom as they are built and verified.",
   "type": "module",
@@ -71,6 +71,7 @@
     "README.zh-CN.md",
     "docs/codex.md",
     "docs/map-api.md",
+    "docs/locking.md",
     "scripts/codex-register.mjs",
     "scripts/codex-cli.mjs",
     "scripts/install-mmap-command.mjs",
@@ -82,7 +83,7 @@
     "scripts/watcher-command.mjs"
   ],
   "engines": {
-    "node": ">=18"
+    "node": "^18.17.0 || >=20.3.0"
   },
   "scripts": {
     "test": "vitest run",
@@ -94,9 +95,10 @@
     "check:package": "node scripts/check-package-surface.mjs",
     "check:codex": "node scripts/check-codex-package.mjs",
     "check:reuse": "node scripts/check-reuse.mjs",
+    "check:locks": "node scripts/check-store-lock.mjs",
     "benchmark:render": "node scripts/benchmark-render.mjs",
     "prepack": "npm run build",
-    "verify": "npm run typecheck && npm run test && npm run build && npm run check:reuse && npm run check:package && npm run check:codex && npm run check:release"
+    "verify": "npm run typecheck && npm run test && npm run build && npm run check:locks && npm run check:reuse && npm run check:package && npm run check:codex && npm run check:release"
   },
   "devDependencies": {
     "@modelcontextprotocol/sdk": "^1.12.0",
@@ -105,6 +107,7 @@
     "@xterm/addon-fit": "0.11.0",
     "@xterm/xterm": "6.0.0",
     "esbuild": "^0.25.0",
+    "fs-native-extensions": "1.5.1",
     "typescript": "^5.8.0",
     "vitest": "^4.1.11",
     "ws": "8.21.3",