@ngockhoale/ukit 2.3.6 → 2.3.8

package/CHANGELOG.md

@@ -2,6 +2,88 @@
 
 All notable changes to UKit are documented here.
 
+## 2.3.8 - 2026-09-10
+
+Freeze-sweep wave 4: every fix in this release targets the "agent silently stops working"
+class — lost permission rules that turn into human-waiting prompts, a session start that
+wipes a sibling session's near-cap warnings, torn state files, and lost concurrency
+bookkeeping. All found by verified reproducers and fixed with regression coverage.
+
+**P1 — SessionStart wiped a sibling session's compact-pressure state.** The SessionStart
+reset hook deleted the whole project-wide `compact-pressure.json`, so starting a second
+session in the same project zeroed the first session's pressure history: its near-cap
+warnings stopped firing, it ran into the real context cap, and it stalled. Pressure state
+is now a session-keyed document (`{"v":2,"sessions":{<session_id>:{…}}}`) with the newest
+record's flat fields still projected at the top level for raw readers. Every writer and
+reader threads the hook payload's `session_id` (compact-threshold CLI, output-compression,
+reinject-context PreCompact, context-hardcap-gate); a brand-new session id starts clean
+and never inherits a sibling's totals; the reset hook removes only the calling session's
+record; sessions are pruned to the 8 most recent; a legacy flat file upgrades in place.
+Ships in both `src/core/compact/threshold.js` and the runtime mirror.
+
+**P1 — concurrent auto-allow invocations lost Bash permission rules.** The hook's
+read-modify-write of `settings.local.json` and `permission-usage.json` was unlocked, and
+30 concurrent invocations kept only 26/30 rules and 23/30 usage records — a lost rule
+means the next run of that command prompts for permission again, i.e. the agent sits
+waiting for a human click. Both files are now written inside one mkdir lock on
+`<permission-usage.json>.lock` (protocol-compatible with `withFileLock`, fail-open ≤5s,
+parent directory created before locking) with atomic tmp+rename writes, so Claude Code
+can never observe a torn settings file either — a torn parse would drop the entire allow
+list and prompt on every command. `auto-prune-bash.sh` now shares the same lock and
+atomic writes.
+
+**P2 — index cache writes lost entries under concurrency.** `cache-utils.mjs`
+read-modify-write cycles were unlocked: 20 concurrent writers kept 2 entries. All cache
+reads-modify-writes (write + touch paths) now run under `withFileLock` with atomic
+compact writes (cache files stay single-line JSON by contract).
+
+**P2 — verification-progress and execution-ledger counters lost concurrent updates.**
+`verification-guard.sh` now records attempts through a locked re-read + merge (sync
+`Atomics.wait` backoff so the hook's `process.exit` flow is untouched) and writes
+atomically; `execution-ledger.mjs`'s continuation/notified counters re-read inside
+`withFileLock` so parallel subagent Stop hooks cannot reset each other's budgets.
+
+Full suite: 81 files, 1,373 tests green, including two shuffled runs.
+
+## 2.3.7 - 2026-09-10
+
+Bug-sweep wave 3: a data-loss escape in uninstall, cross-process races on shared runtime
+state, one flaky test, and one vulnerable runtime dependency — all found by verified
+reproducers and fixed with regression coverage.
+
+**P0 — uninstall could delete files outside the project root.** Tracked-path ownership
+checks were purely lexical: with `.claude` (or any parent directory) symlinked to an
+external location, uninstall's recursive remove resolved through the symlink and deleted
+whatever it pointed at — e.g. a shared dotfiles directory. Uninstall now inspects every
+directory component between the project root and each tracked entry (the entry itself may
+still be a tracked symlink, which is unlinked, never followed); entries with a symlinked
+parent are refused, warned, and reported in a new `skippedSymlinkParents` field in both
+dry-run and real results.
+
+**P1 — concurrent state mutations silently lost updates.** `appendOutputHistory` and the
+compact-pressure writers (prompt, output, plan, and the plain state write) performed
+unsynchronized read-modify-write cycles on files under `.ukit/storage/cache/`. With
+parallel subagents firing hooks in separate processes, 30 concurrent updates left a single
+surviving output-history entry (of a 25 cap) and 17 of 510 accumulated sessionTokens —
+understating context pressure and delaying compaction. All mutations now run under
+`withFileLock`, a directory-lock protocol beside each state file (atomic `mkdir`, stale
+reclaim after 10s, fail-open after 5s so a stuck lock can never freeze a hook). The lock
+ships in both `src/core/fileOps.js` and the runtime mirror `token-utils.mjs` using the
+same `<file>.lock` path, so CLI processes and hook processes serialize against each
+other; mirror state writes also became atomic (tmp + rename). An interop test runs both
+implementations against one project root to keep the protocols locked together.
+
+**Flaky test** — the `--no-outline` index-tools test queued a `getFileOutline` once-mock
+its own flag guaranteed would never be consumed; `vi.clearAllMocks()` drains no once-queues,
+so test order leaked the stale value into the next test (18/30 shuffled solo runs failed).
+The dead queue is gone; the full suite is now 3/3 green under `--sequence.shuffle`.
+
+**Dependency** — runtime dependency `yaml` 2.8.2 → 2.9.0 (GHSA-48c2-rrv3-qjmp, stack
+overflow via deeply nested YAML — UKit parses project-supplied manifests). Production
+dependency audit is now 0 findings.
+
+Full suite: 78 files, 1,359 tests green.
+
 ## 2.3.6 - 2026-09-10
 
 Anti-stall coverage parity release: the compact-recovery and stop-reason directives that

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ngockhoale/ukit",
-  "version": "2.3.6",
+  "version": "2.3.8",
   "description": "Install/update an index-first AI workspace for Claude Code, OpenAI Codex, OpenCode, and omp (Oh My Pi).",
   "license": "MIT",
   "type": "module",
@@ -65,7 +65,7 @@
     "node": ">=20"
   },
   "dependencies": {
-    "yaml": "^2.8.2"
+    "yaml": "^2.9.0"
   },
   "devDependencies": {
     "vitest": "^2"

package/src/core/compact/threshold.js

@@ -1,4 +1,4 @@
-import { readJsonIfExists, writeJson } from '../fileOps.js';
+import { readJsonIfExists, writeJson, withFileLock } from '../fileOps.js';
 import { buildRuntimePaths } from '../runtimePaths.js';
 import { buildCompactMachineKey, compressLine, estimateTokenCount } from '../token/index.js';
 import { compactContextBlock } from './index.js';
@@ -553,7 +553,72 @@ export function resolveThresholdCompactBudget({
   };
 }
 
+// ---- session-scoped pressure document ---------------------------------------
+// compact-pressure.json holds ONE record per session id. A session starting must never
+// zero another live session's pressure bookkeeping (its hard-cap gate and compact
+// advisories), and a brand-new session must not inherit a sibling's totals. The newest
+// record's flat fields stay projected at the document's top level so readers that predate
+// this shape (raw JSON readers) still see a coherent state.
+// Mirrors templates/.claude/ukit/runtime/compact-threshold.mjs — keep in lockstep.
+const PRESSURE_SESSIONS_MAX = 8;
+
+function normalizeSessionId(value) {
+  const id = typeof value === 'string' ? value.trim() : '';
+  return id || null;
+}
+
+function readPressureDocument(raw) {
+  if (raw && typeof raw === 'object' && raw.sessions && typeof raw.sessions === 'object') {
+    const sessions = {};
+    for (const [id, record] of Object.entries(raw.sessions)) {
+      if (id && record && typeof record === 'object') {
+        sessions[id] = record;
+      }
+    }
+    return sessions;
+  }
+  // Legacy single-record file: preserve the in-flight session under a shared bucket.
+  return raw && typeof raw === 'object' ? { default: raw } : {};
+}
+
+function pickPressureSession(sessions, sessionId) {
+  if (sessionId) {
+    // A session id that has no record yet starts clean — it must not inherit another
+    // session's totals, and the other way round nothing here touches that session.
+    return { id: sessionId, record: sessions[sessionId] ?? null };
+  }
+  const entries = Object.entries(sessions);
+  if (!entries.length) {
+    return { id: 'default', record: null };
+  }
+  const newest = entries
+    .sort(([, left], [, right]) => finiteNumber(right?.updatedAt, 0) - finiteNumber(left?.updatedAt, 0))[0];
+  return { id: newest[0], record: newest[1] };
+}
+
+function resolvePressureRecord(raw, config) {
+  if (!raw || typeof raw !== 'object' || !raw.sessions || typeof raw.sessions !== 'object') {
+    return raw; // flat legacy state — normalized as-is below
+  }
+  const { record } = pickPressureSession(readPressureDocument(raw), normalizeSessionId(config?.sessionId));
+  return record;
+}
+
+function projectPressureDocument(sessions) {
+  const newestFirst = Object.entries(sessions)
+    .sort(([, left], [, right]) => finiteNumber(right?.updatedAt, 0) - finiteNumber(left?.updatedAt, 0))
+    .slice(0, PRESSURE_SESSIONS_MAX);
+  const newestRecord = newestFirst[0]?.[1] ?? null;
+  return {
+    ...(newestRecord && typeof newestRecord === 'object' ? newestRecord : {}),
+    updatedAt: finiteNumber(newestRecord?.updatedAt, Date.now()),
+    v: 2,
+    sessions: Object.fromEntries(newestFirst),
+  };
+}
+
 export function buildCompactPressureState(rawState = null, config = {}) {
+  rawState = resolvePressureRecord(rawState, config);
   const thresholds = buildCompactThresholds(config);
   const rawSoftThreshold = finiteNumber(rawState?.softThreshold, 0);
   const rawHardThreshold = finiteNumber(rawState?.hardThreshold, 0);
@@ -937,27 +1002,62 @@ export async function readCompactPressureState(projectRoot, config = {}) {
   return buildCompactPressureState(await readJsonIfExists(runtimePaths.compactPressurePath), config);
 }
 
+// All compact-pressure mutations share one lock on the state file: without it,
+// concurrent hook processes (parallel subagents) and same-process flows interleave
+// their read-modify-write cycles and silently drop each other's sessionTokens.
+// Mutations are session-scoped: only the calling session's record is rewritten;
+// sibling session records pass through untouched.
+async function mutateCompactPressureState(projectRoot, mutator, config = {}) {
+  const runtimePaths = buildRuntimePaths(projectRoot);
+  return withFileLock(runtimePaths.compactPressurePath, async () => {
+    const sessions = readPressureDocument(await readJsonIfExists(runtimePaths.compactPressurePath));
+    const { id, record } = pickPressureSession(sessions, normalizeSessionId(config?.sessionId));
+    const current = buildCompactPressureState(record, config);
+    const next = mutator(current);
+    const normalized = buildCompactPressureState(next, config);
+    sessions[id] = { ...normalized, updatedAt: Date.now() };
+    await writeJson(runtimePaths.compactPressurePath, projectPressureDocument(sessions));
+    return normalized;
+  });
+}
+
 export async function writeCompactPressureState(projectRoot, state, config = {}) {
   const runtimePaths = buildRuntimePaths(projectRoot);
-  const normalized = buildCompactPressureState(state, config);
-  await writeJson(runtimePaths.compactPressurePath, normalized);
-  return normalized;
+  return withFileLock(runtimePaths.compactPressurePath, async () => {
+    const sessions = readPressureDocument(await readJsonIfExists(runtimePaths.compactPressurePath));
+    const { id } = pickPressureSession(sessions, normalizeSessionId(config?.sessionId));
+    const normalized = buildCompactPressureState(state, config);
+    sessions[id] = { ...normalized, updatedAt: Date.now() };
+    await writeJson(runtimePaths.compactPressurePath, projectPressureDocument(sessions));
+    return normalized;
+  });
+}
+
+function pressureSessionConfig(payload = {}, config = {}) {
+  const sessionId = normalizeSessionId(payload?.sessionId) ?? normalizeSessionId(config?.sessionId);
+  return sessionId ? { ...config, sessionId } : config;
 }
 
 export async function updateCompactPressureFromPrompt(projectRoot, payload, config = {}) {
-  const current = await readCompactPressureState(projectRoot, config);
-  const next = registerPromptPressure(current, payload, config);
-  return writeCompactPressureState(projectRoot, next, config);
+  return mutateCompactPressureState(
+    projectRoot,
+    (current) => registerPromptPressure(current, payload, config),
+    pressureSessionConfig(payload, config),
+  );
 }
 
 export async function updateCompactPressureFromOutput(projectRoot, payload, config = {}) {
-  const current = await readCompactPressureState(projectRoot, config);
-  const next = registerOutputPressure(current, payload, config);
-  return writeCompactPressureState(projectRoot, next, config);
+  return mutateCompactPressureState(
+    projectRoot,
+    (current) => registerOutputPressure(current, payload, config),
+    pressureSessionConfig(payload, config),
+  );
 }
 
 export async function writeThresholdCompactPlan(projectRoot, plan, config = {}) {
-  const current = await readCompactPressureState(projectRoot, config);
-  const next = registerThresholdCompactPlan(current, plan, config);
-  return writeCompactPressureState(projectRoot, next, config);
+  return mutateCompactPressureState(
+    projectRoot,
+    (current) => registerThresholdCompactPlan(current, plan, config),
+    pressureSessionConfig(plan, config),
+  );
 }

package/src/core/fileOps.js

@@ -131,6 +131,75 @@ export async function writeJson(filePath, data) {
   await writeFileAtomic(filePath, `${JSON.stringify(data, null, 2)}\n`);
 }
 
+const LOCK_STALE_MS = 10_000;
+const LOCK_MAX_WAIT_MS = 5_000;
+
+function lockBackoffDelayMs() {
+  return 3 + Math.floor(Math.random() * 9);
+}
+
+function sleep(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+/**
+ * Serialize read-modify-write mutations of a shared state file — across processes
+ * (hook invocations run as separate node processes) and across concurrent async
+ * flows in one process (parallel subagents). The lock is a directory created next
+ * to the target file: `mkdir` is atomic, so exactly one caller can create it.
+ * A crashed holder is reclaimed once the directory's mtime exceeds staleMs.
+ * Liveness wins over strictness: if the lock cannot be acquired within maxWaitMs
+ * the callback runs anyway (the pre-lock behaviour) — these state files are
+ * advisory caches, and losing an update beats freezing a hook mid-flight.
+ * Holders must keep their critical section far below staleMs; nothing refreshes
+ * the lock mtime, so a section that somehow runs longer can have its lock stolen.
+ * @param {string} filePath - state file the mutation targets (lock lives beside it)
+ * @param {() => Promise<*>} fn - critical section; its result is returned
+ * @returns {Promise<*>} whatever fn resolves with
+ */
+export async function withFileLock(filePath, fn, { staleMs = LOCK_STALE_MS, maxWaitMs = LOCK_MAX_WAIT_MS } = {}) {
+  const lockPath = `${filePath}.lock`;
+  const startedAt = Date.now();
+  let locked = false;
+
+  while (!locked) {
+    try {
+      await ensureDir(path.dirname(lockPath));
+      await fs.mkdir(lockPath); // atomic acquire — EEXIST means another holder exists
+      locked = true;
+      break;
+    } catch (error) {
+      if (error?.code !== 'EEXIST') throw error;
+    }
+
+    // Someone holds the lock. Reclaim it when it looks abandoned; otherwise back off.
+    try {
+      const stat = await fs.stat(lockPath);
+      if (Date.now() - stat.mtimeMs > staleMs) {
+        await fs.rm(lockPath, { recursive: true, force: true });
+        continue; // the slot is free now — retry immediately
+      }
+    } catch {
+      continue; // lock vanished between mkdir and stat — retry immediately
+    }
+
+    if (Date.now() - startedAt >= maxWaitMs) break; // fail open — run unlocked
+    await sleep(lockBackoffDelayMs());
+  }
+
+  try {
+    return await fn();
+  } finally {
+    if (locked) {
+      try {
+        await fs.rm(lockPath, { recursive: true, force: true });
+      } catch {
+        // best-effort release; a stale lock is reclaimed by the next waiter
+      }
+    }
+  }
+}
+
 /**
  * Create a directory symlink (macOS/Linux) or junction (Windows).
  * Junctions on Windows don't require elevated privileges.

package/src/core/output/index.js

@@ -1,6 +1,6 @@
 import fs from 'node:fs/promises';
 import path from 'node:path';
-import { readJsonIfExists, writeJson } from '../fileOps.js';
+import { readJsonIfExists, writeJson, withFileLock } from '../fileOps.js';
 import { recordCompaction } from '../compact/index.js';
 import { buildRuntimePaths } from '../runtimePaths.js';
 import {
@@ -1075,16 +1075,21 @@ export async function appendOutputHistory(projectRoot, entry, options = {}) {
   }
 
   const runtimePaths = buildRuntimePaths(projectRoot);
-  const history = await readOutputHistory(projectRoot, options);
-  const normalizedKey = buildOutputHistoryDedupeKey(normalizedEntry);
-  const dedupedExistingEntries = history.entries.filter((candidate) => !(
-    buildOutputHistoryDedupeKey(candidate) === normalizedKey
-  ));
-  const nextDocument = normalizeOutputHistoryDocument({
-    entries: [normalizedEntry, ...dedupedExistingEntries],
-  }, options);
-  await writeJson(runtimePaths.outputHistoryPath, nextDocument);
-  return nextDocument;
+  // The read-merge-write cycle runs under the state-file lock: concurrent tool-output
+  // captures (parallel subagents fire hooks in separate processes) otherwise read the
+  // same old history and overwrite each other, leaving only one survivor entry.
+  return withFileLock(runtimePaths.outputHistoryPath, async () => {
+    const history = await readOutputHistory(projectRoot, options);
+    const normalizedKey = buildOutputHistoryDedupeKey(normalizedEntry);
+    const dedupedExistingEntries = history.entries.filter((candidate) => !(
+      buildOutputHistoryDedupeKey(candidate) === normalizedKey
+    ));
+    const nextDocument = normalizeOutputHistoryDocument({
+      entries: [normalizedEntry, ...dedupedExistingEntries],
+    }, options);
+    await writeJson(runtimePaths.outputHistoryPath, nextDocument);
+    return nextDocument;
+  });
 }
 
 export function summarizeOutputHistory(rawHistory) {

package/src/core/uninstall.js

@@ -64,6 +64,37 @@ function isSameOrDescendantProjectPath(candidatePath, parentPath) {
   return candidatePath === parentPath || candidatePath.startsWith(`${parentPath}/`);
 }
 
+// A tracked path is only safe to delete when every directory between the project root
+// and the entry itself is a real directory. The prefix allowlist above is purely
+// lexical: if `.claude` (or any parent) is a symlink to somewhere outside the project,
+// `fs.rm(recursive)` resolves THROUGH it and would delete whatever it points at —
+// e.g. a shared dotfiles directory linked in as .claude. The entry itself may be a
+// symlink (tracked links are unlinked, never followed); only its parents matter.
+async function hasSymlinkedParent(projectRoot, absolutePath) {
+  const root = path.resolve(projectRoot);
+  const relative = path.relative(root, path.resolve(absolutePath));
+  if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) {
+    // Outside the project root entirely — nothing here is safe to delete.
+    return true;
+  }
+  const segments = relative.split(path.sep);
+  segments.pop(); // the entry itself is allowed to be a symlink (removeLinkOnly unlinks it)
+  let current = root;
+  for (const segment of segments) {
+    current = path.join(current, segment);
+    let stat;
+    try {
+      stat = await fs.lstat(current);
+    } catch {
+      return false; // missing parent — nothing to resolve through
+    }
+    if (stat.isSymbolicLink()) {
+      return true;
+    }
+  }
+  return false;
+}
+
 // Hardcoded fallback for installs that predate file tracking (no 'files' field
 // in install.json). New installs always have a 'files' list, so this fallback
 // only applies when upgrading from a very old UKit version.
@@ -205,18 +236,36 @@ export async function uninstallUkit({ projectRoot, dryRun = false }) {
     ...linkPaths.map((abs) => ({ abs, useLink: true })),
   ];
 
+  // Never delete through a symlinked parent — see hasSymlinkedParent. Skipped entries
+  // are reported (dryRun and real run alike) so the user sees exactly what was refused.
+  const skippedSymlinkParents = [];
+  const safeEntries = [];
+  for (const entry of allEntries) {
+    if (await hasSymlinkedParent(projectRoot, entry.abs)) {
+      skippedSymlinkParents.push(entry.abs);
+    } else {
+      safeEntries.push(entry);
+    }
+  }
+  if (skippedSymlinkParents.length > 0) {
+    console.warn(
+      '[UKit] Refusing to remove paths whose parent directories are symlinks (deleting them would escape the project root):',
+      skippedSymlinkParents,
+    );
+  }
+
   if (dryRun) {
     // Use lstat (not access) so broken symlinks are included in the report.
     const wouldRemove = [];
-    for (const { abs } of allEntries) {
+    for (const { abs } of safeEntries) {
       if (await pathExistsLstat(abs)) wouldRemove.push(abs);
     }
-    return { removed: 0, attempted: allEntries.length, wasInstalled: true, wouldRemove };
+    return { removed: 0, attempted: allEntries.length, wasInstalled: true, wouldRemove, skippedSymlinkParents };
   }
 
   // Remove all paths in parallel
   const results = await Promise.all(
-    allEntries.map(({ abs, useLink }) => {
+    safeEntries.map(({ abs, useLink }) => {
       const remove = useLink ? removeLinkOnly : removeLinkOrDir;
       return remove(abs).then((didRemove) => ({ abs, didRemove }));
     }),
@@ -240,5 +289,5 @@ export async function uninstallUkit({ projectRoot, dryRun = false }) {
     await cleanupEmptyParents(removedPath, projectRoot);
   }
 
-  return { removed, attempted: allEntries.length, wasInstalled: true };
+  return { removed, attempted: allEntries.length, wasInstalled: true, skippedSymlinkParents };
 }