@ngockhoale/ukit 2.7.6 → 2.7.8

package/CHANGELOG.md

@@ -2,6 +2,111 @@
 
 All notable changes to UKit are documented here.
 
+## 2.7.8 - 2026-09-22
+
+Stall/hang fixes — cycle C43 (TASK-001..008): the indefinite-stall producers
+root-caused in `docs/AI_REPORT/2026-09-21-NORMAL_CHAT_STALL.md` are fixed
+across Claude Code, omp, and Codex runtimes.
+
+- **CX-8**: route-state writes are atomic + locked — torn `skill-router-state.json`
+  writes can no longer wedge the router.
+- **OMP-3/F-2**: omp receipt dedupe — a tool result is recorded once; duplicate
+  receipts no longer double-count execution records.
+- **RC-4/OMP-1/F-3/F-4**: breaker-freeze family — budgeted reclaim, pstart
+  stamps, fail-open streaks; continuation sidecar merge gate stamps
+  `lastContinuationAt` strictly newer than its base and materializes counter
+  state when the main ledger file is absent.
+- **F-5/F-5b**: `withFileLock` policy + reclaim — stale locks reclaimed via
+  pstart mismatch; recycled-pid regression tests added.
+- **F-15**: verifier-classification — verification outcomes classified so a
+  non-verification failure no longer masquerades as a test failure.
+- **F-RC-2**: `hook-chain-runner` exits promptly after verdict — bounded stdout
+  flush then `process.exit`; an abandoned in-proc step promise can no longer
+  pin the event loop for the full registered timeout.
+- **F-10/OMP-2**: omp Stop parity — omp sessions get the same handoff-cursor
+  Stop-gate lane as Claude Code.
+- **OMP-6/F-8/CX-10**: `createPayloadReference` staging runs under a deadline
+  with fail-open to inline payload (never throws, host loop not frozen); omp
+  SessionStart carries `session_id` so `reset-compact-pressure` no longer
+  wipes every session's pressure records; codex `requiredBeforeCompletion`
+  renders only commands that exist in `package.json`.
+- Housekeeping: stale "Kilo" tool labels scrubbed from handoff docs, agent
+  report contracts, and config help text (functional support was already
+  removed; the regression tests and gateway comments documenting that removal
+  are intentionally kept).
+
+## 2.7.7 - 2026-09-21
+
+Audit-report remediation + hook observability — cycle C42 (TASK-001..011): all 12
+findings from the three 2026-09-20 reports closed; the reports moved to
+`docs/AI_REPORT/archive/`.
+
+- B1: `hook-chain-runner` short-circuits on a permission decision under
+  `--emit-verdict`. A `hookSpecificOutput` decision on a step now owns the
+  verdict and stops the chain there, so the decision JSON is emitted as the
+  whole stdout instead of being concatenated in front of the next step's
+  output, and a genuinely-skipped fail-closed gate is no longer exempted from
+  the fail-closed verdict. C8 (same task): an invalid `:N` budget suffix
+  (`:0`, `:-1`) warns and runs the step on its default budget instead of
+  silently parsing; a non-numeric suffix stays a literal path.
+- B2: `yarn test <files>` — the repo's dominant verification form — is now
+  recognized by the verification-minutes estimator alongside `yarn vitest
+  [run]`, so task budgets are no longer inflated by the 1-minute flat
+  fallback.
+- B3: doctor strips `{{...}}` placeholders before `parse()`ing the omp config
+  template, so a placeholder in key position no longer makes the `yaml` lib
+  emit "Keys with collection values will be stringified" on every run.
+- C5: new scoped `scripts/install/sync-installed-mirror.mjs` (`--check` reports
+  drift, bare run rewrites). It reuses the real install pipeline
+  (`buildInstallPlan` → `diffInstallPlan` → `applyDiffResults`) so drift is
+  decided by exactly the code `ukit install` uses, and it only ever touches
+  `.claude/**`/`.omp/**` plan targets — never `src/`, `docs/`, or user-authored
+  files. Parity coverage widened to `.claude/ukit/index/`.
+- C7: hook-chain failure surfaces pinned by
+  `tests/hooks/hookChainFailureSurface.test.js` (output overflow on a
+  fail-closed gate, budget exhausted mid-chain, signal death); the decision to
+  drop non-verdict stderr is documented in `UKIT_INTERNALS.md`.
+- O1: telemetry wired into the four previously unmeasured hooks
+  (`reinject-context`, `auto-prune-bash`, `reset-compact-pressure`,
+  `session-episode`) via a no-staging arm path in the `hook-telemetry` engine.
+  Four hooks now emit rows where they emitted none.
+- O2: audit snapshots carry provenance (`generatedAt` + telemetry row/file
+  counts), and `scripts/perf/diff-perf-findings.mjs` diffs two snapshots per
+  finding id without touching live telemetry.
+- O3: telemetry rows split stdin-stage time from execution time (`stageMs`),
+  so a slow producer is distinguishable from a slow step.
+- O4/O6: telemetry consumers audited and the consolidated row shape pinned
+  (`tests/hooks/chainTelemetryRowShape.test.js`); doctor escalates when a
+  failure actually landed on a chain carrying a fail-closed gate or exhausted
+  its budget — purely additive to the existing detail/remedy.
+- O5: telemetry append overhead measured; numbers recorded in
+  `scripts/perf/perf-measure.md`.
+- Fixed `tests/consistency/ompDocsSync.test.js` case 4: the frozen-history
+  guard treated the `RULES.md` §Clear Handoff step-2 archive rotation (delete
+  the oldest cycle, fold it into `HISTORY.md`) as tampering, so the test was
+  red at HEAD and at the `v2.7.2`/`v2.7.5` tags. The guard now exempts a
+  cycle-dir deletion only when a matching `HISTORY.md` row exists and the
+  post-rotation archive is back within the 3-dir cap; rewrites, renames, and
+  unrecorded deletions still fail.
+- Fixed `src/core/memory/store.js`: the session-archive cap appended and
+  `slice(-cap)`ed, so it evicted by insertion position instead of age — a
+  session archived late carrying an older end time outlived a newer one written
+  earlier. It now drops the oldest by end time (ties keep insertion order),
+  matching the key `archiveSessions` already uses. The regression case in
+  `tests/core/sessionArchiveBound.test.js` had a `Date.now()`-per-session
+  fixture whose outcome depended on whether the clock ticked mid-array, so it
+  passed on an unloaded box and failed under load; the fixture is now
+  deterministic and the case fails against the old code.
+- `withFileLock` (both `templates/.claude/ukit/runtime/token-utils.mjs` and its
+  `src/core/fileOps.js` twin) is now fail-closed: a lock wait that exceeds
+  `maxWaitMs` skips the mutation instead of running it unlocked, and the drop is
+  journaled to `<file>.lock-drops.jsonl` (bounded 128-record JSONL). Both twins
+  delegate to `async-lock.mjs`, so owner stamping (pid + token + pstart),
+  recycled-pid detection, and claim+quarantine stale reclaim exist exactly once;
+  a stale lock whose recorded pid was recycled is now reclaimed instead of
+  wedging every mutation for that file. `writeJson`/`writeFileAtomic` fall back
+  to copy+unlink when `rename` fails with EXDEV (cross-mount tmp dirs).
+
 ## 2.7.6 - 2026-09-20
 
 Permission posture work — cycle C41 (TASK-001..004): omp prompts aligned to

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ngockhoale/ukit",
-  "version": "2.7.6",
+  "version": "2.7.8",
   "description": "Install/update an index-first AI workspace for Claude Code, OpenAI Codex and omp (Oh My Pi).",
   "license": "MIT",
   "type": "module",

package/scripts/install/sync-installed-mirror.mjs

@@ -0,0 +1,250 @@
+#!/usr/bin/env node
+// C5 (docs/AI_REPORT/archive/2026-09-20-CORRECTNESS.md §C5) — scoped mirror sync/check.
+//
+// The installed `.claude/` + `.omp/` mirror is a *rendered* copy of `templates/.claude`
+// + `templates/.omp`, and hand-copying it drifted three separate ways this cycle:
+// a template change never re-copied, a raw copy that kept its literal `{{token}}`
+// because nobody rendered it, and `settings.json`'s two gateway-injected env keys,
+// which make whole-file byte identity impossible by construction. Parallel lanes
+// overwrote each other because there was no tool — only whoever remembered.
+//
+//   node scripts/install/sync-installed-mirror.mjs --check   report drift, exit non-zero
+//   node scripts/install/sync-installed-mirror.mjs           rewrite drifted targets
+//
+// This reuses the real install pipeline (`buildInstallPlan` → `diffInstallPlan` →
+// `applyDiffResults` → `applyGatewayResilienceEnv`) rather than reimplementing it, so
+// "drift" is decided by exactly the code `ukit install` uses — including the
+// gateway-env carve-out in `src/core/diffPlan.js` and the render step in
+// `src/render/buildTemplateVariables.js`. Render-tokenized templates are compared in
+// rendered form; `settings.json` is compared section-wise, never byte-wise.
+//
+// Scope is deliberately narrow — the tool only ever considers plan targets that live
+// under `.claude/**` or `.omp/**`, so it can never touch `src/`, `docs/`, or any
+// user-authored file:
+//
+//   * `mergeStrategy: 'skip'` entries (e.g. `.claude/settings.local.json`) are
+//     user-owned, not template-derived. They are excluded from drift AND from apply —
+//     a hands-off file must never read as drift, and this tool is not an installer.
+//   * Mirror-only detection is limited to the target directories of *selected,
+//     non-link* manifest items — directories UKit owns wholesale from `templates/`.
+//     A blanket walk of `.claude/` reports ~226 false positives, because pack-gated
+//     skill libraries (e.g. `.claude/skills/frontend-vue/**` on a `core`-only project)
+//     are present in the mirror but intentionally outside the current plan. Those
+//     directories are covered by `tests/consistency/templateParity.test.js` instead.
+import fs from 'node:fs/promises';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+import { buildPathConfig } from '../../src/core/paths.js';
+import { loadManifest } from '../../src/manifest/loadManifest.js';
+import { detectStack } from '../../src/stack/detectStack.js';
+import { detectProjectContext } from '../../src/context/detectProjectContext.js';
+import { detectProviders } from '../../src/context/detectProviders.js';
+import { loadRuntimeConfig } from '../../src/core/runtimeConfig.js';
+import { buildTemplateVariables } from '../../src/render/buildVariables.js';
+import { buildInstallPlan } from '../../src/core/buildPlan.js';
+import { diffInstallPlan } from '../../src/core/diffPlan.js';
+import { applyDiffResults } from '../../src/core/applyPlan.js';
+import { applyGatewayResilienceEnv } from '../../src/core/gatewayResilienceEnv.js';
+
+const MANAGED_PREFIXES = ['.claude/', '.omp/'];
+const USAGE = 'usage: sync-installed-mirror.mjs [--check]';
+
+function isManaged(relativePath) {
+  return MANAGED_PREFIXES.some((prefix) => relativePath.startsWith(prefix));
+}
+
+function toRelative(projectRoot, targetPath) {
+  return path.relative(projectRoot, targetPath).split(path.sep).join('/');
+}
+
+async function readPackageVersion(packageRoot) {
+  try {
+    const pkg = JSON.parse(await fs.readFile(path.join(packageRoot, 'package.json'), 'utf8'));
+    return typeof pkg.version === 'string' ? pkg.version : null;
+  } catch {
+    return null;
+  }
+}
+
+// The full set of paths the plan manages, used to decide whether a file found in the
+// mirror is unmanaged. Built from every entry (not just the managed ones) so a path
+// that is managed by an out-of-scope entry is never reported as mirror-only.
+function buildPlanTargetSet(plan, projectRoot) {
+  return new Set(
+    plan.entries
+      .map((entry) => toRelative(projectRoot, entry.targetPath))
+      .filter(Boolean),
+  );
+}
+
+async function collectFilesRecursively(dir, out) {
+  let entries;
+  try {
+    entries = await fs.readdir(dir, { withFileTypes: true });
+  } catch {
+    return out; // missing or unreadable directory — the create-drift lane covers it
+  }
+
+  for (const entry of entries) {
+    const fullPath = path.join(dir, entry.name);
+    // Symlinks are the adapter-link lane's business (`type: 'link'` items are excluded
+    // from mirror-only detection), and following one would walk another tool's tree.
+    if (entry.isSymbolicLink()) continue;
+    if (entry.isDirectory()) {
+      await collectFilesRecursively(fullPath, out);
+      continue;
+    }
+    if (entry.isFile()) out.push(fullPath);
+  }
+
+  return out;
+}
+
+// Directories UKit installs wholesale from a template directory. Any file inside one
+// that the plan does not ship is mirror-only drift rather than user content.
+async function collectManagedDirectoryRoots({ plan, templatesRoot }) {
+  const roots = [];
+
+  for (const item of plan.selectedItems) {
+    if (item.type === 'link') continue;
+    const target = String(item.targetPath ?? '');
+    if (!isManaged(target)) continue;
+
+    const source = path.join(templatesRoot, String(item.sourceTemplate ?? ''));
+    let stat;
+    try {
+      stat = await fs.stat(source);
+    } catch {
+      continue;
+    }
+    if (stat.isDirectory()) roots.push(target.replace(/\/+$/, ''));
+  }
+
+  return roots;
+}
+
+async function collectMirrorOnlyPaths({ plan, projectRoot, templatesRoot }) {
+  const planTargets = buildPlanTargetSet(plan, projectRoot);
+  const roots = await collectManagedDirectoryRoots({ plan, templatesRoot });
+  const mirrorOnly = [];
+
+  for (const root of roots) {
+    const files = await collectFilesRecursively(path.join(projectRoot, root), []);
+    for (const file of files) {
+      const relativePath = toRelative(projectRoot, file);
+      if (!planTargets.has(relativePath)) mirrorOnly.push(relativePath);
+    }
+  }
+
+  return [...new Set(mirrorOnly)].sort();
+}
+
+async function main() {
+  const args = process.argv.slice(2);
+  const unknown = args.filter((arg) => arg !== '--check');
+  if (unknown.length > 0) {
+    console.error(`${USAGE}\nunknown flag: ${unknown[0]}`);
+    process.exit(2);
+  }
+
+  const checkOnly = args.includes('--check');
+  const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');
+  const projectRoot = process.cwd();
+  const pathConfig = buildPathConfig({ packageRoot, projectRoot });
+
+  const manifest = await loadManifest(pathConfig.manifestPath);
+  const stackContext = await detectStack(projectRoot);
+  const projectContext = await detectProjectContext(projectRoot);
+  const providerContext = await detectProviders(projectRoot);
+  const runtimeConfig = await loadRuntimeConfig(projectRoot);
+
+  // Same variable set as `runInstallPipeline`, so render-tokenized templates compare
+  // in the exact form `ukit install` would have written them.
+  const variables = buildTemplateVariables({
+    projectContext,
+    stackContext,
+    packageVersion: await readPackageVersion(packageRoot),
+    providerContext,
+    runtimeConfig,
+  });
+
+  const plan = await buildInstallPlan({
+    manifest,
+    stackContext,
+    templatesRoot: pathConfig.templatesRoot,
+    variables,
+    projectRoot,
+    // Omitted: `ukit install`'s own default adapter set (claude + codex + omp), so the
+    // `.omp/**` half of the mirror is in scope.
+    selectedAdapterItemIds: undefined,
+  });
+
+  const managedEntries = plan.entries.filter((entry) => {
+    if (entry.mergeStrategy === 'skip') return false;
+    return isManaged(toRelative(projectRoot, entry.targetPath));
+  });
+  const managedRelativePaths = new Set(managedEntries.map((entry) => toRelative(projectRoot, entry.targetPath)));
+
+  const diffResults = await diffInstallPlan({ entries: managedEntries });
+  const drift = diffResults
+    .filter((entry) => entry.action === 'create' || entry.action === 'update')
+    .map((entry) => ({ entry, relativePath: toRelative(projectRoot, entry.targetPath) }))
+    .sort((a, b) => a.relativePath.localeCompare(b.relativePath));
+
+  const mirrorOnly = await collectMirrorOnlyPaths({
+    plan,
+    projectRoot,
+    templatesRoot: pathConfig.templatesRoot,
+  });
+
+  if (checkOnly) {
+    for (const { entry, relativePath } of drift) {
+      console.log(`DRIFT ${relativePath} (${entry.action})`);
+    }
+    for (const relativePath of mirrorOnly) {
+      console.log(`MIRROR-ONLY ${relativePath}`);
+    }
+
+    if (drift.length === 0 && mirrorOnly.length === 0) {
+      console.log(`ukit mirror check: clean (${managedRelativePaths.size} scoped targets)`);
+      return 0;
+    }
+
+    console.log(
+      `ukit mirror check: ${drift.length} drifted, ${mirrorOnly.length} mirror-only `
+        + `(${managedRelativePaths.size} scoped targets)`,
+    );
+    return 1;
+  }
+
+  const { writes } = await applyDiffResults(
+    drift.map(({ entry }) => entry),
+    { backupRoot: pathConfig.backupRoot, projectRoot },
+  );
+  for (const write of writes) {
+    console.log(`${write.action === 'create' ? 'CREATED' : 'UPDATED'} ${toRelative(projectRoot, write.targetPath)}`);
+  }
+
+  // Re-running the managed-gateway step is what makes a rewritten `settings.json`
+  // whole again: the rendered template does not carry the two injected env keys, so
+  // serializing the template alone would drop them.
+  const gateway = await applyGatewayResilienceEnv({ projectRoot });
+  if (gateway.changed) {
+    console.log(`ukit mirror sync: re-applied managed gateway env keys (${[...gateway.applied, ...gateway.migrated].join(', ')})`);
+  }
+
+  for (const relativePath of mirrorOnly) {
+    console.log(`MIRROR-ONLY ${relativePath} — not shipped by any template; left in place, remove it by hand if stale`);
+  }
+
+  console.log(`ukit mirror sync: ${writes.length} written, ${mirrorOnly.length} mirror-only left in place`);
+  return 0;
+}
+
+try {
+  process.exit(await main());
+} catch (error) {
+  console.error(error && error.message ? error.message : error);
+  process.exit(1);
+}