akm-cli 0.9.4 → 0.9.6

package/CHANGELOG.md

@@ -4,6 +4,271 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [0.9.6] - 2026-08-31
+
+The deletion release: **net −2,100 lines**, almost all of it machinery that
+gated, refused, verified, or cached a judgment. Nearly every guard removed
+here had **zero confirmed firings** across 38,341 production telemetry events,
+while several had already broken real installs.
+
+The standard applied is now written down in `AGENTS.md` (`## Defensive Code`):
+a guard survives only if it has demonstrably helped a real user, its failure
+mode costs less than the hazard it prevents, and the operation is not already
+gated behind a deliberate human command. "This hazard is conceivable" is not a
+justification.
+
+### Fixed
+
+- **Embedding no longer discards an entire index because one batch was too
+  big (#874).** Remote embeddings batched by document count (100) against a
+  fixed 30s timeout; a single oversized batch failed the whole phase, leaving
+  `embeddings` at 0 rows on a real 23,857-entry bundle and silently disabling
+  semantic search. Batching is now bounded by a token budget, failures are
+  skipped-and-reported per batch, and an oversized single document is a named
+  skip rather than a phase failure.
+- **`akm lint` no longer silently skips user directories named `.cache` or
+  `registry`.** Two name-based exclusion sites remained after the 0.9.5 fix, so
+  a bundle's own `knowledge/registry/` was never linted and reported clean.
+  Exclusion is now anchored to akm's resolved registry-cache path.
+- **One directory can no longer register as two bundles (#870).** When
+  `AKM_BUNDLE_DIR` pointed at a directory already configured under another id,
+  akm minted a second bundle for it; `akm migrate` then enumerated every task
+  file twice and failed with `duplicate task migration file path` (exit 70) —
+  permanently, while health checks kept passing. Bundle identity is now the
+  resolved content root (`path.resolve(entry.path, component.root ?? ".")`) at
+  both registration sites, existing duplicates reconcile instead of throwing,
+  and a genuinely irreconcilable pair reports both bundle ids and the shared
+  path.
+
+### Removed
+
+- **The index writer lease (#872).** It guarded a *regenerable cache*, had zero
+  lease events in telemetry, and a live-but-wedged holder stranded all indexing
+  for 12 hours — `probeLock` only reclaims a dead PID. It blocked legitimate
+  work twice in a single day of real use. `withAssetMutationLease` is **kept**
+  (it guards authored, git-backed asset writes) but its identical 12h
+  age-based stale reclaim is gone; only a verifiably-dead holder is reclaimed.
+- **`semantic-status.ts` and its cached `blocked` verdict (#873).** A failed
+  probe was persisted with a 24h TTL, and search consulted that verdict
+  *before attempting semantic search at all* — so one failure silently
+  disabled a working feature for a day. Semantic search now attempts per query
+  and falls back to FTS with a live warning. The remaining pre-flight check is
+  a real-time embedding count, not a stored judgment.
+- **The improve-lock 4h stale reclaim.** Same hole: `--skip-if-locked`
+  silently no-oped nightly `improve` for up to four hours and reported success.
+- **The persisted `supportsJsonSchema` capability cache**, which was never
+  invalidated by `akm config set`; a stale `true` sent `response_format:
+  json_schema` to an incompatible endpoint with no fallback (`isRetryable`
+  excludes 4xx). Replaced by attempt-then-fallback held in memory for the
+  process only. The config field survives as an explicit user override.
+- **Workflow authoring resource caps** — steps, params, route branches,
+  inputs, outputs, gate loops, retries, JSON depth/node, composition depth,
+  and exec argv/env caps. None ever fired; several duplicated OS limits.
+- **`LIFETIME_UNIT_CAP` (10,000)**, which hard-aborted workflows mid-run. The
+  maximum ever observed was 14 units.
+- **`state.db` open-path identity re-verification**, which ran on *every*
+  command and threw a bare `Error`.
+- **The task-source 1 MiB cap** at four sites, restating the `guarded-source`
+  cap already deleted in 0.9.5.
+- **TOCTOU identity checks** — `assertGitPublicationIdentity` (git's
+  `--force-with-lease` already covers it), `assertFrozenDirectoryIdentity`
+  (replaced with a path-containment recheck: containment and resolved-path
+  identity stay, device/inode comparison goes, so a remount or container
+  rebuild no longer aborts a dispatch), `assertTaskSourceExpectation`'s stat
+  fields (its content hash stays), and redundant repeat HEAD-generation
+  compares in `write-source`.
+- **`assertNotFlag`** on persona/system-prompt content; **`MAX_ENV_BYTES`**,
+  **`MAX_SECRET_BYTES`**, **`MAX_FEEDBACK_TAGS`**, **`MAX_CONFIG_FILE_BYTES`**,
+  model-map JSON budgets, memory-contradiction family caps, and two launchd /
+  schtasks re-checks that duplicated an existing fallback. `map.concurrency`
+  now clamps instead of rejecting a human-authored value.
+- **17 dead symbols**, including several whose docstrings described behavior
+  nothing implemented. Three "canonical" constants that call sites were
+  ignoring in favour of hardcoded literals were **wired in** rather than
+  deleted, closing the drift instead of removing the evidence of it.
+- **Speculative flexibility** — `AKM_ABLATE_CONTRIBUTORS` ablation plumbing
+  and the unexercised `graphBoost.confidenceMode` branches.
+
+### Changed
+
+- **Drifted duplicate implementations consolidated.** HTTP retry/backoff (the
+  generic copy's `Retry-After` parsing was unbounded and numeric-only; the
+  capped, date-aware one now applies everywhere), the child-process env
+  allowlist (the opencode-sdk copy was missing `AKM_EVENT_SOURCE` and the
+  Windows HOME equivalents), portable synchronous sleep (five call sites had
+  been silently taking the Node fallback instead of the Bun fast path), a
+  stacked LLM chunk retry, and a duplicate `isProcessAlive`.
+
+### Kept, deliberately
+
+Not everything unused is disposable. `isVecFastPathReady` stays: a partial
+`entries_vec` table does not throw, it silently returns wrong neighbours, so
+there is no error for a fallback to catch. `assertSupportedKind` stays: it has
+a proven independent bypass path and is the real last-line check, not a
+duplicate. The maintenance barrier and registry TTL cache stay. The four
+improve strategies with zero recorded invocations stay — that measures one
+install's cron schedule, not their worth.
+
+## [0.9.5] - 2026-08-30
+
+### Action required after upgrading
+
+- **Run `akm index --full` once (#862).** Search ranking previously counted
+  every search *hit* as a small win for that entry's utility score, whether
+  or not you ever opened it — a plain impression, not a selection. Over
+  enough repeat searches this compounded into a real feedback loop: appear
+  in results -> score goes up -> rank higher next time -> appear again. On
+  the install this was diagnosed against, one entry had been searched 19
+  times, opened 0 times, and still carried a utility score of 0.83 — on par
+  with entries a user had actually picked every time; the single
+  highest-utility entry in the whole table had an 8% select rate. That bias
+  is baked into every existing install's stored scores and does not go away
+  on its own — the live bump has been removed (search impressions no longer
+  write utility scores at all; only an actual `akm show`/select or explicit
+  `akm feedback` feeds the offline recompute now), but the already-inflated
+  numbers stay in `utility_scores`/`utility_scores_scoped` until you
+  recompute them. Run `akm index --full` once after upgrading to rebuild
+  scores purely from selection rate and feedback. **Search result order will
+  change after that rebuild — that is intended**, not a regression.
+- **The first `akm task sync` after upgrading may report a large number of
+  updates.** This is a backlog of reconciles that were being silently
+  refused, not new changes to your tasks. Two independent bugs combined to
+  make `sync` refuse work it should have done: (1) an installed scheduler
+  entry written by a pre-`--bundle` akm release couldn't prove ownership
+  under the newer, stricter check and was reported `(unproven owner)`,
+  which made `sync` refuse the *entire* run rather than reconcile everything
+  else; (2) separately, one task or workflow source that failed to compile
+  (e.g. a task still on schema v2 in a shape the shim can't convert) also
+  blocked every other, unrelated task from reconciling. Both are fixed —
+  ownership is now re-derived from the entry's own akm markers instead of
+  requiring a literal `--bundle` token, and a source that fails to compile
+  is now excluded and reported in `failed`/`failures` while every source
+  that DID compile still reconciles. On the install this was diagnosed
+  against, all 18 scheduled tasks showed up as updates on the first `sync`
+  after the fix — that is the backlog, not a sign your tasks changed.
+
+### Added
+
+- **`akm task prune` reclaims orphaned scheduler entries `sync` cannot reach
+  (#851).** `akm task sync` only ever reconciles entries that resolve to a
+  desired task/workflow definition in a live bundle; an entry whose own
+  `--scheduler-context` descriptor is corrupt/missing, or whose owning
+  bundle directory has since been deleted, was permanently invisible to it
+  and had to be removed by hand-editing the crontab/launchd plist/Task
+  Scheduler. `akm task prune` finds those and nothing else: it never
+  touches an entry that still resolves to a live bundle, and there is no
+  `--force`/"remove everything silently" mode. Defaults to a dry-run
+  preview that makes zero scheduler writes and exits non-zero when it finds
+  removal candidates (usable as a CI/health-check guard, mirroring `task
+  sync --dry-run`'s exit-code convention); `--yes` executes the printed
+  plan; `--id a,b` narrows a run to specific binding ids and refuses (with
+  no writes) any id that isn't a current orphan candidate — including a
+  live entry, which cannot be pruned even if you name it explicitly.
+- **`configVersion` read shim (#863).** Config loading now tolerates a
+  known older `configVersion` by upgrading the parsed document in memory
+  (never rewriting the file), with a one-line stderr warning naming the old
+  and new versions. The upgrade is silenced the next time any command
+  writes the config (`akm config set`, etc.), since every config write
+  already stamps the current version. Anything else — unknown, newer, or
+  malformed `configVersion` — still fails closed with the same actionable
+  error as before. No current release needed this shim yet (akm has only
+  ever shipped `configVersion: "0.9.0"`); it's in place now so the next
+  real bump doesn't break every existing config on upgrade.
+- **Truncated indexed content is no longer silent.** The two indexer caps
+  that truncate an entry's content before it's written to the search index
+  (a markdown-body cap and a search-text cap) previously truncated without
+  any signal. Indexed entries now carry a `contentTruncated: true` flag
+  when either cap fired, and truncation logs a `--verbose` diagnostic
+  naming the entry, so an unexpectedly-worse search match on a very large
+  file has a visible cause instead of a silent one. The cap sizes
+  themselves are unchanged.
+
+### Fixed
+
+- **`akm health --report` and `akm proposal list --status accepted` no
+  longer crash, and `improve`'s accepted-proposal counts are no longer
+  silently zero, on proposals written before every optional envelope field
+  existed (#859).** A prior fix already tolerated a missing `changes` key;
+  this closes the same gap for `proposedTarget`, which was still hard-required
+  at decode and — on the real archive this was checked against — absent
+  from 93% of accepted rows. Decoding a proposal now tolerates a genuinely
+  *absent* `proposedTarget` (accept-time resolution already had a
+  ref-derived fallback for this case, previously unreachable because decode
+  threw first); a *present but malformed* value still throws, so real
+  corruption is never silently accepted. Writing a new (`pending`) proposal
+  still requires the full envelope — this only widens what can be *read*,
+  not what akm will *write*.
+- **`akm task sync` no longer refuses to reconcile an entry it can't fully
+  re-prove ownership of, and no longer lets one broken task/workflow source
+  block every other source (#867, plus the scheduler-ownership fix
+  described above).** The v2 task-source shim also no longer hard-fails a
+  command wrapped in `env NAME=value... cmd`, a shape that's common for
+  cron entries (e.g. `env AKM_BIN=/path/akm bash script.sh`) — the shim now
+  looks past a leading `env` and its assignments to the real command before
+  deciding whether the conversion to v3/v4 is safe, instead of checking
+  `env` itself.
+- **`akm task sync --dry-run` no longer crashes on a real install with a
+  pre-`--bundle` cron entry.** The reconcile fix above made those entries
+  reachable for the first time, and the dry-run preview's frozen result
+  object was then mutated in place while stamping output metadata, throwing
+  "Attempting to define property on object that is not extensible" (exit
+  70). Output stamping now copies instead of mutating, so it tolerates a
+  frozen (or any) result uniformly.
+- **`akm migrate apply` no longer aborts an entire batch because one file in
+  it is blocked (#866).** The task v2->v3 and v3->v4 migrators refused to
+  write *any* file — including files with no problem at all — the moment a
+  single file in the batch was classified `blocked`. Both migrators now
+  skip and report the blocked file(s) and still migrate everything else;
+  the run still exits non-zero whenever anything was skipped.
+- Two flaky integration/unit tests fixed at the root cause rather than
+  quarantined (#864): a `state.db` byte-identity assertion that raced
+  SQLite's own WAL checkpoint timing (now forces a checkpoint before
+  comparing, so the assertion reflects real durable mutations only), and a
+  "resolveProjectContext when cwd is the home directory" test that did
+  real, un-mocked filesystem walks against the actual `$HOME` (now isolates
+  `os.homedir()` and the stash/XDG dirs like the rest of the suite).
+
+### Changed
+
+- **`test:integration` is green-by-default (#861).** Verified there is no
+  CI mechanism silently swallowing a failing test (no `continue-on-error`,
+  no lost exit codes, no allowlist of "expected" failures) — the fixes in
+  this release were the actual cause of the prior red baseline. Raised the
+  integration suite's minimum-test-count floor (`AKM_MIN_INTEGRATION_TESTS`,
+  5500 -> 5700) so a large silent test loss is still caught with headroom
+  for ordinary future deletions, and documented in `AGENTS.md` that `TMPDIR`
+  must be a real `/tmp`-family path for the suite to pass — some guards
+  (stash-path safety, `akm-eval`'s Docker twin) intentionally hardcode that
+  assumption rather than reading `TMPDIR`.
+- Simplified several areas flagged in the #866 complexity review without
+  behavior changes: `improve_runs.result_json`'s `schemaVersion` decoding is
+  now an explicit, extensible table instead of one large inline
+  conditional, ready for a future schema bump to add a case rather than
+  restructure the function.
+
+### Removed
+
+- **Three defensive checks that only ever refused work a human explicitly
+  asked for, with no evidence (via telemetry) they ever caught a real
+  problem, were removed:**
+  - The 1 MiB hard cap on reading a bundle file (command/agent/script/task/
+    workflow source, or a frozen workflow secret/env source) into memory.
+    Reads remain exact and integrity-checked (hashed, CAS-verified, fail-
+    closed on read-time mutation) — there is simply no longer a size ceiling
+    that refuses to read a file you put in your own bundle.
+  - The task migrator's inode/hard-link-count/change-time identity
+    fencing on top of its existing lockfile + backup-before-write +
+    byte-for-byte drift check. The extra fencing modeled a multi-tenant
+    race that doesn't apply to a single-user CLI holding an exclusive lock,
+    and its failure mode (refusing to migrate a file that's byte-identical
+    to what was already previewed, or refusing to touch a file merely
+    because it has a hard link elsewhere) was worse than the risk it
+    guarded against. Symlink/path-escape rejection — a real hazard — is
+    unchanged.
+  - Dead anti-collapse "hard refusal" guard functions
+    (`checkGenerationGuard`, `checkMergeInformationFloor`) that had zero
+    production callers.
+
 ## [0.9.4] - 2026-08-30
 
 ### Changed

package/dist/commands/env/env-cli.js

@@ -127,8 +127,7 @@ const envCreateCommand = defineJsonCommand({
                 content = fs.readFileSync(fromFile, "utf8");
             }
             else {
-                const MAX_ENV_BYTES = 1024 * 1024; // 1 MB
-                const buf = await readStdin(MAX_ENV_BYTES, () => new UsageError("Env file exceeds 1 MB limit.", "INVALID_FLAG_VALUE"));
+                const buf = await readStdin();
                 content = buf.toString("utf8");
             }
         }

package/dist/commands/env/secret-cli.js

@@ -122,16 +122,12 @@ const secretSetCommand = defineJsonCommand({
         if (fromEnv !== undefined && fromFile !== undefined) {
             throw new UsageError("Pass only one of --from-file or --from-env (or use stdin).", "INVALID_FLAG_VALUE");
         }
-        const MAX_SECRET_BYTES = 5 * 1024 * 1024; // 5 MB
         let value;
         if (fromFile !== undefined) {
             if (!fs.existsSync(fromFile)) {
                 throw new NotFoundError(`File not found: ${fromFile}`, "FILE_NOT_FOUND");
             }
             value = fs.readFileSync(fromFile);
-            if (value.byteLength > MAX_SECRET_BYTES) {
-                throw new UsageError("Secret exceeds the 5 MB limit.");
-            }
         }
         else if (fromEnv !== undefined) {
             const envVal = process.env[fromEnv];
@@ -144,7 +140,7 @@ const secretSetCommand = defineJsonCommand({
             if (process.stdin.isTTY) {
                 process.stderr.write(`Enter value for secret "${name}" (Ctrl-D when done):\n`);
             }
-            const stdinBuf = await readStdin(MAX_SECRET_BYTES, () => new UsageError("Secret exceeds the 5 MB limit."));
+            const stdinBuf = await readStdin();
             // Strip a single trailing newline so `echo "$TOKEN" | akm secret set`
             // stores the token without the shell-added newline. Use --from-file for
             // byte-exact storage of multi-line material (PEM keys, certs).

package/dist/commands/feedback-cli.js

@@ -25,7 +25,6 @@ import { findEntryIdByRef, getEntryFilePathById, getItemRefById, } from "../stor
 import { applyFeedbackToUtilityScore } from "../storage/repositories/index-utility-repository.js";
 // ── Tag validation ────────────────────────────────────────────────────────────
 const TAG_KEY_RE = /^[a-z_][a-z0-9_]*$/;
-const MAX_FEEDBACK_TAGS = 10;
 function validateFeedbackTags(raw) {
     const seen = new Set();
     const out = [];
@@ -43,9 +42,6 @@ function validateFeedbackTags(raw) {
         seen.add(tag);
         out.push(tag);
     }
-    if (out.length > MAX_FEEDBACK_TAGS) {
-        throw new UsageError(`Too many tags: ${out.length}. Maximum is ${MAX_FEEDBACK_TAGS}.`, "INVALID_FLAG_VALUE");
-    }
     return out;
 }
 // ── Lesson strength helper ────────────────────────────────────────────────────

package/dist/commands/health/checks.js

@@ -607,38 +607,6 @@ export const HEALTH_CHECKS = [
             };
         },
     },
-    {
-        name: "semantic-search-runtime",
-        channel: "advisory",
-        run: (ctx) => {
-            const blocked = ctx.semanticStatus?.status === "blocked";
-            // The generic "status: blocked" line is not actionable when the real
-            // problem is a configured remote embedding endpoint that is down while
-            // semanticSearchMode leaves semantic search enabled — every index run
-            // burns time failing against it and searches silently degrade to
-            // keyword-only. Name the endpoint and the two ways out.
-            const remoteReason = ctx.semanticStatus?.reason?.startsWith("remote-") === true;
-            const endpointAdvisory = blocked && remoteReason && ctx.embeddingEndpoint
-                ? `Configured embedding endpoint ${ctx.embeddingEndpoint} is failing ` +
-                    `(${ctx.semanticStatus?.reason}${ctx.semanticStatus?.message ? `: ${ctx.semanticStatus.message}` : ""}) ` +
-                    `while semanticSearchMode is "${ctx.semanticSearchMode ?? "off"}". Searches fall back to keyword-only. ` +
-                    `Restore the endpoint, or set semanticSearchMode to "off" (or remove embedding.endpoint to use the local model).`
-                : undefined;
-            return {
-                name: "semantic-search-runtime",
-                kind: "deterministic",
-                status: !ctx.semanticStatus || !blocked ? "pass" : "warn",
-                confidence: "medium",
-                message: endpointAdvisory ??
-                    (ctx.semanticStatus
-                        ? `Semantic search status: ${ctx.semanticStatus.status}`
-                        : "No semantic-search runtime status recorded yet."),
-                evidence: ctx.semanticStatus
-                    ? { ...ctx.semanticStatus, ...(ctx.embeddingEndpoint ? { embeddingEndpoint: ctx.embeddingEndpoint } : {}) }
-                    : undefined,
-            };
-        },
-    },
     {
         name: "session-extraction",
         channel: "advisory",

package/dist/commands/health/surfaces.js

@@ -14,7 +14,7 @@
  * the stash-exposure pattern. `egress-endpoints` is the one informational
  * (pass-status) entry: it emits whenever any remote endpoint is configured.
  */
-import { MAX_CONFIG_FILE_BYTES, readTextFileWithLimit } from "../../core/common.js";
+import { readTextFile } from "../../core/common.js";
 import { CURRENT_CONFIG_VERSION } from "../../core/config/config-schema.js";
 import { compareConfigVersion } from "../../core/config/config-version.js";
 import { formatRegistryUrl } from "../../core/registry-url.js";
@@ -28,7 +28,7 @@ import { formatRegistryUrl } from "../../core/registry-url.js";
 export function collectConfigSkewAdvisory(configPath) {
     let raw;
     try {
-        raw = JSON.parse(readTextFileWithLimit(configPath, MAX_CONFIG_FILE_BYTES, "Config file"));
+        raw = JSON.parse(readTextFile(configPath, "Config file"));
     }
     catch {
         return undefined;

package/dist/commands/health.js

@@ -13,7 +13,6 @@ import { classifyPathAccess, describeInaccessiblePath } from "../core/path-acces
 import { getConfigPath, getDataDir, getDbPath, getStateDbPathInDataDir } from "../core/paths.js";
 import { listExistingTableNames, openStateDatabase } from "../core/state-db.js";
 import { DURATION_UNITS, parseDuration, parseSinceToIso } from "../core/time.js";
-import { readSemanticStatus } from "../indexer/search/semantic-status.js";
 import { closeDatabase, openReadonlyExistingDatabase } from "../storage/repositories/index-connection.js";
 import { getAllEntries } from "../storage/repositories/index-entries-repository.js";
 import { queryTaskHistory } from "../storage/repositories/task-history-repository.js";
@@ -183,25 +182,20 @@ function gatherStaleTxnJournalsPhase(now) {
     }
 }
 /**
- * Semantic-search status + the config fields the embedding-endpoint and
- * surfaces advisories need. Best-effort: an unloadable config leaves the
- * config-derived fields undefined and callers fall back to generic messages.
+ * Config fields the surfaces advisory needs. Best-effort: an unloadable
+ * config leaves the field undefined and the caller falls back to a generic
+ * message.
  */
-function gatherSemanticConfigPhase() {
-    const semanticStatus = readSemanticStatus();
-    let semanticSearchMode;
-    let embeddingEndpoint;
+function gatherEgressConfigPhase() {
     let egressConfigView;
     try {
         const config = loadConfig();
-        semanticSearchMode = config.semanticSearchMode;
-        embeddingEndpoint = config.embedding?.endpoint;
         egressConfigView = config;
     }
     catch {
         // fall through with undefined
     }
-    return { semanticStatus, semanticSearchMode, embeddingEndpoint, egressConfigView };
+    return { egressConfigView };
 }
 /**
  * Assemble the window's improve-pipeline summary: invoked/completed/skipped
@@ -519,7 +513,7 @@ export function akmHealth(options = {}) {
         const taskHistory = gatherTaskHistoryPhase(db, logsDb, since, stateDbPath, now);
         const { tableNames, missingTables, probe } = taskHistory;
         const staleTxnJournals = gatherStaleTxnJournalsPhase(now);
-        const { semanticStatus, semanticSearchMode, embeddingEndpoint, egressConfigView } = gatherSemanticConfigPhase();
+        const { egressConfigView } = gatherEgressConfigPhase();
         const { improveSummary } = gatherImproveSummaryPhase(db, stateDbPath, since, now);
         advisories.push(...gatherAncillaryAdvisories(db, stateDbPath, since, improveSummary, options, egressConfigView));
         const engineProbes = runHealthEngineProbes();
@@ -542,9 +536,6 @@ export function akmHealth(options = {}) {
             stuckActiveTasks: taskHistory.stuckActiveTasks,
             worstTaskFailRate: taskHistory.worstTaskFailRate,
             staleTxnJournals,
-            semanticStatus,
-            semanticSearchMode,
-            embeddingEndpoint,
             sessionExtraction: improveSummary.sessionExtraction,
             autoAccept: improveSummary.autoAccept,
             engineProbes,

package/dist/commands/improve/anti-collapse.js

@@ -4,11 +4,11 @@
 /**
  * WS-3b Step 8 — Anti-collapse merge guards.
  *
- *   (a) Generation counter: merged.generation = max(sources)+1; refuse merge
- *       of two assets both above generation N (default 2); merges cite sources.
+ *   (a) Generation counter: merged.generation = max(sources)+1; merges cite
+ *       sources. `over_generation_count` (collapse-detector.ts) tracks assets
+ *       above the generation threshold as an advisory metric only — there is
+ *       no merge-refusal path wired in.
  *   (b) Lexical-diversity check: low n-gram diversity ⇒ raise merge threshold.
- *   (c) Merge-information floor (R5 §4.2): provenance union must not shrink and
- *       the merged body must retain a minimum fraction of the source tokens.
  *   (d) Occasional random non-similar cluster in the pool.
  *
  * @module anti-collapse
@@ -37,93 +37,6 @@ export function computeMergedGeneration(sourceGenerations) {
         return 1;
     return Math.max(...sourceGenerations) + 1;
 }
-/**
- * Check whether a merge of the given assets should be refused due to the
- * anti-collapse generation guard.
- *
- * Returns `{ refused: true, reason }` when BOTH assets have generation > maxGeneration.
- * Returns `{ refused: false }` when the merge is allowed.
- *
- * @param sourceGenerations - Generation values for all merge participants.
- * @param config - Anti-collapse config.
- */
-export function checkGenerationGuard(sourceGenerations, config) {
-    // R5: default ON — only an explicit opt-out disables the guard.
-    if (config.enabled === false)
-        return { refused: false };
-    const maxGen = config.maxGeneration ?? DEFAULT_MAX_GENERATION;
-    const highGenCount = sourceGenerations.filter((g) => g > maxGen).length;
-    if (highGenCount >= 2) {
-        return {
-            refused: true,
-            reason: `Anti-collapse: ${highGenCount} merge participants have generation > ${maxGen} (${sourceGenerations.join(", ")}); refusing to merge over-consolidated assets.`,
-        };
-    }
-    return { refused: false };
-}
-/** Distinct-token retention floor default (R5 §4.2). */
-export const DEFAULT_MIN_SPECIFICITY_RETENTION = 0.6;
-function distinctTokens(text) {
-    // Same lowercase whitespace tokenization computeBigramDiversity uses.
-    return new Set(text
-        .toLowerCase()
-        .split(/\s+/)
-        .filter((w) => w.length > 0));
-}
-/**
- * A merge must strictly increase information (R5 §4.2):
- *  1. Provenance: the merged asset's `xrefs` must be a superset of the union of
- *     all participants' `xrefs` plus the participant refs
- *     themselves — provenance never shrinks through a merge.
- *  2. Specificity: distinctTokens(mergedBody) ≥ minSpecificityRetention ×
- *     |union(distinctTokens(participant bodies))| — a merge that only
- *     shortens/genericizes fails.
- *
- * Pure and deterministic; ADVISORY in v1 (the caller counts violations, it
- * does not refuse the merge). Returns `passed: true` immediately when the
- * anti-collapse suite or the floor itself is opted out.
- */
-export function checkMergeInformationFloor(mergedBody, mergedSourceRefs, participants, config) {
-    if (config.enabled === false || config.mergeInformationFloor === false || participants.length === 0) {
-        return { passed: true, provenanceBefore: 0, provenanceAfter: 0, specificityRetention: 1 };
-    }
-    // 1. Provenance union: participants + everything they already cited.
-    const required = new Set();
-    for (const p of participants) {
-        required.add(p.ref);
-        for (const xref of p.xrefs)
-            required.add(xref);
-    }
-    const after = new Set(mergedSourceRefs);
-    const missing = [...required].filter((r) => !after.has(r));
-    // 2. Specificity retention over the union of source tokens.
-    const sourceTokens = new Set();
-    for (const p of participants) {
-        for (const t of distinctTokens(p.body))
-            sourceTokens.add(t);
-    }
-    const mergedTokens = distinctTokens(mergedBody);
-    // Clamped at computation so the pass/fail decision, the reason string, and
-    // the reported field all describe the same value.
-    const specificityRetention = Math.min(1, sourceTokens.size === 0 ? 1 : mergedTokens.size / sourceTokens.size);
-    const minRetention = config.minSpecificityRetention ?? DEFAULT_MIN_SPECIFICITY_RETENTION;
-    const provenanceOk = missing.length === 0;
-    const specificityOk = specificityRetention >= minRetention;
-    const reasons = [];
-    if (!provenanceOk) {
-        reasons.push(`provenance shrank: merged xrefs missing ${missing.length} ref(s) (e.g. ${missing[0]})`);
-    }
-    if (!specificityOk) {
-        reasons.push(`specificity retention ${specificityRetention.toFixed(2)} < ${minRetention} (merge genericized/shortened)`);
-    }
-    return {
-        passed: provenanceOk && specificityOk,
-        provenanceBefore: required.size,
-        provenanceAfter: after.size,
-        specificityRetention,
-        ...(reasons.length > 0 ? { reason: reasons.join("; ") } : {}),
-    };
-}
 /**
  * Compute the bigram n-gram diversity of a text string.
  * Returns a value in [0, 1] where 0 = all identical bigrams, 1 = all unique.

package/dist/commands/improve/autonomy-gate.js

@@ -29,7 +29,7 @@ export function describeGatedLanes(lanes) {
 }
 /** Configured capabilities that tasks doctor reports behind the autonomy gate. */
 export function configuredDirectAutonomyLanes() {
-    return ["memoryCleanup"];
+    return [...DIRECT_AUTONOMY_LANES];
 }
 /**
  * True when a lane may mutate. Used by the lane that bypasses the strategy

package/dist/commands/improve/consolidate.js

@@ -27,6 +27,7 @@ import { findEntryIdByRef, getAllEntries, getEntryById } from "../../storage/rep
 import { getNeighborsByEntryId } from "../../storage/repositories/index-vec-repository.js";
 import { isProposalSkipped, listProposals, listProposalsReadOnly, proposalContent, } from "../proposal/repository.js";
 import { hasSupersededStatus, validateProposalFrontmatter } from "../proposal/validators/proposal-quality-validators.js";
+import { DEFAULT_RANDOM_CLUSTER_FRACTION } from "./anti-collapse.js";
 import { cacheHash } from "./content-hash.js";
 import { resolveImproveLlmExecution } from "./execution.js";
 import { resolveImproveStrategy, resolveProcessEnabled } from "./improve-strategies.js";
@@ -190,14 +191,15 @@ async function clusterMemoriesBySimilarity(memories, config, stateDb, signal) {
         finally {
             embedMs += Date.now() - embedStart;
         }
-        // Upsert newly computed vectors into the cache.
+        // Upsert newly computed vectors into the cache. A skipped document
+        // (embedBatch reports it via `undefined` rather than throwing, #874) has
+        // no vector to cache — omit it rather than writing a bogus embedding.
         if (stateDb && missVecs.length === missTexts.length) {
             try {
-                const toUpsert = missIndices.map((idx, pos) => ({
-                    contentHash: contentHashes[idx],
-                    embedding: missVecs[pos],
-                    modelId,
-                }));
+                const toUpsert = missIndices.flatMap((idx, pos) => {
+                    const embedding = missVecs[pos];
+                    return embedding ? [{ contentHash: contentHashes[idx], embedding, modelId }] : [];
+                });
                 upsertBodyEmbeddings(stateDb, toUpsert);
             }
             catch {
@@ -783,26 +785,23 @@ async function judgeConsolidationChunks(args) {
                 ...(opts.onNotices ? { onNotices: opts.onNotices } : {}),
             });
         };
-        let raw = await callChunkLlm(`chunk ${chunkIdx + 1} failed`);
+        // callChunkLlm already retries once internally (llm/client.ts's
+        // chatCompletion, jittered 200-800ms backoff) — a second, outer retry
+        // here stacked an uncoordinated fixed 2s backoff on top of it. Removed;
+        // only mark the chunk failed once the single retry the client already
+        // performs has been exhausted.
+        const raw = await callChunkLlm(`chunk ${chunkIdx + 1} failed`);
         if (!raw.ok) {
-            // Single retry with 2s backoff before recording chunk as lost.
-            // Recovers transient Shredder LM Studio timeouts without significantly
-            // extending run time. Only marks failed if both attempts fail.
-            await new Promise((r) => setTimeout(r, 2_000));
-            const retry = await callChunkLlm(`chunk ${chunkIdx + 1} retry failed`);
-            if (!retry.ok) {
-                warn(retry.error ?? `chunk ${chunkIdx + 1} failed after retry`);
-                warnings.push(retry.error ?? `chunk ${chunkIdx + 1} failed after retry`);
-                totalChunksProcessed++;
-                accounting.totalChunksFailed++;
-                // Account for the chunk's memories under the failed-chunk bucket.
-                // judgedNoAction does NOT run on this path (it's after the success
-                // guards) so without this the accounting invariant breaks on every
-                // chunk-level transport/parse failure.
-                accounting.failedChunkMemories += chunk.length;
-                continue;
-            }
-            raw = retry;
+            warn(raw.error ?? `chunk ${chunkIdx + 1} failed`);
+            warnings.push(raw.error ?? `chunk ${chunkIdx + 1} failed`);
+            totalChunksProcessed++;
+            accounting.totalChunksFailed++;
+            // Account for the chunk's memories under the failed-chunk bucket.
+            // judgedNoAction does NOT run on this path (it's after the success
+            // guards) so without this the accounting invariant breaks on every
+            // chunk-level transport/parse failure.
+            accounting.failedChunkMemories += chunk.length;
+            continue;
         }
         // C9 action 1: AKM_DEBUG_LLM was a separate, undocumented env var for this
         // one diagnostic; folded into the standard AKM_VERBOSE gate (warnVerbose)
@@ -922,7 +921,7 @@ async function planConsolidation(opts, config, stashDir, _startMs, memories, war
             const antiCollapseForCluster = getImproveProcessConfig("consolidate", opts.improveProfile)?.antiCollapse ??
                 {};
             if (antiCollapseForCluster.enabled !== false && clusteredMemories.length > 2) {
-                const fraction = antiCollapseForCluster.randomClusterFraction ?? 0.05;
+                const fraction = antiCollapseForCluster.randomClusterFraction ?? DEFAULT_RANDOM_CLUSTER_FRACTION;
                 const randomCount = Math.max(1, Math.floor(clusteredMemories.length * fraction));
                 // Pick `randomCount` positions to inject random (un-clustered) members.
                 // Use a seeded-ish shuffle: sort by hash of the name so it's deterministic