akm-cli 0.9.24 → 0.9.25-alpha.2

package/CHANGELOG.md

@@ -4,6 +4,179 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [Unreleased]
+
+## [0.9.25-alpha.2] - 2026-10-03
+
+### Added
+
+- **`akm improve judge`** runs reflect's quality judge on one revision, read as
+  `{"source", "candidate", "feedback", "ref"}` JSON from stdin, with the engine
+  `processes.reflect.qualityGate.engine` names, and prints the verdict. It
+  writes nothing: it tests a judge engine on revisions whose right answer you
+  know.
+
+### Changed
+
+- **Reflect refuses three kinds of defective revision before the judge runs,**
+  with the quality gate on or off: one that adds placeholder text ("please
+  confirm", "to be confirmed"; not `TODO`, `TBD` or `FIXME`), one that talks about
+  its own edit ("the feedback says", "this revision", "the source asset", a
+  quoted gate rejection), and one that copies frontmatter into its body (key
+  lines such as `sources:` or `updated:` outside code, or a `sources`,
+  `xrefs` or `contradictedBy` value). Each rule counts only what the revision
+  adds to its source. A hit is a `quality_rejected` refusal with no proposal
+  and no judge call, and the event names the rule (`reflectDefect`). Each
+  rule's wording is a list that `processes.reflect.defectFilter` can replace:
+  `placeholders` and `metaCommentary` are plain phrases, matched as whole words
+  in any case, and `frontmatterKeys` are exact key names. A list left out keeps
+  its default, and `[]` turns its rule off. On 396 labelled reflect edits the
+  default lists hit 22 of 313 bad edits and none of 83 good ones.
+- **The reflect quality judge's rubric names what it kept missing.** Tuned on
+  the labelled judge-gate set (plain judge, qwen3.8-27b): a description with a
+  sentence split by a stray period or an unbalanced quote is broken text; a
+  missing title, description or `when_to_use` is a concrete problem whether or
+  not the feedback mentions it, while a bare `type:` or a provenance stamp is
+  not; PRESERVATION and QUALITY are judged line by line in the changed region,
+  so a fix elsewhere no longer excuses a dropped fact, an invented or
+  strengthened claim, a hedge or a placeholder. On a stratified 100-case sample
+  the gate passed 31 of 40 good edits instead of 18 and 3 of 60 bad instead of 2.
+- **A reflect quality judge on an agent engine reads only to verify what a
+  revision adds.** Its prompt names the revised asset's ref and adds one
+  paragraph: read that asset, or one the changed region names, with `akm_show`
+  only to check a fact the revision adds or alters, at most twice, and never
+  search; before scoring, find each added statement in the asset or the
+  feedback (a step or cause that merely seems to follow does not count), each
+  source fact in the revision, and each feedback point in a change to the text
+  it is about. The plain judge's prompt is unchanged. On the same 100 cases
+  (qwen3.8-27b, thinking on) the agent judge passed 38 of 40 good edits and 14
+  of 60 bad, against 34 and 12 for the plain judge on that model. Give the
+  engine's `llmEngine` `enableThinking: true`: without thinking the model
+  looped on tool calls.
+- **A failed reflect reply reads the same on every engine kind:** the parser's
+  own message. 0.9.25-alpha.1 named the engine on an agent engine.
+- **Inference reaches opencode only where akm writes opencode's config:** an
+  improve process's `llm` overlay on model work's agent, and an `opencode-sdk`
+  engine's `llmEngine` fallback model. Set the rest in your opencode config; a
+  task's or workflow's `inference` is an `untranslated-field` notice, as before
+  0.9.25-alpha.1. `claude` takes no `--effort`.
+- **An `opencode-sdk` session the dispatch times out on or aborts is aborted on
+  the server for every dispatch,** not only model work.
+- **An improve stage retries a reply only when the stage cannot read it,** not
+  when it misses the JSON Schema. A reply it can read costs no second call.
+- **opencode model work can read the stash and search it.** The `akm-model-work`
+  agent reads, greps and globs in the stash and its working directory, edits
+  only in the working directory, and has `akm_search` and `akm_show` from the
+  akm-opencode plugin (0.9.21 or later, in your own opencode config). The
+  plugin's curation, learning and write gate are off for these dispatches, and
+  its state goes to akm's state directory, not `~/.local/state/akm-opencode`.
+
+### Removed
+
+- **The agent-engine inference fields,** which 0.9.25-alpha.1 accepted: they
+  fail to load again.
+- **The `opencode-sdk` step watcher and the git-repository refusal for model
+  work's scratch directory.**
+- **opencode model work's step limit.** At the limit opencode sends a "maximum
+  steps" message as a trailing assistant message, which a qwen chat template
+  (LM Studio, llama-server) renders as the start of the model's reply: LM
+  Studio returned nothing, llama-server returned that message as the answer,
+  and the dispatch failed. The dispatch timeout bounds a run.
+- **A prompt builder that nothing called** (`buildSchemaRepairPrompt`).
+- **The `--track-usage` / `--no-track-usage` flag on `akm search`, `akm curate`
+  and `akm show`.** A successful read always records its usage event, stamped
+  with its source (`user`, `improve`, `task` or `audit`); only `user` events
+  feed ranking and eval, so machine reads never skew them. Either spelling now
+  fails as an unknown flag.
+
+### Fixed
+
+- **The reflect size guard no longer flags a body that does not grow.** The
+  expansion ceiling is capped at 25,000 characters, so a source body longer than
+  that was flagged `EXCESSIVE_EXPANSION` even when the proposed body was its own
+  length (ratio 1.00), and went to review instead of the judge. A body no longer
+  than its source is never expansion; one that grows past the cap still is, by
+  any amount. The reflect prompt agrees: it told the model its body could be at
+  most 25,000 characters even when the source was longer, and now gives such a
+  source's own length.
+- **An asset's or a task's own `tools:` can no longer name the model-work
+  policy.** In 0.9.25-alpha.1 exactly `read`, `edit`, `akm search`, `akm show`,
+  in that order, skipped `execution.allowedTools`. It is ordinary tools now:
+  only akm's own model-work callers ask for the policy.
+- **An agent engine that names its model only in `args` is named in the usage
+  report,** where it showed `unattributed`.
+- **`opencode` and `opencode-sdk` engines receive the XDG base-directory
+  variables.** Under a custom `XDG_CONFIG_HOME` the spawned opencode missed its
+  provider config and failed every dispatch with `Unexpected server error`.
+
+## [0.9.25-alpha.1] - 2026-10-02
+
+### Changed
+
+- **Unattended model work runs under one tool policy, on any engine that
+  confines it.** The model may read, edit inside a scratch directory akm removes
+  after the dispatch, and run `akm search` and `akm show`; the stash stays
+  read-only. An LLM engine has no tools, `claude` confines the policy, and
+  `opencode` and `opencode-sdk` run an injected `akm-model-work` agent with
+  read and edit only. Every other harness refuses it before the request starts.
+- **One config rule names the engines model work may use.** Every key it reads
+  its engine from (`defaults.llmEngine`, `index.*.engine`, a strategy's or
+  process's `engine`, an enabled triage `judgment.engine`, a
+  `qualityGate.engine`) must name an LLM engine or a `claude`, `opencode` or
+  `opencode-sdk` agent engine, or the config fails to load. `--require-engines`
+  and `akm health` check agent engines too.
+- **Model work is bounded at 600 seconds on every engine kind** whose engine
+  sets no `timeoutMs`; agent and `opencode-sdk` engines ran until they finished.
+- **`akm proposal new` returns the proposal as JSON on every engine kind,**
+  with no draft file and no live session. A reply that is not a proposal gets
+  one retry.
+- **Reflect asks every engine kind for the same JSON reply and repairs it
+  once.** An agent or `opencode-sdk` engine gets reflect's JSON Schema and a
+  repair turn, as an LLM engine did. An LLM engine's requests are unchanged.
+- **An improve stage's reply that fails its JSON Schema gets one corrective
+  retry,** then the stage reads the last reply with its own parser.
+- **One schema instruction for every agent engine,** appended by the shared
+  request lowering: `opencode` and `opencode-sdk` now receive a requested
+  schema, and a workflow unit on seven harnesses no longer carries it twice.
+- **Agent and `opencode-sdk` dispatches leave a usage record.**
+- **An `opencode-sdk` engine's LLM fallback comes only from its own
+  `llmEngine`; `defaults.llmEngine` no longer supplies one.** If you relied on
+  that, set `llmEngine` on the SDK engine. Without one, opencode uses its own
+  provider, model and auth.
+- **Inference reaches `opencode`, `opencode-sdk` and `claude` engines.**
+  `temperature`, `maxTokens`, `contextLength`, `enableThinking` and
+  `reasoningEffort` from an engine, an improve process's `llm` overlay, an
+  asset, a workflow or a `models.json` alias were dropped on every agent
+  engine. An agent engine may set them, and `claude` takes `reasoningEffort` as
+  `--effort`.
+- **Reasoning effort has one word, `reasoningEffort`.** `effort` in a
+  `models.json` alias or an asset is read as it, so an LLM engine now sends it
+  as `reasoning_effort`.
+
+### Removed
+
+- **Dead code; nothing you run changes:** each harness's unused `pattern` and
+  `structuredOutput` fields, `resolveLlmEngineUse`'s swap of an agent engine
+  for an LLM engine, the agent request's unread `effort` hint, and the agent
+  file-write contract (`DRAFT_WRITTEN`, reflect's `ref_mismatch` check).
+
+### Fixed
+
+- **An `opencode-sdk` engine with an LLM fallback reaches its endpoint, and a
+  failed dispatch is a failure (#1015).** An SDK error or provider rejection is
+  `ok: false` with opencode's message, and a reply's last text part is its answer.
+- **An LLM engine reports a provider error sent with HTTP 200 as a failure**
+  (OpenRouter does this), not as an empty reply.
+- **An `opencode` engine runs a persona** instead of failing on
+  `--system-prompt`, which opencode 1.18 rejects.
+- **An LLM engine sends a requested schema** unless it sets
+  `supportsJsonSchema: false`; it sent one only when it set `true`.
+- **A feature gate's timeout stops the call it bounds,** and a stage call
+  reports a timeout or abort as `timeout` or `aborted`, not `error`.
+- **`akm proposal new` keeps the reply's `confidence`.**
+- **Model work on an agent or `opencode-sdk` engine with no `timeoutMs` stops
+  after 600 seconds** as documented; it resolved to no timeout.
+
 ## [0.9.24] - 2026-10-02
 
 ### Changed

package/dist/cli.js

@@ -177,7 +177,7 @@ const setupCommand = defineCommand({
         // the work, matching the `sync --push/--no-push` pattern. A flag
         // DECLARED as `no-init` can never be negated: `--no-init` parses as
         // "negate `init`", a name nothing declared, leaving the real key at its
-        // default forever — see `search --no-track-usage`'s identical fix.
+        // default forever.
         init: {
             type: "boolean",
             default: true,

package/dist/commands/health/checks.js

@@ -3,7 +3,6 @@
 // file, You can obtain one at https://mozilla.org/MPL/2.0/.
 import { spawnSync } from "node:child_process";
 import { loadConfig } from "../../core/config/config.js";
-import { IMPROVE_PROCESS_ENGINE_CAPABILITIES } from "../../core/config/engine-semantics.js";
 import { listEnvsRecursive } from "../../core/env-secret-ref.js";
 import { ConfigError } from "../../core/errors.js";
 import { EXTRACT_INFRASTRUCTURE_SKIP_REASONS } from "../../core/improve-types.js";
@@ -15,7 +14,7 @@ import { loadModelMap, mergeModelMapLayers, parseModelMapLayer, readInstalledMod
 import { probeEndpointOnce } from "../../llm/client.js";
 import { STATE_DB_FREELIST_WARN_RATIO, } from "../../storage/state-db-integrity.js";
 import { listKeys } from "../env/env.js";
-import { resolveImprovePlan } from "../improve/improve-strategies.js";
+import { MODEL_CALLING_PROCESSES, resolveImprovePlan } from "../improve/improve-strategies.js";
 import { ENGINE_LAST_USED_LOOKBACK_DAYS } from "./engine-usage.js";
 import { ACTIVE_RUN_WARN_MS, TASK_FAIL_RATE_WARN, } from "./types.js";
 /** Probe one connection's reachability, once per endpoint; `undefined` when no probe seam is supplied. */
@@ -142,7 +141,7 @@ async function runConfiguredEngineProbe(checkName, engineName, config, deps, rea
                 evidence: { engine: engineName, runtimeKind: "sdk", binaryAvailable: false },
             };
         }
-        const fallbackEngine = configuredEngine.llmEngine ?? config.defaults?.llmEngine;
+        const fallbackEngine = configuredEngine.llmEngine;
         let fallback;
         let fallbackCredential;
         let fallbackApiKeyFile;
@@ -523,17 +522,17 @@ export function probeActiveImproveStrategy(deps = {}) {
             .sort(([a], [b]) => a.localeCompare(b))
             .map(([process, engine]) => `${process}: "${engine}"`)
             .join(", ");
-        // #957: fail only when the strategy's LLM-backed work would be a total
-        // no-op — every process the strategy actually enabled among the
-        // `capability: "llm"` set (see IMPROVE_PROCESS_ENGINE_CAPABILITIES) ended
-        // up unavailable. A partial failure (some processes still have a working
+        // #957: fail only when the strategy's model work would be a total no-op —
+        // every process the strategy actually enabled among the ones that call a
+        // model themselves (MODEL_CALLING_PROCESSES, any engine kind) ended up
+        // unavailable. A partial failure (some processes still have a working
         // engine) stays a `warn`, matching #914's "a credential warn stays a warn"
         // policy for the general per-engine probes; this is the strategy-scoped
         // "is the whole run a no-op" question instead.
-        const llmProcessNames = Object.keys(IMPROVE_PROCESS_ENGINE_CAPABILITIES).filter((name) => IMPROVE_PROCESS_ENGINE_CAPABILITIES[name] === "llm");
-        const requiredLlmProcessNames = llmProcessNames.filter((name) => plan.processes[name].enabled || plan.engineUnavailable.some((item) => item.process === name));
-        const availableLlmProcessNames = llmProcessNames.filter((name) => plan.processes[name].enabled);
-        const allRequiredUnavailable = requiredLlmProcessNames.length > 0 && availableLlmProcessNames.length === 0;
+        const modelProcessNames = [...MODEL_CALLING_PROCESSES];
+        const requiredModelProcessNames = modelProcessNames.filter((name) => plan.processes[name].enabled || plan.engineUnavailable.some((item) => item.process === name));
+        const availableModelProcessNames = modelProcessNames.filter((name) => plan.processes[name].enabled);
+        const allRequiredUnavailable = requiredModelProcessNames.length > 0 && availableModelProcessNames.length === 0;
         const status = unavailableProcesses.length === 0 ? "pass" : allRequiredUnavailable ? "fail" : "warn";
         return {
             check: {

package/dist/commands/improve/consolidate/pair-pass.js

@@ -36,6 +36,7 @@ import { concurrentMap } from "../../../core/concurrent.js";
 import { parseEmbeddedJsonResponse } from "../../../core/parse.js";
 import { DERIVED_SUFFIX } from "../../../core/recognition-util.js";
 import { warnOnce } from "../../../core/warn.js";
+import { runnerLlmConnection } from "../../../integrations/agent/runner.js";
 import { assertRunnerCredentials } from "../../../integrations/agent/runner-dispatch.js";
 import { runGit } from "../../../sources/providers/git-install.js";
 import { closeDatabase, openExistingDatabase, openReadonlyExistingDatabase, } from "../../../storage/repositories/index-connection.js";
@@ -501,10 +502,11 @@ async function judgeOne(ctx, candidate) {
         request: {
             responseSchema: PAIR_JUDGE_JSON_SCHEMA,
             enableThinking: false,
-            timeoutMs: ctx.llmRunner.timeoutMs,
+            ...(Object.hasOwn(ctx.llmRunner, "timeoutMs") ? { timeoutMs: ctx.llmRunner.timeoutMs } : {}),
             signal: ctx.opts.signal,
             ...(ctx.chat ? { chat: ctx.chat } : {}),
         },
+        parse: parsePairJudgeResponse,
         ...(ctx.opts.onNotices ? { onNotices: ctx.opts.onNotices } : {}),
     });
     if (!outcome.ok)
@@ -747,7 +749,7 @@ seams = {}) {
         // entirely and needs no credential.
         if (!seams.chat)
             assertRunnerCredentials(llmRunner);
-        const results = await concurrentMap(judgeable, (candidate) => judgeOne(ctx, candidate), llmRunner.connection.concurrency ?? 1, { signal: opts.signal });
+        const results = await concurrentMap(judgeable, (candidate) => judgeOne(ctx, candidate), runnerLlmConnection(llmRunner)?.concurrency ?? 1, { signal: opts.signal });
         results.forEach((r, idx) => {
             // Must-fix 3: `concurrentMap` leaves an entry `undefined` for a call an
             // aborted run never sent at all — `r?.failed === true` reads that as

package/dist/commands/improve/consolidate.js

@@ -36,6 +36,7 @@ import { resolveWriteTarget } from "../../core/write-source.js";
 import { deriveInstallations } from "../../indexer/installations.js";
 import { resolveSourceEntries } from "../../indexer/search/search-source.js";
 import { USAGE_EVENT_RETENTION_DAYS } from "../../indexer/usage/usage-events.js";
+import { runnerLlmConnection } from "../../integrations/agent/runner.js";
 import { assertRunnerCredentials } from "../../integrations/agent/runner-dispatch.js";
 import { cosineSimilarity, embedBatch, resolveEmbeddingModelId } from "../../llm/embedder.js";
 import { getBodyEmbeddings, upsertBodyEmbeddings } from "../../storage/repositories/embeddings-repository.js";
@@ -52,6 +53,10 @@ import { resolveImproveStrategy, resolveProcessEnabled } from "./improve-strateg
 import { isContentDrivenRow, isLedgerBlocked, ledgerKey, loadLedgerSnapshot, recordLedgerAttempt } from "./ledger.js";
 import { isInRetrievalScope, loadRetrievalScope } from "./retrieval-scope.js";
 import { callStage, mintProposal, noticeSet, stageRunner } from "./stage.js";
+function parsePlan(raw) {
+    const plan = parseEmbeddedJsonResponse(raw);
+    return plan && Array.isArray(plan.operations) ? { ...plan, operations: plan.operations } : undefined;
+}
 /** A plan op worth acting on. Retired advisory ops (merge/delete/contradict) are dropped, never thrown on. */
 export function isValidOp(op) {
     if (typeof op !== "object" || op === null)
@@ -566,9 +571,10 @@ async function judgeConsolidationChunks(args) {
             request: {
                 responseSchema: CONSOLIDATE_PLAN_JSON_SCHEMA,
                 enableThinking: false,
-                timeoutMs: llmRunner.timeoutMs,
+                ...(Object.hasOwn(llmRunner, "timeoutMs") ? { timeoutMs: llmRunner.timeoutMs } : {}),
                 signal: opts.signal,
             },
+            parse: parsePlan,
             ...(opts.onNotices ? { onNotices: opts.onNotices } : {}),
         });
         if (!outcome.ok) {
@@ -576,8 +582,8 @@ async function judgeConsolidationChunks(args) {
             continue;
         }
         warnVerbose(`[akm:consolidate] ${label} raw response (first 500 chars): ${outcome.raw.slice(0, 500)}`);
-        const parsed = parseEmbeddedJsonResponse(outcome.raw);
-        if (!parsed || !Array.isArray(parsed.operations)) {
+        const parsed = parsePlan(outcome.raw);
+        if (!parsed) {
             const hint = outcome.raw.trim() === "" ? " (empty response — if using a thinking model, disable thinking mode)" : "";
             const msg = `Chunk ${chunkIdx + 1}: invalid plan from AI — skipping.${hint}`;
             warn(msg);
@@ -619,7 +625,7 @@ async function planConsolidation(opts, config, stashDir, memories, warnings, sta
     const llmRunner = opts.llmRunner ?? undefined;
     // 500 body chars per memory keep the judgement useful; chunk size varies instead.
     const bodyTruncation = 500;
-    const chunkSize = computeSafeChunkSize(llmRunner?.connection.contextLength ?? DEFAULT_CONTEXT_LENGTH_TOKENS, bodyTruncation, opts.maxChunkSize);
+    const chunkSize = computeSafeChunkSize((llmRunner && runnerLlmConnection(llmRunner)?.contextLength) ?? DEFAULT_CONTEXT_LENGTH_TOKENS, bodyTruncation, opts.maxChunkSize);
     const sourceName = opts.target ?? stashDir;
     let budgeted = memories;
     const budgetMs = opts.signal?.remainingBudgetMs;

package/dist/commands/improve/execution.js

@@ -22,6 +22,8 @@ function mergeDefaults(farther, nearer) {
 /**
  * Resolve improve-owned model work through the canonical execution cascade:
  * defaults.llmEngine -> strategy -> index.<pass> -> process -> current invocation.
+ * The engine may be of any kind that confines the model-work tool policy: one
+ * that cannot, chosen with `--engine` say, is refused here, before any work.
  */
 export function resolveImproveExecution(options) {
     const defaultEngine = options.config.defaults?.llmEngine;
@@ -33,22 +35,13 @@ export function resolveImproveExecution(options) {
     if (selectedEngine === undefined || selectedEngine === null)
         return null;
     const invocationDefaults = mergeDefaults(mergeDefaults(defaultEngine ? { engine: defaultEngine } : {}, profileDefaults), indexDefaults);
-    const current = mergeDefaults(processDefaults, currentDefaults);
     const prepared = resolveExecution({
         content: `improve ${options.processName} execution selection`,
         config: options.config,
         invocationDefaults,
-        current,
+        current: mergeDefaults(processDefaults, currentDefaults),
+        modelWork: true,
     });
     const lowered = buildExecution(prepared.request, prepared.runner);
     return Object.freeze({ runner: lowered.runner, notices: lowered.notices });
 }
-export function resolveImproveLlmExecution(options) {
-    const resolved = resolveImproveExecution(options);
-    if (!resolved)
-        return null;
-    if (resolved.runner.kind !== "llm") {
-        return null;
-    }
-    return { runner: resolved.runner, notices: resolved.notices };
-}

package/dist/commands/improve/extract-prompt.js

@@ -9,9 +9,9 @@
  * session data into the markdown template loaded from
  * `src/assets/prompts/extract-session.md`.
  *
- * The schema is intentionally strict — providers with `supportsJsonSchema:
- * true` enforce shape upstream, so the parser only has to handle the
- * happy path. `additionalProperties: false` means any hallucinated keys
+ * The schema is intentionally strict — a provider that honours
+ * `response_format` enforces shape upstream, so the parser only has to handle
+ * the happy path. `additionalProperties: false` means any hallucinated keys
  * the model emits get dropped before we parse.
  */
 import promptTemplate from "../../assets/prompts/extract-session.md" with { type: "text" };
@@ -20,7 +20,7 @@ const EXTRACT_CANDIDATE_NAME_PATTERN = "^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?(?:/[a-z
 const EXTRACT_CANDIDATE_NAME_RE = new RegExp(EXTRACT_CANDIDATE_NAME_PATTERN);
 /**
  * JSON Schema for the structured extract output. Passed to `chatCompletion`
- * when the configured LLM connection has `supportsJsonSchema: true`.
+ * unless the configured LLM connection sets `supportsJsonSchema: false`.
  *
  * Shape:
  *   {

package/dist/commands/improve/extract.js

@@ -32,16 +32,15 @@ import { indexWrittenAssets } from "../../indexer/index-written-assets.js";
 import { assertRunnerCredentials } from "../../integrations/agent/runner-dispatch.js";
 import { getAvailableHarnesses } from "../../integrations/session-logs/index.js";
 import { preFilterSession } from "../../integrations/session-logs/pre-filter.js";
-import { isJsonSchemaKnownUnsupported } from "../../llm/client.js";
 import { getExtractedSessionsMap, getLastExtractRunAt, shouldSkipAlreadyExtractedSession, upsertExtractedSession, } from "../../storage/repositories/extract-sessions-repository.js";
 import { openSqliteReadSnapshot } from "../../storage/sqlite-read-snapshot.js";
 import { contentHash } from "./content-hash.js";
-import { resolveImproveLlmExecution } from "./execution.js";
+import { resolveImproveExecution } from "./execution.js";
 import { buildExtractPrompt, EXTRACT_JSON_SCHEMA, parseExtractPayload, } from "./extract-prompt.js";
 import { cloneAndFreeze, resolveImproveStrategy, resolveProcessEnabled } from "./improve-strategies.js";
 import { isLedgerBlocked, ledgerKey, loadLedgerSnapshot } from "./ledger.js";
 import { buildSessionSummaryPrompt, parseSessionSummary, SESSION_SUMMARY_JSON_SCHEMA, sessionMeetsDurationGate, writeSessionAsset, } from "./session-asset.js";
-import { callStage, mintProposal, noticeSet } from "./stage.js";
+import { callStage, callStageOnce, mintProposal, noticeSet } from "./stage.js";
 /** Minimum session duration (minutes) for writing a session asset. */
 const DEFAULT_MIN_SESSION_DURATION_MINUTES = 5;
 /** Raw session size (chars) below which the LLM call is skipped; only truly empty sessions are safe to skip. */
@@ -119,7 +118,7 @@ export function resolveStandaloneExtractPlan(config, selection) {
     }
     const selected = resolveImproveStrategy(selection.strategy, config);
     const process = cloneAndFreeze(getImproveProcessConfig("extract", selected.config) ?? {});
-    const resolved = resolveImproveLlmExecution({
+    const resolved = resolveImproveExecution({
         config,
         profile: selected.config,
         process,
@@ -130,7 +129,7 @@ export function resolveStandaloneExtractPlan(config, selection) {
         processName: "extract",
     });
     if (!resolved) {
-        throw new ConfigError("No LLM engine configured for extract. Set defaults.llmEngine, pass --engine, or select an improve strategy with processes.extract.engine.", "LLM_NOT_CONFIGURED");
+        throw new ConfigError("No engine configured for extract. Set defaults.llmEngine, pass --engine, or select an improve strategy with processes.extract.engine.", "LLM_NOT_CONFIGURED");
     }
     const runner = resolved.runner;
     return Object.freeze({
@@ -358,15 +357,16 @@ function planExtractSessions(args) {
 }
 const EXTRACT_LLM_UNAVAILABLE = Symbol("extract-llm-unavailable");
 /**
- * One session's extraction call. A connection without structured output gets
- * one corrective retry; configuration errors escape before any state is written.
+ * One session's extraction call, with one corrective retry; configuration
+ * errors escape before any state is written.
  */
 async function extractFromSession(run, prompt) {
     const { llmRunner } = run;
     try {
         const result = await runStructured({
             dispatch: async (feedback) => {
-                const outcome = await callStage({
+                // This loop parses and repairs the reply itself, so each attempt is one unvalidated dispatch.
+                const outcome = await callStageOnce({
                     feature: "session_extraction",
                     runner: llmRunner,
                     prompt: feedback ? `${prompt}\n\n## Corrective output instruction\n\n${feedback}` : prompt,
@@ -388,9 +388,6 @@ async function extractFromSession(run, prompt) {
                 return payload.parseFailure ? undefined : payload;
             },
             validate: (payload) => ({ ok: true, value: payload }),
-            maxAttempts: llmRunner.connection.supportsJsonSchema !== false && !isJsonSchemaKnownUnsupported(llmRunner.connection)
-                ? 1
-                : 2,
             buildFeedback: () => "Your previous response did not contain a valid extraction payload. Respond with ONLY a JSON object matching the requested schema, with a candidates array and no prose or code fences.",
         });
         if (result.ok)
@@ -714,13 +711,13 @@ function resolveExtractRun(options, config, process, activeProfile) {
         llmRunner = options.llmRunner;
     }
     else {
-        const resolved = resolveImproveLlmExecution({ config, profile: activeProfile, process, processName: "extract" });
+        const resolved = resolveImproveExecution({ config, profile: activeProfile, process, processName: "extract" });
         llmRunner = resolved?.runner;
         if (resolved)
             notices.add(resolved.notices);
     }
     if (!llmRunner) {
-        throw new ConfigError("No LLM engine configured for extract. Set defaults.llmEngine or improve.strategies.<name>.processes.extract.engine.", "LLM_NOT_CONFIGURED");
+        throw new ConfigError("No engine configured for extract. Set defaults.llmEngine or improve.strategies.<name>.processes.extract.engine.", "LLM_NOT_CONFIGURED");
     }
     const runner = llmRunner;
     const timeoutMs = options.resolvedPlan
@@ -744,6 +741,7 @@ function resolveExtractRun(options, config, process, activeProfile) {
                 ...(options.signal ? { signal: options.signal } : {}),
                 ...(options.chat ? { chat: options.chat } : {}),
             },
+            parse: parseSessionSummary,
             onNotices: notices.add,
         });
         return parseSessionSummary(outcome.ok ? outcome.raw : "");

package/dist/commands/improve/improve-cli.js

@@ -15,17 +15,20 @@ import { redactSensitiveText } from "../../core/redaction.js";
 import { clearLogFile, setLogFile, warn } from "../../core/warn.js";
 import { resolveWriteTarget } from "../../core/write-source.js";
 import { DEFAULT_LLM_TIMEOUT_MS } from "../../integrations/agent/config.js";
+import { defaultWhich } from "../../integrations/agent/detect.js";
 import { collectEngineCredentialValues } from "../../integrations/agent/engine-resolution.js";
 import { probeLlmReachable } from "../../llm/client.js";
 import { getOutputMode } from "../../output/context.js";
 import { deliverRendered } from "../../output/html-render.js";
+import { readStdin } from "../../runtime.js";
 import { akmImprove, IMPROVE_TARGET_FLAG, resolveImproveReadSource } from "./improve.js";
 import { runImproveReportQuery } from "./improve-report.js";
 import { buildImproveRunId, recordImproveRunResult, recordTerminatedImproveRun, } from "./improve-result-file.js";
 import { runImproveSession } from "./improve-session.js";
-import { resolveImprovePlan, } from "./improve-strategies.js";
+import { resolveImprovePlan, resolveImproveStrategy, } from "./improve-strategies.js";
 import { formatUsageReportTable } from "./improve-usage-report.js";
 import { renderReflectPromptPreview } from "./reflect.js";
+import { resolveQualityGateJudge, runReflectQualityJudge } from "./stage.js";
 let akmImproveForRun = akmImprove;
 /** Swap the CLI's improve work implementation in deterministic subprocess tests. */
 export function _setAkmImproveForTests(fake) {
@@ -73,26 +76,19 @@ function assertRequiredEnginesAvailable(plan) {
     const lines = plan.engineUnavailable.map((item) => `  - ${item.process} (${item.configKey}): ${item.reason}`);
     throw new ConfigError(`--require-engines: ${plan.engineUnavailable.length} improve process${plan.engineUnavailable.length === 1 ? "" : "es"} cannot run because ${plan.engineUnavailable.length === 1 ? "its" : "their"} engine is unavailable:\n${lines.join("\n")}`, "LLM_NOT_CONFIGURED");
 }
-/** Every LLM connection the plan would dispatch to, triage's judgment engine included. */
+/** Every engine the plan would dispatch to, triage's judgment engine included. */
 function collectRequiredEngineTargets(plan) {
-    const targets = [];
-    for (const [processName, process] of Object.entries(plan.processes)) {
-        if (process.runner) {
-            targets.push({
-                process: processName,
-                engine: process.runner.engine,
-                connection: probeConnection(process.runner),
-            });
-        }
-    }
-    if (plan.triageJudgment?.kind === "llm") {
-        targets.push({
-            process: "triage.judgment",
-            engine: plan.triageJudgment.engine,
-            connection: probeConnection(plan.triageJudgment),
-        });
-    }
-    return targets;
+    const runners = Object.entries(plan.processes)
+        .flatMap(([processName, process]) => (process.runner ? [[processName, process.runner]] : []))
+        .concat(plan.triageJudgment ? [["triage.judgment", plan.triageJudgment]] : []);
+    return runners.map(([processName, runner]) => ({
+        process: processName,
+        engine: runner.engine,
+        ...(runner.kind === "llm" ? { connection: probeConnection(runner) } : { bin: runner.profile.bin }),
+        ...(runner.kind === "sdk" && runner.fallbackConnection
+            ? { connection: probeConnection({ connection: runner.fallbackConnection, timeoutMs: runner.fallbackTimeoutMs }) }
+            : {}),
+    }));
 }
 /** The resolved engine keeps its request timeout beside the connection (the runtime merges it in); the probe needs it on the connection. */
 function probeConnection(runner) {
@@ -112,37 +108,42 @@ const REQUIRED_ENGINE_PROBE_MAX_MS = 120_000;
 /**
  * `--require-engines`, live: probe each connection's real completion path
  * (a gateway can list a model whose completion route is dead, #980), once per
- * endpoint + model, within {@link requiredEngineProbeTimeoutMs}. Returns each
- * target's latency for the run result (R17); an unreachable one fails the run.
+ * endpoint + model, within {@link requiredEngineProbeTimeoutMs}, and look each
+ * agent harness's binary up on PATH. Returns each target's latency for the run
+ * result (R17); an unreachable one fails the run.
  */
-export async function assertRequiredEnginesReachable(plan, probeReachable = (connection) => probeLlmReachable(connection, requiredEngineProbeTimeoutMs(connection))) {
+export async function assertRequiredEnginesReachable(plan, probeReachable = (connection) => probeLlmReachable(connection, requiredEngineProbeTimeoutMs(connection)), which = defaultWhich) {
     const targets = collectRequiredEngineTargets(plan);
     if (targets.length === 0)
         return [];
     const probesByConnection = new Map();
-    const probed = await Promise.all(targets.map(async (target) => {
-        const key = `${target.connection.endpoint.replace(/\/+$/, "")}|${target.connection.model}`;
+    const probeConnectionOnce = (connection) => {
+        const key = `${connection.endpoint.replace(/\/+$/, "")}|${connection.model}`;
         let pending = probesByConnection.get(key);
         if (!pending) {
             const probeStartedAt = Date.now();
-            pending = probeReachable(target.connection).then((reach) => ({
-                reach,
-                latencyMs: Date.now() - probeStartedAt,
-            }));
+            pending = probeReachable(connection).then((reach) => ({ reach, latencyMs: Date.now() - probeStartedAt }));
             probesByConnection.set(key, pending);
         }
-        const { reach, latencyMs } = await pending;
-        return { ...target, reach, latencyMs };
+        return pending;
+    };
+    const probed = await Promise.all(targets.map(async (target) => {
+        if (target.bin !== undefined && which(target.bin) === undefined) {
+            return { ...target, reach: { reachable: false, error: `${target.bin} is not on PATH` }, latencyMs: 0 };
+        }
+        if (!target.connection)
+            return { ...target, reach: { reachable: true }, latencyMs: 0 };
+        return { ...target, ...(await probeConnectionOnce(target.connection)) };
     }));
     const unreachable = probed.filter((item) => !item.reach.reachable);
     if (unreachable.length > 0) {
-        const lines = unreachable.map((item) => `  - ${item.process} (engine "${item.engine}", ${item.connection.endpoint}): ${item.reach.error ?? "did not respond"}`);
-        throw new ConfigError(`--require-engines: ${unreachable.length} improve process${unreachable.length === 1 ? "" : "es"} cannot run because ${unreachable.length === 1 ? "its" : "their"} engine completion path is not reachable:\n${lines.join("\n")}`, "LLM_NOT_CONFIGURED", "Check that each listed endpoint is up and serves its model. The probe is one short completion, bounded by the engine's timeoutMs (at most two minutes).");
+        const lines = unreachable.map((item) => `  - ${item.process} (engine "${item.engine}", ${item.connection?.endpoint ?? item.bin}): ${item.reach.error ?? "did not respond"}`);
+        throw new ConfigError(`--require-engines: ${unreachable.length} improve process${unreachable.length === 1 ? "" : "es"} cannot run because ${unreachable.length === 1 ? "its" : "their"} engine completion path is not reachable:\n${lines.join("\n")}`, "LLM_NOT_CONFIGURED", "Check that each listed endpoint is up and serves its model, and that each listed agent binary is installed. The endpoint probe is one short completion, bounded by the engine's timeoutMs (at most two minutes).");
     }
     return probed.map((item) => ({
         process: item.process,
         engine: item.engine,
-        endpoint: item.connection.endpoint,
+        endpoint: item.connection?.endpoint ?? item.bin,
         reachable: item.reach.reachable,
         latencyMs: item.latencyMs,
     }));
@@ -189,6 +190,32 @@ function rejectReportOnlyFlags(args) {
         return;
     throw new UsageError(`\`${flag}\` only applies to \`akm improve report\`. Use \`akm improve report ${flag} <value>\` instead.`, "INVALID_FLAG_VALUE");
 }
+/**
+ * `akm improve judge`: reflect's quality judge on one revision, read as
+ * `{"source", "candidate", "feedback"}` JSON from stdin, with the engine the
+ * strategy's reflect quality gate names. It writes nothing.
+ */
+async function runImproveJudgeCli(strategyName) {
+    const input = process.stdin.isTTY
+        ? {}
+        : JSON.parse((await readStdin()).toString("utf8"));
+    const { source, candidate, feedback, ref } = input;
+    if (typeof source !== "string" || typeof candidate !== "string") {
+        throw new UsageError('`akm improve judge` reads {"source": "...", "candidate": "...", "feedback": "...", "ref": "..."} JSON from stdin.', "MISSING_REQUIRED_ARGUMENT");
+    }
+    const config = loadConfig();
+    const judge = resolveQualityGateJudge(config, resolveImproveStrategy(strategyName, config).config, "reflect");
+    if (!judge) {
+        throw new ConfigError("`akm improve judge` judges with the reflect quality gate's engine. Set processes.reflect.qualityGate.engine.", "INVALID_CONFIG_FILE");
+    }
+    const notes = typeof feedback === "string" && feedback.trim() !== "" ? [feedback.trim()] : [];
+    const verdict = await runReflectQualityJudge(config, candidate, source, notes, undefined, {
+        runnerSelectionFrozen: true,
+        llmRunner: judge,
+        ...(typeof ref === "string" && ref ? { ref } : {}),
+    });
+    output("improve-judge", { engine: judge.engine, ...verdict });
+}
 export const improveCommand = defineCommand({
     meta: {
         name: "improve",
@@ -271,6 +298,10 @@ export const improveCommand = defineCommand({
                 return;
             }
             rejectReportOnlyFlags(args);
+            if (getStringArg(args, "scope") === "judge") {
+                await runImproveJudgeCli(getStringArg(args, "strategy"));
+                return;
+            }
             rejectRetiredImproveTargetFlag();
             const jsonToStdout = args["json-to-stdout"];
             const targetArg = getStringArg(args, "bundle");