akm-cli 0.9.0-rc.0 → 0.9.0-rc.1

package/CHANGELOG.md

@@ -6,6 +6,129 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Added
+
+- **Workflow orchestration engine (experimental).** akm can now execute
+  multi-step workflows as deterministic **YAML programs**, driven either by a
+  native engine or by any agent session. This is a new, self-contained
+  surface; classic linear **markdown workflows and the stable workflow CLI
+  contract (`start`/`next`/`complete`/`status`/`list`) are unchanged**. What
+  ships:
+  - **Authoring.** Orchestrated workflows are YAML programs
+    (`workflows/*.yaml`, `version: 1`) validated against a published JSON
+    Schema (`schemas/akm-workflow.json`) by `akm workflow validate`; scaffold
+    one with `akm workflow template --yaml` or `akm workflow create
+    <name>.yaml`. A closed `${{ … }}` expression language (exactly
+    `params.<name>`, `steps.<id>.output.<path>`, `item`, `item_index`, parsed
+    once into an AST) wires steps together. `validate` also surfaces non-fatal
+    **warnings** (a step with no typed `output:` schema; a `${{ params.<name> }}`
+    reference to a param the declared `params:` block omits) that never change
+    the frozen plan or its hash. Creating a workflow whose canonical name
+    collides with an existing asset of a **different** extension (`foo.yaml`
+    while `foo.md` exists, or vice-versa) is refused, since the two would
+    silently shadow each other.
+  - **Compilation + frozen plans.** `akm workflow start` compiles the program
+    into a backend-agnostic Workflow Plan Graph IR (`src/workflows/ir/`) and
+    freezes it on the run row (`plan_json` + `plan_hash`); a run executes the
+    plan compiled at start, and edits to the source file require a new run.
+  - **Per-step orchestration.** A step can declare a runner, model, timeout,
+    fan-out (`map`/`over` with a `concurrency` cap and a `collect` | `vote`
+    reducer), a typed `output` JSON Schema (validated via a `runStructured`
+    retry-with-feedback loop), `env` bindings (resolved through the existing
+    `akm env run` machinery — secret tokens, dangerous-key policy, keys-only
+    audit events), classify-and-dispatch `route` steps, and `depends_on`
+    ordering.
+  - **Determinism + replay.** Journaled unit identity is content-derived
+    (`<step>:<sha256(item)[:12]>`, `:solo` for a single unit), so cached
+    results survive item-list reordering; a completed unit whose recorded
+    inputs differ on replan is a hard **replay-divergence** failure naming the
+    unit, never a silent re-dispatch. Every unit is recorded in the new
+    `workflow_run_units` table behind a serialized writer queue.
+  - **Execution (`akm workflow run`).** A semaphore-bounded scheduler fans a
+    step's units out (concurrency defaults to 1 per the local-model
+    LLM-defaults rule, capped by the engine-wide `workflow.maxConcurrency`
+    config knob — unset = `min(16, max(1, cores − 2))`, explicit values
+    clamped to `[1, 64]`), enforces per-unit
+    timeouts (default 10 m) and run **budget ceilings** (`budget.max_tokens` /
+    `budget.max_units`, seeded from the journal so they span resumes), and
+    advances the run **strictly through `completeWorkflowStep`** so completion
+    gates are never bypassed. Every dispatched unit gets a standard akm
+    preamble (run/unit ids, knowledge + env/secret + reporting contract).
+  - **Typed artifacts + honest gates.** A step's promoted artifact is
+    validated against its declared `output` schema before completion; a
+    criteria-bearing gate judges that **artifact** (canonical JSON, clipped)
+    rather than machine prose, and each engine-driven evaluation is journaled
+    as a gate unit row. `gate.max_loops` bounds an evaluator-optimizer retry
+    loop (feedback threaded into re-dispatched unit prompts); `gate.required`
+    (or the run-wide `--require-gates`) makes a gate with no available judge
+    **block** for a human instead of failing open.
+  - **Failure policy.** Per-unit `on_error: fail | continue` (fail-fast
+    default) plus bounded `retry: { max, on: [<failure_reason>…] }` keyed on
+    the persisted failure taxonomy.
+  - **Isolation + leases.** `isolation: worktree` runs each file-mutating unit
+    in a fresh detached git worktree (journaled path; clean trees
+    auto-removed, dirty ones retained). A run **lease** (`engine_lease_*`)
+    ensures a run is driven by exactly one engine or one external driver at a
+    time; manual `complete` is refused while a live engine lease is held.
+  - **Harness-neutral driver protocol.** An orchestrated run can be driven by
+    ANY agent session (Claude Code, opencode, Codex, a human at a shell), not
+    only the native engine. **`akm workflow brief <run>`** is read-only (takes
+    no lease, mutates nothing) and emits the active step's expected work-list —
+    per-unit content-derived id, resolved instructions + input hash
+    (byte-identical to the engine's dispatch), output schema, env binding
+    NAMES only, and the exact `report` command lines. **`akm workflow report
+    <run> --unit <id> --status completed|failed|running`** is the one mutating
+    verb, ingesting a unit's result through the SAME shared step semantics the
+    engine uses (idempotent same-hash re-report, replay-divergence on a
+    differing hash, budget enforcement, schema validation, and the
+    artifact-judged gate/`max_loops` completion path). `--status running`
+    claims/heartbeats a unit for stale-driver detection without advancing the
+    spine; `--rerun` records a fresh attempt for a failed unit (carrying its
+    prior token total forward). Every report command carries `--expect-step`
+    (refused if the spine has moved since the brief), and `report --settle`
+    (no `--unit`) advances a step that dispatches no reportable units — a
+    params-only route, an empty fan-out, or an all-unresolvable work-list — so
+    a driver is never wedged. The engine and the brief/report surfaces are
+    proven to produce **identical unit graphs**
+    (`tests/workflows/conformance/driver-parity.test.ts`).
+  - **Observability.** `akm workflow watch <run>` tails the run's `workflow_*`
+    / `workflow_unit_*` events as NDJSON (`--stream` foreground-polls to a
+    terminal status, no daemon); `akm workflow status --units` lists per-unit
+    diagnostics (failure reason + result/error text) without feeding them into
+    the deterministic artifact graph; unit lifecycle emits
+    `workflow_unit_started` / `workflow_unit_finished` events carrying
+    ids/status/enums only. `akm show workflow:<name>` summarizes each step's
+    orchestration.
+  - **Harness adapters.** Seven local coding-agent CLIs are first-class
+    dispatch targets — Codex, Copilot CLI, Pi, Gemini, Aider, Amazon Q, and
+    OpenHands — each registered in `HARNESS_REGISTRY` with a command builder +
+    result extractor; agent-identity detection and the session-log provider
+    list are derived from the registry, and harness-native session ids are
+    journaled opportunistically for future session reuse.
+  - **Storage.** Additive `workflow.db` migrations 004–009 (unit journal,
+    harness session ids, frozen plans + run leases, check-in heartbeats,
+    attempt counter, unit claims); migrations 001–003 are untouched and linear
+    workflows behave exactly as before.
+
+  See "Orchestrated steps" and "Driving a run from any agent" in
+  `docs/features/workflows.md`, the redesign addendum in
+  `docs/technical/akm-workflows-orchestration-plan.md`, and `STABILITY.md`
+  (Experimental).
+- **`fable` built-in model alias** — resolves to `claude-fable-5`
+  (`opencode/claude-fable-5` on opencode); recommended resolution target for
+  the `deep` workflow model tier.
+
+### Fixed
+
+- **Check-in directives now survive plain-text output and `workflow
+  status`** (check-in review C2/M1): `formatWorkflowNextPlain` and
+  `formatWorkflowStatusPlain` render the `CONTINUE` directive, and every
+  run-detail response (status/start/complete) evaluates the check-in instead
+  of only `workflow next`.
+- Workflow frontmatter validator error message now lists the actually-allowed
+  keys (`name`, `updated` were missing); removed the documented-but-nonexistent
+  `akm workflow step` alias from `docs/features/workflows.md`.
+
 ## [0.9.0] — 2026-06-30
 
 ### Fixed

package/dist/assets/prompts/workflow-unit-preamble.md

@@ -0,0 +1,26 @@
+You are executing one unit of an akm workflow run.
+
+- Workflow run: {{RUN_ID}}
+- Step: {{STEP_ID}}
+- Unit: {{UNIT_ID}}
+- Run parameters: {{PARAMS_JSON}}
+
+Ground rules for this unit:
+
+1. Pull knowledge on demand instead of guessing: `akm search '<query>'` to find
+   relevant assets, `akm show <ref>` to read one, `akm curate '<query>'` to let
+   akm select the best match. Only pull what this unit actually needs.
+2. Environment values and secrets are provided through your process
+   environment when the workflow declares them. Never print secret values to
+   stdout or embed them in your answer. If you need an env file path, use
+   `akm env path <ref>`; never `cat` secrets.
+3. Do exactly the work described in the instructions below — no more. Other
+   units may be running concurrently on sibling items; do not touch files or
+   state outside the scope this unit was given.
+4. Your final output IS the unit result recorded by the engine. When a JSON
+   schema is requested, respond with ONLY the JSON value (no prose, no code
+   fences). Otherwise finish with a concise factual summary of what you did.
+
+Unit instructions follow.
+
+---

package/dist/commands/env/env-binding.js

@@ -0,0 +1,95 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+/**
+ * Env-ref → resolved values, shared between `akm env run` and the workflow
+ * engine's per-unit env bindings (orchestration plan, *Dispatch context*).
+ *
+ * Extracted from `env-cli.ts`'s `runEnvInjected` so the native executor
+ * REUSES the exact same machinery instead of forking it. Every safety
+ * invariant carries over unchanged:
+ *
+ *   - `${secret:NAME}` tokens resolve against the env's own stash; a missing
+ *     secret is a hard error and NOTHING is injected (no partial injection).
+ *   - Known process-hijacking keys (LD_PRELOAD, PATH, …) are blocked for
+ *     third-party-sourced stashes and warned about for first-party ones.
+ *   - An `env_access` audit event records key NAMES only, never values.
+ *   - Resolved values must never be written to stdout or logs by callers.
+ */
+import fs from "node:fs";
+import path from "node:path";
+import { resolveAssetPathFromName } from "../../core/asset/asset-spec.js";
+import { isWithin } from "../../core/common.js";
+import { makeEnvRef, resolveEnvPath } from "../../core/env-secret-ref.js";
+import { NotFoundError, UsageError } from "../../core/errors.js";
+import { appendEvent } from "../../core/events.js";
+import { isDangerousEnvKey } from "../lint/env-key-rules.js";
+import { loadEnv, resolveSecretTokens } from "./env.js";
+import { readValue } from "./secret.js";
+/**
+ * Resolve an env ref to its injectable values: load the file, apply key
+ * filtering, substitute `${secret:NAME}` tokens from the sibling secrets
+ * directory, enforce the dangerous-key policy, and append the keys-only
+ * `env_access` audit event.
+ */
+export function resolveEnvBinding(target, options = {}) {
+    const warn = options.warn ?? ((message) => process.stderr.write(`warning: ${message}\n`));
+    const { name, absPath, source } = resolveEnvPath(target);
+    const envRef = makeEnvRef(name, source);
+    if (!fs.existsSync(absPath)) {
+        throw new NotFoundError(`Env not found: ${envRef}`);
+    }
+    const allValues = loadEnv(absPath);
+    // Value-safe key filtering (--only / --except operate on key NAMES only).
+    let envValues = allValues;
+    if (options.only && options.except) {
+        throw new UsageError("Pass only one of --only or --except.", "INVALID_FLAG_VALUE");
+    }
+    if (options.only) {
+        const wanted = new Set(options.only);
+        const missing = options.only.filter((k) => !(k in allValues));
+        if (missing.length > 0) {
+            warn(`--only key(s) not present in ${envRef}: ${missing.join(", ")}`);
+        }
+        envValues = Object.fromEntries(Object.entries(allValues).filter(([k]) => wanted.has(k)));
+    }
+    else if (options.except) {
+        const excluded = new Set(options.except);
+        envValues = Object.fromEntries(Object.entries(allValues).filter(([k]) => !excluded.has(k)));
+    }
+    // Substitute `${secret:NAME}` tokens with the sibling secret asset in the
+    // SAME stash. A missing secret is a hard error — inject NOTHING.
+    const secretsRoot = path.join(source.path, "secrets");
+    const resolveSecret = (secretName) => {
+        const secretPath = resolveAssetPathFromName("secret", secretsRoot, secretName);
+        // Defense-in-depth: ensure the resolved path stays inside the secrets dir.
+        if (!isWithin(secretPath, secretsRoot)) {
+            throw new UsageError(`Secret name "${secretName}" escapes the secrets directory.`);
+        }
+        if (!fs.existsSync(secretPath))
+            return undefined;
+        // Match `secret run`: read utf8, do not trim (stay consistent with that path).
+        return readValue(secretPath).toString("utf8");
+    };
+    const { values: substituted, missing } = resolveSecretTokens(envValues, resolveSecret);
+    if (missing.length > 0) {
+        throw new NotFoundError(`Env "${envRef}" references secret(s) not found in its stash: ${missing.map((n) => `secret:${n}`).join(", ")}. Nothing was injected.`, "FILE_NOT_FOUND", `Create the missing secret, e.g. \`akm secret set secret:${missing[0]}\`.`);
+    }
+    envValues = substituted;
+    const keys = Object.keys(envValues);
+    // Scan injected keys for known process-hijacking variables. Block for
+    // third-party-sourced stashes (origin has a registryId); warn for the
+    // operator's own first-party stash, where they own the file.
+    const dangerous = keys.filter(isDangerousEnvKey);
+    if (dangerous.length > 0) {
+        const detail = `Env "${envRef}" injects process-hijacking variable(s): ${dangerous.join(", ")}.`;
+        if (source.registryId) {
+            throw new UsageError(`Refusing to inject env from a third-party stash. ${detail}\n` +
+                `       Review the file, then copy the values into a first-party env if you trust them.`, "INVALID_FLAG_VALUE");
+        }
+        warn(`${detail} Injecting anyway (first-party stash).`);
+    }
+    // Audit trail: keys only, never values.
+    appendEvent({ eventType: "env_access", ref: envRef, metadata: { keys } });
+    return { ref: envRef, values: envValues, keys };
+}

package/dist/commands/env/env-cli.js

@@ -27,7 +27,6 @@ import { isWithin, writeFileAtomic } from "../../core/common.js";
 import { loadConfig } from "../../core/config/config.js";
 import { findEnvSource, makeEnvRef, parseEnvRef, resolveEnvPath } from "../../core/env-secret-ref.js";
 import { ConfigError, NotFoundError, UsageError } from "../../core/errors.js";
-import { appendEvent } from "../../core/events.js";
 import { isQuiet } from "../../core/warn.js";
 import { resolveSourceEntries } from "../../indexer/search/search-source.js";
 import { parseFlagValue } from "../../output/context.js";
@@ -232,64 +231,14 @@ async function runEnvInjected(target, opts) {
         }
         throw new NotFoundError(`Env not found: ${makeEnvRef(name, source)}`);
     }
-    const { loadEnv } = await import("./env.js");
-    const allValues = loadEnv(absPath);
-    // Value-safe key filtering (--only / --except operate on key NAMES only).
-    let envValues = allValues;
-    if (opts.only && opts.except) {
-        throw new UsageError("Pass only one of --only or --except.", "INVALID_FLAG_VALUE");
-    }
-    if (opts.only) {
-        const wanted = new Set(opts.only);
-        const missing = opts.only.filter((k) => !(k in allValues));
-        if (missing.length > 0) {
-            process.stderr.write(`warning: --only key(s) not present in ${makeEnvRef(name, source)}: ${missing.join(", ")}\n`);
-        }
-        envValues = Object.fromEntries(Object.entries(allValues).filter(([k]) => wanted.has(k)));
-    }
-    else if (opts.except) {
-        const excluded = new Set(opts.except);
-        envValues = Object.fromEntries(Object.entries(allValues).filter(([k]) => !excluded.has(k)));
-    }
-    // Substitute `${secret:NAME}` tokens in values with the value of the sibling
-    // secret asset in the SAME stash. The lookup is injected so commands/env.ts
-    // keeps its narrow dependency surface; we resolve each name against this env's
-    // own `source`. A missing secret is a hard error — inject NOTHING (no partial
-    // injection). Resolved values are never logged or printed.
-    const { resolveSecretTokens } = await import("./env.js");
-    const { readValue } = await import("./secret.js");
-    const secretsRoot = path.join(source.path, "secrets");
-    const resolveSecret = (secretName) => {
-        const secretPath = resolveAssetPathFromName("secret", secretsRoot, secretName);
-        // Defense-in-depth: ensure the resolved path stays inside the secrets dir.
-        if (!isWithin(secretPath, secretsRoot)) {
-            throw new UsageError(`Secret name "${secretName}" escapes the secrets directory.`);
-        }
-        if (!fs.existsSync(secretPath))
-            return undefined;
-        // Match `secret run`: read utf8, do not trim (stay consistent with that path).
-        return readValue(secretPath).toString("utf8");
-    };
-    const { values: substituted, missing } = resolveSecretTokens(envValues, resolveSecret);
-    if (missing.length > 0) {
-        const envRef = makeEnvRef(name, source);
-        throw new NotFoundError(`Env "${envRef}" references secret(s) not found in its stash: ${missing.map((n) => `secret:${n}`).join(", ")}. Nothing was injected.`, "FILE_NOT_FOUND", `Create the missing secret, e.g. \`akm secret set secret:${missing[0]}\`.`);
-    }
-    envValues = substituted;
-    const keys = Object.keys(envValues);
-    // Scan injected keys for known process-hijacking variables (LD_PRELOAD,
-    // PATH, ...). Block for third-party-sourced stashes (origin has a registryId);
-    // warn for the operator's own first-party stash, where they own the file.
-    const { isDangerousEnvKey } = await import("../lint/env-key-rules.js");
-    const dangerous = keys.filter(isDangerousEnvKey);
-    if (dangerous.length > 0) {
-        const detail = `Env "${makeEnvRef(name, source)}" injects process-hijacking variable(s): ${dangerous.join(", ")}.`;
-        if (source.registryId) {
-            throw new UsageError(`Refusing to inject env from a third-party stash. ${detail}\n` +
-                `       Review the file, then copy the values into a first-party env if you trust them.`, "INVALID_FLAG_VALUE");
-        }
-        process.stderr.write(`warning: ${detail} Injecting anyway (first-party stash).\n`);
-    }
+    // Load → filter → secret-substitute → dangerous-key policy → keys-only
+    // audit event. Shared with the workflow engine's per-unit env bindings —
+    // see env-binding.ts for the extracted core and its safety invariants.
+    const { resolveEnvBinding } = await import("./env-binding.js");
+    const { values: envValues } = resolveEnvBinding(target, {
+        only: opts.only,
+        except: opts.except,
+    });
     const mergedEnv = buildChildEnv(process.env, {
         clean: opts.clean === true,
         inherit: opts.inherit ?? [],
@@ -297,12 +246,6 @@ async function runEnvInjected(target, opts) {
     for (const [envKey, envValue] of Object.entries(envValues)) {
         mergedEnv[envKey] = envValue;
     }
-    // Audit trail: keys only, never values.
-    appendEvent({
-        eventType: "env_access",
-        ref: makeEnvRef(name, source),
-        metadata: { keys },
-    });
     const result = spawnSync(command[0], command.slice(1), {
         stdio: "inherit",
         env: mergedEnv,

package/dist/commands/sources/migration-help.js

@@ -116,12 +116,15 @@ export function renderMigrationHelp(versionInput, changelogText = loadChangelog(
     if (changelogText) {
         for (const candidate of candidates) {
             const section = extractChangelogSection(changelogText, candidate);
-            if (section) {
-                const bundled = loadReleaseNote(candidate);
-                if (!bundled)
-                    return `${section.trim()}\n\nFull changelog: ${CHANGELOG_URL}\n`;
+            const bundled = loadReleaseNote(candidate);
+            if (bundled) {
+                if (!section)
+                    return `${bundled.trim()}\n\nFull changelog: ${CHANGELOG_URL}\n`;
                 return `${bundled.trim()}\n\nRelease notes\n-------------\n${section.trim()}\n\nFull changelog: ${CHANGELOG_URL}\n`;
             }
+            if (section) {
+                return `${section.trim()}\n\nFull changelog: ${CHANGELOG_URL}\n`;
+            }
         }
     }
     const fallbackVersion = candidates.find((candidate) => candidate !== "latest") ?? requested;