akm-cli 0.10.261012-beta → 0.10.261014-beta

package/docs/architecture/README.md

@@ -1,67 +1,32 @@
 # Architecture Index
 
-How akm is built: system overview, normative specs, decision history, and
-subsystem internals.
+How akm is built: system overview, normative specs, and subsystem internals.
 
 - [Architecture](architecture.md) -- How akm's bundles, cache, index, and registries fit together
-- [Core Principles](akm-core-principles.md) -- Design principles and constraints
+- [Core Principles](akm-core-principles.md) -- How the code, tests and docs in this repo are built: the engineering rules and what enforces each
 - [Adapters](adapters.md) -- The `BundleAdapter` contract: what a bundle's files *are*, how they're indexed, linted, and written
 - [Architecture: The Workflow Engine](workflow-engine.md) -- How a workflow compiles to a frozen plan, persists run/unit state, dispatches work, and resumes without replaying completed steps
 - [Architecture: The Improvement Loop](improvement.md) -- Purpose and boundary of the improvement loop
-- [Runtime Boundary Design](runtime-boundary-design.md) -- Isolating `bun:sqlite`/`Bun.*` from the core
-- [Architecture Decision History](akm-architecture-decision-history.md) -- ADR-style record of the major architecture rulings
 
 ## Specs (`specs/`)
 
 Normative specifications and binding conventions.
 
 - [OKF format support](specs/okf-support.md) -- OKF is a first-class format supported through the built-in `okf` adapter
-- [0.9.0 surface decisions](specs/0.9.0-decisions.md) -- The decision record (D1-D12) behind the 0.9.0 breaking changes
-- [Bundle & Adapter Spec (0.9.0)](specs/akm-0.9.0-bundle-adapter-spec.md) -- Normative spec for bundles, adapters, and bundle recognition
-- [Ref Grammar Decision (0.9.0)](specs/akm-0.9.0-ref-grammar-decision.md) -- The `[bundle//]conceptId` ref grammar
 - [Ref Format](specs/ref.md) -- Wire format for asset references
-- [Format-Neutral Bundle Workspace Spec](specs/akm-format-neutral-bundle-workspace-spec.md) -- The format-neutral workspace model
-- [Agent, Command, Engine, and Model Resolution](specs/agent-command-engine-model-design.md) -- Approved target semantics for native agent/command bundles, configuration cascading, model maps, and dispatch
-- [0.9.2 Agent/Command/Task/Workflow Implementation Plan](../plans/0.9.2-agent-command-workflow-plan.md) -- Sequenced work packages, [GitHub milestone tracker](https://github.com/itlackey/akm/issues/801), migration rules, support boundaries, and release evidence for the coherent execution MVP
-- [Fact Asset Type](specs/fact-asset-type.md) -- The `fact` asset type
-- [Bundle Conventions Code Spec](specs/stash-conventions-code-spec.md) -- Code-level bundle conventions
-- [Bundle Organization Conventions](specs/stash-organization-conventions.md) -- How a bundle is laid out
-- [DI Seams Plan](specs/di-seams-plan.md) -- Dependency-injection seams used by the test suite
-- [Improve Collapse/Churn Detector](specs/improve-collapse-churn-detector-design.md) -- Longitudinal collapse/churn detection design (§6.3 is the operator runbook referenced by `akm health`)
-
-Historical review registers (0.9.0 release-review audit trail, kept for provenance, not normative going forward):
-
-- [0.9.0 Docs–Code Drift Register](specs/0.9.0-docs-code-drift-register.md) -- Ruled record of places documentation and code disagreed, and open-intent questions (Q-01..Q-19)
-- [0.9.0 Public API Issue Backlog](specs/0.9.0-public-api-issue-backlog.md) -- Implementation-only findings from the CLI surface review
-- [0.9.0 Release Surface Review](specs/0.9.0-release-surface-review.md) -- Reconciled action list from the end-to-end user-facing surface sweep
 
 ## Internals (`internals/`)
 
 Current-truth subsystem references.
 
 - [Storage Locations](internals/storage-locations.md) -- Authoritative inventory of every on-disk read/write path
-- [Search](internals/search.md) -- Hybrid search architecture and scoring
 - [Indexing](internals/indexing.md) -- How the search index is built
 - [Classification](internals/classification.md) -- Matcher and renderer behavior
 - [Improve Workflow](internals/improve-workflow.md) -- `akm improve` command surface and pipeline reference
 - [Health Advisories](internals/health-advisories.md) -- `akm health` advisory-to-action map for operators
-- [Registry Network Boundary](internals/registry-network-boundary.md) -- Outbound registry request inventory, destination policy, redirects, and DNS guarantees
-- [Functional Contract Patterns](internals/functional-contract-patterns.md) -- Quick reference for contributor pipelines and small process contracts
 - [Fresh-Host Rebuild Runbook](internals/fresh-host-rebuild-runbook.md) -- Rebuild an akm install on a new machine
 - [Persisted-Data Compatibility Contract](persisted-data-compat.md) -- What a reader owes data an earlier release wrote, per persisted format
 
-## Reviews (`reviews/`)
-
-Point-in-time architecture reviews of a specific subsystem question.
-
-- [Runtime secret resolution for in-process source fetchers](reviews/env-secret-access.md) -- Why secret reads are injected as a `SecretResolver` capability instead of imported inside the fetcher subgraph
-
-## Comparisons (`comparisons/`)
-
-- [Workflow architecture: Claude Code workflows vs. akm workflows](comparisons/claude-code-vs-akm-workflows-full.md) -- Full technical comparison of the Claude Code `Workflow` tool and akm workflows; see [the short vendor-neutral guide](../guides/claude-code-vs-akm-workflows.md) for the decision-level version
-
 ## Testing (`testing/`)
 
 - [Testing Workflow](testing/testing-workflow.md) -- End-to-end, Docker, deployment, and upgrade validation, plus the coverage gap guide
-- [Manual Testing Checklist](testing/manual-testing-checklist.md) -- Pre-release manual QA checklist
-- [OKF v0.2 Conformance Re-Evaluation Runbook](testing/okf-v0.2-conformance-runbook.md) -- Acceptance procedure to re-run after changing adapters, indexing, refs, writes, or lint behavior

package/docs/architecture/adapters.md

@@ -11,8 +11,8 @@ For read-only native tool bundles, recognition is runtime translation, not
 conversion: the adapter projects native files into AKM's index/runtime shapes
 while the source file remains authoritative. AKM does not synchronize native
 formats, maintain translated canonical copies, or write agent/command assets
-back through the Claude/OpenCode adapters. See
-[Agent, Command, Engine, and Model Resolution](specs/agent-command-engine-model-design.md).
+back through the Claude/OpenCode adapters. Execution of agent and command
+assets is described in [Architecture: Connections and Harnesses](architecture.md#connections-and-harnesses).
 
 > **Stability note.** The adapter set, bundle-recognition rules, and the
 > `bundles` config shape are still evolving — see [STABILITY.md](../../STABILITY.md).

package/docs/architecture/akm-core-principles.md

@@ -1,126 +1,264 @@
 # akm Core Principles
 
-akm is a portable capability library for AI agents — give every coding agent
-the capabilities your team has already built. It indexes existing agent
-assets in place, loads only what a task needs, packages capabilities into
-shareable bundles, improves the library through reviewable proposals, and
-runs durable workflows, locally and without tying the library to one
-assistant.
+How akm is built. These are the rules the code, the tests and the
+documentation in this repository follow. What akm does for a user is
+described in [Architecture](architecture.md), whose "Invariants" section
+lists the behaviors every command keeps.
 
-This document lists the design constraints that keep that library
-predictable.
+Each principle states the rule, why it exists, and what enforces it. A rule
+with nothing enforcing it is held by review, and says so.
 
-## What akm Does
+## Precedence
 
-An agent has a task. Across configured sources (local filesystem paths and
-cache-backed git/website/npm mirrors) and registry catalogs, akm helps it
-discover assets such as scripts, skills, commands, agents, knowledge docs,
-workflows, env files, secrets, and wiki pages.
-
-Core flow:
-
-```text
-connect -> index -> curate -> show -> use/run -> feedback -> proposal
-```
-
-Not every task uses every stage.
+When two principles pull against each other: a user's data comes first,
+then honesty (code and documentation that do what they say), then
+simplicity, then convenience.
 
 ## Principles
 
-### 1. Capabilities remain portable files
-
-Capabilities are ordinary files on disk — skills, scripts, workflows, agent
-definitions, instructions, memories, knowledge docs. akm indexes what's
-already there; the source files remain the source of truth, and akm's index
-is a derived, rebuildable view rather than a copy of record.
-
-### 2. Materialize sources locally and use one retrieval layer
-
-Local filesystem paths and cache-backed git/website/npm mirrors all
-materialize into one local index, so `search`, `curate`, and `show` work the
-same way regardless of where a capability originated. Registries stay a
-conceptually separate, read-only catalog of installable bundles: registry
-results live in `registryHits`, never merged into source `hits`.
-
-### 3. Every token must earn its place
-
-Default output should stay lean. Search and curate are for choosing; show is
-for using.
-
-- **Search is a menu.** Default search output should expose only enough to
-  choose the next asset. In the current CLI that usually means:
-  - `brief`: `type`, `name`, `ref`, `action`, `estimatedTokens`, `keys` —
-    `ref` is deliberately present at the leanest level so an agent can run
-    `akm show <ref>` without asking for fuller detail
-  - `normal`: adds `description` and `score`
-
-  Richer provenance/debug fields belong behind fuller detail modes.
-
-- **Show is a dispatch envelope.** Show should return the payload that lets
-  the consumer act:
-  - script execution hints
-  - skill instructions
-  - command templates
-  - agent prompts
-  - knowledge/wiki content
-  - workflow steps and parameters
-  - env key names without secret values
-  - secret metadata without the secret value
-
-### 4. Discovery chooses; show delivers
-
-Search and curate should not accumulate show-level detail. `full` detail
-modes can expose more metadata, but the base mental model stays:
-
-```text
-search/curate decide
-show delivers
-filesystem is optional depth
-```
-
-### 5. Refs are opaque durable handles
-
-Consumers should treat refs as opaque lookup handles. The current wire
-format is `[bundle//]conceptId[#fragment]`, but agents should pass refs
-through rather than parse them. See `docs/architecture/specs/ref.md`.
-
-### 6. Execution requires an explicit supported surface
-
-akm retrieves every supported capability type. It directly orchestrates
-defined execution surfaces such as workflows, agent dispatch, tasks, and
-guarded subprocess injection. It does not blindly execute arbitrary indexed
-content merely because that content appears in search results.
-
-### 7. Improvement is evidence-driven and reviewable
-
-Feedback and usage signal drive `akm improve`, which produces reviewable,
-diffable proposals rather than silent rewrites. Accept, reject, and revert
-stay human- or policy-gated, and proposals target only writable bundles.
-
-### 8. Writes are destination-aware and fail closed
-
-Writes target only writable bundles/adapters. Read-only adapters and
-protected env/secret values are never silently written through — a write
-aimed at a non-writable destination fails rather than falling back to an
-unintended target.
-
-### 9. Output serves agents first
-
-The default consumer is structured-output automation. JSON-first and
-concise detail levels are the right defaults.
-
-### 10. Complexity belongs behind indexing and source management
-
-The hard parts should stay inside indexing, source resolution, registry
-install, and provider plumbing, not in the hot path from `search`/`curate`
-to `show`/`use`.
-
-## What akm Does Not Do
-
-- blindly execute arbitrary indexed content just because it appears in
-  search results — akm orchestrates only defined execution surfaces
-  (workflows, agent dispatch, tasks, guarded subprocess injection)
-- expose secret/env values through `show`
-- replace live-service integrations such as MCP — akm complements MCP and
-  assistant-native skills rather than replacing them
-- require agents to understand source layouts or provider internals
+### 1. Delete before you add
+
+The default answer to a new guard, cap, version gate, option, mode or layer
+is no. A guard, cap, version gate, or fencing check survives only if it
+passes **all three** tests:
+
+1. **It has demonstrably helped a real user.** Check the local evidence before defending it — the `events` table in `state.db`, task logs, issue history. A guard that has never fired in production is a candidate for deletion, not preservation.
+2. **Its failure mode costs less than the hazard it prevents.** A cap whose trip makes the CLI unusable is worse than the slowness it was guarding against.
+3. **The operation is not already gated behind a deliberate human command.** Anything a person explicitly typed does not get a second machine-level gate. The operator is not racing themselves.
+
+"This hazard is conceivable" is not a justification — that always answers yes,
+and it is how this codebase accumulated footguns. When in doubt, delete.
+
+Preference order when something must change: **remove the limit** > degrade
+with a warning > abort. Aborting is the last resort, not the default.
+
+What this does not apply to: machinery that prevents **data loss or
+corruption** passes test 2 on its own merits and stays: backups, atomic
+writes, write-path validation that keeps malformed data out of the database,
+and path-containment checks that stop writes outside the bundle. The target is
+machinery that makes the tool refuse to do its job.
+
+The same rule applies to code that is not a guard: build what the current
+design needs and nothing for a future one (YAGNI). No speculative option,
+abstraction, extension point or "flexibility" goes in until a real caller
+needs it, and a mechanism nothing calls any more comes out.
+
+Why: every footgun this codebase has removed was a guard added because a
+hazard was conceivable; the limits made the tool refuse to do its job
+more often than they protected anyone.
+
+Worked examples, which are the precedent:
+
+- **#857** — single-ref lookup walked the entire bundle tree, capped at 16,384 files, and aborted when tripped. Zero useful firings in months of local event history; its only production behavior was making the CLI unusable on a large bundle. Walk and cap deleted, replaced with closed-form candidate enumeration. Net −13 lines of source, every collision guarantee intact.
+- **`extra-params` (#815/#816)** — hard-rejected legacy keys. Degraded to warn-and-auto-lift; only genuinely conflicting keys still reject. This is the template for a guard that had to stay but had the wrong failure mode.
+- **`guarded-source` 1MiB cap** — kept once on the argument that the bytes are hashed, so a truncated read would be silently wrong. That argument rejects *truncation*; it never justified a *cap*. Correct answer was no truncation and no cap.
+- **Migrator TOCTOU fencing** — inode/device/ctime identity checks defending a window inside a human-typed `akm migrate apply` that already holds a cross-process lock and writes backups. A multi-tenant-daemon threat model applied to a single-user CLI.
+
+Enforced by: review.
+
+### 2. One place per policy
+
+A rule lives in exactly one module and everything else calls it. Write
+destinations branch on source kind only in `src/core/write-source.ts`.
+Fan-out widths are decided once in `src/workflows/concurrency-policy.ts`.
+A call's timeout is resolved once in `src/integrations/agent/resolution.ts`,
+and no other layer adds one.
+A child process's environment is built in `childProcessEnv`
+(`src/core/paths.ts`): akm's own, with the directories this akm resolved set
+over it.
+Every agent harness registers once in `HARNESS_REGISTRY`
+(`src/integrations/harnesses/index.ts`) and core imports the registry, not
+a harness. Runtime-specific APIs (`bun:sqlite`, `Bun.*`) are reached only
+through `src/storage/database.ts` and `src/runtime.ts`.
+
+Why: a policy copied into three places drifts into three policies, and
+the drift is invisible until a user hits the one that differs.
+
+Enforced by: `tests/harnesses-registry.test.ts` for the registry;
+`scripts/lint-secret-resolver-boundary.ts` for secret reads;
+`tests/contracts/module-boundaries.test.ts` for the module table;
+`tests/contracts/runtime-port-boundary.test.ts` for akm's own runtime, which
+lowering and dispatch reach only through the `AgentRuntime` port
+(`src/integrations/agent/runtime-port.ts`), composed in
+`src/integrations/agent/runtime.ts`. The write chokepoint and the
+no-direct-harness-imports rule are held by review; the OpenCode plugin upgrade
+and staleness checks still import the OpenCode folder's installation helpers
+directly, which is open work.
+
+### 3. One current shape per format, one converter
+
+Each stored format has one current shape in code and one converter from older
+shapes. A reader uses the converter and keeps no tolerance of its own. A user's
+file (`config.json`, a task source, bundle content) is rewritten only by
+`akm migrate apply`, with a backup, running that same converter; akm's own
+stores (`state.db`, `index.db`) migrate forward when opened, with a backup
+before any destructive step. Reads reach back to the 0.9.15 floor: above it a
+reader converts in memory, warns once and names `akm migrate apply`; below it,
+and for an older task source (the one named exception), the command stops and
+names the migrator. What happens to data from a newer release differs by
+format (a newer index layout or frozen plan is refused with "upgrade akm";
+`state.db` and `config.json` warn and continue, and `akm health` reports the
+config skew), so [Persisted-Data Compatibility](persisted-data-compat.md)
+states the rule per format. Readers that still carry their own conversion, such
+as the frozen-plan decoder, are listed there and move to the converter as they
+are touched.
+
+Why: tolerance scattered through readers is a shim nobody can delete. A
+converter that the readers and the migrator share is written once, tested once
+and leaves one shape behind.
+
+Enforced by: `tests/integration/upgrade-rehearsal/`, which installs the
+previous release, builds real state and runs the current build against it, and
+`tests/integration/previous-release-corpus.test.ts`, which pins each older
+shape as refused or converted. The compat page lists each format with its test;
+a change to a format adds its old shape to the rehearsal.
+
+### 4. Sane defaults, flexible overrides
+
+Every default works out of the box for the common case, which for anything
+touching a model means a slow local model on a single-threaded server: no
+`max_tokens` sent unless asked, one timeout, concurrency one against a
+loopback endpoint. Every default is overridable where it applies, per
+connection or harness under `connections.<name>.*` and `harnesses.<name>.*`, per workflow step, per command flag, and
+the override is honored as given. The tool never second-guesses a value
+the user set; the one place it still does, clamping a workflow fan-out width
+above 64, is debt rather than policy.
+
+Why: defaults that need tuning before the first useful result lose the
+user; overrides that are ignored or capped behind the user's back lose
+their trust.
+
+Enforced by: `tests/workflows/concurrency-defaults.test.ts` and
+`tests/resolution.test.ts`; the rest is review.
+
+### 5. If it matters, a test or lint checks it
+
+Any rule we want to hold is pinned: output shapes and exit codes, the ref
+grammar, the module table, the execution transport contract, that every command and
+flag named in a doc exists, that shipped assets carry no dead refs. The pin
+is a test under `tests/contracts/`, a unit test, or a script in the `lint`
+chain. A rule nobody pinned is a preference, and may change without
+ceremony.
+
+Why: the documentation drifted from the code for months because the pages
+said "normative" and nothing checked.
+
+Enforced by: `bun run lint` (`scripts/lint-doc-examples.ts`,
+`scripts/lint-shipped-assets.ts`, `scripts/lint-secret-resolver-boundary.ts`),
+`tests/contracts/`, `tests/output-shapes-unit.test.ts`.
+
+### 6. The suite is green by default and sandboxed through the environment
+
+`test:unit` and `test:integration` report zero failures at all times. A
+red run is a real break, never noise to route around; a test that must be
+disabled is disabled explicitly with a linked issue.
+
+No test reads or writes a developer's real files: the runner scripts and the
+preload point everything a test can resolve from the environment at a
+sandbox, and a test that leaks fails.
+[Testing Workflow](testing/testing-workflow.md) states the two targets, where
+a test belongs, how the sandbox works and what it cannot cover.
+
+Why: a suite that is "usually green" is a suite nobody trusts, and a test
+that reads a developer's real config passes on one machine and fails on
+the next.
+
+Enforced by: `tests/_preload.ts` (the sandbox and the tripwire),
+`scripts/test-unit.sh` and `scripts/test-integration.sh` (the split and the
+per-shard `HOME`), `TEST_ISOLATION_MISSING` in `src/core/paths.ts`. Reads
+through `os.homedir()` outside the runner scripts are held by review.
+
+### 7. Prose lives in files; comments say why
+
+Multi-line markdown, XML or prose belongs in a standalone file next to the
+module that uses it, imported as text
+(`import x from "./x.md" with { type: "text" }`) and filled with
+`{{PLACEHOLDER}}` tokens through `.replace` or `.replaceAll` at call time.
+`src/tasks/backends/schtasks.ts` (which imports
+`src/assets/backends/schtasks-template.xml`), `src/output/cli-hints.ts` and
+`scripts/copy-assets.ts` are the established pattern.
+
+A code comment explains a constraint or a decision the code cannot show; it
+does not cite plans, specs, review rounds or history, and it does not restate
+the line beneath it.
+
+Why: templates inside template literals cannot be edited without touching
+source, and a comment that points at a document nobody can find is noise
+that looks like information.
+
+Enforced by: review.
+
+### 8. Documentation describes the product that exists
+
+One current-truth document per subsystem; no second one is forked. Plans,
+decision records, review registers and future-work promises do not live
+under `docs/`. Code is cited by file and symbol, not line number. A
+document claims no guarantee the code does not enforce, and names no
+command, flag or file that does not exist. When code and documentation
+disagree, one of them is fixed in the same change.
+
+Why: a user who reads a stale page loses more time than if the page had
+not existed, and a "normative" document the code ignores teaches
+contributors that documents do not matter.
+
+Enforced by: `scripts/lint-doc-examples.ts`, which reads the fenced examples
+in `docs/`, `src/assets/hints/`, `README.md`, `AGENTS.md` and `STABILITY.md`
+and checks that every command and flag in them exists (the migration documents
+of earlier releases, named in `HISTORICAL_MIGRATION_DOCS`, are exempt; the
+current migration guide and release note are read), and the npm pack check in
+`tests/integration/install/npm-pack-contents.test.ts` (no dead link ships).
+Whether a statement is true of the product is held by review.
+
+### 9. CLI-first, Bun-first
+
+This package is a command-line tool. `src/cli.ts` dispatches and commands
+live under `src/commands/`. The one programmatic entry point is `akm-cli/api`
+(`src/api.ts`, which exports only `curate`), so the akm plugins can recall
+in-process; the `exports` map in `package.json` lists only `./api`,
+`./package.json` and `./dist/*`, and there are no barrel exports. Bun is the
+preferred runtime and the toolchain. Node 22 is the floor (`engines.node`):
+the npm package bootstraps through Node, and its launcher falls back to Node
+when no usable Bun is on `PATH`. Anything that would become another API for
+other programs is a separate decision, not an incidental export.
+
+Why: an accidental API is a compatibility promise nobody agreed to make.
+
+Enforced by: `tests/contracts/api-surface.test.ts` (the export list),
+`tests/integration/install/package-exports.test.ts` (the `exports` map),
+`package.json` (`engines.node` and the `preinstall` check), and the
+`node-smoke` job in `.github/workflows/ci.yml` (Node 22 and 24).
+
+### 10. Agent harnesses are thin, replaceable internals
+
+akm is simple, lightweight tooling for agents and automated processes. A
+harness such as OpenCode (pi or another later) is an internal detail akm
+uses to enable agentic work. It sits behind one thin seam, is not woven into
+akm core, and can be replaced without touching the rest of akm. What akm adds
+to a harness run, for agent work and for model work, is stated once in
+[Configuration](../reference/configuration.md#how-akm-runs-a-harness).
+
+Why: heavy machinery and tight coupling are the opposite of akm's goals.
+
+Enforced by: review. The seam is `HARNESS_REGISTRY` (principle 2); akm's own
+runtime sits behind the `AgentRuntime` port (pinned by
+`tests/contracts/runtime-port-boundary.test.ts`), and only the OpenCode plugin
+checks still bypass the seam.
+
+### 11. Simple is not sloppy
+
+Principle 1 removes machinery that is not justified; it does not lower the
+standard of what remains. Code is kept simple with SOLID structure, not
+instead of it: a module has one responsibility and one reason to change;
+behaviour is extended by adding a case in its one place (principle 2), not
+by editing every caller; a caller depends on the narrow interface it uses,
+not on the internals behind it; types say what a value is. Shrinking a diff
+by merging unrelated concerns, dropping the error handling the design calls
+for, or flattening clear structure into one long function is not
+simplification and is reverted in review.
+
+Why: the layers this codebase has removed were added without need, but the
+remedy for over-engineering is a small clean design, not a monolith. Both
+cost the next reader.
+
+Enforced by: review; `tests/contracts/module-boundaries.test.ts` for the
+module seams it pins.