akm-cli 0.9.30 → 0.9.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/README.md +10 -10
- package/ROADMAP.md +112 -0
- package/dist/commands/improve/improve-cli.js +33 -12
- package/dist/core/file-lock.js +10 -1
- package/dist/llm/client.js +33 -3
- package/dist/scripts/akm-migrate-node.js +5 -1
- package/dist/scripts/akm-migrate.js +5 -1
- package/docs/README.md +42 -41
- package/docs/agents/README.md +15 -0
- package/docs/agents/agent-install.md +14 -0
- package/docs/architecture/README.md +67 -0
- package/docs/architecture/adapters.md +272 -0
- package/docs/architecture/akm-architecture-decision-history.md +2055 -0
- package/docs/architecture/akm-core-principles.md +126 -0
- package/docs/architecture/architecture.md +579 -0
- package/docs/architecture/comparisons/claude-code-vs-akm-workflows-full.md +550 -0
- package/docs/architecture/decisions/0001-no-interpolation-attached-structured-context.md +72 -0
- package/docs/architecture/decisions/0002-unit-reuse-and-input-hash-scope.md +142 -0
- package/docs/architecture/decisions/0003-child-env-allowlist-and-provenance.md +189 -0
- package/docs/architecture/decisions/0004-task-input-contract-and-flag-coercion.md +97 -0
- package/docs/architecture/decisions/0005-task-result-vocabulary-and-legacy-read-mapping.md +108 -0
- package/docs/architecture/decisions/0006-task-source-version-routing.md +124 -0
- package/docs/architecture/decisions/0007-workflow-composition-bounds.md +83 -0
- package/docs/architecture/decisions/0008-task-binding-normalization.md +88 -0
- package/docs/architecture/decisions/0009-child-run-publication-column-parity.md +80 -0
- package/docs/architecture/decisions/0010-driver-protocol-cut.md +36 -0
- package/docs/architecture/decisions/0011-engine-run-loop-invariants.md +125 -0
- package/docs/architecture/decisions/README.md +62 -0
- package/docs/architecture/improvement.md +440 -0
- package/docs/architecture/internals/classification.md +154 -0
- package/docs/architecture/internals/fresh-host-rebuild-runbook.md +40 -0
- package/docs/architecture/internals/functional-contract-patterns.md +340 -0
- package/docs/architecture/internals/health-advisories.md +110 -0
- package/docs/architecture/internals/improve-workflow.md +713 -0
- package/docs/architecture/internals/indexing.md +498 -0
- package/docs/architecture/internals/registry-network-boundary.md +57 -0
- package/docs/architecture/internals/search.md +249 -0
- package/docs/architecture/internals/storage-locations.md +943 -0
- package/docs/architecture/persisted-data-compat.md +59 -0
- package/docs/architecture/reviews/env-secret-access.md +319 -0
- package/docs/architecture/runtime-boundary-design.md +153 -0
- package/docs/architecture/specs/0.9.0-close-out-plan.md +361 -0
- package/docs/architecture/specs/0.9.0-decisions.md +528 -0
- package/docs/architecture/specs/0.9.0-docs-code-drift-register.md +745 -0
- package/docs/architecture/specs/0.9.0-open-items-register.md +135 -0
- package/docs/architecture/specs/0.9.0-public-api-issue-backlog.md +740 -0
- package/docs/architecture/specs/0.9.0-release-surface-review.md +636 -0
- package/docs/architecture/specs/agent-command-engine-model-design.md +513 -0
- package/docs/architecture/specs/akm-0.9.0-bundle-adapter-spec.md +501 -0
- package/docs/architecture/specs/akm-0.9.0-ref-grammar-decision.md +291 -0
- package/docs/architecture/specs/akm-format-neutral-bundle-workspace-spec.md +2059 -0
- package/docs/architecture/specs/di-seams-plan.md +539 -0
- package/docs/architecture/specs/driver-protocol-keep-or-cut.md +275 -0
- package/docs/architecture/specs/fact-asset-type.md +142 -0
- package/docs/architecture/specs/improve-collapse-churn-detector-design.md +634 -0
- package/docs/architecture/specs/okf-support.md +148 -0
- package/docs/architecture/specs/ref.md +329 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/README.md +72 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/findings-consolidated.md +152 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/library-eval-alternatives.md +352 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/library-eval-bunqueue.md +300 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/library-eval-openworkflow.md +402 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/opus-review-1.md +758 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/opus-review-2.md +645 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/opus-review-3.md +602 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/panel-r1-adversarial.md +27 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/panel-r1-architecture.md +27 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/panel-r1-coverage.md +74 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/panel-r1-security.md +15 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/panel-r2-adversarial.md +132 -0
- package/docs/architecture/specs/reviews/task-workflow-unification/panel-r2-verification.md +147 -0
- package/docs/architecture/specs/stash-conventions-code-spec.md +275 -0
- package/docs/architecture/specs/stash-organization-conventions.md +377 -0
- package/docs/architecture/specs/task-workflow-format-unification.md +721 -0
- package/docs/architecture/specs/workflow-engine-buy-vs-build.md +231 -0
- package/docs/architecture/specs/workflow-format-unification.md +437 -0
- package/docs/architecture/testing/manual-testing-checklist.md +3420 -0
- package/docs/architecture/testing/okf-v0.2-conformance-runbook.md +924 -0
- package/docs/architecture/testing/task-workflow-manual-tests.md +1817 -0
- package/docs/architecture/testing/testing-workflow.md +446 -0
- package/docs/architecture/workflow-engine.md +274 -0
- package/docs/guides/README.md +33 -0
- package/docs/guides/author-bundles.md +310 -0
- package/docs/guides/author-workflows.md +520 -0
- package/docs/guides/bundles.md +321 -0
- package/docs/guides/capture-knowledge.md +136 -0
- package/docs/guides/claude-code-vs-akm-workflows.md +30 -0
- package/docs/guides/concepts.md +93 -0
- package/docs/guides/discover-and-load.md +158 -0
- package/docs/guides/getting-started.md +240 -0
- package/docs/guides/improve-the-library.md +246 -0
- package/docs/guides/knowledge-management.md +18 -0
- package/docs/guides/recipes/headless-install.md +97 -0
- package/docs/guides/recipes/website-source.md +28 -0
- package/docs/guides/run-workflows.md +279 -0
- package/docs/guides/scheduling.md +123 -0
- package/docs/guides/use-with-any-agent.md +129 -0
- package/docs/guides/wikis.md +81 -0
- package/docs/maintainers/README.md +19 -0
- package/docs/maintainers/curate-workmap.md +336 -0
- package/docs/maintainers/eval.md +3 -0
- package/docs/maintainers/local-development.md +108 -0
- package/docs/maintainers/release-checklist.md +103 -0
- package/docs/migration/README.md +2 -2
- package/docs/migration/release-notes/0.0.13.md +1 -1
- package/docs/migration/release-notes/0.1.0.md +1 -1
- package/docs/migration/release-notes/0.2.0.md +1 -1
- package/docs/migration/release-notes/0.3.0.md +1 -1
- package/docs/migration/release-notes/0.5.0.md +1 -1
- package/docs/migration/release-notes/0.6.0.md +1 -1
- package/docs/migration/release-notes/0.7.0.md +1 -1
- package/docs/migration/release-notes/0.7.3.md +2 -2
- package/docs/migration/release-notes/0.7.4.md +1 -1
- package/docs/migration/release-notes/0.7.5.md +1 -1
- package/docs/migration/release-notes/0.8.0.md +1 -1
- package/docs/migration/release-notes/0.9.0.md +1 -1
- package/docs/migration/release-notes/0.9.14.md +1 -1
- package/docs/migration/release-notes/0.9.15.md +1 -1
- package/docs/migration/release-notes/0.9.16.md +1 -1
- package/docs/migration/release-notes/0.9.17.md +1 -1
- package/docs/migration/release-notes/0.9.19.md +1 -1
- package/docs/migration/release-notes/0.9.2.md +1 -1
- package/docs/migration/release-notes/README.md +27 -31
- package/docs/migration/v0.5-to-v0.6.md +759 -0
- package/docs/migration/v0.8-to-v0.9.md +1 -1
- package/docs/plans/0.9.2-agent-command-workflow-plan.md +423 -0
- package/docs/posts/README.md +30 -0
- package/docs/posts/akm-command-tour.md +495 -0
- package/docs/posts/akm-logo-sized.webp +0 -0
- package/docs/posts/belief-aware-memory-12.md +139 -0
- package/docs/posts/config-v2-lessons-16.md +191 -0
- package/docs/posts/improve-loop-autonomous-10.md +225 -0
- package/docs/posts/improve-pipeline-debugging-0803.md +179 -0
- package/docs/posts/intro-part-01.md +76 -0
- package/docs/posts/intro-part-02.md +221 -0
- package/docs/posts/intro-part-03.md +190 -0
- package/docs/posts/intro-part-04.md +221 -0
- package/docs/posts/llm-mode-reflect-13.md +166 -0
- package/docs/posts/openpalm-built-on-akm.md +86 -0
- package/docs/posts/progressive-disclosure-06.md +129 -0
- package/docs/posts/proposal-safety-net-15.md +112 -0
- package/docs/posts/release-0.5.0.md +247 -0
- package/docs/posts/release-0.6.0.md +117 -0
- package/docs/posts/release-0.7.0.md +159 -0
- package/docs/posts/release-0.8.0.md +332 -0
- package/docs/posts/scattered-skills-05.md +100 -0
- package/docs/posts/task-assets-persistent-workflows-11.md +222 -0
- package/docs/posts/team-skills-07.md +198 -0
- package/docs/posts/wiki-agent-knowledge-08.md +223 -0
- package/docs/posts/workflows-vaults-09.md +321 -0
- package/docs/product-surface.md +54 -0
- package/docs/reference/README.md +1 -2
- package/docs/reference/asset-types.md +168 -0
- package/docs/reference/cli.md +5 -6
- package/docs/reference/env-and-secrets.md +126 -0
- package/docs/reference/memory.md +124 -0
- package/docs/reference/refs.md +156 -0
- package/docs/reference/registry.md +489 -0
- package/docs/reference/website-sources.md +141 -0
- package/package.json +12 -15
|
@@ -0,0 +1,3420 @@
|
|
|
1
|
+
# AKM Manual Testing Runbook
|
|
2
|
+
|
|
3
|
+
This is the authoritative manual QA document for AKM `0.9.x`. Other testing
|
|
4
|
+
documents describe automation and internals; this document owns manual test
|
|
5
|
+
scope, setup, expected behavior, evidence, and release-facing gates.
|
|
6
|
+
|
|
7
|
+
Last audited against the implementation: **2026-08-05** (`0.9.0-rc.15`).
|
|
8
|
+
|
|
9
|
+
## Table of Contents
|
|
10
|
+
|
|
11
|
+
- [1. Purpose and Use](#1-purpose-and-use)
|
|
12
|
+
- [2. Safety and Prerequisites](#2-safety-and-prerequisites)
|
|
13
|
+
- [3. Sandbox Setup](#3-sandbox-setup)
|
|
14
|
+
- [4. Fixtures and Local Services](#4-fixtures-and-local-services)
|
|
15
|
+
- [5. Fast Offline Pass](#5-fast-offline-pass)
|
|
16
|
+
- [6. Startup, Setup, Help, and Config](#6-startup-setup-help-and-config)
|
|
17
|
+
- [7. Index, Search, Curate, and Show](#7-index-search-curate-and-show)
|
|
18
|
+
- [8. Output, Errors, and Exit Codes](#8-output-errors-and-exit-codes)
|
|
19
|
+
- [9. Bundles, Adapters, Sources, and Registries](#9-bundles-adapters-sources-and-registries)
|
|
20
|
+
- [10. Writes, Import, Clone, and Sync](#10-writes-import-clone-and-sync)
|
|
21
|
+
- [11. Env and Secret](#11-env-and-secret)
|
|
22
|
+
- [12. Lint](#12-lint)
|
|
23
|
+
- [13. Feedback, Log, and Health](#13-feedback-log-and-health)
|
|
24
|
+
- [14. Workflow](#14-workflow)
|
|
25
|
+
- [15. Task](#15-task)
|
|
26
|
+
- [16. Proposal Queue](#16-proposal-queue)
|
|
27
|
+
- [17. Agent, LLM, and Improve](#17-agent-llm-and-improve)
|
|
28
|
+
- [18. Semantic Search](#18-semantic-search)
|
|
29
|
+
- [19. Migration, Durability, and Concurrency](#19-migration-durability-and-concurrency)
|
|
30
|
+
- [20. Security and Adversarial Testing](#20-security-and-adversarial-testing)
|
|
31
|
+
- [21. Package, Runtime, Platform, and Release](#21-package-runtime-platform-and-release)
|
|
32
|
+
- [22. Evidence and Cleanup](#22-evidence-and-cleanup)
|
|
33
|
+
- [23. Change-Based Test Selection](#23-change-based-test-selection)
|
|
34
|
+
- [24. Known Constraints and Blocked Gates](#24-known-constraints-and-blocked-gates)
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## 1. Purpose and Use
|
|
39
|
+
|
|
40
|
+
### 1.1 Scope
|
|
41
|
+
|
|
42
|
+
This runbook covers:
|
|
43
|
+
|
|
44
|
+
- CLI startup, help, setup, config, output shaping, and error classification.
|
|
45
|
+
- Indexing, ranking, search, curate, show, refs, fragments, and usage tracking.
|
|
46
|
+
- Every built-in bundle adapter and every supported source provider.
|
|
47
|
+
- Writes, target precedence, git publication, envs, secrets, and redaction.
|
|
48
|
+
- Workflow, task, proposal, agent, LLM, and improve behavior.
|
|
49
|
+
- Semantic search, migration, crash recovery, locking, and concurrent processes.
|
|
50
|
+
- npm packages, standalone binaries, installers, self-upgrade, schedulers, and
|
|
51
|
+
release artifacts.
|
|
52
|
+
- Security boundaries including SSRF, path traversal, symlinks, dangerous env
|
|
53
|
+
keys, malformed input, rollback, and partial publication.
|
|
54
|
+
|
|
55
|
+
This is a catalog, not an instruction to run every gate for every change. Use
|
|
56
|
+
[Change-Based Test Selection](#23-change-based-test-selection) to choose scope.
|
|
57
|
+
|
|
58
|
+
### 1.2 Test tiers
|
|
59
|
+
|
|
60
|
+
Every check carries one or more labels.
|
|
61
|
+
|
|
62
|
+
| Label | Meaning | Typical time | Token/network cost |
|
|
63
|
+
| --- | --- | ---: | --- |
|
|
64
|
+
| **[CORE]** | Fast deterministic offline gate | 25-35 min | None |
|
|
65
|
+
| **[LOCAL]** | Extended local filesystem/process gate | +45-75 min | None |
|
|
66
|
+
| **[SERVICE]** | Controlled loopback fixture services | +15-25 min | None |
|
|
67
|
+
| **[LIVE]** | Controlled public git/npm/HTTPS service | +15-30 min | Network only |
|
|
68
|
+
| **[AI]** | Fake or bounded real agent/LLM checks | +15-30 min | Fake: none; real: bounded |
|
|
69
|
+
| **[PLATFORM]** | Native OS/runtime/install checks | Per platform | Runner/VM time |
|
|
70
|
+
| **[DESTRUCTIVE]** | Crash, migration, and recovery rehearsal | 1-3 hr | Disposable VM only |
|
|
71
|
+
| **[RELEASE]** | Exact package and release-artifact acceptance | 1-3 hr | CI/VM/network |
|
|
72
|
+
|
|
73
|
+
### 1.3 Result states
|
|
74
|
+
|
|
75
|
+
Use these states exactly:
|
|
76
|
+
|
|
77
|
+
| Status | Meaning |
|
|
78
|
+
| --- | --- |
|
|
79
|
+
| `PASS` | Behavior, exit code, output, and side effects match the stated expectation. |
|
|
80
|
+
| `FAIL` | Any observable result differs. |
|
|
81
|
+
| `BLOCKED` | A named prerequisite is unavailable. Record it. |
|
|
82
|
+
| `N/A` | The target platform genuinely does not support the surface. |
|
|
83
|
+
|
|
84
|
+
Do not turn an implementation defect into an expected result. Some adversarial
|
|
85
|
+
gates below intentionally expose unresolved behavior. A release waiver must name
|
|
86
|
+
the issue, impact, owner, and expiry.
|
|
87
|
+
|
|
88
|
+
### 1.4 Evidence header
|
|
89
|
+
|
|
90
|
+
Record this once per run:
|
|
91
|
+
|
|
92
|
+
```text
|
|
93
|
+
commit:
|
|
94
|
+
package version:
|
|
95
|
+
artifact digest (when applicable):
|
|
96
|
+
OS / architecture:
|
|
97
|
+
Bun / Node versions:
|
|
98
|
+
CLI invocation path:
|
|
99
|
+
sandbox path:
|
|
100
|
+
tiers run:
|
|
101
|
+
started / finished UTC:
|
|
102
|
+
tester:
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
For every failure retain the command, exit code, stdout, stderr, relevant file
|
|
106
|
+
or database diff, and whether a second identical run reproduced it. Never
|
|
107
|
+
attach real credentials or unredacted env/secret payloads.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## 2. Safety and Prerequisites
|
|
112
|
+
|
|
113
|
+
### 2.1 Non-negotiable safety rules
|
|
114
|
+
|
|
115
|
+
- [ ] Use a disposable shell and preferably a disposable clean worktree.
|
|
116
|
+
- [ ] Isolate `HOME`, all four `XDG_*_HOME` variables, all four `AKM_*_DIR`
|
|
117
|
+
variables, temp files, GitHub CLI config, Git config, npm config/cache,
|
|
118
|
+
and Hugging Face cache under one sandbox root.
|
|
119
|
+
- [ ] Set `AKM_CONFIG_DIR`. It overrides `XDG_CONFIG_HOME`; omitting it can
|
|
120
|
+
overwrite a real config before path verification.
|
|
121
|
+
- [ ] Clear inherited AKM debug/registry settings and provider, agent, GitHub,
|
|
122
|
+
LLM, and embedding credentials.
|
|
123
|
+
- [ ] Set both `AKM_FORCE_SETUP_TMP_STASH=1` and
|
|
124
|
+
`AKM_FORCE_INIT_TMP_STASH=1`; setup and bundle-create use separate guards.
|
|
125
|
+
- [ ] Use `dist/akm` for release-facing checks. Use `bun ./src/cli.ts` only when
|
|
126
|
+
intentionally validating source execution.
|
|
127
|
+
- [ ] Never run `akm upgrade`, native scheduler mutation, migration failure
|
|
128
|
+
injection, or kill-point tests on the developer account or host install.
|
|
129
|
+
- [ ] Never point `akm sync` at a real writable remote.
|
|
130
|
+
- [ ] Never use production secrets as fixtures.
|
|
131
|
+
- [ ] Do not use internal migration test-hook variables manually. Run the
|
|
132
|
+
shipped recovery suites or use a dedicated disposable recovery harness.
|
|
133
|
+
- [ ] Guard recursive cleanup with the exact sandbox prefix.
|
|
134
|
+
|
|
135
|
+
The sandbox isolates AKM state. `bun install` and `bun run build` still operate
|
|
136
|
+
inside the checkout, so release QA should use a disposable worktree and start
|
|
137
|
+
from a clean `git status`.
|
|
138
|
+
|
|
139
|
+
### 2.2 Prerequisites
|
|
140
|
+
|
|
141
|
+
**[CORE]** and **[LOCAL]** require Bash, Bun, Node.js 22+, Git, `jq`, standard
|
|
142
|
+
POSIX filesystem tools, and the repository checkout.
|
|
143
|
+
|
|
144
|
+
| Gate | Additional prerequisite |
|
|
145
|
+
| --- | --- |
|
|
146
|
+
| YAML parsing | Repository `yaml` dependency |
|
|
147
|
+
| Git source/sync | Disposable remote or local no-push repository |
|
|
148
|
+
| npm source | Disposable package/version or controlled registry |
|
|
149
|
+
| Production website | Controlled public HTTPS hostname; loopback is blocked |
|
|
150
|
+
| Local semantic model | Cold-download network, disk, isolated `HF_HOME` |
|
|
151
|
+
| Docker | Docker/BuildKit and base-image network access |
|
|
152
|
+
| Native scheduler | Disposable OS account or VM |
|
|
153
|
+
| Release artifact | Exact candidate tarball/binaries/checksums and native runners |
|
|
154
|
+
|
|
155
|
+
### 2.3 Automated baseline before manual release QA
|
|
156
|
+
|
|
157
|
+
```sh
|
|
158
|
+
git status --short
|
|
159
|
+
bun install --frozen-lockfile
|
|
160
|
+
bun run check
|
|
161
|
+
bun run build
|
|
162
|
+
test -x dist/akm
|
|
163
|
+
test ! -e dist/tests
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
- [ ] **[RELEASE]** `bun run check` exits `0`.
|
|
167
|
+
- [ ] **[RELEASE]** Build exits `0`; `dist/tests` does not exist.
|
|
168
|
+
- [ ] **[RELEASE]** Recheck `git status --short`. Only understood generated
|
|
169
|
+
changes are present. Do not format/rewrite the release candidate during
|
|
170
|
+
acceptance.
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
## 3. Sandbox Setup
|
|
175
|
+
|
|
176
|
+
### 3.1 Create the sandbox
|
|
177
|
+
|
|
178
|
+
Run from the repository root. The shell function is deliberate; it is more
|
|
179
|
+
reliable than an alias in copied scripts.
|
|
180
|
+
|
|
181
|
+
```sh
|
|
182
|
+
set -u
|
|
183
|
+
umask 077
|
|
184
|
+
|
|
185
|
+
export REPO="$(git rev-parse --show-toplevel)"
|
|
186
|
+
export AKM_BIN="$REPO/dist/akm"
|
|
187
|
+
export AKM_SANDBOX="$(mktemp -d /tmp/akm-sandbox.XXXXXX)"
|
|
188
|
+
|
|
189
|
+
export HOME="$AKM_SANDBOX/home"
|
|
190
|
+
export XDG_CONFIG_HOME="$AKM_SANDBOX/xdg-config"
|
|
191
|
+
export XDG_CACHE_HOME="$AKM_SANDBOX/xdg-cache"
|
|
192
|
+
export XDG_DATA_HOME="$AKM_SANDBOX/xdg-data"
|
|
193
|
+
export XDG_STATE_HOME="$AKM_SANDBOX/xdg-state"
|
|
194
|
+
export AKM_CONFIG_DIR="$AKM_SANDBOX/config"
|
|
195
|
+
export AKM_CACHE_DIR="$AKM_SANDBOX/cache"
|
|
196
|
+
export AKM_DATA_DIR="$AKM_SANDBOX/data"
|
|
197
|
+
export AKM_STATE_DIR="$AKM_SANDBOX/state"
|
|
198
|
+
export AKM_BUNDLE_DIR="$AKM_SANDBOX/bundle"
|
|
199
|
+
export TMPDIR="$AKM_SANDBOX/tmp"
|
|
200
|
+
export TMP="$TMPDIR"
|
|
201
|
+
export TEMP="$TMPDIR"
|
|
202
|
+
export HF_HOME="$AKM_SANDBOX/huggingface"
|
|
203
|
+
export GH_CONFIG_DIR="$AKM_SANDBOX/gh"
|
|
204
|
+
export GIT_CONFIG_GLOBAL="$AKM_SANDBOX/gitconfig"
|
|
205
|
+
export GIT_CONFIG_NOSYSTEM=1
|
|
206
|
+
export NPM_CONFIG_USERCONFIG="$AKM_SANDBOX/npmrc"
|
|
207
|
+
export npm_config_cache="$AKM_SANDBOX/npm-cache"
|
|
208
|
+
export AKM_FORCE_SETUP_TMP_STASH=1
|
|
209
|
+
export AKM_FORCE_INIT_TMP_STASH=1
|
|
210
|
+
|
|
211
|
+
unset AKM_VERBOSE AKM_DEBUG AKM_REGISTRY_URL AKM_NPM_REGISTRY
|
|
212
|
+
unset AKM_LLM_API_KEY AKM_EMBED_API_KEY AKM_NO_AUTO_MIGRATE
|
|
213
|
+
unset AKM_EMBED_DETERMINISTIC AKM_UPGRADE_SKIP_CHECKSUM
|
|
214
|
+
unset GITHUB_TOKEN GH_TOKEN OPENAI_API_KEY ANTHROPIC_API_KEY
|
|
215
|
+
unset OPENCODE_API_KEY X_BEARER_TOKEN X_RSS_TEMPLATE
|
|
216
|
+
|
|
217
|
+
mkdir -p \
|
|
218
|
+
"$HOME" "$XDG_CONFIG_HOME" "$XDG_CACHE_HOME" "$XDG_DATA_HOME" \
|
|
219
|
+
"$XDG_STATE_HOME" "$AKM_CONFIG_DIR" "$AKM_CACHE_DIR" \
|
|
220
|
+
"$AKM_DATA_DIR" "$AKM_STATE_DIR" "$AKM_BUNDLE_DIR" "$TMPDIR" \
|
|
221
|
+
"$HF_HOME" "$GH_CONFIG_DIR" "$npm_config_cache"
|
|
222
|
+
|
|
223
|
+
akm() { "$AKM_BIN" "$@"; }
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
For source-entrypoint testing only:
|
|
227
|
+
|
|
228
|
+
```sh
|
|
229
|
+
akm() { bun "$REPO/src/cli.ts" "$@"; }
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
### 3.2 Verify isolation before mutation
|
|
233
|
+
|
|
234
|
+
`config path --all` is a read-only recovery surface and works before config
|
|
235
|
+
exists.
|
|
236
|
+
|
|
237
|
+
```sh
|
|
238
|
+
akm config path --all --format json >"$AKM_SANDBOX/paths.json"
|
|
239
|
+
jq -e --arg root "$AKM_SANDBOX/" \
|
|
240
|
+
'all(to_entries[]; (.value | tostring | startswith($root)))' \
|
|
241
|
+
"$AKM_SANDBOX/paths.json"
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
- [ ] **[CORE]** Every reported path starts with `$AKM_SANDBOX/`.
|
|
245
|
+
- [ ] **[CORE]** `akm --version` matches `jq -r .version package.json`.
|
|
246
|
+
- [ ] **[CORE]** No real provider/agent credential remains in `env`.
|
|
247
|
+
|
|
248
|
+
Stop immediately if a path escapes the sandbox.
|
|
249
|
+
|
|
250
|
+
### 3.3 Deterministic non-interactive setup
|
|
251
|
+
|
|
252
|
+
Do not use `setup --yes` to seed ranking checks. It can perform host-dependent
|
|
253
|
+
environment/agent detection. Avoid scaffolding so fixture counts remain exact.
|
|
254
|
+
|
|
255
|
+
```sh
|
|
256
|
+
akm setup \
|
|
257
|
+
--config '{"semanticSearchMode":"off","registries":[]}' \
|
|
258
|
+
--dir "$AKM_BUNDLE_DIR" \
|
|
259
|
+
--no-init \
|
|
260
|
+
--format json >"$AKM_SANDBOX/setup.json"
|
|
261
|
+
|
|
262
|
+
jq -e --arg dir "$AKM_BUNDLE_DIR" \
|
|
263
|
+
'.written == true and .bundleDir == $dir' \
|
|
264
|
+
"$AKM_SANDBOX/setup.json"
|
|
265
|
+
test ! -e "$AKM_BUNDLE_DIR/skills"
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
- [ ] **[CORE]** Setup exits `0`, writes only sandbox config, and does not
|
|
269
|
+
scaffold under `--no-init`.
|
|
270
|
+
- [ ] **[CORE]** `config list` contains `configVersion: "0.9.0"`, `bundles`, and
|
|
271
|
+
`defaultBundle`; retired root keys are absent.
|
|
272
|
+
|
|
273
|
+
### 3.4 Classified-error helper
|
|
274
|
+
|
|
275
|
+
Use this only for thrown/classified failures. `--quiet` keeps stderr to one JSON
|
|
276
|
+
envelope.
|
|
277
|
+
|
|
278
|
+
```sh
|
|
279
|
+
expect_error() {
|
|
280
|
+
expected_status="$1"
|
|
281
|
+
expected_code="$2"
|
|
282
|
+
shift 2
|
|
283
|
+
|
|
284
|
+
set +e
|
|
285
|
+
akm --quiet "$@" \
|
|
286
|
+
>"$AKM_SANDBOX/error.stdout" \
|
|
287
|
+
2>"$AKM_SANDBOX/error.stderr"
|
|
288
|
+
actual_status=$?
|
|
289
|
+
set -e
|
|
290
|
+
|
|
291
|
+
test "$actual_status" -eq "$expected_status"
|
|
292
|
+
test ! -s "$AKM_SANDBOX/error.stdout"
|
|
293
|
+
jq -e --arg code "$expected_code" \
|
|
294
|
+
'.ok == false and .code == $code and (.error | type == "string")' \
|
|
295
|
+
"$AKM_SANDBOX/error.stderr"
|
|
296
|
+
}
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
Do not use this helper for health warnings/failures, lint findings,
|
|
300
|
+
workflow/task/agent domain failures, or blocked migration plans. Those emit a
|
|
301
|
+
normal result on stdout with a nonzero status.
|
|
302
|
+
|
|
303
|
+
### 3.5 Byte snapshot helper
|
|
304
|
+
|
|
305
|
+
```sh
|
|
306
|
+
snapshot_tree() {
|
|
307
|
+
root="$1"
|
|
308
|
+
output="$2"
|
|
309
|
+
(
|
|
310
|
+
cd "$root"
|
|
311
|
+
find . -type f -print0 | LC_ALL=C sort -z | xargs -0 sha256sum
|
|
312
|
+
) >"$output"
|
|
313
|
+
}
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
On platforms without `sha256sum`, use the native SHA-256 utility and record it.
|
|
317
|
+
For failed stateful commands, snapshot config, lockfile, bundle, cache, data,
|
|
318
|
+
state, and event cursor, not only asset files.
|
|
319
|
+
|
|
320
|
+
---
|
|
321
|
+
|
|
322
|
+
## 4. Fixtures and Local Services
|
|
323
|
+
|
|
324
|
+
### 4.1 Exact ranking baseline
|
|
325
|
+
|
|
326
|
+
Copy fixture contents, including dotfiles, without setup scaffolding:
|
|
327
|
+
|
|
328
|
+
```sh
|
|
329
|
+
cp -R "$REPO/tests/fixtures/stashes/ranking-baseline/." "$AKM_BUNDLE_DIR/"
|
|
330
|
+
|
|
331
|
+
for entry in agents commands knowledge skills MANIFEST.json; do
|
|
332
|
+
test -e "$AKM_BUNDLE_DIR/$entry"
|
|
333
|
+
done
|
|
334
|
+
test ! -e "$AKM_BUNDLE_DIR/scripts"
|
|
335
|
+
|
|
336
|
+
akm index --full --format json >"$AKM_SANDBOX/index-baseline.json"
|
|
337
|
+
jq -e '.totalEntries == 18 and .verification.ok == true' \
|
|
338
|
+
"$AKM_SANDBOX/index-baseline.json"
|
|
339
|
+
|
|
340
|
+
akm info --format json | jq -e '
|
|
341
|
+
.indexStats.entryCount == 18 and
|
|
342
|
+
.indexStats.byType == {
|
|
343
|
+
"skill": 4,
|
|
344
|
+
"knowledge": 10,
|
|
345
|
+
"command": 2,
|
|
346
|
+
"agent": 2
|
|
347
|
+
}
|
|
348
|
+
'
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
- [ ] **[CORE]** The fixture itself has exactly `agents`, `commands`,
|
|
352
|
+
`knowledge`, `skills`, and `MANIFEST.json` at its root.
|
|
353
|
+
- [ ] **[CORE]** Full index reports exactly 18 entries and exact type counts.
|
|
354
|
+
|
|
355
|
+
Stable refs:
|
|
356
|
+
|
|
357
|
+
| Type | Ref |
|
|
358
|
+
| --- | --- |
|
|
359
|
+
| skill | `skills/k8s-deploy` |
|
|
360
|
+
| skill | `skills/docker-homelab` |
|
|
361
|
+
| knowledge | `knowledge/incident-response-runbook` |
|
|
362
|
+
| agent | `agents/code-reviewer` |
|
|
363
|
+
| command | `commands/release-manager` |
|
|
364
|
+
|
|
365
|
+
### 4.2 Supplemental manual-QA bundle
|
|
366
|
+
|
|
367
|
+
Copy this only after exact ranking assertions:
|
|
368
|
+
|
|
369
|
+
```sh
|
|
370
|
+
cp -R "$REPO/tests/fixtures/manual-qa/bundle/." "$AKM_BUNDLE_DIR/"
|
|
371
|
+
akm index
|
|
372
|
+
akm show knowledge/qa-guide | jq -e '.name == "qa-guide"'
|
|
373
|
+
akm show workflows/typed-route | jq -e '.type == "workflow"'
|
|
374
|
+
akm show tasks/manual-success | jq -e '.type == "task"'
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
It supplies `.meta`, typed/gated workflows, command tasks, and fake-agent input
|
|
378
|
+
assets. It contains no credentials.
|
|
379
|
+
|
|
380
|
+
### 4.3 Start controlled fixture services
|
|
381
|
+
|
|
382
|
+
The service binds only to `127.0.0.1`, records route/model/count/presence
|
|
383
|
+
metadata, and exposes chat, embedding, registry, and website routes.
|
|
384
|
+
|
|
385
|
+
```sh
|
|
386
|
+
export AKM_QA_SERVICE_META="$AKM_SANDBOX/service.json"
|
|
387
|
+
export AKM_QA_SERVICE_LOG="$AKM_SANDBOX/service-requests.jsonl"
|
|
388
|
+
export AKM_QA_SITE_VERSION="$AKM_SANDBOX/site-version.txt"
|
|
389
|
+
printf 'v1\n' >"$AKM_QA_SITE_VERSION"
|
|
390
|
+
|
|
391
|
+
bun "$REPO/tests/fixtures/manual-qa/fake-services.ts" \
|
|
392
|
+
"$AKM_QA_SERVICE_META" \
|
|
393
|
+
"$AKM_QA_SERVICE_LOG" \
|
|
394
|
+
"$AKM_QA_SITE_VERSION" \
|
|
395
|
+
>"$AKM_SANDBOX/service.stdout" \
|
|
396
|
+
2>"$AKM_SANDBOX/service.stderr" &
|
|
397
|
+
export AKM_QA_SERVICE_PID=$!
|
|
398
|
+
|
|
399
|
+
for _ in $(seq 1 100); do
|
|
400
|
+
test -s "$AKM_QA_SERVICE_META" && break
|
|
401
|
+
kill -0 "$AKM_QA_SERVICE_PID"
|
|
402
|
+
sleep 0.05
|
|
403
|
+
done
|
|
404
|
+
test -s "$AKM_QA_SERVICE_META"
|
|
405
|
+
|
|
406
|
+
export AKM_QA_BASE_URL="$(jq -r .baseUrl "$AKM_QA_SERVICE_META")"
|
|
407
|
+
export AKM_QA_CHAT_URL="$(jq -r .chat.ok "$AKM_QA_SERVICE_META")"
|
|
408
|
+
export AKM_QA_PROPOSAL_URL="$(jq -r .chat.proposal "$AKM_QA_SERVICE_META")"
|
|
409
|
+
export AKM_QA_EMBED_URL="$(jq -r .embeddings "$AKM_QA_SERVICE_META")"
|
|
410
|
+
export AKM_QA_REGISTRY_URL="$(jq -r .registry "$AKM_QA_SERVICE_META")"
|
|
411
|
+
export AKM_QA_WEBSITE_URL="$(jq -r .website "$AKM_QA_SERVICE_META")"
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
- [ ] **[SERVICE]** Metadata appears within five seconds and all endpoints are
|
|
415
|
+
loopback.
|
|
416
|
+
- [ ] **[SERVICE]** Request logs contain no authorization or prompt values.
|
|
417
|
+
|
|
418
|
+
Loopback website ingestion needs command-scoped `NODE_ENV=test` because
|
|
419
|
+
production correctly blocks private hosts. That proves crawler protocol, not
|
|
420
|
+
production SSRF behavior.
|
|
421
|
+
|
|
422
|
+
### 4.4 Configure fake agent engines
|
|
423
|
+
|
|
424
|
+
```sh
|
|
425
|
+
export AKM_QA_BUN="$(command -v bun)"
|
|
426
|
+
export AKM_QA_FAKE_AGENT="$REPO/tests/fixtures/manual-qa/fake-agent.ts"
|
|
427
|
+
|
|
428
|
+
agent_engine_json() {
|
|
429
|
+
mode="$1"
|
|
430
|
+
shift
|
|
431
|
+
extra='[]'
|
|
432
|
+
if test "$#" -gt 0; then
|
|
433
|
+
extra="$(printf '%s\n' "$@" | jq -R . | jq -s .)"
|
|
434
|
+
fi
|
|
435
|
+
jq -nc \
|
|
436
|
+
--arg bin "$AKM_QA_BUN" \
|
|
437
|
+
--arg script "$AKM_QA_FAKE_AGENT" \
|
|
438
|
+
--arg mode "$mode" \
|
|
439
|
+
--argjson extra "$extra" \
|
|
440
|
+
'{kind:"agent",platform:"opencode",bin:$bin,args:([$script,$mode] + $extra)}'
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
akm config set engines.qa-agent "$(agent_engine_json success)" >/dev/null
|
|
444
|
+
akm config set engines.qa-agent-fail "$(agent_engine_json fail)" >/dev/null
|
|
445
|
+
akm config set engines.qa-agent-capture "$(agent_engine_json capture)" >/dev/null
|
|
446
|
+
akm config set engines.qa-agent-secret "$(agent_engine_json echo-secret)" >/dev/null
|
|
447
|
+
akm config set engines.qa-judge-pass "$(agent_engine_json judge-pass)" >/dev/null
|
|
448
|
+
akm config set engines.qa-judge-reject "$(agent_engine_json judge-reject)" >/dev/null
|
|
449
|
+
akm config set defaults.engine qa-agent >/dev/null
|
|
450
|
+
akm config set workflow.judgeEngine qa-judge-pass >/dev/null
|
|
451
|
+
|
|
452
|
+
export AKM_QA_SIGNAL_MARKER="$AKM_SANDBOX/fake-agent.signal"
|
|
453
|
+
akm config set engines.qa-agent-sleep \
|
|
454
|
+
"$(agent_engine_json sleep 10000 "$AKM_QA_SIGNAL_MARKER")" >/dev/null
|
|
455
|
+
akm config set engines.qa-agent-sleep.timeoutMs 100 >/dev/null
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
### 4.5 Configure fake LLM and embedding engines
|
|
459
|
+
|
|
460
|
+
Run after starting fixture services:
|
|
461
|
+
|
|
462
|
+
```sh
|
|
463
|
+
export QA_LLM_KEY='manual-qa-not-a-real-secret'
|
|
464
|
+
|
|
465
|
+
akm config set engines.qa-chat "$(jq -nc \
|
|
466
|
+
--arg endpoint "$AKM_QA_CHAT_URL" \
|
|
467
|
+
'{kind:"llm",endpoint:$endpoint,model:"qa-model",apiKey:"$QA_LLM_KEY",timeoutMs:1000}')" >/dev/null
|
|
468
|
+
akm config set defaults.llmEngine qa-chat >/dev/null
|
|
469
|
+
|
|
470
|
+
akm config set embedding "$(jq -nc \
|
|
471
|
+
--arg endpoint "$AKM_QA_EMBED_URL" \
|
|
472
|
+
'{provider:"openai",endpoint:$endpoint,model:"qa-embedding",dimension:4,batchSize:4}')" >/dev/null
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
### 4.6 Seed proposal cases
|
|
476
|
+
|
|
477
|
+
```sh
|
|
478
|
+
bun "$REPO/tests/fixtures/manual-qa/seed-proposals.ts" \
|
|
479
|
+
"$AKM_BUNDLE_DIR" >"$AKM_SANDBOX/proposals.json"
|
|
480
|
+
akm index
|
|
481
|
+
jq -e '.update.id and .newAsset.id and .emptyDiff.id and .defer.id' \
|
|
482
|
+
"$AKM_SANDBOX/proposals.json"
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
The seeder creates a fixed update, new asset, empty diff, and deferred candidate.
|
|
486
|
+
It is destructive and must target only the sandbox.
|
|
487
|
+
|
|
488
|
+
---
|
|
489
|
+
|
|
490
|
+
## 5. Fast Offline Pass
|
|
491
|
+
|
|
492
|
+
The fast gate uses no network, model download, provider token, agent process,
|
|
493
|
+
scheduler, or migration. Run:
|
|
494
|
+
|
|
495
|
+
| Order | Required coverage |
|
|
496
|
+
| ---: | --- |
|
|
497
|
+
| 1 | Sections 3.1-3.3: sandbox and deterministic setup |
|
|
498
|
+
| 2 | Section 4.1: exact ranking baseline |
|
|
499
|
+
| 3 | Section 6.1: version/root/help surface |
|
|
500
|
+
| 4 | Sections 7.1-7.4: index/search/curate/show |
|
|
501
|
+
| 5 | Sections 8.1-8.5: representative formats and errors |
|
|
502
|
+
| 6 | Sections 10.1-10.3: local remember/import writes |
|
|
503
|
+
| 7 | Sections 11.1-11.3: env/secret redaction |
|
|
504
|
+
| 8 | Section 12.1: lint detect/fix/idempotence |
|
|
505
|
+
| 9 | Section 13.1: feedback/log smoke |
|
|
506
|
+
|
|
507
|
+
Success criteria:
|
|
508
|
+
|
|
509
|
+
- [ ] Every **[CORE]** check passes.
|
|
510
|
+
- [ ] A second index/read pass preserves exact fixture counts and ranking.
|
|
511
|
+
- [ ] No secret fixture value appears in output, logs, reports, config, or events.
|
|
512
|
+
- [ ] No file outside `$AKM_SANDBOX` changes after setup begins.
|
|
513
|
+
|
|
514
|
+
---
|
|
515
|
+
|
|
516
|
+
## 6. Startup, Setup, Help, and Config
|
|
517
|
+
|
|
518
|
+
### 6.1 Version, root help, and command help
|
|
519
|
+
|
|
520
|
+
```sh
|
|
521
|
+
test "$(akm --version)" = "$(jq -r .version "$REPO/package.json")"
|
|
522
|
+
akm --help >"$AKM_SANDBOX/root-help.txt"
|
|
523
|
+
|
|
524
|
+
for command in \
|
|
525
|
+
setup index health info bundle upgrade search curate show workflow \
|
|
526
|
+
remember import sync clone registry migrate config feedback log agent \
|
|
527
|
+
lint improve proposal help hints completions env secret task; do
|
|
528
|
+
test "$(grep -Ec "^ ${command}[[:space:]]{2,}" "$AKM_SANDBOX/root-help.txt")" -eq 1
|
|
529
|
+
```
|
|
530
|
+
|
|
531
|
+
- [ ] **[CORE]** Version matches `package.json` exactly.
|
|
532
|
+
- [ ] **[CORE]** Root help lists every command above exactly once and includes
|
|
533
|
+
`migrate`.
|
|
534
|
+
- [ ] **[CORE]** Root help does not list retired top-level `init`, `add`, `list`,
|
|
535
|
+
`remove`, `update`, `wiki`, `mv`, `history`, `graph`, `lessons`, `extract`,
|
|
536
|
+
or `propose`.
|
|
537
|
+
- [ ] **[CORE]** Bare `akm` and `akm help` print the sectioned overview without
|
|
538
|
+
reading or rewriting invalid config.
|
|
539
|
+
- [ ] **[CORE]** `help agents`, `help agents --full`, `hints`, and
|
|
540
|
+
`hints --detail brief` are nonempty and have the expected relative detail.
|
|
541
|
+
- [ ] **[LOCAL]** `help bundle|env|secret|workflow|task|proposal` agrees with the
|
|
542
|
+
corresponding `--help` command tree.
|
|
543
|
+
- [ ] **[LOCAL]** Deep usage includes the full prefix, such as `akm task run`.
|
|
544
|
+
- [ ] **[LOCAL]** Misspelled command/flag suggestions are concise and contain no
|
|
545
|
+
ANSI controls inside JSON.
|
|
546
|
+
|
|
547
|
+
### 6.2 Non-interactive setup
|
|
548
|
+
|
|
549
|
+
- [ ] **[LOCAL]** `setup --yes` in a second sandbox runs without prompts,
|
|
550
|
+
initializes a bundle, and is idempotent on the second run.
|
|
551
|
+
- [ ] **[LOCAL]** JSON and YAML passed through `setup --from` normalize to
|
|
552
|
+
equivalent config.
|
|
553
|
+
- [ ] **[LOCAL]** `--from ... --yes --dir ... --no-init` writes config but does
|
|
554
|
+
not scaffold.
|
|
555
|
+
- [ ] **[LOCAL]** Passing both `--from` and `--config` exits `2` with
|
|
556
|
+
`INVALID_FLAG_VALUE` before write.
|
|
557
|
+
- [ ] **[LOCAL]** Malformed JSON/YAML or invalid shape exits `78` with
|
|
558
|
+
`INVALID_CONFIG_FILE`, preserves prior config bytes, and prints no stack.
|
|
559
|
+
- [ ] **[SERVICE]** `setup --probe` against the fixture `.chat.probe` endpoint
|
|
560
|
+
verifies connectivity before write and records no credential value.
|
|
561
|
+
- [ ] **[LOCAL]** `setup --config '{}' --dir "$HOME" --no-init` rejects the
|
|
562
|
+
unsafe root. `--no-init` must not bypass target safety.
|
|
563
|
+
- [ ] **[LOCAL]** Invalid embedding shape is a config error, not exit `70`.
|
|
564
|
+
|
|
565
|
+
### 6.3 Interactive setup
|
|
566
|
+
|
|
567
|
+
Use a real TTY in the sandbox.
|
|
568
|
+
|
|
569
|
+
- [ ] **[LOCAL]** Setup explains semantic download, small-model setup, agent
|
|
570
|
+
engines, registries, sources, and task review.
|
|
571
|
+
- [ ] **[LOCAL]** Cancel at each boundary; config/bundle stay byte-identical
|
|
572
|
+
unless final save was confirmed.
|
|
573
|
+
- [ ] **[LOCAL]** Redirected stdin without a noninteractive flag exits `2` with
|
|
574
|
+
`NON_INTERACTIVE_REQUIRES_YES` instead of hanging.
|
|
575
|
+
- [ ] **[AI]** Detected CLIs/model servers are suggestions only; selecting or
|
|
576
|
+
skipping persists exactly the reviewed choice.
|
|
577
|
+
|
|
578
|
+
### 6.4 Config management
|
|
579
|
+
|
|
580
|
+
```sh
|
|
581
|
+
akm config list --format json | jq -e '.bundles and .defaultBundle'
|
|
582
|
+
akm config get semanticSearchMode | jq -e '. == "off"'
|
|
583
|
+
akm config set semanticSearchMode auto --silent >"$AKM_SANDBOX/silent.out"
|
|
584
|
+
test ! -s "$AKM_SANDBOX/silent.out"
|
|
585
|
+
akm config unset semanticSearchMode --silent >"$AKM_SANDBOX/silent.out"
|
|
586
|
+
test ! -s "$AKM_SANDBOX/silent.out"
|
|
587
|
+
```
|
|
588
|
+
|
|
589
|
+
- [ ] **[CORE]** `list|get|set|unset` preserve unrelated keys and return JSON
|
|
590
|
+
values; use `jq -r` for string comparisons.
|
|
591
|
+
- [ ] **[CORE]** `--silent` changes config with empty stdout; errors still emit.
|
|
592
|
+
- [ ] **[LOCAL]** Objects deep-merge, arrays replace, and `unset` is deletion.
|
|
593
|
+
- [ ] **[LOCAL]** Generic walker cannot set/unset `configVersion`.
|
|
594
|
+
- [ ] **[LOCAL]** `defaultWriteTarget` rejects unknown/disabled bundles and can
|
|
595
|
+
be set/unset for a configured writable bundle.
|
|
596
|
+
- [ ] **[LOCAL]** Retired config roots and defaults fail without rewrite.
|
|
597
|
+
- [ ] **[LOCAL]** API key references accept `$VAR`/`${VAR}` only; config/list/info
|
|
598
|
+
never materialize values.
|
|
599
|
+
- [ ] **[LOCAL]** Concurrent independent `config set` processes preserve both
|
|
600
|
+
changes or one fails cleanly; JSON stays valid and no temp file remains.
|
|
601
|
+
- [ ] **[LOCAL]** `config path --all` honors output formatting. Bare
|
|
602
|
+
`config path` is a raw path primitive; if format/output are ignored it
|
|
603
|
+
must warn explicitly. Silent ignoring is a failure.
|
|
604
|
+
|
|
605
|
+
### 6.5 Help migration notes
|
|
606
|
+
|
|
607
|
+
- [ ] **[CORE]** `help migrate 0.6.0`, `v0.6.0-rc1`, and `latest` print bundled
|
|
608
|
+
guidance without config startup.
|
|
609
|
+
- [ ] **[CORE]** Unknown version lists available notes gracefully.
|
|
610
|
+
- [ ] **[CORE]** Missing version exits `2` with `MISSING_REQUIRED_ARGUMENT`.
|
|
611
|
+
- [ ] **[LOCAL]** Global flags do not steal the version positional.
|
|
612
|
+
|
|
613
|
+
### 6.6 Completions
|
|
614
|
+
|
|
615
|
+
```sh
|
|
616
|
+
akm completions --format yaml \
|
|
617
|
+
>"$AKM_SANDBOX/akm-completion.bash" \
|
|
618
|
+
2>"$AKM_SANDBOX/completion.stderr"
|
|
619
|
+
bash -n "$AKM_SANDBOX/akm-completion.bash"
|
|
620
|
+
grep -q "'--format' has no effect" "$AKM_SANDBOX/completion.stderr"
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
- [ ] **[CORE]** Generated Bash parses; warning stays on stderr.
|
|
624
|
+
- [ ] **[LOCAL]** Dynamically source it and verify each group returns only real
|
|
625
|
+
subcommands. Registry completion is `add list remove`.
|
|
626
|
+
- [ ] **[LOCAL]** Unsupported shell exits `2` with structured usage error.
|
|
627
|
+
- [ ] **[LOCAL]** `--install` writes only in sandbox completion paths, emits no
|
|
628
|
+
script on stdout, and reports destination on stderr.
|
|
629
|
+
|
|
630
|
+
### 6.7 Upgrade check
|
|
631
|
+
|
|
632
|
+
`upgrade --check` is network-backed, not part of first-run/offline coverage.
|
|
633
|
+
|
|
634
|
+
- [ ] **[LIVE]** With GitHub credentials cleared, check reports current/latest,
|
|
635
|
+
boolean availability, and install method without mutation.
|
|
636
|
+
- [ ] **[LIVE]** Repeat with a disposable token and prove redaction.
|
|
637
|
+
- [ ] **[LIVE]** Source/dist may report `installMethod: "unknown"`; do not count
|
|
638
|
+
that as standalone-upgrade success.
|
|
639
|
+
|
|
640
|
+
---
|
|
641
|
+
|
|
642
|
+
## 7. Index, Search, Curate, and Show
|
|
643
|
+
|
|
644
|
+
### 7.1 Index
|
|
645
|
+
|
|
646
|
+
```sh
|
|
647
|
+
akm index --full --format json >"$AKM_SANDBOX/index-full.json"
|
|
648
|
+
akm index --verbose --format json \
|
|
649
|
+
>"$AKM_SANDBOX/index-verbose.json" \
|
|
650
|
+
2>"$AKM_SANDBOX/index-verbose.stderr"
|
|
651
|
+
jq -e . "$AKM_SANDBOX/index-verbose.json"
|
|
652
|
+
test -s "$AKM_SANDBOX/index-verbose.stderr"
|
|
653
|
+
```
|
|
654
|
+
|
|
655
|
+
- [ ] **[CORE]** Full index reports 18 baseline entries, successful verification,
|
|
656
|
+
scanned directories, and timings.
|
|
657
|
+
- [ ] **[CORE]** Immediate incremental index preserves count and skips unchanged
|
|
658
|
+
directories.
|
|
659
|
+
- [ ] **[CORE]** Verbose progress is stderr-only; stdout remains valid JSON.
|
|
660
|
+
- [ ] **[LOCAL]** Add, modify, rename, and delete one asset; new bytes appear and
|
|
661
|
+
removed bytes never remain discoverable.
|
|
662
|
+
- [ ] **[LOCAL]** Malformed workflow warns/skips only that file.
|
|
663
|
+
- [ ] **[LOCAL]** `--clean --dry-run` reports stale refs without deleting them;
|
|
664
|
+
normal clean removes them. Account for normal indexing discovering a
|
|
665
|
+
deletion before clean runs.
|
|
666
|
+
- [ ] **[LOCAL]** `--dry-run` without `--clean` exits `2`.
|
|
667
|
+
- [ ] **[LOCAL]** Removed `--enrich`/`--re-enrich` exit `2` with guidance.
|
|
668
|
+
- [ ] **[LOCAL]** SIGINT/SIGTERM release locks, preserve prior valid index, and
|
|
669
|
+
allow the next index.
|
|
670
|
+
- [ ] **[LOCAL]** Two simultaneous full indexes serialize or return typed lock
|
|
671
|
+
failure, never DB corruption/permanent lock.
|
|
672
|
+
|
|
673
|
+
### 7.2 Search
|
|
674
|
+
|
|
675
|
+
```sh
|
|
676
|
+
akm search k8s-deploy \
|
|
677
|
+
--detail full --format json |
|
|
678
|
+
jq -e '
|
|
679
|
+
.hits[0].ref == "skills/k8s-deploy" and
|
|
680
|
+
.hits[0].type == "skill" and
|
|
681
|
+
.hits[0].score >= 0 and .hits[0].score <= 1
|
|
682
|
+
'
|
|
683
|
+
|
|
684
|
+
json_count="$(akm search docker | jq '.hits | length')"
|
|
685
|
+
jsonl_count="$(akm search docker --format jsonl | jq -s 'length')"
|
|
686
|
+
test "$json_count" -gt 0
|
|
687
|
+
test "$json_count" -eq "$jsonl_count"
|
|
688
|
+
```
|
|
689
|
+
|
|
690
|
+
- [ ] **[CORE]** Exact/phrase queries rank expected fixtures first.
|
|
691
|
+
- [ ] **[CORE]** `--type skill` returns only skills. Unknown free-form type
|
|
692
|
+
returns zero hits, not usage failure.
|
|
693
|
+
- [ ] **[CORE]** Empty search browses all 18 baseline entries deterministically.
|
|
694
|
+
- [ ] **[CORE]** JSON and JSONL hit counts match.
|
|
695
|
+
- [ ] **[CORE]** No-result search exits `0` with empty hits.
|
|
696
|
+
- [ ] **[LOCAL]** Prefix queries (`knowledge/`, `memories/projectA/`,
|
|
697
|
+
`<bundle>//`) enforce exact boundaries; `projectAlpha/` does not leak.
|
|
698
|
+
- [ ] **[LOCAL]** Named bundle/local/registry/all modes keep local `hits` and
|
|
699
|
+
`registryHits` separate.
|
|
700
|
+
- [ ] **[LOCAL]** Repeated valid filters AND-join. Missing/malformed/unknown or
|
|
701
|
+
conflicting filters fail without broadening scope.
|
|
702
|
+
- [ ] **[LOCAL]** Scoped memory appears only for matching filters; unscoped
|
|
703
|
+
assets disappear once filters are supplied.
|
|
704
|
+
- [ ] **[LOCAL]** Belief current/historical/all partitions state correctly.
|
|
705
|
+
- [ ] **[LOCAL]** Proposed quality and sessions are excluded by default and
|
|
706
|
+
included only through explicit flags.
|
|
707
|
+
- [ ] **[LOCAL]** A successful read advances usage-event state in
|
|
708
|
+
`$AKM_DATA_DIR` only.
|
|
709
|
+
- [ ] **[LOCAL]** Zero/negative/fractional/prefix numeric limits (`2x`), overflow,
|
|
710
|
+
and conflicting repeats are rejected. `parseInt` prefix acceptance fails.
|
|
711
|
+
|
|
712
|
+
### 7.3 Curate
|
|
713
|
+
|
|
714
|
+
```sh
|
|
715
|
+
akm curate "docker homelab" --detail full --format json |
|
|
716
|
+
jq -e '.items[0].ref == "skills/docker-homelab" and (.summary | type == "string")'
|
|
717
|
+
```
|
|
718
|
+
|
|
719
|
+
- [ ] **[CORE]** Returns `{items,query,summary}` and actionable follow-ups.
|
|
720
|
+
- [ ] **[CORE]** Empty query exits `2`, `MISSING_REQUIRED_ARGUMENT`.
|
|
721
|
+
- [ ] **[LOCAL]** Type, limit, source, detail, agent shape, related refs, and
|
|
722
|
+
family collapse are correct.
|
|
723
|
+
- [ ] **[LOCAL]** Invalid limits match search strictness.
|
|
724
|
+
- [ ] **[SERVICE]** Registry-only items have install guidance and no local
|
|
725
|
+
path/editability.
|
|
726
|
+
|
|
727
|
+
### 7.4 Show, fragments, scope, and meta
|
|
728
|
+
|
|
729
|
+
```sh
|
|
730
|
+
akm show skills/k8s-deploy --format json |
|
|
731
|
+
jq -e '
|
|
732
|
+
.type == "skill" and
|
|
733
|
+
.ref == "skills/k8s-deploy" and
|
|
734
|
+
(.path | startswith("/")) and
|
|
735
|
+
(.content | contains("Kubernetes Deployment"))
|
|
736
|
+
'
|
|
737
|
+
|
|
738
|
+
akm show 'knowledge/incident-response-runbook#severity-levels' \
|
|
739
|
+
--format json |
|
|
740
|
+
jq -e '(.content | contains("SEV1")) and ((.content | contains("Escalation Path")) | not)'
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
- [ ] **[CORE]** Default/full/agent/summary retain canonical ref; human/agent
|
|
744
|
+
shapes expose path/editability as documented.
|
|
745
|
+
- [ ] **[CORE]** Valid fragment returns only that section. Invalid fragment exits
|
|
746
|
+
`1`, `ASSET_NOT_FOUND`, and lists slugs.
|
|
747
|
+
- [ ] **[CORE]** Missing asset exits `1` with structured stderr.
|
|
748
|
+
- [ ] **[LOCAL]** `.md` suffix normalization preserves durable identity.
|
|
749
|
+
- [ ] **[LOCAL]** Removed `toc`, `--scope`, and type-colon refs fail with useful
|
|
750
|
+
migration guidance.
|
|
751
|
+
- [ ] **[LOCAL]** `show meta`, `meta:<name>`, and `<bundle>//meta` direct-read
|
|
752
|
+
`.meta`; those docs never appear in search.
|
|
753
|
+
- [ ] **[LOCAL]** Qualified duplicates return selected bundle bytes; unqualified
|
|
754
|
+
follows default then installation precedence.
|
|
755
|
+
- [ ] **[LOCAL]** Scope match resolves; mismatch returns not-found without leak.
|
|
756
|
+
- [ ] **[LOCAL]** Read-only source reports `editable:false` plus edit hint;
|
|
757
|
+
writable local reports `true`.
|
|
758
|
+
- [ ] **[LOCAL]** Show is local-index only and does not remote-fallback.
|
|
759
|
+
|
|
760
|
+
---
|
|
761
|
+
|
|
762
|
+
## 8. Output, Errors, and Exit Codes
|
|
763
|
+
|
|
764
|
+
### 8.1 Exit and channel oracle
|
|
765
|
+
|
|
766
|
+
| Outcome | Exit | Channel |
|
|
767
|
+
| --- | ---: | --- |
|
|
768
|
+
| Success | `0` | Normal payload on stdout |
|
|
769
|
+
| Not found/general failure | `1` | Classified stderr or domain stdout result |
|
|
770
|
+
| Usage/bad input | `2` | Structured JSON stderr |
|
|
771
|
+
| Health warning | `4` | Health stdout result |
|
|
772
|
+
| Unexpected throw | `70` | Structured internal stderr |
|
|
773
|
+
| Config error | `78` | Structured JSON stderr |
|
|
774
|
+
|
|
775
|
+
Domain-result exceptions:
|
|
776
|
+
|
|
777
|
+
| Command | Nonzero behavior |
|
|
778
|
+
| --- | --- |
|
|
779
|
+
| `health` | Warn `4`, fail `1`; result stays on stdout |
|
|
780
|
+
| `lint --fail-on-flagged` | Exit `1`; lint result stays on stdout |
|
|
781
|
+
| `workflow run` | Failure/rejection/timeout emits workflow stdout result |
|
|
782
|
+
| `agent` | Dispatch failure exits `1` with `agent-result` stdout |
|
|
783
|
+
| `task run` | Blocked/failed `1`, command config failure `78`; child code in detail |
|
|
784
|
+
| `migrate` | Child status preserved; progress can precede final formatted plan |
|
|
785
|
+
| `env run` / `secret run` | Raw child streams and exact child status |
|
|
786
|
+
|
|
787
|
+
### 8.2 Six formats
|
|
788
|
+
|
|
789
|
+
```sh
|
|
790
|
+
akm info --format json | jq -e .
|
|
791
|
+
akm info --format jsonl | jq -e .
|
|
792
|
+
akm info --format yaml | bun -e '
|
|
793
|
+
import { parse } from "yaml";
|
|
794
|
+
const value = parse(await Bun.stdin.text());
|
|
795
|
+
if (!value?.version) process.exit(1);
|
|
796
|
+
'
|
|
797
|
+
akm info --format text >"$AKM_SANDBOX/info.txt"
|
|
798
|
+
akm info --format md >"$AKM_SANDBOX/info.md"
|
|
799
|
+
akm info --format html >"$AKM_SANDBOX/info.html"
|
|
800
|
+
grep -qi '<html' "$AKM_SANDBOX/info.html"
|
|
801
|
+
```
|
|
802
|
+
|
|
803
|
+
- [ ] **[CORE]** JSON/JSONL/YAML parse; text is plain; Markdown is not JSON;
|
|
804
|
+
HTML is self-contained.
|
|
805
|
+
- [ ] **[LOCAL]** Repeat representative search/show/config/bundle/log/health/task/
|
|
806
|
+
proposal/workflow/setup/migrate commands in supported formats.
|
|
807
|
+
- [ ] **[LOCAL]** Global flags work before/after command and with space/equals.
|
|
808
|
+
- [ ] **[LOCAL]** Persisted output defaults apply only when CLI flags are absent.
|
|
809
|
+
|
|
810
|
+
### 8.3 Shape, detail, and output destination
|
|
811
|
+
|
|
812
|
+
- [ ] **[CORE]** Brief/normal/full increase detail without identity drift.
|
|
813
|
+
- [ ] **[CORE]** Summary succeeds only on show; all other commands reject it
|
|
814
|
+
before side effects with exit `2`, `INVALID_SHAPE_VALUE`.
|
|
815
|
+
- [ ] **[LOCAL]** Agent shape keeps action fields and strips non-action metadata.
|
|
816
|
+
- [ ] **[LOCAL]** `show --format md --output <file>` writes file, empty stdout.
|
|
817
|
+
- [ ] **[LOCAL]** Output replacement is atomic; unwritable/directory target does
|
|
818
|
+
not truncate existing content.
|
|
819
|
+
- [ ] **[LOCAL]** JSONL stays on stdout and does not create `--output` file.
|
|
820
|
+
- [ ] **[LOCAL]** Health report HTML data matches JSON report data.
|
|
821
|
+
|
|
822
|
+
### 8.4 Format-exempt commands
|
|
823
|
+
|
|
824
|
+
`completions`, `help`, `help migrate`, `env path`, `env run`, and `secret run`
|
|
825
|
+
do not emit normal result envelopes.
|
|
826
|
+
|
|
827
|
+
- [ ] **[CORE]** Passing format emits one stderr warning and preserves raw
|
|
828
|
+
payload/child streams.
|
|
829
|
+
- [ ] **[LOCAL]** Child help/format flags after `--` reach child, not AKM parser.
|
|
830
|
+
- [ ] **[LOCAL]** Interactive setup is terminal UI; scripted setup is formatted.
|
|
831
|
+
|
|
832
|
+
### 8.5 Error gauntlet
|
|
833
|
+
|
|
834
|
+
```sh
|
|
835
|
+
expect_error 2 MISSING_REQUIRED_ARGUMENT curate ""
|
|
836
|
+
expect_error 1 ASSET_NOT_FOUND show skills/does-not-exist
|
|
837
|
+
expect_error 2 UNKNOWN_FLAG info --totally-bogus
|
|
838
|
+
expect_error 2 INVALID_FORMAT_VALUE info --format xml
|
|
839
|
+
expect_error 2 INVALID_DETAIL_VALUE info --detail maximum
|
|
840
|
+
expect_error 2 INVALID_SHAPE_VALUE search docker --shape summary
|
|
841
|
+
expect_error 2 MISSING_REQUIRED_ARGUMENT help migrate
|
|
842
|
+
expect_error 2 INVALID_FLAG_VALUE completions --shell zsh
|
|
843
|
+
```
|
|
844
|
+
|
|
845
|
+
- [ ] **[CORE]** Classified throw leaves stdout empty and emits one stderr JSON
|
|
846
|
+
object with `ok:false`, `error`, `code`, optional hint.
|
|
847
|
+
- [ ] **[CORE]** No ordinary failure prints stack, bare `Error:`, ANSI JSON, or
|
|
848
|
+
output-shape registration failure.
|
|
849
|
+
- [ ] **[LOCAL]** Errors stay JSON under text/yaml/md/html/output flags; no error
|
|
850
|
+
output file is created.
|
|
851
|
+
- [ ] **[LOCAL]** Test-harness unclassified throw exits `70`; `AKM_DEBUG=1` adds
|
|
852
|
+
stack only on stderr.
|
|
853
|
+
- [ ] **[LOCAL]** Unknown commands/bare groups exit `2` with concise current hint.
|
|
854
|
+
|
|
855
|
+
---
|
|
856
|
+
|
|
857
|
+
## 9. Bundles, Adapters, Sources, and Registries
|
|
858
|
+
|
|
859
|
+
### 9.1 Bundle create
|
|
860
|
+
|
|
861
|
+
Use a second sandbox or disposable secondary paths.
|
|
862
|
+
|
|
863
|
+
- [ ] **[LOCAL]** First create scaffolds canonical dirs/current config/default.
|
|
864
|
+
- [ ] **[LOCAL]** `--dir <secondary>` backfills without changing existing
|
|
865
|
+
default; `--set-default` changes it explicitly.
|
|
866
|
+
- [ ] **[LOCAL]** Repeat is idempotent and preserves files.
|
|
867
|
+
- [ ] **[LOCAL]** `$HOME`, root, config, cache, data, and state roots are rejected
|
|
868
|
+
even if no scaffolding work would occur.
|
|
869
|
+
- [ ] **[LOCAL]** Env/secrets paths have owner-only POSIX permissions.
|
|
870
|
+
- [ ] **[LOCAL]** Temp authorization uses `AKM_FORCE_INIT_TMP_STASH`, not setup's
|
|
871
|
+
separate variable.
|
|
872
|
+
|
|
873
|
+
### 9.2 Identity, precedence, and enabled state
|
|
874
|
+
|
|
875
|
+
```sh
|
|
876
|
+
mkdir -p "$AKM_SANDBOX/bundle-a" "$AKM_SANDBOX/bundle-b"
|
|
877
|
+
akm bundle add "$AKM_SANDBOX/bundle-a" --name bundle-a
|
|
878
|
+
akm bundle add "$AKM_SANDBOX/bundle-b" --name bundle-b
|
|
879
|
+
akm remember "body A" --name shared --bundle bundle-a
|
|
880
|
+
akm remember "body B" --name shared --bundle bundle-b
|
|
881
|
+
akm show bundle-a//memories/shared | jq -e '.content | contains("body A")'
|
|
882
|
+
akm show bundle-b//memories/shared | jq -e '.content | contains("body B")'
|
|
883
|
+
```
|
|
884
|
+
|
|
885
|
+
- [ ] **[LOCAL]** Qualified identity remains stable through index/search/show.
|
|
886
|
+
- [ ] **[LOCAL]** Unqualified resolution follows default then installation
|
|
887
|
+
precedence, never random duplicate.
|
|
888
|
+
- [ ] **[LOCAL]** Names obey slug rules; `team/sync-qa` is rejected.
|
|
889
|
+
- [ ] **[LOCAL]** Kind filter accepts exact supported kinds; `local`/`managed`
|
|
890
|
+
groupings exit `2`.
|
|
891
|
+
- [ ] **[LOCAL]** `bundle show` reports one descriptor/components/effective
|
|
892
|
+
writable state.
|
|
893
|
+
- [ ] **[LOCAL]** `enabled:false` removes source from index/search/refresh/write
|
|
894
|
+
selection. Singular update/write must not bypass disabled state.
|
|
895
|
+
|
|
896
|
+
### 9.3 Built-in adapter matrix
|
|
897
|
+
|
|
898
|
+
| Adapter | Fixture |
|
|
899
|
+
| --- | --- |
|
|
900
|
+
| `website-snapshot` | `tests/fixtures/bundles/website-snapshot` |
|
|
901
|
+
| `agent-skills` | `tests/fixtures/bundles/agent-skills` |
|
|
902
|
+
| `claude` | `tests/fixtures/bundles/claude` |
|
|
903
|
+
| `opencode` | `tests/fixtures/bundles/opencode` |
|
|
904
|
+
| `dotenv` | `tests/fixtures/bundles/dotenv` |
|
|
905
|
+
| `akm-workflow` | `tests/fixtures/bundles/akm-workflow` |
|
|
906
|
+
| `akm-task` | `tests/fixtures/bundles/akm-task` |
|
|
907
|
+
| `llm-wiki` | `tests/fixtures/bundles/llm-wiki` |
|
|
908
|
+
| `akm` | `tests/fixtures/stashes/ranking-baseline` |
|
|
909
|
+
| `okf` | `tests/fixtures/bundles/okf-sample-v2` |
|
|
910
|
+
| `generic-files` | `tests/fixtures/bundles/generic-files` |
|
|
911
|
+
|
|
912
|
+
For each adapter:
|
|
913
|
+
|
|
914
|
+
- [ ] **[LOCAL]** Add copied fixture under unique name; run bundle show, full
|
|
915
|
+
index, search, representative show, and lint.
|
|
916
|
+
- [ ] **[LOCAL]** Expected component wins precedence without double indexing.
|
|
917
|
+
- [ ] **[LOCAL]** Placement/ref/type/rendering match format-family goldens.
|
|
918
|
+
- [ ] **[LOCAL]** Intentional invalid files produce only named lint findings.
|
|
919
|
+
- [ ] **[LOCAL]** Read-only adapter rejects write before mutation.
|
|
920
|
+
|
|
921
|
+
### 9.4 LLM Wiki adapter
|
|
922
|
+
|
|
923
|
+
```sh
|
|
924
|
+
cp -R "$REPO/tests/fixtures/bundles/llm-wiki" "$AKM_SANDBOX/sample-wiki"
|
|
925
|
+
akm bundle add "$AKM_SANDBOX/sample-wiki" --name sample-wiki
|
|
926
|
+
akm bundle show sample-wiki --format json
|
|
927
|
+
akm index --full
|
|
928
|
+
akm show sample-wiki//pages/http-caching
|
|
929
|
+
akm show sample-wiki//raw/2026-07-http-rfc
|
|
930
|
+
akm lint --dir "$AKM_SANDBOX/sample-wiki"
|
|
931
|
+
```
|
|
932
|
+
|
|
933
|
+
- [ ] **[LOCAL]** Adapter is `llm-wiki`; pages use `pageKind`, raw is
|
|
934
|
+
`wiki-source`.
|
|
935
|
+
- [ ] **[LOCAL]** `schema.md`, `index.md`, `log.md` are not indexed concepts.
|
|
936
|
+
- [ ] **[LOCAL]** Intentional broken xref is reported.
|
|
937
|
+
- [ ] **[LOCAL]** Removing schema or pages prevents recognition.
|
|
938
|
+
- [ ] **[LOCAL]** No network/LLM request occurs.
|
|
939
|
+
- [ ] **[LOCAL]** There is no `akm wiki`; agents write normal files.
|
|
940
|
+
|
|
941
|
+
### 9.5 Filesystem source
|
|
942
|
+
|
|
943
|
+
```sh
|
|
944
|
+
mkdir -p "$AKM_SANDBOX/fs-source/knowledge"
|
|
945
|
+
printf '%s\n' '# Filesystem Marker' '' 'qa-filesystem-source-marker' \
|
|
946
|
+
>"$AKM_SANDBOX/fs-source/knowledge/marker.md"
|
|
947
|
+
akm bundle add "$AKM_SANDBOX/fs-source" --name fs-source
|
|
948
|
+
akm bundle list --kind filesystem
|
|
949
|
+
akm search qa-filesystem-source-marker --from fs-source
|
|
950
|
+
```
|
|
951
|
+
|
|
952
|
+
- [ ] **[LOCAL]** Add indexes in place with kind `filesystem`, writable true.
|
|
953
|
+
- [ ] **[LOCAL]** Single update and update-all reconcile current files into the
|
|
954
|
+
index without provider hydration.
|
|
955
|
+
- [ ] **[LOCAL]** Non-TTY remove requires `--yes`, removes ownership/index, and
|
|
956
|
+
leaves source files.
|
|
957
|
+
- [ ] **[LOCAL]** Real path plus symlink alias does not create ambiguous owner or
|
|
958
|
+
duplicate durable refs.
|
|
959
|
+
|
|
960
|
+
### 9.6 Git source
|
|
961
|
+
|
|
962
|
+
Use a small disposable HTTPS/SSH repository. `file://` is rejected by design.
|
|
963
|
+
|
|
964
|
+
- [ ] **[LIVE]** Inferred add clones/locks/configures/indexes read-only.
|
|
965
|
+
- [ ] **[LIVE]** Branch/tag/ref pin resolves exact revision.
|
|
966
|
+
- [ ] **[LIVE]** Change unique upstream marker, update by legal name, and prove
|
|
967
|
+
new bytes/revision. Exit zero alone is insufficient.
|
|
968
|
+
- [ ] **[LIVE]** Force rematerializes complete content.
|
|
969
|
+
- [ ] **[LIVE]** Declarative `--provider git` is config-only; first update
|
|
970
|
+
materializes and runs security audit.
|
|
971
|
+
- [ ] **[LIVE]** Writable source pushes only under explicit policy; no-push never
|
|
972
|
+
contacts remote.
|
|
973
|
+
- [ ] **[LIVE]** Plain `git+http` cannot bypass insecure transport gate.
|
|
974
|
+
- [ ] **[LIVE]** Remove cleans AKM-owned config/lock/index/cache only.
|
|
975
|
+
|
|
976
|
+
### 9.7 npm source
|
|
977
|
+
|
|
978
|
+
Use a disposable package/version or controlled registry.
|
|
979
|
+
|
|
980
|
+
- [ ] **[LIVE]** Inferred add materializes/locks/indexes read-only.
|
|
981
|
+
- [ ] **[LIVE]** Scoped/unscoped refs, exact versions, dist-tags, malformed specs,
|
|
982
|
+
missing versions, auth, checksum/tar safety, and oversized archives behave
|
|
983
|
+
structurally.
|
|
984
|
+
- [ ] **[LIVE]** Publish second unique marker/version; update proves new bytes.
|
|
985
|
+
- [ ] **[LIVE]** Declarative provider is config-only; update makes managed.
|
|
986
|
+
- [ ] **[LIVE]** `writable:true` is rejected.
|
|
987
|
+
- [ ] **[LIVE]** Remove cleans all AKM-owned extraction roots only.
|
|
988
|
+
|
|
989
|
+
### 9.8 Website source
|
|
990
|
+
|
|
991
|
+
Production must reject loopback/private hosts:
|
|
992
|
+
|
|
993
|
+
```sh
|
|
994
|
+
expect_error 78 INVALID_CONFIG_FILE \
|
|
995
|
+
bundle add "$AKM_QA_WEBSITE_URL" --name private-site --allow-insecure-transport
|
|
996
|
+
```
|
|
997
|
+
|
|
998
|
+
Protocol-only fixture:
|
|
999
|
+
|
|
1000
|
+
```sh
|
|
1001
|
+
NODE_ENV=test akm bundle add "$AKM_QA_WEBSITE_URL" \
|
|
1002
|
+
--name qa-site --allow-insecure-transport --max-pages 2 --max-depth 1
|
|
1003
|
+
NODE_ENV=test akm search qa-site --from qa-site
|
|
1004
|
+
```
|
|
1005
|
+
|
|
1006
|
+
- [ ] **[SERVICE]** Production blocks private start/redirect hosts;
|
|
1007
|
+
`--allow-insecure-transport` accepts transport risk only, not SSRF.
|
|
1008
|
+
- [ ] **[SERVICE]** Test-seam crawl respects page/depth/robots; request log proves
|
|
1009
|
+
private path not fetched.
|
|
1010
|
+
- [ ] **[SERVICE]** Change site version to `v2`, named-update, and prove v2 bytes.
|
|
1011
|
+
Stale-fallback success with v1 fails this gate.
|
|
1012
|
+
- [ ] **[SERVICE]** Failed first crawl leaves no config/cache/index/lock/success
|
|
1013
|
+
event residue.
|
|
1014
|
+
- [ ] **[SERVICE]** Interrupted publication preserves previous complete snapshot.
|
|
1015
|
+
- [ ] **[SERVICE]** Update-all skips website; named update refreshes.
|
|
1016
|
+
- [ ] **[LIVE]** Repeat against controlled public HTTPS in production mode.
|
|
1017
|
+
- [ ] **[LIVE]** Specialized YouTube/Bluesky/X/feed fetchers use controlled data
|
|
1018
|
+
or disposable credentials and retain safe fallback.
|
|
1019
|
+
|
|
1020
|
+
### 9.9 Registry
|
|
1021
|
+
|
|
1022
|
+
```sh
|
|
1023
|
+
expect_error 2 INVALID_FLAG_VALUE registry add "$AKM_QA_REGISTRY_URL" --name qa-registry
|
|
1024
|
+
akm registry add "$AKM_QA_REGISTRY_URL" --name qa-registry --allow-insecure-transport
|
|
1025
|
+
akm registry list | jq -e '.registries[] | select(.name == "qa-registry")'
|
|
1026
|
+
akm search kubernetes --from registry --detail full
|
|
1027
|
+
akm search kubernetes --from registry --assets --detail full
|
|
1028
|
+
akm registry remove qa-registry --yes
|
|
1029
|
+
```
|
|
1030
|
+
|
|
1031
|
+
- [ ] **[SERVICE]** HTTP requires `--allow-insecure-transport` and warns.
|
|
1032
|
+
- [ ] **[SERVICE]** Duplicate URL is idempotent `added:false`; unknown remove is
|
|
1033
|
+
exit `1`, `SOURCE_NOT_FOUND`.
|
|
1034
|
+
- [ ] **[SERVICE]** Registry hits remain `registryHits`, not local `hits`.
|
|
1035
|
+
- [ ] **[SERVICE]** Full hit exposes actionable `installRef`.
|
|
1036
|
+
- [ ] **[SERVICE]** `--assets` makes asset-level fixture result observable;
|
|
1037
|
+
discarding provider asset hits is failure.
|
|
1038
|
+
- [ ] **[SERVICE]** Scores remain provider-local and may exceed one.
|
|
1039
|
+
- [ ] **[SERVICE]** Malformed/wrong schema/HTTP/timeout/stale/oversized response
|
|
1040
|
+
does not poison other registries.
|
|
1041
|
+
- [ ] **[SERVICE]** Options require JSON object; malformed/non-object fails before
|
|
1042
|
+
config mutation.
|
|
1043
|
+
- [ ] **[SERVICE]** Install uses `bundle add <installRef>`; no registry install
|
|
1044
|
+
subcommand exists.
|
|
1045
|
+
|
|
1046
|
+
### 9.10 Source failure atomicity and dangerous env
|
|
1047
|
+
|
|
1048
|
+
```sh
|
|
1049
|
+
before_config="$(sha256sum "$AKM_CONFIG_DIR/config.json")"
|
|
1050
|
+
set +e
|
|
1051
|
+
akm bundle add "$REPO/tests/fixtures/manual-qa/dangerous-bundle" \
|
|
1052
|
+
--name qa-dangerous </dev/null
|
|
1053
|
+
status=$?
|
|
1054
|
+
set -e
|
|
1055
|
+
test "$status" -eq 1
|
|
1056
|
+
test "$(sha256sum "$AKM_CONFIG_DIR/config.json")" = "$before_config"
|
|
1057
|
+
! akm bundle list --format json | jq -e '.sources[] | select(.name == "qa-dangerous")'
|
|
1058
|
+
```
|
|
1059
|
+
|
|
1060
|
+
- [ ] **[LOCAL]** Inferred noninteractive add blocks raw dangerous env, rolls
|
|
1061
|
+
back config/lock/cache/index, no success event/envelope.
|
|
1062
|
+
- [ ] **[LOCAL]** Allow-insecure permits reviewed fixture with warning.
|
|
1063
|
+
- [ ] **[LOCAL]** Non-`.env` file under env is ignored.
|
|
1064
|
+
- [ ] **[LOCAL]** Publisher lint suppression does not silently bypass untrusted
|
|
1065
|
+
install safety; test suppression fixture separately.
|
|
1066
|
+
- [ ] **[LOCAL]** Declarative provider and later update receive equivalent audit.
|
|
1067
|
+
- [ ] **[LOCAL]** New dangerous key on update preserves previous complete source.
|
|
1068
|
+
- [ ] **[LOCAL]** A held WAL index reader observes only the prior generation
|
|
1069
|
+
through a failed update; index/state main, WAL, and SHM inodes are not
|
|
1070
|
+
replaced during compensation.
|
|
1071
|
+
- [ ] **[LOCAL]** Writable Git rejects physical-root/component symlink escapes,
|
|
1072
|
+
a changed HEAD or dirty state at publication, and dangerous tracked,
|
|
1073
|
+
untracked, ignored, filter-produced, or submodule materialization.
|
|
1074
|
+
- [ ] **[LOCAL]** Lock publication and rollback fence both exact raw bytes and
|
|
1075
|
+
file mode; a concurrent chmod blocks publication without being reverted.
|
|
1076
|
+
- [ ] **[LOCAL]** A simulated durable index/state split is reported as
|
|
1077
|
+
`index-state-generation`; after writers stop, `akm index --full` clears
|
|
1078
|
+
the advisory and restores coherent searchable usage links.
|
|
1079
|
+
- [ ] **[LOCAL]** Corrupt/unreadable lock fails structurally; no managed/plain
|
|
1080
|
+
reinterpretation or split state.
|
|
1081
|
+
- [ ] **[LOCAL]** Failed add/update/remove compare config, lock, cache, index,
|
|
1082
|
+
source bytes, and event cursor, not only exit.
|
|
1083
|
+
|
|
1084
|
+
---
|
|
1085
|
+
|
|
1086
|
+
## 10. Writes, Import, Clone, and Sync
|
|
1087
|
+
|
|
1088
|
+
### 10.1 Remember
|
|
1089
|
+
|
|
1090
|
+
```sh
|
|
1091
|
+
akm remember "test memory body" --name test-memory
|
|
1092
|
+
akm show memories/test-memory | jq -e '.content | contains("test memory body")'
|
|
1093
|
+
expect_error 1 RESOURCE_ALREADY_EXISTS remember "duplicate" --name test-memory
|
|
1094
|
+
akm remember "replacement" --name test-memory --force
|
|
1095
|
+
printf 'stdin body\n' | akm remember --name from-stdin
|
|
1096
|
+
```
|
|
1097
|
+
|
|
1098
|
+
- [ ] **[CORE]** Plain/stdin/force paths write one valid indexed memory and
|
|
1099
|
+
preserve body bytes.
|
|
1100
|
+
- [ ] **[CORE]** Duplicate without force fails before mutation.
|
|
1101
|
+
- [ ] **[LOCAL]** Description/repeated tags/expiry/source/auto/nested path/xrefs
|
|
1102
|
+
serialize once without nested frontmatter/duplicates.
|
|
1103
|
+
- [ ] **[LOCAL]** Scope-only memory is valid without tags and uses canonical
|
|
1104
|
+
scope fields.
|
|
1105
|
+
- [ ] **[LOCAL]** Tag-bearing metadata with no resulting tag fails before write.
|
|
1106
|
+
- [ ] **[LOCAL]** Invalid absolute/traversal name/path, self/unresolved xref,
|
|
1107
|
+
unknown bundle, retired target/stash flags fail before all side effects.
|
|
1108
|
+
- [ ] **[AI]** Enrich success/failure uses fake LLM and never silently writes
|
|
1109
|
+
missing required metadata.
|
|
1110
|
+
|
|
1111
|
+
### 10.2 Target precedence
|
|
1112
|
+
|
|
1113
|
+
- [ ] **[LOCAL]** Explicit bundle wins over default write target, then default
|
|
1114
|
+
bundle; no fallback to first writable source.
|
|
1115
|
+
- [ ] **[LOCAL]** Read-only/disabled/missing/unsupported targets reject before
|
|
1116
|
+
write.
|
|
1117
|
+
- [ ] **[LOCAL]** Writable git target produces one complete boundary commit;
|
|
1118
|
+
push requires writable + remote policy.
|
|
1119
|
+
- [ ] **[LOCAL]** Qualified source cannot redirect to conflicting target.
|
|
1120
|
+
|
|
1121
|
+
### 10.3 Import
|
|
1122
|
+
|
|
1123
|
+
```sh
|
|
1124
|
+
cat >"$AKM_SANDBOX/incoming.md" <<'EOF'
|
|
1125
|
+
---
|
|
1126
|
+
description: Imported manual QA document
|
|
1127
|
+
custom:
|
|
1128
|
+
nested: preserved
|
|
1129
|
+
---
|
|
1130
|
+
# Imported Guide
|
|
1131
|
+
|
|
1132
|
+
qa-import-marker
|
|
1133
|
+
EOF
|
|
1134
|
+
|
|
1135
|
+
akm import "$AKM_SANDBOX/incoming.md" --name imported-guide
|
|
1136
|
+
akm import - --name imported-stdin <"$AKM_SANDBOX/incoming.md"
|
|
1137
|
+
```
|
|
1138
|
+
|
|
1139
|
+
- [ ] **[CORE]** File/stdin import preserve custom frontmatter/body and produce
|
|
1140
|
+
deterministic refs.
|
|
1141
|
+
- [ ] **[LOCAL]** Path/target/force/xref/supersession use same preflight/write
|
|
1142
|
+
rules as remember.
|
|
1143
|
+
- [ ] **[LOCAL]** Malformed frontmatter fails losslessly; missing input/invalid
|
|
1144
|
+
name leaves no partial file.
|
|
1145
|
+
- [ ] **[LOCAL]** Loopback URL production import exits `2` under SSRF policy;
|
|
1146
|
+
no allow-insecure bypass exists.
|
|
1147
|
+
- [ ] **[LIVE]** Controlled public HTTPS import follows bounded redirects,
|
|
1148
|
+
converts content, derives URL-path name, blocks private redirect.
|
|
1149
|
+
|
|
1150
|
+
### 10.4 Supersession
|
|
1151
|
+
|
|
1152
|
+
- [ ] **[LOCAL]** Correction xrefs old asset; old gains superseded state/by ref
|
|
1153
|
+
while preserving unrelated metadata/body.
|
|
1154
|
+
- [ ] **[LOCAL]** Belief-current hides old; all can return both.
|
|
1155
|
+
- [ ] **[LOCAL]** Unresolved/self supersession changes neither file.
|
|
1156
|
+
- [ ] **[LOCAL]** Cross-bundle non-applicable demotion is reported explicitly.
|
|
1157
|
+
- [ ] **[DESTRUCTIVE]** Failure between correction and demotion has pinned
|
|
1158
|
+
all-or-nothing behavior; undocumented half-correction fails.
|
|
1159
|
+
|
|
1160
|
+
### 10.5 Clone
|
|
1161
|
+
|
|
1162
|
+
- [ ] **[CORE]** Clone a skill and single file to unmanaged destination; skill
|
|
1163
|
+
references copy recursively, source unchanged.
|
|
1164
|
+
- [ ] **[LOCAL]** Managed clone follows explicit/default/working target and
|
|
1165
|
+
indexes immediately.
|
|
1166
|
+
- [ ] **[LOCAL]** Destination works without write target and conflicts with
|
|
1167
|
+
bundle flag.
|
|
1168
|
+
- [ ] **[LOCAL]** Existing destination requires force; invalid rename/traversal/
|
|
1169
|
+
missing source leaves no partial copy.
|
|
1170
|
+
- [ ] **[LIVE]** Clone from uninstalled controlled package caches but does not
|
|
1171
|
+
register source.
|
|
1172
|
+
|
|
1173
|
+
### 10.6 Sync
|
|
1174
|
+
|
|
1175
|
+
```sh
|
|
1176
|
+
git -C "$AKM_BUNDLE_DIR" init
|
|
1177
|
+
git -C "$AKM_BUNDLE_DIR" config user.name 'AKM Manual QA'
|
|
1178
|
+
git -C "$AKM_BUNDLE_DIR" config user.email 'manual-qa@example.invalid'
|
|
1179
|
+
git -C "$AKM_BUNDLE_DIR" add .
|
|
1180
|
+
git -C "$AKM_BUNDLE_DIR" commit -m 'Manual QA baseline'
|
|
1181
|
+
|
|
1182
|
+
akm remember "sync marker" --name sync-marker
|
|
1183
|
+
akm sync --no-push -m "Manual QA sync"
|
|
1184
|
+
git -C "$AKM_BUNDLE_DIR" show --stat --oneline -1
|
|
1185
|
+
```
|
|
1186
|
+
|
|
1187
|
+
- [ ] **[LOCAL]** Non-git bundle returns structured skipped/no-op.
|
|
1188
|
+
- [ ] **[LOCAL]** Git commits sandbox changes/message and never pushes with
|
|
1189
|
+
no-push.
|
|
1190
|
+
- [ ] **[LOCAL]** Named source uses legal slug such as `team-sync-qa`, not slash.
|
|
1191
|
+
- [ ] **[LOCAL]** Source literally named `json` remains selectable alongside
|
|
1192
|
+
format flag.
|
|
1193
|
+
- [ ] **[LOCAL]** Whole-bundle staging policy is represented honestly; no claim
|
|
1194
|
+
that unrelated dirty paths were excluded if they were committed.
|
|
1195
|
+
- [ ] **[LIVE]** Disposable writable remote pushes; read-only/no-remote commits
|
|
1196
|
+
locally only; failed push retains complete local commit.
|
|
1197
|
+
|
|
1198
|
+
### 10.7 Write concurrency, atomicity, and paths
|
|
1199
|
+
|
|
1200
|
+
- [ ] **[LOCAL]** Two same-name remember creates: exactly one succeeds, one
|
|
1201
|
+
already-exists; final is one complete payload.
|
|
1202
|
+
- [ ] **[LOCAL]** Repeat for import and content-bearing env create.
|
|
1203
|
+
- [ ] **[LOCAL]** Concurrent different-asset writes both survive/index.
|
|
1204
|
+
- [ ] **[LOCAL]** Read-only/unwritable/disk-full/path-is-directory failures leave
|
|
1205
|
+
old bytes/index intact.
|
|
1206
|
+
- [ ] **[LOCAL]** Internal symlink resolves consistently; escaping symlink is
|
|
1207
|
+
rejected and outside sentinel unchanged.
|
|
1208
|
+
- [ ] **[DESTRUCTIVE]** Kill replacement at publication boundaries; recovery is
|
|
1209
|
+
old or new complete bytes, no temp/stale lock/truncation.
|
|
1210
|
+
|
|
1211
|
+
---
|
|
1212
|
+
|
|
1213
|
+
## 11. Env and Secret
|
|
1214
|
+
|
|
1215
|
+
Use conspicuous dummy values only. AKM keeps env and secret values out of its
|
|
1216
|
+
structured results, but a child launched by `env run` or `secret run` owns its
|
|
1217
|
+
raw stdout/stderr and can print injected values.
|
|
1218
|
+
|
|
1219
|
+
### 11.1 Create, list, path, and indexing
|
|
1220
|
+
|
|
1221
|
+
```sh
|
|
1222
|
+
export QA_SECRET_VALUE='manual-qa-secret-not-a-credential'
|
|
1223
|
+
|
|
1224
|
+
printf '%s\n' \
|
|
1225
|
+
'QA_PUBLIC=manual-qa-public' \
|
|
1226
|
+
'QA_SECOND=manual-qa-second' \
|
|
1227
|
+
'QA_TOKEN=Bearer ${secret:qa-token}' \
|
|
1228
|
+
>"$AKM_SANDBOX/qa-runtime.env"
|
|
1229
|
+
|
|
1230
|
+
akm env create qa-runtime \
|
|
1231
|
+
--from-file "$AKM_SANDBOX/qa-runtime.env" --format json \
|
|
1232
|
+
>"$AKM_SANDBOX/env-create.json"
|
|
1233
|
+
printf '%s\n' "$QA_SECRET_VALUE" |
|
|
1234
|
+
akm secret set secrets/qa-token --format json \
|
|
1235
|
+
>"$AKM_SANDBOX/secret-set.json"
|
|
1236
|
+
|
|
1237
|
+
akm env list --format json >"$AKM_SANDBOX/env-list.json"
|
|
1238
|
+
akm secret list --format json >"$AKM_SANDBOX/secret-list.json"
|
|
1239
|
+
akm index
|
|
1240
|
+
akm show env/qa-runtime --format json \
|
|
1241
|
+
>"$AKM_SANDBOX/env-show.json"
|
|
1242
|
+
akm show secrets/qa-token --format json \
|
|
1243
|
+
>"$AKM_SANDBOX/secret-show.json"
|
|
1244
|
+
|
|
1245
|
+
jq -e 'any(.envs[]; .ref == "env/qa-runtime" and (.keys | index("QA_PUBLIC")))' \
|
|
1246
|
+
"$AKM_SANDBOX/env-list.json"
|
|
1247
|
+
jq -e 'any(.secrets[]; .ref == "secrets/qa-token")' \
|
|
1248
|
+
"$AKM_SANDBOX/secret-list.json"
|
|
1249
|
+
|
|
1250
|
+
for capture in \
|
|
1251
|
+
"$AKM_SANDBOX/env-create.json" \
|
|
1252
|
+
"$AKM_SANDBOX/secret-set.json" \
|
|
1253
|
+
"$AKM_SANDBOX/env-list.json" \
|
|
1254
|
+
"$AKM_SANDBOX/secret-list.json" \
|
|
1255
|
+
"$AKM_SANDBOX/env-show.json" \
|
|
1256
|
+
"$AKM_SANDBOX/secret-show.json"; do
|
|
1257
|
+
! grep -F "$QA_SECRET_VALUE" "$capture"
|
|
1258
|
+
! grep -F 'manual-qa-public' "$capture"
|
|
1259
|
+
done
|
|
1260
|
+
```
|
|
1261
|
+
|
|
1262
|
+
- [ ] **[CORE]** Env list exposes refs and key names only; secret list exposes
|
|
1263
|
+
refs only. Values, comments, and absolute paths are absent.
|
|
1264
|
+
- [ ] **[CORE]** Env is searchable by key name, never value/comment. Secret is
|
|
1265
|
+
searchable by filename only, never bytes.
|
|
1266
|
+
- [ ] **[CORE]** `env path env/qa-runtime` emits one absolute path on stdout and
|
|
1267
|
+
a do-not-source warning on stderr; `--quiet` suppresses only the warning.
|
|
1268
|
+
- [ ] **[CORE]** Env/secret directories are `0700` and files are `0600` on
|
|
1269
|
+
POSIX. Record the ACL equivalent on Windows.
|
|
1270
|
+
- [ ] **[CORE]** Empty env create is idempotent. Content-bearing create refuses
|
|
1271
|
+
an existing destination with exit `2`, preserving old bytes.
|
|
1272
|
+
- [ ] **[LOCAL]** `--from-file` and `--from-stdin` conflict. Env stdin over 1 MiB,
|
|
1273
|
+
secret stdin/file over 5 MiB, missing input, invalid path, and traversal
|
|
1274
|
+
fail before publication.
|
|
1275
|
+
- [ ] **[LOCAL REGRESSION]** Secret `--from-env` obeys the same 5 MiB cap as
|
|
1276
|
+
stdin/file. Current behavior must not be accepted if it bypasses the cap.
|
|
1277
|
+
- [ ] **[LOCAL]** `--sensitive` and sibling `.sensitive` markers remove an asset
|
|
1278
|
+
from list/index/search/show while preserving direct run/path use.
|
|
1279
|
+
- [ ] **[LOCAL]** Mutations are not automatically reindexed; list reads disk
|
|
1280
|
+
immediately, while search/show change only after `akm index`.
|
|
1281
|
+
|
|
1282
|
+
### 11.2 Safe value use and raw child channels
|
|
1283
|
+
|
|
1284
|
+
```sh
|
|
1285
|
+
export QA_INHERITED='manual-qa-inherited'
|
|
1286
|
+
|
|
1287
|
+
akm env run env/qa-runtime \
|
|
1288
|
+
--only QA_PUBLIC,QA_TOKEN --clean --inherit QA_INHERITED \
|
|
1289
|
+
-- bun -e '
|
|
1290
|
+
if (process.env.QA_PUBLIC !== "manual-qa-public") process.exit(11);
|
|
1291
|
+
if (process.env.QA_TOKEN !== "Bearer manual-qa-secret-not-a-credential") process.exit(12);
|
|
1292
|
+
if (process.env.QA_INHERITED !== "manual-qa-inherited") process.exit(13);
|
|
1293
|
+
if (process.env.QA_SECOND !== undefined) process.exit(14);
|
|
1294
|
+
'
|
|
1295
|
+
|
|
1296
|
+
akm secret run secrets/qa-token QA_INJECTED --clean \
|
|
1297
|
+
-- bun -e '
|
|
1298
|
+
if (process.env.QA_INJECTED !== "manual-qa-secret-not-a-credential") process.exit(15);
|
|
1299
|
+
'
|
|
1300
|
+
test -z "${QA_INJECTED+x}"
|
|
1301
|
+
|
|
1302
|
+
set +e
|
|
1303
|
+
akm env run env/qa-runtime -- bun -e 'process.exit(7)'
|
|
1304
|
+
env_child_status=$?
|
|
1305
|
+
akm secret run secrets/qa-token QA_INJECTED -- bun -e 'process.exit(9)'
|
|
1306
|
+
secret_child_status=$?
|
|
1307
|
+
set -e
|
|
1308
|
+
test "$env_child_status" -eq 7
|
|
1309
|
+
test "$secret_child_status" -eq 9
|
|
1310
|
+
```
|
|
1311
|
+
|
|
1312
|
+
- [ ] **[CORE]** Values enter only the child environment and never mutate the
|
|
1313
|
+
invoking shell.
|
|
1314
|
+
- [ ] **[CORE]** `${secret:NAME}` resolves from the env asset's own bundle,
|
|
1315
|
+
including embedded/multiple tokens. Missing secret fails before spawn.
|
|
1316
|
+
- [ ] **[CORE]** `--only` and `--except` are mutually exclusive; filtering occurs
|
|
1317
|
+
before dangerous-key policy. Clean mode inherits only the documented
|
|
1318
|
+
minimum plus explicit names.
|
|
1319
|
+
- [ ] **[CORE]** Child stdout/stderr pass through unchanged and exact numeric
|
|
1320
|
+
child status is returned. Do not claim those raw streams are redacted.
|
|
1321
|
+
- [ ] **[LOCAL]** Missing command is usage exit `2`; missing executable is
|
|
1322
|
+
`FILE_NOT_FOUND`, exit `1`; non-executable target is config exit `78`.
|
|
1323
|
+
- [ ] **[LOCAL]** Access events contain only env ref/key names or secret ref/
|
|
1324
|
+
target variable. They contain no values and occur only after resolution.
|
|
1325
|
+
- [ ] **[LOCAL]** Timeout/signal behavior returns a nonzero signal-equivalent
|
|
1326
|
+
result and leaves no child. A null child status must not become success.
|
|
1327
|
+
- [ ] **[LOCAL]** Format flags on `env path`, `env run`, and `secret run` warn on
|
|
1328
|
+
stderr and leave the raw payload/child streams unchanged.
|
|
1329
|
+
|
|
1330
|
+
### 11.3 Export, overwrite, targets, and removal
|
|
1331
|
+
|
|
1332
|
+
```sh
|
|
1333
|
+
printf 'QA_LITERAL=$(touch %s)\n' "$AKM_SANDBOX/export-payload-ran" \
|
|
1334
|
+
>"$AKM_SANDBOX/qa-export.env"
|
|
1335
|
+
akm env create qa-export --from-file "$AKM_SANDBOX/qa-export.env"
|
|
1336
|
+
akm env export env/qa-export --out "$AKM_SANDBOX/qa-export.sh" \
|
|
1337
|
+
--format json >"$AKM_SANDBOX/env-export.json"
|
|
1338
|
+
|
|
1339
|
+
test ! -e "$AKM_SANDBOX/export-payload-ran"
|
|
1340
|
+
sh -c '. "$1"' sh "$AKM_SANDBOX/qa-export.sh"
|
|
1341
|
+
test ! -e "$AKM_SANDBOX/export-payload-ran"
|
|
1342
|
+
|
|
1343
|
+
printf 'replacement-without-trailing-newline' |
|
|
1344
|
+
akm secret set secrets/qa-token
|
|
1345
|
+
akm env remove env/qa-export --yes
|
|
1346
|
+
```
|
|
1347
|
+
|
|
1348
|
+
- [ ] **[CORE]** Export requires `--out`, prints no values, writes atomically at
|
|
1349
|
+
`0600`, and single-quotes substitutions/backticks/dollars/quotes/globs.
|
|
1350
|
+
- [ ] **[CORE]** Export does not resolve `${secret:...}`. `env run` is the safe
|
|
1351
|
+
default for secret substitution.
|
|
1352
|
+
- [ ] **[CORE]** Secret stdin strips one trailing newline; file input preserves
|
|
1353
|
+
bytes exactly. Secret overwrite is atomic and serialized.
|
|
1354
|
+
- [ ] **[LOCAL]** Explicit target wins over `defaultWriteTarget`, then default
|
|
1355
|
+
bundle. Read-only/disabled/unsupported targets fail before mutation.
|
|
1356
|
+
- [ ] **[LOCAL]** Writable Git target publishes one boundary commit unless the
|
|
1357
|
+
files are intentionally ignored and stored as local-only overlays.
|
|
1358
|
+
- [ ] **[LOCAL]** Env remove needs confirmation, removes the env and marker, and
|
|
1359
|
+
leaves unrelated assets unchanged.
|
|
1360
|
+
- [ ] **[LOCAL]** `secret path` and `secret remove` are removed and exit `2`.
|
|
1361
|
+
Locate/delete a secret under the reviewed bundle root, then reindex.
|
|
1362
|
+
- [ ] **[LOCAL REGRESSION]** Duplicate env refs across bundles resolve to the
|
|
1363
|
+
same file for path/run/export/remove. Read/write resolver disagreement is
|
|
1364
|
+
a failure.
|
|
1365
|
+
|
|
1366
|
+
### 11.4 Dangerous keys and redaction boundary
|
|
1367
|
+
|
|
1368
|
+
- [ ] **[CORE]** Secret run rejects dangerous target variables such as `PATH`,
|
|
1369
|
+
`LD_PRELOAD`, `NODE_OPTIONS`, and `GIT_CONFIG_*` before spawn/event.
|
|
1370
|
+
- [ ] **[LOCAL]** First-party env activation warns and proceeds; third-party
|
|
1371
|
+
activation blocks exit `2`. Excluding all dangerous keys permits the safe
|
|
1372
|
+
subset.
|
|
1373
|
+
- [ ] **[LOCAL]** `--allow-dangerous-env-keys` can authorize reviewed installation but
|
|
1374
|
+
never authorizes later dangerous activation.
|
|
1375
|
+
- [ ] **[LOCAL REGRESSION]** Lint suppression cannot authorize untrusted install;
|
|
1376
|
+
declarative first materialization and every update receive the same audit.
|
|
1377
|
+
- [ ] **[LOCAL REGRESSION]** Env export applies an equivalent dangerous-key
|
|
1378
|
+
warning/gate before producing sourceable output.
|
|
1379
|
+
- [ ] **[LOCAL]** Search all captured AKM results, events, state/log databases,
|
|
1380
|
+
reports, and config for the literal dummy value. Exclude deliberate child
|
|
1381
|
+
passthrough evidence from the AKM-redaction claim.
|
|
1382
|
+
|
|
1383
|
+
---
|
|
1384
|
+
|
|
1385
|
+
## 12. Lint
|
|
1386
|
+
|
|
1387
|
+
### 12.1 Detect, gate, fix, and repeat
|
|
1388
|
+
|
|
1389
|
+
Create a disposable native-AKM lint root:
|
|
1390
|
+
|
|
1391
|
+
```sh
|
|
1392
|
+
export QA_LINT_DIR="$AKM_SANDBOX/lint-bundle"
|
|
1393
|
+
mkdir -p "$QA_LINT_DIR/knowledge"
|
|
1394
|
+
cat >"$QA_LINT_DIR/knowledge/fixable.md" <<'EOF'
|
|
1395
|
+
---
|
|
1396
|
+
description: Manual QA: fixable colon
|
|
1397
|
+
---
|
|
1398
|
+
# Fixable
|
|
1399
|
+
EOF
|
|
1400
|
+
|
|
1401
|
+
akm lint --dir "$QA_LINT_DIR" --format json \
|
|
1402
|
+
>"$AKM_SANDBOX/lint-detect.json"
|
|
1403
|
+
jq -e '.ok == true and .summary.flagged >= 1' \
|
|
1404
|
+
"$AKM_SANDBOX/lint-detect.json"
|
|
1405
|
+
|
|
1406
|
+
set +e
|
|
1407
|
+
akm lint --dir "$QA_LINT_DIR" --fail-on-flagged --format json \
|
|
1408
|
+
>"$AKM_SANDBOX/lint-gate.json" \
|
|
1409
|
+
2>"$AKM_SANDBOX/lint-gate.stderr"
|
|
1410
|
+
lint_gate_status=$?
|
|
1411
|
+
set -e
|
|
1412
|
+
test "$lint_gate_status" -eq 1
|
|
1413
|
+
test ! -s "$AKM_SANDBOX/lint-gate.stderr"
|
|
1414
|
+
jq -e '.ok == true and .summary.flagged >= 1' \
|
|
1415
|
+
"$AKM_SANDBOX/lint-gate.json"
|
|
1416
|
+
|
|
1417
|
+
akm lint --dir "$QA_LINT_DIR" --fix --format json \
|
|
1418
|
+
>"$AKM_SANDBOX/lint-fix.json"
|
|
1419
|
+
jq -e '.summary.fixed >= 1' "$AKM_SANDBOX/lint-fix.json"
|
|
1420
|
+
|
|
1421
|
+
snapshot_tree "$QA_LINT_DIR" "$AKM_SANDBOX/lint-fixed.sha256"
|
|
1422
|
+
akm lint --dir "$QA_LINT_DIR" --auto-fix --format json \
|
|
1423
|
+
>"$AKM_SANDBOX/lint-fix-second.json"
|
|
1424
|
+
jq -e '.summary.fixed == 0' "$AKM_SANDBOX/lint-fix-second.json"
|
|
1425
|
+
snapshot_tree "$QA_LINT_DIR" "$AKM_SANDBOX/lint-fixed-again.sha256"
|
|
1426
|
+
cmp "$AKM_SANDBOX/lint-fixed.sha256" \
|
|
1427
|
+
"$AKM_SANDBOX/lint-fixed-again.sha256"
|
|
1428
|
+
```
|
|
1429
|
+
|
|
1430
|
+
- [ ] **[CORE]** Plain lint exits `0` with `ok:true` even when findings exist.
|
|
1431
|
+
- [ ] **[CORE]** Fail-on-flagged exits `1` but retains the complete successful
|
|
1432
|
+
lint result on stdout and leaves stderr empty.
|
|
1433
|
+
- [ ] **[CORE]** Fix applies only fixable findings, reports failed fixes as
|
|
1434
|
+
flagged, and is byte-idempotent on the second run.
|
|
1435
|
+
- [ ] **[CORE]** Fix plus fail-on-flagged exits `1` only if findings remain.
|
|
1436
|
+
- [ ] **[CORE]** `--auto-fix` equals `--fix`; single-dash `-auto-fix` is an
|
|
1437
|
+
unknown flag, exit `2`.
|
|
1438
|
+
- [ ] **[CORE]** Each finding has relative file, issue, detail, and tri-state
|
|
1439
|
+
fixed status. No absolute bundle path or secret value appears.
|
|
1440
|
+
|
|
1441
|
+
### 12.2 Native and adapter scopes
|
|
1442
|
+
|
|
1443
|
+
```sh
|
|
1444
|
+
akm lint --dir "$AKM_BUNDLE_DIR" --type workflows --format json \
|
|
1445
|
+
>"$AKM_SANDBOX/lint-workflows.json"
|
|
1446
|
+
akm lint --dir "$REPO/tests/fixtures/stashes/all-types" --format json \
|
|
1447
|
+
>"$AKM_SANDBOX/lint-all-types.json"
|
|
1448
|
+
akm lint --dir "$REPO/tests/fixtures/bundles/llm-wiki" --format json \
|
|
1449
|
+
>"$AKM_SANDBOX/lint-wiki.json"
|
|
1450
|
+
```
|
|
1451
|
+
|
|
1452
|
+
- [ ] **[CORE]** Native sweep covers agents, commands, memories, skills,
|
|
1453
|
+
workflows, lessons, tasks, knowledge, and facts, plus a separate dangerous
|
|
1454
|
+
`.env` pass.
|
|
1455
|
+
- [ ] **[LOCAL]** Workflow Markdown compiles structurally; task `.yml` validates
|
|
1456
|
+
version/schedule/enabled/target; missing `SKILL.md` is flagged.
|
|
1457
|
+
- [ ] **[LOCAL]** `--type` accepts exact plural native directories and genuinely
|
|
1458
|
+
narrows structural and dangerous-env scanning.
|
|
1459
|
+
- [ ] **[LOCAL REGRESSION]** Singular/unknown type and missing/unreadable `--dir`
|
|
1460
|
+
fail instead of returning a false-clean result.
|
|
1461
|
+
- [ ] **[LOCAL]** Non-AKM roots dispatch through the detected/configured adapter
|
|
1462
|
+
`validate()` implementation and remain read-only even with `--fix`.
|
|
1463
|
+
- [ ] **[LOCAL]** LLM Wiki reports uncited raw, missing description, broken xref,
|
|
1464
|
+
and broken source; dotenv reports dangerous keys without exposing values.
|
|
1465
|
+
- [ ] **[LOCAL REGRESSION]** `--type` is not silently ignored by non-AKM adapter
|
|
1466
|
+
validation.
|
|
1467
|
+
- [ ] **[CORE]** Unsupported `--profile`, `--category`, `--severity`, and
|
|
1468
|
+
`--bundle` flags fail exit `2`; do not invent selector semantics.
|
|
1469
|
+
|
|
1470
|
+
### 12.3 Rules, suppressions, and mutation boundaries
|
|
1471
|
+
|
|
1472
|
+
Core issue codes include `unquoted-colon`, `missing-updated`, `orphaned-stub`,
|
|
1473
|
+
`placeholder-stub`, `missing-name-or-type`, `missing-type`, `stale-path`,
|
|
1474
|
+
`missing-skill-md`, `invalid-task-yaml`, `missing-ref`,
|
|
1475
|
+
`dangerous-env-key`, `invalid-workflow-structure`, and `missing-category`.
|
|
1476
|
+
Adapters may add named diagnostics; unknown adapter codes map to
|
|
1477
|
+
`adapter-diagnostic` while preserving the original code in detail.
|
|
1478
|
+
|
|
1479
|
+
- [ ] **[LOCAL]** Only unquoted description, missing updated, orphaned stub, and
|
|
1480
|
+
placeholder stub fixes mutate; all structural/security findings remain
|
|
1481
|
+
flagged.
|
|
1482
|
+
- [ ] **[LOCAL]** `lint_skip` scalar/list suppresses only supported base rules,
|
|
1483
|
+
exactly and case-sensitively. It cannot suppress dangerous env or per-type
|
|
1484
|
+
structure.
|
|
1485
|
+
- [ ] **[LOCAL]** `# akm-lint-ok: dangerous-env-key` suppresses only the next
|
|
1486
|
+
matching dotenv assignment for lint display, never installation/runtime.
|
|
1487
|
+
- [ ] **[LOCAL REGRESSION]** Suppression also associates with `export KEY=...`;
|
|
1488
|
+
malformed task YAML and `.yaml` spelling produce explicit findings rather
|
|
1489
|
+
than disappearing.
|
|
1490
|
+
- [ ] **[LOCAL]** Fenced-code refs are ignored; body refs and refs/xrefs/
|
|
1491
|
+
supersededBy/contradictedBy fields are checked across configured bundles.
|
|
1492
|
+
- [ ] **[LOCAL]** Lint appends no event, performs no reindex, and adapter
|
|
1493
|
+
validation never writes.
|
|
1494
|
+
- [ ] **[LOCAL REGRESSION]** Native fix honors read-only/Git publication
|
|
1495
|
+
boundaries and either behaves transactionally or reports partial mutation
|
|
1496
|
+
explicitly. Silent direct edits/deletes are a failure.
|
|
1497
|
+
|
|
1498
|
+
### 12.4 Advisory channel
|
|
1499
|
+
|
|
1500
|
+
Non-fatal workflow compile advisories (a step with no `output:` schema, a
|
|
1501
|
+
reference to an undeclared param) travel in their own channel so a hint cannot
|
|
1502
|
+
fail a build.
|
|
1503
|
+
|
|
1504
|
+
```sh
|
|
1505
|
+
akm lint --type workflows --format json > "$AKM_SANDBOX/lint-advisory.json"
|
|
1506
|
+
jq -e '.warnings | length > 0' "$AKM_SANDBOX/lint-advisory.json"
|
|
1507
|
+
jq -e '[.flagged[].issue] | index("workflow-warning") == null' "$AKM_SANDBOX/lint-advisory.json"
|
|
1508
|
+
akm lint --type workflows --fail-on-flagged; echo "exit=$?"
|
|
1509
|
+
```
|
|
1510
|
+
|
|
1511
|
+
- [ ] **[CORE]** A compile advisory appears in `warnings` with issue code
|
|
1512
|
+
`workflow-warning`, never in `flagged`, and `summary.warnings` counts it.
|
|
1513
|
+
- [ ] **[CORE]** `--fail-on-flagged` exits `0` for a bundle whose only findings
|
|
1514
|
+
are advisories, and non-zero once a real error is present.
|
|
1515
|
+
- [ ] **[LOCAL]** Both lint paths agree: the akm sweep and a bundle linted
|
|
1516
|
+
through its own adapter classify the same code the same way.
|
|
1517
|
+
- [ ] **[LOCAL]** A workflow file is parsed and compiled **once** per sweep, not
|
|
1518
|
+
once per output channel — check with a large workflow and compare sweep
|
|
1519
|
+
time against a bundle with the same file count and no workflows.
|
|
1520
|
+
|
|
1521
|
+
### 12.5 Output parity and automated coverage
|
|
1522
|
+
|
|
1523
|
+
- [ ] **[CORE]** JSON/JSONL/YAML/text/Markdown/HTML preserve the same finding
|
|
1524
|
+
set and summary. Text renders flagged before fixed.
|
|
1525
|
+
- [ ] **[LOCAL]** Hard usage/config/internal failures use standard structured
|
|
1526
|
+
stderr and do not emit a partial lint result.
|
|
1527
|
+
- [ ] **[LOCAL]** Run focused lint, adapter dispatch, workflow structure,
|
|
1528
|
+
dangerous-key, and format-family golden suites after linter changes.
|
|
1529
|
+
|
|
1530
|
+
---
|
|
1531
|
+
|
|
1532
|
+
## 13. Feedback, Log, and Health
|
|
1533
|
+
|
|
1534
|
+
### 13.1 Feedback taxonomy and ranking
|
|
1535
|
+
|
|
1536
|
+
```sh
|
|
1537
|
+
akm feedback skills/k8s-deploy --positive --tag slice:manual --tag team:qa
|
|
1538
|
+
expect_error 2 MISSING_REQUIRED_ARGUMENT \
|
|
1539
|
+
feedback skills/k8s-deploy --negative
|
|
1540
|
+
akm feedback skills/k8s-deploy --negative \
|
|
1541
|
+
--reason "manual QA relevance check"
|
|
1542
|
+
```
|
|
1543
|
+
|
|
1544
|
+
- [ ] **CORE** Positive and reasoned negative feedback return stable signal/reason fields.
|
|
1545
|
+
- [ ] **CORE** Negative feedback without a reason fails by default with `MISSING_REQUIRED_ARGUMENT`, exit `2`.
|
|
1546
|
+
- [ ] **LOCAL** Both signals, missing/nonexistent ref, malformed tag, and more than ten tags fail before durable mutation.
|
|
1547
|
+
- [ ] **LOCAL** `feedback.requireReason:false` produces the documented warning rather than the default hard failure.
|
|
1548
|
+
- [ ] **LOCAL** User feedback updates ranking immediately and reports `rankingUpdate.applied`; machine-origin signals do not impersonate user demand.
|
|
1549
|
+
- [ ] **LOCAL** Feedback survives full reindex and contains no credential fixture value.
|
|
1550
|
+
|
|
1551
|
+
### 13.2 Durable log and cursors
|
|
1552
|
+
|
|
1553
|
+
```sh
|
|
1554
|
+
akm log --type feedback --ref skills/k8s-deploy --detail full --format json \
|
|
1555
|
+
> "$AKM_SANDBOX/feedback-log.json"
|
|
1556
|
+
event_id="$(jq -r '.events[-1].id' "$AKM_SANDBOX/feedback-log.json")"
|
|
1557
|
+
akm log --since "@offset:$event_id" --format json
|
|
1558
|
+
```
|
|
1559
|
+
|
|
1560
|
+
- [ ] **CORE** Mutations produce numeric durable ids, normalized refs, type, timestamp, and structured non-secret metadata.
|
|
1561
|
+
- [ ] **LOCAL** Type/ref/run/since/limit/include-tags/exclude-tags filters compose correctly.
|
|
1562
|
+
- [ ] **LOCAL** `@offset:<id>` returns only later events across a fresh process and exposes a reusable next offset.
|
|
1563
|
+
- [ ] **LOCAL** Include-tags requires all requested tags; exclude removes any matching event.
|
|
1564
|
+
- [ ] **LOCAL** Retired `log list` and `log tail` fail with current `akm log` guidance.
|
|
1565
|
+
- [ ] **LOCAL** Full rebuild/source update/restart preserve events and monotonic cursors.
|
|
1566
|
+
- [ ] **LOCAL** Event metadata stores ids/status/counts, not workflow instructions, child output, prompts, env values, or credentials.
|
|
1567
|
+
|
|
1568
|
+
### 13.3 Health pass, warn, and fail
|
|
1569
|
+
|
|
1570
|
+
```sh
|
|
1571
|
+
set +e
|
|
1572
|
+
akm health --format json > "$AKM_SANDBOX/health.json"
|
|
1573
|
+
health_status=$?
|
|
1574
|
+
set -e
|
|
1575
|
+
|
|
1576
|
+
jq -e --argjson status "$health_status" '
|
|
1577
|
+
(.status == "pass" and $status == 0) or
|
|
1578
|
+
(.status == "warn" and $status == 4) or
|
|
1579
|
+
(.status == "fail" and $status == 1)
|
|
1580
|
+
' "$AKM_SANDBOX/health.json"
|
|
1581
|
+
```
|
|
1582
|
+
|
|
1583
|
+
- [ ] **CORE** Health status and exit mapping agree; warn remains `ok:true`, hard failure is `ok:false`.
|
|
1584
|
+
- [ ] **LOCAL** Fresh data dir initializes `state.db`, validates schema/round-trip, and does not call initial absence corruption.
|
|
1585
|
+
- [ ] **LOCAL** Fake default agent provides deterministic healthy engine evidence.
|
|
1586
|
+
- [ ] **SERVICE** Semantic config `auto` plus blocked runtime produces nonfatal advisory/exit `4` when hard checks pass.
|
|
1587
|
+
- [ ] **LOCAL** A copied secondary state DB with a failed hard check returns health stdout result and exit `1`; never corrupt primary evidence DB.
|
|
1588
|
+
- [ ] **LOCAL** Active/stuck runs, task backing/fail rate, agent failure rate, improve/session metrics, and graph summary are accurate.
|
|
1589
|
+
- [ ] **LOCAL** Since accepts duration/date/ISO/epoch and rejects invalid/future values predictably.
|
|
1590
|
+
- [ ] **LOCAL** Up to four repeated windows work; explicit windows and window-compare are mutually exclusive.
|
|
1591
|
+
|
|
1592
|
+
### 13.4 Health reports
|
|
1593
|
+
|
|
1594
|
+
```sh
|
|
1595
|
+
akm health --report --format json > "$AKM_SANDBOX/health-report.json"
|
|
1596
|
+
akm health --report --format html --output "$AKM_SANDBOX/health-report.html"
|
|
1597
|
+
akm health --group-by run --format md > "$AKM_SANDBOX/health-runs.md"
|
|
1598
|
+
```
|
|
1599
|
+
|
|
1600
|
+
- [ ] **LOCAL** JSON/Markdown/HTML contain the same per-run/trend/proposal/accept-rate data.
|
|
1601
|
+
- [ ] **LOCAL** Absolute since is not misparsed as implicit duration comparison.
|
|
1602
|
+
- [ ] **LOCAL** HTML is self-contained, safely escaped, and credential-free.
|
|
1603
|
+
- [ ] **LOCAL** Report read failure is not overwritten by stale status-derived exit.
|
|
1604
|
+
|
|
1605
|
+
---
|
|
1606
|
+
|
|
1607
|
+
## 14. Workflow
|
|
1608
|
+
|
|
1609
|
+
### 14.1 Authoring and validation
|
|
1610
|
+
|
|
1611
|
+
```sh
|
|
1612
|
+
akm workflow list --format json | jq -e '.runs == []'
|
|
1613
|
+
akm workflow create qa-template --print > "$AKM_SANDBOX/qa-template.md"
|
|
1614
|
+
test ! -e "$AKM_BUNDLE_DIR/workflows/qa-template.md"
|
|
1615
|
+
akm workflow create qa-created --from "$AKM_SANDBOX/qa-template.md"
|
|
1616
|
+
akm lint --type workflows --fail-on-flagged
|
|
1617
|
+
```
|
|
1618
|
+
|
|
1619
|
+
- [ ] **CORE** Create-print emits raw valid Markdown and writes nothing.
|
|
1620
|
+
- [ ] **CORE** Create-from writes/indexes valid workflow with preamble and exact sections.
|
|
1621
|
+
- [ ] **LOCAL** Flat name plus safe path creates hierarchy; slash/traversal/duplicate step/missing section/unknown route/invalid params fail before write.
|
|
1622
|
+
- [ ] **LOCAL** Existing workflow requires force plus from/reset; force alone fails.
|
|
1623
|
+
- [ ] **LOCAL** Lint catches structure without rewriting valid prose.
|
|
1624
|
+
- [ ] **CORE** A schema keyword outside the enforced subset (`format`, `pattern`,
|
|
1625
|
+
`$ref`, `const`, `uniqueItems`, `patternProperties`, …) in `output:` or
|
|
1626
|
+
`params:` is a **line-anchored parse error naming the keyword**, not a
|
|
1627
|
+
silently non-constraining schema. Confirm the same document fails
|
|
1628
|
+
`workflow run`, `workflow show`, `workflow create`, and `akm lint`.
|
|
1629
|
+
- [ ] **LOCAL** `akm index` skips such a workflow with a scan warning rather
|
|
1630
|
+
than indexing it — the quietest surface, so check it deliberately.
|
|
1631
|
+
- [ ] **LOCAL** `allOf`/`anyOf`/`oneOf`/`not` are accepted and **enforced** at
|
|
1632
|
+
runtime; a bounded-evaluation overrun is reported as an error, never as a
|
|
1633
|
+
truncated pass.
|
|
1634
|
+
- [ ] **LOCAL** A document-level `defaults.llm` combined with any step on an
|
|
1635
|
+
agent engine fails at freeze, naming the step and the engine. Moving the
|
|
1636
|
+
`llm:` block onto the LLM step's `unit:` is the documented remedy; `llm: {}`
|
|
1637
|
+
is a no-op and `llm: null` is a parse error.
|
|
1638
|
+
|
|
1639
|
+
### 14.2 Typed params and partial run without an engine call
|
|
1640
|
+
|
|
1641
|
+
The `typed-route` fixture executes a route-only first step. `--max-steps 1`
|
|
1642
|
+
tests parameter parsing and durable run creation without agent/LLM dispatch.
|
|
1643
|
+
|
|
1644
|
+
```sh
|
|
1645
|
+
akm workflow run workflows/typed-route \
|
|
1646
|
+
--include_processes=true --count 2 \
|
|
1647
|
+
--labels api --labels worker --max-steps 1 \
|
|
1648
|
+
> "$AKM_SANDBOX/typed-run.json"
|
|
1649
|
+
|
|
1650
|
+
export QA_RUN_ID="$(jq -r .run.id "$AKM_SANDBOX/typed-run.json")"
|
|
1651
|
+
jq -e '
|
|
1652
|
+
.run.status == "active" and
|
|
1653
|
+
.run.currentStepId == "finish" and
|
|
1654
|
+
.run.params == {
|
|
1655
|
+
"include_processes": true,
|
|
1656
|
+
"count": 2,
|
|
1657
|
+
"labels": ["api", "worker"]
|
|
1658
|
+
}
|
|
1659
|
+
' "$AKM_SANDBOX/typed-run.json"
|
|
1660
|
+
```
|
|
1661
|
+
|
|
1662
|
+
- [ ] **LOCAL** Boolean/integer/number/string/null/enum/object/repeated array/JSON array coerce to schema types.
|
|
1663
|
+
- [ ] **LOCAL** Unknown param, underscore-hyphen alias, invalid JSON/enum/range, missing required, duplicate scalar, flags before target, and retired params bag fail exit `2`.
|
|
1664
|
+
- [ ] **LOCAL** Params supplied to active run fail because creation-only.
|
|
1665
|
+
- [ ] **LOCAL** Invalid max-steps values fail; bounded partial exits `0`, remains active.
|
|
1666
|
+
- [ ] **LOCAL** `--max-steps` counts **finished spine steps**: a step's whole
|
|
1667
|
+
bounded gate loop counts as one, and a route-skipped step consumes none.
|
|
1668
|
+
It does not bound the dispatches inside a single step's gate loop — pair
|
|
1669
|
+
it with `budget.max_units` when that is what you need capped.
|
|
1670
|
+
|
|
1671
|
+
### 14.3 Status, list, abandon, resume, and scope
|
|
1672
|
+
|
|
1673
|
+
```sh
|
|
1674
|
+
akm workflow status "$QA_RUN_ID"
|
|
1675
|
+
akm workflow status workflows/typed-route
|
|
1676
|
+
akm workflow list --active --ref workflows/typed-route
|
|
1677
|
+
akm workflow status "$QA_RUN_ID" --units
|
|
1678
|
+
akm workflow abandon "$QA_RUN_ID"
|
|
1679
|
+
akm workflow resume "$QA_RUN_ID"
|
|
1680
|
+
```
|
|
1681
|
+
|
|
1682
|
+
- [ ] **LOCAL** Direct id status works from another dir; ref status/list remain current-scope only.
|
|
1683
|
+
- [ ] **LOCAL** Units adds diagnostics without state change.
|
|
1684
|
+
- [ ] **LOCAL** Abandon marks failed/removes active; resume reopens failed/blocked, never completed.
|
|
1685
|
+
- [ ] **LOCAL** Run by ref while active resumes it; direct id continues exact run.
|
|
1686
|
+
- [ ] **LOCAL** Source edit after start does not change frozen plan; new run sees edit.
|
|
1687
|
+
|
|
1688
|
+
### 14.4 Deterministic execution and gates
|
|
1689
|
+
|
|
1690
|
+
Configure fake engines from section 4.4, then continue the active run:
|
|
1691
|
+
|
|
1692
|
+
```sh
|
|
1693
|
+
akm workflow run "$QA_RUN_ID" --max-steps 1 \
|
|
1694
|
+
> "$AKM_SANDBOX/typed-finished.json"
|
|
1695
|
+
jq -e '.run.status == "completed"' "$AKM_SANDBOX/typed-finished.json"
|
|
1696
|
+
```
|
|
1697
|
+
|
|
1698
|
+
Create separate copies of `gated-agent` for pass/reject/malformed judges so each
|
|
1699
|
+
run freezes its own judge selection.
|
|
1700
|
+
|
|
1701
|
+
- [ ] **AI** Pass verdict completes.
|
|
1702
|
+
- [ ] **AI** Reject verdict returns missing/feedback evidence and nonzero domain result.
|
|
1703
|
+
- [ ] **AI** Malformed verdict, agent/HTTP failure, missing judge, and timeout fail closed.
|
|
1704
|
+
- [ ] **AI** Max loops bounds retries and prior gate feedback enters next unit.
|
|
1705
|
+
- [ ] **AI** Gate journal distinguishes running/error/pass/reject and survives resume.
|
|
1706
|
+
- [ ] **AI** Config changes after start do not replace frozen execution/judge engine/model/timeout.
|
|
1707
|
+
|
|
1708
|
+
### 14.5 Exec (shell) units
|
|
1709
|
+
|
|
1710
|
+
An exec unit spawns a command instead of dispatching to an engine, so run this
|
|
1711
|
+
whole subsection with **no engine configured** — that is the first assertion,
|
|
1712
|
+
not a setup shortcut. Author the fixtures first:
|
|
1713
|
+
|
|
1714
|
+
```sh
|
|
1715
|
+
cat > "$AKM_BUNDLE_DIR/workflows/exec-basic.md" <<'EOF'
|
|
1716
|
+
---
|
|
1717
|
+
type: workflow
|
|
1718
|
+
description: Exec unit smoke
|
|
1719
|
+
steps:
|
|
1720
|
+
- id: emit
|
|
1721
|
+
unit:
|
|
1722
|
+
exec:
|
|
1723
|
+
command: ["printf", "hello\n\n"]
|
|
1724
|
+
---
|
|
1725
|
+
|
|
1726
|
+
# Exec Basic
|
|
1727
|
+
|
|
1728
|
+
## emit
|
|
1729
|
+
|
|
1730
|
+
Print a greeting. Stdout is the promoted artifact; this prose never reaches the
|
|
1731
|
+
command.
|
|
1732
|
+
EOF
|
|
1733
|
+
|
|
1734
|
+
cat > "$AKM_BUNDLE_DIR/workflows/exec-json.md" <<'EOF'
|
|
1735
|
+
---
|
|
1736
|
+
type: workflow
|
|
1737
|
+
description: Exec unit with a typed artifact
|
|
1738
|
+
steps:
|
|
1739
|
+
- id: emit
|
|
1740
|
+
unit:
|
|
1741
|
+
exec:
|
|
1742
|
+
command: ["echo", '{"verdict":"pass"}']
|
|
1743
|
+
output: { type: object, required: [verdict], properties: { verdict: { type: string } } }
|
|
1744
|
+
---
|
|
1745
|
+
|
|
1746
|
+
# Exec JSON
|
|
1747
|
+
|
|
1748
|
+
## emit
|
|
1749
|
+
|
|
1750
|
+
Emit exactly one JSON value.
|
|
1751
|
+
EOF
|
|
1752
|
+
|
|
1753
|
+
akm index
|
|
1754
|
+
akm lint --type workflows --fail-on-flagged
|
|
1755
|
+
akm workflow run workflows/exec-basic --format json > "$AKM_SANDBOX/exec-basic.json"
|
|
1756
|
+
jq -e '.run.status == "completed"' "$AKM_SANDBOX/exec-basic.json"
|
|
1757
|
+
akm workflow status "$(jq -r .run.id "$AKM_SANDBOX/exec-basic.json")" --units
|
|
1758
|
+
```
|
|
1759
|
+
|
|
1760
|
+
Authoring and dispatch:
|
|
1761
|
+
|
|
1762
|
+
- [ ] **CORE** A workflow of only exec steps freezes and completes with **no
|
|
1763
|
+
engine configured at all**; the run spends no tokens.
|
|
1764
|
+
- [ ] **CORE** Without `output:`, the artifact is stdout with trailing newlines
|
|
1765
|
+
stripped (`hello`, not `hello\n\n`); empty stdout is *no output*.
|
|
1766
|
+
- [ ] **CORE** With `output:`, stdout must be exactly one JSON value — leading
|
|
1767
|
+
and trailing whitespace tolerated, any other trailing text fails the unit
|
|
1768
|
+
— and the value is validated against the schema.
|
|
1769
|
+
- [ ] **CORE** `engine`, `model`, or `llm` alongside `exec:` fails at parse,
|
|
1770
|
+
naming the conflict.
|
|
1771
|
+
- [ ] **LOCAL** `command` is argv, never a shell string: an argument containing
|
|
1772
|
+
`;`, `|`, `&&`, `$(…)`, backticks, `>`, or `*` reaches the child as those
|
|
1773
|
+
literal bytes. `["bash", "-lc", "…"]` is the supported way to get a shell.
|
|
1774
|
+
- [ ] **LOCAL** Argv bounds hold: 1–64 entries, each non-empty and at most 4096
|
|
1775
|
+
bytes; violations fail at parse and at frozen-plan decode.
|
|
1776
|
+
- [ ] **LOCAL** A step body is still required and is **not** passed to the
|
|
1777
|
+
command.
|
|
1778
|
+
- [ ] **LOCAL** stderr is a diagnostic channel only — it never becomes the
|
|
1779
|
+
artifact, and a passing command that writes to stderr still succeeds.
|
|
1780
|
+
|
|
1781
|
+
Failure taxonomy — each reason, and whether `retry.on` may name it:
|
|
1782
|
+
|
|
1783
|
+
- [ ] **CORE** Non-zero exit is `non_zero_exit` and, under the default
|
|
1784
|
+
`on_error: fail`, fails the step and the run (this is what makes a `test`
|
|
1785
|
+
step a gate). `on_error: continue` records it in evidence instead.
|
|
1786
|
+
- [ ] **LOCAL** Wall-clock expiry is `timeout`; run cancellation is `aborted`; a
|
|
1787
|
+
missing binary or unusable working directory is `spawn_failed`. All three
|
|
1788
|
+
are retryable via `retry.on`.
|
|
1789
|
+
- [ ] **LOCAL** `exec_output_limit`, `exec_context_too_large`, `exec_cwd_escape`
|
|
1790
|
+
and `exec_capture_incomplete` are **rejected** in a `retry.on` list —
|
|
1791
|
+
each is deterministic, an authoring problem, or work that already ran.
|
|
1792
|
+
- [ ] **LOCAL** On timeout or cancellation the child's whole **process group**
|
|
1793
|
+
takes the SIGTERM→SIGKILL ladder; no orphaned grandchildren survive.
|
|
1794
|
+
|
|
1795
|
+
Capture and retention:
|
|
1796
|
+
|
|
1797
|
+
- [ ] **LOCAL** Past the 8 MiB per-stream retention cap akm keeps draining the
|
|
1798
|
+
pipe, so the command never blocks: with no `output:` schema the unit
|
|
1799
|
+
**succeeds** with the artifact marked truncated; with a schema it fails
|
|
1800
|
+
`exec_output_limit`.
|
|
1801
|
+
- [ ] **LOCAL** A command that backgrounds a descendant inheriting **stderr**
|
|
1802
|
+
(`sh -c 'daemon 2>&1 & echo ok'` shapes) and exits 0 **succeeds** with its
|
|
1803
|
+
complete stdout artifact, and warns that the stderr tail may be missing.
|
|
1804
|
+
Only an incomplete **stdout** capture fails the unit.
|
|
1805
|
+
- [ ] **LOCAL** That same unit settles promptly after its leader exits — it does
|
|
1806
|
+
**not** sit until its declared timeout expires.
|
|
1807
|
+
- [ ] **LOCAL** `timeout: none` is genuinely unbounded: a command that runs well
|
|
1808
|
+
past an hour still completes and promotes its artifact.
|
|
1809
|
+
|
|
1810
|
+
Environment scope and context:
|
|
1811
|
+
|
|
1812
|
+
- [ ] **LOCAL** The child gets the default allowlist only: credentials and cloud
|
|
1813
|
+
or CI variables present in akm's own environment are **absent** unless
|
|
1814
|
+
named. `pass_env:` widens it by name; exact named `env:` bindings provide
|
|
1815
|
+
fixed or secret values.
|
|
1816
|
+
- [ ] **CORE** A new durable-v4-family (`irVersion: 5`) start containing
|
|
1817
|
+
`inherit_env: true` is rejected before dispatch and directs the author to
|
|
1818
|
+
named env bindings. A pre-`irVersion`-5 stored plan is also rejected and
|
|
1819
|
+
directs the user to start a new run.
|
|
1820
|
+
- [ ] **LOCAL** `env:` bindings inject resolved values, and the `AKM_*` context
|
|
1821
|
+
is applied *after* them, so a binding cannot shadow it.
|
|
1822
|
+
- [ ] **LOCAL** `AKM_RUN_ID`, `AKM_STEP_ID`, `AKM_UNIT_ID`, `AKM_PARAMS`,
|
|
1823
|
+
`AKM_INPUTS`, and (in a `map`) `AKM_ITEM` / `AKM_ITEM_INDEX` reach the
|
|
1824
|
+
command with the documented shapes.
|
|
1825
|
+
- [ ] **LOCAL** An oversized `AKM_*` context fails `exec_context_too_large`
|
|
1826
|
+
**before** the spawn, naming the variable, rather than surfacing a raw
|
|
1827
|
+
`E2BIG`.
|
|
1828
|
+
|
|
1829
|
+
`cwd` and isolation:
|
|
1830
|
+
|
|
1831
|
+
- [ ] **LOCAL** `cwd` is relative and contained: absolute paths, drive letters,
|
|
1832
|
+
`~`, and `..` **segments** are rejected by the parser *and* the
|
|
1833
|
+
frozen-plan decoder, and containment is re-checked against the resolved
|
|
1834
|
+
base (symlinks included) immediately before the spawn.
|
|
1835
|
+
- [ ] **LOCAL** A directory whose *name* merely begins with dots (`..data`) is a
|
|
1836
|
+
legal contained `cwd` and must **not** fail `exec_cwd_escape`.
|
|
1837
|
+
- [ ] **LOCAL** Under `isolation: worktree` each unit gets a fresh detached
|
|
1838
|
+
worktree; a clean one is gone once the step resolves, a dirty one is
|
|
1839
|
+
retained and its path logged.
|
|
1840
|
+
- [ ] **LOCAL** When a dirty leftover is moved aside and the worktree then fails
|
|
1841
|
+
to mint, the failure still reports **where the preserved work went**.
|
|
1842
|
+
- [ ] **DESTRUCTIVE** The stale-worktree sweep removes abandoned trees after
|
|
1843
|
+
seven days but leaves a tree that is still in use by a live run.
|
|
1844
|
+
|
|
1845
|
+
Gates, retry, and reuse:
|
|
1846
|
+
|
|
1847
|
+
- [ ] **LOCAL** A `### gate` rubric on an exec step **evaluates** — a rejection
|
|
1848
|
+
still fails the step — but never loops: the command runs exactly **once**
|
|
1849
|
+
regardless of `gate.max_loops`, because a frozen argv cannot answer
|
|
1850
|
+
feedback. Verify with a command that appends to a file.
|
|
1851
|
+
- [ ] **LOCAL** A declared `retry:` re-runs the command only for reasons it
|
|
1852
|
+
names; a schema miss is never re-prompted (re-running could deploy twice).
|
|
1853
|
+
- [ ] **LOCAL** A completed exec unit is reused on resume, never re-executed —
|
|
1854
|
+
confirm with a command whose side effect is observable.
|
|
1855
|
+
- [ ] **LOCAL** `map` fan-out over exec units respects the frozen concurrency
|
|
1856
|
+
width, and each unit sees its own `AKM_ITEM`.
|
|
1857
|
+
|
|
1858
|
+
### 14.6 Failure, interruption, concurrency, and events
|
|
1859
|
+
|
|
1860
|
+
- [ ] **LOCAL** Execution nonzero returns workflow stdout result, exit `1`.
|
|
1861
|
+
- [ ] **LOCAL** Whole-run timeout leaves current step resumable and records evidence.
|
|
1862
|
+
- [ ] **LOCAL** A run that **completes** after its `--timeout` deadline landed is
|
|
1863
|
+
reported as completed, not as timed out.
|
|
1864
|
+
- [ ] **LOCAL** SIGINT/SIGTERM map to `130`/`143`, release claims, leave recoverable run.
|
|
1865
|
+
- [ ] **LOCAL** Concurrent same-ref/scope starts create one active run; second driver loses lease safely.
|
|
1866
|
+
- [ ] **LOCAL** Map fan-out respects workflow and frozen engine concurrency caps.
|
|
1867
|
+
- [ ] **LOCAL** Log by run correlates start/unit/gate/step/finish only.
|
|
1868
|
+
- [ ] **DESTRUCTIVE** Kill before unit, after unit, after judge; resume reclaims only unfinished work, no duplicate unit.
|
|
1869
|
+
- [ ] **DESTRUCTIVE** Run focused lease/crash/contention/publication suites.
|
|
1870
|
+
|
|
1871
|
+
### 14.7 Step artifacts, bounds, and evidence
|
|
1872
|
+
|
|
1873
|
+
A step's promoted artifact is bounded for *persistence* only. The distinction
|
|
1874
|
+
between what the running invocation sees and what the row keeps is the point.
|
|
1875
|
+
|
|
1876
|
+
- [ ] **LOCAL** A step promoting an artifact larger than the 1 MiB persistence
|
|
1877
|
+
cap still hands the **complete** value to the next step of the same run —
|
|
1878
|
+
both a whole-value `inputs:` reference and a path reference into it.
|
|
1879
|
+
- [ ] **LOCAL** The persisted row for that step carries a truncation marker, and
|
|
1880
|
+
**resuming** into the truncated artifact fails loudly, naming truncation
|
|
1881
|
+
as the cause rather than reporting a missing property.
|
|
1882
|
+
- [ ] **LOCAL** Over-cap evidence sacrifices the byte-largest values first, and
|
|
1883
|
+
the persisted row is never left over cap.
|
|
1884
|
+
|
|
1885
|
+
### 14.8 Retired surfaces and fallback
|
|
1886
|
+
|
|
1887
|
+
- [ ] **CORE** Start/next/complete/brief/report fail `UNKNOWN_COMMAND` with run/status guidance.
|
|
1888
|
+
- [ ] **LOCAL** No template/validate/watch; replacements are create-print/lint/log.
|
|
1889
|
+
- [ ] **AI** No defaults engine plus actual opencode binary enables announced SDK fallback; SDK package alone does not.
|
|
1890
|
+
- [ ] **AI** Binary absent fails `78`, names both remedies; no arbitrary first-engine fallback.
|
|
1891
|
+
- [ ] **LOCAL** Secret-shaped param names warn because params persist; use dummy values only.
|
|
1892
|
+
|
|
1893
|
+
---
|
|
1894
|
+
|
|
1895
|
+
## 15. Task
|
|
1896
|
+
|
|
1897
|
+
### 15.1 Surface and schema
|
|
1898
|
+
|
|
1899
|
+
The current group has only `add`, `run`, `history`, `sync`, and `doctor`.
|
|
1900
|
+
Inspection uses generic search/show. Enable/disable is YAML edit plus sync;
|
|
1901
|
+
removal is file deletion plus sync.
|
|
1902
|
+
|
|
1903
|
+
- [ ] **CORE** Task help shows those five; bare task exits `2`; doctor is diagnostic.
|
|
1904
|
+
- [ ] **CORE** Search `tasks/` and show `tasks/manual-success` expose fixtures.
|
|
1905
|
+
- [ ] **LOCAL** Task list/show/remove/enable/disable and plural top-level tasks fail with guidance.
|
|
1906
|
+
- [ ] **LOCAL** Version `2` and schedule are required; enabled default correct; exactly one target.
|
|
1907
|
+
- [ ] **LOCAL** Command arrays preserve argv. String commands whitespace-split without shell quote semantics.
|
|
1908
|
+
- [ ] **LOCAL** Old Markdown task is not indexed; YAML is canonical; legacy same-id blocks add unless force.
|
|
1909
|
+
|
|
1910
|
+
### 15.2 Doctor and manual execution without scheduler mutation
|
|
1911
|
+
|
|
1912
|
+
```sh
|
|
1913
|
+
akm task doctor --format json > "$AKM_SANDBOX/task-doctor.json"
|
|
1914
|
+
akm task run manual-success --format json > "$AKM_SANDBOX/task-success.json"
|
|
1915
|
+
|
|
1916
|
+
set +e
|
|
1917
|
+
akm task run manual-failure --format json > "$AKM_SANDBOX/task-failure.json"
|
|
1918
|
+
task_failure_status=$?
|
|
1919
|
+
set -e
|
|
1920
|
+
|
|
1921
|
+
test "$task_failure_status" -eq 1
|
|
1922
|
+
jq -e '.result.detail.exitCode == 7' "$AKM_SANDBOX/task-failure.json"
|
|
1923
|
+
akm task history --id manual-failure --limit 1
|
|
1924
|
+
```
|
|
1925
|
+
|
|
1926
|
+
- [ ] **CORE** Doctor reports backend/executable/log dir/schedule subset without scheduler mutation.
|
|
1927
|
+
- [ ] **CORE** Manual run executes disabled success task; scheduled marker skips disabled task.
|
|
1928
|
+
- [ ] **LOCAL** Failure is stdout result/CLI `1`, with child `7` retained in detail/history.
|
|
1929
|
+
- [ ] **LOCAL** Command child `78` maps CLI `78`; other failed status maps `1`.
|
|
1930
|
+
- [ ] **LOCAL** Missing/malformed task, invalid id, prompt/workflow failure, timeout, and signal record bounded history.
|
|
1931
|
+
- [ ] **LOCAL** Flat log and logs DB agree and redact credentials.
|
|
1932
|
+
- [ ] **LOCAL** A **workflow** task that declares no `timeoutMs:` gets the
|
|
1933
|
+
6-hour unattended default: the log records `timed_out=true` with the
|
|
1934
|
+
applied `timeout_ms`, the attempt is `failed` so the scheduler sees a
|
|
1935
|
+
non-zero exit, and the aborted run is left **resumable** with the error
|
|
1936
|
+
naming `akm workflow resume <id>`.
|
|
1937
|
+
- [ ] **LOCAL** `timeoutMs: null` opts a workflow task out entirely and arms no
|
|
1938
|
+
timer; an explicit number overrides the default.
|
|
1939
|
+
- [ ] **LOCAL** A workflow task whose run **completes** as the deadline lands is
|
|
1940
|
+
recorded `completed` with no timeout error and no resume hint.
|
|
1941
|
+
|
|
1942
|
+
### 15.3 Scheduler binding and native lifecycle
|
|
1943
|
+
|
|
1944
|
+
- [ ] **LOCAL** Source/dist scheduler write without rebind rejects ineligible executable before task/scheduler mutation.
|
|
1945
|
+
- [ ] **LOCAL** Explicit rebind records reviewed invocation and warns not release-eligible.
|
|
1946
|
+
- [ ] **PLATFORM** Use installed npm/standalone candidate in disposable account.
|
|
1947
|
+
- [ ] **PLATFORM** Add a disabled command/prompt/workflow; confirm no native entry exists; use `akm task enable` and inspect the native entry; edit schedule/target + sync; execute; inspect history/log; use `akm task disable` and confirm removal; delete YAML + sync remains converged.
|
|
1948
|
+
- [ ] **PLATFORM** Existing binding remains unless rebind; upgrade behavior is explicit.
|
|
1949
|
+
|
|
1950
|
+
| Platform | Evidence |
|
|
1951
|
+
| --- | --- |
|
|
1952
|
+
| Linux cron | Before/after crontab, generated line, execution log, history, removal |
|
|
1953
|
+
| macOS launchd | Plist, launchctl state, execution log, history, unload/removal |
|
|
1954
|
+
| Windows schtasks | XML, scheduler query, execution result, history, deletion |
|
|
1955
|
+
|
|
1956
|
+
```sh
|
|
1957
|
+
AKM_NATIVE_SCHEDULER_TESTS=1 bun test tests/integration/native-scheduler.test.ts
|
|
1958
|
+
```
|
|
1959
|
+
|
|
1960
|
+
- [ ] **PLATFORM** Native test actually runs rather than skips and all changes are removed from the disposable account.
|
|
1961
|
+
|
|
1962
|
+
---
|
|
1963
|
+
|
|
1964
|
+
## 16. Proposal Queue
|
|
1965
|
+
|
|
1966
|
+
### 16.1 Seed, list, show, diff, and resolution
|
|
1967
|
+
|
|
1968
|
+
Run the deterministic seeder from section 4.6.
|
|
1969
|
+
|
|
1970
|
+
```sh
|
|
1971
|
+
update_id="$(jq -r .update.id "$AKM_SANDBOX/proposals.json")"
|
|
1972
|
+
new_id="$(jq -r .newAsset.id "$AKM_SANDBOX/proposals.json")"
|
|
1973
|
+
|
|
1974
|
+
akm proposal list --status pending --format json
|
|
1975
|
+
akm proposal show "$update_id" --format json
|
|
1976
|
+
akm proposal show 11111111 --format json
|
|
1977
|
+
akm proposal show memories/qa-proposal-update --format json
|
|
1978
|
+
akm proposal diff "$update_id" --format text
|
|
1979
|
+
```
|
|
1980
|
+
|
|
1981
|
+
- [ ] **LOCAL** List filters queue/status/ref/type without mutation; bare proposal is a usage error.
|
|
1982
|
+
- [ ] **LOCAL** Pending proposal resolves by full UUID, unique prefix, and canonical asset ref.
|
|
1983
|
+
- [ ] **LOCAL** Ambiguous prefix `33333333` fails with an ambiguity error and candidate guidance.
|
|
1984
|
+
- [ ] **LOCAL** Missing proposal is `PROPOSAL_NOT_FOUND`, exit `1`.
|
|
1985
|
+
- [ ] **LOCAL** Show includes payload, provenance, target, validation, status, and no credential values.
|
|
1986
|
+
- [ ] **LOCAL** Diff distinguishes create/update/empty diff and renders all supported formats.
|
|
1987
|
+
- [ ] **LOCAL** Queue/target selectors that conflict with recorded target fail before mutation.
|
|
1988
|
+
|
|
1989
|
+
### 16.2 Accept and revert exact bytes
|
|
1990
|
+
|
|
1991
|
+
```sh
|
|
1992
|
+
cp "$AKM_BUNDLE_DIR/memories/qa-proposal-update.md" \
|
|
1993
|
+
"$AKM_SANDBOX/qa-proposal-update.before"
|
|
1994
|
+
|
|
1995
|
+
akm proposal accept "$update_id" --format json
|
|
1996
|
+
grep -q 'qa-proposal-after' "$AKM_BUNDLE_DIR/memories/qa-proposal-update.md"
|
|
1997
|
+
|
|
1998
|
+
akm proposal revert "$update_id" --format json
|
|
1999
|
+
cmp "$AKM_SANDBOX/qa-proposal-update.before" \
|
|
2000
|
+
"$AKM_BUNDLE_DIR/memories/qa-proposal-update.md"
|
|
2001
|
+
```
|
|
2002
|
+
|
|
2003
|
+
- [ ] **LOCAL** Accept validates/publishes/indexes one proposal and archives it accepted with backup metadata.
|
|
2004
|
+
- [ ] **LOCAL** Update revert accepts full archived UUID or asset ref, restores exact prior bytes, reindexes, and marks reverted.
|
|
2005
|
+
- [ ] **LOCAL** Archived UUID prefixes are not supported by revert; use full UUID.
|
|
2006
|
+
- [ ] **LOCAL** Accepting new asset succeeds, but reverting it fails with usage exit `2` because no prior-content backup exists.
|
|
2007
|
+
- [ ] **LOCAL** Reverting pending/rejected/reverted/missing proposal has the correct usage/not-found classification and changes no bytes.
|
|
2008
|
+
- [ ] **LOCAL** Stale before-hash, read-only/disabled target, lint blocker, adapter precommit failure, git commit/push failure, and publication interruption preserve coherent queue/asset state.
|
|
2009
|
+
|
|
2010
|
+
### 16.3 Reject and bulk dry-runs
|
|
2011
|
+
|
|
2012
|
+
```sh
|
|
2013
|
+
akm proposal reject "$new_id" --reason "manual QA" --yes
|
|
2014
|
+
akm proposal accept --generator reflect --dry-run --format json
|
|
2015
|
+
akm proposal reject --generator distill \
|
|
2016
|
+
--reason "manual QA dry-run" --dry-run --format json
|
|
2017
|
+
```
|
|
2018
|
+
|
|
2019
|
+
- [ ] **LOCAL** Single reject requires nonempty reason and `--yes` in non-TTY mode; it archives one row and preserves payload/audit.
|
|
2020
|
+
- [ ] **LOCAL** Bulk dry-run needs no prompt, reports matching proposals, and leaves statuses/assets unchanged.
|
|
2021
|
+
- [ ] **LOCAL** Max-diff-lines and older-than strictly validate integers/ranges and select exact rows.
|
|
2022
|
+
- [ ] **LOCAL** Dry-run does not create backups, commits, or asset/index mutations.
|
|
2023
|
+
|
|
2024
|
+
### 16.4 Drain and observability
|
|
2025
|
+
|
|
2026
|
+
```sh
|
|
2027
|
+
before_event_id="$(akm log --format json | jq -r '.events[-1].id // 0')"
|
|
2028
|
+
akm proposal drain --dry-run --format json \
|
|
2029
|
+
> "$AKM_SANDBOX/drain-dry-run.json"
|
|
2030
|
+
after_event_id="$(akm log --format json | jq -r '.events[-1].id // 0')"
|
|
2031
|
+
```
|
|
2032
|
+
|
|
2033
|
+
- [ ] **LOCAL** Without a judgment tier, drain accepts only proposals a quality judge passed (a `staged` gate decision whose content hash still matches), rejects empty diffs, and defers everything else; it never accepts an unjudged proposal by itself.
|
|
2034
|
+
- [ ] **LOCAL** Dry-run leaves assets/statuses unchanged but appends the documented `triage_drained` observability event. It is not globally side-effect-free.
|
|
2035
|
+
- [ ] **LOCAL** Max accepts, older-than, queue mode versus `--promote`, and hard cap buckets are exact.
|
|
2036
|
+
- [ ] **AI** Judgment-enabled drain uses selected frozen engine and fails closed on malformed/failed judgment.
|
|
2037
|
+
- [ ] **LOCAL** Noninteractive promotion without `--yes` fails before writes.
|
|
2038
|
+
|
|
2039
|
+
### 16.5 Proposal generation and crash recovery
|
|
2040
|
+
|
|
2041
|
+
- [ ] **AI** `proposal new` with the fake/controlled agent either queues one valid proposal with provenance or returns a structured engine failure; use one tiny asset/task.
|
|
2042
|
+
- [ ] **AI** `proposal extract --type <harness>` and `--auto` use disposable session fixtures, bounded since/limit, and never ingest real session secrets.
|
|
2043
|
+
- [ ] **LOCAL** Generated proposal source/sourceRun/model/target attribution is complete and queryable.
|
|
2044
|
+
- [ ] **DESTRUCTIVE** Kill accept/reject/revert at each durable transaction phase; retry converges with one archive status and old/new complete bytes.
|
|
2045
|
+
- [ ] **LOCAL** Run `tests/integration/proposal-durable-recovery.test.ts`, proposal storage/target-binding suites, and proposal stuck-repair before release changes to queue transactions.
|
|
2046
|
+
|
|
2047
|
+
---
|
|
2048
|
+
|
|
2049
|
+
## 17. Agent, LLM, and Improve
|
|
2050
|
+
|
|
2051
|
+
### 17.1 Fake agent success, failure, capture, timeout, and redaction
|
|
2052
|
+
|
|
2053
|
+
Configure fake engines from section 4.4.
|
|
2054
|
+
|
|
2055
|
+
```sh
|
|
2056
|
+
akm agent agents/qa-reviewer \
|
|
2057
|
+
--engine qa-agent --prompt "Return the fixture marker" --format json \
|
|
2058
|
+
> "$AKM_SANDBOX/agent-success.json"
|
|
2059
|
+
jq -e '.ok == true and .shape == "agent-result" and (.stdout | contains("qa-agent-success"))' \
|
|
2060
|
+
"$AKM_SANDBOX/agent-success.json"
|
|
2061
|
+
|
|
2062
|
+
akm agent --engine qa-agent-capture \
|
|
2063
|
+
--prompt "capture prompt" --cwd "$AKM_SANDBOX" --format json \
|
|
2064
|
+
> "$AKM_SANDBOX/agent-capture.json"
|
|
2065
|
+
|
|
2066
|
+
set +e
|
|
2067
|
+
akm agent --engine qa-agent-fail --prompt "fail" --format json \
|
|
2068
|
+
> "$AKM_SANDBOX/agent-failure.json"
|
|
2069
|
+
agent_failure_status=$?
|
|
2070
|
+
set -e
|
|
2071
|
+
test "$agent_failure_status" -eq 1
|
|
2072
|
+
jq -e '.ok == false and .exitCode == 7' "$AKM_SANDBOX/agent-failure.json"
|
|
2073
|
+
```
|
|
2074
|
+
|
|
2075
|
+
- [ ] **AI** Agent asset contributes system prompt/model/tool policy; CLI model override wins.
|
|
2076
|
+
- [ ] **AI** Prompt, prompt-stdin, cwd, and interactive no-prompt paths are each covered; stored commands and workflows use their canonical commands.
|
|
2077
|
+
- [ ] **AI** Capture proves exact argv/prompt/cwd and credential-presence boolean without logging credential value.
|
|
2078
|
+
- [ ] **AI** Nonzero child returns `agent-result` on stdout, CLI exit `1`, child code/reason/stderr retained and redacted.
|
|
2079
|
+
- [ ] **AI** Missing executable is a bounded spawn failure; timeout kills process group and writes the fake signal marker with no orphan.
|
|
2080
|
+
- [ ] **AI** `OPENCODE_API_KEY=manual-qa-credential-value` plus echo-secret mode produces no literal value in AKM result, stderr, event, or logs.
|
|
2081
|
+
- [ ] **AI REGRESSION** Prompt-stdin remains mutually exclusive with `--prompt`; removed aliases such as agent `--command` are absent from help.
|
|
2082
|
+
- [ ] **AI** Named LLM engine is rejected where agent-only dispatch is required; invalid platform/kind fails before spawn.
|
|
2083
|
+
- [ ] **AI** Default-engine fallback is announced exactly once when used and never rescues an explicitly invalid named engine.
|
|
2084
|
+
|
|
2085
|
+
### 17.2 Fake OpenAI-compatible service
|
|
2086
|
+
|
|
2087
|
+
Start/configure sections 4.3 and 4.5.
|
|
2088
|
+
|
|
2089
|
+
```sh
|
|
2090
|
+
: >"$AKM_QA_SERVICE_LOG"
|
|
2091
|
+
|
|
2092
|
+
akm remember "Observed manual QA on 2026-08-05" \
|
|
2093
|
+
--name qa-enriched --tag manual-qa-seed --enrich --format json \
|
|
2094
|
+
> "$AKM_SANDBOX/enriched.json"
|
|
2095
|
+
|
|
2096
|
+
grep -q 'manual-qa' "$AKM_BUNDLE_DIR/memories/qa-enriched.md"
|
|
2097
|
+
|
|
2098
|
+
akm config set engines.qa-chat.endpoint "$AKM_QA_PROPOSAL_URL" --silent
|
|
2099
|
+
akm proposal new lesson qa-llm-proposal \
|
|
2100
|
+
--task "Create the controlled manual QA lesson" \
|
|
2101
|
+
--engine qa-chat --format json >"$AKM_SANDBOX/llm-proposal-default.json"
|
|
2102
|
+
jq -e '.ok == true and (.ref | endswith("//lessons/qa-llm-proposal"))' \
|
|
2103
|
+
"$AKM_SANDBOX/llm-proposal-default.json"
|
|
2104
|
+
|
|
2105
|
+
akm config set engines.qa-chat.maxTokens 32 --silent
|
|
2106
|
+
akm proposal new lesson qa-llm-proposal \
|
|
2107
|
+
--task "Repeat the controlled manual QA lesson with an engine cap" \
|
|
2108
|
+
--engine qa-chat --format json >"$AKM_SANDBOX/llm-proposal-capped.json"
|
|
2109
|
+
akm config unset engines.qa-chat.maxTokens --silent
|
|
2110
|
+
akm config set engines.qa-chat.endpoint "$AKM_QA_CHAT_URL" --silent
|
|
2111
|
+
|
|
2112
|
+
jq -s -e '
|
|
2113
|
+
([.[] | select(.pathname == "/ok/chat/completions")]) as $enrich |
|
|
2114
|
+
([.[] | select(.pathname == "/proposal/chat/completions")]) as $proposal |
|
|
2115
|
+
($enrich | length) == 1 and
|
|
2116
|
+
$enrich[0].authorizationPresent == true and
|
|
2117
|
+
$enrich[0].maxTokensPresent == true and
|
|
2118
|
+
($proposal | length) == 2 and
|
|
2119
|
+
all($proposal[]; .authorizationPresent == true) and
|
|
2120
|
+
$proposal[0].maxTokensPresent == false and
|
|
2121
|
+
$proposal[1].maxTokensPresent == true and
|
|
2122
|
+
$enrich[0].responseFormatPresent == false and
|
|
2123
|
+
all($proposal[]; .responseFormatPresent == true)
|
|
2124
|
+
' \
|
|
2125
|
+
"$AKM_QA_SERVICE_LOG"
|
|
2126
|
+
```
|
|
2127
|
+
|
|
2128
|
+
- [ ] **AI** The seed tag forces the structured remember path; valid response enriches tags/description/date with one bounded request.
|
|
2129
|
+
- [ ] **AI REGRESSION** `remember --enrich` without another structured argument must still dispatch enrichment. A zero-request raw write is a failure.
|
|
2130
|
+
- [ ] **AI** HTTP 500, invalid envelope, malformed content, connection refused, and slow endpoint produce typed/fail-soft behavior with no stack/hang/partial write.
|
|
2131
|
+
- [ ] **AI** Timeout honors engine `timeoutMs`; no removed feature-gate timeout is used.
|
|
2132
|
+
- [ ] **AI** Explicit symbolic credential is required and authoritative; missing value fails config/dispatch without trying another secret.
|
|
2133
|
+
- [ ] **AI** Echo-auth 500 response cannot leak the Authorization value into error, log, event, or persisted run result.
|
|
2134
|
+
- [ ] **AI** Remember's enrichment call sends its intentional caller cap; default proposal chat omits `max_tokens`; configured engine `maxTokens` sends it.
|
|
2135
|
+
- [ ] **AI** Local endpoint work defaults to one concurrent request on indexing paths; remote defaults remain bounded.
|
|
2136
|
+
- [ ] **AI** Probe route exercises capability setup using tiny fixed responses only.
|
|
2137
|
+
|
|
2138
|
+
Run the standalone enrichment regression separately. It currently fails because
|
|
2139
|
+
the command takes the raw-write path and makes no request:
|
|
2140
|
+
|
|
2141
|
+
```sh
|
|
2142
|
+
before_requests="$(wc -l <"$AKM_QA_SERVICE_LOG")"
|
|
2143
|
+
akm remember "Standalone enrichment dispatch oracle" \
|
|
2144
|
+
--name qa-enrich-only --enrich --format json \
|
|
2145
|
+
>"$AKM_SANDBOX/enrich-only.json"
|
|
2146
|
+
after_requests="$(wc -l <"$AKM_QA_SERVICE_LOG")"
|
|
2147
|
+
|
|
2148
|
+
test "$after_requests" -eq "$((before_requests + 1))"
|
|
2149
|
+
grep -q 'manual-qa' "$AKM_BUNDLE_DIR/memories/qa-enrich-only.md"
|
|
2150
|
+
```
|
|
2151
|
+
|
|
2152
|
+
### 17.3 Improve dry-run invariants
|
|
2153
|
+
|
|
2154
|
+
Dry-run still preflights enabled model-backed processes, so configure the fake
|
|
2155
|
+
LLM first. It must not call the model or mutate any durable surface.
|
|
2156
|
+
|
|
2157
|
+
```sh
|
|
2158
|
+
snapshot_tree "$AKM_SANDBOX" "$AKM_SANDBOX/before-improve.sha256"
|
|
2159
|
+
before_requests="$(wc -l < "$AKM_QA_SERVICE_LOG")"
|
|
2160
|
+
|
|
2161
|
+
akm improve skills/k8s-deploy \
|
|
2162
|
+
--strategy quick --limit 1 --dry-run --no-sync --no-push \
|
|
2163
|
+
--format json > "$AKM_SANDBOX/improve-dry-run.json"
|
|
2164
|
+
|
|
2165
|
+
after_requests="$(wc -l < "$AKM_QA_SERVICE_LOG")"
|
|
2166
|
+
test "$before_requests" -eq "$after_requests"
|
|
2167
|
+
jq -e '
|
|
2168
|
+
.schemaVersion == 2 and
|
|
2169
|
+
.shape == "improve" and
|
|
2170
|
+
.dryRun == true and
|
|
2171
|
+
.strategy == "quick"
|
|
2172
|
+
' "$AKM_SANDBOX/improve-dry-run.json"
|
|
2173
|
+
```
|
|
2174
|
+
|
|
2175
|
+
Do not compare the snapshot file against itself. Produce before/after manifests
|
|
2176
|
+
outside the hashed tree or exclude those evidence files, then compare config,
|
|
2177
|
+
bundle, data, state, and cache independently.
|
|
2178
|
+
|
|
2179
|
+
- [ ] **AI** Dry-run emits schema-v2 improve result, performs zero model requests, creates no proposal/event/run/lock/log/asset/config/index/git change.
|
|
2180
|
+
- [ ] **AI** Scope by type/ref, task guidance, bundle validation, limit, strategy, require-feedback-signal, and format shaping are represented in plan.
|
|
2181
|
+
- [ ] **AI** Retired `--target`, `canary`, and removed auto-accept behavior produce current errors/warnings without poisoned scope.
|
|
2182
|
+
- [ ] **AI** Unknown strategy/engine/incompatible process fails during preflight before logging/locks/mutation.
|
|
2183
|
+
|
|
2184
|
+
### 17.4 Bounded live improve and lock behavior
|
|
2185
|
+
|
|
2186
|
+
Use a controlled test-capable model, one asset, and explicit no-sync/no-push:
|
|
2187
|
+
|
|
2188
|
+
```sh
|
|
2189
|
+
akm improve skills/k8s-deploy \
|
|
2190
|
+
--task "tighten the description" \
|
|
2191
|
+
--strategy quick --limit 1 --no-sync --no-push --json-to-stdout
|
|
2192
|
+
```
|
|
2193
|
+
|
|
2194
|
+
- [ ] **AI** Successful live run persists exactly one `improve_runs` row, emits expected process/proposal events, and queues reviewable changes with provenance.
|
|
2195
|
+
- [ ] **AI** Default live mode emits progress on stderr and no stdout; json-to-stdout returns the same persisted result.
|
|
2196
|
+
- [ ] **AI** Lock contention without skip is a transient error, exit `75` (`IMPROVE_LOCK_HELD`); `--skip-if-locked` exits `0`, reports `skipped.reason:"lock-held"`, and emits no `improve_invoked`.
|
|
2197
|
+
- [ ] **AI** Stale lock is reclaimed with an observable recovery event; active lock is never stolen.
|
|
2198
|
+
- [ ] **AI** Timeout/SIGINT/SIGTERM/SIGHUP persist one terminated run with redacted reason and release locks/children.
|
|
2199
|
+
- [ ] **AI** Default/quick/thorough/reflect-distill/consolidate/catchup/proactive-maintenance strategies enable exactly documented processes. Autonomy-gated mutations remain off without explicit experimental opt-in.
|
|
2200
|
+
- [ ] **AI** No sync/push occurs under explicit flags; git publication is tested separately only against disposable remote.
|
|
2201
|
+
- [ ] **AI** Credential/prompt/session values are absent from improve result, state DB, proposal provenance, health report, event stream, and logs.
|
|
2202
|
+
|
|
2203
|
+
---
|
|
2204
|
+
|
|
2205
|
+
## 18. Semantic Search
|
|
2206
|
+
|
|
2207
|
+
Semantic intent (`off`/`auto`) and runtime readiness are separate. Use a fresh
|
|
2208
|
+
data/cache/state tier for each case because vectors, fingerprints, and blocked
|
|
2209
|
+
status are durable.
|
|
2210
|
+
|
|
2211
|
+
### 18.1 Isolated semantic tiers and state oracle
|
|
2212
|
+
|
|
2213
|
+
```sh
|
|
2214
|
+
export AKM_QA_BASE_DATA_DIR="$AKM_DATA_DIR"
|
|
2215
|
+
export AKM_QA_BASE_CACHE_DIR="$AKM_CACHE_DIR"
|
|
2216
|
+
export AKM_QA_BASE_STATE_DIR="$AKM_STATE_DIR"
|
|
2217
|
+
|
|
2218
|
+
use_semantic_tier() {
|
|
2219
|
+
tier="$1"
|
|
2220
|
+
export AKM_DATA_DIR="$AKM_SANDBOX/semantic/$tier/data"
|
|
2221
|
+
export AKM_CACHE_DIR="$AKM_SANDBOX/semantic/$tier/cache"
|
|
2222
|
+
export AKM_STATE_DIR="$AKM_SANDBOX/semantic/$tier/state"
|
|
2223
|
+
mkdir -p "$AKM_DATA_DIR" "$AKM_CACHE_DIR" "$AKM_STATE_DIR"
|
|
2224
|
+
}
|
|
2225
|
+
|
|
2226
|
+
restore_semantic_dirs() {
|
|
2227
|
+
export AKM_DATA_DIR="$AKM_QA_BASE_DATA_DIR"
|
|
2228
|
+
export AKM_CACHE_DIR="$AKM_QA_BASE_CACHE_DIR"
|
|
2229
|
+
export AKM_STATE_DIR="$AKM_QA_BASE_STATE_DIR"
|
|
2230
|
+
}
|
|
2231
|
+
```
|
|
2232
|
+
|
|
2233
|
+
| Effective state | Meaning | Search behavior |
|
|
2234
|
+
| --- | --- | --- |
|
|
2235
|
+
| `disabled` | Mode is `off` | FTS only, no embedding request |
|
|
2236
|
+
| `pending` | Enabled but not verified/current fingerprint changed | FTS fallback with advisory |
|
|
2237
|
+
| `ready-js` | Complete vectors (scanned by cosine similarity in JavaScript) | Hybrid/vector search |
|
|
2238
|
+
| `blocked` | Recent embedding failure | FTS fallback until retry/expiry |
|
|
2239
|
+
|
|
2240
|
+
- [ ] **[CORE]** With mode off, full index reports disabled, stores zero
|
|
2241
|
+
embeddings, and search remains exact FTS with no model/network request.
|
|
2242
|
+
- [ ] **[LOCAL]** Info, index verification, health advisory, and search event
|
|
2243
|
+
agree on effective state. Config intent remains `auto` after transient
|
|
2244
|
+
failure.
|
|
2245
|
+
- [ ] **[LOCAL]** A blocked status expires to pending after its documented TTL;
|
|
2246
|
+
a fingerprint change also returns to pending without deleting keyword data.
|
|
2247
|
+
|
|
2248
|
+
### 18.2 Deterministic model-free path
|
|
2249
|
+
|
|
2250
|
+
`AKM_EMBED_DETERMINISTIC=1` is a test-only stable hash embedder, not a production
|
|
2251
|
+
model or relevance-quality claim.
|
|
2252
|
+
|
|
2253
|
+
```sh
|
|
2254
|
+
use_semantic_tier deterministic
|
|
2255
|
+
akm config unset embedding --silent
|
|
2256
|
+
akm config set semanticSearchMode auto --silent
|
|
2257
|
+
export AKM_EMBED_DETERMINISTIC=1
|
|
2258
|
+
|
|
2259
|
+
akm index --full --format json \
|
|
2260
|
+
>"$AKM_SANDBOX/semantic-deterministic-index.json"
|
|
2261
|
+
jq -e '
|
|
2262
|
+
.verification.ok == true and
|
|
2263
|
+
.verification.semanticStatus == "ready-js" and
|
|
2264
|
+
.verification.embeddingCount == .verification.entryCount
|
|
2265
|
+
' "$AKM_SANDBOX/semantic-deterministic-index.json"
|
|
2266
|
+
|
|
2267
|
+
akm search "deploy docker compose in a homelab" \
|
|
2268
|
+
--detail full --format json \
|
|
2269
|
+
>"$AKM_SANDBOX/semantic-deterministic-search.json"
|
|
2270
|
+
akm log --type search --limit 1 --detail full --format json \
|
|
2271
|
+
>"$AKM_SANDBOX/semantic-deterministic-log.json"
|
|
2272
|
+
jq -e '.events[-1].metadata.mode == "semantic"' \
|
|
2273
|
+
"$AKM_SANDBOX/semantic-deterministic-log.json"
|
|
2274
|
+
|
|
2275
|
+
unset AKM_EMBED_DETERMINISTIC
|
|
2276
|
+
```
|
|
2277
|
+
|
|
2278
|
+
- [ ] **[LOCAL]** Index/search make no network or model request.
|
|
2279
|
+
- [ ] **[LOCAL]** Every embeddable entry has one vector; repeated full index
|
|
2280
|
+
yields stable vectors and relevant result set.
|
|
2281
|
+
- [ ] **[LOCAL]** Full-detail hit explains hybrid/semantic matching and durable
|
|
2282
|
+
search telemetry records actual semantic mode. Timing alone is not proof.
|
|
2283
|
+
|
|
2284
|
+
### 18.3 Controlled remote embedding service
|
|
2285
|
+
|
|
2286
|
+
Start section 4.3 first. The fixture proves transport, batching, persistence,
|
|
2287
|
+
readiness, and activation, not natural-language relevance.
|
|
2288
|
+
|
|
2289
|
+
```sh
|
|
2290
|
+
use_semantic_tier fake-remote
|
|
2291
|
+
unset AKM_EMBED_DETERMINISTIC AKM_EMBED_API_KEY
|
|
2292
|
+
: >"$AKM_QA_SERVICE_LOG"
|
|
2293
|
+
|
|
2294
|
+
akm config set embedding "$(jq -nc \
|
|
2295
|
+
--arg endpoint "$AKM_QA_EMBED_URL" \
|
|
2296
|
+
'{provider:"openai",endpoint:$endpoint,model:"qa-embedding",dimension:4,batchSize:4}')" \
|
|
2297
|
+
--silent
|
|
2298
|
+
akm config set semanticSearchMode auto --silent
|
|
2299
|
+
|
|
2300
|
+
akm index --full --format json >"$AKM_SANDBOX/semantic-remote-index.json"
|
|
2301
|
+
jq -e '
|
|
2302
|
+
.verification.ok == true and
|
|
2303
|
+
.verification.embeddingProvider == "remote" and
|
|
2304
|
+
.verification.embeddingCount == .verification.entryCount and
|
|
2305
|
+
.verification.semanticStatus == "ready-js"
|
|
2306
|
+
' "$AKM_SANDBOX/semantic-remote-index.json"
|
|
2307
|
+
|
|
2308
|
+
akm search deploy --detail full --format json \
|
|
2309
|
+
>"$AKM_SANDBOX/semantic-remote-search.json"
|
|
2310
|
+
jq -s -e '
|
|
2311
|
+
[.[] | select(.pathname == "/v1/embeddings")] as $requests |
|
|
2312
|
+
($requests | length) > 0 and
|
|
2313
|
+
all($requests[];
|
|
2314
|
+
.model == "qa-embedding" and
|
|
2315
|
+
.authorizationPresent == false and
|
|
2316
|
+
.inputCount >= 1 and .inputCount <= 4)
|
|
2317
|
+
' "$AKM_QA_SERVICE_LOG"
|
|
2318
|
+
```
|
|
2319
|
+
|
|
2320
|
+
- [ ] **[SERVICE]** Batch/model/auth metadata is exact; request log contains no
|
|
2321
|
+
input text or credential value.
|
|
2322
|
+
- [ ] **[SERVICE]** Query creates one additional embedding request and records
|
|
2323
|
+
semantic execution.
|
|
2324
|
+
- [ ] **[SERVICE]** Wrong envelope, wrong vector count/dimension, HTTP failure,
|
|
2325
|
+
slow response, cancellation, and connection refusal are bounded and do not
|
|
2326
|
+
invalidate the FTS index.
|
|
2327
|
+
- [ ] **[SERVICE]** Changing `embedding.dimension` invalidates old vectors and
|
|
2328
|
+
an ordinary index regenerates them. Changing indexed text regenerates
|
|
2329
|
+
exactly that asset's vector.
|
|
2330
|
+
- [ ] **[SERVICE]** Changing `embedding.model` to a string that still resolves
|
|
2331
|
+
to a compatible model (same dimension, near-identical vectors on
|
|
2332
|
+
re-embedding a sample) is verified and KEPT — no purge, the stored
|
|
2333
|
+
vectors survive, and `akm index --format json` reports the same
|
|
2334
|
+
`embeddingCount` as before. `akm index --reembed` bypasses this check
|
|
2335
|
+
and forces a purge + rebuild regardless.
|
|
2336
|
+
- [ ] **[SERVICE REGRESSION]** A provider-returned vector dimension mismatch is
|
|
2337
|
+
rejected rather than stored as ready.
|
|
2338
|
+
|
|
2339
|
+
### 18.4 Blocked indexing and query-time fallback
|
|
2340
|
+
|
|
2341
|
+
```sh
|
|
2342
|
+
use_semantic_tier blocked
|
|
2343
|
+
akm config set embedding "$(jq -nc \
|
|
2344
|
+
--arg endpoint "$AKM_QA_BASE_URL/error/embeddings" \
|
|
2345
|
+
'{provider:"openai",endpoint:$endpoint,model:"qa-failing",dimension:4,batchSize:4}')" \
|
|
2346
|
+
--silent
|
|
2347
|
+
akm config set semanticSearchMode auto --silent
|
|
2348
|
+
|
|
2349
|
+
set +e
|
|
2350
|
+
akm index --full --format json \
|
|
2351
|
+
>"$AKM_SANDBOX/semantic-blocked-index.json" \
|
|
2352
|
+
2>"$AKM_SANDBOX/semantic-blocked-index.stderr"
|
|
2353
|
+
blocked_index_status=$?
|
|
2354
|
+
set -e
|
|
2355
|
+
|
|
2356
|
+
test "$blocked_index_status" -eq 0
|
|
2357
|
+
jq -e '
|
|
2358
|
+
.verification.ok == false and
|
|
2359
|
+
.verification.semanticStatus == "blocked"
|
|
2360
|
+
' "$AKM_SANDBOX/semantic-blocked-index.json"
|
|
2361
|
+
|
|
2362
|
+
akm search deploy --detail full --format json \
|
|
2363
|
+
>"$AKM_SANDBOX/semantic-blocked-search.json"
|
|
2364
|
+
jq -e '(.hits | length) > 0 and (.warnings | length) > 0' \
|
|
2365
|
+
"$AKM_SANDBOX/semantic-blocked-search.json"
|
|
2366
|
+
```
|
|
2367
|
+
|
|
2368
|
+
- [ ] **[SERVICE]** Index exits `0` because keyword indexing succeeded, but
|
|
2369
|
+
semantic verification is explicitly false/blocked and warns.
|
|
2370
|
+
- [ ] **[SERVICE]** Search exits `0`, returns FTS hits, and records keyword mode.
|
|
2371
|
+
- [ ] **[SERVICE]** If hard health checks pass, blocked semantic readiness is a
|
|
2372
|
+
health warning on stdout with exit `4`, not a hard failure.
|
|
2373
|
+
- [ ] **[SERVICE REGRESSION]** Query-time provider failure updates durable
|
|
2374
|
+
readiness and structured warnings, not stderr only while status stays ready.
|
|
2375
|
+
- [ ] **[LOCAL REGRESSION]** Targeted writes that skip embeddings cannot leave a
|
|
2376
|
+
misleading complete/ready count; next index converges deterministically.
|
|
2377
|
+
|
|
2378
|
+
### 18.5 Real local and bounded live gates
|
|
2379
|
+
|
|
2380
|
+
```sh
|
|
2381
|
+
use_semantic_tier local-model
|
|
2382
|
+
unset AKM_EMBED_DETERMINISTIC AKM_EMBED_API_KEY
|
|
2383
|
+
akm config unset embedding --silent
|
|
2384
|
+
akm config set semanticSearchMode auto --silent
|
|
2385
|
+
akm index --full --verbose --format json \
|
|
2386
|
+
>"$AKM_SANDBOX/semantic-local-index.json"
|
|
2387
|
+
```
|
|
2388
|
+
|
|
2389
|
+
- [ ] **[AI]** Package install loads the default local model, keeps all model
|
|
2390
|
+
files under isolated `HF_HOME`, reaches ready, and passes a paraphrase
|
|
2391
|
+
query through an actual CLI subprocess.
|
|
2392
|
+
- [ ] **[AI]** Record cold download and warm offline reuse separately. A model
|
|
2393
|
+
download failure blocks semantic only and leaves FTS usable.
|
|
2394
|
+
- [ ] **[AI]** Optional remote provider uses a non-production symbolic key,
|
|
2395
|
+
exact model/dimension, batch at most four, bounded spend, and redaction.
|
|
2396
|
+
- [ ] **[PLATFORM]** Standalone release binary cannot load the externalized local
|
|
2397
|
+
transformer dependency; use remote embeddings or mark local gate `N/A`.
|
|
2398
|
+
|
|
2399
|
+
Automated real-model gate:
|
|
2400
|
+
|
|
2401
|
+
```sh
|
|
2402
|
+
AKM_SEMANTIC_TESTS=1 \
|
|
2403
|
+
bun test --timeout=120000 tests/integration/semantic-search-e2e.test.ts
|
|
2404
|
+
```
|
|
2405
|
+
|
|
2406
|
+
Restore the primary tier after semantic checks:
|
|
2407
|
+
|
|
2408
|
+
```sh
|
|
2409
|
+
restore_semantic_dirs
|
|
2410
|
+
unset AKM_EMBED_DETERMINISTIC
|
|
2411
|
+
akm config set semanticSearchMode off --silent
|
|
2412
|
+
akm config unset embedding --silent
|
|
2413
|
+
```
|
|
2414
|
+
|
|
2415
|
+
---
|
|
2416
|
+
|
|
2417
|
+
## 19. Migration, Durability, and Concurrency
|
|
2418
|
+
|
|
2419
|
+
### 19.1 Task migration boundary
|
|
2420
|
+
|
|
2421
|
+
`akm migrate` has exactly one responsibility: explicit task migration, run as
|
|
2422
|
+
two generations in one pass — task-v2 to task-v3 source conversion, then
|
|
2423
|
+
task-v3 to task source v4 conversion against the resulting files. It never
|
|
2424
|
+
rewrites config or databases.
|
|
2425
|
+
|
|
2426
|
+
| Operation | Classification |
|
|
2427
|
+
| --- | --- |
|
|
2428
|
+
| `akm migrate status` | Read-only task inventory, both generations |
|
|
2429
|
+
| `akm migrate apply --dry-run` | Read-only validated conversion plan, both generations |
|
|
2430
|
+
| `akm migrate apply` | Per-file backup plus atomic task-source replacement, both generations |
|
|
2431
|
+
|
|
2432
|
+
- [ ] **[LOCAL]** Status and dry-run report the same generation for each pass
|
|
2433
|
+
and change no source, config, database, lock, scheduler, event, or usage
|
|
2434
|
+
row.
|
|
2435
|
+
- [ ] **[LOCAL]** Apply validates complete v3 bytes before replacement,
|
|
2436
|
+
preserves mode, and creates the backup immediately before the write; the
|
|
2437
|
+
v3-to-v4 generation then runs the same way against the resulting files.
|
|
2438
|
+
- [ ] **[LOCAL]** A changed generation, ambiguous argv array, unwritable source,
|
|
2439
|
+
invalid YAML, or unsafe shell translation is blocked with original bytes
|
|
2440
|
+
intact.
|
|
2441
|
+
- [ ] **[LOCAL]** Normal run/sync/doctor rejects task v2 and task v3 and never
|
|
2442
|
+
invokes the migrator as a side effect.
|
|
2443
|
+
|
|
2444
|
+
### 19.2 Automatic current database upgrades
|
|
2445
|
+
|
|
2446
|
+
Managed `state.db` opens validate the exact migration-ledger prefix and apply
|
|
2447
|
+
known pending additive migrations automatically. Unknown or reordered ledger
|
|
2448
|
+
entries fail closed. There is no external storage migration command.
|
|
2449
|
+
|
|
2450
|
+
- [ ] **[LOCAL]** A fresh database applies the complete current registry.
|
|
2451
|
+
- [ ] **[LOCAL]** A database with an exact older prefix advances on open without
|
|
2452
|
+
dropping or rewriting durable rows.
|
|
2453
|
+
- [ ] **[LOCAL]** A newer/unknown or inconsistent ledger is rejected before a
|
|
2454
|
+
write.
|
|
2455
|
+
- [ ] **[LOCAL]** Concurrent first opens serialize each migration exactly once.
|
|
2456
|
+
- [ ] **[LOCAL]** `index.db` remains regenerable; `state.db` is never deleted as
|
|
2457
|
+
a generic recovery step.
|
|
2458
|
+
|
|
2459
|
+
### 19.3 Package upgrade boundary
|
|
2460
|
+
|
|
2461
|
+
`akm upgrade` updates executable code only. A 0.8 installation moves its old
|
|
2462
|
+
config/state aside, creates current config/state, and selectively brings
|
|
2463
|
+
authored assets forward. The explicit task migrator can then convert task-v2
|
|
2464
|
+
sources through to task source v4. No current runtime loads old
|
|
2465
|
+
config/storage layouts.
|
|
2466
|
+
|
|
2467
|
+
- [ ] **[LIVE DISPOSABLE]** `akm upgrade --check` is nonmutating and the chosen
|
|
2468
|
+
npm/Bun/pnpm or standalone path installs the expected version.
|
|
2469
|
+
- [ ] **[LIVE DISPOSABLE]** A standalone replacement verifies its checksum and
|
|
2470
|
+
preflights the staged binary before atomic replacement.
|
|
2471
|
+
- [ ] **[LIVE DISPOSABLE]** Old and current data sets stay separate; rollback
|
|
2472
|
+
restores executable code together with its matching archived data.
|
|
2473
|
+
|
|
2474
|
+
### 19.4 Concurrent writers and readers
|
|
2475
|
+
|
|
2476
|
+
```sh
|
|
2477
|
+
akm config set semanticSearchMode off --silent
|
|
2478
|
+
|
|
2479
|
+
set +e
|
|
2480
|
+
akm index --full --format json \
|
|
2481
|
+
>"$AKM_SANDBOX/index-race-1.json" 2>"$AKM_SANDBOX/index-race-1.stderr" &
|
|
2482
|
+
index_pid_1=$!
|
|
2483
|
+
akm index --full --format json \
|
|
2484
|
+
>"$AKM_SANDBOX/index-race-2.json" 2>"$AKM_SANDBOX/index-race-2.stderr" &
|
|
2485
|
+
index_pid_2=$!
|
|
2486
|
+
wait "$index_pid_1"; index_status_1=$?
|
|
2487
|
+
wait "$index_pid_2"; index_status_2=$?
|
|
2488
|
+
set -e
|
|
2489
|
+
|
|
2490
|
+
test "$index_status_1" -eq 0
|
|
2491
|
+
test "$index_status_2" -eq 0
|
|
2492
|
+
! grep -qi 'database is locked' "$AKM_SANDBOX"/index-race-*.stderr
|
|
2493
|
+
test ! -e "$AKM_DATA_DIR/index.db.write.lock"
|
|
2494
|
+
|
|
2495
|
+
akm config set search.graphBoost.directBoostPerEntity 0.11 --silent &
|
|
2496
|
+
config_pid_1=$!
|
|
2497
|
+
akm config set search.graphBoost.directBoostCap 1.25 --silent &
|
|
2498
|
+
config_pid_2=$!
|
|
2499
|
+
wait "$config_pid_1"
|
|
2500
|
+
wait "$config_pid_2"
|
|
2501
|
+
jq -e '
|
|
2502
|
+
.search.graphBoost.directBoostPerEntity == 0.11 and
|
|
2503
|
+
.search.graphBoost.directBoostCap == 1.25
|
|
2504
|
+
' "$AKM_CONFIG_DIR/config.json"
|
|
2505
|
+
```
|
|
2506
|
+
|
|
2507
|
+
- [ ] **[LOCAL]** Index contender waits/serializes; both complete, verification
|
|
2508
|
+
passes, no SQLite lock error or stale writer lease remains.
|
|
2509
|
+
- [ ] **[LOCAL]** Concurrent independent config writes both survive or one fails
|
|
2510
|
+
cleanly after the bounded lock wait; config remains complete JSON.
|
|
2511
|
+
- [ ] **[LOCAL]** Same-name create races produce one asset/one already-exists;
|
|
2512
|
+
different names both survive. Repeat for remember/import/env/secret.
|
|
2513
|
+
- [ ] **[LOCAL]** Reader during full index sees old or finalized new generation,
|
|
2514
|
+
never an empty/intermediate FTS generation.
|
|
2515
|
+
- [ ] **[LOCAL REGRESSION]** A live writer is never reclaimed solely because a
|
|
2516
|
+
wall-clock stale threshold elapsed.
|
|
2517
|
+
- [ ] **[DESTRUCTIVE]** Sync cannot commit an intermediate multi-file write;
|
|
2518
|
+
source update/website/npm refresh are old-or-new complete after kill.
|
|
2519
|
+
|
|
2520
|
+
### 19.5 Database and artifact recovery matrix
|
|
2521
|
+
|
|
2522
|
+
| Artifact | Recovery rule |
|
|
2523
|
+
| --- | --- |
|
|
2524
|
+
| `config.json` | Corruption exits `78`; preserve bytes and restore manually from verified backup |
|
|
2525
|
+
| `akm.lock` | Preserve corrupt bytes; mutation fails before config/cache/index changes |
|
|
2526
|
+
| `index.db` | Regenerable after evidence capture: remove/quarantine, then full index |
|
|
2527
|
+
| `state.db` | Non-regenerable; never delete as repair; restore verified backup |
|
|
2528
|
+
| transaction journal | Stop writers, retain evidence, use domain recovery |
|
|
2529
|
+
| Git source cache | Staged swap for read-only sources; writable checkout policy is explicit |
|
|
2530
|
+
| npm/website cache | Must preserve prior complete generation or report a known failing gate |
|
|
2531
|
+
|
|
2532
|
+
- [ ] **[LOCAL]** WAL is default; explicit DELETE/TRUNCATE works; detected network
|
|
2533
|
+
filesystems fall back safely; managed connections use bounded busy timeout.
|
|
2534
|
+
- [ ] **[LOCAL]** State writes use immediate transactions and nested same-
|
|
2535
|
+
connection calls join the outer transaction.
|
|
2536
|
+
- [ ] **[LOCAL]** Atomic write failure yields complete old or new bytes. A parent
|
|
2537
|
+
directory fsync failure after rename may report failure with new bytes;
|
|
2538
|
+
record that ambiguity rather than claiming rollback.
|
|
2539
|
+
- [ ] **[LOCAL]** Full rebuild preserves state/events/proposals/workflow history;
|
|
2540
|
+
only derived index data is disposable.
|
|
2541
|
+
|
|
2542
|
+
### 19.6 Focused automated gates
|
|
2543
|
+
|
|
2544
|
+
```sh
|
|
2545
|
+
bun test --timeout=120000 \
|
|
2546
|
+
tests/integration/migrate-format.test.ts \
|
|
2547
|
+
tests/migrate/task-v2-to-v3-files.test.ts \
|
|
2548
|
+
tests/tasks/migrate-v2-to-v3.test.ts \
|
|
2549
|
+
tests/integration/config-recovery-concurrency.test.ts \
|
|
2550
|
+
tests/integration/file-lock.test.ts \
|
|
2551
|
+
tests/integration/index-writer-lock.test.ts \
|
|
2552
|
+
tests/integration/index-writer-lock-crossproc.test.ts \
|
|
2553
|
+
tests/integration/proposal-durable-recovery.test.ts
|
|
2554
|
+
```
|
|
2555
|
+
|
|
2556
|
+
---
|
|
2557
|
+
|
|
2558
|
+
## 20. Security and Adversarial Testing
|
|
2559
|
+
|
|
2560
|
+
Security gates are fail-closed. A green regression suite proves only represented
|
|
2561
|
+
cases; it does not waive an uncovered boundary. Never turn a known vulnerability
|
|
2562
|
+
into expected behavior. Hostile archives, DNS/rebinding, kill points, and native
|
|
2563
|
+
process tests run only in a bounded disposable container or VM.
|
|
2564
|
+
|
|
2565
|
+
### 20.1 Failure and evidence oracle
|
|
2566
|
+
|
|
2567
|
+
- [ ] Use only dummy credentials, controlled loopback services, local archives,
|
|
2568
|
+
and outside-sandbox sentinels. Never contact metadata services, a LAN host,
|
|
2569
|
+
production registry, or private infrastructure.
|
|
2570
|
+
- [ ] Rejected usage normally exits `2`, invalid configuration `78`, dangerous
|
|
2571
|
+
install `1`, and health warning `4`. Exit `70`, signal death, resource
|
|
2572
|
+
exhaustion, partial publication, or silent success is a failure.
|
|
2573
|
+
- [ ] For stateful rejection compare config, lock, cache, index, source bytes,
|
|
2574
|
+
databases, event cursor, process/listener set, stdout, and stderr.
|
|
2575
|
+
- [ ] Capture terminal-control fixtures to files and inspect bytes through a
|
|
2576
|
+
non-rendering program. Never print them directly to a terminal.
|
|
2577
|
+
- [ ] Security `FAIL` or `BLOCKED` blocks release unless the release record has a
|
|
2578
|
+
named issue, impact, owner, mitigation, approver, and expiry.
|
|
2579
|
+
|
|
2580
|
+
### 20.2 Filesystem containment and links
|
|
2581
|
+
|
|
2582
|
+
```sh
|
|
2583
|
+
outside="$AKM_SANDBOX/outside-sentinel"
|
|
2584
|
+
mkdir -p "$outside"
|
|
2585
|
+
printf 'outside-must-not-change\n' >"$outside/sentinel"
|
|
2586
|
+
ln -s "$outside" "$AKM_BUNDLE_DIR/memories/qa-link"
|
|
2587
|
+
|
|
2588
|
+
set +e
|
|
2589
|
+
akm remember 'must not escape' --name escaped --path qa-link \
|
|
2590
|
+
>"$AKM_SANDBOX/link-write.stdout" \
|
|
2591
|
+
2>"$AKM_SANDBOX/link-write.stderr"
|
|
2592
|
+
link_status=$?
|
|
2593
|
+
set -e
|
|
2594
|
+
|
|
2595
|
+
test "$link_status" -eq 2
|
|
2596
|
+
test ! -e "$outside/escaped.md"
|
|
2597
|
+
test "$(cat "$outside/sentinel")" = outside-must-not-change
|
|
2598
|
+
rm "$AKM_BUNDLE_DIR/memories/qa-link"
|
|
2599
|
+
```
|
|
2600
|
+
|
|
2601
|
+
- [ ] **[LOCAL]** Remember/import/env/secret/workflow/task/proposal writes reject
|
|
2602
|
+
absolute, parent, mixed-separator, NUL, drive/device, and descendant-
|
|
2603
|
+
symlink escapes before mutation.
|
|
2604
|
+
- [ ] **[LOCAL]** Manual and Git-backed walkers skip escaping file/directory
|
|
2605
|
+
symlinks, including tracked and untracked Git links. Outside marker never
|
|
2606
|
+
enters search/show/snippets/embeddings/model prompts/index DB.
|
|
2607
|
+
- [ ] **[LOCAL]** Internal links follow one documented containment rule; hardlink,
|
|
2608
|
+
FIFO/device, unreadable, disappearing, and rename-race cases stay bounded.
|
|
2609
|
+
- [ ] **[LOCAL REGRESSION]** Direct write primitives and Git-listed asset reads
|
|
2610
|
+
enforce the same realpath boundary. Any outside read/write blocks release.
|
|
2611
|
+
- [ ] **[LOCAL]** Archive extraction rejects absolute/traversing names, unsafe
|
|
2612
|
+
strip results, escaping symlink/hardlink targets, duplicate/type-conflict
|
|
2613
|
+
members, devices/FIFOs/sockets, and unsupported types before publication.
|
|
2614
|
+
|
|
2615
|
+
### 20.3 SSRF, redirects, DNS, and network budgets
|
|
2616
|
+
|
|
2617
|
+
Section 9.8's `NODE_ENV=test` crawl is a protocol seam only. Prove an ordinary
|
|
2618
|
+
installed CLI cannot activate that seam merely through ambient environment:
|
|
2619
|
+
|
|
2620
|
+
```sh
|
|
2621
|
+
before_requests="$(wc -l <"$AKM_QA_SERVICE_LOG")"
|
|
2622
|
+
set +e
|
|
2623
|
+
NODE_ENV=test akm bundle add "$AKM_QA_WEBSITE_URL" \
|
|
2624
|
+
--name qa-ambient-bypass --allow-insecure-transport \
|
|
2625
|
+
>"$AKM_SANDBOX/ambient-bypass.stdout" \
|
|
2626
|
+
2>"$AKM_SANDBOX/ambient-bypass.stderr" </dev/null
|
|
2627
|
+
ambient_status=$?
|
|
2628
|
+
set -e
|
|
2629
|
+
|
|
2630
|
+
installed="$(akm bundle list --format json |
|
|
2631
|
+
jq '[.sources[]? | select(.name == "qa-ambient-bypass")] | length')"
|
|
2632
|
+
if test "$installed" -ne 0; then
|
|
2633
|
+
akm bundle remove qa-ambient-bypass --yes >/dev/null 2>&1 || true
|
|
2634
|
+
fi
|
|
2635
|
+
|
|
2636
|
+
test "$ambient_status" -eq 78
|
|
2637
|
+
test "$installed" -eq 0
|
|
2638
|
+
test "$(wc -l <"$AKM_QA_SERVICE_LOG")" -eq "$before_requests"
|
|
2639
|
+
```
|
|
2640
|
+
|
|
2641
|
+
- [ ] **[SERVICE REGRESSION]** Ambient `NODE_ENV=test`/`BUN_TEST=1` cannot bypass
|
|
2642
|
+
production private-host policy. Current failure requires a fix, not PASS.
|
|
2643
|
+
- [ ] **[SERVICE]** Reject loopback/private/link-local/metadata/CGNAT/multicast/
|
|
2644
|
+
reserved/documentation IPv4, IPv6, mapped IPv6, NAT64, 6to4, userinfo, and
|
|
2645
|
+
non-HTTP schemes.
|
|
2646
|
+
- [ ] **[SERVICE]** Mixed public/private, empty, and failed DNS results fail
|
|
2647
|
+
closed. Validate and constrain every redirect hop, not only the first URL.
|
|
2648
|
+
- [ ] **[DESTRUCTIVE]** Rebinding fixture proves the connection uses the approved
|
|
2649
|
+
address or equivalent enforcement; a second unpinned DNS lookup fails.
|
|
2650
|
+
- [ ] **[SERVICE]** Strip auth/cookie/proxy-auth on cross-origin redirect and
|
|
2651
|
+
bound redirects, DNS, headers/body, bytes, rate-limit wait, and total time.
|
|
2652
|
+
- [ ] **[SERVICE REGRESSION]** Equivalent egress policy covers website pages/
|
|
2653
|
+
robots/feeds, response-derived media URLs, URL health checks, registries,
|
|
2654
|
+
npm metadata/tarballs, setup recommendations, and self-update downloads.
|
|
2655
|
+
- [ ] Request logs prove no forbidden destination was contacted; error text alone
|
|
2656
|
+
is not evidence.
|
|
2657
|
+
|
|
2658
|
+
### 20.4 Registry, package, Git, and archive trust
|
|
2659
|
+
|
|
2660
|
+
- [ ] **[LOCAL]** Git source/ref rejects option-shaped refs, control chars,
|
|
2661
|
+
`ext::`, `fd::`, `file:`, unknown/insecure schemes, and revision mismatch.
|
|
2662
|
+
Subprocesses use argv, bounded time, and noninteractive auth.
|
|
2663
|
+
- [ ] **[SERVICE]** npm honors strong SRI before SHA-1, requires integrity,
|
|
2664
|
+
validates every redirect host, and executes no package lifecycle script
|
|
2665
|
+
while materializing a bundle.
|
|
2666
|
+
- [ ] **[DESTRUCTIVE]** Archive budgets cover compressed/expanded bytes, member
|
|
2667
|
+
count, per-member size, depth, compression ratio, listing/extraction time,
|
|
2668
|
+
and partial staging cleanup.
|
|
2669
|
+
- [ ] **[SERVICE]** Registry response size/count/depth/string limits apply to
|
|
2670
|
+
static and skills providers before parse/cache/render. Malformed/control-
|
|
2671
|
+
character data cannot poison another registry.
|
|
2672
|
+
- [ ] **[SERVICE REGRESSION]** Registry list/config/info/search never expose
|
|
2673
|
+
literal option credentials or credential-bearing URLs; install guidance is
|
|
2674
|
+
inert and safely quoted.
|
|
2675
|
+
- [ ] **[SERVICE]** Setup binds an official recommendation ID to its expected
|
|
2676
|
+
authenticated install ref; an arbitrary configured registry cannot replace
|
|
2677
|
+
that ref under the same display ID.
|
|
2678
|
+
- [ ] **[LOCAL REGRESSION]** Failed add/update/remove restores prior config,
|
|
2679
|
+
lock, cache, root, index, and event cursor, including re-add of an existing
|
|
2680
|
+
source. No success event precedes security audit.
|
|
2681
|
+
|
|
2682
|
+
### 20.5 Dangerous env, secrets, output, and permissions
|
|
2683
|
+
|
|
2684
|
+
```sh
|
|
2685
|
+
before_event="$(akm log --format json | jq -r '.events[-1].id // 0')"
|
|
2686
|
+
set +e
|
|
2687
|
+
akm bundle add \
|
|
2688
|
+
"$REPO/tests/fixtures/manual-qa/suppressed-dangerous-bundle" \
|
|
2689
|
+
--name qa-suppressed-dangerous \
|
|
2690
|
+
>"$AKM_SANDBOX/suppressed.stdout" \
|
|
2691
|
+
2>"$AKM_SANDBOX/suppressed.stderr" </dev/null
|
|
2692
|
+
suppressed_status=$?
|
|
2693
|
+
set -e
|
|
2694
|
+
|
|
2695
|
+
test "$suppressed_status" -eq 1
|
|
2696
|
+
test "$(akm log --format json | jq -r '.events[-1].id // 0')" = "$before_event"
|
|
2697
|
+
! akm bundle list --format json |
|
|
2698
|
+
jq -e '.sources[]? | select(.name == "qa-suppressed-dangerous")'
|
|
2699
|
+
```
|
|
2700
|
+
|
|
2701
|
+
- [ ] **[LOCAL REGRESSION]** Publisher lint suppression cannot suppress
|
|
2702
|
+
untrusted install security; scan read errors fail closed; inferred,
|
|
2703
|
+
declarative, normal update, and force update use the same pre-publication
|
|
2704
|
+
audit.
|
|
2705
|
+
- [ ] **[LOCAL]** Runtime blocks source-origin `LD_PRELOAD`, `DYLD_*`, `PATH`,
|
|
2706
|
+
`NODE_OPTIONS`, `BASH_FUNC_*`, and `GIT_CONFIG_*`; first-party use warns.
|
|
2707
|
+
- [ ] **[LOCAL]** Exact dummy values and credential-shaped URL/userinfo/query/
|
|
2708
|
+
fragment variants are absent from AKM stdout/stderr, errors, events,
|
|
2709
|
+
task logs, databases, reports, proposals, workflows, backups, and config.
|
|
2710
|
+
- [ ] **[LOCAL]** JSON/YAML stay parseable; text/Markdown visibly encode terminal
|
|
2711
|
+
controls; HTML escapes attacker fields. Inspect captured bytes only.
|
|
2712
|
+
- [ ] **[LOCAL]** Env/secret directories and files akm creates are `0700`/`0600`
|
|
2713
|
+
even under umask 022, as are config backups and scheduler invocation
|
|
2714
|
+
files. The data directory, the databases and their sidecars, and per-run
|
|
2715
|
+
task logs take the process umask: akm neither sets nor reports on those.
|
|
2716
|
+
- [ ] **[PLATFORM]** Windows ACL evidence replaces POSIX mode checks and is marked
|
|
2717
|
+
N/A only when genuinely unsupported.
|
|
2718
|
+
|
|
2719
|
+
### 20.6 Execution, agents, model output, and listeners
|
|
2720
|
+
|
|
2721
|
+
- [ ] **[LOCAL]** Command arrays preserve exact argv for whitespace, quotes,
|
|
2722
|
+
semicolons, substitutions, newlines, percent signs, and platform meta-
|
|
2723
|
+
characters. No shell appears unless explicitly selected by contract.
|
|
2724
|
+
- [ ] **[PLATFORM]** Cron/launchd/schtasks serialization round-trips argv without
|
|
2725
|
+
shell/XML injection; timeout/cancel removes process tree and scheduler entry.
|
|
2726
|
+
- [ ] **[AI]** Empty, fenced, think-block, malformed, nested, oversized, and
|
|
2727
|
+
schema-invalid model output fails closed within bounded attempts/bytes.
|
|
2728
|
+
- [ ] **[AI]** Asset text containing fake verdict JSON, rubric overrides, tool
|
|
2729
|
+
instructions, and exfiltration requests cannot make a rejecting fake judge
|
|
2730
|
+
pass or publish a proposal.
|
|
2731
|
+
- [ ] **[AI REGRESSION]** Every AKM-managed listener has an unpredictable
|
|
2732
|
+
per-process credential. Unauthenticated local requests get `401`, the AKM
|
|
2733
|
+
client authenticates, and the credential is never durable.
|
|
2734
|
+
- [ ] **[AI REGRESSION]** The OpenCode SDK fallback listener is not considered
|
|
2735
|
+
safe merely because it binds loopback or chooses a random port.
|
|
2736
|
+
- [ ] **[LOCAL]** Active-bundle custom wiki fetchers are trusted executable code;
|
|
2737
|
+
installation alone cannot import one, and activation requires an explicit
|
|
2738
|
+
reviewed trust decision.
|
|
2739
|
+
|
|
2740
|
+
### 20.7 Malformed input and resource exhaustion
|
|
2741
|
+
|
|
2742
|
+
- [ ] **[LOCAL]** Reject `__proto__`, `constructor`, and `prototype` at every
|
|
2743
|
+
config/import merge depth; ordinary object prototypes remain unchanged.
|
|
2744
|
+
- [ ] **[LOCAL]** Wrong top-level types, duplicate keys, invalid UTF-8/NUL,
|
|
2745
|
+
extreme numbers, oversized strings/arrays/nesting, truncated JSON/YAML,
|
|
2746
|
+
and YAML alias expansion terminate within bounded memory/time.
|
|
2747
|
+
- [ ] **[LOCAL]** Workflow source/step/param/fan-out/plan/depth limits fail before
|
|
2748
|
+
durable run creation or engine dispatch.
|
|
2749
|
+
- [ ] **[LOCAL]** Malformed input emits no stack, spawns no child, makes no model
|
|
2750
|
+
or public request, writes no partial state, and never exits `70`.
|
|
2751
|
+
- [ ] Fuzz/property failures retain the seed as a deterministic fixture.
|
|
2752
|
+
|
|
2753
|
+
### 20.8 Focused security suite and disposition
|
|
2754
|
+
|
|
2755
|
+
```sh
|
|
2756
|
+
bun test --timeout=120000 \
|
|
2757
|
+
tests/redaction.test.ts \
|
|
2758
|
+
tests/registry-resolve.test.ts \
|
|
2759
|
+
tests/opencode-sdk-runner.test.ts \
|
|
2760
|
+
tests/integration/website-ssrf.test.ts \
|
|
2761
|
+
tests/integration/tar-utils-scan.test.ts \
|
|
2762
|
+
tests/integration/walker.test.ts \
|
|
2763
|
+
tests/integration/vault-dangerous-key-install-gate.test.ts \
|
|
2764
|
+
tests/integration/vault-dangerous-key-lint.test.ts \
|
|
2765
|
+
tests/integration/env-run-dangerous-key-block.test.ts \
|
|
2766
|
+
tests/integration/config-sanitize-secrets.test.ts \
|
|
2767
|
+
tests/integration/self-update.test.ts
|
|
2768
|
+
```
|
|
2769
|
+
|
|
2770
|
+
Current known failing gates must be fixed or explicitly waived with expiry:
|
|
2771
|
+
|
|
2772
|
+
| Boundary | Required disposition |
|
|
2773
|
+
| --- | --- |
|
|
2774
|
+
| Ambient test-mode SSRF bypass and secondary URL sinks | `FAIL` until guarded |
|
|
2775
|
+
| Git/direct-write symlink escape | `FAIL` until contained |
|
|
2776
|
+
| Archive expanded-resource/type/link budgets | `FAIL` until bounded |
|
|
2777
|
+
| Registry option/control-data rendering | `FAIL` until redacted |
|
|
2778
|
+
| Registry URL credential persistence and rendering | `PASS` — #811 passed independent review after seven hardening cycles |
|
|
2779
|
+
| Suppressible/add-only dangerous-key audit and event ordering | `FAIL` until pre-publication |
|
|
2780
|
+
| Source lifecycle rollback across config/lock/root/index/events | `FAIL` until atomic/recoverable |
|
|
2781
|
+
| OpenCode local-listener authentication/documentation | `FAIL` until authenticated |
|
|
2782
|
+
| Exact-value command-target task-log redaction | `PASS` — fixed in 0.9.1 (#755) |
|
|
2783
|
+
| Package-manager upgrade exact-version verification | `PASS` — fixed 2026-08-06 (`self-update.ts`) |
|
|
2784
|
+
|
|
2785
|
+
---
|
|
2786
|
+
|
|
2787
|
+
## 21. Package, Runtime, Platform, and Release
|
|
2788
|
+
|
|
2789
|
+
Release acceptance runs against the exact candidate commit and bytes. A local
|
|
2790
|
+
source build, packed npm artifact, standalone binary, installer, and published
|
|
2791
|
+
release are different subjects and require separate evidence.
|
|
2792
|
+
|
|
2793
|
+
### 21.1 Supported matrix and build boundaries
|
|
2794
|
+
|
|
2795
|
+
| Surface | Current support/constraint |
|
|
2796
|
+
| --- | --- |
|
|
2797
|
+
| npm package | Node `>=22` bootstrap; Bun `>=1.0` preferred, Node fallback supported |
|
|
2798
|
+
| Node fallback | Test Node 22 and 24; requires built dist and usable SQLite dependency |
|
|
2799
|
+
| Standalone | Linux x64/arm64 glibc, macOS x64/arm64, Windows x64 |
|
|
2800
|
+
| Unsupported standalone | Alpine/musl, 32-bit, native Windows ARM64 |
|
|
2801
|
+
| POSIX installer | Linux/macOS x64/arm64; `AKM_INSTALL_DIR` override |
|
|
2802
|
+
| Windows installer | PowerShell 5.1+, x64 binary; ARM64 uses x64 emulation |
|
|
2803
|
+
| Scheduler | Linux crontab, macOS LaunchAgent, Windows Task Scheduler |
|
|
2804
|
+
|
|
2805
|
+
```sh
|
|
2806
|
+
bun install --frozen-lockfile
|
|
2807
|
+
bun run check
|
|
2808
|
+
bun run build
|
|
2809
|
+
|
|
2810
|
+
test -x dist/akm
|
|
2811
|
+
test -x dist/akm-migrate
|
|
2812
|
+
test -f dist/cli.js
|
|
2813
|
+
test -f dist/cli-node.mjs
|
|
2814
|
+
test -f dist/scripts/akm-migrate.js
|
|
2815
|
+
test -f dist/scripts/akm-migrate-node.js
|
|
2816
|
+
test ! -e dist/tests
|
|
2817
|
+
|
|
2818
|
+
node dist/akm --version
|
|
2819
|
+
node dist/cli-node.mjs --version
|
|
2820
|
+
bun dist/cli.js --version
|
|
2821
|
+
```
|
|
2822
|
+
|
|
2823
|
+
- [ ] **[RELEASE]** Build emits `src/**` only plus deliberately copied runtime
|
|
2824
|
+
assets and migration entries; no test/general helper tree leaks into dist.
|
|
2825
|
+
- [ ] **[RELEASE]** Package bins are exactly `dist/akm` and `dist/akm-migrate`,
|
|
2826
|
+
both Node-shebang launchers with Bun preference and Node fallback.
|
|
2827
|
+
- [ ] **[RELEASE]** Relative ESM imports are Node-compatible and all module-local
|
|
2828
|
+
text/YAML/XML assets needed at runtime are present.
|
|
2829
|
+
- [ ] **[RELEASE]** Build may regenerate schema but leaves no unexplained tracked
|
|
2830
|
+
diff. Release acceptance uses verify-only lint, not formatting mutation.
|
|
2831
|
+
|
|
2832
|
+
### 21.2 npm pack, isolated install, and runtime parity
|
|
2833
|
+
|
|
2834
|
+
```sh
|
|
2835
|
+
bun run test:package
|
|
2836
|
+
bun test --timeout=120000 \
|
|
2837
|
+
tests/package-install.test.ts \
|
|
2838
|
+
tests/integration/package-install.test.ts \
|
|
2839
|
+
tests/integration/package-launcher.test.ts \
|
|
2840
|
+
tests/integration/npm-bin-contract.test.ts
|
|
2841
|
+
|
|
2842
|
+
bun run build
|
|
2843
|
+
# Install the exact spec package.json declares — never a range of your own.
|
|
2844
|
+
# better-sqlite3 compiles from source whenever there is no prebuilt binary for
|
|
2845
|
+
# the running Node's ABI, and a from-source build against Node 24.19+ headers
|
|
2846
|
+
# aborts at teardown (#790). CI does the same read-back. Remove it first:
|
|
2847
|
+
# `npm install pkg@version` is a no-op when that version is already installed
|
|
2848
|
+
# and will NOT rebuild the binding for the Node you are about to test with.
|
|
2849
|
+
rm -rf node_modules/better-sqlite3
|
|
2850
|
+
npm install --no-save \
|
|
2851
|
+
"better-sqlite3@$(node -p "require('./package.json').optionalDependencies['better-sqlite3']")"
|
|
2852
|
+
AKM_SMOKE_NODE=node bun run test:node-smoke
|
|
2853
|
+
AKM_SMOKE_NODE=node bun run test:node-compat
|
|
2854
|
+
```
|
|
2855
|
+
|
|
2856
|
+
- [ ] **[RELEASE]** Test-package packs and installs under a temporary prefix,
|
|
2857
|
+
verifies package/version/bin ownership, runs both launchers, and cleans up.
|
|
2858
|
+
- [ ] **[PLATFORM]** Repeat Node smoke/compat on Node 22 and 24. Every gated test
|
|
2859
|
+
runs rather than skips; no Bun global leaks into the forced Node path.
|
|
2860
|
+
- [ ] **[PLATFORM]** Missing/failed/pre-1.0 Bun probe falls back to Node; current
|
|
2861
|
+
Bun is selected. Paths with spaces and Windows `%*` preserve argv.
|
|
2862
|
+
- [ ] **[PLATFORM]** Node 20 package install fails at preinstall with the Node 22
|
|
2863
|
+
diagnostic and leaves no usable bins.
|
|
2864
|
+
- [ ] **[RELEASE]** Packed payload contains required dist/docs/schemas and omits
|
|
2865
|
+
source, tests, repository scripts, `.git`, `.akm`, and `node_modules`.
|
|
2866
|
+
- [ ] **[RELEASE]** Packed README equals `.github/README.npm.md`. Run lifecycle
|
|
2867
|
+
staging only in a disposable worktree because it overwrites `README.md`.
|
|
2868
|
+
|
|
2869
|
+
`bun run build:install` replaces the configured machine-global installation and
|
|
2870
|
+
is never an ordinary manual QA command.
|
|
2871
|
+
|
|
2872
|
+
### 21.3 Docker and standalone artifacts
|
|
2873
|
+
|
|
2874
|
+
```sh
|
|
2875
|
+
./tests/docker/run-docker-tests.sh
|
|
2876
|
+
|
|
2877
|
+
# Focused alternatives:
|
|
2878
|
+
./tests/docker/run-docker-tests.sh ubuntu-binary
|
|
2879
|
+
./tests/docker/run-docker-tests.sh --bun-only
|
|
2880
|
+
./tests/docker/run-docker-tests.sh --binary-only
|
|
2881
|
+
|
|
2882
|
+
# Test wrapper; without the variable the file skips:
|
|
2883
|
+
AKM_DOCKER_TESTS=1 bun test tests/integration/docker-install.test.ts
|
|
2884
|
+
```
|
|
2885
|
+
|
|
2886
|
+
- [ ] **[PLATFORM]** Four Bun-linked source variants pass: Ubuntu, Debian,
|
|
2887
|
+
Alpine, Fedora. Three local linux-x64 binary variants pass: Ubuntu,
|
|
2888
|
+
Debian, Fedora. Summary is seven passes, zero failures/skips.
|
|
2889
|
+
- [ ] **[PLATFORM]** Matrix proves CLI startup, bundle create, index/search/show/
|
|
2890
|
+
info/list, and incremental indexing. It does not prove npm pack, published
|
|
2891
|
+
artifact, installer, self-upgrade, ARM, macOS, or Windows.
|
|
2892
|
+
- [ ] **[RELEASE]** Cross-compile five exact standalone names; run each on native
|
|
2893
|
+
OS/architecture and verify the embedded version and supported command surface.
|
|
2894
|
+
- [ ] **[RELEASE]** `checksums.txt` covers exactly five binaries and both
|
|
2895
|
+
installers. The npm tarball is compared separately because it is generated
|
|
2896
|
+
later.
|
|
2897
|
+
|
|
2898
|
+
### 21.4 Installers and self-upgrade
|
|
2899
|
+
|
|
2900
|
+
Use release-attached installers, not raw `main`, and pin a concrete tag:
|
|
2901
|
+
|
|
2902
|
+
```sh
|
|
2903
|
+
VERSION="$(jq -r .version package.json)"
|
|
2904
|
+
TAG="v$VERSION"
|
|
2905
|
+
INSTALL_ROOT="$(mktemp -d)"
|
|
2906
|
+
|
|
2907
|
+
curl -fsSL \
|
|
2908
|
+
"https://github.com/itlackey/akm/releases/download/$TAG/install.sh" \
|
|
2909
|
+
-o "$INSTALL_ROOT/install.sh"
|
|
2910
|
+
AKM_INSTALL_DIR="$INSTALL_ROOT/bin" bash "$INSTALL_ROOT/install.sh" "$TAG"
|
|
2911
|
+
"$INSTALL_ROOT/bin/akm" --version
|
|
2912
|
+
```
|
|
2913
|
+
|
|
2914
|
+
- [ ] **[LIVE PLATFORM]** Correct OS/arch asset downloads; exact checksum entry
|
|
2915
|
+
verifies before replacement; unsupported arch/tool/permission/mismatch
|
|
2916
|
+
fails closed.
|
|
2917
|
+
- [ ] **[PLATFORM]** Windows installer uses isolated `AKM_INSTALL_DIR`, verifies
|
|
2918
|
+
SHA-256, records/reverses PATH mutation, and runs x64 candidate.
|
|
2919
|
+
- [ ] **[RELEASE]** Installers support `AKM_INSTALL_DIR`; do not document a
|
|
2920
|
+
nonexistent `--prefix` option.
|
|
2921
|
+
|
|
2922
|
+
Real self-upgrade runs only in a disposable VM/container from a compatible
|
|
2923
|
+
older 0.9 release:
|
|
2924
|
+
|
|
2925
|
+
```sh
|
|
2926
|
+
unset AKM_UPGRADE_SKIP_CHECKSUM
|
|
2927
|
+
akm --version
|
|
2928
|
+
akm migrate status
|
|
2929
|
+
akm upgrade --check --format json
|
|
2930
|
+
akm upgrade --format json
|
|
2931
|
+
akm --version
|
|
2932
|
+
akm migrate status
|
|
2933
|
+
akm info
|
|
2934
|
+
```
|
|
2935
|
+
|
|
2936
|
+
- [ ] **[LIVE DESTRUCTIVE]** Check is nonmutating. npm/Bun/pnpm global installs
|
|
2937
|
+
invoke their package manager; standalone streams bounded bytes, verifies
|
|
2938
|
+
checksum, preflights the staged binary, and atomically replaces it.
|
|
2939
|
+
- [ ] **[LIVE DESTRUCTIVE]** Failure retains explicit recoverable old/new
|
|
2940
|
+
artifacts. Integrity bypass variable is never set for acceptance.
|
|
2941
|
+
- [ ] **[LIVE]** A 0.8 package can be replaced, but its config/state are not
|
|
2942
|
+
runtime inputs: archive them, initialize current paths, then explicitly
|
|
2943
|
+
migrate retained task-v2 sources.
|
|
2944
|
+
- [ ] **[LIVE]** RC/`next` cannot be validated through latest-stable discovery;
|
|
2945
|
+
record post-publication stable self-upgrade as blocked until discoverable.
|
|
2946
|
+
|
|
2947
|
+
### 21.5 Native scheduler acceptance
|
|
2948
|
+
|
|
2949
|
+
Use an installed npm or standalone candidate and a disposable OS account. Do
|
|
2950
|
+
not hide an ineligible checkout behind `--rebind` in release acceptance.
|
|
2951
|
+
|
|
2952
|
+
- [ ] **[PLATFORM]** Doctor reports eligible `npm`/`standalone` binding with
|
|
2953
|
+
absolute candidate paths and no credentials in the row.
|
|
2954
|
+
- [ ] **[PLATFORM]** Add disabled task, inspect native definition, enable/edit +
|
|
2955
|
+
sync, trigger, verify candidate version/history/log, delete YAML + sync,
|
|
2956
|
+
and prove native removal.
|
|
2957
|
+
- [ ] **[PLATFORM]** Preserve unrelated crontab/plist/schtasks entries and exact
|
|
2958
|
+
argv quoting. Context descriptor is content-addressed and mode `0600` on
|
|
2959
|
+
POSIX.
|
|
2960
|
+
- [ ] **[PLATFORM]** Linux standalone gate runs with its required candidate bin,
|
|
2961
|
+
version, and architecture variables.
|
|
2962
|
+
- [ ] **[PLATFORM]** macOS/Windows native test runs only on disposable GitHub
|
|
2963
|
+
Actions with `CI=true`, `GITHUB_ACTIONS=true`, compiled native artifact,
|
|
2964
|
+
real runner home, unique task ID, gate dirs, version, and architecture.
|
|
2965
|
+
- [ ] **[PLATFORM]** Windows Node variant also provides packed `akm.cmd`, exact
|
|
2966
|
+
`node.exe`, path-with-spaces gate, and unique node task ID.
|
|
2967
|
+
|
|
2968
|
+
A bare `AKM_NATIVE_SCHEDULER_TESTS=1 bun test ...` is insufficient and must not
|
|
2969
|
+
be recorded as PASS when the test skipped.
|
|
2970
|
+
|
|
2971
|
+
### 21.6 Exact release gate and publication
|
|
2972
|
+
|
|
2973
|
+
```sh
|
|
2974
|
+
bun run release:check
|
|
2975
|
+
|
|
2976
|
+
# Partial local gate only:
|
|
2977
|
+
./tests/release-check.sh --skip-docker
|
|
2978
|
+
```
|
|
2979
|
+
|
|
2980
|
+
- [ ] **[RELEASE]** Full script runs workflow syntax/contract, verify-only lint,
|
|
2981
|
+
typecheck, build/bin/migration checks, package acceptance, setup/install
|
|
2982
|
+
regression, explicit legacy-task migration, Linux standalone scheduler,
|
|
2983
|
+
unit, integration, then Docker.
|
|
2984
|
+
- [ ] **[RELEASE]** Every gated file reports executed tests. Skip-docker is
|
|
2985
|
+
partial unless a separate exact-commit matrix transcript exists.
|
|
2986
|
+
- [ ] **[RELEASE]** Cut the changelog BEFORE triggering the workflow: bump
|
|
2987
|
+
`package.json` `version`, rename `## [Unreleased]` to
|
|
2988
|
+
`## [<version>] - <YYYY-MM-DD>`, and leave a fresh empty `## [Unreleased]`
|
|
2989
|
+
above it. This is functional, not cosmetic —
|
|
2990
|
+
`src/commands/sources/migration-help.ts:84` resolves release notes by
|
|
2991
|
+
skipping the `Unreleased` heading, so shipping an un-renamed section makes
|
|
2992
|
+
the released binary's `akm help migrate latest` report the PREVIOUS
|
|
2993
|
+
release's notes. Enforced by `tests/integration/workflow-release.test.ts`
|
|
2994
|
+
("the changelog is cut…"), which `tests/release-check.sh` runs as
|
|
2995
|
+
`run_step "Workflow Release Contract"` — so a missed cut fails the release
|
|
2996
|
+
gate rather than depending on this checklist being read.
|
|
2997
|
+
- [ ] **[RELEASE]** Official workflow input version equals committed package
|
|
2998
|
+
version and targets the tested SHA. Stable uses npm `latest`; prerelease
|
|
2999
|
+
uses `next` and GitHub prerelease.
|
|
3000
|
+
- [ ] **[RELEASE]** After publication, download every asset, verify checksums,
|
|
3001
|
+
compare GitHub/npm tarballs, install under fresh Node 22/24, and run native
|
|
3002
|
+
binaries/installers on supported targets.
|
|
3003
|
+
- [ ] **[RELEASE]** npm version is immutable. Existing GitHub assets may be
|
|
3004
|
+
clobbered by rerun, so rerun only identical commit/bytes and compare hashes.
|
|
3005
|
+
|
|
3006
|
+
---
|
|
3007
|
+
|
|
3008
|
+
## 22. Evidence and Cleanup
|
|
3009
|
+
|
|
3010
|
+
### 22.1 Required evidence
|
|
3011
|
+
|
|
3012
|
+
Retain outside `/tmp/akm-*` because test cleanup can remove old matching paths:
|
|
3013
|
+
|
|
3014
|
+
- [ ] Commit SHA, package version, candidate/artifact digests, OS/architecture,
|
|
3015
|
+
Bun/Node/npm versions, exact launcher path, UTC window, tester, tiers.
|
|
3016
|
+
- [ ] Exact command, exit status, stdout, and stderr as separate files. Preserve
|
|
3017
|
+
`PIPESTATUS[0]` when using `tee`.
|
|
3018
|
+
- [ ] Before/after worktree status and config/lock/bundle/cache/data/state/event
|
|
3019
|
+
manifests for stateful failures.
|
|
3020
|
+
- [ ] Test totals and skip list. An opt-in suite that skipped is `BLOCKED`, never
|
|
3021
|
+
`PASS`.
|
|
3022
|
+
- [ ] Pack manifest, isolated install/bin ownership, Node matrix, Docker summary,
|
|
3023
|
+
native scheduler before/after, self-upgrade transcript, release/run URLs,
|
|
3024
|
+
npm metadata, release inventory, and checksum transcript where applicable.
|
|
3025
|
+
- [ ] Every failure's reproduction result and retained rollback/stage/rescue path.
|
|
3026
|
+
- [ ] Every waiver's issue, impact, owner, mitigation, approver, and expiry.
|
|
3027
|
+
- [ ] No real token, npmrc, GitHub auth file, raw environment dump, prompt, or
|
|
3028
|
+
unredacted secret-bearing state enters evidence.
|
|
3029
|
+
|
|
3030
|
+
### 22.2 Cleanup procedure
|
|
3031
|
+
|
|
3032
|
+
```sh
|
|
3033
|
+
if test -n "${AKM_QA_SERVICE_PID:-}"; then
|
|
3034
|
+
kill "$AKM_QA_SERVICE_PID" 2>/dev/null || true
|
|
3035
|
+
wait "$AKM_QA_SERVICE_PID" 2>/dev/null || true
|
|
3036
|
+
fi
|
|
3037
|
+
```
|
|
3038
|
+
|
|
3039
|
+
- [ ] Delete test task YAML, sync, and inspect native scheduler to prove no AKM
|
|
3040
|
+
test marker/plist/task remains.
|
|
3041
|
+
- [ ] Remove isolated npm prefixes, installer directories, migration clones,
|
|
3042
|
+
writable remotes, containers, and VMs.
|
|
3043
|
+
- [ ] Remove only the seven exact `akm-test-*` Docker image tags; never broad-
|
|
3044
|
+
prune a shared Docker host.
|
|
3045
|
+
- [ ] Archive failure shard logs/evidence before deleting announced temp roots.
|
|
3046
|
+
- [ ] Remove `tests/docker/.build` only by exact path after verifying it belongs
|
|
3047
|
+
to this checkout.
|
|
3048
|
+
- [ ] Copy evidence out, then guard sandbox deletion:
|
|
3049
|
+
|
|
3050
|
+
```sh
|
|
3051
|
+
case "$AKM_SANDBOX" in
|
|
3052
|
+
/tmp/akm-sandbox.*) rm -rf -- "$AKM_SANDBOX" ;;
|
|
3053
|
+
*) printf 'Refusing unsafe cleanup: %s\n' "$AKM_SANDBOX" >&2; false ;;
|
|
3054
|
+
esac
|
|
3055
|
+
```
|
|
3056
|
+
|
|
3057
|
+
- [ ] Compare final worktree/status to baseline. Explain every remaining change;
|
|
3058
|
+
do not use broad Git restore on a shared or dirty worktree.
|
|
3059
|
+
|
|
3060
|
+
---
|
|
3061
|
+
|
|
3062
|
+
## 23. Change-Based Test Selection
|
|
3063
|
+
|
|
3064
|
+
Every production change gets focused tests during iteration and `bun run check`
|
|
3065
|
+
before merge. `check:changed` is a fixed fast contract battery, not diff-aware
|
|
3066
|
+
and not a substitute for the full check.
|
|
3067
|
+
|
|
3068
|
+
| Changed area | Required escalation beyond focused tests |
|
|
3069
|
+
| --- | --- |
|
|
3070
|
+
| CLI dispatcher/output/command CLI | check:changed, envelope/output suites, sections 6 and 8, full check |
|
|
3071
|
+
| Config/paths/setup/schema | setup/install regression, first-run package path, sections 3 and 6, full check |
|
|
3072
|
+
| Refs/adapters/index/search/show | contract/ranking/index suites, sections 4 and 7; semantic if vectors/status changed |
|
|
3073
|
+
| Sources/registry/write-source | provider/write/publication suites, controlled service; LIVE git/npm/HTTPS when lifecycle changed |
|
|
3074
|
+
| Storage/migration/transactions | crash/concurrency/property/published-upgrade gates, section 19, destructive rehearsal for cutover changes |
|
|
3075
|
+
| Workflow | workflow unit/integration, gate fake agents, slow expansion, crash/contention when scheduler changed |
|
|
3076
|
+
| Workflow exec units / subprocess capture | section 14.5 end to end on an engine-less install, plus the worktree isolation and stale-sweep gates; any capture, timeout, or process-group change also runs the agent spawn suites, which share that subprocess layer |
|
|
3077
|
+
| Workflow child environment / allowlist | section 14.5 environment gates plus section 11; a change to the shared floor is native Windows, not Linux emulation |
|
|
3078
|
+
| Task/scheduler | task suites, Linux standalone; native macOS/Windows for backend/quoting/binding; published upgrade for schema changes |
|
|
3079
|
+
| Env/secret/security path/archive/network | env/secret plus traversal/SSRF/archive/redaction/dangerous-key suites, section 20 |
|
|
3080
|
+
| Agent/LLM/improve/proposal | family suites and fake-service AI pass; live bounded provider only for changed external dispatch |
|
|
3081
|
+
| Package/runtime/build dependencies | build/package/bin, Node 22/24, Bun launcher, standalone, Docker, release check |
|
|
3082
|
+
| Installers/self-update/standalone | installer/update suites, exact checksum/artifact tests, native install/upgrade, full release gate |
|
|
3083
|
+
| Workflows/release-check/Docker | workflow syntax/contract, actual Docker matrix, complete artifact inventory |
|
|
3084
|
+
| Test preload/helpers/runners | lint isolation, both sharded targets, multiple shard counts, leaked temp/log review |
|
|
3085
|
+
| Documentation only | doc-example lint and named contract tests; local evidence required because normal CI may skip docs-only changes |
|
|
3086
|
+
|
|
3087
|
+
Minimum selection rules:
|
|
3088
|
+
|
|
3089
|
+
- [ ] Changes to providers, refs, search/show, config, or output also run
|
|
3090
|
+
`tests/contracts/` and read architecture guidance before acceptance.
|
|
3091
|
+
- [ ] Contract/security/migration/release changes run the full relevant section,
|
|
3092
|
+
not one happy-path command.
|
|
3093
|
+
- [ ] A platform-specific change cannot be passed solely on Linux emulation.
|
|
3094
|
+
- [ ] A documentation-only change that alters commands runs those examples in a
|
|
3095
|
+
sandbox even if code tests are unaffected.
|
|
3096
|
+
|
|
3097
|
+
---
|
|
3098
|
+
|
|
3099
|
+
## 24. Known Constraints and Blocked Gates
|
|
3100
|
+
|
|
3101
|
+
These are current constraints, not historical expected failures. Re-audit this
|
|
3102
|
+
section before each release and remove an item when its implementation/gate is
|
|
3103
|
+
fixed.
|
|
3104
|
+
|
|
3105
|
+
### 24.1 Supported constraints
|
|
3106
|
+
|
|
3107
|
+
1. npm bootstrap requires Node `>=22`; Bun does not remove that requirement.
|
|
3108
|
+
2. No Alpine/musl standalone, native Windows ARM64, 32-bit, or non-listed target.
|
|
3109
|
+
3. Standalone binaries cannot load the externalized local transformer model.
|
|
3110
|
+
4. macOS/Windows schedules are per-user interactive; schedule grammar is narrower
|
|
3111
|
+
than Linux cron and Windows expansion is bounded.
|
|
3112
|
+
5. Docker, semantic E2E, Node compatibility, native scheduler, published upgrade,
|
|
3113
|
+
standalone scheduler, real-agent/model, and slow property gates are opt-in.
|
|
3114
|
+
6. Current normal CI is Ubuntu-centric and does not itself run Docker, real
|
|
3115
|
+
embeddings, native macOS/Windows schedulers, or release binaries.
|
|
3116
|
+
7. The release workflow runs `tests/release-check.sh --skip-docker`
|
|
3117
|
+
(build/publication follow it) but not Docker, the slow gate, or the Node
|
|
3118
|
+
matrix — those stay separate, opt-in local/CI gates.
|
|
3119
|
+
8. Docker source variants use `bun link`; binary variants use a locally compiled
|
|
3120
|
+
linux-x64 artifact. They do not prove npm/published bytes.
|
|
3121
|
+
9. Self-upgrade discovery follows latest stable/npm `@latest`, so an unpublished
|
|
3122
|
+
or prerelease candidate cannot complete that happy path.
|
|
3123
|
+
10. 0.8-to-0.9 self-upgrade is deliberately blocked; use explicit migration.
|
|
3124
|
+
|
|
3125
|
+
### 24.2 Open release-blocking regression gates
|
|
3126
|
+
|
|
3127
|
+
Triaged 2026-08-06 for the 0.9.0 release: every sub-item was assessed against
|
|
3128
|
+
the actual test/lint/code evidence (an item counts as covered only when a test
|
|
3129
|
+
*asserts* the property, not merely exercises the path). itlackey reviewed the
|
|
3130
|
+
triage the same day and directed per-row outcomes: three rows resolved
|
|
3131
|
+
outright, four targeted fixes landed (registry stale fallback, upgrade
|
|
3132
|
+
version verification, lint fail-closed, plus the full Semantic row), and the
|
|
3133
|
+
remaining gaps carry approved waivers with the expiries recorded below.
|
|
3134
|
+
|
|
3135
|
+
- [x] **Test harness:** curation cannot leave a schema-less shared `index.db`
|
|
3136
|
+
that makes later proposal acceptance tests fail by file order.
|
|
3137
|
+
**Resolved 2026-08-06.** Root cause: `openExistingDatabase` created the
|
|
3138
|
+
missing file on open (no schema), so `curate`'s fire-and-forget usage
|
|
3139
|
+
telemetry planted a broken `index.db` that later proposal acceptance saw
|
|
3140
|
+
as existing-but-corrupt. It now refuses to create (`create: false` down
|
|
3141
|
+
to the driver) and a missing index throws with an `akm index` remedy.
|
|
3142
|
+
Regression pins: `tests/storage/open-existing-database-no-create.test.ts`
|
|
3143
|
+
(order-independent distillation of the two-file repro: telemetry no-op →
|
|
3144
|
+
write-path fail-open) and the original pair passing in one process
|
|
3145
|
+
(`tests/curate-logic.test.ts` + `tests/commands/proposal/adapter-precommit-check.test.ts`).
|
|
3146
|
+
- [x] **Agent/LLM:** standalone `remember --enrich` dispatches enrichment instead
|
|
3147
|
+
of silently taking the raw-write path.
|
|
3148
|
+
**Resolved 2026-08-06.** The feared bug was real: `hasStructuredArgs`
|
|
3149
|
+
(`src/commands/read/remember-cli.ts`) omitted `args.enrich`, so a bare
|
|
3150
|
+
`--enrich` took the zero-flag hot path and never attempted enrichment.
|
|
3151
|
+
Fixed (routes like `--auto`; fail-soft unchanged) and the standalone test
|
|
3152
|
+
in `tests/integration/remember-frontmatter.test.ts` now asserts the
|
|
3153
|
+
enrichment branch ran (warning surfaced), which it previously did not.
|
|
3154
|
+
- [x] **Semantic:** vector dimension validation, truthful ready-vec, JS fallback
|
|
3155
|
+
after fast-path failure, runtime failure status, targeted-write readiness,
|
|
3156
|
+
and cancellation propagation.
|
|
3157
|
+
**Resolved 2026-08-06** (fix-now directed by itlackey at 0.9.0 triage):
|
|
3158
|
+
- Truthful ready-vec: `verifyIndexState` now reports `ready-vec` only when
|
|
3159
|
+
the vec fast path is actually serving (`isVecAvailable` AND
|
|
3160
|
+
`isVecFastPathReady`); a degraded run reports `ready-js` with an
|
|
3161
|
+
`akm index --full` hint. Pinned by
|
|
3162
|
+
`tests/integration/index-verification-truth.test.ts` (forces real vec
|
|
3163
|
+
insert failures via a width-mismatched mock endpoint; verified to fail
|
|
3164
|
+
against the old logic).
|
|
3165
|
+
- Dimension validation: `embedding.dimension` is bounded to the vec-table
|
|
3166
|
+
guard's 1–4096 in the config schema (fails at `config set` with a clear
|
|
3167
|
+
message instead of crashing `akm index`); the DB-layer guard itself is
|
|
3168
|
+
pinned in `tests/integration/db.test.ts`.
|
|
3169
|
+
- Runtime failure status: a real provider failure (HTTP 500) driven through
|
|
3170
|
+
`akmIndex()` to `semanticStatus: "blocked"` is now pinned in
|
|
3171
|
+
`index-verification-truth.test.ts`.
|
|
3172
|
+
- Cancellation propagation: pre-aborted and mid-run-aborted signals
|
|
3173
|
+
rejecting `akmIndex()` are pinned in the same file.
|
|
3174
|
+
- JS fallback after fast-path failure: already covered
|
|
3175
|
+
(`tests/integration/db.test.ts`, `tests/integration/vector-search.test.ts`).
|
|
3176
|
+
- Targeted-write readiness: first-class writes use the same embedding
|
|
3177
|
+
materializer as full indexing for the entry IDs they changed. A healthy
|
|
3178
|
+
provider makes the write vector-fresh before return; provider failure keeps
|
|
3179
|
+
the authored file and FTS row while publishing canonical blocked status.
|
|
3180
|
+
Pinned in `tests/integration/indexer/index-written-assets.test.ts`.
|
|
3181
|
+
- [x] **Durability:** atomic reader-visible index generations, live-lock age,
|
|
3182
|
+
full source lifecycle rollback, safe website/npm refresh, strict lockfile,
|
|
3183
|
+
registry stale fallback, and sync/write serialization.
|
|
3184
|
+
**Closed for 0.9.1** — the last open item (#758) landed; see the
|
|
3185
|
+
per-item history below.
|
|
3186
|
+
- Covered: strict lockfile (`tests/integration/lockfile.test.ts` corrupt-
|
|
3187
|
+
lockfile fail-closed + CAS cases).
|
|
3188
|
+
- **Fixed 2026-08-06** (fix-now directed at 0.9.0 triage): registry stale
|
|
3189
|
+
fallback — a failed registry fetch now serves the cached index even past
|
|
3190
|
+
its TTL, with a warning, instead of hard-failing
|
|
3191
|
+
(`src/storage/repositories/registry-cache.ts`; pinned by
|
|
3192
|
+
`tests/integration/registry-stale-fallback.test.ts`).
|
|
3193
|
+
- **Fixed for 0.9.1** ([#757](https://github.com/itlackey/akm/issues/757),
|
|
3194
|
+
[#759](https://github.com/itlackey/akm/issues/759)): four of the five
|
|
3195
|
+
partial/open items landed.
|
|
3196
|
+
- Live-lock age — a lock whose holder PID is genuinely ALIVE but whose
|
|
3197
|
+
mtime has aged past the threshold is now pinned in both directions, at
|
|
3198
|
+
the production thresholds, for the index-writer lease and the improve
|
|
3199
|
+
run lock (`tests/integration/index-writer-lock.test.ts`,
|
|
3200
|
+
`tests/integration/commands/improve/improve-lock-invariants.test.ts`).
|
|
3201
|
+
Each positive test fails when `probeLock`'s age branch is disabled; each
|
|
3202
|
+
negative control fails when it is made unconditional.
|
|
3203
|
+
- Atomic index generations — a second reader connection opened from inside
|
|
3204
|
+
`insertTransaction` now proves it observes the complete previous
|
|
3205
|
+
generation, never zero or partial rows
|
|
3206
|
+
(`tests/integration/indexer/reindex-generation-atomicity.test.ts`).
|
|
3207
|
+
- Sync serialization — `createExactPathCommit`'s own CAS is raced by a real
|
|
3208
|
+
concurrent commit landing inside the `update-ref` window, mirroring the
|
|
3209
|
+
`write-source.ts` pattern
|
|
3210
|
+
(`tests/integration/sync-exact-commit-cas.test.ts`).
|
|
3211
|
+
- Safe website refresh — this one was a REAL BUG, not just missing
|
|
3212
|
+
coverage: the refresh deleted the whole mirror and then wrote pages in
|
|
3213
|
+
place, so an interrupted run left a partial mirror that a later
|
|
3214
|
+
non-forced run would serve (the freshness marker is only rewritten on
|
|
3215
|
+
success, so the stale marker still looked fresh). Fixed with
|
|
3216
|
+
staging-dir-then-rename plus an age-gated sweep of abandoned staging
|
|
3217
|
+
dirs (`src/sources/snapshot-fetchers/website-ingest.ts`; pinned by
|
|
3218
|
+
`tests/integration/website-refresh-interruption.test.ts`, all four cases
|
|
3219
|
+
of which fail against the old in-place write). Residual, documented in
|
|
3220
|
+
code: publication is two renames, so a kill in that one-syscall window
|
|
3221
|
+
leaves the mirror absent — `requireStashDir` callers refresh immediately,
|
|
3222
|
+
others recover on the next expiry or `--force`.
|
|
3223
|
+
- **Closed for 0.9.1** ([#758](https://github.com/itlackey/akm/issues/758)):
|
|
3224
|
+
full source lifecycle rollback — the waiver's named verification test now
|
|
3225
|
+
exists as `tests/integration/vault-dangerous-key-blocked-install-rollback.test.ts`.
|
|
3226
|
+
It drives the real `akm bundle add` CLI end-to-end against a materialized
|
|
3227
|
+
copy of `tests/fixtures/manual-qa/dangerous-bundle/` (whose
|
|
3228
|
+
`env/runtime.env` carries `NODE_OPTIONS`), non-TTY and without
|
|
3229
|
+
`--allow-dangerous-env-keys`, and asserts every lifecycle surface after the block:
|
|
3230
|
+
**byte-level** parity of `config.json` and `akm.lock`, no surviving content
|
|
3231
|
+
root, and no orphaned index row. Three workspace states are covered —
|
|
3232
|
+
pristine, pre-existing operator config, and a bundle already installed —
|
|
3233
|
+
the last of which pins the other direction too: a rollback that reverts
|
|
3234
|
+
the FIRST bundle's records is as wrong as one that leaves the refused
|
|
3235
|
+
bundle's behind.
|
|
3236
|
+
Rollback was found already correct in every case constructible here; the
|
|
3237
|
+
test's value is as a pin, so it was mutation-verified against five separate
|
|
3238
|
+
breaks (skip the rollback; drop `removeLockEntry`; delete every bundle
|
|
3239
|
+
instead of the target; leave the content root; skip the post-rollback
|
|
3240
|
+
reindex) and fails on each.
|
|
3241
|
+
- Residue accepted by design, asserted explicitly rather than left implicit:
|
|
3242
|
+
(a) on a pristine workspace the attempt still SCAFFOLDS `config.json` and
|
|
3243
|
+
`akm.lock` — the config/lock writers create their file on first mutation —
|
|
3244
|
+
so the test pins that the created `config.json` deep-equals `DEFAULT_CONFIG`
|
|
3245
|
+
with no `bundles` key and that `akm.lock` is the empty array, rather than
|
|
3246
|
+
asserting absence; (b) the events stream keeps the `add` event for the
|
|
3247
|
+
refused ref, because it is an append-only audit log of ATTEMPTS and an
|
|
3248
|
+
operator investigating a blocked install needs it on record.
|
|
3249
|
+
- [ ] **Security:** Git/direct-write symlink containment, authenticated OpenCode
|
|
3250
|
+
listener, registry credential/control-data redaction, archive expansion
|
|
3251
|
+
budgets, universal redirect/SSRF policy, and unsuppressible update audit.
|
|
3252
|
+
- Partial: symlink containment (walker escape cases tested; not exercised
|
|
3253
|
+
through a real git-repo walk), registry control-data redaction
|
|
3254
|
+
(response-shape limits tested), redirect/SSRF policy (website fetcher has
|
|
3255
|
+
the policy; registry fetch does not apply the same private-IP guard).
|
|
3256
|
+
Registry URL credential handling is implemented under #811. Strict
|
|
3257
|
+
configured-value rejection and conservative diagnostic redaction passed
|
|
3258
|
+
independent review after seven hardening cycles; the final focused corpus
|
|
3259
|
+
covers 102 cases with 36,895 assertions.
|
|
3260
|
+
- Open: authenticated OpenCode listener (localhost listener carries no
|
|
3261
|
+
per-process credential), archive expansion budgets (download size capped;
|
|
3262
|
+
expansion ratio/member budgets untested).
|
|
3263
|
+
- Closed in #765: the unsuppressible update audit stages and re-verifies every
|
|
3264
|
+
refreshed bundle before publication; keep the rejection, approval, and
|
|
3265
|
+
rollback cases above in the release matrix.
|
|
3266
|
+
- Tracking: [#763](https://github.com/itlackey/akm/issues/763) (listener credential),
|
|
3267
|
+
[#764](https://github.com/itlackey/akm/issues/764) (archive expansion budgets),
|
|
3268
|
+
[#766](https://github.com/itlackey/akm/issues/766) (git symlink containment),
|
|
3269
|
+
[#767](https://github.com/itlackey/akm/issues/767) (registry SSRF),
|
|
3270
|
+
[#811](https://github.com/itlackey/akm/issues/811) (registry URL credentials,
|
|
3271
|
+
completed for 0.9.2).
|
|
3272
|
+
- issue: local-attack-surface and hostile-upstream hardening gaps, all
|
|
3273
|
+
pre-existing (none regressed in 0.9.0). impact: requires a local
|
|
3274
|
+
co-resident attacker, a malicious registry/source, or a compromised
|
|
3275
|
+
upstream to exploit. owner: itlackey. verification test: the checklist's
|
|
3276
|
+
own §23 items (lines cited in the per-item notes). temporary mitigation:
|
|
3277
|
+
sources/registries are operator-chosen; dangerous-env audit runs at
|
|
3278
|
+
install; opencode listener binds loopback only. waiver approver:
|
|
3279
|
+
itlackey — approved 2026-08-06 (0.9.0 release triage). waiver expiry:
|
|
3280
|
+
0.10.0 (hardening series).
|
|
3281
|
+
- [x] **Secrets/permissions:** exact-value task-log redaction; managed-file
|
|
3282
|
+
permission coverage (rejected — see below). **Row closed in 0.9.1:** both
|
|
3283
|
+
halves are resolved — the redaction gap is fixed, and permission
|
|
3284
|
+
management is a rejected concept rather than outstanding work. No waiver
|
|
3285
|
+
remains on this row.
|
|
3286
|
+
- Covered: exact-value redaction for prompt-target and workflow-target task
|
|
3287
|
+
logs (`tests/integration/tasks-runner.test.ts` "redacts echoed agent
|
|
3288
|
+
credentials before task logs are persisted", webhook-URL case).
|
|
3289
|
+
- **REJECTED, not fixed** ([#756](https://github.com/itlackey/akm/issues/756),
|
|
3290
|
+
reverted in [#791](https://github.com/itlackey/akm/issues/791)). akm briefly
|
|
3291
|
+
chmodded the databases, their `-wal`/`-shm` sidecars, and the containing
|
|
3292
|
+
data directory to owner-only on every `openManagedDatabase`, and wrote task
|
|
3293
|
+
logs `0600` into a `0700` directory. That is no longer the intended
|
|
3294
|
+
behavior and should not be re-attempted: enforcing a mode on a directory the
|
|
3295
|
+
operator owns — on the most-traveled path in the CLI, including read-only
|
|
3296
|
+
opens — silently converted legacy `0755` data dirs and broke installs
|
|
3297
|
+
sharing `$XDG_DATA_HOME` across uids, which worked in 0.9.0.
|
|
3298
|
+
**Decision: akm does not manage permissions on the data directory, the
|
|
3299
|
+
databases, or task logs.** Those take the process umask; protecting the
|
|
3300
|
+
data directory is the operator's call and umask/`chmod` is their lever.
|
|
3301
|
+
This is scoped, not blanket — akm still creates `env`/`secret` assets,
|
|
3302
|
+
config backups and scheduler invocation files at `0600`/`0700` at creation
|
|
3303
|
+
time, which is a different act from re-permissioning a directory the
|
|
3304
|
+
operator already owned. Nothing survives from that work:
|
|
3305
|
+
the health advisory that reported on those modes is gone too — akm does not
|
|
3306
|
+
nag about permissions it does not set.
|
|
3307
|
+
- **Fixed** ([#755](https://github.com/itlackey/akm/issues/755), 0.9.1).
|
|
3308
|
+
Exact-value redaction now runs in `persistRunLog`, the one sink all three
|
|
3309
|
+
target kinds funnel through, so the command arm is covered alongside the
|
|
3310
|
+
other two. Secret values come from three places, and the distinction matters:
|
|
3311
|
+
config-declared credentials and a task's own `redact:` names are redacted at
|
|
3312
|
+
any length, while values merely *inferred* from a variable's name must clear
|
|
3313
|
+
an 8-character floor. Applying the naive rule the issue proposed — treat
|
|
3314
|
+
every non-allowlisted value in the inherited environment as secret —
|
|
3315
|
+
classified 127 of 132 variables as credentials, 25 of them one character
|
|
3316
|
+
long, and rewrote `3 tests passed` into `[REDACTED] tests passed`. Pinned by
|
|
3317
|
+
`tests/tasks/log-redaction.test.ts` and the `runTask — command target` cases
|
|
3318
|
+
in `tests/integration/tasks-runner.test.ts`, including an over-redaction
|
|
3319
|
+
guard that fails if ordinary build output is ever mangled.
|
|
3320
|
+
- [ ] **Package/release:** exact package-manager upgrade version verification,
|
|
3321
|
+
native installer/scheduler coverage, action/dependency provenance hardening,
|
|
3322
|
+
and post-publication artifact parity.
|
|
3323
|
+
- **Fixed 2026-08-06** (fix-now directed at 0.9.0 triage): upgrade version
|
|
3324
|
+
verification — after a package-manager install "succeeds", `akm upgrade`
|
|
3325
|
+
now re-reads `akm --version` from PATH; a confirmed mismatch (lagging
|
|
3326
|
+
`@latest` dist-tag) reports `upgraded: false` with the exact-version pin
|
|
3327
|
+
remedy and skips `migrate apply` (which would have run against the old
|
|
3328
|
+
binary). Fail-open only when verification itself is unavailable.
|
|
3329
|
+
(`src/commands/sources/self-update.ts`; pinned in
|
|
3330
|
+
`tests/integration/self-update.test.ts`.)
|
|
3331
|
+
- **Fixed** ([#771](https://github.com/itlackey/akm/issues/771), 0.9.1):
|
|
3332
|
+
publication is gated on the real release-acceptance script.
|
|
3333
|
+
`.github/workflows/release.yml` runs `./tests/release-check.sh
|
|
3334
|
+
--skip-docker` before version-verify/build/publish, so the npm bin shim,
|
|
3335
|
+
the migration bundle under both the Bun and Node entry points, and the
|
|
3336
|
+
packed-artifact install are all checked by CI instead of depending on the
|
|
3337
|
+
operator remembering `bun run release:check` locally.
|
|
3338
|
+
- **Fixed** ([#768](https://github.com/itlackey/akm/issues/768)): every
|
|
3339
|
+
third-party `uses:` in `.github/workflows/*.yml` is now pinned to a full
|
|
3340
|
+
commit SHA (with the tag preserved as a trailing comment) instead of a
|
|
3341
|
+
mutable tag.
|
|
3342
|
+
- Partial ([#770](https://github.com/itlackey/akm/issues/770)): native
|
|
3343
|
+
scheduler macOS/Windows coverage is no longer unreachable —
|
|
3344
|
+
`.github/workflows/gated-ci.yml`'s `native-scheduler` job now runs
|
|
3345
|
+
`tests/integration/native-scheduler.test.ts` with
|
|
3346
|
+
`AKM_NATIVE_SCHEDULER_TESTS=1` on real `macos-15`/`windows-2022` runners
|
|
3347
|
+
(path-gated on PRs, full on the weekly schedule and release-candidate
|
|
3348
|
+
tags). `install.ps1` still has no execution harness — its PATH-mutation
|
|
3349
|
+
step (`[Environment]::SetEnvironmentVariable(..., "User")`) throws off
|
|
3350
|
+
Windows, so a fake-harness approach like `install-script.test.ts` cannot
|
|
3351
|
+
deterministically drive it outside a real Windows runner — but
|
|
3352
|
+
`tests/integration/install-ps1-syntax.test.ts` now statically parses it
|
|
3353
|
+
with PowerShell's own parser (runs on any host with `pwsh`, including
|
|
3354
|
+
GitHub's `ubuntu-latest`), catching syntax regressions though not
|
|
3355
|
+
semantic ones.
|
|
3356
|
+
- Open: post-publish artifact parity (nothing compares npm tarball to
|
|
3357
|
+
release assets).
|
|
3358
|
+
- Tracking: [#769](https://github.com/itlackey/akm/issues/769) (post-publish artifact parity),
|
|
3359
|
+
[#770](https://github.com/itlackey/akm/issues/770) (install.ps1 execution harness, if ever pursued on a real Windows runner).
|
|
3360
|
+
- issue: release-infrastructure hardening. impact: a clobbered release
|
|
3361
|
+
asset ships silently; tag-hijack of a third-party action could
|
|
3362
|
+
compromise the release job. owner: itlackey. verification test: per-item
|
|
3363
|
+
notes (workflow-lint for SHA pins; post-publish parity job). temporary
|
|
3364
|
+
mitigation: releases are operator-triggered (`workflow_dispatch`) with a
|
|
3365
|
+
human watching; `release:check` covers the Linux artifact end-to-end
|
|
3366
|
+
when run. waiver approver: itlackey — approved 2026-08-06 (0.9.0 release
|
|
3367
|
+
triage). waiver expiry: 0.10.0.
|
|
3368
|
+
- [ ] **Lint:** invalid type/dir fail-closed behavior, adapter type scoping,
|
|
3369
|
+
malformed task coverage, and transactional/publication-aware fixing.
|
|
3370
|
+
- **Fixed 2026-08-06** (fix-now directed at 0.9.0 triage): invalid
|
|
3371
|
+
`--type`/`--dir` fail closed — a nonexistent `--dir` and an unknown
|
|
3372
|
+
`--type` on an akm bundle (the singular/plural typo) are now usage
|
|
3373
|
+
errors instead of a false-clean `ok:true, flagged:0`
|
|
3374
|
+
(`src/commands/lint/index.ts`; pinned by
|
|
3375
|
+
`tests/integration/lint-fail-closed.test.ts`).
|
|
3376
|
+
- **Fixed for 0.9.1** (waiver discharged): all three tracked rows landed.
|
|
3377
|
+
- [#760](https://github.com/itlackey/akm/issues/760) malformed task YAML —
|
|
3378
|
+
the parse failure is now its own `invalid-task-yaml` finding instead of
|
|
3379
|
+
collapsing onto `{}` (which every task rule short-circuits on), and a
|
|
3380
|
+
`tasks/*.yaml` near miss is collected and flagged for the extension
|
|
3381
|
+
rather than skipped by the walk. Fixed on all three task-lint surfaces
|
|
3382
|
+
(the CLI sweep, the `akm` adapter's `validate`, and `akm-task`) from one
|
|
3383
|
+
shared parse in `src/tasks/schema.ts`; pinned by
|
|
3384
|
+
`tests/integration/lint-task-yaml.test.ts`.
|
|
3385
|
+
- [#761](https://github.com/itlackey/akm/issues/761) `--fix` safety —
|
|
3386
|
+
`--fix` against a bundle configured `writable: false` is now a usage
|
|
3387
|
+
error raised before any file is touched, and a per-file fix-write
|
|
3388
|
+
failure is reported in-band as `fixed: "failed"` (plus a `lint-failed`
|
|
3389
|
+
finding if the per-file dispatch throws) instead of aborting the sweep
|
|
3390
|
+
with an uncaught error that hid the fixes already on disk. Pinned by
|
|
3391
|
+
`tests/integration/lint-fix-safety.test.ts`.
|
|
3392
|
+
- [#762](https://github.com/itlackey/akm/issues/762) adapter type scoping —
|
|
3393
|
+
a `--type` passed to a non-akm bundle now emits a warning naming the
|
|
3394
|
+
flag and the adapter; findings are unchanged (validation was already a
|
|
3395
|
+
superset). Pinned in `tests/integration/lint-adapter-dispatch.test.ts`.
|
|
3396
|
+
- Also closed alongside these: [#774](https://github.com/itlackey/akm/issues/774)
|
|
3397
|
+
— `missing-skill-md` is reachable again for `agent-skills` (a real
|
|
3398
|
+
directory pass over `ValidateContext.list`, replacing the dangling
|
|
3399
|
+
`{@link directorySkillDiagnostics}` reference), and opencode's singular
|
|
3400
|
+
`skill/` alias is checked identically to `skills/`. Pinned by
|
|
3401
|
+
`tests/integration/lint-missing-skill-md.test.ts`.
|
|
3402
|
+
|
|
3403
|
+
For each unchecked row record:
|
|
3404
|
+
|
|
3405
|
+
```text
|
|
3406
|
+
issue:
|
|
3407
|
+
impact:
|
|
3408
|
+
owner:
|
|
3409
|
+
verification test:
|
|
3410
|
+
temporary mitigation:
|
|
3411
|
+
waiver approver:
|
|
3412
|
+
waiver expiry:
|
|
3413
|
+
```
|
|
3414
|
+
|
|
3415
|
+
The stale-consolidation CLI recovery surface was removed in 0.9.1; its two
|
|
3416
|
+
obsolete skipped improve-memory integration cases were deleted during the 0.9.2
|
|
3417
|
+
cleanup ([#794](https://github.com/itlackey/akm/issues/794)).
|
|
3418
|
+
Stale transaction journals remain covered through the `stale-txn-journals`
|
|
3419
|
+
health check. Do not carry historical failures into this list unless they remain
|
|
3420
|
+
reproducible against the exact candidate.
|