akm-cli 0.9.16-alpha.1 → 0.9.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (147) hide show
  1. package/CHANGELOG.md +56 -132
  2. package/dist/assets/hints/cli-hints-full.md +13 -6
  3. package/dist/assets/tasks/core/index-refresh.yml +1 -1
  4. package/dist/assets/tasks/improve/akm-improve-catchup.yml +3 -6
  5. package/dist/cli/retired-commands.js +0 -4
  6. package/dist/cli/unknown-flags.js +3 -36
  7. package/dist/commands/env/env-binding.js +4 -4
  8. package/dist/commands/env/env-cli.js +3 -3
  9. package/dist/commands/improve/collapse-detector.js +2 -2
  10. package/dist/commands/improve/consolidate.js +4 -6
  11. package/dist/commands/improve/improve-cli.js +20 -15
  12. package/dist/commands/improve/reflect.js +23 -2
  13. package/dist/commands/lint/base-linter.js +9 -0
  14. package/dist/commands/lint/env-key-rules.js +2 -2
  15. package/dist/commands/proposal/propose.js +15 -1
  16. package/dist/commands/proposal/repository.js +3 -12
  17. package/dist/commands/proposal/validators/proposal-quality-validators.js +40 -3
  18. package/dist/commands/proposal/validators/proposal-validators.js +5 -4
  19. package/dist/commands/read/curate.js +44 -34
  20. package/dist/commands/read/search.js +35 -54
  21. package/dist/commands/read/show.js +21 -2
  22. package/dist/commands/registry-cli.js +5 -5
  23. package/dist/commands/sources/add-cli.js +59 -16
  24. package/dist/commands/sources/bundle-cli.js +35 -11
  25. package/dist/commands/sources/bundle-config-ops.js +30 -0
  26. package/dist/commands/sources/dangerous-env-audit.js +4 -4
  27. package/dist/commands/sources/info.js +8 -8
  28. package/dist/commands/sources/installed-stashes.js +55 -61
  29. package/dist/commands/sources/source-add.js +39 -38
  30. package/dist/commands/sources/source-manage.js +34 -12
  31. package/dist/commands/sources/stash-cli.js +111 -119
  32. package/dist/commands/sources/stash-skeleton.js +6 -3
  33. package/dist/commands/tasks/explain.js +4 -1
  34. package/dist/commands/tasks/tasks-cli.js +31 -9
  35. package/dist/commands/tasks/tasks.js +239 -194
  36. package/dist/commands/tasks/validate.js +20 -32
  37. package/dist/core/activation-policy.js +4 -4
  38. package/dist/core/adapter/adapters/akm-adapter.js +8 -35
  39. package/dist/core/adapter/adapters/akm-metadata.js +1 -11
  40. package/dist/core/adapter/execution-source.js +10 -29
  41. package/dist/core/asset/asset-placement.js +0 -35
  42. package/dist/core/config/config-schema.js +64 -8
  43. package/dist/core/config/config-sources.js +96 -2
  44. package/dist/core/config/config.js +190 -24
  45. package/dist/core/config/legacy-source-shape-shim.js +9 -0
  46. package/dist/core/config/schema/embedding.js +30 -7
  47. package/dist/core/config/schema/execution.js +23 -0
  48. package/dist/core/config/schema/experimental.js +1 -1
  49. package/dist/core/config/schema/scheduler.js +20 -0
  50. package/dist/core/config/schema/search.js +10 -12
  51. package/dist/core/config/schema/sources-bundles.js +32 -1
  52. package/dist/core/content-safety.js +52 -0
  53. package/dist/core/errors.js +2 -5
  54. package/dist/core/maintenance-barrier.js +11 -13
  55. package/dist/core/paths.js +11 -0
  56. package/dist/core/run-lock.js +2 -5
  57. package/dist/core/state/migrations.js +1 -26
  58. package/dist/core/state-db.js +27 -63
  59. package/dist/core/type-presentation.js +1 -1
  60. package/dist/core/write-source.js +13 -8
  61. package/dist/indexer/bundle-identity-guard.js +45 -8
  62. package/dist/indexer/ensure-index.js +0 -5
  63. package/dist/indexer/index-db-contention.js +56 -0
  64. package/dist/indexer/index-rebuild-lock.js +73 -0
  65. package/dist/indexer/index-written-assets.js +171 -133
  66. package/dist/indexer/indexer.js +1621 -458
  67. package/dist/indexer/lookup/adapter-concept-owner.js +5 -19
  68. package/dist/indexer/materialize-embeddings.js +785 -0
  69. package/dist/indexer/passes/dir-staleness.js +161 -0
  70. package/dist/indexer/passes/metadata.js +1 -18
  71. package/dist/indexer/scan/drain-dir.js +70 -27
  72. package/dist/indexer/search/db-search.js +89 -373
  73. package/dist/indexer/search/ranking-contributors.js +16 -21
  74. package/dist/indexer/search/ranking.js +57 -135
  75. package/dist/indexer/search/search-source.js +29 -11
  76. package/dist/integrations/agent/execution-lowering.js +3 -2
  77. package/dist/integrations/agent/execution-preparation.js +32 -1
  78. package/dist/integrations/agent/prompts.js +1 -1
  79. package/dist/integrations/agent/request-lowering.js +3 -2
  80. package/dist/llm/client.js +3 -11
  81. package/dist/llm/embedder.js +3 -10
  82. package/dist/llm/embedders/remote.js +104 -133
  83. package/dist/llm/feature-gate.js +2 -4
  84. package/dist/llm/rerank-client.js +3 -3
  85. package/dist/output/html-render.js +2 -1
  86. package/dist/output/shapes/passthrough.js +2 -1
  87. package/dist/output/stdout.js +24 -0
  88. package/dist/output/text/command-format.js +13 -19
  89. package/dist/output/text/helpers.js +1 -1
  90. package/dist/output/text/index.js +2 -5
  91. package/dist/output/text.js +4 -3
  92. package/dist/registry/resolve.js +37 -10
  93. package/dist/scripts/akm-migrate-node.js +15197 -11351
  94. package/dist/scripts/akm-migrate.js +15514 -11668
  95. package/dist/setup/semantic-assets.js +2 -2
  96. package/dist/setup/setup.js +3 -3
  97. package/dist/setup/steps/connection.js +2 -3
  98. package/dist/setup/steps/tasks.js +29 -36
  99. package/dist/sources/providers/git-install.js +17 -11
  100. package/dist/sources/providers/git-provider.js +12 -5
  101. package/dist/sources/providers/git-stash.js +38 -16
  102. package/dist/sources/snapshot-fetchers/website-ingest.js +3 -3
  103. package/dist/storage/repositories/embedding-salvage-repository.js +184 -0
  104. package/dist/storage/repositories/index-connection.js +3 -1
  105. package/dist/storage/repositories/index-entries-repository.js +68 -77
  106. package/dist/storage/repositories/index-entry-schema.js +25 -16
  107. package/dist/storage/repositories/index-fts-repository.js +263 -29
  108. package/dist/storage/repositories/index-meta-repository.js +29 -0
  109. package/dist/storage/repositories/index-schema.js +122 -115
  110. package/dist/storage/repositories/index-utility-repository.js +1 -1
  111. package/dist/storage/repositories/index-vec-repository.js +435 -22
  112. package/dist/tasks/activation-config.js +90 -0
  113. package/dist/tasks/backends/cron.js +9 -0
  114. package/dist/tasks/backends/launchd.js +1 -0
  115. package/dist/tasks/backends/schtasks.js +2 -0
  116. package/dist/tasks/embedded.js +4 -5
  117. package/dist/tasks/scheduler-binding.js +2 -2
  118. package/dist/tasks/scheduler-sync-preview.js +8 -1
  119. package/dist/tasks/scheduler-sync.js +19 -10
  120. package/dist/tasks/source/parse-task-source.js +10 -113
  121. package/dist/tasks/source/project-v4.js +2 -2
  122. package/dist/tasks/source/task-source-v4.js +4 -12
  123. package/dist/tasks/source/task-to-v3.js +4 -12
  124. package/dist/tasks/source/task-to-v4.js +40 -7
  125. package/docs/migration/README.md +1 -0
  126. package/docs/migration/release-notes/0.9.15.md +36 -34
  127. package/docs/migration/release-notes/0.9.16.md +60 -98
  128. package/docs/migration/release-notes/README.md +0 -5
  129. package/docs/migration/v0.9.1-to-v0.9.2.md +6 -9
  130. package/docs/reference/cli.md +124 -122
  131. package/docs/reference/configuration.md +137 -133
  132. package/docs/reference/data-and-telemetry.md +1 -2
  133. package/docs/reference/tasks.md +34 -29
  134. package/package.json +1 -1
  135. package/schemas/akm-config.json +170 -6
  136. package/schemas/akm-task.json +1 -2
  137. package/dist/commands/sources/index-status.js +0 -99
  138. package/dist/core/hash.js +0 -18
  139. package/dist/indexer/drain.js +0 -306
  140. package/dist/indexer/embedding-identity.js +0 -20
  141. package/dist/indexer/enrich.js +0 -260
  142. package/dist/indexer/reconcile.js +0 -890
  143. package/dist/indexer/scan/parse-file.js +0 -66
  144. package/dist/indexer/units/unit.js +0 -159
  145. package/dist/llm/embedders/provider-limits.js +0 -288
  146. package/dist/storage/repositories/files-repository.js +0 -181
  147. package/dist/storage/repositories/units-repository.js +0 -510
@@ -1,110 +1,72 @@
1
1
  Migration notes for akm v0.9.16
2
2
 
3
- The index is redesigned end to end (`docs/plans/index-redesign.md`): `akm
4
- index` is now reconcile-then-drain instead of a walk/clean/embed/finalize
5
- phase pipeline, everything derived from a file is keyed by the hash of what
6
- it was derived from, and there is exactly one text table
7
- (`unit_texts`/`units_fts`) and one vector table (`units`/`units_vec`) instead
8
- of the old entry-level and fragment-level duplicates. The derived `index.db`
9
- generation changes from v23 to v24.
3
+ Scheduled execution is now authorized by host-local config and bound to the
4
+ specific source installed under a bundle id. Existing
5
+ `scheduler.enabled` entries need a `sourceId` before the 0.9.16 runtime will
6
+ load them. Run this once after upgrading:
10
7
 
11
- **First run after upgrade.** On the first normal read (or an explicit `akm
12
- index`), akm detects the v23 cache cannot serve queries under v24 and
13
- rebuilds the derived generation: `entries` and every unit are re-derived from
14
- your files (a stat walk plus a parse of everything, since the old generation
15
- has no `files` stat cache to diff against — this is CPU-bound and fast, not
16
- network-bound). The new unit vector store starts empty: the old
17
- `embeddings`/`entries_vec` tables it replaces are not carried forward, so
18
- every unit is embedded once at your configured provider, the same as a fresh
19
- install. On a large corpus this is a real, paid cost, once per embedding
20
- model, and it is never repeated for unchanged text again:
21
- not on a rename, not on the next `akm index --full`, not on a future
22
- generation bump, only on a genuine content or model change. Search works
23
- lexically from the first minute of the rebuild and gains semantic results as
24
- the embedding queue drains in the background. `akm index status` reports the
25
- queue's progress (units with a vector for the active identity, versus
26
- still-pending) without triggering any writes. Do not run an older akm against
27
- an index this release has already rebuilt: upgrade that binary instead.
8
+ ```sh
9
+ akm migrate apply
10
+ akm task sync
11
+ ```
28
12
 
29
- **Config keys removed.** `embedding.maxInputTokens`, `embedding.maxTokens`,
30
- `embedding.batchSize`, and `embedding.contextLength` are gone from the
31
- schema entirely (0.9.15 had already stopped reading them for packing;
32
- 0.9.16 removes them outright). `akm index` packs every embedding request
33
- against the provider's own probed context window and slot count — this is
34
- unchanged from 0.9.15, only what it packs (units instead of whole entries)
35
- moved. `search.minScore` is also gone: reciprocal-rank fusion was tried and
36
- measured worse (0.918 mean vs. a 0.933 pre-redesign baseline, full table in
37
- `docs/plans/index-redesign.md`) and rejected. What ships instead scores
38
- lexical evidence by magnitude through the calibrated BM25 transform
39
- (`stableFtsScore`) and fuses it with cosine similarity at a 0.7/0.3 split,
40
- with the exact/prefix/relaxed lexical tier a hit came from ranking ahead of
41
- that fused score — the transform's realized output band turned out to be
42
- only 0.021 wide after weighting, too narrow on its own to separate a strong
43
- match from a weak one. No separate floor is applied over any of it, which
44
- is why `minScore` has nothing left to tune. A config file that still sets any of these five keys
45
- loads without error — they are simply ignored, like `embedding.chunkSize`
46
- before them — see [Retired
47
- Configuration](../../reference/configuration.md#retired-configuration).
13
+ The migrator binds each existing grant to the bundle source currently in
14
+ config, drops grants whose bundle no longer exists, and writes a config backup
15
+ before changing the file. `akm migrate status` and `akm migrate apply
16
+ --dry-run` report this as a blocking migration because silently trusting a new
17
+ source would defeat the boundary. Task frontmatter cannot enable scheduling.
18
+ Use `akm task enable <bundle//tasks/name>` or `akm task disable ...`; a plain
19
+ `akm task sync` reconciles all enabled bundles and removes installed entries
20
+ for bundles that have since been disabled.
48
21
 
49
- **`--skip-if-locked` is deprecated and does nothing.** Index runs no longer
50
- take a rebuild lock at all — every write is a short, idempotent,
51
- content-addressed transaction, and two concurrent `akm index` runs converge
52
- on the same end state instead of contending — so there is nothing left to
53
- skip around. Passing the flag prints one deprecation warning and the run
54
- proceeds exactly as an ordinary `akm index` would; it is kept only so an
55
- existing script or scheduled task does not fail on an unknown flag.
22
+ Replacing a bundle's source locator or component root invalidates its old
23
+ scheduler grants. Re-enable the reviewed task after the replacement. Normal
24
+ content updates, adapter detection, and website crawl-policy changes keep the
25
+ same source identity.
26
+ Removing a bundle revokes all grants owned by that bundle. A bundle with
27
+ `enabled: false` is inert for content reads, writes, indexing, execution, and
28
+ scheduling, and it cannot remain `defaultBundle` or `defaultWriteTarget`.
29
+ Explicit lifecycle operations such as `akm bundle update <name>` may still
30
+ refresh a disabled bundle without activating its content.
56
31
 
57
- **`--enrich` and `--re-enrich` are removed.** Plain `akm index` now always
58
- performs metadata enrichment when an LLM engine is configured (the same
59
- behavior `--enrich` used to opt into); re-enrichment of index-time LLM
60
- passes is not exposed in this slice. Both flags now fail with a `UsageError`
61
- (exit 2) naming the replacement, rather than silently doing nothing.
32
+ Two bundle ids may no longer point at the same physical content root, including
33
+ through symbolic links. This alias is not safe to rewrite automatically because
34
+ durable refs name the bundle id. Keep the id whose refs should survive and
35
+ remove the duplicate config entry; config errors name both ids and the shared
36
+ root.
62
37
 
63
- **`--clean` and `--dry-run` are removed.** Every `akm index` run already
64
- removes stale entries as part of reconcile — the same work `--clean` used to
65
- opt into — so there is no longer a distinct pass for `--dry-run` to preview.
66
- Both flags now fail with a `UsageError` (exit 2) instead of being silently
67
- accepted.
38
+ Config inheritance now has a portable-data boundary. An inherited config may
39
+ supply ordinary portable settings, including engine endpoint and model fields,
40
+ but host authority is always local. Inherited bundle/default declarations,
41
+ scheduler grants, execution policy, credentials, executable paths/arguments,
42
+ setup state, registry declarations, and write-capable strategy hooks are
43
+ ignored with a warning. Bundle-relative `extends` paths are checked against
44
+ the bundle's physical root, including every link in an extends chain.
68
45
 
69
- **`akm index`'s result envelope renames two fields and drops one.**
70
- `generatedMetadata` is now `entriesUpserted`: it always counted the files
71
- reconcile added or changed, never LLM-generated metadata, so a script reading
72
- it as enrichment coverage was being misled. `directoriesScanned` is now
73
- `sourcesScanned`, because reconcile is a flat per-file stat walk with no
74
- directory granularity left to count, and it reports configured source roots.
75
- `directoriesSkipped` is gone: it could only ever be `0` under the new walk.
76
- The text output's summary line follows the same change. A script reading the
77
- old names gets `undefined` and should switch to the new ones.
46
+ Commands and personas can no longer set `workspace`, `environment`, or
47
+ `runtime` in frontmatter. Those values select host execution context and now
48
+ produce a configuration error. Asset-requested tools are constrained by the
49
+ local allowlist:
78
50
 
79
- **`akm index status` is new.** A cheap, read-only snapshot — files tracked,
80
- entries, distinct units, how many have a vector for the active embedding
81
- identity, the identity itself, and the last reconcile/build times — with no
82
- writes. It mirrors `akm info`'s handling of a missing or unreadable index: an
83
- absent index reads as the ordinary first-run state, and one that exists but
84
- cannot be read is reported, never silently presented as empty.
51
+ ```json
52
+ {
53
+ "execution": {
54
+ "allowedTools": ["read_file", "search"]
55
+ }
56
+ }
57
+ ```
85
58
 
86
- **Exit codes are unchanged.** `INDEX_DB_CONTENDED` (exit 75) already existed
87
- and still means the same thing — a second connection held a write
88
- transaction long enough to exhaust SQLite's own busy timeout — but it now
89
- covers genuinely rare contention instead of also standing in for the old
90
- rebuild lock's own failure modes, which no longer exist because the rebuild
91
- lock itself is gone.
59
+ Use `"*"` only when every asset the host may execute is trusted. If an asset
60
+ requests tools that the selected transport cannot enforce, akm fails before
61
+ dispatch instead of sending an over-privileged request.
92
62
 
93
- **The optional cross-encoder rerank pass moves from `akm curate` to `akm
94
- search`.** It was always meant for search; `search.curateRerank` (0.9.15,
95
- #951) is renamed `search.rerank` with no compatibility alias — the old key
96
- shipped hours earlier and default-off, so nobody has it meaningfully set.
97
- `akm curate` no longer reranks at all.
63
+ The old combined `--allow-insecure` switch has been removed. The two unrelated
64
+ decisions are now explicit:
98
65
 
99
- **`"ready-js"` is retired from `akm info`'s `semanticSearch.status` and from
100
- `akm index`'s verification status.** It named a pure-JS cosine-similarity
101
- fallback for when the `sqlite-vec` extension was unavailable; the new unit
102
- vector store has no BLOB-table fallback to fall back to, so nothing produces
103
- that value any more. The two surfaces now diverge in what replaces it:
104
- `akm info`'s `semanticSearch.status` reports `"pending"` until the
105
- embedding queue finishes draining and `"ready-vec"` once it has — a host
106
- without the extension can never reach `"ready-vec"` (`vecAvailable` false
107
- rules it out) and stays `"pending"` indefinitely; that status has no
108
- `"blocked"` value at all. `akm index`'s own verification status reports the
109
- same `"pending"`/`"ready-vec"`, plus `"blocked"` when a run makes no
110
- embedding progress at all.
66
+ - `--allow-insecure-transport` permits a reviewed plain-HTTP bundle or
67
+ registry endpoint.
68
+ - `--allow-dangerous-env-keys` permits reviewed dangerous environment-key
69
+ findings during bundle add/update or env activation.
70
+
71
+ Update scripts to use the narrow flag that matches the risk being accepted.
72
+ Neither flag implies the other.
@@ -7,11 +7,6 @@ live one level up in `docs/migration/`.
7
7
 
8
8
  ## Available notes
9
9
 
10
- - [0.9.16](0.9.16.md) — the index redesign: `akm index` is now
11
- reconcile-then-drain over one content-addressed text table and one vector
12
- table, index generation v23-to-v24, five retired config keys, `--enrich`/
13
- `--re-enrich`/`--clean`/`--dry-run` removed, `--skip-if-locked` now a
14
- no-op, and the new `akm index status`
15
10
  - [0.9.15](0.9.15.md) — exit-code 75 for lease/state.db contention,
16
11
  `--require-engines` scheduled task templates, `--no-probe` cli-version
17
12
  skip, thinking-control wire forms, embedding re-embed safety and
@@ -104,7 +104,6 @@ with:
104
104
  content: Review the execution contract.
105
105
  schedule:
106
106
  - cron: "15 4 * * 1"
107
- enabled: false
108
107
  engine: reviewer
109
108
  model: exact-model-id
110
109
  timeout: 45000
@@ -134,12 +133,11 @@ earlier 0.9.x releases and are summarized further down):
134
133
  parses, runs with `akm task run`, and is silently skipped by
135
134
  `akm task sync` (zero bindings, zero failures) — it never has to declare
136
135
  a trigger just to be a valid document.
137
- - **`akm.enabled` becomes per-schedule-binding.** v3's single
138
- document-level `enabled: false` becomes that schedule entry's own
139
- `enabled: false` in v4 — there is no longer a document-level flag at all.
140
- A disabled task with no cron trigger (only `on.workflow_dispatch`) has no
141
- v4 representation and is `blocked` for manual review (see
142
- [The migration procedure](#the-migration-procedure)).
136
+ - **Source-owned enablement is removed.** Current task source v4 carries no
137
+ `enabled` field. `akm-migrate` removes v2/v3/v4 source flags and seeds the
138
+ host-local `scheduler.enabled` allow-list only from native scheduler
139
+ bindings it can prove are currently enabled. A source cannot activate
140
+ itself merely by being installed from a bundle.
143
141
  - **Typed `inputs:` with defaults and `required:`.** v4 tasks can declare
144
142
  named, bounded-JSON-Schema parameters, each optionally carrying a
145
143
  `default` or `required: true` (mutually exclusive). `akm task run`
@@ -224,7 +222,7 @@ by hand using this field mapping:
224
222
  |---|---|
225
223
  | `command:` (array, argv style) | `run:` (string) plus `shell:` |
226
224
  | `timeoutMs:` | `timeout:` |
227
- | `enabled:` (document level) | removed — use `schedule:` as a list of `{cron, enabled}` entries |
225
+ | `enabled:` (document level) | removed — use host-local `scheduler.enabled` (`akm task enable` / `disable`) |
228
226
  | `schedule:` (cron string) | still accepted as a bare string, or as the list form above |
229
227
 
230
228
  validate it, and rerun the preview. You can either write the replacement
@@ -248,7 +246,6 @@ Common blocked reasons and what to do about each:
248
246
  | `github-action-target-removed` | The task's `uses:` is a GitHub Action locator (`owner/repo[/path]@ref`); that spelling has no task source v4 equivalent. | Rewrite the target as `commands/`, `scripts/`, `workflows/`, or `akm/command` by hand. |
249
247
  | `with-on-non-command-target` | A `with:` block is authored on a target other than `uses: akm/command`. | Declare `inputs:` on the task instead; a workflow step composing it binds them with its own `with:`. |
250
248
  | `ambiguous-scheduling-source` | The document declares both `akm.schedule` and `on:`. | Pick one; the migrator will not guess which one wins. |
251
- | `enabled-false-has-no-schedule-entry` | `akm.enabled: false` with no cron trigger to attach it to (the only trigger is `on.workflow_dispatch`). | Task source v4 has no document-level `enabled` flag — decide whether the task should be scheduled (add a cron) or left manual-only (drop `akm.enabled`), then re-run. |
252
249
  | `read-only-source` | The owning source or file is not writable. | Move or re-source the file somewhere writable, or edit it by hand. |
253
250
  | `invalid-v3-task` | The v3 document itself is structurally invalid (unknown fields, missing selector, malformed trigger, etc). | Fix the underlying v3 document first — the migrator translates structure, it does not repair it. |
254
251