akm-cli 0.9.15-beta.2 → 0.9.15-beta.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +116 -3
- package/dist/cli.js +3 -3
- package/dist/commands/health/checks.js +13 -9
- package/dist/commands/health/scheduler-binary.js +120 -0
- package/dist/commands/health.js +9 -0
- package/dist/commands/improve/improve-cli.js +107 -6
- package/dist/commands/improve/improve.js +47 -3
- package/dist/commands/improve/reflect.js +159 -64
- package/dist/core/config/schema/embedding.js +2 -3
- package/dist/core/errors.js +2 -0
- package/dist/core/maintenance-barrier.js +35 -6
- package/dist/indexer/indexer.js +44 -3
- package/dist/indexer/materialize-embeddings.js +51 -9
- package/dist/llm/client.js +18 -0
- package/dist/llm/embedders/remote.js +106 -8
- package/dist/llm/usage-persist.js +7 -1
- package/dist/scripts/akm-migrate-node.js +77 -17
- package/dist/scripts/akm-migrate.js +77 -17
- package/docs/migration/release-notes/0.9.15.md +83 -0
- package/docs/reference/cli.md +60 -6
- package/docs/reference/configuration.md +55 -5
- package/package.json +1 -1
- package/schemas/akm-config.json +0 -12
|
@@ -8,6 +8,20 @@ now, retry shortly." If a script or scheduler wrapper special-cases exit 2 to
|
|
|
8
8
|
detect a held lease, switch it to exit 75, or read the JSON envelope's `code`
|
|
9
9
|
field instead.
|
|
10
10
|
|
|
11
|
+
A concurrent `akm index` now also exits 75 instead of exit 78 or exit 70. A
|
|
12
|
+
2026-09-10 field report found a second `akm index` (no `--skip-if-locked`)
|
|
13
|
+
colliding on the short internal barrier that registers the opt-in rebuild
|
|
14
|
+
lock could fail with `{"code":"INVALID_CONFIG_FILE"}` at exit 78 — a
|
|
15
|
+
config-error exit that told a supervisor to stop retrying ordinary
|
|
16
|
+
contention between two legitimate runs. That barrier now retries briefly
|
|
17
|
+
before giving up, and a busy barrier is reclassified as `TransientError`
|
|
18
|
+
code `MAINTENANCE_BARRIER_BUSY` (exit 75) instead. Contention on index.db
|
|
19
|
+
itself was already reclassified from a raw `database is locked` error (exit
|
|
20
|
+
70) to `TransientError` code `INDEX_DB_CONTENDED` (exit 75). Either way, the
|
|
21
|
+
fix for a scheduled or opportunistic index run is the same:
|
|
22
|
+
`akm index --skip-if-locked` steps aside (exit 0) instead of contending at
|
|
23
|
+
all.
|
|
24
|
+
|
|
11
25
|
The six shipped scheduled `improve` task templates now run with
|
|
12
26
|
`--require-engines`, which aborts (exit 78) before any index work when a
|
|
13
27
|
process's engine or credential cannot be resolved in the task's own
|
|
@@ -17,6 +31,23 @@ already-materialized task files are not rewritten. To get the same protection
|
|
|
17
31
|
on an existing scheduled task, add `--require-engines` to its `run:` command
|
|
18
32
|
yourself, then run `akm task sync`.
|
|
19
33
|
|
|
34
|
+
`--require-engines` now also runs a bounded reachability probe against each
|
|
35
|
+
distinct engine endpoint (the same probe `akm health` already uses), not
|
|
36
|
+
only a config/credential check. A field re-test found the flag let a run
|
|
37
|
+
through to a fully dead endpoint, which then sat silent for minutes making
|
|
38
|
+
no progress and no exit — the flag's exit-78 abort now catches that case up
|
|
39
|
+
front, naming the unreachable engine and endpoint, before any index work.
|
|
40
|
+
If a scheduled `--require-engines` run starts failing at exit 78 after
|
|
41
|
+
upgrading, check that the engine's endpoint actually answers — this is the
|
|
42
|
+
flag doing its documented job on a condition it previously missed, not a
|
|
43
|
+
new failure mode. Separately, `--timeout-ms` and an engine's own configured
|
|
44
|
+
timeout already aborted an in-flight request correctly, and SIGTERM/SIGINT
|
|
45
|
+
already ended a run within its documented grace period — both confirmed,
|
|
46
|
+
not changed, by this investigation. A live run now also prints one
|
|
47
|
+
default-level line if it waits more than a few seconds on its first engine
|
|
48
|
+
response, so a slow-but-alive run and a dead one are never indistinguishable
|
|
49
|
+
from silence alone.
|
|
50
|
+
|
|
20
51
|
`akm health --no-probe` now also skips the `cli-version` update check (a GitHub
|
|
21
52
|
release lookup), alongside the engine-reachability checks it already skipped.
|
|
22
53
|
An air-gapped or offline host's existing `--no-probe` habit now suppresses both
|
|
@@ -38,6 +69,17 @@ scheduled or opportunistic run step aside instead of contending with a rebuild
|
|
|
38
69
|
already in progress; the shipped `index-refresh` scheduled task already passes
|
|
39
70
|
it.
|
|
40
71
|
|
|
72
|
+
`embedding.maxTokens`'s default (the per-request token budget) is now 6000,
|
|
73
|
+
down from 8000: a field report on an 8192-token llama.cpp embedder showed the
|
|
74
|
+
4-chars-per-token estimator undercounts dense technical text by 7-55%, so the
|
|
75
|
+
old default regularly overshot the endpoint's real context window. If you
|
|
76
|
+
already set `embedding.maxTokens` explicitly, this default change does not
|
|
77
|
+
affect you — your configured value is unchanged. `akm index` also now
|
|
78
|
+
recovers automatically within a run: on the first request rejected for
|
|
79
|
+
exceeding the endpoint's context window, it lowers its effective budget for
|
|
80
|
+
the rest of that run (reported with one line) rather than continuing to hit
|
|
81
|
+
the same wall on every following batch.
|
|
82
|
+
|
|
41
83
|
`embedding.concurrency` (positive integer, 1-16) overrides the number of
|
|
42
84
|
embedding requests kept in flight at once, which otherwise defaults to 1 for
|
|
43
85
|
a loopback endpoint and 2 for a remote one. Set it only for an endpoint that
|
|
@@ -103,6 +145,40 @@ instead of ever failing a whole batch over one oversized document.
|
|
|
103
145
|
set `contextLength` specifically to control request batching (not your
|
|
104
146
|
Ollama server's context window), set `embedding.maxTokens` instead.
|
|
105
147
|
|
|
148
|
+
**Which token knob fixed the original 8k-context overflow.** A 0.9.15-beta
|
|
149
|
+
field report described documents estimated under the request budget that
|
|
150
|
+
still tokenized to 8.5k-12.4k real tokens against an 8192-token endpoint,
|
|
151
|
+
because the 4-chars-per-token estimator undercounts dense technical text.
|
|
152
|
+
`maxInputTokens`, `maxTokens`, and `contextLength` are easy to confuse, and
|
|
153
|
+
only one of them makes that overflow structurally unreachable:
|
|
154
|
+
|
|
155
|
+
- `embedding.maxInputTokens` (default `512`) is the per-DOCUMENT cap. It
|
|
156
|
+
truncates a document's embedded text to its head before the document is
|
|
157
|
+
ever counted toward a request, so no single document can contribute more
|
|
158
|
+
than 512 estimated tokens. This is the fix: it makes the original
|
|
159
|
+
single-document overflow structurally unreachable, independent of the
|
|
160
|
+
other two knobs.
|
|
161
|
+
- `embedding.maxTokens` (default `6000`) is the per-REQUEST budget — how
|
|
162
|
+
many already-capped documents fit in one HTTP request — plus a same-run
|
|
163
|
+
adaptive shrink on the first context-size rejection. It reduces how often
|
|
164
|
+
a request lands near an endpoint's real limit, but a request-level budget
|
|
165
|
+
alone cannot stop one oversized document from overflowing a request.
|
|
166
|
+
- `embedding.contextLength` sets Ollama's `num_ctx` only. It no longer feeds
|
|
167
|
+
the request token budget the way it used to (the two fields used to share
|
|
168
|
+
this one value), and it has no effect at all against a non-Ollama
|
|
169
|
+
endpoint.
|
|
170
|
+
|
|
171
|
+
The field's exact 0.9.15-beta config — `contextLength: 8192` and
|
|
172
|
+
`maxTokens: 8000` — produces no 400s on 0.9.15. `maxTokens` now defaults
|
|
173
|
+
lower anyway (6000), but that is not why the overflow stopped: every
|
|
174
|
+
document is truncated to `maxInputTokens` (512 tokens) before it is counted
|
|
175
|
+
toward any request, so the 8.5k-12.4k-token documents that used to overflow
|
|
176
|
+
an 8192-token endpoint can no longer reach the request budget in the first
|
|
177
|
+
place. Set `embedding.maxInputTokens` higher only if you need documents
|
|
178
|
+
longer than ~2000 characters embedded in full — for a corpus with such
|
|
179
|
+
documents, size `embedding.maxTokens` to still fit the worst case, or the
|
|
180
|
+
overflow risk returns.
|
|
181
|
+
|
|
106
182
|
`akm index --full` and an index-generation bump no longer re-embed
|
|
107
183
|
unchanged content: vectors about to be discarded are salvaged and handed
|
|
108
184
|
back to unchanged entries at the start of the next embedding pass instead
|
|
@@ -131,3 +207,10 @@ effective config and another config file or bundle-relative file. `akm config
|
|
|
131
207
|
unset` now refuses to unset a key whose value comes only from an
|
|
132
208
|
`extends`-inherited base, naming the source, since there would be nothing local
|
|
133
209
|
to remove.
|
|
210
|
+
|
|
211
|
+
The scheduler runs the binary path `akm task sync` recorded at sync time, not
|
|
212
|
+
whichever akm your shell now resolves to. After upgrading akm through a
|
|
213
|
+
different installer than the one active at your last `task sync` (e.g.
|
|
214
|
+
npm-global to a standalone download), run `akm task sync` again so the
|
|
215
|
+
schedule points at the new binary; `akm health --probe` now warns via a new
|
|
216
|
+
`scheduler-binary` advisory when the two diverge.
|
package/docs/reference/cli.md
CHANGED
|
@@ -117,7 +117,7 @@ Every command exits with one of the following codes:
|
|
|
117
117
|
| 2 | Usage / bad input | `UsageError` |
|
|
118
118
|
| 4 | Health warning (`akm health` only) | — |
|
|
119
119
|
| 70 | Internal / unclassified error | unexpected throw |
|
|
120
|
-
| 75 | Transient — retry shortly (sysexits `EX_TEMPFAIL`); another akm process holds a lock or is writing `state.db` right now, not a bad command line | `TransientError` |
|
|
120
|
+
| 75 | Transient — retry shortly (sysexits `EX_TEMPFAIL`); another akm process holds a lock or is writing `state.db` or `index.db` right now, not a bad command line | `TransientError` |
|
|
121
121
|
| 78 | Configuration error | `ConfigError` |
|
|
122
122
|
|
|
123
123
|
Failures classified by akm emit a JSON error envelope on **stderr** before
|
|
@@ -279,7 +279,19 @@ opt-in, PID-liveness-only rebuild lock and releases it on exit — this is
|
|
|
279
279
|
advisory, never the blocking lock #872 removed (see
|
|
280
280
|
[Locks](https://github.com/itlackey/akm/blob/main/docs/architecture/internals/indexing.md#locks)). A human-typed
|
|
281
281
|
`akm index` with no flag is never gated by it: if another run already holds
|
|
282
|
-
the lock, it warns and proceeds anyway, contending with the existing run.
|
|
282
|
+
the lock, it warns and proceeds anyway, contending with the existing run. If
|
|
283
|
+
that contention makes index.db genuinely busy (SQLite `database is locked`)
|
|
284
|
+
long enough to exhaust the driver's retry window, the run now fails with
|
|
285
|
+
exit 75 (`TransientError`, code `INDEX_DB_CONTENDED`) instead of the raw
|
|
286
|
+
driver error at exit 70 — the same retry-shortly contract as
|
|
287
|
+
`STATE_DB_CONTENDED`, so a scheduler can branch on it instead of alerting.
|
|
288
|
+
The rebuild lock itself is registered through a brief internal barrier
|
|
289
|
+
(`getMaintenanceBarrierPath()`) shared with every other akm lock/lease; two
|
|
290
|
+
`akm index` runs launched close enough together to collide on that
|
|
291
|
+
registration step retry briefly and then, if it is still busy, also exit 75
|
|
292
|
+
(code `MAINTENANCE_BARRIER_BUSY`) rather than the config-error exit 78 a
|
|
293
|
+
2026-09-10 field report found — a busy registration barrier is ordinary
|
|
294
|
+
contention between two legitimate runs, never a broken config file.
|
|
283
295
|
`--skip-if-locked` changes that only for the invocation that passes it: if
|
|
284
296
|
the lock is already held by a live process, it skips gracefully (exit 0,
|
|
285
297
|
`{ ok: true, skipped: { reason: "lock-held", pid, launcherPid, startedAt } }`
|
|
@@ -358,15 +370,17 @@ akm health --report --window-compare 7d --format html
|
|
|
358
370
|
| `--window-compare` | Compare the current window against the prior window of the same duration (e.g. `24h`, `7d`). With `--report`, overrides the default trend window. |
|
|
359
371
|
| `--group-by` | Group rows by `run` (one row per `improve_runs` entry). Omit for the default summary. |
|
|
360
372
|
| `--windows` | Explicit comparison window(s) as `name=...,since=ISO,until=ISO` (repeatable, up to 4). Mutually exclusive with `--window-compare`. |
|
|
361
|
-
| `--no-probe` | Skip the `default-llm-engine` / `configured-engines` reachability probes
|
|
373
|
+
| `--no-probe` | Skip the `default-llm-engine` / `configured-engines` reachability probes, the `cli-version` update check, and the `scheduler-binary` version check (for an offline or air-gapped host). |
|
|
362
374
|
|
|
363
375
|
The command reads `state.db`, verifies that the required tables exist, performs a
|
|
364
376
|
write-read probe against the events stream, inspects `task_history`, checks the
|
|
365
377
|
default agent engine, and summarizes recent `improve_*` events. Unless
|
|
366
378
|
`--no-probe` is given, it also sends a bounded (3s timeout) reachability probe
|
|
367
379
|
to the `default-llm-engine` and every `configured-engines` LLM connection (and
|
|
368
|
-
an SDK engine's LLM fallback), one probe per distinct endpoint,
|
|
369
|
-
installed akm-cli version against the latest GitHub release (`cli-version`)
|
|
380
|
+
an SDK engine's LLM fallback), one probe per distinct endpoint, checks the
|
|
381
|
+
installed akm-cli version against the latest GitHub release (`cli-version`),
|
|
382
|
+
and runs the scheduler's recorded akm binary with `--version` to check it
|
|
383
|
+
against the running CLI (`scheduler-binary`).
|
|
370
384
|
|
|
371
385
|
Primary result fields:
|
|
372
386
|
|
|
@@ -1239,6 +1253,14 @@ Shipping akm inside your own product (a Docker image, a plugin's own
|
|
|
1239
1253
|
`node_modules`)? See [Bundling akm](../integration/bundling-akm.md) for the
|
|
1240
1254
|
full boot contract, JSON shapes, and exit codes.
|
|
1241
1255
|
|
|
1256
|
+
`akm upgrade` replaces the binary in place for its own install method, but a
|
|
1257
|
+
scheduler binding recorded by an earlier `akm task sync` under a *different*
|
|
1258
|
+
install method is not repointed automatically — the scheduler runs the
|
|
1259
|
+
binary path recorded at sync time, not whichever akm `upgrade` just
|
|
1260
|
+
installed. Run `akm task sync` after switching installers so scheduled runs
|
|
1261
|
+
pick up the new binary; see [`task sync`](#task) and `akm health`'s
|
|
1262
|
+
`scheduler-binary` advisory.
|
|
1263
|
+
|
|
1242
1264
|
### clone
|
|
1243
1265
|
|
|
1244
1266
|
Copy an asset from any source into a managed writable bundle or an unmanaged
|
|
@@ -2321,6 +2343,7 @@ akm improve --require-engines # for scheduled runs: abort (exit 78) ins
|
|
|
2321
2343
|
akm improve --no-sync # skip the end-of-run git commit entirely (default: on for git-backed bundles)
|
|
2322
2344
|
akm improve --sync --no-push # commit only, skip the push after it
|
|
2323
2345
|
akm improve --plan --strategy thorough # preview thorough's resolved engine/model routing; nothing is dispatched
|
|
2346
|
+
akm improve lessons/my-lesson --show-prompt --format text # print the composed reflect prompt for one asset, unwrapped; no lock/index/engine call
|
|
2324
2347
|
akm improve report # LLM usage/routing report for the most recent real run
|
|
2325
2348
|
akm improve report --run <id> # ...for one specific improve_runs id
|
|
2326
2349
|
akm improve report --since 7d # ...aggregated over every real run started in the last 7 days
|
|
@@ -2340,7 +2363,8 @@ akm improve report --since 7d # ...aggregated over every real run start
|
|
|
2340
2363
|
| `--strategy <name>` | Override the active improve strategy (a built-in or entry under `improve.strategies`) |
|
|
2341
2364
|
| `--json-to-stdout` | Also emit the full persisted JSON result on stdout for a live run. Without this flag, stdout stays empty. Dry-runs always emit their result and are never persisted. |
|
|
2342
2365
|
| `--skip-if-locked` | If another improve run already holds the lock, skip gracefully (exit 0) instead of failing with "already running" (exit 78). Use for high-frequency scheduled runs so they don't pile up failures while a longer run is in progress. |
|
|
2343
|
-
| `--require-engines` | Abort (exit 78, before any indexing, lock, or log side effect) if the active strategy would enable a process whose engine or credential cannot be resolved in this process's environment. Without this flag, improve degrades gracefully: it skips the affected processes and reports them in the result's `skippedProcesses`. Recommended alongside `--skip-if-locked` for scheduled runs, since the operator's own shell can pass config validation while a scheduler's stripped-down environment (see #953) cannot. |
|
|
2366
|
+
| `--require-engines` | Abort (exit 78, before any indexing, lock, or log side effect) if the active strategy would enable a process whose engine or credential cannot be resolved in this process's environment, OR whose endpoint fails a bounded reachability probe — the same probe `akm health`'s `default-llm-engine`/`configured-engines` checks run, once per distinct endpoint. Without this flag, improve degrades gracefully: it skips the affected processes and reports them in the result's `skippedProcesses`. Recommended alongside `--skip-if-locked` for scheduled runs, since the operator's own shell can pass config validation while a scheduler's stripped-down environment (see #953) cannot. |
|
|
2367
|
+
| `--show-prompt` | Print the composed reflect prompt (#952) for one asset and exit — before any lock, index write, or engine dispatch. Requires a fully-qualified asset ref as the scope (`akm improve lessons/my-lesson --show-prompt`); rejected with a type or whole-bundle scope. The default output format is JSON, which carries the prompt as a `prompt` field (escaped into one line) alongside the resolved `engine`/`engineKind`; pass `--format text` to print the prompt itself, unwrapped and readable by eye. |
|
|
2344
2368
|
| `--sync` / `--no-sync` | Commit (and optionally push) the git-backed primary bundle when the run finishes. Default: on for git-backed bundles (per profile config). |
|
|
2345
2369
|
| `--push` / `--no-push` | Push after the end-of-run sync commit when writable with a remote configured. `--no-push` commits only, skipping the push. Default: per profile config (`true`). `sync.push` stays outside the autonomy gate — this is a per-run opt-out, not a default change. |
|
|
2346
2370
|
|
|
@@ -2396,6 +2420,18 @@ on an unavailable credential either — even a strategy left with every process
|
|
|
2396
2420
|
disabled this way still returns its plan, with the affected processes in
|
|
2397
2421
|
`skippedProcesses`.
|
|
2398
2422
|
|
|
2423
|
+
`--timeout-ms` is a run-wide wall-clock budget: when it expires, the run
|
|
2424
|
+
cooperatively aborts any in-flight engine request (the same `AbortSignal`
|
|
2425
|
+
every LLM call already honors) instead of waiting out the engine's own,
|
|
2426
|
+
much longer, per-call timeout — the run then finishes and reports normally
|
|
2427
|
+
rather than hanging past its budget. SIGTERM/SIGINT/SIGHUP end a live run
|
|
2428
|
+
the same way, within a short bounded grace period, and the process exits
|
|
2429
|
+
with a stable per-signal code (`143`/`130`/`129`) rather than needing a
|
|
2430
|
+
`kill -9`. If a live run has waited more than a few seconds without any
|
|
2431
|
+
engine response at all, one default-level line ("Still waiting for the
|
|
2432
|
+
first engine response...") is printed so a scheduled run's log is never
|
|
2433
|
+
silently empty while an engine is slow or dead.
|
|
2434
|
+
|
|
2399
2435
|
For dry runs, `plannedRefs` is the effective post-limit work set, not every
|
|
2400
2436
|
ref in the requested scope. The `plan` object preserves both views: raw scope
|
|
2401
2437
|
size and per-gate removals, configured and effective caps, final ranked refs
|
|
@@ -2437,6 +2473,17 @@ no model or per-process notices. Neither `--dry-run` nor `--plan` probes
|
|
|
2437
2473
|
engine reachability over the network — pair with `akm health --probe` (or the
|
|
2438
2474
|
default probe-on behavior) to check whether a named engine actually answers.
|
|
2439
2475
|
|
|
2476
|
+
`--show-prompt` (#952) is the cheapest way to exercise reflect alone: it
|
|
2477
|
+
builds the exact prompt reflect would send for one asset — the same source
|
|
2478
|
+
resolution, runner selection, feedback/schema-hint/related-lesson/rejected-
|
|
2479
|
+
proposal gathering `akm improve`'s live reflect step uses — and prints it
|
|
2480
|
+
without acquiring a dispatch lease, so it never calls an engine. Add
|
|
2481
|
+
`--format text` (the default JSON/yaml envelope escapes the prompt into one
|
|
2482
|
+
line, which defeats a by-eye read) to confirm by eye that recent feedback is
|
|
2483
|
+
framed as an unverified report to investigate (never a fact to insert
|
|
2484
|
+
verbatim) and that the response contract tells the model never to emit the
|
|
2485
|
+
truncation marker or any content from outside the shown asset.
|
|
2486
|
+
|
|
2440
2487
|
When reinforced facts need promotion, `knowledge` is the higher-authority
|
|
2441
2488
|
destination than `memory`. The deterministic search ranking also prefers
|
|
2442
2489
|
`knowledge` over `memory` hits, including inferred `.derived` memories, when
|
|
@@ -2872,6 +2919,13 @@ task template (both the core set and the improve-schedule set) and asks once
|
|
|
2872
2919
|
before changing task files or scheduler state; non-interactive setup changes
|
|
2873
2920
|
neither.
|
|
2874
2921
|
|
|
2922
|
+
Because the scheduler runs the exact binary path recorded at the last `task
|
|
2923
|
+
sync`, upgrading akm through a different installer than the one active at
|
|
2924
|
+
that sync (npm-global to a standalone download, or vice versa) leaves
|
|
2925
|
+
scheduled runs invoking the old, now-stale binary — `task sync` re-resolves
|
|
2926
|
+
the current path and repoints them. `akm health --probe`'s `scheduler-binary`
|
|
2927
|
+
advisory warns when the two diverge, naming both versions.
|
|
2928
|
+
|
|
2875
2929
|
Setup reconfiguration preserves existing scheduler runtime bindings. Changing
|
|
2876
2930
|
the AKM storage path or installed runtime path therefore requires an explicit
|
|
2877
2931
|
`akm task sync --rebind`; setup does not silently migrate those entries.
|
|
@@ -396,8 +396,8 @@ unless a remote `embedding` config is provided.
|
|
|
396
396
|
`akm improve`'s memory-inference/consolidate passes when they call an
|
|
397
397
|
embedding model: `provider`, `endpoint`, `model`, `apiKey` (symbolic
|
|
398
398
|
reference, same rules as engine `apiKey`), `dimension`, `localModel`,
|
|
399
|
-
`maxInputTokens`, `maxTokens`, `batchSize`, `
|
|
400
|
-
`
|
|
399
|
+
`maxInputTokens`, `maxTokens`, `batchSize`, `contextLength`, `timeoutMs`,
|
|
400
|
+
`concurrency`, and `ollamaOptions.num_ctx`.
|
|
401
401
|
|
|
402
402
|
The knobs that bound request/document size and rate, all optional (defaults
|
|
403
403
|
apply when unset), for a remote endpoint (`src/llm/embedders/remote.ts`):
|
|
@@ -405,12 +405,46 @@ apply when unset), for a remote endpoint (`src/llm/embedders/remote.ts`):
|
|
|
405
405
|
| Key | Default | Bounds |
|
|
406
406
|
| --- | --- | --- |
|
|
407
407
|
| `embedding.maxInputTokens` | `512` | Per-DOCUMENT cap, applied before batching (#956). A document's embedded text is truncated to its head (unicode-safe) at this many estimated tokens instead of ever being skipped for size alone — a document is skipped only when its truncated head is empty. |
|
|
408
|
-
| `embedding.maxTokens` | `
|
|
408
|
+
| `embedding.maxTokens` | `6000` (`DEFAULT_TOKEN_BUDGET`) | Per-REQUEST token budget: how many (already-capped) documents' estimated tokens fit in one HTTP request. With the 512-token default document cap, a request carries about 11 documents by default. Lowered from 8000 to 6000 (#954): the 4-chars-per-token estimator undercounts dense technical text by 7-55%, so 8000 regularly overshot an 8192-token endpoint's real context window. |
|
|
409
409
|
| `embedding.batchSize` | `100` | Per-REQUEST document-COUNT safety cap, independent of the token budget — guards against many tiny documents packing an oversized request. |
|
|
410
410
|
| `embedding.contextLength` | unset | Ollama's `num_ctx` ONLY, forwarded verbatim as `options.num_ctx` on the native `/api/embed` request. Does **not** feed the request token budget above (#956) — the two used to share this one field, so setting it for the server's context window silently changed request batching too. |
|
|
411
411
|
| `embedding.timeoutMs` | `120000` (120s) | Per-request wall timeout — see below. |
|
|
412
412
|
| `embedding.concurrency` | `1` loopback / `2` remote | In-flight request window — see below. |
|
|
413
413
|
|
|
414
|
+
**Which knob fixed the field's 8k-context overflow, worked examples.** A
|
|
415
|
+
0.9.15-beta field report described documents estimated under the request
|
|
416
|
+
budget that still tokenized to 8.5k-12.4k real tokens against an
|
|
417
|
+
8192-token endpoint, because the 4-chars-per-token estimator undercounts
|
|
418
|
+
dense technical text. Three knobs changed shape between beta and this
|
|
419
|
+
release; only one of them makes that overflow structurally unreachable:
|
|
420
|
+
|
|
421
|
+
- `embedding.maxInputTokens: 512` — per-DOCUMENT cap, applied before
|
|
422
|
+
batching. Example: a 6,000-character API reference page is truncated to
|
|
423
|
+
its first ~2,000 characters (512 estimated tokens) before it is ever
|
|
424
|
+
counted toward a request. This is the fix for the original overflow: no
|
|
425
|
+
single document can contribute more than 512 estimated tokens to a
|
|
426
|
+
request, no matter how `maxTokens` or `contextLength` are set.
|
|
427
|
+
- `embedding.maxTokens: 6000` — per-REQUEST budget: how many already-capped
|
|
428
|
+
documents' estimated tokens fit in one HTTP request. Example: with the
|
|
429
|
+
default 512-token document cap, a request packs about 11 documents before
|
|
430
|
+
this budget is reached and the request is sent; if the run's first
|
|
431
|
+
request is still rejected for exceeding the endpoint's real context
|
|
432
|
+
window, akm shrinks this budget to three quarters of its value (floored
|
|
433
|
+
at twice `maxInputTokens`) for every later request in the same run. A
|
|
434
|
+
request-level budget alone cannot stop one oversized document from
|
|
435
|
+
overflowing a request — only the per-document cap above does that.
|
|
436
|
+
- `embedding.contextLength: 8192` — Ollama's `num_ctx` only, forwarded
|
|
437
|
+
verbatim on a native `/api/embed` request. It has no effect on request or
|
|
438
|
+
document sizing, and no effect at all against a non-Ollama endpoint — see
|
|
439
|
+
below for why that used not to be true.
|
|
440
|
+
|
|
441
|
+
A field config of `contextLength: 8192` + `maxTokens: 8000` (the exact
|
|
442
|
+
0.9.15-beta values from the original report) produces no 400s on 0.9.15:
|
|
443
|
+
`maxInputTokens` (512, new this release) caps every document before it is
|
|
444
|
+
counted, so the original 8.5k-12.4k-token documents that overflowed the
|
|
445
|
+
8192-token endpoint can never reach the request budget in the first place —
|
|
446
|
+
independent of whatever `maxTokens` or `contextLength` are set to.
|
|
447
|
+
|
|
414
448
|
`embedding.timeoutMs` (positive integer, default `120000` — 120s) is the
|
|
415
449
|
budget for a request at the FULL token budget (`embedding.maxTokens`); a
|
|
416
450
|
local model server on a large, token-budget-bounded batch legitimately takes
|
|
@@ -421,6 +455,18 @@ dead endpoint is still detected in seconds on the common case of small
|
|
|
421
455
|
documents. Set `embedding.timeoutMs` lower to fail fast against a
|
|
422
456
|
known-fast endpoint, or higher for a slow local server on large batches.
|
|
423
457
|
|
|
458
|
+
`embedding.maxTokens` (or its default) is also a run-scoped adaptive
|
|
459
|
+
starting point, not a hard ceiling (#954): on the FIRST rejection of an
|
|
460
|
+
`akm index` run for exceeding the endpoint's context window, akm shrinks
|
|
461
|
+
the request budget to three quarters of its current value — floored at
|
|
462
|
+
twice `embedding.maxInputTokens` — for every request not yet sent, and
|
|
463
|
+
prints one line naming the new value. This never changes the rejected
|
|
464
|
+
request's own split-and-retry (below), never shrinks a second time in the
|
|
465
|
+
same run, and never grows the budget back up. Users who set
|
|
466
|
+
`embedding.maxTokens` explicitly are unaffected by the LOWERED DEFAULT
|
|
467
|
+
above but still benefit from this same-run recovery if their own value
|
|
468
|
+
turns out to be too high for the endpoint.
|
|
469
|
+
|
|
424
470
|
A request TIMEOUT (not a rejection for exceeding the context window) never
|
|
425
471
|
drops its batch immediately: field confirmation showed that once akm
|
|
426
472
|
abandons a timed-out request, the endpoint (e.g. llama-server) keeps
|
|
@@ -447,10 +493,10 @@ serves parallel requests — a local server started with a multi-slot flag
|
|
|
447
493
|
single-slot model server, which the default already protects from
|
|
448
494
|
reload-thrash. Request SIZE remains the first throughput lever regardless:
|
|
449
495
|
`embedding.batchSize` (a document-count cap, default 100) together with
|
|
450
|
-
`embedding.maxTokens` (an estimated token budget per request, default
|
|
496
|
+
`embedding.maxTokens` (an estimated token budget per request, default 6000
|
|
451
497
|
— NOT `embedding.contextLength`, see the table above) control how many
|
|
452
498
|
documents land in one request — with the default 512-token
|
|
453
|
-
`embedding.maxInputTokens` document cap, that is about
|
|
499
|
+
`embedding.maxInputTokens` document cap, that is about 11 documents,
|
|
454
500
|
taking about the same wall time as a single one against a healthy endpoint.
|
|
455
501
|
|
|
456
502
|
## Search tuning
|
|
@@ -737,3 +783,7 @@ network filesystem for the data directory and falls back to `DELETE`.
|
|
|
737
783
|
configuration using `engines`, `defaults.engine`, `defaults.llmEngine`, and
|
|
738
784
|
`improve.strategies`; AKM deliberately does not infer or rename ambiguous
|
|
739
785
|
profile identities.
|
|
786
|
+
|
|
787
|
+
`embedding.chunkSize` was never read by anything under `src/` (#954), so a
|
|
788
|
+
config that still sets it is simply ignored — it still loads, unvalidated
|
|
789
|
+
and without warning.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "akm-cli",
|
|
3
|
-
"version": "0.9.15-beta.
|
|
3
|
+
"version": "0.9.15-beta.4",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "akm (Agent Knowledge Manager) — a portable, local-first capability library for AI agents. Discover, load, share, and improve reusable skills, scripts, workflows, and knowledge across any shell-capable coding agent, including Claude Code, OpenCode, and Cursor.",
|
|
6
6
|
"keywords": [
|
package/schemas/akm-config.json
CHANGED
|
@@ -223,10 +223,6 @@
|
|
|
223
223
|
"type": "integer",
|
|
224
224
|
"exclusiveMinimum": 0
|
|
225
225
|
},
|
|
226
|
-
"chunkSize": {
|
|
227
|
-
"type": "integer",
|
|
228
|
-
"exclusiveMinimum": 0
|
|
229
|
-
},
|
|
230
226
|
"contextLength": {
|
|
231
227
|
"type": "integer",
|
|
232
228
|
"exclusiveMinimum": 0
|
|
@@ -1927,10 +1923,6 @@
|
|
|
1927
1923
|
"type": "integer",
|
|
1928
1924
|
"exclusiveMinimum": 0
|
|
1929
1925
|
},
|
|
1930
|
-
"chunkSize": {
|
|
1931
|
-
"type": "integer",
|
|
1932
|
-
"exclusiveMinimum": 0
|
|
1933
|
-
},
|
|
1934
1926
|
"contextLength": {
|
|
1935
1927
|
"type": "integer",
|
|
1936
1928
|
"exclusiveMinimum": 0
|
|
@@ -3509,10 +3501,6 @@
|
|
|
3509
3501
|
"type": "integer",
|
|
3510
3502
|
"exclusiveMinimum": 0
|
|
3511
3503
|
},
|
|
3512
|
-
"chunkSize": {
|
|
3513
|
-
"type": "integer",
|
|
3514
|
-
"exclusiveMinimum": 0
|
|
3515
|
-
},
|
|
3516
3504
|
"contextLength": {
|
|
3517
3505
|
"type": "integer",
|
|
3518
3506
|
"exclusiveMinimum": 0
|