akm-cli 0.9.15-beta.1 → 0.9.15-beta.3

package/docs/reference/configuration.md

@@ -396,21 +396,74 @@ unless a remote `embedding` config is provided.
 `akm improve`'s memory-inference/consolidate passes when they call an
 embedding model: `provider`, `endpoint`, `model`, `apiKey` (symbolic
 reference, same rules as engine `apiKey`), `dimension`, `localModel`,
-`maxTokens`, `batchSize`, `chunkSize`, `contextLength`, and
-`ollamaOptions.num_ctx`.
-
-`akm index` keeps a small, fixed number of `/v1/embeddings` requests in
-flight at once (a remote endpoint only; the local transformer path is
-unaffected): `1` for a loopback endpoint (`localhost`, `127.0.0.0/8`, etc. —
-a local model server serves one inference at a time, and parallel requests
-thrash it) and `2` for a remote one. This width is not configurable. The
-actual throughput knob is request SIZE, not request count: `embedding.batchSize`
-(a document-count cap, default 100) together with `embedding.maxTokens` /
-`embedding.contextLength` (an estimated token budget per request, default
-8000) control how many documents land in one request — a batch of 16-32
-documents takes about the same wall time as a single one against a healthy
-endpoint, so growing the batch is where most of the win is, not adding more
-concurrent requests.
+`maxInputTokens`, `maxTokens`, `batchSize`, `contextLength`, `timeoutMs`,
+`concurrency`, and `ollamaOptions.num_ctx`.
+
+The knobs that bound request/document size and rate, all optional (defaults
+apply when unset), for a remote endpoint (`src/llm/embedders/remote.ts`):
+
+| Key | Default | Bounds |
+| --- | --- | --- |
+| `embedding.maxInputTokens` | `512` | Per-DOCUMENT cap, applied before batching (#956). A document's embedded text is truncated to its head (unicode-safe) at this many estimated tokens instead of ever being skipped for size alone — a document is skipped only when its truncated head is empty. |
+| `embedding.maxTokens` | `6000` (`DEFAULT_TOKEN_BUDGET`) | Per-REQUEST token budget: how many (already-capped) documents' estimated tokens fit in one HTTP request. With the 512-token default document cap, a request carries about 11 documents by default. Lowered from 8000 to 6000 (#954): the 4-chars-per-token estimator undercounts dense technical text by 7-55%, so 8000 regularly overshot an 8192-token endpoint's real context window. |
+| `embedding.batchSize` | `100` | Per-REQUEST document-COUNT safety cap, independent of the token budget — guards against many tiny documents packing an oversized request. |
+| `embedding.contextLength` | unset | Ollama's `num_ctx` ONLY, forwarded verbatim as `options.num_ctx` on the native `/api/embed` request. Does **not** feed the request token budget above (#956) — the two used to share this one field, so setting it for the server's context window silently changed request batching too. |
+| `embedding.timeoutMs` | `120000` (120s) | Per-request wall timeout — see below. |
+| `embedding.concurrency` | `1` loopback / `2` remote | In-flight request window — see below. |
+
+`embedding.timeoutMs` (positive integer, default `120000` — 120s) is the
+budget for a request at the FULL token budget (`embedding.maxTokens`); a
+local model server on a large, token-budget-bounded batch legitimately takes
+longer than the prior fixed 30s cut off. A smaller request gets a
+proportionally smaller timeout —
+`clamp(timeoutMs × requestTokens / tokenBudget, 30000, timeoutMs)` — so a
+dead endpoint is still detected in seconds on the common case of small
+documents. Set `embedding.timeoutMs` lower to fail fast against a
+known-fast endpoint, or higher for a slow local server on large batches.
+
+`embedding.maxTokens` (or its default) is also a run-scoped adaptive
+starting point, not a hard ceiling (#954): on the FIRST rejection of an
+`akm index` run for exceeding the endpoint's context window, akm shrinks
+the request budget to three quarters of its current value — floored at
+twice `embedding.maxInputTokens` — for every request not yet sent, and
+prints one line naming the new value. This never changes the rejected
+request's own split-and-retry (below), never shrinks a second time in the
+same run, and never grows the budget back up. Users who set
+`embedding.maxTokens` explicitly are unaffected by the LOWERED DEFAULT
+above but still benefit from this same-run recovery if their own value
+turns out to be too high for the endpoint.
+
+A request TIMEOUT (not a rejection for exceeding the context window) never
+drops its batch immediately: field confirmation showed that once akm
+abandons a timed-out request, the endpoint (e.g. llama-server) keeps
+computing it anyway, so dropping it right away just grows the provider's
+queue while every following batch dies the same way. Instead akm backs off
+(5s, doubling, capped at 60s) and retries the same request once; a second
+timeout splits it in half and retries each half the same way, down to
+individual documents, and a single document that still times out is finally
+skipped (logged at the default `warn` level). After 3 consecutive failures
+at single-document size (timeout or network error), or 3 consecutive
+network errors at any size, the embedding phase stops dispatching further
+requests and reports failure — batches already committed are kept; rerun
+`akm index` once the endpoint is healthy.
+
+`akm index` keeps a small number of `/v1/embeddings` requests in flight at
+once (a remote endpoint only; the local transformer path is unaffected):
+`1` for a loopback endpoint (`localhost`, `127.0.0.0/8`, etc. — a local
+model server serves one inference at a time, and parallel requests thrash
+it) and `2` for a remote one, unless `embedding.concurrency` (positive
+integer, 1-16) overrides it. This default holds for the overwhelming
+majority of setups; set the override only for an endpoint that genuinely
+serves parallel requests — a local server started with a multi-slot flag
+(llama.cpp's `--parallel N`, vLLM) — not to "speed up" an ordinary
+single-slot model server, which the default already protects from
+reload-thrash. Request SIZE remains the first throughput lever regardless:
+`embedding.batchSize` (a document-count cap, default 100) together with
+`embedding.maxTokens` (an estimated token budget per request, default 6000
+— NOT `embedding.contextLength`, see the table above) control how many
+documents land in one request — with the default 512-token
+`embedding.maxInputTokens` document cap, that is about 11 documents,
+taking about the same wall time as a single one against a healthy endpoint.
 
 ## Search tuning
 
@@ -674,6 +727,17 @@ embedding calls have since 0.9.13 (#917); resolution order for a single
 `apiKeyFile`, then `secret://<name>` — though in practice a config sets only
 one of the three per engine.
 
+`embedding.apiKey` accepts the same three forms and resolves `secret://` the
+same way, on every path that sends an embedding request: `akm index`
+(including its `bundle update` post-commit embedding pass and the targeted
+re-embed a write command like `akm remember` triggers), `akm improve`'s
+consolidate pass (memory dedup and similarity clustering), and the
+fingerprint-rename canary `akm index` runs when the embedding config
+changes. All of them build the
+provider request through the same `RemoteEmbedder`/`resolveSecret` boundary,
+so a `secret://` reference resolves identically regardless of which command
+triggered the request (#953).
+
 Use `AKM_SQLITE_JOURNAL_MODE=DELETE` or `TRUNCATE` when WAL is unavailable,
 such as on some NFS/SMB mounts. With the default `WAL` setting, AKM detects a
 network filesystem for the data directory and falls back to `DELETE`.
@@ -685,3 +749,7 @@ network filesystem for the data directory and falls back to `DELETE`.
 configuration using `engines`, `defaults.engine`, `defaults.llmEngine`, and
 `improve.strategies`; AKM deliberately does not infer or rename ambiguous
 profile identities.
+
+`embedding.chunkSize` was never read by anything under `src/` (#954), so a
+config that still sets it is simply ignored — it still loads, unvalidated
+and without warning.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "akm-cli",
-  "version": "0.9.15-beta.1",
+  "version": "0.9.15-beta.3",
   "type": "module",
   "description": "akm (Agent Knowledge Manager) — a portable, local-first capability library for AI agents. Discover, load, share, and improve reusable skills, scripts, workflows, and knowledge across any shell-capable coding agent, including Claude Code, OpenCode, and Cursor.",
   "keywords": [

package/schemas/akm-config.json

@@ -211,15 +211,15 @@
           "type": "string",
           "minLength": 1
         },
-        "maxTokens": {
+        "maxInputTokens": {
           "type": "integer",
           "exclusiveMinimum": 0
         },
-        "batchSize": {
+        "maxTokens": {
           "type": "integer",
           "exclusiveMinimum": 0
         },
-        "chunkSize": {
+        "batchSize": {
           "type": "integer",
           "exclusiveMinimum": 0
         },
@@ -236,6 +236,15 @@
             }
           },
           "additionalProperties": true
+        },
+        "timeoutMs": {
+          "type": "integer",
+          "exclusiveMinimum": 0
+        },
+        "concurrency": {
+          "type": "integer",
+          "exclusiveMinimum": 0,
+          "maximum": 16
         }
       },
       "additionalProperties": true
@@ -1902,15 +1911,15 @@
               "type": "string",
               "minLength": 1
             },
-            "maxTokens": {
+            "maxInputTokens": {
               "type": "integer",
               "exclusiveMinimum": 0
             },
-            "batchSize": {
+            "maxTokens": {
               "type": "integer",
               "exclusiveMinimum": 0
             },
-            "chunkSize": {
+            "batchSize": {
               "type": "integer",
               "exclusiveMinimum": 0
             },
@@ -1927,6 +1936,15 @@
                 }
               },
               "additionalProperties": true
+            },
+            "timeoutMs": {
+              "type": "integer",
+              "exclusiveMinimum": 0
+            },
+            "concurrency": {
+              "type": "integer",
+              "exclusiveMinimum": 0,
+              "maximum": 16
             }
           },
           "additionalProperties": true
@@ -3471,15 +3489,15 @@
           "type": "string",
           "minLength": 1
         },
-        "maxTokens": {
+        "maxInputTokens": {
           "type": "integer",
           "exclusiveMinimum": 0
         },
-        "batchSize": {
+        "maxTokens": {
           "type": "integer",
           "exclusiveMinimum": 0
         },
-        "chunkSize": {
+        "batchSize": {
           "type": "integer",
           "exclusiveMinimum": 0
         },
@@ -3496,6 +3514,15 @@
             }
           },
           "additionalProperties": true
+        },
+        "timeoutMs": {
+          "type": "integer",
+          "exclusiveMinimum": 0
+        },
+        "concurrency": {
+          "type": "integer",
+          "exclusiveMinimum": 0,
+          "maximum": 16
         }
       },
       "additionalProperties": true