@aria-framework/ai 0.24.1 → 0.26.0

package/README.md

@@ -29,11 +29,14 @@ const ai = createAiClient({
     timeoutMs: 60000, maxTokens: 1024, contextTokens: 8192
   }),
   budget: { async assertWithinBudget(cfg, meta) {}, async record(cfg, result, meta) {} }, // optional
-  logger: console,                                                                       // optional
-  maxRetryAfterMs: 10000   // optional: the longest Retry-After a call will wait out (see 429 below)
+  logger: console                                                                        // optional
 });
 
-const r = await ai.complete({ system, messages, maxTokens: 400, schema /* optional */ });
+const r = await ai.complete({
+  system, messages, maxTokens: 400,
+  schema,                 // optional: constrained JSON
+  // reasoning: 'full',   // optional, lmx only: let the model think fully (see Reasoning below)
+});
 // r.text, r.json (when schema), r.model, r.usage.total, r.ms
 ```
 
@@ -55,16 +58,16 @@ to write yourself:
 
 | Contract rule | What the adapter does |
 |---|---|
-| Poll `/status` about every 2 s | One poller per `(instance, statusUrl)`, shared by every engine on that stack, with `unref()` called on it |
+| Poll `/status` about every 2 s | One poller per stack (keyed by `instance`), shared by every engine on it, `unref()`'d. Each read is bounded (`statusTimeoutMs`, default 5 s), concurrent reads share one fetch, and `pollMs` is floored at 500 ms |
 | Route only to `healthy`; `draining` gets nothing new | Selects **for** `healthy`, so a state lmx adds later can never receive work by accident |
 | Engine URLs come from the document | Resolved on every call, and never stored |
 | A status outage is not an inference outage | Keeps routing on the last good document for 3 minutes (`staleMs`), logging loudly |
 | Pin the certificate; never disable verification | Uses an undici `Agent({ connect: { ca } })` per stack. Never `NODE_EXTRA_CA_CERTS`, never `rejectUnauthorized:false` |
-| Choose the reasoning flag from the model | `qwen` → `chat_template_kwargs.enable_thinking=false`, `gpt-oss` → `reasoning_effort:'low'`, anything else gets neither (with a warning) |
+| Choose the reasoning flag from the model | By default thinking is kept to a minimum: `qwen` → `chat_template_kwargs.enable_thinking=false`, `gpt-oss` → `reasoning_effort:'low'`, anything else gets neither (with a warning). `reasoning: 'full'` on the call (since 0.26.0) sends `enable_thinking=true` / `reasoning_effort:'high'` instead. The flag always overrides a raw `extra`; only the named option changes it. See [Reasoning](#reasoning-reasoning-full) |
 | Size requests against `maxInputTokens` (per slot) | `contextTokens` is taken from the engine, not from config |
 | Identity is `(instance, id)`, falling back to `name` | `lmx.engineId` is matched first; `rekeyPlan()` migrates rows when ids are minted or engines renamed |
-| A `429` is about the key, not the engine | Waits `Retry-After` once (up to `maxRetryAfterMs`), then throws with `lmxSkip: 'lmx_throttled'` |
-| The document's `instance` must match | A document for another instance is refused, because engine names collide across stacks |
+| A `429` is about the key, not the engine | Fails fast (since 0.25.0): throws `rate_limit` with `lmxSkip: 'lmx_throttled'` and `retryAfterMs`. The client never waits; whether to wait or move on is the caller's decision |
+| The document's `instance` must match | A document for another instance is refused, because engine names collide across stacks. `lmxVerify` then returns `engines: null`, so nothing downstream (`rekeyPlan`) can act on another stack's list |
 
 ### Ask the operator for
 
@@ -94,7 +97,8 @@ The adapter takes the status URL exactly as given; it does not append `/status`.
     ca: '-----BEGIN CERTIFICATE-----…',          // PEM to pin; null = system trust store
     engine: 'advanced',              // engine name: always stored, used for display and logs
     engineId: '7c9e6679-…',          // engine id when the stack publishes one; null before mint-ids
-    pollMs: 2000, staleMs: 180000    // optional
+    pollMs: 2000, staleMs: 180000,   // optional; pollMs is floored at 500, null/undefined = default
+    statusTimeoutMs: 5000            // optional; one status read's deadline
   }
 }
 ```
@@ -142,6 +146,14 @@ const { findEngine } = require('@aria-framework/ai');
 const engine = findEngine(doc.engines, { id: row.lmx_engine_id || null, name: row.lmx_engine });
 ```
 
+### Rotating credentials, and removing a stack
+
+The poller takes its token, certificate and intervals from the config you pass on **every** call
+(`discoveryFor` reconfigures the live poller since 0.25.0), so a rotated token or a replaced
+certificate takes effect on the next call — no restart. A changed `statusUrl` replaces the poller.
+When you delete or disable a supervisor, call
+`require('@aria-framework/ai/providers/lmx').forgetDiscovery(instance)` so its poller stops.
+
 ### Errors and routing
 
 When an lmx call cannot run, the thrown `AiError` carries `err.lmxSkip`. None of these mean the
@@ -153,18 +165,65 @@ instead:
 | `lmx_not_healthy` | draining, restarting or starting; the supervisor is doing planned work |
 | `lmx_stale` | no fresh status document, so we cannot see the stack |
 | `lmx_unknown_engine` | the engine is not in the document (removed, or hidden from this gateway key) |
-| `lmx_throttled` | a `429` on the key; the one `Retry-After` wait has already been spent |
+| `lmx_throttled` | a `429` on the gateway key. Not retried by the client; `err.retryAfterMs` says how long the gateway asked for. Every engine on the stack shares the key, so the next endpoint on the same stack will likely be throttled too |
 
 Failover across several engines for one job is the app's job; lmx provides none. List
 endpoints in preference order and take the first one that serves.
 
+### Reasoning (`reasoning: 'full'`)
+
+By default the adapter makes the model think as little as its family allows, so a quick answer
+stays quick. Pass `reasoning: 'full'` on a `complete()` call to let the model think fully:
+
+```js
+const r = await ai.complete({
+  system, messages, schema,
+  reasoning: 'full',
+  maxTokens: 4096,        // at least 4096; see the trap below
+});                       // and give the endpoint timeoutMs: 120000 or more
+```
+
+**When to use it.** Use it for judgement tasks where the answer depends on weighing evidence,
+such as alert triage. Do not use it for quick summaries, rewrites or extraction. They do not get
+better, only slower.
+
+**What it costs.** It is much slower, roughly 30–45 s per answer on lab hardware instead of a few
+seconds, and it spends many more tokens, because the thinking is generated and billed like any
+other output. Budget for both.
+
+**The trap.** Thinking comes out of `maxTokens`. If the limit is too small, the model spends all
+of it thinking and returns **an empty answer, with no error from the server** (`finish_reason:
+"length"`). The adapter turns that, and a reply cut off inside an unclosed `<think>` block, into
+a `bad_response` AiError saying it ran out of room (and, when the server says so, that the
+tokens went on internal reasoning), but the work is lost either way. Use **`maxTokens` ≥ 4096** and a
+**`timeoutMs` ≥ 120 s** (120000) on the endpoint, or the deadline cuts the answer off first.
+
+**What is sent, per model family** (read from the model the engine is running on each call):
+
+| Family | Default (no `reasoning`) | `reasoning: 'full'` |
+|---|---|---|
+| qwen | `{"chat_template_kwargs":{"enable_thinking":false}}` | `{"chat_template_kwargs":{"enable_thinking":true}}` |
+| gpt-oss | `{"reasoning_effort":"low"}` | `{"reasoning_effort":"high"}` |
+| anything else | nothing (warning logged) | nothing (warning logged that `'full'` could not be honoured) |
+
+Rules:
+
+- `'full'` is the only value. Leaving the option out (or `null`) means the default. Any other value,
+  for example `'high'`, `true` or `''`, is refused with `AiError` kind `refused` before any
+  request is made, whichever provider the endpoint uses.
+- The flag is merged **after** `extra`, so a raw `extra: { chat_template_kwargs: … }` or
+  `extra: { reasoning_effort: … }` cannot change it. Only `reasoning` can.
+- Only `lmx` honours it. On `openai-compatible`, `lmstudio` or `anthropic` the client logs a
+  warning, removes the option, and nothing reaches that provider's request body.
+- `REASONING_MODES` (package root) lists the accepted values for an app that validates its own
+  settings.
+
 ### Embeddings
 
 Call `client.embed(cfg, texts, { signal })` and pass your own resolved embedding config; it never
-falls back to `resolveConfig()`. It applies the same retry as `complete()`: a `429` with
-`Retry-After` is waited out once, and a fast dropped connection is retried once. Calling the
-provider adapter's `embed()` directly skips both, and a throttled batch then fails outright.
-Embeddings are not metered against the token budget.
+falls back to `resolveConfig()`. It applies the same policy as `complete()`: a fast dropped
+connection is retried once, a `429` fails fast with `retryAfterMs` on the error, and a config with
+`enabled: false` is refused. Embeddings are not metered against the token budget.
 
 On lmx, `embed()` puts the engine's `dimensions` on the config as `embeddingDimensions`. Record the width
 you built an index with and compare it on startup. A model swap changes the width, and an index