@aria-framework/ai 0.24.0 → 0.25.0

package/README.md

@@ -29,8 +29,7 @@ const ai = createAiClient({
     timeoutMs: 60000, maxTokens: 1024, contextTokens: 8192
   }),
   budget: { async assertWithinBudget(cfg, meta) {}, async record(cfg, result, meta) {} }, // optional
-  logger: console,                                                                       // optional
-  maxRetryAfterMs: 10000   // optional: the longest Retry-After a call will wait out (see 429 below)
+  logger: console                                                                        // optional
 });
 
 const r = await ai.complete({ system, messages, maxTokens: 400, schema /* optional */ });
@@ -55,7 +54,7 @@ to write yourself:
 
 | Contract rule | What the adapter does |
 |---|---|
-| Poll `/status` about every 2 s | One poller per `(instance, statusUrl)`, shared by every engine on that stack, with `unref()` called on it |
+| Poll `/status` about every 2 s | One poller per stack (keyed by `instance`), shared by every engine on it, `unref()`'d. Each read is bounded (`statusTimeoutMs`, default 5 s), concurrent reads share one fetch, and `pollMs` is floored at 500 ms |
 | Route only to `healthy`; `draining` gets nothing new | Selects **for** `healthy`, so a state lmx adds later can never receive work by accident |
 | Engine URLs come from the document | Resolved on every call, and never stored |
 | A status outage is not an inference outage | Keeps routing on the last good document for 3 minutes (`staleMs`), logging loudly |
@@ -63,8 +62,8 @@ to write yourself:
 | Choose the reasoning flag from the model | `qwen` → `chat_template_kwargs.enable_thinking=false`, `gpt-oss` → `reasoning_effort:'low'`, anything else gets neither (with a warning) |
 | Size requests against `maxInputTokens` (per slot) | `contextTokens` is taken from the engine, not from config |
 | Identity is `(instance, id)`, falling back to `name` | `lmx.engineId` is matched first; `rekeyPlan()` migrates rows when ids are minted or engines renamed |
-| A `429` is about the key, not the engine | Waits `Retry-After` once (up to `maxRetryAfterMs`), then throws with `lmxSkip: 'lmx_throttled'` |
-| The document's `instance` must match | A document for another instance is refused, because engine names collide across stacks |
+| A `429` is about the key, not the engine | Fails fast (since 0.25.0): throws `rate_limit` with `lmxSkip: 'lmx_throttled'` and `retryAfterMs`. The client never waits; whether to wait or move on is the caller's decision |
+| The document's `instance` must match | A document for another instance is refused, because engine names collide across stacks. `lmxVerify` then returns `engines: null`, so nothing downstream (`rekeyPlan`) can act on another stack's list |
 
 ### Ask the operator for
 
@@ -94,7 +93,8 @@ The adapter takes the status URL exactly as given; it does not append `/status`.
     ca: '-----BEGIN CERTIFICATE-----…',          // PEM to pin; null = system trust store
     engine: 'advanced',              // engine name: always stored, used for display and logs
     engineId: '7c9e6679-…',          // engine id when the stack publishes one; null before mint-ids
-    pollMs: 2000, staleMs: 180000    // optional
+    pollMs: 2000, staleMs: 180000,   // optional; pollMs is floored at 500, null/undefined = default
+    statusTimeoutMs: 5000            // optional; one status read's deadline
   }
 }
 ```
@@ -142,6 +142,14 @@ const { findEngine } = require('@aria-framework/ai');
 const engine = findEngine(doc.engines, { id: row.lmx_engine_id || null, name: row.lmx_engine });
 ```
 
+### Rotating credentials, and removing a stack
+
+The poller takes its token, certificate and intervals from the config you pass on **every** call
+(`discoveryFor` reconfigures the live poller since 0.25.0), so a rotated token or a replaced
+certificate takes effect on the next call — no restart. A changed `statusUrl` replaces the poller.
+When you delete or disable a supervisor, call
+`require('@aria-framework/ai/providers/lmx').forgetDiscovery(instance)` so its poller stops.
+
 ### Errors and routing
 
 When an lmx call cannot run, the thrown `AiError` carries `err.lmxSkip`. None of these mean the
@@ -153,7 +161,7 @@ instead:
 | `lmx_not_healthy` | draining, restarting or starting; the supervisor is doing planned work |
 | `lmx_stale` | no fresh status document, so we cannot see the stack |
 | `lmx_unknown_engine` | the engine is not in the document (removed, or hidden from this gateway key) |
-| `lmx_throttled` | a `429` on the key; the one `Retry-After` wait has already been spent |
+| `lmx_throttled` | a `429` on the gateway key. Not retried by the client; `err.retryAfterMs` says how long the gateway asked for. Every engine on the stack shares the key, so the next endpoint on the same stack will likely be throttled too |
 
 Failover across several engines for one job is the app's job; lmx provides none. List
 endpoints in preference order and take the first one that serves.
@@ -161,10 +169,9 @@ endpoints in preference order and take the first one that serves.
 ### Embeddings
 
 Call `client.embed(cfg, texts, { signal })` and pass your own resolved embedding config; it never
-falls back to `resolveConfig()`. It applies the same retry as `complete()`: a `429` with
-`Retry-After` is waited out once, and a fast dropped connection is retried once. Calling the
-provider adapter's `embed()` directly skips both, and a throttled batch then fails outright.
-Embeddings are not metered against the token budget.
+falls back to `resolveConfig()`. It applies the same policy as `complete()`: a fast dropped
+connection is retried once, a `429` fails fast with `retryAfterMs` on the error, and a config with
+`enabled: false` is refused. Embeddings are not metered against the token budget.
 
 On lmx, `embed()` puts the engine's `dimensions` on the config as `embeddingDimensions`. Record the width
 you built an index with and compare it on startup. A model swap changes the width, and an index

package/index.js

@@ -50,8 +50,6 @@ const DEFAULTS = {
 
 const RETRY_AFTER_MS = 400;
 const RETRY_ONLY_IF_FAILED_WITHIN_MS = 5000;
-/** The longest Retry-After a call will sit out before giving up on this endpoint. */
-const MAX_RETRY_AFTER_MS = 10000;
 
 const NOOP_LOGGER = { info() {}, warn() {}, error() {} };
 const NOOP_BUDGET = { async assertWithinBudget() {}, async record() {} };
@@ -67,13 +65,17 @@ function createAiClient(deps = {}) {
   }
   const log = deps.logger || NOOP_LOGGER;
   const meter = deps.budget || NOOP_BUDGET;
-  const maxRetryAfterMs = Number.isFinite(deps.maxRetryAfterMs) ? deps.maxRetryAfterMs : MAX_RETRY_AFTER_MS;
 
   /**
-   * Try once more, but only for the failures where trying again could help — a local provider that
-   * dropped the connection while loading a model, or a rate limit that said how long to wait. A
-   * cancelled call, a timeout, an open-ended rate limit or a slow failure is never retried (see the
-   * guards below).
+   * Try once more, but only for the failure where trying again could help — a local provider that
+   * dropped the connection while loading a model. A cancelled call, a timeout, a rate limit or a slow
+   * failure is never retried (see the guards below).
+   *
+   * A 429 FAILS FAST (0.25.0). 0.23/0.24 slept its Retry-After here (up to 10 s), below every app's
+   * dispatcher: every engine on an lmx stack shares one gateway key, so each failover re-hit the same
+   * throttle and paid the wait again, and an OpenAI-compatible primary that sent Retry-After delayed a
+   * healthy backup by the full wait. Decided 2026-10-01: the client does not wait. The error carries
+   * `retryAfterMs` (and `lmxSkip: 'lmx_throttled'` from an lmx engine) for a caller that chooses to.
    */
   async function withOneRetry(run, opts = {}) {
     const startedAt = Date.now();
@@ -82,17 +84,6 @@ function createAiClient(deps = {}) {
     } catch (err) {
       if (opts.signal && opts.signal.aborted) throw err;
       if (!err || !err.retryable) throw err;
-      // A 429 THAT SAYS HOW LONG. The lmx gateway's per-key limits answer with Retry-After, and its
-      // contract is explicit: the limit is on the KEY, so wait and retry rather than blaming the
-      // engine. Bounded, because a request a person is waiting on cannot sit out a minute; past the
-      // cap it throws as before and the caller's routing decides.
-      if (err.kind === 'rate_limit' && Number.isFinite(err.retryAfterMs)
-          && err.retryAfterMs <= maxRetryAfterMs) {
-        log.warn(`AI: rate limited — waiting ${err.retryAfterMs}ms as asked, then trying once more`);
-        await new Promise((r) => setTimeout(r, err.retryAfterMs));
-        if (opts.signal && opts.signal.aborted) throw err;
-        return run();
-      }
       const elapsed = Date.now() - startedAt;
       // A TIMEOUT is the deadline itself being reached — retrying waits the whole deadline again. A
       // RATE LIMIT is the provider asking for less pressure. A slow `unreachable` is not the
@@ -146,12 +137,9 @@ function createAiClient(deps = {}) {
   }
 
   /**
-   * Embed one or more strings, with the same retry policy as complete().
-   *
-   * THE RETRY IS THE POINT. complete() always went through withOneRetry; embeddings were called on
-   * the adapter directly, so a gateway 429 failed the whole batch even though Retry-After said how
-   * long to wait - the README promised otherwise, and lmx's CLIENT.md requires a wait-and-retry for
-   * every request through the gateway. Embeddings are idempotent, so a resend is always safe.
+   * Embed one or more strings, with the same retry policy as complete() — a fast dropped connection
+   * is retried once; a 429 fails fast (0.25.0, see withOneRetry). Embeddings are idempotent, so the
+   * one resend is always safe.
    *
    * THE CONFIG IS THE CALLER'S. An app resolves its embedding settings separately from completion
    * (embeddingModel, often a different endpoint), so this never falls back to resolveConfig().
@@ -161,6 +149,11 @@ function createAiClient(deps = {}) {
    * @param {{signal?: AbortSignal}} [opts]
    */
   async function embed(cfg, texts, opts = {}) {
+    // EXPLICITLY DISABLED is refused, as complete() refuses it (the review found embed ignored it).
+    // Strictly `false`: an embedding config the caller built without the flag is still honoured.
+    if (cfg && cfg.enabled === false) {
+      throw new AiError('disabled', 'This endpoint is switched off.');
+    }
     const adapter = cfg && PROVIDERS[cfg.provider];
     if (!adapter) {
       throw new AiError('unconfigured', `No adapter is registered for provider "${cfg && cfg.provider}".`);

package/lmxStatus.js

@@ -155,12 +155,16 @@ const rowRef = (row) => ({ id: row.lmx_engine_id || null, name: row.lmx_engine }
  */
 function rekeyPlan(rows, engines) {
   if (!Array.isArray(engines)) return [];
+  // IDS ROWS ALREADY TRACK (0.25.0). CLIENT.md's condition for minting is "an id you have never
+  // seen". Without it, renaming row A's engine to the name a stale, id-less row B still carries
+  // handed B A's id — B's routes silently went to A's engine.
+  const held = new Set((rows || []).map((r) => r && r.lmx_engine_id).filter(Boolean));
   const plan = [];
   for (const row of rows || []) {
     if (!row) continue;
     if (!row.lmx_engine_id) {
       const e = findEngine(engines, { name: row.lmx_engine });
-      if (e && e.id) plan.push({ id: row.id, set: { lmx_engine_id: e.id }, why: 'minted' });
+      if (e && e.id && !held.has(e.id)) plan.push({ id: row.id, set: { lmx_engine_id: e.id }, why: 'minted' });
       continue;
     }
     const e = findEngine(engines, { id: row.lmx_engine_id });

package/lmxVerify.js

@@ -216,6 +216,11 @@ async function verify(o = {}) {
     if (doc && doc.instance && String(doc.instance) !== instance) {
       checks.push(fail('instance',
         `this stack reports itself as “${doc.instance}”, not “${instance}” — the address points at a different deployment`));
+      // AND ITS ENGINES ARE NOT HANDED BACK (0.25.0). The apps' scheduled refresh feeds
+      // report.engines to rekeyPlan; returning another deployment's list wrote THAT stack's engine
+      // ids onto this stack's rows — irreversibly, since a row with an id is never re-pointed.
+      // `null` is "no document for this stack", which every consumer treats as "write nothing".
+      found = null;
     }
 
     checks.push(await enginesKeyCheck(o, engines, timeoutMs));

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@aria-framework/ai",
   "description": "Aria App Framework \u2014 AI module. A dependency-injected model seam (createAiClient) over several providers (LM Studio / OpenAI-compatible / Anthropic), with a fact-preservation guard, generic Polish and Generate writing engines, and a browser polish widget. Prompts and config stay in the consuming app.",
-  "version": "0.24.0",
+  "version": "0.25.0",
   "license": "UNLICENSED",
   "private": false,
   "publishConfig": {

package/providers/lmx.js

@@ -27,16 +27,33 @@ const { createLmxDiscovery } = require('./lmxDiscovery');
 const { lmxTransport } = require('./lmxTransport');
 const { AiError } = require('../error');
 
-/** Discovery instances, keyed so several engines on one stack share a single poller. */
+/**
+ * Discovery instances, ONE PER STACK (keyed by instance), so several engines on one stack share a
+ * single poller.
+ *
+ * Keyed by instance alone since 0.25.0. Keyed by (instance, statusUrl), an edited status URL created
+ * a SECOND poller and left the first polling the old address every 2 s for the life of the process.
+ */
 const registry = new Map();
 
-const keyOf = (lmx) => `${lmx.instance}\u0000${lmx.statusUrl}`;
+/** The settings a live poller takes from the app's supervisor row, applied on every use. */
+const settingsOf = (lmx) => ({
+  token: lmx.statusToken,
+  ca: lmx.ca || null,
+  staleMs: lmx.staleMs,
+  pollMs: lmx.pollMs,
+  fetchTimeoutMs: lmx.statusTimeoutMs
+});
 
 /**
- * The discovery for this supervisor, created once and started on first use.
+ * The discovery for this supervisor, created once and started on first use — and RECONFIGURED on
+ * every use (0.25.0).
  *
- * Credentials are NOT part of the key: rotating a token should not orphan a running poller and
- * leave the old one polling with a dead credential. The live instance is updated in place instead.
+ * Credentials are not part of the key, so rotating a token does not orphan the poller; until 0.25.0
+ * nothing then applied the new token either (the comment here said "updated in place"; nothing did),
+ * so a rotated key or a replaced certificate never reached the poller and every lmx endpoint went
+ * stale until a restart. configure() applies whatever the app hands over each time — the app
+ * resolves its supervisor row per call, so the row is the source of truth.
  */
 function discoveryFor(lmx, logger) {
   if (!lmx || !lmx.instance || !lmx.statusUrl) {
@@ -44,29 +61,44 @@ function discoveryFor(lmx, logger) {
       'This endpoint is an lmx engine but no supervisor is configured for it — an engine name '
       + 'without a status listener cannot be resolved to an address.');
   }
-  const key = keyOf(lmx);
+  const key = lmx.instance;
   let d = registry.get(key);
-  if (!d) {
-    d = createLmxDiscovery({
+  if (d && d.status().statusUrl !== lmx.statusUrl) {
+    // A NEW ADDRESS IS A NEW POLLER: the document, its age and its last error belong to the old one.
+    d.stop();
+    registry.delete(key);
+    d = null;
+  }
+  if (d) {
+    d.configure(settingsOf(lmx));
+  } else {
+    d = createLmxDiscovery(Object.assign({
       instance: lmx.instance,
       statusUrl: lmx.statusUrl,
-      token: lmx.statusToken,
-      ca: lmx.ca || null,
-      staleMs: lmx.staleMs,
-      // Was accepted from the app's supervisor row and then dropped here, so a configured poll
-      // interval never took effect. undefined falls through to POLL_MS.
-      pollMs: lmx.pollMs,
       // Test seam, the same one createLmxDiscovery already takes: it is the only way to exercise
       // complete() end-to-end without a live stack, and the cold-start bug lived exactly there.
       fetchImpl: lmx.fetchImpl,
       logger
-    });
+    }, settingsOf(lmx)));
     d.start();
     registry.set(key, d);
   }
   return d;
 }
 
+/**
+ * Stop and drop the poller for a stack — call it when the app deletes or disables a supervisor
+ * (0.25.0). Otherwise that stack's poller kept polling every 2 s for the life of the process.
+ * @returns {boolean} whether there was one to forget
+ */
+function forgetDiscovery(instance) {
+  const d = registry.get(instance);
+  if (!d) return false;
+  d.stop();
+  registry.delete(instance);
+  return true;
+}
+
 /**
  * Why a resolution failed, in the vocabulary the dispatcher classifies on.
  *
@@ -257,5 +289,5 @@ function _resetRegistry() {
 module.exports = {
   complete, embed, listModels, listModelsResult,
   apiRoot: openai.apiRoot,
-  reasoningFor, discoveryFor, _resetRegistry, SKIP_CODE
+  reasoningFor, discoveryFor, forgetDiscovery, _resetRegistry, SKIP_CODE
 };

package/providers/lmxDiscovery.js

@@ -56,6 +56,25 @@ const STALE_MS = 3 * 60 * 1000;
 /** The only state that may receive new work. */
 const HEALTHY = 'healthy';
 
+/**
+ * The fastest a poller may run (0.25.0). A poll interval of 0 — a NULL coerced, a typo — is not a
+ * faster poll, it is a busy loop against the supervisor.
+ */
+const MIN_POLL_MS = 500;
+
+/**
+ * How long one status read may take (0.25.0). There was no bound at all — only undici's 300 s
+ * defaults — so after a restart an lmx call awaited a hung listener far past its own timeout and
+ * the dispatcher never failed over. The document is small and served no-store: seconds is ample.
+ */
+const FETCH_TIMEOUT_MS = 5000;
+
+// null/undefined mean "the default" (lmxStore stores NULL for exactly that); any number — 0, a
+// typo, NaN — is floored rather than trusted.
+const floorPoll = (ms) => (ms === undefined || ms === null || ms === ''
+  ? POLL_MS
+  : Math.max(MIN_POLL_MS, Number(ms) || 0));
+
 /**
  * @param {object} opts
  * @param {string} opts.statusUrl      e.g. https://host:9443/status
@@ -63,7 +82,8 @@ const HEALTHY = 'healthy';
  * @param {string} opts.token          bearer for the status listener
  * @param {string} [opts.ca]           PEM of the certificate to pin
  * @param {number} [opts.staleMs]
- * @param {number} [opts.pollMs]
+ * @param {number} [opts.pollMs]         floored at MIN_POLL_MS
+ * @param {number} [opts.fetchTimeoutMs] one status read's deadline (default FETCH_TIMEOUT_MS)
  * @param {object} [opts.logger]
  * @param {function} [opts.fetchImpl]  injectable for tests; defaults to global fetch
  * @param {function} [opts.now]        injectable clock
@@ -72,10 +92,6 @@ function createLmxDiscovery(opts = {}) {
   const {
     statusUrl,
     instance,
-    token,
-    ca = null,
-    staleMs = STALE_MS,
-    pollMs = POLL_MS,
     logger = console,
     fetchImpl,
     now = () => Date.now()
@@ -84,10 +100,21 @@ function createLmxDiscovery(opts = {}) {
   if (!statusUrl) throw new Error('createLmxDiscovery: statusUrl is required');
   if (!instance) throw new Error('createLmxDiscovery: instance is required — identity is (instance, name)');
 
+  // MUTABLE, AND THAT IS THE 0.25.0 FIX. These were captured once for the life of the process, so a
+  // rotated token or a replaced certificate never reached the poller: Check (a fresh fetch) said
+  // "verified" while every poll got 401 and, after staleMs, every engine was skipped as stale until
+  // a restart. configure() below updates them in place; the registry calls it on every use.
+  let token = opts.token;
+  let ca = opts.ca || null;
+  let staleMs = opts.staleMs || STALE_MS;
+  let pollMs = floorPoll(opts.pollMs);
+  let fetchTimeoutMs = Number(opts.fetchTimeoutMs) || FETCH_TIMEOUT_MS;
+
   let doc = null;         // the last document that parsed and matched our instance
   let docAt = 0;
   let lastError = null;
   let timer = null;
+  let inFlight = null;    // the one refresh running now; concurrent callers share it
 
   /**
    * The pinned transport — see providers/lmxTransport.js for why this is not NODE_EXTRA_CA_CERTS
@@ -102,7 +129,9 @@ function createLmxDiscovery(opts = {}) {
     const t = transport();
     const res = await t.fetch(statusUrl, {
       headers: { Authorization: `Bearer ${token}` },
-      dispatcher: t.dispatcher
+      dispatcher: t.dispatcher,
+      // BOUNDED (0.25.0): a listener that accepts and never answers is abandoned, not awaited.
+      signal: AbortSignal.timeout(fetchTimeoutMs)
     });
     if (!res.ok) {
       const err = new Error(`status listener answered ${res.status}`);
@@ -119,7 +148,17 @@ function createLmxDiscovery(opts = {}) {
    * failure — the listener answered perfectly — so it is reported as a configuration error, which
    * is what it is. Adopting it would route `analysis` to another deployment's `analysis`.
    */
-  async function refresh() {
+  /**
+   * SINGLE-FLIGHT (0.25.0). The 2 s poller and every warming AI call each started their own read, so
+   * a hung listener stacked a new pending fetch per tick and per request. Concurrent callers now
+   * share the one in flight.
+   */
+  function refresh() {
+    if (!inFlight) inFlight = refreshOnce().finally(() => { inFlight = null; });
+    return inFlight;
+  }
+
+  async function refreshOnce() {
     try {
       const next = await fetchOnce();
       if (next && next.instance && next.instance !== instance) {
@@ -196,8 +235,28 @@ function createLmxDiscovery(opts = {}) {
     timer = null;
   }
 
+  /**
+   * Apply new settings to a live poller (0.25.0). `undefined` keeps the current value. A new poll
+   * interval restarts a running timer so it takes effect now, not after a process restart.
+   * The status URL and instance are identity, not settings: a different URL is a different poller
+   * (the registry replaces it — see providers/lmx.js).
+   */
+  function configure(next = {}) {
+    if (next.token !== undefined) token = next.token;
+    if (next.ca !== undefined) ca = next.ca || null;
+    if (next.staleMs !== undefined) staleMs = next.staleMs || STALE_MS;
+    if (next.fetchTimeoutMs !== undefined) fetchTimeoutMs = Number(next.fetchTimeoutMs) || FETCH_TIMEOUT_MS;
+    if (next.pollMs !== undefined) {
+      const p = floorPoll(next.pollMs);
+      if (p !== pollMs) {
+        pollMs = p;
+        if (timer) { stop(); timer = setInterval(refresh, pollMs); if (timer.unref) timer.unref(); }
+      }
+    }
+  }
+
   return {
-    start, stop, refresh, resolve, engines, engine,
+    start, stop, refresh, resolve, engines, engine, configure,
     /** The pinned transport, so ENGINE calls reach the same host over the same trust. */
     transport,
     /** For a diagnostics panel: what we know and how old it is. */
@@ -208,7 +267,9 @@ function createLmxDiscovery(opts = {}) {
       engineCount: engines().length,
       ageSec: doc ? ageSec() : null,
       stale: doc ? isStale() : true,
-      lastError
+      lastError,
+      pollMs,
+      running: !!timer
     })
   };
 }
@@ -227,4 +288,4 @@ function findEngine(list, ref) {
   return all.find((e) => e && e.name === r.name) || null;
 }
 
-module.exports = { createLmxDiscovery, findEngine, POLL_MS, STALE_MS, HEALTHY };
+module.exports = { createLmxDiscovery, findEngine, POLL_MS, STALE_MS, HEALTHY, MIN_POLL_MS, FETCH_TIMEOUT_MS };

package/providers/openai-compatible.js

@@ -404,6 +404,14 @@ async function embed(cfg, texts) {
   } finally {
     clearTimeout(timer);
   }
+  // THE STATUS FIRST, the body second — the order complete() already had. Parsing first turned a
+  // 429 (often an empty body) into "not JSON", and a 429 with a JSON body into "could not embed",
+  // both `bad_response`: never `rate_limit`, so client.embed's Retry-After wait (0.24.0) could not
+  // fire and an lmx gateway throttle read as a broken engine. httpError is handed the body already
+  // read — a real Response cannot be read twice.
+  if (res.status === 401 || res.status === 403 || res.status === 429) {
+    throw await httpError({ status: res.status, headers: res.headers, text: async () => raw }, label, cfg.apiKey);
+  }
   let payload;
   try { payload = JSON.parse(raw); } catch (err) {
     throw new AiError('bad_response', `${label} returned something that is not JSON from ${url}.`);