@axlsdk/axl 0.23.2 → 0.24.0

package/README.md

@@ -67,9 +67,9 @@ agent({ model: 'deepseek:deepseek-reasoner' });           // DEEPSEEK_API_KEY
 agent({ model: 'ollama:llama3' });                        // local — no key, $0
 ```
 
-Presets: `openrouter`, `azure`, `xai`, `deepseek`, `mistral`, `groq`, `bedrock`, and self-hosted `ollama` / `vllm` / `lmstudio` / `llamacpp` / `sglang`. The unified `effort` knob and per-call cost tracking work across them. OpenAI Responses, Anthropic, Google, and OpenRouter accept image input without hard-coding a model catalog; the selected upstream model remains authoritative. Completed-file transcription is a separate, narrower contract. Build your own compatible provider by cloning a `ProviderProfile`; see [provider details](../../docs/providers.md) and [multimodal input](../../docs/multimodal-input.md).
+Presets: `openrouter`, `azure`, `xai`, `deepseek`, `mistral`, `groq`, `bedrock`, and self-hosted `ollama` / `vllm` / `lmstudio` / `llamacpp` / `sglang`. The unified `effort` knob and per-call cost tracking work across them. OpenAI Responses, Anthropic, Google, and OpenRouter accept image input, and OpenAI Chat Completions, Google, and OpenRouter accept general recorded-audio input, without hard-coding a model catalog; the selected upstream model remains authoritative. Completed-file transcription is a separate, narrower contract. Build your own compatible provider by cloning a `ProviderProfile`; see [provider details](../../docs/providers.md) and [multimodal input](../../docs/multimodal-input.md).
 
-Each provider also accepts an opt-in `rateLimit` (`{ maxConcurrent?, minIntervalMs?, acquireTimeoutMs? }`) for proactive client-side pacing on top of the automatic 429/503/529 backoff — useful when a large fan-out (e.g. an eval) shares one API key. It caps in-flight request concurrency (not token throughput) for that provider's chat calls. See [Providers → Rate limiting](../../docs/providers.md#rate-limiting-opt-in).
+On every built-in chat provider, a rate-limit 429 pauses every call on that account and model for an exponential backoff (lengthened by a longer `Retry-After`, never shortened), retries on its own budget and paces the scope adaptively until it stops being throttled. First-party OpenAI and Anthropic (their own endpoints, not a proxy `baseUrl`) also fail a spend-cap 429 fast; elsewhere every 429 is treated as a rate limit. `rateLimit: { adaptive: false }` turns this off. Each provider also accepts an opt-in `rateLimit` (`{ maxConcurrent?, minIntervalMs?, acquireTimeoutMs?, adaptive?, maxRateLimitRetries? }`) for proactive client-side pacing on top of the automatic 429/503/529 handling — useful when a large fan-out (e.g. an eval) shares one API key. It caps in-flight request concurrency (not token throughput) for that provider's chat calls. See [Providers → Rate limiting](../../docs/providers.md#rate-limiting).
 
 Built-in providers require HTTPS for remote endpoints and never follow redirects.
 Literal loopback HTTP remains automatic for local inference; Docker service names,
@@ -585,6 +585,18 @@ const relevant = await ctx.recall('knowledge-base', {
 
 Vector store implementations: `InMemoryVectorStore` (testing), `SqliteVectorStore` (production, requires `better-sqlite3`).
 
+**Captured requests (opt-in).** Configure `diagnostics.artifacts` with a `root`
+(or your own `DiagnosticArtifactStore`) and an eval run can record the
+provider-neutral request Axl submitted for every model call — including the
+repaired message list on a retry turn and the prompt a judge built for itself.
+It is off by default, bounded per record / run / queue, redacted when
+`trace.redact` is on, and strictly separate from accounting: a capture that
+fails or hits a limit changes no number. Artifacts are committed only after the
+eval history row that owns them is saved, deleted with it, and reclaimed by a
+startup pass plus a periodic sweep. See
+[observability.md](../../docs/observability.md#captured-requests-opt-in) and
+[integration.md](../../docs/integration.md#diagnostic-artifact-storage).
+
 **Embedder cost attribution.** `OpenAIEmbedder` reports `{ tokens, cost, model }` on every embed call — computed from the response's `prompt_tokens` and a pricing table (`text-embedding-3-small` $0.02/1M, `-large` $0.13/1M, `ada-002` $0.10/1M). The cost flows through `runtime.trackExecution()` the same way agent-call cost does, counts against `ctx.budget()`, and shows up in Studio's Cost Dashboard under "Memory (Embedder)". See [observability.md](../../docs/observability.md#event-types) for the trace-event shape.
 
 **Custom `Embedder` implementations** (breaking change in 0.15.0 — `embed()` previously returned `Promise<number[][]>`):
@@ -817,6 +829,27 @@ google:gemini-3.1-pro-preview          # Google Gemini
 
 See [docs/providers.md](../../docs/providers.md) for the full model list including reasoning models.
 
+### General recorded-audio input
+
+An `InputAudioPart` puts a finite recording directly in front of a chat model so
+it can reason about speech *and* non-speech sound — the journey transcription
+cannot serve.
+
+```typescript
+await ctx.ask(listener, [
+  { type: 'audio', source: { type: 'bytes', data: wav, mediaType: 'audio/wav' } },
+  { type: 'text', text: 'What kind of sound is this?' },
+]);
+```
+
+Sources are `bytes | base64 | provider-file`; an audio URL is unrepresentable by
+construction. Audio is a per-provider opt-in: `openrouter:` is live-certified for
+a text answer, a tool continuation, and streaming; `openai:` (Chat Completions
+`input_audio`) and `google:` (Interactions) are implemented but not yet
+certified; `anthropic:`, `openai-responses:`, and every other compatible preset
+reject an audio part with zero requests and no transcription fallback. See
+[Multimodal input](../../docs/multimodal-input.md#general-recorded-audio-input).
+
 ### Completed-file transcription
 
 Use `ctx.transcribe()` for finite recordings, then explicitly pass
@@ -826,7 +859,9 @@ contracts; any nonblank `openrouter-transcription:<vendor/model>` URI can
 dispatch supported bytes/base64 to OpenRouter, where the selected model/route
 decides endpoint compatibility. See [Multimodal input](../../docs/multimodal-input.md)
 for the 25 MiB inline bound, source types, options, transcript redaction,
-temporary Gemini Files cleanup, and live-verification commands.
+temporary Gemini Files cleanup, and live-verification commands. Transcription and
+general audio input are separate products: neither is ever a hidden
+implementation of the other.
 
 ## License
 

package/dist/{chunk-NVPA4RY6.js → chunk-O7Y6WXE2.js}

@@ -1,3 +1,5 @@
+import { createRequire as __axlCreateRequire } from 'node:module';
+const require = __axlCreateRequire(import.meta.url);
 var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
   get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
 }) : x)(function(x) {
@@ -78,4 +80,4 @@ export {
   __require,
   OTelSpanManager
 };
-//# sourceMappingURL=chunk-NVPA4RY6.js.map
+//# sourceMappingURL=chunk-O7Y6WXE2.js.map

package/dist/{chunk-NVPA4RY6.js.map → chunk-O7Y6WXE2.js.map}

@@ -1 +1 @@
-{"version":3,"sources":["../src/telemetry/span-manager.ts"],"sourcesContent":["import type { SpanHandle, SpanManager, TelemetryConfig } from './types.js';\n\n/** Minimal interface for an OTel span. */\ninterface OTelSpan {\n  setAttribute(key: string, value: string | number | boolean): void;\n  addEvent(name: string, attributes?: Record<string, string | number | boolean>): void;\n  setStatus(status: { code: unknown; message?: string }): void;\n  end(): void;\n}\n\n/** Minimal interface for an OTel tracer. */\ninterface OTelTracer {\n  startActiveSpan<T>(\n    name: string,\n    options: { attributes: Record<string, string | number | boolean> },\n    fn: (span: OTelSpan) => Promise<T>,\n  ): Promise<T>;\n}\n\n/** Minimal interface for an OTel tracer provider. */\ninterface OTelTracerProvider {\n  getTracer(name: string): OTelTracer;\n  shutdown?(): Promise<void>;\n}\n\n/** Minimal interface for the @opentelemetry/api module. */\ninterface OTelApi {\n  SpanStatusCode: { UNSET: unknown; OK: unknown; ERROR: unknown };\n  trace: {\n    getTracerProvider(): OTelTracerProvider;\n    getActiveSpan?(): OTelSpan | undefined;\n  };\n}\n\n/**\n * OpenTelemetry-backed SpanManager.\n * Dynamically imports @opentelemetry/api to avoid hard dependency.\n */\nexport class OTelSpanManager implements SpanManager {\n  private tracer: OTelTracer;\n  private otelApi: OTelApi;\n  private tracerProvider: OTelTracerProvider;\n\n  private constructor(otelApi: OTelApi, tracer: OTelTracer, tracerProvider: OTelTracerProvider) {\n    this.otelApi = otelApi;\n    this.tracer = tracer;\n    this.tracerProvider = tracerProvider;\n  }\n\n  static async create(config: TelemetryConfig): Promise<OTelSpanManager> {\n    let otelApi: OTelApi;\n    try {\n      otelApi = (await import('@opentelemetry/api')) as unknown as OTelApi;\n    } catch {\n      throw new Error(\n        '@opentelemetry/api is required for telemetry. Install it with: npm install @opentelemetry/api',\n      );\n    }\n\n    const tracerProvider =\n      (config.tracerProvider as OTelTracerProvider | undefined) ??\n      otelApi.trace.getTracerProvider();\n    const serviceName = config.serviceName ?? 'axl';\n    const tracer = tracerProvider.getTracer(serviceName);\n\n    return new OTelSpanManager(otelApi, tracer, tracerProvider);\n  }\n\n  async withSpanAsync<T>(\n    name: string,\n    attributes: Record<string, string | number | boolean>,\n    fn: (span: SpanHandle) => Promise<T>,\n  ): Promise<T> {\n    const otelApi = this.otelApi;\n\n    return this.tracer.startActiveSpan(name, { attributes }, async (otelSpan: OTelSpan) => {\n      let statusWasSet = false;\n      const handle: SpanHandle = {\n        setAttribute(key: string, value: string | number | boolean) {\n          otelSpan.setAttribute(key, value);\n        },\n        addEvent(eventName: string, attrs?: Record<string, string | number | boolean>) {\n          otelSpan.addEvent(eventName, attrs);\n        },\n        setStatus(code: 'unset' | 'ok' | 'error', message?: string) {\n          statusWasSet = true;\n          const statusCode =\n            code === 'ok'\n              ? otelApi.SpanStatusCode.OK\n              : code === 'error'\n                ? otelApi.SpanStatusCode.ERROR\n                : otelApi.SpanStatusCode.UNSET;\n          otelSpan.setStatus({ code: statusCode, message });\n        },\n        end() {\n          otelSpan.end();\n        },\n      };\n\n      try {\n        const result = await fn(handle);\n        if (!statusWasSet) handle.setStatus('ok');\n        return result;\n      } catch (err) {\n        // Error text can contain provider, prompt, or sensitive tool data.\n        // Keep exported status structural; host diagnostics remain in Axl's\n        // own error and redacted event channels.\n        handle.setStatus('error');\n        throw err;\n      } finally {\n        handle.end();\n      }\n    });\n  }\n\n  addEventToActiveSpan(name: string, attributes?: Record<string, string | number | boolean>): void {\n    const activeSpan = this.otelApi.trace.getActiveSpan?.();\n    if (activeSpan) {\n      activeSpan.addEvent(name, attributes);\n    }\n  }\n\n  async shutdown(): Promise<void> {\n    if (this.tracerProvider && typeof this.tracerProvider.shutdown === 'function') {\n      await this.tracerProvider.shutdown();\n    }\n  }\n}\n"],"mappings":";;;;;;;;AAsCO,IAAM,kBAAN,MAAM,iBAAuC;AAAA,EAC1C;AAAA,EACA;AAAA,EACA;AAAA,EAEA,YAAY,SAAkB,QAAoB,gBAAoC;AAC5F,SAAK,UAAU;AACf,SAAK,SAAS;AACd,SAAK,iBAAiB;AAAA,EACxB;AAAA,EAEA,aAAa,OAAO,QAAmD;AACrE,QAAI;AACJ,QAAI;AACF,gBAAW,MAAM,OAAO,oBAAoB;AAAA,IAC9C,QAAQ;AACN,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,UAAM,iBACH,OAAO,kBACR,QAAQ,MAAM,kBAAkB;AAClC,UAAM,cAAc,OAAO,eAAe;AAC1C,UAAM,SAAS,eAAe,UAAU,WAAW;AAEnD,WAAO,IAAI,iBAAgB,SAAS,QAAQ,cAAc;AAAA,EAC5D;AAAA,EAEA,MAAM,cACJ,MACA,YACA,IACY;AACZ,UAAM,UAAU,KAAK;AAErB,WAAO,KAAK,OAAO,gBAAgB,MAAM,EAAE,WAAW,GAAG,OAAO,aAAuB;AACrF,UAAI,eAAe;AACnB,YAAM,SAAqB;AAAA,QACzB,aAAa,KAAa,OAAkC;AAC1D,mBAAS,aAAa,KAAK,KAAK;AAAA,QAClC;AAAA,QACA,SAAS,WAAmB,OAAmD;AAC7E,mBAAS,SAAS,WAAW,KAAK;AAAA,QACpC;AAAA,QACA,UAAU,MAAgC,SAAkB;AAC1D,yBAAe;AACf,gBAAM,aACJ,SAAS,OACL,QAAQ,eAAe,KACvB,SAAS,UACP,QAAQ,eAAe,QACvB,QAAQ,eAAe;AAC/B,mBAAS,UAAU,EAAE,MAAM,YAAY,QAAQ,CAAC;AAAA,QAClD;AAAA,QACA,MAAM;AACJ,mBAAS,IAAI;AAAA,QACf;AAAA,MACF;AAEA,UAAI;AACF,cAAM,SAAS,MAAM,GAAG,MAAM;AAC9B,YAAI,CAAC,aAAc,QAAO,UAAU,IAAI;AACxC,eAAO;AAAA,MACT,SAAS,KAAK;AAIZ,eAAO,UAAU,OAAO;AACxB,cAAM;AAAA,MACR,UAAE;AACA,eAAO,IAAI;AAAA,MACb;AAAA,IACF,CAAC;AAAA,EACH;AAAA,EAEA,qBAAqB,MAAc,YAA8D;AAC/F,UAAM,aAAa,KAAK,QAAQ,MAAM,gBAAgB;AACtD,QAAI,YAAY;AACd,iBAAW,SAAS,MAAM,UAAU;AAAA,IACtC;AAAA,EACF;AAAA,EAEA,MAAM,WAA0B;AAC9B,QAAI,KAAK,kBAAkB,OAAO,KAAK,eAAe,aAAa,YAAY;AAC7E,YAAM,KAAK,eAAe,SAAS;AAAA,IACrC;AAAA,EACF;AACF;","names":[]}
+{"version":3,"sources":["../src/telemetry/span-manager.ts"],"sourcesContent":["import type { SpanHandle, SpanManager, TelemetryConfig } from './types.js';\n\n/** Minimal interface for an OTel span. */\ninterface OTelSpan {\n  setAttribute(key: string, value: string | number | boolean): void;\n  addEvent(name: string, attributes?: Record<string, string | number | boolean>): void;\n  setStatus(status: { code: unknown; message?: string }): void;\n  end(): void;\n}\n\n/** Minimal interface for an OTel tracer. */\ninterface OTelTracer {\n  startActiveSpan<T>(\n    name: string,\n    options: { attributes: Record<string, string | number | boolean> },\n    fn: (span: OTelSpan) => Promise<T>,\n  ): Promise<T>;\n}\n\n/** Minimal interface for an OTel tracer provider. */\ninterface OTelTracerProvider {\n  getTracer(name: string): OTelTracer;\n  shutdown?(): Promise<void>;\n}\n\n/** Minimal interface for the @opentelemetry/api module. */\ninterface OTelApi {\n  SpanStatusCode: { UNSET: unknown; OK: unknown; ERROR: unknown };\n  trace: {\n    getTracerProvider(): OTelTracerProvider;\n    getActiveSpan?(): OTelSpan | undefined;\n  };\n}\n\n/**\n * OpenTelemetry-backed SpanManager.\n * Dynamically imports @opentelemetry/api to avoid hard dependency.\n */\nexport class OTelSpanManager implements SpanManager {\n  private tracer: OTelTracer;\n  private otelApi: OTelApi;\n  private tracerProvider: OTelTracerProvider;\n\n  private constructor(otelApi: OTelApi, tracer: OTelTracer, tracerProvider: OTelTracerProvider) {\n    this.otelApi = otelApi;\n    this.tracer = tracer;\n    this.tracerProvider = tracerProvider;\n  }\n\n  static async create(config: TelemetryConfig): Promise<OTelSpanManager> {\n    let otelApi: OTelApi;\n    try {\n      otelApi = (await import('@opentelemetry/api')) as unknown as OTelApi;\n    } catch {\n      throw new Error(\n        '@opentelemetry/api is required for telemetry. Install it with: npm install @opentelemetry/api',\n      );\n    }\n\n    const tracerProvider =\n      (config.tracerProvider as OTelTracerProvider | undefined) ??\n      otelApi.trace.getTracerProvider();\n    const serviceName = config.serviceName ?? 'axl';\n    const tracer = tracerProvider.getTracer(serviceName);\n\n    return new OTelSpanManager(otelApi, tracer, tracerProvider);\n  }\n\n  async withSpanAsync<T>(\n    name: string,\n    attributes: Record<string, string | number | boolean>,\n    fn: (span: SpanHandle) => Promise<T>,\n  ): Promise<T> {\n    const otelApi = this.otelApi;\n\n    return this.tracer.startActiveSpan(name, { attributes }, async (otelSpan: OTelSpan) => {\n      let statusWasSet = false;\n      const handle: SpanHandle = {\n        setAttribute(key: string, value: string | number | boolean) {\n          otelSpan.setAttribute(key, value);\n        },\n        addEvent(eventName: string, attrs?: Record<string, string | number | boolean>) {\n          otelSpan.addEvent(eventName, attrs);\n        },\n        setStatus(code: 'unset' | 'ok' | 'error', message?: string) {\n          statusWasSet = true;\n          const statusCode =\n            code === 'ok'\n              ? otelApi.SpanStatusCode.OK\n              : code === 'error'\n                ? otelApi.SpanStatusCode.ERROR\n                : otelApi.SpanStatusCode.UNSET;\n          otelSpan.setStatus({ code: statusCode, message });\n        },\n        end() {\n          otelSpan.end();\n        },\n      };\n\n      try {\n        const result = await fn(handle);\n        if (!statusWasSet) handle.setStatus('ok');\n        return result;\n      } catch (err) {\n        // Error text can contain provider, prompt, or sensitive tool data.\n        // Keep exported status structural; host diagnostics remain in Axl's\n        // own error and redacted event channels.\n        handle.setStatus('error');\n        throw err;\n      } finally {\n        handle.end();\n      }\n    });\n  }\n\n  addEventToActiveSpan(name: string, attributes?: Record<string, string | number | boolean>): void {\n    const activeSpan = this.otelApi.trace.getActiveSpan?.();\n    if (activeSpan) {\n      activeSpan.addEvent(name, attributes);\n    }\n  }\n\n  async shutdown(): Promise<void> {\n    if (this.tracerProvider && typeof this.tracerProvider.shutdown === 'function') {\n      await this.tracerProvider.shutdown();\n    }\n  }\n}\n"],"mappings":";;;;;;;;;;AAsCO,IAAM,kBAAN,MAAM,iBAAuC;AAAA,EAC1C;AAAA,EACA;AAAA,EACA;AAAA,EAEA,YAAY,SAAkB,QAAoB,gBAAoC;AAC5F,SAAK,UAAU;AACf,SAAK,SAAS;AACd,SAAK,iBAAiB;AAAA,EACxB;AAAA,EAEA,aAAa,OAAO,QAAmD;AACrE,QAAI;AACJ,QAAI;AACF,gBAAW,MAAM,OAAO,oBAAoB;AAAA,IAC9C,QAAQ;AACN,YAAM,IAAI;AAAA,QACR;AAAA,MACF;AAAA,IACF;AAEA,UAAM,iBACH,OAAO,kBACR,QAAQ,MAAM,kBAAkB;AAClC,UAAM,cAAc,OAAO,eAAe;AAC1C,UAAM,SAAS,eAAe,UAAU,WAAW;AAEnD,WAAO,IAAI,iBAAgB,SAAS,QAAQ,cAAc;AAAA,EAC5D;AAAA,EAEA,MAAM,cACJ,MACA,YACA,IACY;AACZ,UAAM,UAAU,KAAK;AAErB,WAAO,KAAK,OAAO,gBAAgB,MAAM,EAAE,WAAW,GAAG,OAAO,aAAuB;AACrF,UAAI,eAAe;AACnB,YAAM,SAAqB;AAAA,QACzB,aAAa,KAAa,OAAkC;AAC1D,mBAAS,aAAa,KAAK,KAAK;AAAA,QAClC;AAAA,QACA,SAAS,WAAmB,OAAmD;AAC7E,mBAAS,SAAS,WAAW,KAAK;AAAA,QACpC;AAAA,QACA,UAAU,MAAgC,SAAkB;AAC1D,yBAAe;AACf,gBAAM,aACJ,SAAS,OACL,QAAQ,eAAe,KACvB,SAAS,UACP,QAAQ,eAAe,QACvB,QAAQ,eAAe;AAC/B,mBAAS,UAAU,EAAE,MAAM,YAAY,QAAQ,CAAC;AAAA,QAClD;AAAA,QACA,MAAM;AACJ,mBAAS,IAAI;AAAA,QACf;AAAA,MACF;AAEA,UAAI;AACF,cAAM,SAAS,MAAM,GAAG,MAAM;AAC9B,YAAI,CAAC,aAAc,QAAO,UAAU,IAAI;AACxC,eAAO;AAAA,MACT,SAAS,KAAK;AAIZ,eAAO,UAAU,OAAO;AACxB,cAAM;AAAA,MACR,UAAE;AACA,eAAO,IAAI;AAAA,MACb;AAAA,IACF,CAAC;AAAA,EACH;AAAA,EAEA,qBAAqB,MAAc,YAA8D;AAC/F,UAAM,aAAa,KAAK,QAAQ,MAAM,gBAAgB;AACtD,QAAI,YAAY;AACd,iBAAW,SAAS,MAAM,UAAU;AAAA,IACtC;AAAA,EACF;AAAA,EAEA,MAAM,WAA0B;AAC9B,QAAI,KAAK,kBAAkB,OAAO,KAAK,eAAe,aAAa,YAAY;AAC7E,YAAM,KAAK,eAAe,SAAS;AAAA,IACrC;AAAA,EACF;AACF;","names":[]}