@combycode/llm-sdk 3.1.0 → 3.3.0

package/CHANGELOG.md

@@ -6,6 +6,288 @@ All notable changes to `@combycode/llm-sdk` are documented here. The format foll
 
 ## [Unreleased]
 
+## [3.3.0] - 2026-09-06
+
+### Fixed
+
+- **xAI's batch API had never worked, and could not say so.** Three readings were wrong. The
+  status counts live under `state`, not at the top level, so `total` was always 0, the job never
+  reached a terminal state, and a polling caller waited on a batch that had completed in seconds --
+  no error, no output, just a wait. The results are nested and TAGGED,
+  `batch_result.response.<variant>`, so reading `row.response` found nothing and every answer came
+  back a failure with no error to explain it. And the cancel route is `<id>:cancel`; the slash form
+  answers 404 (`DELETE` and `PATCH` on the bare batch answer 405). All three measured live against
+  `api.x.ai` on 2026-09-04, which is also when the corpus cell for xAI batch went from unsupported
+  to green in 17 seconds -- the fastest of the four providers.
+
+- **A hosted tool the provider cannot run was dropped without a word.** Asking OpenRouter for
+  `code_interpreter` put `tools: []` on the wire and said nothing: the catalog is right (OpenRouter
+  routes function tools and its own plugins, not hosted code execution), so omitting it is correct,
+  but the silent loss of a capability the caller asked for is the exact thing this library's
+  tool-constraint mechanic exists to prevent. An unsupported builtin now produces a
+  `request_adjusted` warning naming the tool and what the provider does run, and an empty `tools`
+  array no longer reaches any provider -- it used to be dropped only on the `web_search` path.
+
+- **The local chunker silently dropped space-free text: CJK prose, minified JSON, base64 blobs,
+  long URLs.** `snapStep` moved the cursor to the next ASCII space no matter how far away it was,
+  and to the END of the document when there was none -- in both cases without emitting a chunk for
+  the span it stepped over. So a document with a base64 image in the middle lost the image and
+  everything the walk skipped with it (measured: 17983 of 26991 characters), and one that turns
+  space-free and stays that way was indexed as its first window alone. The text was never embedded
+  and no query could retrieve it. A space is now a boundary only while the next window would still
+  start inside the chunk just emitted; past that the walk advances by the step. Verified live
+  against real OpenAI embeddings, with the answer inside a minified payload: no hit before, the
+  right hit after, for the payload at the end of the document and in the middle of it.
+
+- **A document that mixed prose with a space-free run was chunked into dozens of runts.** The
+  step is derived from the SNAPPED window length, so a window trimmed back hard -- its only space
+  near its start, which is exactly what the last window before a base64 blob or a minified payload
+  looks like -- left a step below the overlap, and the `max(step, 1)` floor became the actual step.
+  The walk then crawled one word at a time across the whole approach to the blob: on a README with
+  one embedded image, 42 of 58 chunks came out under half the budget and the smallest was 8
+  characters. Every one was embedded at the caller's expense, and because they are the same
+  sentence shifted by a word they crowded each other out of the results. A boundary is now taken
+  only when it leaves at least half the window; below that the window is used whole. The cut that
+  buys can only ever land inside a run with no space in it for 1024 characters, where there is no
+  word to split. Same README: 16 chunks, no runts. Prose is byte-for-byte unaffected.
+
+- **The tail of every locally-indexed document was embedded several times over.** Once a window
+  reached the end of the text the walk kept going, re-emitting the same tail as a run of
+  ever-shorter chunks that all ended in the same place. Not lost text -- duplicate index entries,
+  each one paid for at the embedding endpoint and each one competing with the others for a result
+  slot. The walk now stops on the window that reaches the end.
+
+- **Every OpenAI model was routed to the Responses API, whatever the model.** `resolveApi` chose by
+  PROVIDER alone, while the catalog had carried `preferredApi` per model from the start and nothing
+  read it. Six catalogued models cannot be called on Responses at all: `gpt-audio`, `gpt-audio-1.5`,
+  `gpt-audio-mini` and the three `*-search` models, which are Chat Completions-only. Asking for any
+  of them failed with *"The requested model 'gpt-audio' is not supported with the Responses API"*.
+  Routing now consults the model's own preference and falls back to the provider default. Verified
+  live: `gpt-audio` returns audio and `gpt-5-search` answers, both of which previously could not run.
+
+- **`gpt-audio*` was described wrongly by the catalog.** `preferredApi: 'responses'` (see above),
+  `outputModalities: ['text']` for a model that returns audio bytes AND a transcript, and
+  `capabilities.audioGeneration: false` for the audio-generation model. OpenAI's guide is explicit:
+  *"For this audio-chat pattern, use Chat Completions with an audio-capable model."*
+
+- **Streamed audio produced nothing at all.** `delta.audio` was ignored by the Chat Completions
+  stream parser, so a `gpt-audio` turn streamed 68 SSE events and yielded exactly ONE unified event:
+  `usage`. No text (the words are in `audio.transcript`, not `delta.content`), no media, and —
+  because these chunks never carry a `finish_reason` — no terminal event either, so a caller
+  awaiting `done` waited forever. Now the transcript surfaces as `text`, the fragments as
+  `media_start` / `media_chunk` / `media_end`, and the final `expires_at`-only delta closes the
+  stream. Streamed audio is always `pcm16` (the API refuses any other format when `stream=true`) and
+  raw PCM has no magic bytes, so the mime is stated rather than sniffed.
+
+- **Audio output was silently dropped on the OpenAI chat path.** `openai-completions` gated its
+  `modalities` block on `hasAudioInput`, so `outputModalities: ['text', 'audio']` travelled from
+  `ExecuteOptions` all the way to the wire builder and died there. `gpt-audio` then refused the call
+  outright — *"this model requires that either input content or output modality contain audio"*.
+  The parser had always known how to build an `audio_output` part from `message.audio`, so both ends
+  of the feature existed and only this guard kept them apart. The guard now fires on audio in OR
+  audio out. OpenRouter inherits the same spec, so models routed through it now forward the audio
+  request the caller actually made.
+
+- **Every audio clip was labelled `audio/wav`, whatever it was.** OpenAI returns `message.audio` as
+  `{ id, data, expires_at, transcript }` — there is no `format` key — so the adapter's
+  `audio/${format ?? 'wav'}` fell through to the default on every response. Request mp3, receive
+  `ID3`-prefixed mp3 bytes, be told it is wav. A new `sniffAudioMime` reads the container from the
+  magic bytes (mp3/wav/ogg/flac/aac), mirroring the existing `sniffImageMime`; the old template
+  remains as the last resort.
+
+### Changed
+
+- **Stream parsing is spec-driven too.** The other half of the same migration: all seven
+  `createStreamParser` implementations now run one driver over a declarative spec, and **682 more
+  lines of hand-written parsing are gone**. The driver is the buffered interpreter with two
+  differences — state is created once per STREAM rather than per call, and one reserved
+  accumulator is drained and returned after each SSE event.
+
+  Measured before starting: of 518 lines across the five parsers, 38 touched state. The rest was
+  dispatch, which is what a spec expresses. Behaviour is unchanged and checked the same way, against
+  29 recorded streams whose event sequences are frozen in the corpus, plus one live streaming call
+  per provider after the switch.
+
+  `parseStreamEvent` stays on the adapter interface as a stateless one-shot; `createStreamParser` is
+  the stateful one callers should use.
+
+- **Response parsing is spec-driven.** Requests have been built from specs since 3.0.0; the parse
+  side was seven hand-written `parseResponse` implementations doing the same four things in four
+  spellings. All seven now run one interpreter over a declarative spec, and **599 lines of
+  hand-written parsing are gone**.
+
+  The evaluator is the request one, unchanged: `$`, `$map`, `$call`, `$table`, `$join`, `$when` and
+  `$default` never cared what the root object was. Only classification is new — `collect` walks a
+  discriminated array and emits into named accumulators, and naming several places ONE object in
+  each rather than copies, which is what the adapters did and what consumers depend on.
+
+  Behaviour is unchanged, and that is checked rather than asserted: the differential replays every
+  recorded provider body through the adapters and compares against `parsed` values frozen in the
+  corpus — data neither path recomputes, so it still means something now the code that produced
+  it is deleted. 46 of 46 buffered cells match, and one live call per provider was made against the
+  real API after the switch.
+
+  `OpenRouterAdapter.parseResponse` is gone entirely: its `:online` web-search rule is a delta of
+  the shared spec, so naming the spec is the override now, exactly as `wireFlavor` already was for
+  requests.
+
+### Internal
+
+- **The response corpus covers the parse paths it never reached.** Measured against the fixtures,
+  `citations`, `files`, `builtinToolCalls`, `media` and `moderation` appeared in **zero** buffered
+  cells, and `thinking` in one target — so a parser that stopped producing any of them kept the
+  differential green. Five scenarios were added (`builtin.search`, `builtin.codeexec`, `media.audio`,
+  `thinking`, `moderation`), taking the corpus from 42 to 57 recorded cells. Both bugs above were
+  found by recording them.
+
+  Scenarios now declare which targets they apply to, because the matrix is no longer a full cross
+  product: Chat Completions has no hosted web search, and only `gpt-audio` returns audio. The
+  recorder and the differential read the same `expectedCells()`, so they cannot disagree about what
+  is missing.
+
+- **The streaming corpus covers the branches that carry state.** Measured before this change, the
+  14 streaming cells produced only `usage`, `done`, `text`, `tool_call_*` and `thinking`: NINE of the
+  sixteen `StreamEvent` types had no coverage, and they were exactly the stateful ones — the
+  accumulate-JSON-then-pair-with-its-result machine, the emit-once-per-stream flags, the three-event
+  media reassembly. Five streaming scenarios were added (search, code execution, audio, thinking,
+  moderation), taking the corpus from 60 to 75 cells and streaming coverage from 7/16 to **15/16**.
+  The sixteenth, `error`, is emitted only by the Realtime adapter, which this corpus does not cover.
+
+  The audio bug above was found by the first run of the new cell: it tripped the existing invariant
+  that every streaming cell must produce a terminal event.
+
+- **`XAIAdapter` emitted every reasoning delta TWICE.** Its `parseStreamEvent` override prepended a
+  `thinking` event for `reasoning_content` while `OpenAIAdapter`, its parent, already appended one
+  for the same field. `xai/completions` is not a corpus target — xAI defaults to the Responses
+  API — so nothing was watching. The override was both redundant and duplicating; it is gone, and
+  a regression test pins the count at one.
+
+- **A test that failed on timing alone.** `tiktoken-optional` builds a counter per case, and the
+  encoder is a ~5.6 MB WASM module loaded lazily per instance, landing either side of bun's 5s
+  default under load. It failed twice in one session, on a different case each time, with no
+  assertion involved. The limit is raised rather than the load hidden: the work is slow, not wrong.
+  (A first attempt to warm the encoder in `beforeAll` made it fail 3/3 instead of intermittently,
+  because the cache is per-instance and the warm-up only added a third load.)
+
+- **The corpus did not catch everything, and that is worth recording.** Switching the adapters
+  dropped xAI's inline code-execution file extraction: `XAIResponsesAdapter` overrides
+  `filesFromOutputItem`, the shared spec transform called the OpenAI module function directly, and
+  the override was simply never consulted. No recorded xAI cell runs code execution, so the response
+  differential stayed green — an existing unit test failed instead. xAI now has its own response
+  registry, the parse-side twin of that adapter override.
+
+- **The three parse branches for a failure reported inside a 200 are covered.** A provider cannot be
+  asked to fail on demand, so those cells are CONSTRUCTED — marked `synthetic: true`, built from the
+  target's own recorded envelope with only the failure fields changed, each traceable to the official
+  SDK type cited in the cell's `provenance`. They cover OpenAI Responses `status:'failed'` with
+  `response.error`, its `incomplete_details.reason: 'content_filter'` (which must not be reported as a
+  length truncation), and Google Interactions `status:'failed'`.
+
+  A synthetic body widens the derived shape book's `known` and `values` but is excluded from
+  `expected`: a failed response carries no `output`, and letting it into that intersection would
+  weaken the check on every genuine recording.
+
+- **The frozen request corpus grew a waiver list.** A deliberate wire change previously had no way to
+  be recorded except re-freezing, which would absorb every *un*noticed change in the same pass. Each
+  waiver is checked in both directions: one whose cells no longer differ fails the suite, so it
+  cannot outlive the change it describes.
+
+## [3.2.1] — 2026-08-31
+
+### Fixed
+
+- **Three TypeScript errors in an MCP test file** that made v3.2.0 unpublishable: `searchParams.get()`
+  returns `string | null` while `.at(-1)` returns `string | undefined`, and `toBe` has no overload
+  spanning both. The assertions are unchanged; only their types are aligned.
+
+- **Two lint warnings** left standing in `network/engine.ts` (an unused type import) and an MCP test
+  helper (`let x!` forward declarations that nothing forward-references).
+
+### Internal
+
+- **The release gate now runs lint, typecheck and the tests** (`G1 build-green` in CombyCode's shared
+  quality-gate). v3.2.0 was tagged with a green gate and a red typecheck, because the gate checked
+  documentation and consumers while lint and typecheck lived in a playbook sentence — the half a
+  human has to remember. The gate's own README names that failure mode: "a checklist you have to
+  remember to read is guarded by the same attention that failed in the first place." The check ships
+  with the good/bad/blind fixtures the selftest requires, and was verified against the exact error
+  that escaped.
+
+## [3.2.0] — 2026-08-31
+
+### Added
+
+- **`response.citations` — the sources an answer cited, unified across providers.** Reaching them
+  meant regexing `response.raw`; the SDK's own web-search example did exactly that, which is every
+  consumer reimplementing provider knowledge that belongs here. Four wire shapes are read: Anthropic's
+  text-block `citations[]` (the only provider that also reports the cited passage), Google's
+  `groundingMetadata.groundingChunks[]`, OpenAI Responses/Chat `url_citation` annotations, and xAI's
+  bare top-level `citations[]`.
+
+  Distinct from `builtinToolCalls`, which records what the model *invoked*: a turn can run three
+  searches and cite one page. Through an agent run the sources **accumulate across steps**, deduped
+  by URL — a run that searches in step 1 and answers in step 3 keeps the sources its answer rests on.
+  Optional per R3 (`response.citations ?? []`); absent on `stream()`, which holds no raw payload, and
+  on Google's Interactions surface, whose grounding shape has not been measured.
+
+  **Streaming reports them too**, as a `citation` StreamEvent per source, collected onto the streamed
+  final response so `stream()` and `complete()` agree. Four more measured shapes, since a stream never
+  assembles the body the buffered reader parses: Anthropic `citations_delta`, Responses
+  `response.output_text.annotation.added` (OpenAI and xAI), chat-completions `delta.annotations`, and
+  Google's *late* populated `groundingMetadata` chunk — the first one carrying that key is empty, so
+  latching on first sight would report a search and no sources.
+
+  Verified live on Anthropic, OpenAI, Google, xAI and OpenRouter, buffered and streamed: 5/5 providers
+  return real cited URLs for the web-search scenario, which previously reported `no-citation` on all
+  five. Note Google reports each source as a `grounding-api-redirect` URL, not the page itself.
+
+### Fixed
+
+- **xAI file `delete`, `getInfo` and `list` never worked.** `XAIFileAdapter` is exported, and three
+  of its four methods rejected with `unknown spec: xai/files.<op>` before any HTTP — only
+  `xai/files.upload` had ever been written, so roughly twenty lines of response mapping below them
+  had never run. The three specs follow `xai.upload`'s own note that the surface differs from
+  OpenAI's only in the upload `purpose`, so they carry the same paths and the same Bearer base.
+
+- **`listModelsLive` was blocked by CORS in the browser, for Anthropic only.** Anthropic refuses a
+  browser request without an explicit opt-in header. The chat path sends it and the Anthropic files
+  specs carry it, but `models.list` lost it when that request moved to a spec: the header had lived
+  in the helper's own table, which nothing called any more. Measured from a browser with a raw
+  fetch — `/v1/models` returns 200 with the header and fails without it. The sandbox listed models
+  through `Promise.allSettled` and kept only the fulfilled results, so the rejection was swallowed
+  and Anthropic simply showed no models rather than an error.
+
+### Removed
+
+- **Five private functions left behind by the spec migration.** Each was live until its area moved to
+  the spec-driven builder, which renamed it with a leading underscore instead of deleting it:
+  `_buildForm`, `_batchName`, `_toOpenAIAudioFormat`, `_toResponseModalities` and
+  `LiveSpec.headers`. Each was checked field by field against the spec that replaced it before
+  removal — and the last of them was **not** equivalent, which is how the CORS bug above was found.
+
+### Internal
+
+- **Line coverage 91.62% → 99.53%**, function coverage 87.53% → 97.41%, across 2290 → 4051 tests.
+  Eleven modules had no function coverage at all — `batcher.ts` ran 3.5% of its lines,
+  `plugins/internal-tools/runner/runner.ts` 3.7%, `transport-ws.ts` 9.6%. Every new test is mutation-verified: the
+  source line it claims to cover was broken and the test watched to fail.
+
+  The exercise found ten defects no existing test caught, each pinned as a `DEFECT:` test rather
+  than fixed in the same pass: `ResponseStore.list(null)` returning every user's response ids,
+  `chunker.ts` silently dropping space-free text (3000 characters of CJK or base64 yield one
+  400-character chunk), `attachment.ts fromBlob` leaving the MIME type empty because a type-less
+  `Blob` reports `''` rather than `null`, and `oauth.ts tryRefresh` discarding the refresh token it
+  means to preserve.
+
+  Two pre-existing tests turned out to execute no code at all — one asserts against `readFileSync`
+  of `realtime.ts` with regexes over the source text, which is why that file sat at 0% functions
+  while appearing tested.
+
+- **A coverage floor that can actually fail.** `bunfig.toml`'s `coverageThreshold` is accepted and
+  ignored by bun 1.3.14 — set to an impossible 1.0 the run still exits 0 — so the floor is its own
+  step (`bun run coverage:gate`), carrying a `--self-test` that proves it discriminates.
+
+
 ## [3.1.0] — 2026-08-25
 
 ### Added

package/dist/agent/loop-step-state.d.ts

@@ -1,7 +1,7 @@
 /** Mutable accumulator for a single agent loop step.
  *  Passed through stream-event helpers so they don't fight closure state. */
 import type { ToolCallPart } from '../llm/types/messages';
-import type { Usage } from '../llm/types/response';
+import type { Citation, Usage } from '../llm/types/response';
 /** Accumulation bucket for one in-progress tool call (before tool_call_end). */
 export interface ToolCallAccumEntry {
     id: string;
@@ -19,4 +19,8 @@ export interface StepState {
     toolCallAccum: Map<string, ToolCallAccumEntry>;
     stepUsage: Usage;
     stepFinishReason: string;
+    /** Sources cited during this step, keyed by url. A Map rather than an array
+     *  because Google repeats its grounding chunks across late chunks, and one page
+     *  cited twice is one source. */
+    stepCitations: Map<string, Citation>;
 }

package/dist/catalog/catalog.d.ts

@@ -66,6 +66,18 @@ export interface ModelReasoning {
     effortValues?: string[];
     encryptedContent: boolean;
     summaryAvailable: boolean;
+    /** Whether `thinking: { mode: 'off' }` can actually be honoured.
+     *
+     *  Deliberately optional, and deliberately not defaulted. Absent means NOBODY
+     *  HAS ESTABLISHED IT for this model, which is not the same as `true`: only an
+     *  explicit `false` makes the client drop the request and warn, so a provider
+     *  nobody has measured keeps behaving exactly as it did.
+     *
+     *  It exists because some models refuse: Gemini 2.5 Pro answers `Budget 0 is
+     *  invalid. This model only works in thinking mode.`, and 3.7 Flash accepts a
+     *  zero budget and reasons anyway. Sending the disable field to those buys a
+     *  400 or a silent lie; saying so up front costs a warning. */
+    canDisable?: boolean;
 }
 export interface TokenizerInfo {
     strategy: 'heuristic' | 'tiktoken' | 'count_api';