@bastani/pi-ai 0.9.43 → 0.9.44-alpha.2

package/CHANGELOG.md

@@ -4,6 +4,29 @@ This package is a Bastani fork of `@earendil-works/pi-ai`. Upstream history at t
 
 ## [Unreleased]
 
+## [0.9.44-alpha.1] - 2026-10-24
+
+### Breaking Changes
+
+- `appendGrammarToolInputJsonDelta()` (`api/constrained-sampling`) takes the input delta instead of the whole input so far and no longer closes the input; close it with the new `closeGrammarToolInputJson()` ([pi #9265](https://github.com/earendil-works/pi/issues/9265))
+
+### Added
+
+- Assistant messages from the `pi-messages` API now carry `responseModel`, the concrete model the server reports on its terminal `done` or `error` event, so clients of virtual models can see which model answered ([pi #11265](https://github.com/earendil-works/pi/pull/11265))
+- Added `StreamingJsonParser`, an incremental, tolerant parser for JSON text that arrives in pieces (each piece is read once; snapshots of the partial value never change once returned), and `ToolCallArgumentsStream`, which every provider now uses for streamed tool-call arguments; `StreamingJsonParser.append()` also returns the changes each piece made (`JsonChange`), for writers that keep their own copy ([pi #9265](https://github.com/earendil-works/pi/issues/9265))
+
+### Changed
+
+- The faux provider streams partial tool-call arguments, so `toolcall_delta` partials show growing arguments as real providers do ([pi #9265](https://github.com/earendil-works/pi/issues/9265))
+
+### Fixed
+
+- Claude models with mid-conversation effort no longer fail with `Invalid effort level` on accounts whose organization caps effort below `high`, even at `low` or `medium` thinking: the fixed top-level effort is now `low`, and the per-message effort markers still set the level for each reply ([pi #10063](https://github.com/earendil-works/pi/issues/10063))
+- The faux provider no longer counts newly cached prompt tokens both as `input` and as `cacheWrite` when it simulates prompt caching for a `sessionId`. Every prompt token now goes to either `cacheRead` or `cacheWrite`, and `input` is 0 ([pi f3fa8bd9](https://github.com/earendil-works/pi/commit/f3fa8bd92430a2bf443681027a01c58586f35464))
+- Gemini tool schemas sent through `parametersJsonSchema` now express string and number `const` values as single-value `enum`s, which Gemini honors (it ignores `const`). Only schema-bearing keywords are rewritten, so literal data under `default`, `examples`, `enum`, `const` and custom keywords, and property names such as `__proto__`, are sent unchanged ([pi 7314819f](https://github.com/earendil-works/pi/commit/7314819f0fe30fa616f32fa3a1fc8d1a856a1446), [pi #11308](https://github.com/earendil-works/pi/issues/11308))
+- Kimi K3.1 Pro and K3.1 Pro Highspeed on Kimi Coding, Moonshot, and OpenCode Go no longer use Kimi K2 request settings. They now send `reasoning_effort` (low/high/max), return earlier reasoning, accept tools added mid-conversation, and report Moonshot rates on Kimi Coding; OpenRouter and Vercel AI Gateway clamp their output limit like Kimi K3. Kimi Coding `k3-256k` now reports Kimi K3 rates ([pi 1de2d7ad](https://github.com/earendil-works/pi/commit/1de2d7adeacfd1b065d9df091997d44c43120ade))
+- Streamed tool-call arguments are no longer re-parsed in full on every delta, which made CPU time and memory grow quadratically with the size of a tool call (a 90 KB `write` on Anthropic peaked at about 140 MB of heap; a 500 KB one took minutes). Every provider and the pi-messages client now read each delta once, and keep their stream state (`partialJson`, `partialArgs`, `index`, `customInput`, `streamIndex`, Bedrock's encrypted reasoning bytes) off the content blocks, so it no longer reaches partial messages or transcripts of failed streams. Grammar (custom) tool input streaming no longer copies the whole input on every delta, and truncated provider error bodies and pi-messages diagnostics no longer keep the whole text they were cut from alive ([pi #9265](https://github.com/earendil-works/pi/issues/9265))
+
 ## [0.9.42] - 2026-10-23
 
 ### Added

package/dist/api/anthropic-messages.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"anthropic-messages.d.ts","sourceRoot":"","sources":["../../src/api/anthropic-messages.ts"],"names":[],"mappings":"AAAA,OAAO,SAAkD,MAAM,mBAAmB,CAAC;AAiBnF,OAAO,KAAK,EASX,mBAAmB,EAEnB,cAAc,EACd,aAAa,EAOb,MAAM,aAAa,CAAC;AA8KrB,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,GAAG,KAAK,CAAC;AAE1E,MAAM,MAAM,wBAAwB,GAAG,YAAY,GAAG,SAAS,CAAC;AAqPhE,MAAM,WAAW,gBAAiB,SAAQ,aAAa;IACtD;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;;;;;;;OAWG;IACH,MAAM,CAAC,EAAE,eAAe,CAAC;IACzB;;;;;;;;;;;OAWG;IACH,eAAe,CAAC,EAAE,wBAAwB,CAAC;IAC3C;;;;;OAKG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B;;;;OAIG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACtE;;;;OAIG;IACH,MAAM,CAAC,EAAE,SAAS,CAAC;CACnB;AAsTD,eAAO,MAAM,MAAM,EAAE,cAAc,CAAC,oBAAoB,EAAE,gBAAgB,CAiazE,CAAC;AA4BF,eAAO,MAAM,YAAY,EAAE,cAAc,CAAC,oBAAoB,EAAE,mBAAmB,CAgDlF,CAAC"}
+{"version":3,"file":"anthropic-messages.d.ts","sourceRoot":"","sources":["../../src/api/anthropic-messages.ts"],"names":[],"mappings":"AAAA,OAAO,SAAkD,MAAM,mBAAmB,CAAC;AAiBnF,OAAO,KAAK,EASX,mBAAmB,EAEnB,cAAc,EACd,aAAa,EAKb,MAAM,aAAa,CAAC;AA+KrB,MAAM,MAAM,eAAe,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,GAAG,KAAK,CAAC;AAE1E,MAAM,MAAM,wBAAwB,GAAG,YAAY,GAAG,SAAS,CAAC;AAqPhE,MAAM,WAAW,gBAAiB,SAAQ,aAAa;IACtD;;;;;;OAMG;IACH,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;;;;;;;OAWG;IACH,MAAM,CAAC,EAAE,eAAe,CAAC;IACzB;;;;;;;;;;;OAWG;IACH,eAAe,CAAC,EAAE,wBAAwB,CAAC;IAC3C;;;;;OAKG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B;;;;OAIG;IACH,UAAU,CAAC,EAAE,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAC;IACtE;;;;OAIG;IACH,MAAM,CAAC,EAAE,SAAS,CAAC;CACnB;AAsTD,eAAO,MAAM,MAAM,EAAE,cAAc,CAAC,oBAAoB,EAAE,gBAAgB,CAgZzE,CAAC;AA4BF,eAAO,MAAM,YAAY,EAAE,cAAc,CAAC,oBAAoB,EAAE,mBAAmB,CAgDlF,CAAC"}

package/dist/api/anthropic-messages.js

@@ -7,13 +7,14 @@ import { assertSupportedDocumentMimeType } from "../utils/document-input.js";
 import { AssistantMessageEventStream } from "../utils/event-stream.js";
 import { headersToRecord } from "../utils/headers.js";
 import { getIdentityHeaders } from "../utils/identity.js";
-import { parseJsonWithRepair, parseStreamingJson, parseToolCallArguments } from "../utils/json-parse.js";
+import { parseJsonWithRepair } from "../utils/json-parse.js";
 import { applyOpenRouterReportedCost, isOpenRouterModel, } from "../utils/openrouter-cost.js";
 import { getProviderEnvValue } from "../utils/provider-env.js";
 import { retryProviderRequest } from "../utils/provider-retry.js";
 import { getFetchAdmission } from "../utils/request-admission.js";
 import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
 import { createStreamDeadline, withStreamDeadline } from "../utils/stream-deadline.js";
+import { ToolCallArgumentsStream } from "../utils/streaming-json.js";
 import { getSystemMessageText, renderSystemMessageUpdate } from "../utils/text.js";
 import { getCurrentTools, getInitialSystemMessage, resolveTranscript, } from "../utils/transcript.js";
 import { ANTHROPIC_STRICT_JSON_SCHEMA_PROFILE, getJsonSchemaToolParameters, resolveJsonSchemaStrictSampling, } from "./constrained-sampling.js";
@@ -685,7 +686,12 @@ export const stream = (model, context, options) => {
             }
             await options?.onResponse?.({ status: response.status, headers: headersToRecord(response.headers) }, model);
             stream.push({ type: "start", partial: output });
-            const blocks = output.content;
+            // Stream state per Anthropic block index, kept off the content blocks so it never reaches partials.
+            const openBlocks = new Map();
+            const openBlock = (anthropicIndex) => {
+                const open = openBlocks.get(anthropicIndex);
+                return open === undefined ? undefined : { ...open, block: output.content[open.contentIndex] };
+            };
             for await (const event of withStreamDeadline(iterateAnthropicEvents(response, streamDeadline.signal), streamDeadline.deadlineMs, streamDeadline.abort)) {
                 await options?.onProviderStreamEvent?.(event, model);
                 if (event.type === "message_start") {
@@ -715,47 +721,42 @@ export const stream = (model, context, options) => {
                 }
                 else if (event.type === "content_block_start") {
                     if (event.content_block.type === "text") {
-                        const block = {
-                            type: "text",
-                            text: event.content_block.text ?? "",
-                            index: event.index,
-                        };
-                        output.content.push(block);
+                        output.content.push({ type: "text", text: event.content_block.text ?? "" });
+                        openBlocks.set(event.index, { contentIndex: output.content.length - 1 });
                         stream.push({ type: "text_start", contentIndex: output.content.length - 1, partial: output });
                     }
                     else if (event.content_block.type === "thinking") {
-                        const block = {
+                        output.content.push({
                             type: "thinking",
                             thinking: event.content_block.thinking ?? "",
                             thinkingSignature: event.content_block.signature ?? "",
-                            index: event.index,
-                        };
-                        output.content.push(block);
+                        });
+                        openBlocks.set(event.index, { contentIndex: output.content.length - 1 });
                         stream.push({ type: "thinking_start", contentIndex: output.content.length - 1, partial: output });
                     }
                     else if (event.content_block.type === "redacted_thinking") {
-                        const block = {
+                        output.content.push({
                             type: "thinking",
                             thinking: "[Reasoning redacted]",
                             thinkingSignature: event.content_block.data,
                             redacted: true,
-                            index: event.index,
-                        };
-                        output.content.push(block);
+                        });
+                        openBlocks.set(event.index, { contentIndex: output.content.length - 1 });
                         stream.push({ type: "thinking_start", contentIndex: output.content.length - 1, partial: output });
                     }
                     else if (event.content_block.type === "tool_use") {
-                        const block = {
+                        output.content.push({
                             type: "toolCall",
                             id: event.content_block.id,
                             name: isOAuth
                                 ? fromClaudeCodeName(event.content_block.name, currentTools)
                                 : event.content_block.name,
                             arguments: event.content_block.input ?? {},
-                            partialJson: "",
-                            index: event.index,
-                        };
-                        output.content.push(block);
+                        });
+                        openBlocks.set(event.index, {
+                            contentIndex: output.content.length - 1,
+                            arguments: new ToolCallArgumentsStream(),
+                        });
                         stream.push({ type: "toolcall_start", contentIndex: output.content.length - 1, partial: output });
                     }
                     else if (event.content_block.type === "fallback") {
@@ -785,60 +786,52 @@ export const stream = (model, context, options) => {
                     }
                 }
                 else if (event.type === "content_block_delta") {
+                    const open = openBlock(event.index);
                     if (event.delta.type === "text_delta") {
-                        const index = blocks.findIndex((b) => b.index === event.index);
-                        const block = blocks[index];
-                        if (block && block.type === "text") {
-                            block.text += event.delta.text;
+                        if (open?.block.type === "text") {
+                            open.block.text += event.delta.text;
                             stream.push({
                                 type: "text_delta",
-                                contentIndex: index,
+                                contentIndex: open.contentIndex,
                                 delta: event.delta.text,
                                 partial: output,
                             });
                         }
                     }
                     else if (event.delta.type === "thinking_delta") {
-                        const index = blocks.findIndex((b) => b.index === event.index);
-                        const block = blocks[index];
-                        if (block && block.type === "thinking") {
-                            block.thinking += event.delta.thinking;
+                        if (open?.block.type === "thinking") {
+                            open.block.thinking += event.delta.thinking;
                             stream.push({
                                 type: "thinking_delta",
-                                contentIndex: index,
+                                contentIndex: open.contentIndex,
                                 delta: event.delta.thinking,
                                 partial: output,
                             });
                         }
                     }
                     else if (event.delta.type === "input_json_delta") {
-                        const index = blocks.findIndex((b) => b.index === event.index);
-                        const block = blocks[index];
-                        if (block && block.type === "toolCall") {
-                            block.partialJson += event.delta.partial_json;
-                            block.arguments = parseStreamingJson(block.partialJson);
+                        if (open?.block.type === "toolCall" && open.arguments !== undefined) {
+                            open.block.arguments = open.arguments.append(event.delta.partial_json);
                             stream.push({
                                 type: "toolcall_delta",
-                                contentIndex: index,
+                                contentIndex: open.contentIndex,
                                 delta: event.delta.partial_json,
                                 partial: output,
                             });
                         }
                     }
                     else if (event.delta.type === "signature_delta") {
-                        const index = blocks.findIndex((b) => b.index === event.index);
-                        const block = blocks[index];
-                        if (block && block.type === "thinking") {
-                            block.thinkingSignature = block.thinkingSignature || "";
-                            block.thinkingSignature += event.delta.signature;
+                        if (open?.block.type === "thinking") {
+                            open.block.thinkingSignature = open.block.thinkingSignature || "";
+                            open.block.thinkingSignature += event.delta.signature;
                         }
                     }
                 }
                 else if (event.type === "content_block_stop") {
-                    const index = blocks.findIndex((b) => b.index === event.index);
-                    const block = blocks[index];
-                    if (block) {
-                        delete block.index;
+                    const open = openBlock(event.index);
+                    openBlocks.delete(event.index);
+                    if (open) {
+                        const { block, contentIndex: index } = open;
                         if (block.type === "text") {
                             stream.push({
                                 type: "text_end",
@@ -855,11 +848,8 @@ export const stream = (model, context, options) => {
                                 partial: output,
                             });
                         }
-                        else if (block.type === "toolCall") {
-                            Object.assign(block, parseToolCallArguments(block.partialJson));
-                            // Finalize in-place and strip the scratch buffer so replay only
-                            // carries parsed arguments.
-                            delete block.partialJson;
+                        else if (block.type === "toolCall" && open.arguments !== undefined) {
+                            Object.assign(block, open.arguments.finish());
                             stream.push({
                                 type: "toolcall_end",
                                 contentIndex: index,
@@ -948,11 +938,6 @@ export const stream = (model, context, options) => {
                 !output.diagnostics?.some((diagnostic) => diagnostic.type === "anthropic_input_transformations")) {
                 recordInputTransformations(output, inputTransformations);
             }
-            for (const block of output.content) {
-                delete block.index;
-                // partialJson is only a streaming scratch buffer; never persist it.
-                delete block.partialJson;
-            }
             output.stopReason = options?.signal?.aborted ? "aborted" : "error";
             output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
             stream.push({ type: "error", reason: output.stopReason, error: output });
@@ -1241,14 +1226,16 @@ function buildParams(model, context, isOAuthToken, options, disablePreferredStri
         }
     }
     // Managed effort models always use adaptive thinking. Their per-turn markers
-    // carry the actual effort; the top-level high value deliberately stays stable.
+    // carry the actual effort for every reply.
     if (effortMessages) {
         params.thinking = {
             type: "adaptive",
             display: options?.thinkingDisplay ?? "summarized",
             block_binding: { prefix_mismatch_behavior: "drop_block" },
         };
-        params.output_config = { effort: "high" };
+        // The top-level effort is part of the cached prompt, so it stays fixed. It is "low" because
+        // organizations can cap effort and reject requests containing any level above the cap.
+        params.output_config = { effort: "low" };
     }
     else if (model.reasoning) {
         if (options?.thinkingEnabled) {