llmz 0.6.4 → 0.6.6

package/DOCS.md

@@ -1582,6 +1582,95 @@ await execute({
 
 ---
 
+### Mid-Stream Model Fallback (midStreamFallback)
+
+By default, LLMz streams responses: message bodies (`■send` blocks) and completed `Chat.handler` sends are dispatched as soon as they are produced, and the ■run block starts executing while the model is still generating the rest of the response. This gives the lowest time-to-first-token but runs code before the response is complete.
+
+Set `options.midStreamFallback: true` to allow Cognitive to restart a failed stream on another model. There is **no buffered delivery and no final delivery queue**: with fallback enabled, delta text and completed `Chat.handler` sends both stream immediately during generation. Only **TOOL/CODE execution** waits for a stream to complete successfully. When Cognitive abandons an attempt, a reset-only delta (`restart: true`) is delivered through the existing `Chat.onMessageDelta`; it invalidates **all** messages of the current iteration — **including messages already committed through `handler`** — so consumers must retract and replace them:
+
+```typescript
+await execute({
+  // ...
+  chat: new MyChat({
+    onMessageDelta: (delta) => {
+      if (delta.restart) {
+        // invalidates ALL current-iteration messages, including completed handler
+        // sends: retract/replace them before the replacement streams
+        return retractIteration(delta.iterationId)
+      } else {
+        // stream this message's text as it arrives (same iterationId as below)
+        updateMessage(delta)
+      }
+    },
+  }),
+  options: { midStreamFallback: true },
+})
+```
+
+`retractIteration`/`updateMessage` are placeholder names for your consumer's message store. **This option is not safe to enable blindly**: a reset invalidates messages that were already delivered through `handler`. If your transport cannot retract or replace an already-sent message — an external irreversible send such as SMS, email, or a third-party webhook — those sends cannot be undone, so keep `midStreamFallback` disabled unless your consumer can retract and replace invalidated messages.
+
+#### MessageDelta contract
+
+`Chat.onMessageDelta` receives a single typed `MessageDelta` discriminated union:
+
+- **Ordinary text chunk** (`restart: false`) — the next piece of one message's body; all fields are required:
+  ```typescript
+  {
+    restart: false,
+    iterationId: string, // stable across ALL attempts of this LLM generation iteration
+    id: string, // message id — DIFFERENT per attempt (unique when fallback is on)
+    component: string,
+    props: Record<string, unknown>, // final by the time the body starts streaming
+    delta: string, // new chunk of body text
+    content: string, // full body text accumulated so far, including this chunk
+  }
+  ```
+- **Reset-only delta** (`restart: true`) — carries no text; signals the current generation attempt was abandoned and a replacement is being attempted:
+  ```typescript
+  {
+    restart: true,
+    iterationId: string, // the iteration whose messages (incl. handler sends) must be retracted/replaced
+    attempt: number,
+    fromModel: string,
+    toModel: string,
+    reason: string,
+  }
+  ```
+
+IDs: `iterationId` is **stable across attempts** — every restart of the same iteration keeps the same `iterationId` — while `id` is **different per attempt**. After a reset, replacement chunks arrive under new `id`s within the same `iterationId`.
+
+Consumer rules:
+
+- On `restart: true`, retract/replace **all** current-iteration messages bearing that `iterationId` — streamed text **and** messages already delivered through `handler`.
+- On an ordinary chunk, update the message for `delta.id` (create it if new, otherwise append).
+- A reset is emitted and awaited **before** replacement output begins, **even if the replacement yields no message at all**. Return/await the retraction promise in your handler. If it throws or rejects, generation fails without delivering replacement output or executing code; ordinary text-preview errors remain best-effort.
+- A reset only invalidates the messages of its own `iterationId`; it **never** invalidates messages from earlier, completed iterations.
+- Only TOOL/CODE execution is gated on a successful stream: generated code and tool calls never run from an abandoned attempt.
+
+Outside of fallback (the default), `onMessageDelta` behavior is unchanged: ordinary chunks carry `iterationId`/`id`/`component`/`props`/`delta`/`content` with a single stable message id, and no restart/reset deltas are produced.
+
+#### Completed-send correlation
+
+`Chat.handler(component, metadata)` receives `{ iterationId: string, id: string }` for **every send**, including empty-body and non-text components, without requiring `onMessageDelta`. For streamed messages, `metadata.id` matches the text delta's `id`; it changes across attempts while `iterationId` stays stable. Existing one-argument handlers continue to work.
+
+Record **all** persisted message IDs created while awaiting that handler under its metadata (one component may produce multiple persisted messages). A restart can then retract everything for the matching iteration, even after a message's stream tracking has completed. Non-streaming sends also receive metadata. Tool-yielded messages get unique per-yield IDs and the runtime iteration ID; standalone `Tool.execute` calls without an iteration use their `callId` as the scope.
+
+#### Migrating consumers
+
+- **No final delivery queue**: messages are not withheld until the stream ends. Deltas and completed `handler` sends arrive during generation; drop any "promote on success" design — treat every delivered message as provisional until the iteration completes without a reset.
+- **Retract, don't just clear previews**: on `restart: true`, retraction must cascade to every current-iteration message, including ones already committed through `handler`. If your UI or transport cannot do that, keep `midStreamFallback` disabled.
+- **Clear on errors and cancellation**: a failed or cancelled generation can leave current-iteration messages on screen, incomplete, or unresettable. Explicitly clear/roll back current-iteration state when execution errors or ends — there is no exactly-once guarantee, and caller retries can duplicate effects.
+- **Irreversible transports**: external sends that cannot be undone (SMS, email, webhooks) must not blindly enable fallback without a reset-capable gateway.
+
+#### Caveats
+
+- **No extra retry loop**: Cognitive owns fallback, bounded by the model chain. LLMz does not retry abandoned attempts itself.
+- **Cancellation**: cancellation still aborts the original generation request as usual; consumers must clear current-iteration state on abort (see above). Cancellation deadlines are not reset, and the existing stream inactivity guard remains active during handoffs.
+
+This option defaults to false and applies only to streaming clients. Non-streaming Cognitive requests already fall back transparently. It can be combined with `maxTimeToFirstToken` and `transcriptionModel`; do not enable it globally underneath LLMz without also enabling LLMz's fallback option and a reset-capable consumer.
+
+Restarts emit `llm_call_restarted` traces with the attempt, source/destination models, and reason. Token timings remain relative to the original request and describe the surviving attempt (including time spent on prior attempts). Usage and cost retain Cognitive's final reported metadata; LLMz does not infer or sum abandoned-attempt billing.
+
 ## API Reference
 
 ### Core Functions

package/dist/chat.d.ts

@@ -24,7 +24,14 @@ import { TranscriptArray, Transcript } from './transcript.js';
  * }
  * ```
  */
-export type MessageHandler = (input: RenderedComponent) => Promise<void> | void;
+/** Correlates every generated send with its deltas and restart scope. */
+export type MessageMetadata = {
+    /** Stable across generation attempts; standalone tool calls use their callId. */
+    iterationId: string;
+    /** Logical send ID; matches MessageDelta.id when text deltas exist. Unique across attempts. */
+    id: string;
+};
+export type MessageHandler = (input: RenderedComponent, metadata: MessageMetadata) => Promise<void> | void;
 /**
  * A chunk of a message body streamed live from the LLM, before the message is complete.
  *
@@ -32,10 +39,14 @@ export type MessageHandler = (input: RenderedComponent) => Promise<void> | void;
  * Each chunk is forwarded to {@link Chat} `onMessageDelta` as soon as it is parsed, so the
  * client can render the message progressively (e.g. typewriter effect in a chat UI).
  *
- * The complete message is always delivered to the regular `handler` afterwards — deltas
- * are a progressive preview, the `handler` call remains the authoritative delivery.
+ * In mid-stream fallback mode, both deltas and completed `handler` messages
+ * are live and retractable: restart deltas invalidate every message from that
+ * iteration. Consumers must retract them; only code execution waits for success.
  */
 export type MessageDelta = {
+    restart: false;
+    /** Generation whose provisional messages must be cleared on a restart. */
+    iterationId: string;
     /** Identifies the message being streamed. Unique within an `execute()` call. */
     id: string;
     /** Component name of the message being streamed (e.g. 'message', 'button'). */
@@ -46,10 +57,24 @@ export type MessageDelta = {
     delta: string;
     /** The full body text accumulated so far, including this chunk. */
     content: string;
+} | {
+    /**
+     * Reset-only delta, emitted before replacement text (even if none follows).
+     * Clear ALL messages for iterationId, including completed `handler` sends.
+     * Other iterations are unaffected. Irreversible sends cannot be undone.
+     */
+    restart: true;
+    iterationId: string;
+    /** 1-based index of the new attempt. */
+    attempt: number;
+    fromModel: string;
+    toModel: string;
+    reason: string;
 };
 /**
  * Function type for handling message chunks as they are streamed from the LLM.
- * Errors thrown by this handler are ignored — streaming previews are best-effort.
+ * Text-preview errors are ignored. Restart-handler errors stop generation:
+ * replacement output must not be delivered if retraction fails.
  */
 export type MessageDeltaHandler = (delta: MessageDelta) => Promise<void> | void;
 /**

package/dist/context.d.ts

@@ -564,6 +564,17 @@ export declare class Context implements Serializable<Context.JSON> {
      * before the cognitive service falls back to the next model/provider.
      */
     maxTimeToFirstToken?: number;
+    /**
+     * Allow Cognitive to restart a failed stream on another model. With fallback
+     * enabled, there is no buffered delivery and no final delivery queue: delta
+     * text and completed Chat.handler sends stream immediately during generation.
+     * A reset-only delta (restart: true) on Chat.onMessageDelta invalidates ALL
+     * current-iteration messages, including completed sends — consumers must
+     * retract/replace them, which is unsafe for irreversible external transports.
+     * Only TOOL/CODE execution waits for a successful stream. Streaming-only;
+     * defaults to false, which leaves progressive sending unchanged.
+     */
+    midStreamFallback?: boolean;
     /**
      * STT model used by the cognitive service to transcribe audio attachments
      * when the target LLM does not support audio natively. Defaults to 'fast'.
@@ -600,6 +611,7 @@ export declare class Context implements Serializable<Context.JSON> {
         timeout?: number;
         maxTokens?: number;
         maxTimeToFirstToken?: number;
+        midStreamFallback?: boolean;
         transcriptionModel?: SttModels;
     });
     toJSON(): {

package/dist/{custom-client-NUoxrFQh.js → custom-client-CJIThG2H.js}

@@ -4,7 +4,7 @@ import { f as flatRest, l as extractType, m as baseCreate, p as copyArray, t as
 import { _ as getSymbolsIn, b as keysIn, g as getAllKeysIn, o as getTokenizer, x as assignValue, y as baseSlice } from "./utils-UBeAFeEI.js";
 import { t as getPrototype } from "./_getPrototype-CG7Dxrrz.js";
 import { t as isPlainObject } from "./isPlainObject-D39AzPO2.js";
-import { t as Tool } from "./tool-CAo-BQHs.js";
+import { t as Tool } from "./tool-Bv8N4IPj.js";
 import { o as LoopExceededError } from "./errors-Cg1G0Awu.js";
 import { t as Exit } from "./exit-CDdxDBRT.js";
 import { n as assertValidComponent } from "./component-DGG7d_Mz.js";
@@ -2254,6 +2254,17 @@ var Context = class {
 	*/
 	maxTimeToFirstToken;
 	/**
+	* Allow Cognitive to restart a failed stream on another model. With fallback
+	* enabled, there is no buffered delivery and no final delivery queue: delta
+	* text and completed Chat.handler sends stream immediately during generation.
+	* A reset-only delta (restart: true) on Chat.onMessageDelta invalidates ALL
+	* current-iteration messages, including completed sends — consumers must
+	* retract/replace them, which is unsafe for irreversible external transports.
+	* Only TOOL/CODE execution waits for a successful stream. Streaming-only;
+	* defaults to false, which leaves progressive sending unchanged.
+	*/
+	midStreamFallback;
+	/**
 	* STT model used by the cognitive service to transcribe audio attachments
 	* when the target LLM does not support audio natively. Defaults to 'fast'.
 	*/
@@ -2509,10 +2520,12 @@ var Context = class {
 		this.snapshot = props.snapshot;
 		this.maxTokens = props.maxTokens;
 		this.maxTimeToFirstToken = props.maxTimeToFirstToken;
+		this.midStreamFallback = props.midStreamFallback;
 		this.transcriptionModel = props.transcriptionModel;
 		if (this.loop < 1 || this.loop > 100) throw new Error("Invalid loop. Expected a number between 1 and 100.");
 		if (this.maxTokens !== void 0 && (!Number.isFinite(this.maxTokens) || this.maxTokens < 1)) throw new Error("Invalid maxTokens. Expected a positive number.");
 		if (this.maxTimeToFirstToken !== void 0 && (!Number.isFinite(this.maxTimeToFirstToken) || this.maxTimeToFirstToken < 1)) throw new Error("Invalid maxTimeToFirstToken. Expected a positive number of milliseconds.");
+		if (this.midStreamFallback !== void 0 && typeof this.midStreamFallback !== "boolean") throw new Error("Invalid midStreamFallback. Expected a boolean.");
 	}
 	toJSON() {
 		var _this$snapshot4;

package/dist/{custom-client-BPmEoWKu.cjs → custom-client-Cr33K60k.cjs}

@@ -4,7 +4,7 @@ const require_dual_modes = require("./dual-modes-wAOAlaBS.cjs");
 const require_utils = require("./utils-ZpJm00ed.cjs");
 const require__getPrototype = require("./_getPrototype-DQDlNVVM.cjs");
 const require_isPlainObject = require("./isPlainObject-CSxXVKCk.cjs");
-const require_tool = require("./tool-DdPZb0JW.cjs");
+const require_tool = require("./tool-ReA9gxMp.cjs");
 const require_errors = require("./errors-C962uJdS.cjs");
 const require_exit = require("./exit-BGi0Kk0y.cjs");
 const require_component = require("./component-BwixNVH8.cjs");
@@ -2254,6 +2254,17 @@ var Context = class {
 	*/
 	maxTimeToFirstToken;
 	/**
+	* Allow Cognitive to restart a failed stream on another model. With fallback
+	* enabled, there is no buffered delivery and no final delivery queue: delta
+	* text and completed Chat.handler sends stream immediately during generation.
+	* A reset-only delta (restart: true) on Chat.onMessageDelta invalidates ALL
+	* current-iteration messages, including completed sends — consumers must
+	* retract/replace them, which is unsafe for irreversible external transports.
+	* Only TOOL/CODE execution waits for a successful stream. Streaming-only;
+	* defaults to false, which leaves progressive sending unchanged.
+	*/
+	midStreamFallback;
+	/**
 	* STT model used by the cognitive service to transcribe audio attachments
 	* when the target LLM does not support audio natively. Defaults to 'fast'.
 	*/
@@ -2509,10 +2520,12 @@ var Context = class {
 		this.snapshot = props.snapshot;
 		this.maxTokens = props.maxTokens;
 		this.maxTimeToFirstToken = props.maxTimeToFirstToken;
+		this.midStreamFallback = props.midStreamFallback;
 		this.transcriptionModel = props.transcriptionModel;
 		if (this.loop < 1 || this.loop > 100) throw new Error("Invalid loop. Expected a number between 1 and 100.");
 		if (this.maxTokens !== void 0 && (!Number.isFinite(this.maxTokens) || this.maxTokens < 1)) throw new Error("Invalid maxTokens. Expected a positive number.");
 		if (this.maxTimeToFirstToken !== void 0 && (!Number.isFinite(this.maxTimeToFirstToken) || this.maxTimeToFirstToken < 1)) throw new Error("Invalid maxTimeToFirstToken. Expected a positive number of milliseconds.");
+		if (this.midStreamFallback !== void 0 && typeof this.midStreamFallback !== "boolean") throw new Error("Invalid midStreamFallback. Expected a boolean.");
 	}
 	toJSON() {
 		var _this$snapshot4;

package/dist/{execute-BEaq3jkV.cjs → execute-BmEjJjka.cjs}

@@ -1,6 +1,6 @@
 const require__baseForOwn = require("./_baseForOwn-buOZDFbK.cjs");
 const require_dual_modes = require("./dual-modes-wAOAlaBS.cjs");
-const require_custom_client = require("./custom-client-BPmEoWKu.cjs");
+const require_custom_client = require("./custom-client-Cr33K60k.cjs");
 const require_utils = require("./utils-ZpJm00ed.cjs");
 const require_jsx = require("./jsx-BWvT6ukg.cjs");
 const require_stack_traces = require("./stack-traces-ntw_qhoy.cjs");
@@ -121,7 +121,7 @@ const RESPONSE_LENGTH_BUFFER = {
 	PERCENTAGE: .1
 };
 /** Maximum time to wait between two stream chunks before considering the stream stalled. */
-const STREAM_INACTIVITY_TIMEOUT = 6e4;
+const STREAM_INACTIVITY_TIMEOUT = 18e4;
 const getModelOutputLimit = (inputLength) => clamp(RESPONSE_LENGTH_BUFFER.PERCENTAGE * inputLength, RESPONSE_LENGTH_BUFFER.MIN_TOKENS, RESPONSE_LENGTH_BUFFER.MAX_TOKENS);
 /**
 * Models sometimes wrap their whole response in a code fence. The fence has to
@@ -197,6 +197,19 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 	/** Milliseconds between the stream request and the first/last streamed tokens. */
 	let timeToFirstToken;
 	let timeToLastToken;
+	const midStreamFallback = ctx.midStreamFallback === true;
+	let attempt = 1;
+	const messageMetadata = (itemId) => ({
+		iterationId: iteration.id,
+		id: midStreamFallback ? `${iteration.id}:${attempt}:${itemId}` : `${iteration.id}:${itemId}`
+	});
+	const preview = async (delta) => {
+		try {
+			await (onSendDelta === null || onSendDelta === void 0 ? void 0 : onSendDelta(delta));
+		} catch (err) {
+			if (delta.restart) throw new require_errors.CognitiveError(`LLM stream restart handler failed: ${require_utils.getErrorMessage(err)}`);
+		}
+	};
 	const liveItems = /* @__PURE__ */ new Map();
 	const liveContent = /* @__PURE__ */ new Map();
 	let codeGenerationTraced = false;
@@ -217,37 +230,39 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 			if ((item === null || item === void 0 ? void 0 : item.kind) !== "send") continue;
 			const content = (liveContent.get(item.id) ?? "") + event.delta;
 			liveContent.set(item.id, content);
-			try {
-				await onSendDelta({
-					id: `${iteration.id}:${item.id}`,
-					component: item.name,
-					props: item.props,
-					delta: event.delta,
-					content
-				});
-			} catch (err) {}
+			const delta = {
+				restart: false,
+				...messageMetadata(item.id),
+				component: item.name,
+				props: item.props,
+				delta: event.delta,
+				content
+			};
+			await preview(delta);
 		} else if (event.type === "item-complete") {
 			if (event.item.kind === "send" && onSend) await onSend({
 				name: event.item.name,
 				props: event.item.props,
 				body: event.item.body
-			});
+			}, messageMetadata(event.item.id));
 			else if (event.item.kind === "run" && event.item.status === "complete" && !runCompleted) {
 				runCompleted = true;
-				onRunComplete === null || onRunComplete === void 0 || onRunComplete((event.item.body ?? "").trim());
+				const code = (event.item.body ?? "").trim();
+				onRunComplete === null || onRunComplete === void 0 || onRunComplete(code);
 			}
 		}
 	};
 	if (typeof cognitive.generateTextStream === "function") {
-		const parser = new require_dual_modes.StreamingMessageParser();
-		const fence = new LeadingFenceFilter();
+		let parser = new require_dual_modes.StreamingMessageParser();
+		let fence = new LeadingFenceFilter();
 		const streamController = createJoinedAbortController([controller.signal]);
 		const requestedAt = Date.now();
 		const stream = cognitive.generateTextStream({
 			...input,
-			...ctx.maxTimeToFirstToken ? { options: {
+			...ctx.maxTimeToFirstToken || midStreamFallback ? { options: {
 				...input.options,
-				maxTimeToFirstToken: ctx.maxTimeToFirstToken
+				...ctx.maxTimeToFirstToken ? { maxTimeToFirstToken: ctx.maxTimeToFirstToken } : {},
+				...midStreamFallback ? { midStreamFallback: true } : {}
 			} } : {}
 		}, { signal: streamController.signal });
 		const inactivityTimeout = Math.max(STREAM_INACTIVITY_TIMEOUT, ctx.maxTimeToFirstToken ?? 0);
@@ -266,28 +281,66 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 			}
 		};
 		raw = "";
-		while (true) {
-			var _chunk$value, _chunk$value2;
-			let chunk;
-			try {
-				chunk = await nextChunk();
-			} catch (thrown) {
-				throw new require_errors.CognitiveError(`LLM generation failed: ${require_utils.getErrorMessage(thrown)}`);
+		let streamCompleted = false;
+		try {
+			while (true) {
+				var _chunk$value, _chunk$value2, _chunk$value3, _chunk$value4;
+				let chunk;
+				try {
+					chunk = await nextChunk();
+				} catch (thrown) {
+					throw new require_errors.CognitiveError(`LLM generation failed: ${require_utils.getErrorMessage(thrown)}`);
+				}
+				if (chunk.done) {
+					streamCompleted = true;
+					break;
+				}
+				if (((_chunk$value = chunk.value) === null || _chunk$value === void 0 ? void 0 : _chunk$value.restart) && !midStreamFallback) {
+					streamController.abort("Unexpected LLM stream restart");
+					throw new require_errors.CognitiveError("LLM stream restarted without options.midStreamFallback enabled");
+				}
+				if ((_chunk$value2 = chunk.value) === null || _chunk$value2 === void 0 ? void 0 : _chunk$value2.restart) {
+					traces.push({
+						type: "llm_call_restarted",
+						started_at: Date.now(),
+						...chunk.value.restart
+					});
+					raw = "";
+					parser = new require_dual_modes.StreamingMessageParser();
+					fence = new LeadingFenceFilter();
+					liveItems.clear();
+					liveContent.clear();
+					codeGenerationTraced = false;
+					runCompleted = false;
+					responseMetadata = void 0;
+					attempt = chunk.value.restart.attempt;
+					await preview({
+						...chunk.value.restart,
+						restart: true,
+						iterationId: iteration.id
+					});
+					timeToFirstToken = void 0;
+					timeToLastToken = void 0;
+					continue;
+				}
+				if ((_chunk$value3 = chunk.value) === null || _chunk$value3 === void 0 ? void 0 : _chunk$value3.metadata) responseMetadata = chunk.value.metadata;
+				const delta = (_chunk$value4 = chunk.value) === null || _chunk$value4 === void 0 ? void 0 : _chunk$value4.output;
+				if (!delta) continue;
+				timeToLastToken = Date.now() - requestedAt;
+				timeToFirstToken ??= timeToLastToken;
+				raw += delta;
+				await dispatchSends(parser.push(fence.push(delta)));
 			}
-			if (chunk.done) break;
-			if ((_chunk$value = chunk.value) === null || _chunk$value === void 0 ? void 0 : _chunk$value.metadata) responseMetadata = chunk.value.metadata;
-			const delta = (_chunk$value2 = chunk.value) === null || _chunk$value2 === void 0 ? void 0 : _chunk$value2.output;
-			if (!delta) continue;
-			timeToLastToken = Date.now() - requestedAt;
-			timeToFirstToken ??= timeToLastToken;
-			raw += delta;
-			await dispatchSends(parser.push(fence.push(delta)));
+			if (!responseMetadata) throw new require_errors.CognitiveError("LLM streaming completed without metadata");
+			const remaining = fence.flush();
+			if (remaining) await dispatchSends(parser.push(remaining));
+			await dispatchSends(parser.finish());
+			assistantResponse = require_dual_modes.toParsedAssistantResponse(parser.items, raw);
+		} finally {
+			streamController.abort("LLM stream closed");
+			if (!streamCompleted) stream.return(void 0).catch((err) => {});
 		}
-		if (!responseMetadata) throw new require_errors.CognitiveError("LLM streaming completed without metadata");
-		const remaining = fence.flush();
-		if (remaining) await dispatchSends(parser.push(remaining));
-		await dispatchSends(parser.finish());
-		assistantResponse = require_dual_modes.toParsedAssistantResponse(parser.items, raw);
+		if (midStreamFallback) controller.signal.throwIfAborted();
 	} else {
 		const response = await cognitive.generateText(input, { signal: controller.signal }).catch((thrown) => {
 			throw new require_errors.CognitiveError(`LLM generation failed: ${require_utils.getErrorMessage(thrown)}`);
@@ -296,7 +349,7 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		responseMetadata = response.metadata;
 		raw = response.output;
 		assistantResponse = ctx.version.parseAssistantResponse(raw);
-		for (const send of assistantResponse.sends) await (onSend === null || onSend === void 0 ? void 0 : onSend(send));
+		for (const [index, send] of assistantResponse.sends.entries()) await (onSend === null || onSend === void 0 ? void 0 : onSend(send, messageMetadata(`send-${index}`)));
 	}
 	iteration.code = assistantResponse.code;
 	iteration.sends = assistantResponse.sends;
@@ -329,7 +382,7 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		type: "llm_call_success",
 		started_at: startedAt,
 		ended_at: iteration.llm.ended_at,
-		model: model.id,
+		model: iteration.llm.model,
 		code: iteration.code ?? ""
 	});
 };
@@ -572,7 +625,10 @@ function wrapTool({ chat, tool, traces, object, iteration, beforeHook, afterHook
 				toolCallId
 			}));
 			if (typeof (beforeRes === null || beforeRes === void 0 ? void 0 : beforeRes.input) !== "undefined") effectiveInput = beforeRes.input;
-			output = await tool.execute(effectiveInput, { callId: toolCallId }, chat);
+			output = await tool.execute(effectiveInput, {
+				callId: toolCallId,
+				iterationId: iteration.id
+			}, chat);
 			const afterRes = await (afterHook === null || afterHook === void 0 ? void 0 : afterHook({
 				iteration,
 				tool,
@@ -677,7 +733,7 @@ const executeContext = async (props) => {
 	return result;
 };
 const executeContextInternal = async (props) => {
-	var _props$options, _props$options2, _props$options3, _props$options4, _props$options5;
+	var _props$options, _props$options2, _props$options3, _props$options4, _props$options5, _props$options6;
 	const controller = createJoinedAbortController([props.signal]);
 	const { onIterationStart, onIterationEnd, onTrace, onExit, onBeforeExecution, onAfterTool, onBeforeTool } = props;
 	const client = props.client ?? new _botpress_client.Client();
@@ -691,7 +747,8 @@ const executeContextInternal = async (props) => {
 		timeout: (_props$options2 = props.options) === null || _props$options2 === void 0 ? void 0 : _props$options2.timeout,
 		maxTokens: (_props$options3 = props.options) === null || _props$options3 === void 0 ? void 0 : _props$options3.maxTokens,
 		maxTimeToFirstToken: (_props$options4 = props.options) === null || _props$options4 === void 0 ? void 0 : _props$options4.maxTimeToFirstToken,
-		transcriptionModel: (_props$options5 = props.options) === null || _props$options5 === void 0 ? void 0 : _props$options5.transcriptionModel,
+		midStreamFallback: (_props$options5 = props.options) === null || _props$options5 === void 0 ? void 0 : _props$options5.midStreamFallback,
+		transcriptionModel: (_props$options6 = props.options) === null || _props$options6 === void 0 ? void 0 : _props$options6.transcriptionModel,
 		exits: props.exits,
 		snapshot: props.snapshot,
 		model: props.model,
@@ -884,7 +941,7 @@ const executeIteration = async ({ iteration, ctx, cognitive, controller, onExit,
 			};
 		});
 	};
-	const canExecuteEarly = typeof onBeforeExecution !== "function";
+	const canExecuteEarly = typeof onBeforeExecution !== "function" && !ctx.midStreamFallback;
 	let earlyExecution;
 	await generateCode({
 		iteration,
@@ -902,7 +959,7 @@ const executeIteration = async ({ iteration, ctx, cognitive, controller, onExit,
 				promise: runCode(code)
 			};
 		} : void 0,
-		onSend: async (send) => {
+		onSend: async (send, messageMetadata) => {
 			if (!ctx.chat) return;
 			const sendStartedAt = Date.now();
 			const component = require_jsx.createJsxComponent({
@@ -911,7 +968,7 @@ const executeIteration = async ({ iteration, ctx, cognitive, controller, onExit,
 				children: send.body ? [send.body] : []
 			});
 			try {
-				await ctx.chat.handler(component);
+				await ctx.chat.handler(component, messageMetadata);
 			} catch (err) {
 				throw new Error(`Error while sending message (■send=${send.name}): ${require_utils.getErrorMessage(err)}`);
 			}

package/dist/{execute-DYvFRs54.js → execute-sn9RT_IN.js}

@@ -1,5 +1,5 @@
 import { a as StreamingMessageParser, d as isEqual, h as toNumber, i as toParsedAssistantResponse } from "./dual-modes-D_p-jedw.js";
-import { d as Snapshot, f as parseExit, l as PartialExecutionResult, n as Context, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-NUoxrFQh.js";
+import { d as Snapshot, f as parseExit, l as PartialExecutionResult, n as Context, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-CJIThG2H.js";
 import { d as stripInvalidIdentifiers, i as getErrorMessage, s as init } from "./utils-UBeAFeEI.js";
 import { t as createJsxComponent } from "./jsx-CIgdGGVs.js";
 import { t as cleanStackTrace } from "./stack-traces-BIE_5Xwf.js";
@@ -119,7 +119,7 @@ const RESPONSE_LENGTH_BUFFER = {
 	PERCENTAGE: .1
 };
 /** Maximum time to wait between two stream chunks before considering the stream stalled. */
-const STREAM_INACTIVITY_TIMEOUT = 6e4;
+const STREAM_INACTIVITY_TIMEOUT = 18e4;
 const getModelOutputLimit = (inputLength) => clamp(RESPONSE_LENGTH_BUFFER.PERCENTAGE * inputLength, RESPONSE_LENGTH_BUFFER.MIN_TOKENS, RESPONSE_LENGTH_BUFFER.MAX_TOKENS);
 /**
 * Models sometimes wrap their whole response in a code fence. The fence has to
@@ -195,6 +195,19 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 	/** Milliseconds between the stream request and the first/last streamed tokens. */
 	let timeToFirstToken;
 	let timeToLastToken;
+	const midStreamFallback = ctx.midStreamFallback === true;
+	let attempt = 1;
+	const messageMetadata = (itemId) => ({
+		iterationId: iteration.id,
+		id: midStreamFallback ? `${iteration.id}:${attempt}:${itemId}` : `${iteration.id}:${itemId}`
+	});
+	const preview = async (delta) => {
+		try {
+			await (onSendDelta === null || onSendDelta === void 0 ? void 0 : onSendDelta(delta));
+		} catch (err) {
+			if (delta.restart) throw new CognitiveError(`LLM stream restart handler failed: ${getErrorMessage(err)}`);
+		}
+	};
 	const liveItems = /* @__PURE__ */ new Map();
 	const liveContent = /* @__PURE__ */ new Map();
 	let codeGenerationTraced = false;
@@ -215,37 +228,39 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 			if ((item === null || item === void 0 ? void 0 : item.kind) !== "send") continue;
 			const content = (liveContent.get(item.id) ?? "") + event.delta;
 			liveContent.set(item.id, content);
-			try {
-				await onSendDelta({
-					id: `${iteration.id}:${item.id}`,
-					component: item.name,
-					props: item.props,
-					delta: event.delta,
-					content
-				});
-			} catch (err) {}
+			const delta = {
+				restart: false,
+				...messageMetadata(item.id),
+				component: item.name,
+				props: item.props,
+				delta: event.delta,
+				content
+			};
+			await preview(delta);
 		} else if (event.type === "item-complete") {
 			if (event.item.kind === "send" && onSend) await onSend({
 				name: event.item.name,
 				props: event.item.props,
 				body: event.item.body
-			});
+			}, messageMetadata(event.item.id));
 			else if (event.item.kind === "run" && event.item.status === "complete" && !runCompleted) {
 				runCompleted = true;
-				onRunComplete === null || onRunComplete === void 0 || onRunComplete((event.item.body ?? "").trim());
+				const code = (event.item.body ?? "").trim();
+				onRunComplete === null || onRunComplete === void 0 || onRunComplete(code);
 			}
 		}
 	};
 	if (typeof cognitive.generateTextStream === "function") {
-		const parser = new StreamingMessageParser();
-		const fence = new LeadingFenceFilter();
+		let parser = new StreamingMessageParser();
+		let fence = new LeadingFenceFilter();
 		const streamController = createJoinedAbortController([controller.signal]);
 		const requestedAt = Date.now();
 		const stream = cognitive.generateTextStream({
 			...input,
-			...ctx.maxTimeToFirstToken ? { options: {
+			...ctx.maxTimeToFirstToken || midStreamFallback ? { options: {
 				...input.options,
-				maxTimeToFirstToken: ctx.maxTimeToFirstToken
+				...ctx.maxTimeToFirstToken ? { maxTimeToFirstToken: ctx.maxTimeToFirstToken } : {},
+				...midStreamFallback ? { midStreamFallback: true } : {}
 			} } : {}
 		}, { signal: streamController.signal });
 		const inactivityTimeout = Math.max(STREAM_INACTIVITY_TIMEOUT, ctx.maxTimeToFirstToken ?? 0);
@@ -264,28 +279,66 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 			}
 		};
 		raw = "";
-		while (true) {
-			var _chunk$value, _chunk$value2;
-			let chunk;
-			try {
-				chunk = await nextChunk();
-			} catch (thrown) {
-				throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
+		let streamCompleted = false;
+		try {
+			while (true) {
+				var _chunk$value, _chunk$value2, _chunk$value3, _chunk$value4;
+				let chunk;
+				try {
+					chunk = await nextChunk();
+				} catch (thrown) {
+					throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
+				}
+				if (chunk.done) {
+					streamCompleted = true;
+					break;
+				}
+				if (((_chunk$value = chunk.value) === null || _chunk$value === void 0 ? void 0 : _chunk$value.restart) && !midStreamFallback) {
+					streamController.abort("Unexpected LLM stream restart");
+					throw new CognitiveError("LLM stream restarted without options.midStreamFallback enabled");
+				}
+				if ((_chunk$value2 = chunk.value) === null || _chunk$value2 === void 0 ? void 0 : _chunk$value2.restart) {
+					traces.push({
+						type: "llm_call_restarted",
+						started_at: Date.now(),
+						...chunk.value.restart
+					});
+					raw = "";
+					parser = new StreamingMessageParser();
+					fence = new LeadingFenceFilter();
+					liveItems.clear();
+					liveContent.clear();
+					codeGenerationTraced = false;
+					runCompleted = false;
+					responseMetadata = void 0;
+					attempt = chunk.value.restart.attempt;
+					await preview({
+						...chunk.value.restart,
+						restart: true,
+						iterationId: iteration.id
+					});
+					timeToFirstToken = void 0;
+					timeToLastToken = void 0;
+					continue;
+				}
+				if ((_chunk$value3 = chunk.value) === null || _chunk$value3 === void 0 ? void 0 : _chunk$value3.metadata) responseMetadata = chunk.value.metadata;
+				const delta = (_chunk$value4 = chunk.value) === null || _chunk$value4 === void 0 ? void 0 : _chunk$value4.output;
+				if (!delta) continue;
+				timeToLastToken = Date.now() - requestedAt;
+				timeToFirstToken ??= timeToLastToken;
+				raw += delta;
+				await dispatchSends(parser.push(fence.push(delta)));
 			}
-			if (chunk.done) break;
-			if ((_chunk$value = chunk.value) === null || _chunk$value === void 0 ? void 0 : _chunk$value.metadata) responseMetadata = chunk.value.metadata;
-			const delta = (_chunk$value2 = chunk.value) === null || _chunk$value2 === void 0 ? void 0 : _chunk$value2.output;
-			if (!delta) continue;
-			timeToLastToken = Date.now() - requestedAt;
-			timeToFirstToken ??= timeToLastToken;
-			raw += delta;
-			await dispatchSends(parser.push(fence.push(delta)));
+			if (!responseMetadata) throw new CognitiveError("LLM streaming completed without metadata");
+			const remaining = fence.flush();
+			if (remaining) await dispatchSends(parser.push(remaining));
+			await dispatchSends(parser.finish());
+			assistantResponse = toParsedAssistantResponse(parser.items, raw);
+		} finally {
+			streamController.abort("LLM stream closed");
+			if (!streamCompleted) stream.return(void 0).catch((err) => {});
 		}
-		if (!responseMetadata) throw new CognitiveError("LLM streaming completed without metadata");
-		const remaining = fence.flush();
-		if (remaining) await dispatchSends(parser.push(remaining));
-		await dispatchSends(parser.finish());
-		assistantResponse = toParsedAssistantResponse(parser.items, raw);
+		if (midStreamFallback) controller.signal.throwIfAborted();
 	} else {
 		const response = await cognitive.generateText(input, { signal: controller.signal }).catch((thrown) => {
 			throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
@@ -294,7 +347,7 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		responseMetadata = response.metadata;
 		raw = response.output;
 		assistantResponse = ctx.version.parseAssistantResponse(raw);
-		for (const send of assistantResponse.sends) await (onSend === null || onSend === void 0 ? void 0 : onSend(send));
+		for (const [index, send] of assistantResponse.sends.entries()) await (onSend === null || onSend === void 0 ? void 0 : onSend(send, messageMetadata(`send-${index}`)));
 	}
 	iteration.code = assistantResponse.code;
 	iteration.sends = assistantResponse.sends;
@@ -327,7 +380,7 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		type: "llm_call_success",
 		started_at: startedAt,
 		ended_at: iteration.llm.ended_at,
-		model: model.id,
+		model: iteration.llm.model,
 		code: iteration.code ?? ""
 	});
 };
@@ -570,7 +623,10 @@ function wrapTool({ chat, tool, traces, object, iteration, beforeHook, afterHook
 				toolCallId
 			}));
 			if (typeof (beforeRes === null || beforeRes === void 0 ? void 0 : beforeRes.input) !== "undefined") effectiveInput = beforeRes.input;
-			output = await tool.execute(effectiveInput, { callId: toolCallId }, chat);
+			output = await tool.execute(effectiveInput, {
+				callId: toolCallId,
+				iterationId: iteration.id
+			}, chat);
 			const afterRes = await (afterHook === null || afterHook === void 0 ? void 0 : afterHook({
 				iteration,
 				tool,
@@ -675,7 +731,7 @@ const executeContext = async (props) => {
 	return result;
 };
 const executeContextInternal = async (props) => {
-	var _props$options, _props$options2, _props$options3, _props$options4, _props$options5;
+	var _props$options, _props$options2, _props$options3, _props$options4, _props$options5, _props$options6;
 	const controller = createJoinedAbortController([props.signal]);
 	const { onIterationStart, onIterationEnd, onTrace, onExit, onBeforeExecution, onAfterTool, onBeforeTool } = props;
 	const client = props.client ?? new Client();
@@ -689,7 +745,8 @@ const executeContextInternal = async (props) => {
 		timeout: (_props$options2 = props.options) === null || _props$options2 === void 0 ? void 0 : _props$options2.timeout,
 		maxTokens: (_props$options3 = props.options) === null || _props$options3 === void 0 ? void 0 : _props$options3.maxTokens,
 		maxTimeToFirstToken: (_props$options4 = props.options) === null || _props$options4 === void 0 ? void 0 : _props$options4.maxTimeToFirstToken,
-		transcriptionModel: (_props$options5 = props.options) === null || _props$options5 === void 0 ? void 0 : _props$options5.transcriptionModel,
+		midStreamFallback: (_props$options5 = props.options) === null || _props$options5 === void 0 ? void 0 : _props$options5.midStreamFallback,
+		transcriptionModel: (_props$options6 = props.options) === null || _props$options6 === void 0 ? void 0 : _props$options6.transcriptionModel,
 		exits: props.exits,
 		snapshot: props.snapshot,
 		model: props.model,
@@ -882,7 +939,7 @@ const executeIteration = async ({ iteration, ctx, cognitive, controller, onExit,
 			};
 		});
 	};
-	const canExecuteEarly = typeof onBeforeExecution !== "function";
+	const canExecuteEarly = typeof onBeforeExecution !== "function" && !ctx.midStreamFallback;
 	let earlyExecution;
 	await generateCode({
 		iteration,
@@ -900,7 +957,7 @@ const executeIteration = async ({ iteration, ctx, cognitive, controller, onExit,
 				promise: runCode(code)
 			};
 		} : void 0,
-		onSend: async (send) => {
+		onSend: async (send, messageMetadata) => {
 			if (!ctx.chat) return;
 			const sendStartedAt = Date.now();
 			const component = createJsxComponent({
@@ -909,7 +966,7 @@ const executeIteration = async ({ iteration, ctx, cognitive, controller, onExit,
 				children: send.body ? [send.body] : []
 			});
 			try {
-				await ctx.chat.handler(component);
+				await ctx.chat.handler(component, messageMetadata);
 			} catch (err) {
 				throw new Error(`Error while sending message (■send=${send.name}): ${getErrorMessage(err)}`);
 			}

package/dist/index.cjs

@@ -1,8 +1,8 @@
 Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
 const require_dual_modes = require("./dual-modes-wAOAlaBS.cjs");
-const require_custom_client = require("./custom-client-BPmEoWKu.cjs");
+const require_custom_client = require("./custom-client-Cr33K60k.cjs");
 const require_utils = require("./utils-ZpJm00ed.cjs");
-const require_tool = require("./tool-DdPZb0JW.cjs");
+const require_tool = require("./tool-ReA9gxMp.cjs");
 const require_errors = require("./errors-C962uJdS.cjs");
 const require_typings = require("./typings-BwAz_TM9.cjs");
 const require_exit = require("./exit-BGi0Kk0y.cjs");
@@ -11,7 +11,7 @@ const require_truncator = require("./truncator-B6aaJaN1.cjs");
 const require_quickjs_variant = require("./quickjs-variant-CsIwTVFC.cjs");
 let _bpinternal_zui = require("@bpinternal/zui");
 //#region package.json
-var version = "0.6.4";
+var version = "0.6.6";
 //#endregion
 //#region src/hoist.ts
 /**
@@ -1774,7 +1774,7 @@ const utils = {
 * })
 */
 const execute = async (props) => {
-	const { executeContext } = await Promise.resolve().then(() => require("./execute-BEaq3jkV.cjs"));
+	const { executeContext } = await Promise.resolve().then(() => require("./execute-BmEjJjka.cjs"));
 	return executeContext(props);
 };
 /**
@@ -1783,9 +1783,9 @@ const execute = async (props) => {
 * It's recommended to call this function at the beginning of your application without awaiting it (void init())
 */
 const init = async () => {
-	await Promise.resolve().then(() => require("./execute-BEaq3jkV.cjs"));
+	await Promise.resolve().then(() => require("./execute-BmEjJjka.cjs"));
 	await Promise.resolve().then(() => require("./component-BwixNVH8.cjs")).then((n) => n.component_exports);
-	await Promise.resolve().then(() => require("./tool-DdPZb0JW.cjs")).then((n) => n.tool_exports);
+	await Promise.resolve().then(() => require("./tool-ReA9gxMp.cjs")).then((n) => n.tool_exports);
 	await Promise.resolve().then(() => require("./exit-BGi0Kk0y.cjs")).then((n) => n.exit_exports);
 	await Promise.resolve().then(() => require("./jsx-BWvT6ukg.cjs")).then((n) => n.jsx_exports);
 	await Promise.resolve().then(() => require("./vm-BjkJqSQH.cjs"));

package/dist/index.d.ts

@@ -8,7 +8,7 @@ export { Component, type RenderedComponent, type LeafComponentDefinition, type C
 export { type Citation, CitationsManager } from './citations.js';
 export { DefaultComponents } from './component.default.js';
 export { Snapshot } from './snapshots.js';
-export { Chat, type MessageHandler, type MessageDelta, type MessageDeltaHandler } from './chat.js';
+export { Chat, type MessageHandler, type MessageMetadata, type MessageDelta, type MessageDeltaHandler } from './chat.js';
 import { ExecutionResult } from './result.js';
 import { type ExecutionProps } from './runtime/types.js';
 import { stripTruncationTags, truncateWrappedContent, wrapContent } from './truncator.js';

package/dist/index.js

@@ -1,7 +1,7 @@
 import { a as StreamingMessageParser, c as NAME_REGEX, o as tryParseJson, r as generateInstructions, s as MARKER } from "./dual-modes-D_p-jedw.js";
-import { a as ThinkExit, c as ExecutionResult, d as Snapshot, f as parseExit, i as ListenExit, l as PartialExecutionResult, o as getValue, r as DefaultExit, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-NUoxrFQh.js";
+import { a as ThinkExit, c as ExecutionResult, d as Snapshot, f as parseExit, i as ListenExit, l as PartialExecutionResult, o as getValue, r as DefaultExit, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-CJIThG2H.js";
 import { a as getMultilineComment, c as isJsonSchema, l as isValidIdentifier, m as toValidObjectName, n as escapeString, p as toValidFunctionName, t as configureTokenizer, u as isZuiSchema, v as upperFirst } from "./utils-UBeAFeEI.js";
-import { t as Tool } from "./tool-CAo-BQHs.js";
+import { t as Tool } from "./tool-Bv8N4IPj.js";
 import { c as SnapshotSignal, l as ThinkSignal, o as LoopExceededError, r as CodeFormattingError } from "./errors-Cg1G0Awu.js";
 import { a as isPunct, i as isGroup, o as parse, r as formatTypings, s as tokenize, t as getTypings } from "./typings-CrBa7Tyy.js";
 import { t as Exit } from "./exit-CDdxDBRT.js";
@@ -10,7 +10,7 @@ import { i as wrapContent, n as truncateWrappedContent, t as stripTruncationTags
 import { t as configureQuickJS } from "./quickjs-variant-BYhMBpwU.js";
 import { transforms, z } from "@bpinternal/zui";
 //#region package.json
-var version = "0.6.4";
+var version = "0.6.6";
 //#endregion
 //#region src/hoist.ts
 /**
@@ -1773,7 +1773,7 @@ const utils = {
 * })
 */
 const execute = async (props) => {
-	const { executeContext } = await import("./execute-DYvFRs54.js");
+	const { executeContext } = await import("./execute-sn9RT_IN.js");
 	return executeContext(props);
 };
 /**
@@ -1782,9 +1782,9 @@ const execute = async (props) => {
 * It's recommended to call this function at the beginning of your application without awaiting it (void init())
 */
 const init = async () => {
-	await import("./execute-DYvFRs54.js");
+	await import("./execute-sn9RT_IN.js");
 	await import("./component-DGG7d_Mz.js").then((n) => n.r);
-	await import("./tool-CAo-BQHs.js").then((n) => n.n);
+	await import("./tool-Bv8N4IPj.js").then((n) => n.n);
 	await import("./exit-CDdxDBRT.js").then((n) => n.n);
 	await import("./jsx-CIgdGGVs.js").then((n) => n.i);
 	await import("./vm-C5vf1LH3.js");

package/dist/index.workerd.js

@@ -52,7 +52,7 @@ var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__ge
 }) : target, mod));
 //#endregion
 //#region package.json
-var version = "0.6.4";
+var version = "0.6.6";
 //#endregion
 //#region ../../node_modules/.pnpm/lodash-es@4.17.21/node_modules/lodash-es/_freeGlobal.js
 var freeGlobal;
@@ -7682,7 +7682,10 @@ var init_tool = __esmMin((() => {
 				if (done) return value;
 				if (yieldIndex >= yieldedCount) {
 					setYieldedCount(yieldIndex + 1);
-					await chat?.handler?.(value);
+					await chat?.handler?.(value, {
+						iterationId: ctx.iterationId ?? ctx.callId,
+						id: `${ctx.callId}:yield-${yieldIndex}`
+					});
 				}
 				yieldIndex++;
 			}
@@ -12215,6 +12218,17 @@ var init_context = __esmMin((() => {
 		*/
 		maxTimeToFirstToken;
 		/**
+		* Allow Cognitive to restart a failed stream on another model. With fallback
+		* enabled, there is no buffered delivery and no final delivery queue: delta
+		* text and completed Chat.handler sends stream immediately during generation.
+		* A reset-only delta (restart: true) on Chat.onMessageDelta invalidates ALL
+		* current-iteration messages, including completed sends — consumers must
+		* retract/replace them, which is unsafe for irreversible external transports.
+		* Only TOOL/CODE execution waits for a successful stream. Streaming-only;
+		* defaults to false, which leaves progressive sending unchanged.
+		*/
+		midStreamFallback;
+		/**
 		* STT model used by the cognitive service to transcribe audio attachments
 		* when the target LLM does not support audio natively. Defaults to 'fast'.
 		*/
@@ -12449,10 +12463,12 @@ var init_context = __esmMin((() => {
 			this.snapshot = props.snapshot;
 			this.maxTokens = props.maxTokens;
 			this.maxTimeToFirstToken = props.maxTimeToFirstToken;
+			this.midStreamFallback = props.midStreamFallback;
 			this.transcriptionModel = props.transcriptionModel;
 			if (this.loop < 1 || this.loop > 100) throw new Error("Invalid loop. Expected a number between 1 and 100.");
 			if (this.maxTokens !== void 0 && (!Number.isFinite(this.maxTokens) || this.maxTokens < 1)) throw new Error("Invalid maxTokens. Expected a positive number.");
 			if (this.maxTimeToFirstToken !== void 0 && (!Number.isFinite(this.maxTimeToFirstToken) || this.maxTimeToFirstToken < 1)) throw new Error("Invalid maxTimeToFirstToken. Expected a positive number of milliseconds.");
+			if (this.midStreamFallback !== void 0 && typeof this.midStreamFallback !== "boolean") throw new Error("Invalid midStreamFallback. Expected a boolean.");
 		}
 		toJSON() {
 			return {
@@ -17023,7 +17039,7 @@ var init_generate = __esmMin((() => {
 		MAX_TOKENS: 16e3,
 		PERCENTAGE: .1
 	};
-	STREAM_INACTIVITY_TIMEOUT = 6e4;
+	STREAM_INACTIVITY_TIMEOUT = 18e4;
 	getModelOutputLimit = (inputLength) => clamp(RESPONSE_LENGTH_BUFFER.PERCENTAGE * inputLength, RESPONSE_LENGTH_BUFFER.MIN_TOKENS, RESPONSE_LENGTH_BUFFER.MAX_TOKENS);
 	LeadingFenceFilter = class {
 		_buffer = "";
@@ -17093,6 +17109,19 @@ var init_generate = __esmMin((() => {
 		/** Milliseconds between the stream request and the first/last streamed tokens. */
 		let timeToFirstToken;
 		let timeToLastToken;
+		const midStreamFallback = ctx.midStreamFallback === true;
+		let attempt = 1;
+		const messageMetadata = (itemId) => ({
+			iterationId: iteration.id,
+			id: midStreamFallback ? `${iteration.id}:${attempt}:${itemId}` : `${iteration.id}:${itemId}`
+		});
+		const preview = async (delta) => {
+			try {
+				await onSendDelta?.(delta);
+			} catch (err) {
+				if (delta.restart) throw new CognitiveError(`LLM stream restart handler failed: ${getErrorMessage(err)}`);
+			}
+		};
 		const liveItems = /* @__PURE__ */ new Map();
 		const liveContent = /* @__PURE__ */ new Map();
 		let codeGenerationTraced = false;
@@ -17113,37 +17142,39 @@ var init_generate = __esmMin((() => {
 				if (item?.kind !== "send") continue;
 				const content = (liveContent.get(item.id) ?? "") + event.delta;
 				liveContent.set(item.id, content);
-				try {
-					await onSendDelta({
-						id: `${iteration.id}:${item.id}`,
-						component: item.name,
-						props: item.props,
-						delta: event.delta,
-						content
-					});
-				} catch (err) {}
+				const delta = {
+					restart: false,
+					...messageMetadata(item.id),
+					component: item.name,
+					props: item.props,
+					delta: event.delta,
+					content
+				};
+				await preview(delta);
 			} else if (event.type === "item-complete") {
 				if (event.item.kind === "send" && onSend) await onSend({
 					name: event.item.name,
 					props: event.item.props,
 					body: event.item.body
-				});
+				}, messageMetadata(event.item.id));
 				else if (event.item.kind === "run" && event.item.status === "complete" && !runCompleted) {
 					runCompleted = true;
-					onRunComplete?.((event.item.body ?? "").trim());
+					const code = (event.item.body ?? "").trim();
+					onRunComplete?.(code);
 				}
 			}
 		};
 		if (typeof cognitive.generateTextStream === "function") {
-			const parser = new StreamingMessageParser();
-			const fence = new LeadingFenceFilter();
+			let parser = new StreamingMessageParser();
+			let fence = new LeadingFenceFilter();
 			const streamController = createJoinedAbortController([controller.signal]);
 			const requestedAt = Date.now();
 			const stream = cognitive.generateTextStream({
 				...input,
-				...ctx.maxTimeToFirstToken ? { options: {
+				...ctx.maxTimeToFirstToken || midStreamFallback ? { options: {
 					...input.options,
-					maxTimeToFirstToken: ctx.maxTimeToFirstToken
+					...ctx.maxTimeToFirstToken ? { maxTimeToFirstToken: ctx.maxTimeToFirstToken } : {},
+					...midStreamFallback ? { midStreamFallback: true } : {}
 				} } : {}
 			}, { signal: streamController.signal });
 			const inactivityTimeout = Math.max(STREAM_INACTIVITY_TIMEOUT, ctx.maxTimeToFirstToken ?? 0);
@@ -17162,27 +17193,65 @@ var init_generate = __esmMin((() => {
 				}
 			};
 			raw = "";
-			while (true) {
-				let chunk;
-				try {
-					chunk = await nextChunk();
-				} catch (thrown) {
-					throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
+			let streamCompleted = false;
+			try {
+				while (true) {
+					let chunk;
+					try {
+						chunk = await nextChunk();
+					} catch (thrown) {
+						throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
+					}
+					if (chunk.done) {
+						streamCompleted = true;
+						break;
+					}
+					if (chunk.value?.restart && !midStreamFallback) {
+						streamController.abort("Unexpected LLM stream restart");
+						throw new CognitiveError("LLM stream restarted without options.midStreamFallback enabled");
+					}
+					if (chunk.value?.restart) {
+						traces.push({
+							type: "llm_call_restarted",
+							started_at: Date.now(),
+							...chunk.value.restart
+						});
+						raw = "";
+						parser = new StreamingMessageParser();
+						fence = new LeadingFenceFilter();
+						liveItems.clear();
+						liveContent.clear();
+						codeGenerationTraced = false;
+						runCompleted = false;
+						responseMetadata = void 0;
+						attempt = chunk.value.restart.attempt;
+						await preview({
+							...chunk.value.restart,
+							restart: true,
+							iterationId: iteration.id
+						});
+						timeToFirstToken = void 0;
+						timeToLastToken = void 0;
+						continue;
+					}
+					if (chunk.value?.metadata) responseMetadata = chunk.value.metadata;
+					const delta = chunk.value?.output;
+					if (!delta) continue;
+					timeToLastToken = Date.now() - requestedAt;
+					timeToFirstToken ??= timeToLastToken;
+					raw += delta;
+					await dispatchSends(parser.push(fence.push(delta)));
 				}
-				if (chunk.done) break;
-				if (chunk.value?.metadata) responseMetadata = chunk.value.metadata;
-				const delta = chunk.value?.output;
-				if (!delta) continue;
-				timeToLastToken = Date.now() - requestedAt;
-				timeToFirstToken ??= timeToLastToken;
-				raw += delta;
-				await dispatchSends(parser.push(fence.push(delta)));
+				if (!responseMetadata) throw new CognitiveError("LLM streaming completed without metadata");
+				const remaining = fence.flush();
+				if (remaining) await dispatchSends(parser.push(remaining));
+				await dispatchSends(parser.finish());
+				assistantResponse = toParsedAssistantResponse(parser.items, raw);
+			} finally {
+				streamController.abort("LLM stream closed");
+				if (!streamCompleted) stream.return(void 0).catch((err) => {});
 			}
-			if (!responseMetadata) throw new CognitiveError("LLM streaming completed without metadata");
-			const remaining = fence.flush();
-			if (remaining) await dispatchSends(parser.push(remaining));
-			await dispatchSends(parser.finish());
-			assistantResponse = toParsedAssistantResponse(parser.items, raw);
+			if (midStreamFallback) controller.signal.throwIfAborted();
 		} else {
 			const response = await cognitive.generateText(input, { signal: controller.signal }).catch((thrown) => {
 				throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
@@ -17191,7 +17260,7 @@ var init_generate = __esmMin((() => {
 			responseMetadata = response.metadata;
 			raw = response.output;
 			assistantResponse = ctx.version.parseAssistantResponse(raw);
-			for (const send of assistantResponse.sends) await onSend?.(send);
+			for (const [index, send] of assistantResponse.sends.entries()) await onSend?.(send, messageMetadata(`send-${index}`));
 		}
 		iteration.code = assistantResponse.code;
 		iteration.sends = assistantResponse.sends;
@@ -17224,7 +17293,7 @@ var init_generate = __esmMin((() => {
 			type: "llm_call_success",
 			started_at: startedAt,
 			ended_at: iteration.llm.ended_at,
-			model: model.id,
+			model: iteration.llm.model,
 			code: iteration.code ?? ""
 		});
 	};
@@ -17471,7 +17540,10 @@ function wrapTool({ chat, tool, traces, object, iteration, beforeHook, afterHook
 				toolCallId
 			});
 			if (typeof beforeRes?.input !== "undefined") effectiveInput = beforeRes.input;
-			output = await tool.execute(effectiveInput, { callId: toolCallId }, chat);
+			output = await tool.execute(effectiveInput, {
+				callId: toolCallId,
+				iterationId: iteration.id
+			}, chat);
 			const afterRes = await afterHook?.({
 				iteration,
 				tool,
@@ -17614,6 +17686,7 @@ var init_execute = __esmMin((() => {
 			timeout: props.options?.timeout,
 			maxTokens: props.options?.maxTokens,
 			maxTimeToFirstToken: props.options?.maxTimeToFirstToken,
+			midStreamFallback: props.options?.midStreamFallback,
 			transcriptionModel: props.options?.transcriptionModel,
 			exits: props.exits,
 			snapshot: props.snapshot,
@@ -17805,7 +17878,7 @@ var init_execute = __esmMin((() => {
 				};
 			});
 		};
-		const canExecuteEarly = typeof onBeforeExecution !== "function";
+		const canExecuteEarly = typeof onBeforeExecution !== "function" && !ctx.midStreamFallback;
 		let earlyExecution;
 		await generateCode({
 			iteration,
@@ -17823,7 +17896,7 @@ var init_execute = __esmMin((() => {
 					promise: runCode(code)
 				};
 			} : void 0,
-			onSend: async (send) => {
+			onSend: async (send, messageMetadata) => {
 				if (!ctx.chat) return;
 				const sendStartedAt = Date.now();
 				const component = createJsxComponent({
@@ -17832,7 +17905,7 @@ var init_execute = __esmMin((() => {
 					children: send.body ? [send.body] : []
 				});
 				try {
-					await ctx.chat.handler(component);
+					await ctx.chat.handler(component, messageMetadata);
 				} catch (err) {
 					throw new Error(`Error while sending message (■send=${send.name}): ${getErrorMessage(err)}`);
 				}

package/dist/runtime/generate.d.ts

@@ -1,4 +1,4 @@
-import type { MessageDelta } from '../chat.js';
+import type { MessageDelta, MessageMetadata } from '../chat.js';
 import { Context, Iteration } from '../context.js';
 import type { ParsedSend } from '../prompts/prompt.js';
 import { RuntimeCognitive } from './types.js';
@@ -12,10 +12,11 @@ type GenerateCodeProps = {
      * Called for each completed `■send` block. On streaming clients this fires
      * while the model is still generating — messages are delivered progressively.
      */
-    onSend?: (send: ParsedSend) => Promise<void>;
+    onSend?: (send: ParsedSend, metadata: MessageMetadata) => Promise<void>;
     /**
      * Called for each `■send` body chunk as it is parsed from the stream
-     * (streaming clients only). Best-effort: errors are ignored.
+     * (streaming clients only), or with a restart delta before replacement output.
+     * Text errors are best-effort; restart errors terminate generation.
      */
     onSendDelta?: (delta: MessageDelta) => Promise<void> | void;
     /**

package/dist/runtime/types.d.ts

@@ -122,6 +122,16 @@ type Options = Partial<Pick<Context, 'loop' | 'timeout'>> & {
      * `model` is an array of fallback models.
      */
     maxTimeToFirstToken?: number;
+    /**
+     * Allow Cognitive to restart failed streams on another model. No buffered
+     * delivery and no final delivery queue: delta text and completed Chat.handler
+     * sends stream immediately during generation. A reset-only delta (restart:
+     * true) on Chat.onMessageDelta invalidates ALL current-iteration messages,
+     * including completed sends — consumers must retract/replace them (not safe for
+     * irreversible transports). Only TOOL/CODE execution waits for a successful
+     * stream. Streaming-only; defaults to false, preserving progressive sending.
+     */
+    midStreamFallback?: boolean;
     /**
      * STT model used to transcribe audio attachments (voice messages) when the
      * target LLM does not support audio natively. Audio-capable models receive

package/dist/{tool-CAo-BQHs.js → tool-Bv8N4IPj.js}

@@ -500,7 +500,10 @@ var Tool = class Tool {
 			if (yieldIndex >= yieldedCount) {
 				var _chat$handler;
 				setYieldedCount(yieldIndex + 1);
-				await (chat === null || chat === void 0 || (_chat$handler = chat.handler) === null || _chat$handler === void 0 ? void 0 : _chat$handler.call(chat, value));
+				await (chat === null || chat === void 0 || (_chat$handler = chat.handler) === null || _chat$handler === void 0 ? void 0 : _chat$handler.call(chat, value, {
+					iterationId: ctx.iterationId ?? ctx.callId,
+					id: `${ctx.callId}:yield-${yieldIndex}`
+				}));
 			}
 			yieldIndex++;
 		}

package/dist/{tool-DdPZb0JW.cjs → tool-ReA9gxMp.cjs}

@@ -500,7 +500,10 @@ var Tool = class Tool {
 			if (yieldIndex >= yieldedCount) {
 				var _chat$handler;
 				setYieldedCount(yieldIndex + 1);
-				await (chat === null || chat === void 0 || (_chat$handler = chat.handler) === null || _chat$handler === void 0 ? void 0 : _chat$handler.call(chat, value));
+				await (chat === null || chat === void 0 || (_chat$handler = chat.handler) === null || _chat$handler === void 0 ? void 0 : _chat$handler.call(chat, value, {
+					iterationId: ctx.iterationId ?? ctx.callId,
+					id: `${ctx.callId}:yield-${yieldIndex}`
+				}));
 			}
 			yieldIndex++;
 		}

package/dist/tool.d.ts

@@ -49,6 +49,8 @@ type SmartPartial<T> = IsObject<T> extends true ? Partial<T> : T;
 type ToolCallContext = {
     /** Unique identifier for this specific tool call */
     callId: string;
+    /** Set by the runtime; standalone calls use callId as their message scope. */
+    iterationId?: string;
 };
 export declare namespace Tool {
     type JSON = {

package/dist/types.d.ts

@@ -55,6 +55,13 @@ export declare namespace Traces {
         model: string;
         code: string;
     }>;
+    /** Diagnostic only: content and effects from the preceding attempt are discarded. */
+    type LLMCallRestart = TraceTemplate<'llm_call_restarted', {
+        attempt: number;
+        fromModel: string;
+        toModel: string;
+        reason: string;
+    }>;
     /**
      * Emitted on streaming clients the moment the model starts writing a `■run`
      * block — before the code is fully generated. Useful to show a
@@ -77,7 +84,7 @@ export declare namespace Traces {
         started_at: number;
         ended_at?: number;
     } & Content;
-    type Trace = Comment | Log | ToolCall | ToolSlow | PropertyMutation | YieldTrace | LLMCallStart | LLMCallSuccess | CodeGenerationStart | ThinkSignal | CodeExecution | CodeExecutionException | InvalidCodeExceptionTrace | AbortTrace;
+    type Trace = Comment | Log | ToolCall | ToolSlow | PropertyMutation | YieldTrace | LLMCallStart | LLMCallSuccess | LLMCallRestart | CodeGenerationStart | ThinkSignal | CodeExecution | CodeExecutionException | InvalidCodeExceptionTrace | AbortTrace;
 }
 export type Trace = Traces.Trace;
 export type VMExecutionResult = {

package/package.json

@@ -2,7 +2,7 @@
   "name": "llmz",
   "type": "module",
   "description": "LLMz - An LLM-native Typescript VM built on top of Zui",
-  "version": "0.6.4",
+  "version": "0.6.6",
   "files": [
     "dist",
     "DOCS.md"