llmz 0.6.7 → 0.7.0

package/README.md

@@ -79,6 +79,10 @@ return { ...payment, date: dates[0] }
 
 Because the protocol is parsed incrementally, messages stream to your UI token-by-token, tool calls surface live, and code execution can start before the response has even finished streaming.
 
+Only explicit `■send` blocks produce messages or message deltas. Unexpected text outside protocol blocks is discarded in streaming, fallback restarts, and nonstreaming execution. Explicit Markdown (`■send=md`) is supported. A response with only unexpected text follows the normal invalid-response retry/error path. For debugging, `iteration.llm.output` retains the raw response and `iteration.llm.diagnostics` records parser diagnostics, including `unexpected-text`.
+
+As of 0.7.0, the parser options `strict` and `recoveryComponent` have been removed. All parsing requires explicit send blocks; callers relying on implicit messages must emit `■send=<component>` instead.
+
 ---
 
 ## Quick Start
@@ -323,15 +327,22 @@ With a streaming client (`CognitiveBeta` / Cognitive v2), everything an executio
 const chat = new Chat({
   components: [DefaultComponents.Text],
   transcript: () => transcript,
-  handler: async (component) => finalizeBubble(component),
+  handler: async (component, metadata) => finalizeBubble(component, metadata),
   // Fires per token-chunk while the LLM is still generating
-  onMessageDelta: (delta) => appendToBubble(delta.id, delta.delta),
+  onMessageDelta: (delta) => {
+    if (delta.restart) {
+      // Remove ALL this iteration's messages, including completed handler sends.
+      return clearIterationMessages(delta.iterationId)
+    }
+    return updateBubble(delta.iterationId, delta.id, delta.content)
+  },
 })
 
 const result = await execute({
   client, // CognitiveBeta
   chat,
   tools,
+  options: { midStreamFallback: true },
   onTrace: ({ trace }) => {
     if (trace.type === 'llm_call_started') showSpinner()
     if (trace.type === 'code_generation_started') showStatus('writing code…')
@@ -343,7 +354,14 @@ const result = await execute({
 
 - **Message deltas** stream to your UI token-by-token (`handler` remains the authoritative delivery)
 - **Live traces** cover the full turn lifecycle: `llm_call_started` → message deltas → `code_generation_started` → `llm_call_success` (with the code) → `tool_call`s → exit
-- **Early execution**: the `■run` block starts executing while the tail of the response is still streaming, and the VM pre-warms the moment the model starts writing code
+- **Fallback**: a `restart: true` delta invalidates the current iteration's messages before replacement output arrives. Replacement messages have fresh IDs under the same `iterationId`. Code executes only after the replacement stream succeeds; abandoned code never runs.
+- **Early execution**: without `midStreamFallback`, a closed `■run` block can start executing while the response tail is still streaming. A later generation failure returns an error without retrying the code. The VM pre-warms when code generation begins in either mode.
+
+Terminal failures return an `ErrorExecutionResult`; they do not emit a restart delta because there is no replacement attempt. Handle that result to clear or mark unfinished messages as failed. Already delivered messages remain delivered unless your consumer retracts them. Nonstreaming Cognitive fallback exposes only the successful response, so no message reset is needed.
+
+LLMz rejects explicit Cognitive errors, error metadata (`provider: "unknown"`), and responses stopped by a token limit or content filter before finalizing unfinished blocks. Such failures cannot turn a partial code prefix into executable code. With fallback disabled, this cannot undo code that already ran after a block boundary.
+
+`■run` followed by `■send` is currently accepted, but that message was generated without seeing the code's result. Prefer sending an acknowledgement before `■run`, returning the needed data, then answering in the next iteration. A top-level `return` always requests another iteration; side-effect-only code can finish immediately with `■next`.
 
 Forward these events over a websocket or SSE stream and your frontend renders the agent live — see [example 22](https://github.com/botpress/botpress/tree/master/packages/llmz/examples/22_chat_streaming).
 

package/dist/context.d.ts

@@ -6,6 +6,7 @@ import { SnapshotSignal } from './errors.js';
 import { Exit } from './exit.js';
 import { ValueOrGetter } from './getter.js';
 import { HookedArray } from './handlers.js';
+import type { Diagnostic } from './message-stream/types.js';
 import { ObjectInstance } from './objects.js';
 import { LLMzPrompts, ParsedNext, ParsedSend, Prompt } from './prompts/prompt.js';
 import { Snapshot } from './snapshots.js';
@@ -365,6 +366,7 @@ export declare namespace Iteration {
             tokens: number;
             spend: number;
             output: string;
+            diagnostics?: Diagnostic[];
             model: string;
             time_to_first_token?: number;
             time_to_last_token?: number;
@@ -420,6 +422,8 @@ export declare class Iteration implements Serializable<Iteration.JSON> {
         tokens: number;
         spend: number;
         output: string;
+        /** Syntax diagnostics from the response parser; raw output remains available above. */
+        diagnostics?: Diagnostic[];
         model: string;
         /** Milliseconds between the LLM call start and the first streamed token. Only set on streaming clients. */
         time_to_first_token?: number;
@@ -475,6 +479,8 @@ export declare class Iteration implements Serializable<Iteration.JSON> {
             tokens: number;
             spend: number;
             output: string;
+            /** Syntax diagnostics from the response parser; raw output remains available above. */
+            diagnostics?: Diagnostic[];
             model: string;
             /** Milliseconds between the LLM call start and the first streamed token. Only set on streaming clients. */
             time_to_first_token?: number;
@@ -637,6 +643,8 @@ export declare class Context implements Serializable<Context.JSON> {
                 tokens: number;
                 spend: number;
                 output: string;
+                /** Syntax diagnostics from the response parser; raw output remains available above. */
+                diagnostics?: Diagnostic[];
                 model: string;
                 /** Milliseconds between the LLM call start and the first streamed token. Only set on streaming clients. */
                 time_to_first_token?: number;

package/dist/{custom-client-Cr33K60k.cjs → custom-client-Bp1TgTKk.cjs}

@@ -1,6 +1,6 @@
 const require__overArg = require("./_overArg-C5Hy7oCi.cjs");
 const require__baseIteratee = require("./_baseIteratee-CF4qP2a8.cjs");
-const require_dual_modes = require("./dual-modes-wAOAlaBS.cjs");
+const require_dual_modes = require("./dual-modes-CfqTk9M8.cjs");
 const require_utils = require("./utils-ZpJm00ed.cjs");
 const require__getPrototype = require("./_getPrototype-DQDlNVVM.cjs");
 const require_isPlainObject = require("./isPlainObject-CSxXVKCk.cjs");

package/dist/{custom-client-CJIThG2H.js → custom-client-CMuy7fXU.js}

@@ -1,6 +1,6 @@
 import { a as Symbol, n as isArray, o as root, r as isObjectLike } from "./_overArg-CRYbj1NH.js";
 import { D as isBuffer, E as baseUnary, M as baseAssignValue, R as isObject, T as nodeUtil, _ as baseGet, c as _getTag_default, f as getSymbols, h as Stack, k as isPrototype, s as Uint8Array, u as getAllKeys, v as toKey, x as keys, y as castPath, z as arrayMap } from "./_baseIteratee-P_firB7b.js";
-import { f as flatRest, l as extractType, m as baseCreate, p as copyArray, t as DualModePrompt, u as inspect } from "./dual-modes-D_p-jedw.js";
+import { f as flatRest, l as extractType, m as baseCreate, p as copyArray, t as DualModePrompt, u as inspect } from "./dual-modes-mNyMThtf.js";
 import { _ as getSymbolsIn, b as keysIn, g as getAllKeysIn, o as getTokenizer, x as assignValue, y as baseSlice } from "./utils-UBeAFeEI.js";
 import { t as getPrototype } from "./_getPrototype-CG7Dxrrz.js";
 import { t as isPlainObject } from "./isPlainObject-D39AzPO2.js";

package/dist/{dual-modes-wAOAlaBS.cjs → dual-modes-CfqTk9M8.cjs}

@@ -1615,6 +1615,8 @@ const tryParseJson = (text) => {
 * The parser is purely syntactic: it knows the reserved `■` symbol and the
 * directive grammar, but nothing about registered components, exits or their
 * schemas. Semantic validation is a separate step (see `validator.ts`).
+* Text outside protocol blocks is always discarded with a diagnostic; only
+* explicit `■send` blocks can produce messages.
 *
 * Pushing the same text split across arbitrary chunk boundaries always produces
 * the same items and the same concatenated body deltas. For a higher-level
@@ -1622,10 +1624,9 @@ const tryParseJson = (text) => {
 */
 var StreamingMessageParser = class {
 	_maxPropsLength;
-	_strict;
-	_recoveryComponent;
 	_state = "idle";
 	_items = [];
+	_diagnostics = [];
 	_current;
 	_currentReady = false;
 	_counter = 0;
@@ -1641,13 +1642,15 @@ var StreamingMessageParser = class {
 	_pendingWhitespace = "";
 	constructor(options = {}) {
 		this._maxPropsLength = options.maxPropsLength ?? 1e5;
-		this._strict = options.strict ?? false;
-		this._recoveryComponent = options.recoveryComponent ?? "md";
 	}
 	/** All items parsed so far, in order of appearance. */
 	get items() {
 		return [...this._items];
 	}
+	/** All syntax diagnostics, including discarded text that has no protocol item. */
+	get diagnostics() {
+		return [...this._diagnostics];
+	}
 	push(chunk) {
 		if (this._finished) throw new Error("Parser has already finished. Call reset() before parsing a new stream.");
 		const events = [];
@@ -1693,6 +1696,7 @@ var StreamingMessageParser = class {
 	reset() {
 		this._state = "idle";
 		this._items = [];
+		this._diagnostics = [];
 		this._current = void 0;
 		this._currentReady = false;
 		this._counter = 0;
@@ -1711,7 +1715,7 @@ var StreamingMessageParser = class {
 		switch (this._state) {
 			case "idle":
 				if (char === "■") this._beginItem();
-				else if (!isWhitespace(char)) this._beginRecovery(char, events);
+				else if (!isWhitespace(char)) this._skipUnexpectedText(events);
 				return;
 			case "directive":
 				if (char === "=") this._endDirective(events, "=");
@@ -1790,7 +1794,7 @@ var StreamingMessageParser = class {
 		if (isWhitespace(char)) return;
 		if (item.kind === "next") {
 			this._completeCurrent(events);
-			this._beginRecovery(char, events);
+			this._skipUnexpectedText(events);
 			return;
 		}
 		this._startBody(events);
@@ -2036,46 +2040,15 @@ var StreamingMessageParser = class {
 		this._current = void 0;
 		this._currentReady = false;
 	}
-	_beginRecovery(char, events) {
-		if (this._strict) {
-			this._diagnostic(events, {
-				code: "unexpected-text",
-				message: "Encountered text outside of a ■ block"
-			});
-			this._state = "skip";
-			return;
-		}
-		const item = {
-			id: `item-${this._counter++}`,
-			kind: "send",
-			name: this._recoveryComponent,
-			props: {},
-			body: "",
-			status: "pending",
-			diagnostics: []
-		};
-		this._items.push(item);
-		this._current = item;
-		this._currentReady = false;
-		this._pendingWhitespace = "";
+	_skipUnexpectedText(events) {
 		this._diagnostic(events, {
 			code: "unexpected-text",
-			message: `Encountered text outside of a ■ block; recovered it into an implicit ■send=${this._recoveryComponent}`,
-			itemId: item.id
-		});
-		events.push({
-			type: "item-start",
-			item
+			message: "Encountered text outside of a ■ block"
 		});
-		this._ready(events);
-		events.push({
-			type: "body-start",
-			itemId: item.id
-		});
-		this._state = "body";
-		this._appendBody(char);
+		this._state = "skip";
 	}
 	_diagnostic(events, diagnostic) {
+		this._diagnostics.push(diagnostic);
 		if (diagnostic.itemId) {
 			const item = this._items.find((i) => i.id === diagnostic.itemId);
 			item === null || item === void 0 || item.diagnostics.push(diagnostic);
@@ -2094,7 +2067,7 @@ const stripWrappingFences = (text) => text.trim().split("\n").filter((line, inde
 	return true;
 }).join("\n");
 /** Builds a {@link ParsedAssistantResponse} from parsed protocol items. */
-const toParsedAssistantResponse = (items, raw) => {
+const toParsedAssistantResponse = (items, raw, diagnostics = []) => {
 	var _run$body;
 	const sends = items.filter((item) => item.kind === "send").map((item) => ({
 		name: item.name,
@@ -2106,6 +2079,7 @@ const toParsedAssistantResponse = (items, raw) => {
 	return {
 		raw,
 		items,
+		diagnostics,
 		sends,
 		code: (run === null || run === void 0 || (_run$body = run.body) === null || _run$body === void 0 ? void 0 : _run$body.trim()) || void 0,
 		next: next ? {
@@ -2118,7 +2092,7 @@ const parseAssistantResponse = (response) => {
 	const parser = new StreamingMessageParser();
 	parser.push(stripWrappingFences(response));
 	parser.finish();
-	return toParsedAssistantResponse(parser.items, response);
+	return toParsedAssistantResponse(parser.items, response, parser.diagnostics);
 };
 const replacePlaceholders = (prompt, values) => {
 	const regex = /* @__PURE__ */ new RegExp("■■■([A-Z0-9_\\.-]+)■■■", "gi");

package/dist/{dual-modes-D_p-jedw.js → dual-modes-mNyMThtf.js}

@@ -1614,6 +1614,8 @@ const tryParseJson = (text) => {
 * The parser is purely syntactic: it knows the reserved `■` symbol and the
 * directive grammar, but nothing about registered components, exits or their
 * schemas. Semantic validation is a separate step (see `validator.ts`).
+* Text outside protocol blocks is always discarded with a diagnostic; only
+* explicit `■send` blocks can produce messages.
 *
 * Pushing the same text split across arbitrary chunk boundaries always produces
 * the same items and the same concatenated body deltas. For a higher-level
@@ -1621,10 +1623,9 @@ const tryParseJson = (text) => {
 */
 var StreamingMessageParser = class {
 	_maxPropsLength;
-	_strict;
-	_recoveryComponent;
 	_state = "idle";
 	_items = [];
+	_diagnostics = [];
 	_current;
 	_currentReady = false;
 	_counter = 0;
@@ -1640,13 +1641,15 @@ var StreamingMessageParser = class {
 	_pendingWhitespace = "";
 	constructor(options = {}) {
 		this._maxPropsLength = options.maxPropsLength ?? 1e5;
-		this._strict = options.strict ?? false;
-		this._recoveryComponent = options.recoveryComponent ?? "md";
 	}
 	/** All items parsed so far, in order of appearance. */
 	get items() {
 		return [...this._items];
 	}
+	/** All syntax diagnostics, including discarded text that has no protocol item. */
+	get diagnostics() {
+		return [...this._diagnostics];
+	}
 	push(chunk) {
 		if (this._finished) throw new Error("Parser has already finished. Call reset() before parsing a new stream.");
 		const events = [];
@@ -1692,6 +1695,7 @@ var StreamingMessageParser = class {
 	reset() {
 		this._state = "idle";
 		this._items = [];
+		this._diagnostics = [];
 		this._current = void 0;
 		this._currentReady = false;
 		this._counter = 0;
@@ -1710,7 +1714,7 @@ var StreamingMessageParser = class {
 		switch (this._state) {
 			case "idle":
 				if (char === "■") this._beginItem();
-				else if (!isWhitespace(char)) this._beginRecovery(char, events);
+				else if (!isWhitespace(char)) this._skipUnexpectedText(events);
 				return;
 			case "directive":
 				if (char === "=") this._endDirective(events, "=");
@@ -1789,7 +1793,7 @@ var StreamingMessageParser = class {
 		if (isWhitespace(char)) return;
 		if (item.kind === "next") {
 			this._completeCurrent(events);
-			this._beginRecovery(char, events);
+			this._skipUnexpectedText(events);
 			return;
 		}
 		this._startBody(events);
@@ -2035,46 +2039,15 @@ var StreamingMessageParser = class {
 		this._current = void 0;
 		this._currentReady = false;
 	}
-	_beginRecovery(char, events) {
-		if (this._strict) {
-			this._diagnostic(events, {
-				code: "unexpected-text",
-				message: "Encountered text outside of a ■ block"
-			});
-			this._state = "skip";
-			return;
-		}
-		const item = {
-			id: `item-${this._counter++}`,
-			kind: "send",
-			name: this._recoveryComponent,
-			props: {},
-			body: "",
-			status: "pending",
-			diagnostics: []
-		};
-		this._items.push(item);
-		this._current = item;
-		this._currentReady = false;
-		this._pendingWhitespace = "";
+	_skipUnexpectedText(events) {
 		this._diagnostic(events, {
 			code: "unexpected-text",
-			message: `Encountered text outside of a ■ block; recovered it into an implicit ■send=${this._recoveryComponent}`,
-			itemId: item.id
-		});
-		events.push({
-			type: "item-start",
-			item
+			message: "Encountered text outside of a ■ block"
 		});
-		this._ready(events);
-		events.push({
-			type: "body-start",
-			itemId: item.id
-		});
-		this._state = "body";
-		this._appendBody(char);
+		this._state = "skip";
 	}
 	_diagnostic(events, diagnostic) {
+		this._diagnostics.push(diagnostic);
 		if (diagnostic.itemId) {
 			const item = this._items.find((i) => i.id === diagnostic.itemId);
 			item === null || item === void 0 || item.diagnostics.push(diagnostic);
@@ -2093,7 +2066,7 @@ const stripWrappingFences = (text) => text.trim().split("\n").filter((line, inde
 	return true;
 }).join("\n");
 /** Builds a {@link ParsedAssistantResponse} from parsed protocol items. */
-const toParsedAssistantResponse = (items, raw) => {
+const toParsedAssistantResponse = (items, raw, diagnostics = []) => {
 	var _run$body;
 	const sends = items.filter((item) => item.kind === "send").map((item) => ({
 		name: item.name,
@@ -2105,6 +2078,7 @@ const toParsedAssistantResponse = (items, raw) => {
 	return {
 		raw,
 		items,
+		diagnostics,
 		sends,
 		code: (run === null || run === void 0 || (_run$body = run.body) === null || _run$body === void 0 ? void 0 : _run$body.trim()) || void 0,
 		next: next ? {
@@ -2117,7 +2091,7 @@ const parseAssistantResponse = (response) => {
 	const parser = new StreamingMessageParser();
 	parser.push(stripWrappingFences(response));
 	parser.finish();
-	return toParsedAssistantResponse(parser.items, response);
+	return toParsedAssistantResponse(parser.items, response, parser.diagnostics);
 };
 const replacePlaceholders = (prompt, values) => {
 	const regex = /* @__PURE__ */ new RegExp("■■■([A-Z0-9_\\.-]+)■■■", "gi");

package/dist/{execute-sn9RT_IN.js → execute-DP_17lYZ.js}

@@ -1,5 +1,5 @@
-import { a as StreamingMessageParser, d as isEqual, h as toNumber, i as toParsedAssistantResponse } from "./dual-modes-D_p-jedw.js";
-import { d as Snapshot, f as parseExit, l as PartialExecutionResult, n as Context, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-CJIThG2H.js";
+import { a as StreamingMessageParser, d as isEqual, h as toNumber, i as toParsedAssistantResponse } from "./dual-modes-mNyMThtf.js";
+import { d as Snapshot, f as parseExit, l as PartialExecutionResult, n as Context, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-CMuy7fXU.js";
 import { d as stripInvalidIdentifiers, i as getErrorMessage, s as init } from "./utils-UBeAFeEI.js";
 import { t as createJsxComponent } from "./jsx-CIgdGGVs.js";
 import { t as cleanStackTrace } from "./stack-traces-BIE_5Xwf.js";
@@ -120,11 +120,16 @@ const RESPONSE_LENGTH_BUFFER = {
 };
 /** Maximum time to wait between two stream chunks before considering the stream stalled. */
 const STREAM_INACTIVITY_TIMEOUT = 18e4;
+/** A syntactically valid prefix must not execute when generation did not finish successfully. */
+const assertSuccessfulGeneration = (metadata) => {
+	if (metadata.provider === "unknown") throw new CognitiveError("LLM generation failed: received error metadata with unknown provider");
+	if (metadata.stopReason === "max_tokens" || metadata.stopReason === "content_filter") throw new CognitiveError(`LLM generation did not complete: stopReason=${metadata.stopReason}`);
+};
 const getModelOutputLimit = (inputLength) => clamp(RESPONSE_LENGTH_BUFFER.PERCENTAGE * inputLength, RESPONSE_LENGTH_BUFFER.MIN_TOKENS, RESPONSE_LENGTH_BUFFER.MAX_TOKENS);
 /**
 * Models sometimes wrap their whole response in a code fence. The fence has to
-* be removed before it reaches the incremental parser, otherwise it would be
-* recovered as an unexpected-text message. Works on arbitrary chunk boundaries
+* be removed before it reaches the incremental parser to avoid a spurious
+* unexpected-text diagnostic. Works on arbitrary chunk boundaries
 * by holding content back until the first newline.
 */
 var LeadingFenceFilter = class {
@@ -330,10 +335,11 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 				await dispatchSends(parser.push(fence.push(delta)));
 			}
 			if (!responseMetadata) throw new CognitiveError("LLM streaming completed without metadata");
+			assertSuccessfulGeneration(responseMetadata);
 			const remaining = fence.flush();
 			if (remaining) await dispatchSends(parser.push(remaining));
 			await dispatchSends(parser.finish());
-			assistantResponse = toParsedAssistantResponse(parser.items, raw);
+			assistantResponse = toParsedAssistantResponse(parser.items, raw, parser.diagnostics);
 		} finally {
 			streamController.abort("LLM stream closed");
 			if (!streamCompleted) stream.return(void 0).catch((err) => {});
@@ -343,8 +349,10 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		const response = await cognitive.generateText(input, { signal: controller.signal }).catch((thrown) => {
 			throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
 		});
+		if (response.error) throw new CognitiveError(`LLM generation failed: ${response.error}`);
 		if (!response.output) throw new CognitiveError("LLM did not return any text output");
 		responseMetadata = response.metadata;
+		assertSuccessfulGeneration(responseMetadata);
 		raw = response.output;
 		assistantResponse = ctx.version.parseAssistantResponse(raw);
 		for (const [index, send] of assistantResponse.sends.entries()) await (onSend === null || onSend === void 0 ? void 0 : onSend(send, messageMetadata(`send-${index}`)));
@@ -361,6 +369,7 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		tokens: usage.inputTokens + usage.outputTokens,
 		spend: responseMetadata.cost ?? usage.inputCost + usage.outputCost,
 		output: assistantResponse.raw,
+		diagnostics: assistantResponse.diagnostics,
 		model: `${responseMetadata.provider}:${responseMetadata.model}`,
 		time_to_first_token: timeToFirstToken,
 		time_to_last_token: timeToLastToken,

package/dist/{execute-BmEjJjka.cjs → execute-msui51b2.cjs}

@@ -1,6 +1,6 @@
 const require__baseForOwn = require("./_baseForOwn-buOZDFbK.cjs");
-const require_dual_modes = require("./dual-modes-wAOAlaBS.cjs");
-const require_custom_client = require("./custom-client-Cr33K60k.cjs");
+const require_dual_modes = require("./dual-modes-CfqTk9M8.cjs");
+const require_custom_client = require("./custom-client-Bp1TgTKk.cjs");
 const require_utils = require("./utils-ZpJm00ed.cjs");
 const require_jsx = require("./jsx-BWvT6ukg.cjs");
 const require_stack_traces = require("./stack-traces-ntw_qhoy.cjs");
@@ -122,11 +122,16 @@ const RESPONSE_LENGTH_BUFFER = {
 };
 /** Maximum time to wait between two stream chunks before considering the stream stalled. */
 const STREAM_INACTIVITY_TIMEOUT = 18e4;
+/** A syntactically valid prefix must not execute when generation did not finish successfully. */
+const assertSuccessfulGeneration = (metadata) => {
+	if (metadata.provider === "unknown") throw new require_errors.CognitiveError("LLM generation failed: received error metadata with unknown provider");
+	if (metadata.stopReason === "max_tokens" || metadata.stopReason === "content_filter") throw new require_errors.CognitiveError(`LLM generation did not complete: stopReason=${metadata.stopReason}`);
+};
 const getModelOutputLimit = (inputLength) => clamp(RESPONSE_LENGTH_BUFFER.PERCENTAGE * inputLength, RESPONSE_LENGTH_BUFFER.MIN_TOKENS, RESPONSE_LENGTH_BUFFER.MAX_TOKENS);
 /**
 * Models sometimes wrap their whole response in a code fence. The fence has to
-* be removed before it reaches the incremental parser, otherwise it would be
-* recovered as an unexpected-text message. Works on arbitrary chunk boundaries
+* be removed before it reaches the incremental parser to avoid a spurious
+* unexpected-text diagnostic. Works on arbitrary chunk boundaries
 * by holding content back until the first newline.
 */
 var LeadingFenceFilter = class {
@@ -332,10 +337,11 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 				await dispatchSends(parser.push(fence.push(delta)));
 			}
 			if (!responseMetadata) throw new require_errors.CognitiveError("LLM streaming completed without metadata");
+			assertSuccessfulGeneration(responseMetadata);
 			const remaining = fence.flush();
 			if (remaining) await dispatchSends(parser.push(remaining));
 			await dispatchSends(parser.finish());
-			assistantResponse = require_dual_modes.toParsedAssistantResponse(parser.items, raw);
+			assistantResponse = require_dual_modes.toParsedAssistantResponse(parser.items, raw, parser.diagnostics);
 		} finally {
 			streamController.abort("LLM stream closed");
 			if (!streamCompleted) stream.return(void 0).catch((err) => {});
@@ -345,8 +351,10 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		const response = await cognitive.generateText(input, { signal: controller.signal }).catch((thrown) => {
 			throw new require_errors.CognitiveError(`LLM generation failed: ${require_utils.getErrorMessage(thrown)}`);
 		});
+		if (response.error) throw new require_errors.CognitiveError(`LLM generation failed: ${response.error}`);
 		if (!response.output) throw new require_errors.CognitiveError("LLM did not return any text output");
 		responseMetadata = response.metadata;
+		assertSuccessfulGeneration(responseMetadata);
 		raw = response.output;
 		assistantResponse = ctx.version.parseAssistantResponse(raw);
 		for (const [index, send] of assistantResponse.sends.entries()) await (onSend === null || onSend === void 0 ? void 0 : onSend(send, messageMetadata(`send-${index}`)));
@@ -363,6 +371,7 @@ const generateCode = async ({ iteration, ctx, cognitive, controller, metadata, o
 		tokens: usage.inputTokens + usage.outputTokens,
 		spend: responseMetadata.cost ?? usage.inputCost + usage.outputCost,
 		output: assistantResponse.raw,
+		diagnostics: assistantResponse.diagnostics,
 		model: `${responseMetadata.provider}:${responseMetadata.model}`,
 		time_to_first_token: timeToFirstToken,
 		time_to_last_token: timeToLastToken,

package/dist/index.cjs

@@ -1,6 +1,6 @@
 Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
-const require_dual_modes = require("./dual-modes-wAOAlaBS.cjs");
-const require_custom_client = require("./custom-client-Cr33K60k.cjs");
+const require_dual_modes = require("./dual-modes-CfqTk9M8.cjs");
+const require_custom_client = require("./custom-client-Bp1TgTKk.cjs");
 const require_utils = require("./utils-ZpJm00ed.cjs");
 const require_tool = require("./tool-ReA9gxMp.cjs");
 const require_errors = require("./errors-C962uJdS.cjs");
@@ -11,7 +11,7 @@ const require_truncator = require("./truncator-B6aaJaN1.cjs");
 const require_quickjs_variant = require("./quickjs-variant-CsIwTVFC.cjs");
 let _bpinternal_zui = require("@bpinternal/zui");
 //#region package.json
-var version = "0.6.7";
+var version = "0.7.0";
 //#endregion
 //#region src/hoist.ts
 /**
@@ -1774,7 +1774,7 @@ const utils = {
 * })
 */
 const execute = async (props) => {
-	const { executeContext } = await Promise.resolve().then(() => require("./execute-BmEjJjka.cjs"));
+	const { executeContext } = await Promise.resolve().then(() => require("./execute-msui51b2.cjs"));
 	return executeContext(props);
 };
 /**
@@ -1783,7 +1783,7 @@ const execute = async (props) => {
 * It's recommended to call this function at the beginning of your application without awaiting it (void init())
 */
 const init = async () => {
-	await Promise.resolve().then(() => require("./execute-BmEjJjka.cjs"));
+	await Promise.resolve().then(() => require("./execute-msui51b2.cjs"));
 	await Promise.resolve().then(() => require("./component-BwixNVH8.cjs")).then((n) => n.component_exports);
 	await Promise.resolve().then(() => require("./tool-ReA9gxMp.cjs")).then((n) => n.tool_exports);
 	await Promise.resolve().then(() => require("./exit-BGi0Kk0y.cjs")).then((n) => n.exit_exports);
@@ -1792,7 +1792,7 @@ const init = async () => {
 	await Promise.resolve().then(() => require("./utils-ZpJm00ed.cjs")).then((n) => n.utils_exports);
 	await Promise.resolve().then(() => require("./truncator-B6aaJaN1.cjs")).then((n) => n.truncator_exports);
 	await Promise.resolve().then(() => require("./typings-BwAz_TM9.cjs")).then((n) => n.typings_exports);
-	await Promise.resolve().then(() => require("./dual-modes-wAOAlaBS.cjs")).then((n) => n.dual_modes_exports);
+	await Promise.resolve().then(() => require("./dual-modes-CfqTk9M8.cjs")).then((n) => n.dual_modes_exports);
 };
 //#endregion
 exports.BaseResponse = BaseResponse;

package/dist/index.js

@@ -1,5 +1,5 @@
-import { a as StreamingMessageParser, c as NAME_REGEX, o as tryParseJson, r as generateInstructions, s as MARKER } from "./dual-modes-D_p-jedw.js";
-import { a as ThinkExit, c as ExecutionResult, d as Snapshot, f as parseExit, i as ListenExit, l as PartialExecutionResult, o as getValue, r as DefaultExit, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-CJIThG2H.js";
+import { a as StreamingMessageParser, c as NAME_REGEX, o as tryParseJson, r as generateInstructions, s as MARKER } from "./dual-modes-mNyMThtf.js";
+import { a as ThinkExit, c as ExecutionResult, d as Snapshot, f as parseExit, i as ListenExit, l as PartialExecutionResult, o as getValue, r as DefaultExit, s as ErrorExecutionResult, t as _CustomModelClient, u as SuccessExecutionResult } from "./custom-client-CMuy7fXU.js";
 import { a as getMultilineComment, c as isJsonSchema, l as isValidIdentifier, m as toValidObjectName, n as escapeString, p as toValidFunctionName, t as configureTokenizer, u as isZuiSchema, v as upperFirst } from "./utils-UBeAFeEI.js";
 import { t as Tool } from "./tool-Bv8N4IPj.js";
 import { c as SnapshotSignal, l as ThinkSignal, o as LoopExceededError, r as CodeFormattingError } from "./errors-Cg1G0Awu.js";
@@ -10,7 +10,7 @@ import { i as wrapContent, n as truncateWrappedContent, t as stripTruncationTags
 import { t as configureQuickJS } from "./quickjs-variant-BYhMBpwU.js";
 import { transforms, z } from "@bpinternal/zui";
 //#region package.json
-var version = "0.6.7";
+var version = "0.7.0";
 //#endregion
 //#region src/hoist.ts
 /**
@@ -1773,7 +1773,7 @@ const utils = {
 * })
 */
 const execute = async (props) => {
-	const { executeContext } = await import("./execute-sn9RT_IN.js");
+	const { executeContext } = await import("./execute-DP_17lYZ.js");
 	return executeContext(props);
 };
 /**
@@ -1782,7 +1782,7 @@ const execute = async (props) => {
 * It's recommended to call this function at the beginning of your application without awaiting it (void init())
 */
 const init = async () => {
-	await import("./execute-sn9RT_IN.js");
+	await import("./execute-DP_17lYZ.js");
 	await import("./component-DGG7d_Mz.js").then((n) => n.r);
 	await import("./tool-Bv8N4IPj.js").then((n) => n.n);
 	await import("./exit-CDdxDBRT.js").then((n) => n.n);
@@ -1791,7 +1791,7 @@ const init = async () => {
 	await import("./utils-UBeAFeEI.js").then((n) => n.h);
 	await import("./truncator-DZqEqnpD.js").then((n) => n.r);
 	await import("./typings-CrBa7Tyy.js").then((n) => n.n);
-	await import("./dual-modes-D_p-jedw.js").then((n) => n.n);
+	await import("./dual-modes-mNyMThtf.js").then((n) => n.n);
 };
 //#endregion
 export { BaseResponse, Chat, CitationsManager, Component, ComponentRegistry, DefaultComponents, DefaultExit, ErrorExecutionResult, ExecutionResult, Exit, ListenExit, LoopExceededError, MARKER, NAME_REGEX, NextResponse, ObjectInstance, PartialExecutionResult, ReplayableAsyncIterable, RunResponse, SendResponse, Snapshot, SnapshotSignal, StreamingMessageParser, SuccessExecutionResult, ThinkExit, ThinkSignal, Tool, _CustomModelClient, assertValidComponent, collectText, configureQuickJS, configureTokenizer, execute, generateInstructions, getValue, init, isAnyComponent, isComponent, normalizeComponentDefinition, normalizeComponentName, normalizePropsSchema, parseExit, parseStream, parseText, sanitizeMessageText, tryParseJson, utils, validateComponent, validateProps, version };

package/dist/index.workerd.js

@@ -52,7 +52,7 @@ var __toESM = (mod, isNodeMode, target) => (target = mod != null ? __create(__ge
 }) : target, mod));
 //#endregion
 //#region package.json
-var version = "0.6.7";
+var version = "0.7.0";
 //#endregion
 //#region ../../node_modules/.pnpm/lodash-es@4.17.21/node_modules/lodash-es/_freeGlobal.js
 var freeGlobal;
@@ -10830,10 +10830,9 @@ var init_parser = __esmMin((() => {
 	};
 	StreamingMessageParser = class {
 		_maxPropsLength;
-		_strict;
-		_recoveryComponent;
 		_state = "idle";
 		_items = [];
+		_diagnostics = [];
 		_current;
 		_currentReady = false;
 		_counter = 0;
@@ -10849,13 +10848,15 @@ var init_parser = __esmMin((() => {
 		_pendingWhitespace = "";
 		constructor(options = {}) {
 			this._maxPropsLength = options.maxPropsLength ?? 1e5;
-			this._strict = options.strict ?? false;
-			this._recoveryComponent = options.recoveryComponent ?? "md";
 		}
 		/** All items parsed so far, in order of appearance. */
 		get items() {
 			return [...this._items];
 		}
+		/** All syntax diagnostics, including discarded text that has no protocol item. */
+		get diagnostics() {
+			return [...this._diagnostics];
+		}
 		push(chunk) {
 			if (this._finished) throw new Error("Parser has already finished. Call reset() before parsing a new stream.");
 			const events = [];
@@ -10901,6 +10902,7 @@ var init_parser = __esmMin((() => {
 		reset() {
 			this._state = "idle";
 			this._items = [];
+			this._diagnostics = [];
 			this._current = void 0;
 			this._currentReady = false;
 			this._counter = 0;
@@ -10919,7 +10921,7 @@ var init_parser = __esmMin((() => {
 			switch (this._state) {
 				case "idle":
 					if (char === "■") this._beginItem();
-					else if (!isWhitespace(char)) this._beginRecovery(char, events);
+					else if (!isWhitespace(char)) this._skipUnexpectedText(events);
 					return;
 				case "directive":
 					if (char === "=") this._endDirective(events, "=");
@@ -10998,7 +11000,7 @@ var init_parser = __esmMin((() => {
 			if (isWhitespace(char)) return;
 			if (item.kind === "next") {
 				this._completeCurrent(events);
-				this._beginRecovery(char, events);
+				this._skipUnexpectedText(events);
 				return;
 			}
 			this._startBody(events);
@@ -11244,46 +11246,15 @@ var init_parser = __esmMin((() => {
 			this._current = void 0;
 			this._currentReady = false;
 		}
-		_beginRecovery(char, events) {
-			if (this._strict) {
-				this._diagnostic(events, {
-					code: "unexpected-text",
-					message: "Encountered text outside of a ■ block"
-				});
-				this._state = "skip";
-				return;
-			}
-			const item = {
-				id: `item-${this._counter++}`,
-				kind: "send",
-				name: this._recoveryComponent,
-				props: {},
-				body: "",
-				status: "pending",
-				diagnostics: []
-			};
-			this._items.push(item);
-			this._current = item;
-			this._currentReady = false;
-			this._pendingWhitespace = "";
+		_skipUnexpectedText(events) {
 			this._diagnostic(events, {
 				code: "unexpected-text",
-				message: `Encountered text outside of a ■ block; recovered it into an implicit ■send=${this._recoveryComponent}`,
-				itemId: item.id
-			});
-			events.push({
-				type: "item-start",
-				item
-			});
-			this._ready(events);
-			events.push({
-				type: "body-start",
-				itemId: item.id
+				message: "Encountered text outside of a ■ block"
 			});
-			this._state = "body";
-			this._appendBody(char);
+			this._state = "skip";
 		}
 		_diagnostic(events, diagnostic) {
+			this._diagnostics.push(diagnostic);
 			if (diagnostic.itemId) this._items.find((i) => i.id === diagnostic.itemId)?.diagnostics.push(diagnostic);
 			events.push({
 				type: "diagnostic",
@@ -11301,7 +11272,7 @@ var init_common = __esmMin((() => {
 		if ((index === 0 || index === arr.length - 1) && line.trim().startsWith("```")) return false;
 		return true;
 	}).join("\n");
-	toParsedAssistantResponse = (items, raw) => {
+	toParsedAssistantResponse = (items, raw, diagnostics = []) => {
 		const sends = items.filter((item) => item.kind === "send").map((item) => ({
 			name: item.name,
 			props: item.props,
@@ -11312,6 +11283,7 @@ var init_common = __esmMin((() => {
 		return {
 			raw,
 			items,
+			diagnostics,
 			sends,
 			code: run?.body?.trim() || void 0,
 			next: next ? {
@@ -11324,7 +11296,7 @@ var init_common = __esmMin((() => {
 		const parser = new StreamingMessageParser();
 		parser.push(stripWrappingFences(response));
 		parser.finish();
-		return toParsedAssistantResponse(parser.items, response);
+		return toParsedAssistantResponse(parser.items, response, parser.diagnostics);
 	};
 	replacePlaceholders = (prompt, values) => {
 		const regex = /* @__PURE__ */ new RegExp("■■■([A-Z0-9_\\.-]+)■■■", "gi");
@@ -17025,7 +16997,7 @@ var init_vm = __esmMin((() => {
 }));
 //#endregion
 //#region src/runtime/generate.ts
-var RESPONSE_LENGTH_BUFFER, STREAM_INACTIVITY_TIMEOUT, getModelOutputLimit, LeadingFenceFilter, generateCode;
+var RESPONSE_LENGTH_BUFFER, STREAM_INACTIVITY_TIMEOUT, assertSuccessfulGeneration, getModelOutputLimit, LeadingFenceFilter, generateCode;
 var init_generate = __esmMin((() => {
 	init_lodash();
 	init_abort_signal();
@@ -17040,6 +17012,10 @@ var init_generate = __esmMin((() => {
 		PERCENTAGE: .1
 	};
 	STREAM_INACTIVITY_TIMEOUT = 18e4;
+	assertSuccessfulGeneration = (metadata) => {
+		if (metadata.provider === "unknown") throw new CognitiveError("LLM generation failed: received error metadata with unknown provider");
+		if (metadata.stopReason === "max_tokens" || metadata.stopReason === "content_filter") throw new CognitiveError(`LLM generation did not complete: stopReason=${metadata.stopReason}`);
+	};
 	getModelOutputLimit = (inputLength) => clamp(RESPONSE_LENGTH_BUFFER.PERCENTAGE * inputLength, RESPONSE_LENGTH_BUFFER.MIN_TOKENS, RESPONSE_LENGTH_BUFFER.MAX_TOKENS);
 	LeadingFenceFilter = class {
 		_buffer = "";
@@ -17243,10 +17219,11 @@ var init_generate = __esmMin((() => {
 					await dispatchSends(parser.push(fence.push(delta)));
 				}
 				if (!responseMetadata) throw new CognitiveError("LLM streaming completed without metadata");
+				assertSuccessfulGeneration(responseMetadata);
 				const remaining = fence.flush();
 				if (remaining) await dispatchSends(parser.push(remaining));
 				await dispatchSends(parser.finish());
-				assistantResponse = toParsedAssistantResponse(parser.items, raw);
+				assistantResponse = toParsedAssistantResponse(parser.items, raw, parser.diagnostics);
 			} finally {
 				streamController.abort("LLM stream closed");
 				if (!streamCompleted) stream.return(void 0).catch((err) => {});
@@ -17256,8 +17233,10 @@ var init_generate = __esmMin((() => {
 			const response = await cognitive.generateText(input, { signal: controller.signal }).catch((thrown) => {
 				throw new CognitiveError(`LLM generation failed: ${getErrorMessage(thrown)}`);
 			});
+			if (response.error) throw new CognitiveError(`LLM generation failed: ${response.error}`);
 			if (!response.output) throw new CognitiveError("LLM did not return any text output");
 			responseMetadata = response.metadata;
+			assertSuccessfulGeneration(responseMetadata);
 			raw = response.output;
 			assistantResponse = ctx.version.parseAssistantResponse(raw);
 			for (const [index, send] of assistantResponse.sends.entries()) await onSend?.(send, messageMetadata(`send-${index}`));
@@ -17274,6 +17253,7 @@ var init_generate = __esmMin((() => {
 			tokens: usage.inputTokens + usage.outputTokens,
 			spend: responseMetadata.cost ?? usage.inputCost + usage.outputCost,
 			output: assistantResponse.raw,
+			diagnostics: assistantResponse.diagnostics,
 			model: `${responseMetadata.provider}:${responseMetadata.model}`,
 			time_to_first_token: timeToFirstToken,
 			time_to_last_token: timeToLastToken,

package/dist/message-stream/parser.d.ts

@@ -1,11 +1,7 @@
-import { type MessageStreamEvent, type ParsedItem } from './types.js';
+import { type Diagnostic, type MessageStreamEvent, type ParsedItem } from './types.js';
 export type StreamingParserOptions = {
     /** Maximum number of characters buffered for a props object before the item is marked invalid. Default: 100 000. */
     maxPropsLength?: number;
-    /** When true, unexpected free text is dropped (with a diagnostic) instead of being recovered into an implicit `send`. Default: false. */
-    strict?: boolean;
-    /** Component used to recover unexpected free text into an implicit `■send`. Default: 'md'. */
-    recoveryComponent?: string;
 };
 /**
  * Lenient JSON parsing: JSON5 first (unquoted keys, single quotes, trailing
@@ -18,6 +14,8 @@ export declare const tryParseJson: (text: string) => unknown;
  * The parser is purely syntactic: it knows the reserved `■` symbol and the
  * directive grammar, but nothing about registered components, exits or their
  * schemas. Semantic validation is a separate step (see `validator.ts`).
+ * Text outside protocol blocks is always discarded with a diagnostic; only
+ * explicit `■send` blocks can produce messages.
  *
  * Pushing the same text split across arbitrary chunk boundaries always produces
  * the same items and the same concatenated body deltas. For a higher-level
@@ -25,10 +23,9 @@ export declare const tryParseJson: (text: string) => unknown;
  */
 export declare class StreamingMessageParser {
     private _maxPropsLength;
-    private _strict;
-    private _recoveryComponent;
     private _state;
     private _items;
+    private _diagnostics;
     private _current;
     private _currentReady;
     private _counter;
@@ -45,6 +42,8 @@ export declare class StreamingMessageParser {
     constructor(options?: StreamingParserOptions);
     /** All items parsed so far, in order of appearance. */
     get items(): ParsedItem[];
+    /** All syntax diagnostics, including discarded text that has no protocol item. */
+    get diagnostics(): Diagnostic[];
     push(chunk: string): MessageStreamEvent[];
     /**
      * Signals the end of the transport stream.
@@ -76,6 +75,6 @@ export declare class StreamingMessageParser {
     private _flushBodyDelta;
     private _ready;
     private _completeCurrent;
-    private _beginRecovery;
+    private _skipUnexpectedText;
     private _diagnostic;
 }

package/dist/prompts/common.d.ts

@@ -1,6 +1,6 @@
-import type { ParsedItem } from '../message-stream/types.js';
+import type { Diagnostic, ParsedItem } from '../message-stream/types.js';
 import { ParsedAssistantResponse } from './prompt.js';
 /** Builds a {@link ParsedAssistantResponse} from parsed protocol items. */
-export declare const toParsedAssistantResponse: (items: ParsedItem[], raw: string) => ParsedAssistantResponse;
+export declare const toParsedAssistantResponse: (items: ParsedItem[], raw: string, diagnostics?: Diagnostic[]) => ParsedAssistantResponse;
 export declare const parseAssistantResponse: (response: string) => ParsedAssistantResponse;
 export declare const replacePlaceholders: (prompt: string, values: Record<string, unknown>) => string;

package/dist/prompts/prompt.d.ts

@@ -1,7 +1,7 @@
 import { type CognitiveMessage } from '@botpress/cognitive';
 import { Component } from '../component.js';
 import { Exit } from '../exit.js';
-import type { ParsedItem } from '../message-stream/types.js';
+import type { Diagnostic, ParsedItem } from '../message-stream/types.js';
 import type { ObjectInstance } from '../objects.js';
 import { Snapshot } from '../snapshots.js';
 import { type Tool } from '../tool.js';
@@ -21,6 +21,8 @@ export type ParsedAssistantResponse = {
     raw: string;
     /** All protocol items, in order of appearance. */
     items: ParsedItem[];
+    /** Syntax diagnostics, including unexpected text discarded outside protocol blocks. */
+    diagnostics?: Diagnostic[];
     /** Messages to send to the user, in order. */
     sends: ParsedSend[];
     /** The body of the `■run` block, if any. */

package/dist/result.d.ts

@@ -427,6 +427,7 @@ export declare class SuccessExecutionResult<TOutput = unknown> extends Execution
                     tokens: number;
                     spend: number;
                     output: string;
+                    diagnostics?: import("./index.js").Diagnostic[];
                     model: string;
                     time_to_first_token?: number;
                     time_to_last_token?: number;
@@ -592,6 +593,7 @@ export declare class ErrorExecutionResult extends ExecutionResult implements Ser
                     tokens: number;
                     spend: number;
                     output: string;
+                    diagnostics?: import("./index.js").Diagnostic[];
                     model: string;
                     time_to_first_token?: number;
                     time_to_last_token?: number;
@@ -777,6 +779,7 @@ export declare class PartialExecutionResult extends ExecutionResult implements S
                     tokens: number;
                     spend: number;
                     output: string;
+                    diagnostics?: import("./index.js").Diagnostic[];
                     model: string;
                     time_to_first_token?: number;
                     time_to_last_token?: number;

package/package.json

@@ -2,7 +2,7 @@
   "name": "llmz",
   "type": "module",
   "description": "LLMz - An LLM-native Typescript VM built on top of Zui",
-  "version": "0.6.7",
+  "version": "0.7.0",
   "files": [
     "dist",
     "DOCS.md"
@@ -69,7 +69,7 @@
   },
   "peerDependencies": {
     "@botpress/client": "2.5.0",
-    "@botpress/cognitive": "1.2.0",
+    "@botpress/cognitive": "1.2.1",
     "@bpinternal/thicktoken": "^3.1.0",
     "@bpinternal/zui": "^2.4.1"
   },