libfx 0.0.10 → 0.0.11

package/README.md

@@ -1,8 +1,8 @@
 # libfx
 
 `libfx` is the small fx agent kernel for JavaScript hosts. One agent is one
-in-memory conversation with three operations: `prompt`, `checkpoint`, and
-`close`.
+in-memory conversation with `prompt`, `checkpoint`, and `close` operations,
+plus mid-turn steering on each running turn.
 
 ```sh
 npm install libfx
@@ -41,28 +41,52 @@ await agent.close();
 Agent configuration uses named options; `env` is reserved for
 `createFxTerminal()`.
 
+`effort` sets the reasoning effort for models that advertise effort levels. It
+uses the same vocabulary as the fx CLI's `--effort` flag: `"default"` leaves
+the choice to the model, and named levels such as `"low"`, `"medium"`,
+`"high"`, or `"xhigh"` request a specific level. A named level is validated
+against the selected model's advertised levels at creation; an unsupported
+level rejects with an error (code `LIBFX_UNSUPPORTED_EFFORT`) naming the
+supported set. When `effort` is omitted or `"default"`, the model default
+applies.
+
+`fast` enables the fast lane for models that advertise one, matching the fx
+CLI's `--fast` flag. Enabling it is validated against the selected model at
+creation; a model without a fast path rejects with an error (code
+`LIBFX_UNSUPPORTED_FAST`). When `fast` is omitted or `false`, the model
+default applies.
+
 The host selects the model. Agent creation does not fetch the Gateway model
-catalog. Prompting can resolve model capabilities and context capacity through
-the supplied `fetch`; fx caches that metadata for the agent.
+catalog unless `effort` requests a named level or `fast` is enabled. Prompting
+can resolve model capabilities and context capacity through the supplied
+`fetch`; fx caches that metadata for the agent.
 
 `onEvent` receives runtime diagnostics separately from model output. Transport
 events report request start, response status and elapsed time, safe Gateway
-request metadata, and failures. Credentials and raw headers are never included.
+request metadata, failures, and throttled `transport.activity` liveness while a
+response body is streaming. Activity events include the attempt, current chunk
+bytes, and cumulative response bytes. They call the host directly at most once
+per 250 ms rather than entering the normalized turn queue, so an unread turn
+cannot accumulate heartbeat events. Credentials and raw headers are never
+included.
 
 libfx makes at most one automatic retry after a retryable transport failure and
 only before model output or tool effects escape. Cancellation prevents a retry.
 
-`prompt(input, { signal? })` accepts a string or text/resource blocks. It
-returns an async iterable of normalized events:
+`prompt(input, { signal? })` accepts a string or text, image, and resource
+blocks. It returns an async iterable of normalized events:
 
 - `text_delta`
 - `reasoning_delta` when supplied by the provider
-- `tool_start`
+- `tool_start` with `id`, `name`, and the tool's `input` object. Inputs over
+  64 KiB of JSON arrive as an `inputPreview` string prefix plus
+  `inputTruncated: true` instead; the tool still receives complete arguments.
 - `tool_end`
+- `user_message` with `text` when accepted mid-turn steering enters the turn
 
-Consume the turn while it runs, then await `turn.result`. Output is lossless and
-backpressured: a slow reader pauses production instead of growing an unlimited
-event queue. Awaiting only `turn.result` can wait for an unread stream to drain.
+Consume the turn while it runs, then await `turn.result`. Streamed text and
+tool results are lossless and backpressured: a slow reader pauses production
+instead of growing an unlimited event queue. Awaiting only `turn.result` can wait for an unread stream to drain.
 If you only need the result, explicitly discard events:
 
 ```js
@@ -82,9 +106,50 @@ that threshold when the queue is empty; an individual encoded ACP message is
 limited to 64 MiB on both backends. These are transport bounds, not a total
 answer-size limit or a bound on retained conversation history.
 
-Only one prompt may run at a time. `checkpoint()` is idle-only and returns
-opaque, bounded, versioned bytes. Restore them only when creating a fresh
-agent:
+Image blocks use ACP's content shape and carry canonical base64 (no line
+wrapping) of a PNG, JPEG, GIF, or WebP payload:
+
+```js
+const turn = agent.prompt([
+  { type: "text", text: "What does this screenshot show?" },
+  { type: "image", data: base64Png, mimeType: "image/png" },
+]);
+```
+
+A prompt may contain up to 8 images, each with up to 5 MiB of base64 data,
+with at most 8 MiB of image data per prompt; the SDK rejects larger input with
+typed `RangeError`s before any request. The kernel then validates the decoded
+bytes against the declared `mimeType` and fails the turn with
+`Invalid image prompt block` on a mismatch. Images are routed only to models
+that advertise image input; for any other model the turn fails with
+`Image prompts are unavailable for the selected model` and no image bytes
+leave the process. Prompt images are retained in checkpoints within the
+existing 4 MiB checkpoint bound, so a restored agent can refer to earlier
+images on either backend.
+
+Only one top-level prompt may run at a time. While it runs,
+`await turn.steer(text)` appends guidance at the next safe model boundary
+without discarding the in-flight response or completed tool work. Steering also
+accepts an array of text blocks; image and resource steering blocks are rejected.
+Each message is limited to 64 KiB, with at most 64 queued messages and 1 MiB of
+queued steering text. Accepted steering appears as a `user_message` event before
+the model's continued output. Calling `steer()` after the turn settles rejects
+with `no prompt is running`.
+
+```js
+const turn = agent.prompt("Build the feature.");
+for await (const event of turn) {
+  if (event.type === "tool_end" && event.name === "read_file") {
+    await turn.steer("Keep the public API backward compatible.");
+  }
+}
+```
+
+Cancelling a steered turn drops any guidance that has not reached a safe
+boundary and releases its queue. Applied guidance is part of the same history
+turn, so an idle `checkpoint()` includes the full steered conversation.
+`checkpoint()` returns opaque, bounded, versioned bytes. Restore them only when
+creating a fresh agent:
 
 ```js
 const restored = await createFxAgent({ apiKey, model, checkpoint });
@@ -95,7 +160,10 @@ or a history change. The next prompt can run normally.
 
 The checkpoint contains conversation history and usage only. The host owns
 durable storage and must resupply models, credentials, instructions, tools,
-MCP clients, and skill records.
+MCP clients, and skill records. Reasoning effort and fast mode are
+agent-creation options and are not stored in a checkpoint: recreate the agent
+with new `effort` or `fast` values to change them, the same path as switching
+models.
 
 ## Models
 
@@ -136,11 +204,35 @@ const agent = await createFxAgent({
 });
 ```
 
-The JavaScript host is the authority for tool effects. The same descriptors,
-schemas, cancellation, results, and events are used by N-API and WebAssembly.
+Gateway web search can run at the provider instead of in the JavaScript host.
+Mark its canonical tool name with `providerExecuted: true` and omit `execute`:
+
+```js
+const agent = await createFxAgent({
+  apiKey,
+  tools: [{ name: "web_search", providerExecuted: true }],
+});
+```
+
+The kernel supplies the canonical schema and Gateway advertisement. Currently
+`web_search` is the supported provider-executed descriptor; unknown or local
+names reject agent creation. Provider-executed tools do not
+call host code or require a separate provider key; their `tool_start` and
+`tool_end` events, results, and checkpoint history use the same turn contract.
+Their built-in permission policy is enforced when the request is projected, as
+there is no local call-time effect to approve.
+
+For ordinary tools, the JavaScript host is the authority for effects. The same
+descriptors, schemas, cancellation, results, and events are used by N-API and
+WebAssembly.
 Cancelling a prompt aborts its tools' signals and stops waiting for their
 callbacks. Late results and rejections are ignored. Tools remain responsible
 for stopping their own work when their signal is aborted.
+
+A host tool may use any name, including the kernel's builtin names such as
+`write_file` and `edit_file`: the kernel routes by the registered executor, so
+a host-defined `write_file` calls the host's `execute()` rather than the
+builtin file mutation.
 Instructions are limited to 64 KiB of UTF-8 text, including text assembled by
 the MCP and skills adapters. They are the complete host-owned system context:
 libfx adds no hidden base prompt, and omitting `instructions` sends no system

package/fx-sdk.js

@@ -10,12 +10,31 @@ const workspaceOutputLimit = 64 * 1024;
 const maxInstructionsBytes = 64 * 1024;
 const maxApiKeyBytes = 64 * 1024;
 const maxModelBytes = 1024;
+// Matches the kernel's ReasoningEffort.max_name_bytes.
+const maxEffortBytes = 64;
 const maxUrlBytes = 16 * 1024;
 const maxModelCatalogBytes = 4 * 1024 * 1024;
 const maxModelCatalogEntries = 10_000;
 const streamReadsPerTaskYield = 32;
+const transportActivityIntervalMs = 250;
 const maxUnreadEventBytes = 1024 * 1024;
 const maxUnreadEvents = 256;
+// Prompt image limits mirror the host tool result image contract: the kernel
+// validates content, the SDK bounds the frame before it reaches the core.
+const maxPromptImages = 8;
+const maxPromptImageDataBytes = 5 * 1024 * 1024;
+const maxPromptImagesBytes = 8 * 1024 * 1024;
+// The core's ACP reader drops frames over 8 MiB without a request id to answer
+// (jsonrpc frame_resource_byte_limit), so the SDK must never emit one. The
+// envelope allowance covers the method key and request id.
+const maxPromptFrameBytes = 8 * 1024 * 1024;
+const promptFrameEnvelopeBytes = 128;
+const maxSteeringMessageBytes = 64 * 1024;
+const maxSteeringMessages = 64;
+const maxSteeringQueueBytes = 1024 * 1024;
+// tool_start events carry a bounded preview of the tool input; larger inputs
+// are marked truncated instead of dropped or sent whole.
+const maxToolStartInputBytes = 64 * 1024;
 
 function boundedString(value, name, maxBytes, required) {
   if (value === undefined && !required) return undefined;
@@ -44,6 +63,27 @@ function validateGatewayChatUrl(value) {
   }
 }
 
+// Mirrors the kernel's ReasoningEffort.parse: "auto"/"adaptive"/"default" pick
+// the model default; anything else must be a bounded effort name.
+function normalizeEffort(value) {
+  if (value === undefined) return undefined;
+  boundedString(value, "effort", maxEffortBytes, false);
+  if (!/^[A-Za-z0-9._-]+$/.test(value)) {
+    throw new TypeError('effort must use only letters, digits, ".", "-", or "_"');
+  }
+  return value;
+}
+
+// Mirrors the CLI's --fast/--no-fast toggle: a strict boolean, with undefined
+// leaving the model default in place.
+function normalizeFast(value) {
+  if (value === undefined) return undefined;
+  if (typeof value !== "boolean") {
+    throw new TypeError("fast must be a boolean");
+  }
+  return value;
+}
+
 function normalizeAgentOptions(value) {
   if (!value || typeof value !== "object" || Array.isArray(value)) {
     throw new TypeError("createFxAgent() options must be an object");
@@ -54,6 +94,8 @@ function normalizeAgentOptions(value) {
   }
   options.apiKey = boundedString(options.apiKey, "apiKey", maxApiKeyBytes, true);
   options.model = boundedString(options.model, "model", maxModelBytes, false);
+  options.effort = normalizeEffort(options.effort);
+  options.fast = normalizeFast(options.fast);
   validateGatewayChatUrl(options.gatewayChatUrl);
   return options;
 }
@@ -62,10 +104,26 @@ function agentEnvironment(options) {
   return {
     AI_GATEWAY_API_KEY: options.apiKey,
     ...(options.model === undefined ? {} : { FX_MODEL: options.model }),
+    ...(options.effort === undefined ? {} : { FX_EFFORT: options.effort }),
+    ...(options.fast === undefined ? {} : { FX_FAST: options.fast ? "true" : "false" }),
     ...(options.gatewayChatUrl === undefined ? {} : { FX_GATEWAY_CHAT_URL: options.gatewayChatUrl }),
   };
 }
 
+// The kernel rejects an unsupported effort or fast lane during initialize;
+// these messages originate only from that validation, so the rejection is
+// safe to retype.
+function agentBootstrapError(error) {
+  if (error instanceof Error &&
+    (error.message === "Invalid reasoning effort" || error.message.startsWith("Reasoning effort"))) {
+    error.code ??= "LIBFX_UNSUPPORTED_EFFORT";
+  }
+  if (error instanceof Error && error.message.startsWith("Fast mode")) {
+    error.code ??= "LIBFX_UNSUPPORTED_FAST";
+  }
+  return error;
+}
+
 async function cancelResponseBody(response) {
   try {
     await response.body?.cancel();
@@ -390,6 +448,9 @@ function createRuntime(options) {
   const stdin = new ByteQueue();
   const streams = new Map();
   const httpRequests = new Set();
+  const steering = [];
+  let steeringBytes = 0;
+  let steeringOpen = false;
   const workspaceExecs = new Set();
   const workspace = prepareWorkspaceAdapter(options.workspace);
   const args = ["fx", ...(options.args || [])];
@@ -602,6 +663,7 @@ function createRuntime(options) {
       state.readResult = null;
       if (done) return 0;
       if (!value?.length) return null;
+      options.onTransportChunk?.(value.length);
       return copy(value);
     };
     const immediate = consume();
@@ -655,6 +717,36 @@ function createRuntime(options) {
     });
   }
 
+  function clearSteering() {
+    steering.length = 0;
+    steeringBytes = 0;
+  }
+
+  function queueSteering(text) {
+    if (!steeringOpen) throw new Error("no prompt is running");
+    const value = encoder.encode(text);
+    if (value.length === 0) throw new TypeError("steering text cannot be empty");
+    if (value.length > maxSteeringMessageBytes) {
+      throw new RangeError(`steering text exceeds the ${maxSteeringMessageBytes} byte libfx limit`);
+    }
+    if (steering.length >= maxSteeringMessages || value.length > maxSteeringQueueBytes - steeringBytes) {
+      throw new Error("steering queue is full");
+    }
+    steering.push(value);
+    steeringBytes += value.length;
+  }
+
+  function steeringTake(outputPtr, outputCap) {
+    const value = steering[0];
+    if (!value) return 0;
+    const output = checkedBytes(outputPtr, outputCap);
+    if (!output || value.length > output.length) return -1;
+    output.subarray(0, value.length).set(value);
+    steering.shift();
+    steeringBytes -= value.length;
+    return value.length;
+  }
+
   let pendingHostToolResult = null;
   function hostToolCall(namePtr, nameLen, argumentsPtr, argumentsLen, outputPtr, outputCap, statusPtr) {
     pendingHostToolResult = null;
@@ -1024,6 +1116,8 @@ function createRuntime(options) {
       return chunk.length;
     },
     fx_host_tool_result_release() { pendingHostToolResult = null; },
+    fx_steering_take: steeringTake,
+    fx_steering_close() { steeringOpen = false; clearSteering(); },
     fx_open_url: new WebAssembly.Suspending(openUrl),
     fx_oauth_session_load: new WebAssembly.Suspending(oauthSessionLoad),
     fx_oauth_session_commit: new WebAssembly.Suspending(oauthSessionCommit),
@@ -1051,7 +1145,10 @@ function createRuntime(options) {
     setInstance(value) { instance = value; },
     write(data) { stdin.push(typeof data === "string" ? encoder.encode(data) : data); },
     wake() { stdin.wake(); },
-    closeStdin() { stdin.close(); },
+    closeStdin() { steeringOpen = false; clearSteering(); stdin.close(); },
+    openSteering() { clearSteering(); steeringOpen = true; },
+    steer: queueSteering,
+    closeSteering() { steeringOpen = false; clearSteering(); },
     abortHostEffects,
     abort(error) {
       aborted = true;
@@ -1182,9 +1279,30 @@ export async function createFxTerminal(options) {
 function normalizePromptInput(input) {
   if (typeof input === "string") return [{ type: "text", text: input }];
   if (!Array.isArray(input)) throw new TypeError("prompt input must be a string or an array of prompt blocks");
+  let imageCount = 0;
+  let imageBytes = 0;
   return input.map((block, index) => {
     if (!block || typeof block !== "object") throw new TypeError(`prompt block ${index} must be an object`);
-    if (block.type === "image") throw new TypeError("image prompt blocks are unsupported");
+    if (block.type === "image") {
+      if (typeof block.data !== "string" || block.data.length === 0) {
+        throw new TypeError(`image prompt block ${index} requires base64 data`);
+      }
+      if (typeof block.mimeType !== "string" || block.mimeType.length === 0 || block.mimeType.length > 128) {
+        throw new TypeError(`image prompt block ${index} requires a mimeType`);
+      }
+      if (block.data.length > maxPromptImageDataBytes) {
+        throw new RangeError(`image prompt block ${index} exceeds the ${maxPromptImageDataBytes} byte per-image libfx limit`);
+      }
+      imageCount += 1;
+      if (imageCount > maxPromptImages) {
+        throw new RangeError(`prompt cannot contain more than ${maxPromptImages} images`);
+      }
+      imageBytes += block.data.length;
+      if (imageBytes > maxPromptImagesBytes) {
+        throw new RangeError(`prompt images exceed the ${maxPromptImagesBytes} byte libfx frame limit`);
+      }
+      return { type: "image", data: block.data, mimeType: block.mimeType };
+    }
     if (block.type === "text") {
       if (typeof block.text !== "string") throw new TypeError(`text prompt block ${index} requires text`);
       return { type: "text", text: block.text };
@@ -1199,19 +1317,42 @@ function normalizePromptInput(input) {
   });
 }
 
+function normalizeSteeringInput(input) {
+  const blocks = normalizePromptInput(input);
+  if (blocks.some((block) => block.type !== "text")) {
+    throw new TypeError("steering accepts only text blocks");
+  }
+  const text = blocks.map((block) => block.text).join("\n");
+  if (text.length === 0) throw new TypeError("steering text cannot be empty");
+  if (encoder.encode(text).length > maxSteeringMessageBytes) {
+    throw new RangeError(`steering text exceeds the ${maxSteeringMessageBytes} byte libfx limit`);
+  }
+  return text;
+}
+
 function normalizeHostTools(value) {
   if (value === undefined) return { descriptors: [], executors: new Map() };
   if (!Array.isArray(value)) throw new TypeError("tools must be an array");
   if (value.length > 64) throw new RangeError("tools cannot contain more than 64 entries");
   const descriptors = [];
   const executors = new Map();
+  const names = new Set();
   for (const [index, tool] of value.entries()) {
     if (!tool || typeof tool !== "object") throw new TypeError(`tool ${index} must be an object`);
-    const { name, description, inputSchema, execute } = tool;
+    const { name, description, inputSchema, execute, providerExecuted } = tool;
     if (typeof name !== "string" || !/^[A-Za-z0-9_-]{1,64}$/.test(name)) {
       throw new TypeError(`tool ${index} has an invalid name`);
     }
-    if (executors.has(name)) throw new TypeError(`duplicate tool name: ${name}`);
+    if (names.has(name)) throw new TypeError(`duplicate tool name: ${name}`);
+    names.add(name);
+    if (providerExecuted !== undefined && typeof providerExecuted !== "boolean") {
+      throw new TypeError(`tool ${name} providerExecuted must be a boolean`);
+    }
+    if (providerExecuted === true) {
+      if (execute !== undefined) throw new TypeError(`provider-executed tool ${name} must not define execute()`);
+      descriptors.push({ name, providerExecuted: true });
+      continue;
+    }
     if (typeof description !== "string") throw new TypeError(`tool ${name} requires a description`);
     if (typeof execute !== "function") throw new TypeError(`tool ${name} requires execute()`);
     if (!inputSchema || typeof inputSchema !== "object" || Array.isArray(inputSchema)) {
@@ -1317,6 +1458,10 @@ export async function createFxAgent(options = {}) {
     for (let attemptIndex = 0; attemptIndex < 2; attemptIndex++) {
       const startedAt = performance.now();
       const attempt = activeTurn ? ++activeTurn.transportAttempts : attemptIndex + 1;
+      if (activeTurn) {
+        activeTurn.transportBytes = 0;
+        activeTurn.lastTransportActivityAt = null;
+      }
       emit("transport.start", { attempt, method, endpoint, model: options.model });
       try {
         if (activeTurn?.cancelled) {
@@ -1406,6 +1551,19 @@ export async function createFxAgent(options = {}) {
     args: ["acp"],
     env: agentEnvironment(options),
     hostToolExecutor: executeHostTool,
+    onTransportChunk(byteLength) {
+      const turn = activeTurn;
+      if (!isCurrentTurn(turn) || !Number.isSafeInteger(byteLength) || byteLength <= 0) return;
+      turn.transportBytes += byteLength;
+      const now = performance.now();
+      if (turn.lastTransportActivityAt !== null && now - turn.lastTransportActivityAt < transportActivityIntervalMs) return;
+      turn.lastTransportActivityAt = now;
+      emit("transport.activity", {
+        attempt: turn.transportAttempts,
+        chunkBytes: byteLength,
+        totalBytes: turn.transportBytes,
+      });
+    },
   };
   const runtime = options.runtimeFactory
     ? await options.runtimeFactory(runtimeOptions)
@@ -1490,7 +1648,7 @@ export async function createFxAgent(options = {}) {
     try { runtime.abortHostEffects(); } catch {}
     try { runtime.closeStdin(); } catch {}
     try { await runtime.exited; } catch {}
-    throw error;
+    throw agentBootstrapError(error);
   }
 
   const agent = {
@@ -1531,6 +1689,9 @@ export async function createFxAgent(options = {}) {
         const delta = update.content?.text;
         return delta ? { type: "reasoning_delta", delta } : null;
       }
+      if (update.sessionUpdate === "user_message_chunk" && update.content?.type === "text") {
+        return { type: "user_message", text: update.content.text };
+      }
       if (update.sessionUpdate === "tool_call") {
         toolNames.set(update.toolCallId, update.name || update.toolName || update.title || "tool");
         if (started.has(update.toolCallId)) return null;
@@ -1539,6 +1700,7 @@ export async function createFxAgent(options = {}) {
           type: "tool_start",
           id: update.toolCallId,
           name: toolNames.get(update.toolCallId),
+          ...toolStartInput(update.rawInput),
         };
       }
       if (update.sessionUpdate === "tool_call_update" &&
@@ -1561,6 +1723,7 @@ export async function createFxAgent(options = {}) {
     void result.catch(() => {});
     return {
       cancel() { rawTurn.cancel(); },
+      steer(input) { return rawTurn.steer(normalizeSteeringInput(input)); },
       [Symbol.asyncIterator]() {
         const iterator = (async function* () {
           for await (const update of rawTurn) {
@@ -1589,8 +1752,22 @@ export async function createFxAgent(options = {}) {
     return result;
   }
 
+  // The input object rides the event when it fits the preview budget; larger
+  // inputs become a bounded JSON prefix plus an explicit marker.
+  function toolStartInput(rawInput) {
+    if (rawInput === undefined || rawInput === null) return {};
+    const serialized = JSON.stringify(rawInput);
+    if (serialized === undefined) return {};
+    const bytes = encoder.encode(serialized);
+    if (bytes.length <= maxToolStartInputBytes) return { input: rawInput };
+    return { inputTruncated: true, inputPreview: decoder.decode(utf8Prefix(bytes, maxToolStartInputBytes)) };
+  }
+
   function startTurn(input, promptOptions) {
     const prompt = normalizePromptInput(input);
+    if (encoder.encode(JSON.stringify({ sessionId: "", prompt })).length + promptFrameEnvelopeBytes > maxPromptFrameBytes) {
+      throw new RangeError(`prompt exceeds the ${maxPromptFrameBytes} byte libfx frame limit`);
+    }
     const signal = promptOptions.signal;
     if (signal !== undefined && (typeof signal?.addEventListener !== "function" || typeof signal?.removeEventListener !== "function")) throw new TypeError("prompt signal must be an AbortSignal");
     const queue = [];
@@ -1622,10 +1799,34 @@ export async function createFxAgent(options = {}) {
       },
       toolControllers,
       transportAttempts: 0,
+      transportBytes: 0,
+      lastTransportActivityAt: null,
       get cancelled() { return cancelled; },
+      steer(text) {
+        if (finished || cancelled || activeTurn !== turn) {
+          return Promise.reject(new Error("no prompt is running"));
+        }
+        let accepted;
+        try {
+          if (typeof runtime.steer === "function") {
+            runtime.steer(text);
+            void turn.push({
+              sessionUpdate: "user_message_chunk",
+              content: { type: "text", text },
+            });
+            accepted = Promise.resolve();
+          } else {
+            accepted = request("libfx/steer", { sessionId, text });
+          }
+        } catch (error) {
+          return Promise.reject(error);
+        }
+        return accepted;
+      },
       cancel() {
         if (finished || cancelled) return;
         cancelled = true;
+        runtime.closeSteering?.();
         resumeOutput?.();
         resumeOutput = null;
         send({ jsonrpc: "2.0", method: "session/cancel", params: { sessionId } });
@@ -1658,6 +1859,7 @@ export async function createFxAgent(options = {}) {
       return turn;
     }
     activeTurn = turn;
+    runtime.openSteering?.();
     const abort = () => turn.cancel();
     signal?.addEventListener("abort", abort, { once: true });
     turn.result = request("session/prompt", { sessionId, prompt })
@@ -1673,6 +1875,7 @@ export async function createFxAgent(options = {}) {
         resumeOutput = null;
         signal?.removeEventListener("abort", abort);
         if (activeTurn === turn) activeTurn = null;
+        runtime.closeSteering?.();
         toolControllers.clear();
         if (discardedBytes) emit("output.discarded", { reason: "cancelled", bytes: discardedBytes });
         for (const waiter of waiters.splice(0)) {

package/node.cjs

@@ -188,12 +188,23 @@ var workspaceOutputLimit = 64 * 1024;
 var maxInstructionsBytes = 64 * 1024;
 var maxApiKeyBytes = 64 * 1024;
 var maxModelBytes = 1024;
+var maxEffortBytes = 64;
 var maxUrlBytes = 16 * 1024;
 var maxModelCatalogBytes = 4 * 1024 * 1024;
 var maxModelCatalogEntries = 1e4;
 var streamReadsPerTaskYield = 32;
+var transportActivityIntervalMs = 250;
 var maxUnreadEventBytes = 1024 * 1024;
 var maxUnreadEvents = 256;
+var maxPromptImages = 8;
+var maxPromptImageDataBytes = 5 * 1024 * 1024;
+var maxPromptImagesBytes = 8 * 1024 * 1024;
+var maxPromptFrameBytes = 8 * 1024 * 1024;
+var promptFrameEnvelopeBytes = 128;
+var maxSteeringMessageBytes = 64 * 1024;
+var maxSteeringMessages = 64;
+var maxSteeringQueueBytes = 1024 * 1024;
+var maxToolStartInputBytes = 64 * 1024;
 function boundedString(value, name, maxBytes, required) {
   if (value === undefined && !required)
     return;
@@ -227,6 +238,23 @@ function validateGatewayChatUrl(value) {
     throw new TypeError("gatewayChatUrl must use the canonical Gateway or explicit loopback HTTP");
   }
 }
+function normalizeEffort(value) {
+  if (value === undefined)
+    return;
+  boundedString(value, "effort", maxEffortBytes, false);
+  if (!/^[A-Za-z0-9._-]+$/.test(value)) {
+    throw new TypeError('effort must use only letters, digits, ".", "-", or "_"');
+  }
+  return value;
+}
+function normalizeFast(value) {
+  if (value === undefined)
+    return;
+  if (typeof value !== "boolean") {
+    throw new TypeError("fast must be a boolean");
+  }
+  return value;
+}
 function normalizeAgentOptions(value) {
   if (!value || typeof value !== "object" || Array.isArray(value)) {
     throw new TypeError("createFxAgent() options must be an object");
@@ -237,6 +265,8 @@ function normalizeAgentOptions(value) {
   }
   options.apiKey = boundedString(options.apiKey, "apiKey", maxApiKeyBytes, true);
   options.model = boundedString(options.model, "model", maxModelBytes, false);
+  options.effort = normalizeEffort(options.effort);
+  options.fast = normalizeFast(options.fast);
   validateGatewayChatUrl(options.gatewayChatUrl);
   return options;
 }
@@ -244,9 +274,20 @@ function agentEnvironment(options) {
   return {
     AI_GATEWAY_API_KEY: options.apiKey,
     ...options.model === undefined ? {} : { FX_MODEL: options.model },
+    ...options.effort === undefined ? {} : { FX_EFFORT: options.effort },
+    ...options.fast === undefined ? {} : { FX_FAST: options.fast ? "true" : "false" },
     ...options.gatewayChatUrl === undefined ? {} : { FX_GATEWAY_CHAT_URL: options.gatewayChatUrl }
   };
 }
+function agentBootstrapError(error) {
+  if (error instanceof Error && (error.message === "Invalid reasoning effort" || error.message.startsWith("Reasoning effort"))) {
+    error.code ??= "LIBFX_UNSUPPORTED_EFFORT";
+  }
+  if (error instanceof Error && error.message.startsWith("Fast mode")) {
+    error.code ??= "LIBFX_UNSUPPORTED_FAST";
+  }
+  return error;
+}
 async function cancelResponseBody(response) {
   try {
     await response.body?.cancel();
@@ -600,6 +641,9 @@ function createRuntime(options) {
   const stdin = new ByteQueue;
   const streams = new Map;
   const httpRequests = new Set;
+  const steering = [];
+  let steeringBytes = 0;
+  let steeringOpen = false;
   const workspaceExecs = new Set;
   const workspace = prepareWorkspaceAdapter(options.workspace);
   const args = ["fx", ...options.args || []];
@@ -835,6 +879,7 @@ function createRuntime(options) {
         return 0;
       if (!value?.length)
         return null;
+      options.onTransportChunk?.(value.length);
       return copy(value);
     };
     const immediate = consume();
@@ -891,6 +936,37 @@ function createRuntime(options) {
       httpRequests.delete(controller);
     });
   }
+  function clearSteering() {
+    steering.length = 0;
+    steeringBytes = 0;
+  }
+  function queueSteering(text) {
+    if (!steeringOpen)
+      throw new Error("no prompt is running");
+    const value = encoder.encode(text);
+    if (value.length === 0)
+      throw new TypeError("steering text cannot be empty");
+    if (value.length > maxSteeringMessageBytes) {
+      throw new RangeError(`steering text exceeds the ${maxSteeringMessageBytes} byte libfx limit`);
+    }
+    if (steering.length >= maxSteeringMessages || value.length > maxSteeringQueueBytes - steeringBytes) {
+      throw new Error("steering queue is full");
+    }
+    steering.push(value);
+    steeringBytes += value.length;
+  }
+  function steeringTake(outputPtr, outputCap) {
+    const value = steering[0];
+    if (!value)
+      return 0;
+    const output = checkedBytes(outputPtr, outputCap);
+    if (!output || value.length > output.length)
+      return -1;
+    output.subarray(0, value.length).set(value);
+    steering.shift();
+    steeringBytes -= value.length;
+    return value.length;
+  }
   let pendingHostToolResult = null;
   function hostToolCall(namePtr, nameLen, argumentsPtr, argumentsLen, outputPtr, outputCap, statusPtr) {
     pendingHostToolResult = null;
@@ -1349,6 +1425,11 @@ function createRuntime(options) {
     fx_host_tool_result_release() {
       pendingHostToolResult = null;
     },
+    fx_steering_take: steeringTake,
+    fx_steering_close() {
+      steeringOpen = false;
+      clearSteering();
+    },
     fx_open_url: new WebAssembly.Suspending(openUrl),
     fx_oauth_session_load: new WebAssembly.Suspending(oauthSessionLoad),
     fx_oauth_session_commit: new WebAssembly.Suspending(oauthSessionCommit),
@@ -1383,8 +1464,19 @@ function createRuntime(options) {
       stdin.wake();
     },
     closeStdin() {
+      steeringOpen = false;
+      clearSteering();
       stdin.close();
     },
+    openSteering() {
+      clearSteering();
+      steeringOpen = true;
+    },
+    steer: queueSteering,
+    closeSteering() {
+      steeringOpen = false;
+      clearSteering();
+    },
     abortHostEffects,
     abort(error) {
       aborted = true;
@@ -1547,11 +1639,31 @@ function normalizePromptInput(input) {
     return [{ type: "text", text: input }];
   if (!Array.isArray(input))
     throw new TypeError("prompt input must be a string or an array of prompt blocks");
+  let imageCount = 0;
+  let imageBytes = 0;
   return input.map((block, index) => {
     if (!block || typeof block !== "object")
       throw new TypeError(`prompt block ${index} must be an object`);
-    if (block.type === "image")
-      throw new TypeError("image prompt blocks are unsupported");
+    if (block.type === "image") {
+      if (typeof block.data !== "string" || block.data.length === 0) {
+        throw new TypeError(`image prompt block ${index} requires base64 data`);
+      }
+      if (typeof block.mimeType !== "string" || block.mimeType.length === 0 || block.mimeType.length > 128) {
+        throw new TypeError(`image prompt block ${index} requires a mimeType`);
+      }
+      if (block.data.length > maxPromptImageDataBytes) {
+        throw new RangeError(`image prompt block ${index} exceeds the ${maxPromptImageDataBytes} byte per-image libfx limit`);
+      }
+      imageCount += 1;
+      if (imageCount > maxPromptImages) {
+        throw new RangeError(`prompt cannot contain more than ${maxPromptImages} images`);
+      }
+      imageBytes += block.data.length;
+      if (imageBytes > maxPromptImagesBytes) {
+        throw new RangeError(`prompt images exceed the ${maxPromptImagesBytes} byte libfx frame limit`);
+      }
+      return { type: "image", data: block.data, mimeType: block.mimeType };
+    }
     if (block.type === "text") {
       if (typeof block.text !== "string")
         throw new TypeError(`text prompt block ${index} requires text`);
@@ -1568,6 +1680,20 @@ function normalizePromptInput(input) {
     throw new TypeError(`unsupported prompt block type: ${String(block.type)}`);
   });
 }
+function normalizeSteeringInput(input) {
+  const blocks = normalizePromptInput(input);
+  if (blocks.some((block) => block.type !== "text")) {
+    throw new TypeError("steering accepts only text blocks");
+  }
+  const text = blocks.map((block) => block.text).join(`
+`);
+  if (text.length === 0)
+    throw new TypeError("steering text cannot be empty");
+  if (encoder.encode(text).length > maxSteeringMessageBytes) {
+    throw new RangeError(`steering text exceeds the ${maxSteeringMessageBytes} byte libfx limit`);
+  }
+  return text;
+}
 function normalizeHostTools(value) {
   if (value === undefined)
     return { descriptors: [], executors: new Map };
@@ -1577,15 +1703,26 @@ function normalizeHostTools(value) {
     throw new RangeError("tools cannot contain more than 64 entries");
   const descriptors = [];
   const executors = new Map;
+  const names = new Set;
   for (const [index, tool] of value.entries()) {
     if (!tool || typeof tool !== "object")
       throw new TypeError(`tool ${index} must be an object`);
-    const { name, description, inputSchema, execute } = tool;
+    const { name, description, inputSchema, execute, providerExecuted } = tool;
     if (typeof name !== "string" || !/^[A-Za-z0-9_-]{1,64}$/.test(name)) {
       throw new TypeError(`tool ${index} has an invalid name`);
     }
-    if (executors.has(name))
+    if (names.has(name))
       throw new TypeError(`duplicate tool name: ${name}`);
+    names.add(name);
+    if (providerExecuted !== undefined && typeof providerExecuted !== "boolean") {
+      throw new TypeError(`tool ${name} providerExecuted must be a boolean`);
+    }
+    if (providerExecuted === true) {
+      if (execute !== undefined)
+        throw new TypeError(`provider-executed tool ${name} must not define execute()`);
+      descriptors.push({ name, providerExecuted: true });
+      continue;
+    }
     if (typeof description !== "string")
       throw new TypeError(`tool ${name} requires a description`);
     if (typeof execute !== "function")
@@ -1703,6 +1840,10 @@ async function createFxAgent(options = {}) {
     for (let attemptIndex = 0;attemptIndex < 2; attemptIndex++) {
       const startedAt = performance.now();
       const attempt = activeTurn ? ++activeTurn.transportAttempts : attemptIndex + 1;
+      if (activeTurn) {
+        activeTurn.transportBytes = 0;
+        activeTurn.lastTransportActivityAt = null;
+      }
       emit("transport.start", { attempt, method, endpoint, model: options.model });
       try {
         if (activeTurn?.cancelled) {
@@ -1798,7 +1939,22 @@ async function createFxAgent(options = {}) {
     fetch: transportFetch,
     args: ["acp"],
     env: agentEnvironment(options),
-    hostToolExecutor: executeHostTool
+    hostToolExecutor: executeHostTool,
+    onTransportChunk(byteLength) {
+      const turn = activeTurn;
+      if (!isCurrentTurn(turn) || !Number.isSafeInteger(byteLength) || byteLength <= 0)
+        return;
+      turn.transportBytes += byteLength;
+      const now = performance.now();
+      if (turn.lastTransportActivityAt !== null && now - turn.lastTransportActivityAt < transportActivityIntervalMs)
+        return;
+      turn.lastTransportActivityAt = now;
+      emit("transport.activity", {
+        attempt: turn.transportAttempts,
+        chunkBytes: byteLength,
+        totalBytes: turn.transportBytes
+      });
+    }
   };
   const runtime = options.runtimeFactory ? await options.runtimeFactory(runtimeOptions) : await instantiate(runtimeOptions);
   emit("runtime.ready");
@@ -1902,7 +2058,7 @@ async function createFxAgent(options = {}) {
     try {
       await runtime.exited;
     } catch {}
-    throw error;
+    throw agentBootstrapError(error);
   }
   const agent = {
     prompt(input, promptOptions = {}) {
@@ -1951,6 +2107,9 @@ async function createFxAgent(options = {}) {
         const delta = update.content?.text;
         return delta ? { type: "reasoning_delta", delta } : null;
       }
+      if (update.sessionUpdate === "user_message_chunk" && update.content?.type === "text") {
+        return { type: "user_message", text: update.content.text };
+      }
       if (update.sessionUpdate === "tool_call") {
         toolNames.set(update.toolCallId, update.name || update.toolName || update.title || "tool");
         if (started.has(update.toolCallId))
@@ -1959,7 +2118,8 @@ async function createFxAgent(options = {}) {
         return {
           type: "tool_start",
           id: update.toolCallId,
-          name: toolNames.get(update.toolCallId)
+          name: toolNames.get(update.toolCallId),
+          ...toolStartInput(update.rawInput)
         };
       }
       if (update.sessionUpdate === "tool_call_update" && (update.status === "completed" || update.status === "failed")) {
@@ -1983,6 +2143,9 @@ async function createFxAgent(options = {}) {
       cancel() {
         rawTurn.cancel();
       },
+      steer(input) {
+        return rawTurn.steer(normalizeSteeringInput(input));
+      },
       [Symbol.asyncIterator]() {
         const iterator = async function* () {
           for await (const update of rawTurn) {
@@ -2025,8 +2188,22 @@ async function createFxAgent(options = {}) {
       result.reasoningTokens = usage.reasoningTokens;
     return result;
   }
+  function toolStartInput(rawInput) {
+    if (rawInput === undefined || rawInput === null)
+      return {};
+    const serialized = JSON.stringify(rawInput);
+    if (serialized === undefined)
+      return {};
+    const bytes = encoder.encode(serialized);
+    if (bytes.length <= maxToolStartInputBytes)
+      return { input: rawInput };
+    return { inputTruncated: true, inputPreview: decoder.decode(utf8Prefix(bytes, maxToolStartInputBytes)) };
+  }
   function startTurn(input, promptOptions) {
     const prompt = normalizePromptInput(input);
+    if (encoder.encode(JSON.stringify({ sessionId: "", prompt })).length + promptFrameEnvelopeBytes > maxPromptFrameBytes) {
+      throw new RangeError(`prompt exceeds the ${maxPromptFrameBytes} byte libfx frame limit`);
+    }
     const signal = promptOptions.signal;
     if (signal !== undefined && (typeof signal?.addEventListener !== "function" || typeof signal?.removeEventListener !== "function"))
       throw new TypeError("prompt signal must be an AbortSignal");
@@ -2069,13 +2246,37 @@ async function createFxAgent(options = {}) {
       },
       toolControllers,
       transportAttempts: 0,
+      transportBytes: 0,
+      lastTransportActivityAt: null,
       get cancelled() {
         return cancelled;
       },
+      steer(text) {
+        if (finished || cancelled || activeTurn !== turn) {
+          return Promise.reject(new Error("no prompt is running"));
+        }
+        let accepted;
+        try {
+          if (typeof runtime.steer === "function") {
+            runtime.steer(text);
+            turn.push({
+              sessionUpdate: "user_message_chunk",
+              content: { type: "text", text }
+            });
+            accepted = Promise.resolve();
+          } else {
+            accepted = request("libfx/steer", { sessionId, text });
+          }
+        } catch (error) {
+          return Promise.reject(error);
+        }
+        return accepted;
+      },
       cancel() {
         if (finished || cancelled)
           return;
         cancelled = true;
+        runtime.closeSteering?.();
         resumeOutput?.();
         resumeOutput = null;
         send({ jsonrpc: "2.0", method: "session/cancel", params: { sessionId } });
@@ -2115,6 +2316,7 @@ async function createFxAgent(options = {}) {
       return turn;
     }
     activeTurn = turn;
+    runtime.openSteering?.();
     const abort = () => turn.cancel();
     signal?.addEventListener("abort", abort, { once: true });
     turn.result = request("session/prompt", { sessionId, prompt }).then((response) => ({ stopReason: cancelled ? "cancelled" : response.stopReason, usage: response.usage })).catch((error) => {
@@ -2129,6 +2331,7 @@ async function createFxAgent(options = {}) {
       signal?.removeEventListener("abort", abort);
       if (activeTurn === turn)
         activeTurn = null;
+      runtime.closeSteering?.();
       toolControllers.clear();
       if (discardedBytes)
         emit("output.discarded", { reason: "cancelled", bytes: discardedBytes });
@@ -2439,12 +2642,14 @@ async function getBackendInfo(value = {}) {
   }
 }
 function createNativeCoreRuntime(addon, options) {
-  const { apiKey, model, gatewayChatUrl } = options;
+  const { apiKey, model, effort, fast, gatewayChatUrl } = options;
   const core = addon.createCore({
     apiKey,
     home: options.home ?? import_node_os.homedir(),
     workspaceRoot: options.workspaceRoot ?? process.cwd(),
     ...model === undefined ? {} : { model },
+    ...effort === undefined ? {} : { effort },
+    ...fast === undefined ? {} : { fast },
     ...gatewayChatUrl === undefined ? {} : { gatewayChatUrl }
   });
   let readyFd;
@@ -2510,6 +2715,7 @@ function createNativeCoreRuntime(addon, options) {
       if (response.body) {
         for await (const chunk of response.body) {
           const buffer = Buffer.from(chunk);
+          options.onTransportChunk?.(buffer.length);
           let offset = 0;
           while (offset < buffer.length) {
             const end = Math.min(offset + 65536, buffer.length);

package/node.js

@@ -345,12 +345,14 @@ export async function getBackendInfo(value = {}) {
 }
 
 function createNativeCoreRuntime(addon, options) {
-  const { apiKey, model, gatewayChatUrl } = options;
+  const { apiKey, model, effort, fast, gatewayChatUrl } = options;
   const core = addon.createCore({
     apiKey,
     home: options.home ?? homedir(),
     workspaceRoot: options.workspaceRoot ?? process.cwd(),
     ...(model === undefined ? {} : { model }),
+    ...(effort === undefined ? {} : { effort }),
+    ...(fast === undefined ? {} : { fast }),
     ...(gatewayChatUrl === undefined ? {} : { gatewayChatUrl }),
   });
   let readyFd;
@@ -405,6 +407,7 @@ function createNativeCoreRuntime(addon, options) {
       if (response.body) {
         for await (const chunk of response.body) {
           const buffer = Buffer.from(chunk);
+          options.onTransportChunk?.(buffer.length);
           let offset = 0;
           while (offset < buffer.length) {
             const end = Math.min(offset + 64 * 1024, buffer.length);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "libfx",
-  "version": "0.0.10",
+  "version": "0.0.11",
   "description": "Embed fx agents and terminals in JavaScript hosts",
   "type": "module",
   "repository": {