octaflow 0.34.0 → 0.35.0

package/README.md

@@ -218,7 +218,7 @@ next to what your steps actually do.
 | 🔭 | **Observability** — lifecycle events (run history), per-step spans, and `createMeterObserver` turning the events into OpenTelemetry-style metrics — all pluggable, all structural |
 | 🤖 | **AI add-on** — instrumented models, token/cost capture, quota, daily rollups, a `usage` sink that books each step into `@octabits-io/metering` (the paid APIs around the model in the same table, priced at read time), refusal-aware models, lenient JSON from local models (`withJsonRepair`), a vendor's refusal explained by key and hint (`explainProviderError`), a per-run circuit breaker, a connection test (`testProvider`), Ollama's native API as language and embedding models (`createOllama` in `octaflow/ai/providers`: `num_ctx`, thinking and `keep_alive` per model or per call, no dependency) |
 | 🗂️ | **Model providers an administrator manages** — `createModelProviders` (`octaflow/ai`): providers as rows, each with its kind, server, region, a key that crosses a cipher seam, and its models with declared capabilities (temperature, JSON mode, vision, video, tools, embedding); `createPgModelProviderStore` + `modelProviderDdl` (`octaflow/postgres`), `modelProviderColumns` (`octaflow/drizzle`); `createModelResolver` (`octaflow/ai/providers`) builds the AI SDK model from a row, shapes each call to what the model declares, hands the host its `destination` for the disclosure guard, and rebuilds after a write |
-| 🏃 | **Model runners** — a model the app cannot reach, served by a process that connects *out*: a provider row of kind `runner` carries tags instead of a URL, a runner registered with a token (`octaflow/ai/runner`: `createModelRunnerHub`, shown once, kept as a hash) subscribes with its tags, claims the calls by long-poll through `@octabits-io/server/fetch/octaflow`'s `createModelRunnerHandler` and posts the answers back; the mailbox is two tables (`createPgModelRunnerStore` + `modelRunnerDdl`, `modelRunnerColumns` + `modelRunnerRequestColumns`) whose rows live only while a call is in flight; nobody online fails the call at once, so the breaker trips and the probe answers in milliseconds |
+| 🏃 | **Model runners** — a model the app cannot reach, served by a process that connects *out*: a provider row of kind `runner` carries tags instead of a URL, a runner registered with a token (`octaflow/ai/runner`: `createModelRunnerHub`, shown once, kept as a hash) subscribes with its tags, claims the calls by long-poll through `@octabits-io/server/fetch/octaflow`'s `createModelRunnerHandler` and posts the answers back; the mailbox is three tables (`createPgModelRunnerStore` + `modelRunnerDdl`; `modelRunnerColumns`, `modelRunnerRequestColumns`, `modelRunnerChunkColumns`) whose rows live only while a call is in flight, a streamed answer's parts chunk by chunk among them; nobody online fails the call at once, so the breaker trips and the probe answers in milliseconds; `octaflow-runner` is the runner as a process (Ollama or any OpenAI-compatible server) |
 | 🧱 | **Pluggable everything** — `WorkflowStore`, `Dispatcher`, `StepGate`, `FlowObserver`, hooks |
 
 ---
@@ -251,10 +251,10 @@ pnpm add ai @ai-sdk/provider
 
 ### The Postgres tables
 
-`octaflow/postgres` owns eight tables and ships their DDL: `flowStoreDdl()` (`flow_workflow`,
+`octaflow/postgres` owns nine tables and ships their DDL: `flowStoreDdl()` (`flow_workflow`,
 `flow_workflow_step`), `flowEventDdl()` (`flow_step_event`), `flowGateDdl()`
 (`flow_rate_bucket`, `flow_step_lease`), `modelProviderDdl()` (`model_provider`) and
-`modelRunnerDdl()` (`model_runner`, `model_runner_request`). Each is idempotent `CREATE TABLE IF NOT EXISTS`, takes
+`modelRunnerDdl()` (`model_runner`, `model_runner_request`, `model_runner_chunk`). Each is idempotent `CREATE TABLE IF NOT EXISTS`, takes
 a `schema`, and is meant to be pasted into your own migration system; `applySchema(pool, ddl)`
 is the dev and test shortcut.
 
@@ -273,8 +273,8 @@ migration. Where the store's queries are one SQL statement by nature — every o
 is — `./drizzle` ships the columns only, and the store stays the SQL one. So
 `octaflow/drizzle` exports `flowWorkflowColumns`, `flowWorkflowStepColumns`,
 `flowStepEventColumns`, `flowRateBucketColumns`, `flowStepLeaseColumns`, `modelProviderColumns`,
-`modelRunnerColumns` and `modelRunnerRequestColumns` (`drizzle-orm` an optional peer) and
-nothing that runs a query. The foreign keys, the composite keys, the unique
+`modelRunnerColumns`, `modelRunnerRequestColumns` and `modelRunnerChunkColumns` (`drizzle-orm` an
+optional peer) and nothing that runs a query. The foreign keys, the composite keys, the unique
 constraint and the indexes are yours to declare on the `pgTable`; each column set's doc lists
 the ones its DDL creates, and two of them the store relies on:
 
@@ -343,7 +343,8 @@ const { token } = (await runners.registerRunner({ id: 'dani-mac', label: "Dani's
 import { createModelRunnerHandler } from '@octabits-io/server/fetch/octaflow';
 export const POST = compose(withErrorMapping())(createModelRunnerHandler({ prefix: '/api/model-runner', hub: runners }));
 
-// the runner, on the machine with the model
+// the runner, on the machine with the model — the bin, or the loop in your own process
+//   npx octaflow-runner --url https://app.example/api/model-runner --token omr_… --ollama http://localhost:11434
 import { createHttpRunnerClient, runModelRunner } from 'octaflow/ai/runner';
 import { createOllama } from 'octaflow/ai/providers';
 await runModelRunner({
@@ -352,6 +353,15 @@ await runModelRunner({
 });
 ```
 
+**`octaflow-runner`**, the bin, is the runner as a process: `--url` and `--token` (or
+`OCTAFLOW_RUNNER_URL` / `OCTAFLOW_RUNNER_TOKEN`), one backend — `--ollama <url>` for Ollama's
+native API, `--openai-compatible <url>` (with `--api-key`) for mlx_lm.server, vLLM or llama.cpp
+through the optional `@ai-sdk/openai-compatible` peer — and `--concurrency`. At start it says
+who the token belongs to, which models the app's provider rows declare for its tags (the door
+tells it) and which of them the local server has, with the `ollama pull` for a missing one;
+`--check` does that and exits. A door it cannot reach is retried with a backoff, forever.
+The skill `serve-a-model-from-a-runner` walks through the four pieces.
+
 A runner may claim a request when it has **every tag** of the request — the provider row's
 tags — as a CI job's labels must all be on the runner. A call with nobody online for its
 tags fails at once with `provider_unreachable`, so `withCircuitBreaker` trips and
@@ -363,10 +373,13 @@ every provider, and the disclosure gate reads it the same way.
 
 The wire is the AI SDK's own call and answer as JSON — binary file data as base64, a URL
 restored on the other side — so the runner's backend is any `LanguageModelV4`: `createOllama`
-from this package, or `@ai-sdk/openai-compatible` for mlx_lm.server, vLLM and llama.cpp. The
-first version answers whole: `streamText` over a runner model works, emitted as one text
-block when the answer is there. Files reach the runner as bytes, since the runner model
-declares no URL support and the SDK downloads on the app's side. Embeddings go the same way.
+from this package, or `@ai-sdk/openai-compatible` for mlx_lm.server, vLLM and llama.cpp.
+`streamText` streams: the runner runs `doStream` and posts the parts in chunks as they come
+(every 100 ms, or 64 parts), a third table carries them, and the app emits each chunk as it
+lands; a chunk the door refuses tells the runner the caller gave up, and it cancels the
+stream. A runner on the older wire answers a stream whole, and the app emits that as parts
+too. Files reach the runner as bytes, since the runner model declares no URL support and the
+SDK downloads on the app's side. Embeddings go the same way.
 
 ---
 

package/bin/octaflow-runner.mjs

@@ -0,0 +1,209 @@
+#!/usr/bin/env node
+/**
+ * octaflow-runner — serve a model to an app from a machine the app cannot reach.
+ *
+ * The runner connects *out* to the app's door (`@octabits-io/server/fetch/octaflow`'s
+ * `createModelRunnerHandler`) with the token an administrator minted for it, asks for the model
+ * calls that carry its tags and runs them on the server beside it — Ollama, or anything with
+ * OpenAI's chat API (mlx_lm.server, vLLM, llama.cpp) — and posts the answers back. No port is
+ * opened; the machine needs a way out and nothing else.
+ *
+ *   octaflow-runner --url https://app.example/api/model-runner --token omr_… --ollama http://localhost:11434
+ *   octaflow-runner --url … --token … --openai-compatible http://localhost:8081/v1 [--api-key …]
+ *
+ *   --url <url>                   the door; or OCTAFLOW_RUNNER_URL
+ *   --token <token>               the runner's token, shown once when it was registered; or OCTAFLOW_RUNNER_TOKEN
+ *   --ollama <url>                serve from Ollama's native API at this URL (num_ctx, thinking, keep_alive per model honoured)
+ *   --openai-compatible <url>     serve from an OpenAI-compatible server at this base URL (its `/v1`)
+ *   --api-key <key>               the key that server wants, if any; or OCTAFLOW_RUNNER_API_KEY
+ *   --concurrency <n>             requests in flight at once (default 1 — mlx_lm.server answers one at a time); the door caps it at what was registered
+ *   --check                       connect, say who the token belongs to and whether each model the app declares is installed, then exit
+ *   --quiet                       log failures only
+ *   --help
+ *
+ * At start the runner says who it is and which models the app's provider rows declare for its
+ * tags, and asks the local server which of them it has: a missing one is a warning with the
+ * command that installs it, not a refusal — the app may declare a model it will ask for later.
+ */
+import { parseArgs } from 'node:util';
+import { fileURLToPath } from 'node:url';
+
+const USAGE = `octaflow-runner — serve a model to an app from a machine the app cannot reach
+
+  octaflow-runner --url <door> --token <token> (--ollama <url> | --openai-compatible <url> [--api-key <key>]) [--concurrency <n>] [--check] [--quiet]
+
+  --url                 the app's door (OCTAFLOW_RUNNER_URL)
+  --token               the runner's token (OCTAFLOW_RUNNER_TOKEN)
+  --ollama              Ollama's native API, e.g. http://localhost:11434
+  --openai-compatible   an OpenAI-compatible base URL, e.g. http://localhost:8081/v1 (mlx_lm.server, vLLM, llama.cpp)
+  --api-key             that server's key, if it wants one (OCTAFLOW_RUNNER_API_KEY)
+  --concurrency         requests at once (default 1)
+  --check               connect and report, then exit
+  --quiet               log failures only
+`;
+
+/** The command line and the environment as one configuration, or the reason it is not one. */
+export function readArgs(argv, env = process.env) {
+  let values;
+  try {
+    ({ values } = parseArgs({
+      args: argv,
+      options: {
+        url: { type: 'string' },
+        token: { type: 'string' },
+        ollama: { type: 'string' },
+        'openai-compatible': { type: 'string' },
+        'api-key': { type: 'string' },
+        concurrency: { type: 'string' },
+        check: { type: 'boolean', default: false },
+        quiet: { type: 'boolean', default: false },
+        help: { type: 'boolean', default: false },
+      },
+      strict: true,
+      allowPositionals: false,
+    }));
+  } catch (error) {
+    return { error: error instanceof Error ? error.message : String(error) };
+  }
+  if (values.help) return { help: true };
+  const url = values.url ?? env.OCTAFLOW_RUNNER_URL;
+  const token = values.token ?? env.OCTAFLOW_RUNNER_TOKEN;
+  if (!url) return { error: 'the door is missing: --url or OCTAFLOW_RUNNER_URL' };
+  if (!token) return { error: 'the token is missing: --token or OCTAFLOW_RUNNER_TOKEN' };
+  if (values.ollama && values['openai-compatible']) return { error: 'one backend: --ollama or --openai-compatible, not both' };
+  if (!values.ollama && !values['openai-compatible']) return { error: 'a backend is missing: --ollama <url> or --openai-compatible <url>' };
+  const concurrency = values.concurrency === undefined ? 1 : Number(values.concurrency);
+  if (!Number.isInteger(concurrency) || concurrency < 1) return { error: `--concurrency wants a whole number of 1 or more, not '${values.concurrency}'` };
+  return {
+    url,
+    token,
+    backend: values.ollama ? { kind: 'ollama', url: values.ollama } : { kind: 'openai-compatible', url: values['openai-compatible'], apiKey: values['api-key'] ?? env.OCTAFLOW_RUNNER_API_KEY },
+    concurrency,
+    check: values.check,
+    quiet: values.quiet,
+  };
+}
+
+/** The model ids the local server has, or `null` when it could not be asked. */
+export async function installedModels(backend, fetchImpl = fetch) {
+  try {
+    if (backend.kind === 'ollama') {
+      const res = await fetchImpl(`${backend.url.replace(/\/+$/, '')}/api/tags`, { signal: AbortSignal.timeout(5000) });
+      if (!res.ok) return null;
+      const body = await res.json();
+      return (body.models ?? []).map((m) => m.name ?? m.model).filter(Boolean);
+    }
+    const res = await fetchImpl(`${backend.url.replace(/\/+$/, '')}/models`, {
+      headers: backend.apiKey ? { authorization: `Bearer ${backend.apiKey}` } : {},
+      signal: AbortSignal.timeout(5000),
+    });
+    if (!res.ok) return null;
+    const body = await res.json();
+    return (body.data ?? []).map((m) => m.id).filter(Boolean);
+  } catch {
+    return null;
+  }
+}
+
+/** Ollama lists `name:tag`; an app may declare `name` alone and mean `name:latest`. */
+const sameModel = (declared, installed) => declared === installed || `${declared}:latest` === installed || declared === `${installed}:latest`;
+
+/** The lines the start-up report prints — what the token is, what the app declares, what is there. */
+export function startupReport(self, installed, backend) {
+  const lines = [`connected as ${self.id} (${self.label}) — tags ${self.tags.join(', ')}, up to ${self.concurrency} at once`];
+  if (self.models.length === 0) lines.push('the app declares no models for these tags yet — a provider row of kind `runner` with these tags names them');
+  for (const model of self.models) {
+    if (installed === null) lines.push(`  ${model} — the local server could not be asked whether it is installed`);
+    else if (installed.some((i) => sameModel(model, i))) lines.push(`  ${model} — installed`);
+    else lines.push(`  ${model} — NOT installed${backend.kind === 'ollama' ? ` (ollama pull ${model})` : ''}`);
+  }
+  return lines;
+}
+
+async function makeBackend(backend) {
+  if (backend.kind === 'ollama') {
+    const { createOllama } = await import('../dist/ai/providers/index.js');
+    return createOllama({ baseUrl: backend.url });
+  }
+  let mod;
+  try {
+    mod = await import('@ai-sdk/openai-compatible');
+  } catch {
+    throw new Error("an OpenAI-compatible backend needs the optional peer '@ai-sdk/openai-compatible' installed beside octaflow");
+  }
+  const sdk = mod.createOpenAICompatible({ name: 'runner', baseURL: backend.url, ...(backend.apiKey ? { apiKey: backend.apiKey } : {}), supportsStructuredOutputs: true });
+  return {
+    languageModel: (id) => sdk.languageModel(id),
+    embeddingModel: (id) => sdk.embeddingModel(id),
+  };
+}
+
+export async function main(argv, env = process.env, io = { out: console.log, err: console.error }) {
+  const args = readArgs(argv, env);
+  if (args.help) {
+    io.out(USAGE);
+    return 0;
+  }
+  if (args.error) {
+    io.err(`octaflow-runner: ${args.error}\n\n${USAGE}`);
+    return 2;
+  }
+  const { createHttpRunnerClient, runModelRunner } = await import('../dist/ai/runner/index.js');
+  const client = createHttpRunnerClient({ url: args.url, token: args.token });
+
+  let self;
+  try {
+    self = await client.self();
+  } catch (error) {
+    io.err(`octaflow-runner: cannot reach the door at ${args.url}: ${error instanceof Error ? error.message : String(error)}`);
+    return 1;
+  }
+  const installed = await installedModels(args.backend);
+  for (const line of startupReport(self, installed, args.backend)) io.out(line);
+  if (installed === null) io.err(`octaflow-runner: no model server answered at ${args.backend.url} — requests will fail until it is up`);
+  if (args.check) return installed === null ? 1 : 0;
+
+  const backend = await makeBackend(args.backend);
+  const stop = new AbortController();
+  const onSignal = () => {
+    io.out('stopping — what is in flight is aborted');
+    stop.abort();
+  };
+  process.once('SIGINT', onSignal);
+  process.once('SIGTERM', onSignal);
+
+  io.out(`waiting for work from ${args.url}`);
+  await runModelRunner({
+    client,
+    backend,
+    concurrency: Math.min(args.concurrency, self.concurrency),
+    signal: stop.signal,
+    log: (event) => {
+      switch (event.type) {
+        case 'claimed':
+          if (!args.quiet) io.out(`→ ${event.request.id.slice(0, 8)} ${event.request.kind} ${event.request.model}`);
+          break;
+        case 'delivered':
+          if (!args.quiet || !event.ok) io.out(`${event.ok ? '✓' : '✗'} ${event.requestId.slice(0, 8)} in ${event.elapsedMs} ms`);
+          break;
+        case 'lost':
+          io.err(`✗ ${event.requestId.slice(0, 8)} no longer ours — cancelled by the caller, or the lease ran out`);
+          break;
+        case 'door_unreachable':
+          io.err(`door unreachable (${event.error instanceof Error ? event.error.message : String(event.error)}) — retrying in ${Math.round(event.retryInMs / 1000)} s`);
+          break;
+      }
+    },
+  });
+  return 0;
+}
+
+if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
+  main(process.argv.slice(2)).then(
+    (code) => process.exit(code),
+    (error) => {
+      console.error(`octaflow-runner: ${error instanceof Error ? error.message : String(error)}`);
+      process.exit(1);
+    },
+  );
+}

package/dist/ai/providers/index.d.ts

@@ -1,7 +1,7 @@
 import { b as OpenAiCompatibleProviderId, g as EmbeddingProviderId, i as AiEndpointResidency, o as AiProviderId } from "../../provider-registry-C_mOfdHH.js";
 import { A as maskKeys, C as ProviderErrorExplanation, D as explainProviderError, E as ProviderErrorPreset, O as isFatalProviderError, S as ProviderError, T as ProviderErrorPhrase, _ as ExplainProviderErrorOptions, a as OllamaProbeReport, c as ProviderProbeOptions, d as testProvider, f as CircuitBreaker, h as withCircuitBreaker, i as OllamaProbeOptions, k as isProviderError, l as ProviderProbeReport, m as createCircuitBreaker, o as ProbeableProvider, p as CircuitBreakerOptions, s as ProviderProbeFailure, u as testOllamaModel, v as FATAL_PROVIDER_ERROR_KEYS, w as ProviderErrorKey, x as PROVIDER_ERROR_STATUS, y as PROVIDER_ERROR_PHRASES } from "../../provider-probe-BDg4c1Rk.js";
 import { a as ModelCapabilities, c as ModelProvider, g as ModelProviders, m as ModelProviderRegion, o as ModelEntry } from "../../model-providers-DQU57vAf.js";
-import { u as ModelRunnerHub } from "../../hub-SXfvoqH3.js";
+import { d as ModelRunnerHub } from "../../hub-CTf95SkP.js";
 import { EmbeddingModelV4, LanguageModelV4, LanguageModelV4CallOptions } from "@ai-sdk/provider";
 //#region src/ai/providers/ollama.d.ts
 /** `false` switches thinking off; a level asks a model that has levels for that much. */

package/dist/ai/providers/index.js

@@ -1,7 +1,7 @@
 import { a as testProvider, i as testOllamaModel, o as createCircuitBreaker, s as withCircuitBreaker, u as createRefusalAwareModel } from "../../provider-probe-CwZWwFlo.js";
 import { a as EMBEDDING_PROVIDERS, c as OPENAI_COMPATIBLE_PROVIDER_ID, d as resolveEndpointResidency, t as AI_PROVIDERS } from "../../provider-registry-B91lgOmu.js";
 import { a as PROVIDER_ERROR_STATUS, c as isFatalProviderError, l as isProviderError, n as FATAL_PROVIDER_ERROR_KEYS, o as ProviderError, r as PROVIDER_ERROR_PHRASES, s as explainProviderError, u as maskKeys } from "../../provider-errors-DZWjl8di.js";
-import { t as createRunnerModels } from "../../model-DY-oJny8.js";
+import { t as createRunnerModels } from "../../model-DztyiOGz.js";
 import { wrapLanguageModel } from "ai";
 import { APICallError } from "@ai-sdk/provider";
 //#region src/ai/providers/ollama.ts

package/dist/ai/runner/index.d.ts

@@ -1,5 +1,5 @@
-import { C as modelRunnerSelfSchema, S as modelRunnerInputSchema, _ as StoredModelRunnerRequest, a as ModelRunner, b as modelRunnerDeliverBodySchema, c as ModelRunnerHeartbeatBody, d as ModelRunnerRequestError, f as ModelRunnerRequestKind, g as StoredModelRunner, h as ModelRunnerStore, i as MODEL_RUNNER_TAG, l as ModelRunnerInput, m as ModelRunnerSelf, n as MODEL_RUNNER_REQUEST_KINDS, o as ModelRunnerClaimBody, p as ModelRunnerRequestStatus, r as MODEL_RUNNER_REQUEST_STATUSES, s as ModelRunnerDeliverBody, t as ClaimedModelRunnerRequest, u as ModelRunnerOutcome, v as createInMemoryModelRunnerStore, w as modelRunnerTagsSchema, x as modelRunnerHeartbeatBodySchema, y as modelRunnerClaimBodySchema } from "../../model-runner-wSk3p4m5.js";
-import { a as DEFAULT_MODEL_RUNNER_POLL_MS, c as ModelRunnerCall, d as ModelRunnerHubError, f as ModelRunnerHubOptions, h as hashModelRunnerToken, i as DEFAULT_MODEL_RUNNER_ONLINE_WINDOW_MS, l as ModelRunnerCallResult, m as generateModelRunnerToken, n as DEFAULT_MODEL_RUNNER_LEASE_MS, o as DEFAULT_MODEL_RUNNER_REQUEST_TIMEOUT_MS, p as createModelRunnerHub, r as DEFAULT_MODEL_RUNNER_MAX_ATTEMPTS, s as DEFAULT_MODEL_RUNNER_SWEEP_AFTER_MS, t as DEFAULT_MODEL_RUNNER_CLAIM_WAIT_MS, u as ModelRunnerHub } from "../../hub-SXfvoqH3.js";
+import { C as modelRunnerDeliverBodySchema, D as modelRunnerTagsSchema, E as modelRunnerSelfSchema, S as modelRunnerClaimBodySchema, T as modelRunnerInputSchema, _ as StoredModelRunner, a as ModelRunner, b as createInMemoryModelRunnerStore, c as ModelRunnerDeliverBody, d as ModelRunnerOutcome, f as ModelRunnerRequestError, g as ModelRunnerStore, h as ModelRunnerSelf, i as MODEL_RUNNER_TAG, l as ModelRunnerHeartbeatBody, m as ModelRunnerRequestStatus, n as MODEL_RUNNER_REQUEST_KINDS, o as ModelRunnerChunkBody, p as ModelRunnerRequestKind, r as MODEL_RUNNER_REQUEST_STATUSES, s as ModelRunnerClaimBody, t as ClaimedModelRunnerRequest, u as ModelRunnerInput, v as StoredModelRunnerChunk, w as modelRunnerHeartbeatBodySchema, x as modelRunnerChunkBodySchema, y as StoredModelRunnerRequest } from "../../model-runner-2ONkybAq.js";
+import { _ as generateModelRunnerToken, a as DEFAULT_MODEL_RUNNER_ONLINE_WINDOW_MS, c as DEFAULT_MODEL_RUNNER_SWEEP_AFTER_MS, d as ModelRunnerHub, f as ModelRunnerHubError, g as createModelRunnerHub, h as StartedModelRunnerCall, i as DEFAULT_MODEL_RUNNER_MAX_ATTEMPTS, l as ModelRunnerCall, m as ModelRunnerStart, n as DEFAULT_MODEL_RUNNER_CLAIM_WAIT_MS, o as DEFAULT_MODEL_RUNNER_POLL_MS, p as ModelRunnerHubOptions, r as DEFAULT_MODEL_RUNNER_LEASE_MS, s as DEFAULT_MODEL_RUNNER_REQUEST_TIMEOUT_MS, t as DEFAULT_MODEL_RUNNER_CHUNK_BATCH, u as ModelRunnerCallResult, v as hashModelRunnerToken } from "../../hub-CTf95SkP.js";
 import { EmbeddingModelV4, EmbeddingModelV4CallOptions, EmbeddingModelV4Result, LanguageModelV4, LanguageModelV4CallOptions, LanguageModelV4GenerateResult, LanguageModelV4StreamPart } from "@ai-sdk/provider";
 //#region src/ai/runner/model.d.ts
 export interface RunnerModelsOptions {
@@ -33,6 +33,10 @@ export interface ModelRunnerClient {
     held: string[];
   }>;
   deliver(requestId: string, outcome: ModelRunnerOutcome): Promise<boolean>;
+  /** One batch of a `stream` request's parts; `accepted: false` says to stop. */
+  chunk(requestId: string, seq: number, parts: unknown[]): Promise<{
+    accepted: boolean;
+  }>;
 }
 /** What runs the requests: a model per id, settings as the request's entry carries them. */
 export interface ModelRunnerBackend {
@@ -53,6 +57,10 @@ export interface RunModelRunnerOptions {
   /** The first wait after the door could not be reached; doubles up to `maxBackoffMs`. Default 1 s / 30 s. */
   backoffMs?: number;
   maxBackoffMs?: number;
+  /** A stream's parts are posted at least this often while some are waiting. Default 100 ms. */
+  chunkEveryMs?: number;
+  /** … or as soon as this many are waiting. Default 64. */
+  chunkParts?: number;
   /** Told about each request and each failure to reach the door. */
   log?: (event: ModelRunnerLogEvent) => void;
 }
@@ -72,8 +80,21 @@ export type ModelRunnerLogEvent = {
   error: unknown;
   retryInMs: number;
 };
+/** Where a stream's chunks go: the door, or a test. `false` back means stop. */
+export type ChunkSink = (seq: number, parts: unknown[]) => Promise<boolean>;
+export interface RunRequestOptions {
+  signal?: AbortSignal;
+  /** For a `stream` request: where its chunks go. Without one, the stream is collected and answered whole. */
+  chunks?: ChunkSink;
+  chunkEveryMs?: number;
+  chunkParts?: number;
+}
+/** The answer a runner delivers for a stream it posted chunk by chunk: how many chunks there were. */
+export interface StreamedResult {
+  streamed: number;
+}
 /** Run one claimed request on the backend; never throws — a failure is the outcome. */
-export declare function runModelRunnerRequest(backend: ModelRunnerBackend, request: ClaimedModelRunnerRequest, signal?: AbortSignal): Promise<ModelRunnerOutcome>;
+export declare function runModelRunnerRequest(backend: ModelRunnerBackend, request: ClaimedModelRunnerRequest, options?: RunRequestOptions | AbortSignal): Promise<ModelRunnerOutcome>;
 /** The loop. Returns when `signal` aborts. */
 export declare function runModelRunner(options: RunModelRunnerOptions): Promise<void>;
 /** The hub in the same process as the client — a host that runs its runner inside, and tests. */
@@ -84,7 +105,7 @@ export interface HttpRunnerClientOptions {
   token: string;
   fetch?: typeof fetch;
 }
-/** The door over HTTP: `createModelRunnerHandler`'s four routes. Throws on a door that cannot be reached or answers an error. */
+/** The door over HTTP: `createModelRunnerHandler`'s five routes. Throws on a door that cannot be reached or answers an error. */
 export declare function createHttpRunnerClient(options: HttpRunnerClientOptions): ModelRunnerClient;
 //#endregion
 //#region src/ai/runner/wire.d.ts
@@ -106,9 +127,12 @@ export declare function encodeCallOptions(options: LanguageModelV4CallOptions):
 export declare function decodeCallOptions(wire: unknown): LanguageModelV4CallOptions;
 export declare function encodeGenerateResult(result: LanguageModelV4GenerateResult): unknown;
 export declare function decodeGenerateResult(wire: unknown): LanguageModelV4GenerateResult;
+/** Stream parts as JSON: bytes as base64, URLs as strings; an `error` part carries a sentence, since an `Error` does not cross. */
+export declare function encodeStreamParts(parts: LanguageModelV4StreamPart[]): unknown[];
+export declare function decodeStreamParts(wire: unknown[]): LanguageModelV4StreamPart[];
 export declare function encodeEmbedOptions(options: EmbeddingModelV4CallOptions): EmbedPayload['options'];
 export declare function decodeEmbedOptions(wire: unknown): EmbeddingModelV4CallOptions;
 export declare function encodeEmbedResult(result: EmbeddingModelV4Result): unknown;
 export declare function decodeEmbedResult(wire: unknown): EmbeddingModelV4Result;
 //#endregion
-export { ClaimedModelRunnerRequest, DEFAULT_MODEL_RUNNER_CLAIM_WAIT_MS, DEFAULT_MODEL_RUNNER_LEASE_MS, DEFAULT_MODEL_RUNNER_MAX_ATTEMPTS, DEFAULT_MODEL_RUNNER_ONLINE_WINDOW_MS, DEFAULT_MODEL_RUNNER_POLL_MS, DEFAULT_MODEL_RUNNER_REQUEST_TIMEOUT_MS, DEFAULT_MODEL_RUNNER_SWEEP_AFTER_MS, MODEL_RUNNER_REQUEST_KINDS, MODEL_RUNNER_REQUEST_STATUSES, MODEL_RUNNER_TAG, ModelRunner, ModelRunnerCall, ModelRunnerCallResult, ModelRunnerClaimBody, ModelRunnerDeliverBody, ModelRunnerHeartbeatBody, ModelRunnerHub, ModelRunnerHubError, ModelRunnerHubOptions, ModelRunnerInput, ModelRunnerOutcome, ModelRunnerRequestError, ModelRunnerRequestKind, ModelRunnerRequestStatus, ModelRunnerSelf, ModelRunnerStore, StoredModelRunner, StoredModelRunnerRequest, createInMemoryModelRunnerStore, createModelRunnerHub, generateModelRunnerToken, hashModelRunnerToken, modelRunnerClaimBodySchema, modelRunnerDeliverBodySchema, modelRunnerHeartbeatBodySchema, modelRunnerInputSchema, modelRunnerSelfSchema, modelRunnerTagsSchema };
+export { ClaimedModelRunnerRequest, DEFAULT_MODEL_RUNNER_CHUNK_BATCH, DEFAULT_MODEL_RUNNER_CLAIM_WAIT_MS, DEFAULT_MODEL_RUNNER_LEASE_MS, DEFAULT_MODEL_RUNNER_MAX_ATTEMPTS, DEFAULT_MODEL_RUNNER_ONLINE_WINDOW_MS, DEFAULT_MODEL_RUNNER_POLL_MS, DEFAULT_MODEL_RUNNER_REQUEST_TIMEOUT_MS, DEFAULT_MODEL_RUNNER_SWEEP_AFTER_MS, MODEL_RUNNER_REQUEST_KINDS, MODEL_RUNNER_REQUEST_STATUSES, MODEL_RUNNER_TAG, ModelRunner, ModelRunnerCall, ModelRunnerCallResult, ModelRunnerChunkBody, ModelRunnerClaimBody, ModelRunnerDeliverBody, ModelRunnerHeartbeatBody, ModelRunnerHub, ModelRunnerHubError, ModelRunnerHubOptions, ModelRunnerInput, ModelRunnerOutcome, ModelRunnerRequestError, ModelRunnerRequestKind, ModelRunnerRequestStatus, ModelRunnerSelf, ModelRunnerStart, ModelRunnerStore, StartedModelRunnerCall, StoredModelRunner, StoredModelRunnerChunk, StoredModelRunnerRequest, createInMemoryModelRunnerStore, createModelRunnerHub, generateModelRunnerToken, hashModelRunnerToken, modelRunnerChunkBodySchema, modelRunnerClaimBodySchema, modelRunnerDeliverBodySchema, modelRunnerHeartbeatBodySchema, modelRunnerInputSchema, modelRunnerSelfSchema, modelRunnerTagsSchema };