modelpact-providers 1.0.0 → 1.1.0

package/README.md

@@ -1,6 +1,19 @@
 # modelpact-providers
 
-Language-model backends written against the modelpact contract — one transport per runtime, each green on the contract suite
+[![npm](https://img.shields.io/npm/v/modelpact-providers)](https://www.npmjs.com/package/modelpact-providers)
+[![ci](https://github.com/AvdienkoSergey/modelpact-providers/actions/workflows/ci.yml/badge.svg?event=pull_request)](https://github.com/AvdienkoSergey/modelpact-providers/actions/workflows/ci.yml)
+![node: ≥22](https://img.shields.io/badge/node-%E2%89%A522-339933)
+[![license: MIT](https://img.shields.io/badge/license-MIT-lightgrey)](LICENSE)
+
+**The transports for [modelpact](https://github.com/AvdienkoSergey/modelpact):
+a daemon on your machine, the model inside Chrome, and a model in the tab
+itself. Three backends, one dialect, and the contract suite green on each.**
+
+The engine holds the contract, the lifecycle and one backend with nothing
+behind it. This package holds the ones with something behind them. They are
+apart because they change for different reasons: a daemon's JSON, a browser's
+origin trial and a WebGPU runtime each move on their own clock, and none of
+them should move the contract.
 
 ## Install
 
@@ -8,19 +21,122 @@ Language-model backends written against the modelpact contract — one transport
 npm install modelpact-providers modelpact
 ```
 
-`modelpact` is a peer dependency: this package is built against its contract
-and does not carry a copy of it.
+`modelpact` is a peer dependency: this package is written against its contract
+and carries no copy of it.
+
+## The three
+
+| Provider                | Reaches                           | Wants                                   | Import from                  |
+| ----------------------- | --------------------------------- | --------------------------------------- | ---------------------------- |
+| `makeOllamaProvider`    | a daemon over HTTP, usually local | Ollama on `127.0.0.1:11434`             | `modelpact-providers`        |
+| `makePromptApiProvider` | Chrome's built-in Gemini Nano     | Chrome, and the weights downloaded once | `modelpact-providers`        |
+| `makeWebGpuProvider`    | a model in the tab, on WebGPU     | `@mlc-ai/web-llm`, and a GPU            | `modelpact-providers/webgpu` |
+
+```ts
+import { makeOllamaProvider } from "modelpact-providers";
+
+const access = await makeOllamaProvider({ model: "granite4:350m" }).access();
+if (access.kind !== "ready") return;
+const opened = await access.open({ system: "Answer in one sentence." });
+```
+
+Everything after the provider line is the contract's, and identical across the
+three — see [modelpact's README](https://github.com/AvdienkoSergey/modelpact#readme)
+for what a session promises.
+
+## Two entries, and the reason
+
+`modelpact-providers` is the two that cost a consumer nothing: `fetch` and JSON
+for the daemon, a global for the browser's own model, no runtime dependency
+behind either. `modelpact-providers/webgpu` is the third, because it carries
+`@mlc-ai/web-llm` — an optional peer dependency, so an app on the daemon never
+installs it. The split is by dependency, not by kind: the same reason
+`modelpact/testing` is a separate entry for `vitest`.
+
+## What each one is
+
+**Ollama.** Three endpoints are the whole backend — `/api/tags` says what is
+downloaded, `/api/pull` downloads, `/api/chat` generates. Shapes were read off
+a running daemon, not off the docs: a chat stream is NDJSON whose last line
+carries the counts, a pull line carries `completed` and `total` per layer, and
+an error is an HTTP status with a body. The daemon keeps nothing between
+requests, so the session's record is resent whole every turn.
+
+**Chrome's built-in model.** The one backend that keeps the conversation
+itself: `LanguageModel` is a session object and `prompt()` appends to it, so
+the record travels with the session rather than in the request. It fires its
+own `contextoverflow`, which is forwarded rather than re-derived, and reports
+usage against a window it decides — 9 216 tokens on Chrome 152, measured.
+The declarations are `@types/dom-chromium-ai`, patched under
+[`patches/`](patches) because the IDL is looser than the spec: several states
+the algorithm rejects at runtime are writable in the types, and a TS error at
+the keyboard beats a `TypeError` in the browser.
+
+**WebGPU.** A model in the tab through `@mlc-ai/web-llm`, and the only backend
+whose download costs the user their bandwidth rather than a daemon's. It was
+written before this package existed, in a directory one repository over, to
+answer one question: is the published API enough to write a backend with. It
+was, and it needed nothing added to the contract.
+
+## Tools
+
+All three accept `ModelRequest.tools`, and each executes them the way its
+transport can.
+
+| Provider   | How a call happens                                                                                |
+| ---------- | ------------------------------------------------------------------------------------------------- |
+| Ollama     | native `tool_calls`, answered under the `tool` role, in rounds bounded by `maxToolRounds`         |
+| Prompt API | handed to `create()`, and the browser calls `execute` itself                                      |
+| WebGPU     | not yet: the request is refused at `access`, which is the contract's answer for a backend without |
+
+Chrome 152 answers `available` to `availability()` with tools and then throws
+`InvalidStateError` from `create()` — measured, and it arrives as a refusal at
+`open`. A loop above should expect a refusal in both places and fall back to a
+schema-constrained answer.
+
+## The guard
+
+[`src/surface.ts`](src/surface.ts) names every published type on both sides —
+the engine's and this package's — and
+[`tsconfig.surface.json`](tsconfig.surface.json) compiles it with
+`skipLibCheck` off and `types: []`, which is how a consumer reads a `.d.ts`. A
+declaration that needs an ambient global fails there and nowhere else.
+
+That guard has found the same bug three times, in three packages. A published
+type naming `LanguageModel` broke a consumer who never installed
+`@types/dom-chromium-ai`. A `WebGpuConfig.engine` typed with
+`MLCEngineInterface` broke one the same way, through `@mlc-ai/web-llm`'s own
+declarations, which name packages they do not depend on. Both fixes are the
+same: a structural type of exactly what the backend uses, named locally, and no
+third-party type in any exported signature.
+
+Moving here found a third. `@mlc-ai/web-llm`'s `interruptGenerate()` returns a
+promise, though its own published interface says it returns nothing; the
+adapter called it and dropped that promise on the floor. It is marked `void`
+now, deliberately, because by the time it runs the lifecycle has already
+answered the caller.
+
+Three tsconfigs, and each has one job:
+
+| File                                             | Checks                                                                     |
+| ------------------------------------------------ | -------------------------------------------------------------------------- |
+| [`tsconfig.json`](tsconfig.json)                 | the source. `skipLibCheck` on, because `@mlc-ai/web-llm` cannot survive it |
+| [`tsconfig.patched.json`](tsconfig.patched.json) | everything but WebGPU, with `skipLibCheck` off — the patch, still applying |
+| [`tsconfig.surface.json`](tsconfig.surface.json) | the emitted declarations, as a consumer receives them                      |
 
 ## Scripts
 
-| Script                 | What it does                     |
-| ---------------------- | -------------------------------- |
-| `npm run typecheck`    | `tsc --noEmit` over `src`        |
-| `npm run lint`         | ESLint, type-aware               |
-| `npm run format:check` | Prettier, check only             |
-| `npm test`             | Vitest, once                     |
-| `npm run test:watch`   | Vitest, watching                 |
-| `npm run build`        | `dist/` — JS, declarations, maps |
+| Script                  | What it does                                           |
+| ----------------------- | ------------------------------------------------------ |
+| `npm run typecheck`     | both source configs                                    |
+| `npm run lint`          | ESLint, type-aware                                     |
+| `npm run format:check`  | Prettier, check only                                   |
+| `npm test`              | Vitest; the Ollama contract suite skips with no daemon |
+| `npm run check:surface` | builds, then reads the declarations from outside       |
+| `npm run build`         | `dist/` — JS, declarations, maps                       |
+
+The Ollama suite wants a daemon on `127.0.0.1:11434` holding `granite4:350m`.
+Without one it skips loudly rather than passing quietly.
 
 ## Releases
 

package/dist/index.d.ts

@@ -1,2 +1,12 @@
-export {};
+/**
+ * The transports that cost a consumer nothing but this package: `fetch` and
+ * JSON for the daemon, a global for the browser's own model, no runtime
+ * dependency behind either. Import one and a bundler drops the other.
+ *
+ * The WebGPU backend is `modelpact-providers/webgpu` instead, because it
+ * carries `@mlc-ai/web-llm` — a separate entry for a separate dependency, the
+ * way `modelpact/testing` is separate for `vitest`.
+ */
+export { makeOllamaProvider, type OllamaConfig } from "./ollama.js";
+export { makePromptApiProvider } from "./prompt-api.js";
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,EAAE,kBAAkB,EAAE,KAAK,YAAY,EAAE,MAAM,aAAa,CAAC;AACpE,OAAO,EAAE,qBAAqB,EAAE,MAAM,iBAAiB,CAAC"}

package/dist/index.js

@@ -1,2 +1,12 @@
-export {};
+/**
+ * The transports that cost a consumer nothing but this package: `fetch` and
+ * JSON for the daemon, a global for the browser's own model, no runtime
+ * dependency behind either. Import one and a bundler drops the other.
+ *
+ * The WebGPU backend is `modelpact-providers/webgpu` instead, because it
+ * carries `@mlc-ai/web-llm` — a separate entry for a separate dependency, the
+ * way `modelpact/testing` is separate for `vitest`.
+ */
+export { makeOllamaProvider } from "./ollama.js";
+export { makePromptApiProvider } from "./prompt-api.js";
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,EAAE,kBAAkB,EAAqB,MAAM,aAAa,CAAC;AACpE,OAAO,EAAE,qBAAqB,EAAE,MAAM,iBAAiB,CAAC"}

package/dist/ollama.d.ts

@@ -0,0 +1,39 @@
+/**
+ * Ollama on a machine you can reach over HTTP.
+ *
+ * The daemon keeps nothing between requests: `/api/chat` is handed the whole
+ * conversation every time, which is what `request.history` is for. Three
+ * endpoints are the whole backend — `/api/tags` says what is downloaded,
+ * `/api/pull` downloads, `/api/chat` generates — and everything else the
+ * contract promises is the lifecycle's, back in `modelpact`.
+ *
+ * Shapes here were read off a running daemon, not off the docs: a chat stream
+ * is NDJSON whose last line carries the counts, a pull line carries `completed`
+ * and `total` per layer, and an error is an HTTP status with `{"error": "…"}`.
+ */
+import { type AiProvider } from "modelpact/backend";
+export interface OllamaConfig {
+    /** The tag as `/api/tags` lists it, such as `granite4:350m`. */
+    readonly model: string;
+    /**
+     * `127.0.0.1` and not `localhost`: the daemon binds the one, and the name
+     * can resolve to the other family first and refuse the connection.
+     */
+    readonly host?: string;
+    /**
+     * Sent as `num_ctx`, and therefore the window in force rather than a guess
+     * at one. The default matches the daemon's own; a model that can take more
+     * will, at the price of the memory the cache for it costs.
+     */
+    readonly contextWindow?: number;
+    /** For a proxy, an auth header, or a test with no daemon behind it. */
+    readonly fetch?: typeof globalThis.fetch;
+    /**
+     * How many times one turn may come back with tool calls before it is failed.
+     * Per turn, not per session: a model that keeps asking spends the window on
+     * its own questions and never answers.
+     */
+    readonly maxToolRounds?: number;
+}
+export declare function makeOllamaProvider(config: OllamaConfig): AiProvider;
+//# sourceMappingURL=ollama.d.ts.map

package/dist/ollama.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"ollama.d.ts","sourceRoot":"","sources":["../src/ollama.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AACH,OAAO,EAaL,KAAK,UAAU,EAUhB,MAAM,mBAAmB,CAAC;AAE3B,MAAM,WAAW,YAAY;IAC3B,gEAAgE;IAChE,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB;;;OAGG;IACH,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB;;;;OAIG;IACH,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;IAChC,uEAAuE;IACvE,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,UAAU,CAAC,KAAK,CAAC;IACzC;;;;OAIG;IACH,QAAQ,CAAC,aAAa,CAAC,EAAE,MAAM,CAAC;CACjC;AAsiBD,wBAAgB,kBAAkB,CAAC,MAAM,EAAE,YAAY,GAAG,UAAU,CASnE"}

package/dist/ollama.js

@@ -0,0 +1,442 @@
+/**
+ * Ollama on a machine you can reach over HTTP.
+ *
+ * The daemon keeps nothing between requests: `/api/chat` is handed the whole
+ * conversation every time, which is what `request.history` is for. Three
+ * endpoints are the whole backend — `/api/tags` says what is downloaded,
+ * `/api/pull` downloads, `/api/chat` generates — and everything else the
+ * contract promises is the lifecycle's, back in `modelpact`.
+ *
+ * Shapes here were read off a running daemon, not off the docs: a chat stream
+ * is NDJSON whose last line carries the counts, a pull line carries `completed`
+ * and `total` per layer, and an error is an HTTP status with `{"error": "…"}`.
+ */
+import { AiError, contextUsage, createProvider, err, findTool, fraction, ndjsonLines, ok, runTool, tokens, } from "modelpact/backend";
+const DEFAULTS = {
+    host: "http://127.0.0.1:11434",
+    contextWindow: 4096,
+    maxToolRounds: 8,
+};
+const asRecord = (value) => {
+    const isObject = typeof value === "object" && value !== null;
+    return isObject && !Array.isArray(value)
+        ? value
+        : null;
+};
+const asString = (value) => typeof value === "string" ? value : null;
+const asNumber = (value) => typeof value === "number" && Number.isFinite(value) ? value : null;
+const parseJson = (text) => {
+    try {
+        return JSON.parse(text);
+    }
+    catch {
+        return null;
+    }
+};
+const toEndpoint = (config) => ({
+    host: config.host ?? DEFAULTS.host,
+    // Bound, and not optional: in a browser `fetch` is a method of the window and
+    // throws `TypeError: Illegal invocation` once it is held on its own. Node
+    // does not care, so nothing but a page catches this (measured).
+    call: config.fetch ?? globalThis.fetch.bind(globalThis),
+});
+const postTo = (endpoint, path, body, signal) => endpoint.call(`${endpoint.host}${path}`, {
+    method: "POST",
+    headers: { "content-type": "application/json" },
+    body: JSON.stringify(body),
+    ...(signal === undefined ? {} : { signal }),
+});
+/** The daemon says what went wrong in the body; a status alone would lose it. */
+const failureFromResponse = async (response) => {
+    const text = await response.text().catch(() => "");
+    const errorText = asString(asRecord(parseJson(text))?.error);
+    const detail = errorText ?? `${response.status} from the daemon`;
+    // 400 is the daemon reading the request and refusing it, which is a bug on
+    // this side. Everything else is the daemon's own trouble.
+    return response.status === 400
+        ? { kind: "invalid-input", detail }
+        : { kind: "failed", detail };
+};
+const listModels = async (endpoint) => {
+    const response = await endpoint.call(`${endpoint.host}/api/tags`);
+    if (!response.ok)
+        return [];
+    const listedModels = asRecord(await response.json().catch(() => null))?.models;
+    if (!Array.isArray(listedModels))
+        return [];
+    const names = listedModels.map((entry) => asString(asRecord(entry)?.model));
+    return names.filter((name) => name !== null);
+};
+const getAvailability = async (config) => {
+    const endpoint = toEndpoint(config);
+    let downloadedModels;
+    try {
+        downloadedModels = await listModels(endpoint);
+    }
+    catch (cause) {
+        // Nothing answering is not a failed request, it is no Ollama here.
+        return { kind: "unavailable", reason: { kind: "unsupported", cause } };
+    }
+    const isDownloaded = downloadedModels.includes(config.model);
+    return isDownloaded
+        ? { kind: "ready" }
+        : { kind: "needs-download", started: false };
+};
+/**
+ * Pull progress is per layer, and layers are announced as the pull reaches
+ * them, so the denominator grows while it runs. The share can therefore stall,
+ * and `ProgressMonitor` drops it if it would step back. `success` is what
+ * makes the last report a 1, since that line carries no numbers.
+ */
+const readPullProgress = (reportProgress) => {
+    const layers = new Map();
+    return new TransformStream({
+        transform: (line, controller) => {
+            const parsedLine = asRecord(parseJson(line));
+            if (parsedLine === null)
+                return;
+            const errorText = asString(parsedLine.error);
+            if (errorText !== null)
+                throw new AiError({ kind: "failed", detail: errorText });
+            const digest = asString(parsedLine.digest);
+            const total = asNumber(parsedLine.total);
+            const completed = asNumber(parsedLine.completed) ?? 0;
+            if (digest !== null && total !== null && total > 0) {
+                layers.set(digest, { total, completed });
+            }
+            const isDone = asString(parsedLine.status) === "success";
+            const pulledShare = isDone ? 1 : getPulledShare(layers);
+            const progress = fraction(pulledShare);
+            if (progress !== null)
+                reportProgress(progress);
+            controller.enqueue(line);
+        },
+    });
+};
+const getPulledShare = (layers) => {
+    let total = 0;
+    let completed = 0;
+    for (const layer of layers.values()) {
+        total += layer.total;
+        completed += layer.completed;
+    }
+    return total === 0 ? 0 : completed / total;
+};
+const pullModel = async (endpoint, model, reportProgress) => {
+    const response = await postTo(endpoint, "/api/pull", { model, stream: true });
+    if (!response.ok)
+        return err(await failureFromResponse(response));
+    if (response.body === null)
+        return err({ kind: "failed", detail: "the pull sent no body" });
+    const lines = response.body
+        .pipeThrough(new TextDecoderStream())
+        .pipeThrough(ndjsonLines())
+        .pipeThrough(readPullProgress(reportProgress));
+    const reader = lines.getReader();
+    try {
+        // Read to the end: the transform above is where the reporting happens, and
+        // the body is not finished until it stops yielding.
+        for (;;) {
+            const chunk = await reader.read();
+            if (chunk.done)
+                return ok(null);
+        }
+    }
+    catch (error) {
+        return err(error instanceof AiError
+            ? error.failure
+            : { kind: "failed", detail: "the pull was interrupted", cause: error });
+    }
+};
+const toOllamaTools = (tools) => tools.map((tool) => ({
+    type: "function",
+    function: {
+        name: tool.name,
+        description: tool.description,
+        parameters: tool.inputSchema,
+    },
+}));
+/** A call carries `function.name` and `function.arguments`, an object; one without a name is skipped. */
+const readToolCalls = (message) => {
+    const listedCalls = message?.tool_calls;
+    if (!Array.isArray(listedCalls))
+        return [];
+    const readCalls = [];
+    for (const listedCall of listedCalls) {
+        const calledFunction = asRecord(asRecord(listedCall)?.function);
+        const name = asString(calledFunction?.name);
+        if (name === null)
+            continue;
+        const callArguments = asRecord(calledFunction?.arguments) ?? {};
+        readCalls.push({ function: { name, arguments: callArguments } });
+    }
+    return readCalls;
+};
+const readWhole = async (stream) => {
+    const reader = stream.getReader();
+    const parts = [];
+    try {
+        for (;;) {
+            const chunk = await reader.read();
+            if (chunk.done)
+                return ok(parts.join(""));
+            parts.push(chunk.value);
+        }
+    }
+    catch (error) {
+        return err(error instanceof AiError
+            ? error.failure
+            : { kind: "failed", detail: "the chat stream broke", cause: error });
+    }
+};
+class OllamaModel {
+    #endpoint;
+    #model;
+    #contextWindow;
+    #system;
+    #tools;
+    #maxToolRounds;
+    #reportOverflow;
+    /** The last turn's counts, which is what the context holds now rather than a running sum. */
+    #usedTokens = 0;
+    constructor(config, options) {
+        this.#endpoint = toEndpoint(config);
+        this.#model = config.model;
+        this.#contextWindow = config.contextWindow ?? DEFAULTS.contextWindow;
+        this.#system = options.session.system;
+        this.#tools = options.request.tools ?? [];
+        this.#maxToolRounds = config.maxToolRounds ?? DEFAULTS.maxToolRounds;
+        this.#reportOverflow = options.reportOverflow;
+    }
+    generateStream = async (input, request) => {
+        const conversation = this.#toConversation(input, request);
+        const responseResult = await this.#chat(conversation, request, true);
+        if (!responseResult.ok)
+            return responseResult;
+        const body = responseResult.value.body;
+        if (body === null)
+            return err({ kind: "failed", detail: "the chat sent no body" });
+        return ok(this.#streamRounds(body, conversation, request));
+    };
+    generateWhole = async (input, request) => {
+        // A turn with tools is rounds, and rounds are the streaming path read to
+        // its end; only a plain turn has a whole-answer call worth a second shape.
+        if (this.#tools.length > 0) {
+            const streamResult = await this.generateStream(input, request);
+            return streamResult.ok ? readWhole(streamResult.value) : streamResult;
+        }
+        const conversation = this.#toConversation(input, request);
+        const responseResult = await this.#chat(conversation, request, false);
+        if (!responseResult.ok)
+            return responseResult;
+        const parsedBody = asRecord(await responseResult.value.json().catch(() => null));
+        if (parsedBody === null)
+            return err({ kind: "failed", detail: "the chat sent no JSON" });
+        const answerText = asString(asRecord(parsedBody.message)?.content);
+        if (answerText === null)
+            return err({ kind: "failed", detail: "the chat sent no message" });
+        this.#charge(parsedBody);
+        return ok(answerText);
+    };
+    usage = () => {
+        const used = tokens(this.#usedTokens) ?? ZERO_TOKENS;
+        return contextUsage(used, this.#contextWindow);
+    };
+    /**
+     * Nothing to release. The daemon unloads on its own timer, and telling it to
+     * unload here would take the model out from under whoever else is on it.
+     */
+    dispose = () => undefined;
+    #toConversation(input, request) {
+        const askedMessage = { role: "user", content: input };
+        const conversation = [...request.history, askedMessage];
+        return this.#system === undefined
+            ? conversation
+            : [{ role: "user", content: this.#system }, ...conversation];
+    }
+    async #chat(messages, request, stream) {
+        const body = {
+            model: this.#model,
+            messages,
+            stream,
+            options: { num_ctx: this.#contextWindow },
+            ...(request.schema === undefined ? {} : { format: request.schema }),
+            ...(this.#tools.length === 0
+                ? {}
+                : { tools: toOllamaTools(this.#tools) }),
+        };
+        const response = await postTo(this.#endpoint, "/api/chat", body, request.signal);
+        return response.ok
+            ? ok(response)
+            : err(await failureFromResponse(response));
+    }
+    /**
+     * Rounds: an answer is read to its end, and where it ends in tool calls the
+     * calls are answered and the conversation sent again. Only text reaches the
+     * caller, so a turn that is all calls is silent until its last round.
+     * Bounded, because a model that keeps calling would spend the window on it:
+     * `granite4:350m` asks for the same listing until something stops it.
+     */
+    #streamRounds(firstBody, firstConversation, request) {
+        let conversation = firstConversation;
+        let currentRound = this.#openRound(firstBody);
+        let roundsTaken = 0;
+        const advance = async (controller) => {
+            for (;;) {
+                const chunk = await currentRound.reader.read();
+                if (!chunk.done) {
+                    controller.enqueue(chunk.value);
+                    return;
+                }
+                if (currentRound.toolCalls.length === 0) {
+                    controller.close();
+                    return;
+                }
+                roundsTaken += 1;
+                if (roundsTaken > this.#maxToolRounds) {
+                    throw new AiError({
+                        kind: "failed",
+                        detail: `the model called tools ${roundsTaken} times without answering`,
+                    });
+                }
+                conversation = await this.#answerToolCalls(conversation, currentRound, request.signal);
+                const nextResult = await this.#chat(conversation, request, true);
+                if (!nextResult.ok)
+                    throw new AiError(nextResult.error);
+                const nextBody = nextResult.value.body;
+                if (nextBody === null)
+                    throw new AiError({
+                        kind: "failed",
+                        detail: "the chat sent no body",
+                    });
+                currentRound = this.#openRound(nextBody);
+            }
+        };
+        return new ReadableStream({
+            pull: (controller) => advance(controller),
+            cancel: (reason) => currentRound.reader.cancel(reason),
+        });
+    }
+    /** Content out, calls and counts kept: the last line carries both `done` and the totals. */
+    #openRound(body) {
+        const contentParts = [];
+        const toolCalls = [];
+        const readLines = new TransformStream({
+            transform: (line, controller) => {
+                const parsedLine = asRecord(parseJson(line));
+                if (parsedLine === null)
+                    return;
+                const errorText = asString(parsedLine.error);
+                if (errorText !== null)
+                    throw new AiError({ kind: "failed", detail: errorText });
+                if (parsedLine.done === true)
+                    this.#charge(parsedLine);
+                const message = asRecord(parsedLine.message);
+                toolCalls.push(...readToolCalls(message));
+                const delta = asString(message?.content) ?? "";
+                if (delta === "")
+                    return;
+                contentParts.push(delta);
+                controller.enqueue(delta);
+            },
+        });
+        const deltas = body
+            .pipeThrough(new TextDecoderStream())
+            .pipeThrough(ndjsonLines())
+            .pipeThrough(readLines);
+        return { reader: deltas.getReader(), contentParts, toolCalls };
+    }
+    /**
+     * The model's call goes back as its own turn, then each answer with the
+     * `tool` role, which is what the daemon's template expects. A name the model
+     * made up is answered by name rather than failing the turn: the model can
+     * pick again, where a failed turn could not. A tool that throws cannot be
+     * answered for, and ends the turn.
+     */
+    async #answerToolCalls(conversation, round, signal) {
+        const answered = [
+            ...conversation,
+            {
+                role: "assistant",
+                content: round.contentParts.join(""),
+                tool_calls: round.toolCalls,
+            },
+        ];
+        for (const call of round.toolCalls) {
+            const name = call.function.name;
+            const tool = findTool(this.#tools, name);
+            if (tool === undefined) {
+                answered.push({
+                    role: "tool",
+                    content: `there is no tool called "${name}"`,
+                    tool_name: name,
+                });
+                continue;
+            }
+            const toolResult = await runTool(tool, call.function.arguments, signal);
+            if (!toolResult.ok)
+                throw new AiError(toolResult.error);
+            answered.push({
+                role: "tool",
+                content: toolResult.value,
+                tool_name: name,
+            });
+        }
+        return answered;
+    }
+    /**
+     * `prompt_eval_count` is the whole prompt, history included, so the two
+     * counts together are what the window holds after this turn. Summing across
+     * turns would count the history once per turn.
+     */
+    #charge(finishedLine) {
+        const promptTokens = asNumber(finishedLine.prompt_eval_count) ?? 0;
+        const answerTokens = asNumber(finishedLine.eval_count) ?? 0;
+        this.#usedTokens = promptTokens + answerTokens;
+        if (this.#hasSpentTheWindow(finishedLine))
+            this.#reportOverflow();
+    }
+    /**
+     * A daemon that shifts context answers past `num_ctx` and the counts say so
+     * on their own. One that stops instead ends the turn with `done_reason:
+     * "length"` on counts that reach the window and go no further — the same
+     * overflow, told rather than counted. Both are read here because which one
+     * the daemon does is its build's to decide, not the caller's.
+     *
+     * `num_predict` is left unset, so `length` can only be the window; the count
+     * is checked anyway, in case a model file sets one.
+     */
+    #hasSpentTheWindow(finishedLine) {
+        if (this.#usedTokens < this.#contextWindow)
+            return false;
+        return asString(finishedLine.done_reason) === "length";
+    }
+}
+const ZERO_TOKENS = tokens(0);
+const connectOllama = async (config, options) => {
+    const endpoint = toEndpoint(config);
+    let downloadedModels;
+    try {
+        downloadedModels = await listModels(endpoint);
+    }
+    catch (cause) {
+        return err({ kind: "unsupported", cause });
+    }
+    if (!downloadedModels.includes(config.model)) {
+        const pullResult = await pullModel(endpoint, config.model, options.reportProgress);
+        if (!pullResult.ok)
+            return pullResult;
+    }
+    return ok(new OllamaModel(config, options));
+};
+export function makeOllamaProvider(config) {
+    const backend = {
+        name: "ollama",
+        modalities: ["text"],
+        tools: true,
+        availability: () => getAvailability(config),
+        connect: (options) => connectOllama(config, options),
+    };
+    return createProvider(backend);
+}
+//# sourceMappingURL=ollama.js.map