@tanstack-ai-browser/prompt-api 0.0.0-stage → 0.2.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 daveycodez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,97 @@
-# Temporary Holding Version
+# @tanstack-ai-browser/prompt-api
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A [TanStack AI](https://tanstack.com/ai) adapter for the browser's built-in language model, through the [Prompt API](https://developer.chrome.com/docs/ai/prompt-api) (`LanguageModel`): Gemini Nano in Chrome, Phi-4-mini in Edge. The browser downloads the model once for every site and runs it on the device; the page downloads nothing and nothing leaves the device.
+
+```sh
+npm install @tanstack-ai-browser/prompt-api @tanstack/ai
+```
+
+```ts
+import { chat } from "@tanstack/ai"
+import { promptApiText } from "@tanstack-ai-browser/prompt-api"
+
+const adapter = promptApiText()
+if ((await adapter.availability()) !== "unavailable") {
+  const stream = chat({ adapter, messages, threadId, tools })
+}
+```
+
+The root has `promptApiText(config?)` (or `promptApiText("text", config?)`: the model is whichever the browser has, so naming it is optional), `PromptApiTextAdapter` and the types `PromptApiConfig`, `PromptApiTextModelOptions`, `Availability` and `DownloadProgressCallback`. It also has the catalog: `PROMPT_API_CHAT_MODELS` (one model, `"text"`: whichever the browser provides) and `PROMPT_API_MODEL_META`, with the types `PromptApiChatModel` and `PromptApiModelMeta`.
+
+The Prompt API is in desktop Chrome and Edge (tested in Chrome 156, where web pages have it without flags), on devices with enough memory, GPU and disk (Chrome asks for 22 GB free and either a GPU with more than 4 GB or 16 GB of RAM). It isn't in Web Workers, and a cross-origin iframe needs `allow="language-model"`.
+
+## The model
+
+- `adapter.availability()` is the browser's own `LanguageModel.availability()` for the adapter's `languages` and `inputs`: `"available"`, `"downloadable"`, `"downloading"` or `"unavailable"` (no Prompt API, or the device can't run it). Give the adapter `inputs: ["image"]` or `["audio"]` if users send them (audio needs a GPU).
+- `adapter.availableInputModalities()` resolves to the kinds of input the browser's model takes here (`"text"`, `"image"`, `"audio"`): each kind `LanguageModel.availability()` doesn't call `"unavailable"` for the adapter's `languages`, asked one kind at a time. `inputModalities` lists what the Prompt API can take; this says what this device can, so offer attachments by it. Empty where there's no model.
+- `adapter.create({ onDownloadProgress?, signal? }?)` downloads the model if it isn't on the device and starts it, with the download's progress, 0 to 1, then 1 once it's ready (the config's `onDownloadProgress` hears it too). Chrome wants a click on the page before it downloads, so call it from the click that turns the model on. `chat()` doesn't need it: its first session does the same. Aborting `signal` aborts the browser's `create()`.
+- `onContextOverflow` in the config is called when a thread's session overflows its context window (the session's `contextoverflow` event); Chrome then drops the oldest messages after the system prompt.
+- `adapter.contextUsage(threadId?)` and `adapter.contextWindow(threadId?)` resolve to the tokens a thread's session holds (the system prompt, the tools and the conversation so far) and the tokens it can hold: the session's own `contextUsage` and `contextWindow`. The thread is `chat()`'s `threadId` (or its `promptCache.key`); left out, the thread used last. `undefined` where the thread keeps no session (see Sessions).
+- Token usage: the Prompt API reports none, so the adapter derives it from the session's own counts and sends it with a reply's end (`RUN_FINISHED`, and a structured answer's `usage`). The prompt's tokens are what the session held before the prompt plus the new messages as the session measures them (`measureContextUsage`); what a thread's kept session held is reported as cached, since it doesn't read it again. The reply's are its text measured the same way, less an empty message's count (the turn markers a session counts around any message), so a step that stops at a tool call is counted too. In Chrome 157 that count equals the number of pieces the reply streams in. Where the browser can't measure, no usage is sent.
+- `adapter.destroy()` closes every thread's session (see Sessions); the browser keeps the model.
+
+`languages` (`["en"]` by default) tells the model which languages to expect in and out; Chrome's model takes `en`, `es`, `ja`, `de` and `fr`. `modelOptions` takes what a session takes, with Chrome's names: `samplingMode` (`"most-predictable"` to `"most-creative"`; on web pages only in Chrome's Sampling Parameters origin trial), and `temperature` and `topK` (extensions only). The Prompt API has no output cap, nucleus sampling, penalties or stop sequences, so `modelOptions` has none. The model doesn't reason, so `chat()` offers no `reasoning` option for it, and a `reasoning` request is ignored.
+
+## Sessions
+
+Each thread (`chat()`'s `threadId`) keeps a session, created with the system prompt (and the tools, see below) as its initial prompt. A step prompts it with only the messages it doesn't hold yet: the user's next message, or a tool's result. If the conversation doesn't go on from what the session holds (an edited or regenerated message, a new system prompt or tool list, other sampling settings), the session is closed and a new one reads the whole conversation. A step stopped at a tool call isn't in the session (Chrome drops a stopped turn whole), so the next step sends it along with the result. Structured output prompts the same session.
+
+A session holds its conversation in the model's memory, about 100 MB for 6,000 tokens in Chrome 156, and only `destroy()` frees it (a session that's just dropped keeps it for a while). So four threads keep a session at most, and the least recently used one closes first.
+
+## Tool calls
+
+Pass tools to `chat({ tools })` as with any adapter; `chat()` runs them. The Prompt API's own tool use doesn't work on web pages yet (in Chrome 156 a session that expects tool calls fails to start, and `tools` alone are never called), so tools go by prompt. The tools are added to the system prompt as JSON, with how to call them:
+
+````
+You are a helpful AI assistant with access to tools.
+
+# Available Tools
+[
+  {
+    "name": "get_weather",
+    "description": "Weather for a city",
+    "parameters": { …the tool's input schema… }
+  }
+]
+
+# Tool Calling Instructions
+Only request one tool call at a time. Wait for tool results before asking for another tool.
+
+To call a tool, output JSON in this exact format inside a ```tool_call code fence:
+
+```tool_call
+{"name": "tool_name", "arguments": {"param1": "value1", "param2": "value2"}}
+```
+
+Tool responses will be provided in ```tool_result fences. Each line contains JSON like:
+```tool_result
+{"id": "call_123", "name": "tool_name", "result": {...}, "error": false}
+```
+Use the `result` payload (and treat `error` as a boolean flag) when continuing the conversation.
+
+Important:
+- Use exact tool and parameter names from the schema above
+- Arguments must be a valid JSON object matching the tool's parameters
+- You can include brief reasoning before or after the tool call
+- If no tool is needed, respond directly without tool_call fences
+````
+
+The reply streams as text until a ```` ```tool_call ```` fence starts (```` ```tool-call ````, `<tool_call>…</tool_call>`, `<|tool_call>…<tool_call|>`, `<|tool_call_start|>…<|tool_call_end|>` and a Python-style `[name(key="value")]` work too); the fence is held back, and the first fence that closes with a call in it ends the reply: the generation stops (text after the fence, in the piece that closed it, is shown after the call), and the step ends with `TOOL_CALL_START`, `TOOL_CALL_ARGS`, `TOOL_CALL_END` and `RUN_FINISHED` `finishReason: "tool_calls"`. One call per step. A call reads `name` and `arguments` (or `parameters`), as an object or a JSON string; Python-style calls and `call:name{key:value}` are read too. A fence without a call is shown as text and the reply goes on, as does an unclosed fence at the end of the reply; other code fences are text.
+
+The call is emitted as written: `chat()` checks the tool's name and arguments, and reports a bad call back to the model as the tool's result. The step stopped at the call isn't in the thread's session (Chrome drops a stopped turn), so the next step sends the call along with its result.
+
+In later steps the model sees its earlier calls as the same fences and the results as a user message with a ```` ```tool_result ```` fence, one line per result: `{"id", "name", "result", "error"}`, where `result` is the tool's JSON output (or its text, or the error message) and `error` is a boolean. `toolChoice: "none"` leaves the tools out; the Prompt API can't force a tool, so `"required"` and a named tool work like `"auto"`.
+
+## Structured output
+
+`chat({ outputSchema })` prompts the thread's session once under the Prompt API's `responseConstraint`, with the schema as given; the result is parsed as JSON and `chat()` validates it. A schema Chrome can't enforce fails with Chrome's error.
+
+## Images and audio
+
+User messages can carry images and audio (TanStack AI's `image` and `audio` parts, as base64 data or a URL); they're loaded into Blobs, and the session is created expecting them. A URL from another site needs CORS. The model can't write images or audio, and it can't read video or documents.
+
+## Errors and aborts
+
+A failure ends the stream with `RUN_ERROR` carrying the error's own message (the Prompt API's `DOMException`s: `NotAllowedError` when Chrome wants a click before the download, `QuotaExceededError` for a conversation too long for the model…). Aborting the request (`abortController` or `request.signal`) stops the generation and ends the stream with `RUN_ERROR` (`code: "aborted"`); the thread keeps its session. No token usage is reported.
+
+**Status:** early (0.0.x); the API may change, and so does the Prompt API. See the [repository README](https://github.com/daveycodez/tanstack-ai-browser#readme).

package/dist/adapter.d.ts

@@ -0,0 +1,160 @@
+import type { AdapterYieldChunk, TextOptions } from "@tanstack/ai";
+import { BaseTextAdapter } from "@tanstack/ai/adapters";
+import type { StructuredOutputOptions, StructuredOutputResult } from "@tanstack/ai/adapters";
+import type { Availability, MediaInput } from "./model.js";
+import type { PromptApiChatModel } from "./model-meta.js";
+import type { DownloadProgressCallback, SamplingMode } from "./types.js";
+export type PromptApiConfig = {
+    /**
+     * Languages the conversation is in, input and output: ["en"] by default.
+     * Chrome's model takes "en", "es", "ja", "de" and "fr", and checks its
+     * safety filters for the output language.
+     */
+    languages?: ReadonlyArray<string>;
+    /**
+     * Kinds of input besides text the conversation will have ("image",
+     * "audio"), for `availability()`: images need more of the device, audio a
+     * GPU. Text only by default. Sessions expect what each conversation has.
+     */
+    inputs?: ReadonlyArray<MediaInput>;
+    /**
+     * The browser's download of its model, 0 to 1, when a call starts it; then
+     * 1 once a session has started. To download ahead of time, see `create`.
+     */
+    onDownloadProgress?: DownloadProgressCallback;
+    /**
+     * Called when a thread's session overflows its context window (the
+     * session's `contextoverflow` event): Chrome then drops the oldest
+     * messages after the system prompt to make room.
+     */
+    onContextOverflow?: (event: Event) => void;
+};
+/** Per-call `modelOptions`: the session's sampling. */
+export type PromptApiTextModelOptions = {
+    /**
+     * Sampling preset, "most-predictable" to "most-creative" ("balanced" by
+     * default). Chrome takes it on web pages only in its Sampling Parameters
+     * origin trial.
+     */
+    samplingMode?: SamplingMode;
+    /** Raw sampling, for extensions (web pages ignore it). Chrome refuses it together with `samplingMode`. */
+    temperature?: number;
+    topK?: number;
+};
+type MetadataByModality = {
+    text: unknown;
+    image: unknown;
+    audio: unknown;
+    video: unknown;
+    document: unknown;
+};
+export declare class PromptApiTextAdapter extends BaseTextAdapter<PromptApiChatModel, PromptApiTextModelOptions, readonly ["text", "image", "audio"], MetadataByModality> {
+    readonly kind: "text";
+    readonly name: "prompt-api";
+    readonly provider: "prompt-api";
+    readonly api: "prompt-api";
+    readonly inputModalities: readonly ["text", "image", "audio"];
+    private readonly languages;
+    private readonly inputs;
+    private readonly onDownloadProgress;
+    private readonly onContextOverflow;
+    /** Each thread's session, least recently used first. */
+    private readonly kept;
+    constructor(config: PromptApiConfig, model: PromptApiChatModel);
+    /**
+     * The browser's own `LanguageModel.availability()` for this adapter's
+     * `languages` and `inputs`: "available" (on the device), "downloadable"
+     * or "downloading" (it downloads when a session starts, or at `create`),
+     * or "unavailable" (no Prompt API, or the device can't run it).
+     */
+    availability(): Promise<Availability>;
+    /**
+     * The kinds of input the browser's model takes here, of the
+     * `inputModalities` the Prompt API has: each kind the browser's
+     * `LanguageModel.availability()` doesn't call "unavailable" for this
+     * adapter's `languages` (images need more of the device, audio a GPU).
+     * Empty where there's no model. Sessions expect what each conversation has.
+     */
+    availableInputModalities(): Promise<Array<"text" | MediaInput>>;
+    /**
+     * Downloads the browser's model if it isn't on the device and starts it,
+     * with the download's progress, 0 to 1, then 1 once it's ready. Resolves
+     * to this adapter; rejects if it can't start (fine to leave unawaited).
+     * `chat()` doesn't need it, but the browser asks for a click on the page
+     * before a download: call it from the click that turns the model on.
+     * Aborting `signal` aborts the browser's `create()`.
+     */
+    create(options?: {
+        onDownloadProgress?: DownloadProgressCallback;
+        signal?: AbortSignal;
+    }): Promise<this>;
+    /**
+     * Tokens a thread's session holds (the session's `contextUsage`): the
+     * system prompt, the tools and the conversation so far. The thread is
+     * `chat()`'s `threadId` (or its `promptCache.key`); left out, the thread
+     * used last. Undefined where the thread keeps no session.
+     */
+    contextUsage(threadId?: string): Promise<number | undefined>;
+    /**
+     * Tokens a thread's session can hold (the session's `contextWindow`), for
+     * the thread as `contextUsage` takes it. Undefined where the thread keeps
+     * no session.
+     */
+    contextWindow(threadId?: string): Promise<number | undefined>;
+    /** A thread's kept session, or the last used thread's; undefined if there's none, or it failed to start. */
+    private keptSession;
+    /** Closes every thread's session. The browser keeps the model. */
+    destroy(): Promise<void>;
+    /** Keeps a thread's session, closing the least recently used past the limit. */
+    private keep;
+    /** Closes a thread's session, unless a step is using it. */
+    private drop;
+    /** A session for this call, holding the system prompt, that reports an overflow. */
+    private session;
+    /**
+     * The step's session and the messages to prompt it with: the thread's
+     * kept session if the conversation goes on from what it holds (only the
+     * new messages), else a new one with the whole conversation. The session
+     * is reserved before anything is awaited, so a second step on a busy
+     * thread gets a session of its own (`kept` undefined), closed after it.
+     * Threads go by `promptCache.key`; retention "none" keeps no session.
+     */
+    private begin;
+    /** Frees a step's session: a kept one stays for the thread unless the step failed. */
+    private end;
+    /**
+     * A step's token counts, which the Prompt API doesn't report, from the
+     * session's own: the tokens it held before the prompt, the prompt as the
+     * session measures it, and the reply measured the same way, less an empty
+     * message's tokens (the turn markers the session counts around any
+     * message). What a kept session held is cached: it isn't read again. A
+     * new session read what it holds (the system prompt) in this step.
+     * Nothing where the browser can't measure.
+     */
+    private usage;
+    chatStream(options: TextOptions<PromptApiTextModelOptions>): AsyncIterable<AdapterYieldChunk>;
+    /**
+     * A JSON answer for `chat({ outputSchema })`: the thread's session
+     * answers the conversation under the Prompt API's `responseConstraint`,
+     * which holds the model to the schema as given. `chat()` validates the
+     * result.
+     */
+    structuredOutput(options: StructuredOutputOptions<PromptApiTextModelOptions>): Promise<StructuredOutputResult<unknown>>;
+}
+/**
+ * A TanStack AI text adapter for the browser's built-in model, through the
+ * Prompt API: Gemini Nano in Chrome, Phi-4-mini in Edge. The model is
+ * whichever the browser has ("text", as other Prompt API libraries name it), so naming it is optional.
+ *
+ * @example
+ * ```ts
+ * const adapter = promptApiText()
+ * if ((await adapter.availability()) !== "unavailable") {
+ *   chat({ adapter, messages, tools })
+ * }
+ * ```
+ */
+export declare function promptApiText(config?: PromptApiConfig): PromptApiTextAdapter;
+export declare function promptApiText(model: PromptApiChatModel, config?: PromptApiConfig): PromptApiTextAdapter;
+export {};
+//# sourceMappingURL=adapter.d.ts.map

package/dist/adapter.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"adapter.d.ts","sourceRoot":"","sources":["../src/adapter.ts"],"names":[],"mappings":"AAeA,OAAO,KAAK,EAAE,iBAAiB,EAAgB,WAAW,EAAoB,MAAM,cAAc,CAAA;AAClG,OAAO,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAA;AACvD,OAAO,KAAK,EAAE,uBAAuB,EAAE,sBAAsB,EAAE,MAAM,uBAAuB,CAAA;AAI5F,OAAO,KAAK,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,YAAY,CAAA;AAC1D,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,iBAAiB,CAAA;AAEzD,OAAO,KAAK,EAAE,wBAAwB,EAA4C,YAAY,EAAE,MAAM,YAAY,CAAA;AAElH,MAAM,MAAM,eAAe,GAAG;IAC5B;;;;OAIG;IACH,SAAS,CAAC,EAAE,aAAa,CAAC,MAAM,CAAC,CAAA;IACjC;;;;OAIG;IACH,MAAM,CAAC,EAAE,aAAa,CAAC,UAAU,CAAC,CAAA;IAClC;;;OAGG;IACH,kBAAkB,CAAC,EAAE,wBAAwB,CAAA;IAC7C;;;;OAIG;IACH,iBAAiB,CAAC,EAAE,CAAC,KAAK,EAAE,KAAK,KAAK,IAAI,CAAA;CAC3C,CAAA;AAED,uDAAuD;AACvD,MAAM,MAAM,yBAAyB,GAAG;IACtC;;;;OAIG;IACH,YAAY,CAAC,EAAE,YAAY,CAAA;IAC3B,0GAA0G;IAC1G,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,IAAI,CAAC,EAAE,MAAM,CAAA;CACd,CAAA;AAED,KAAK,kBAAkB,GAAG;IACxB,IAAI,EAAE,OAAO,CAAA;IACb,KAAK,EAAE,OAAO,CAAA;IACd,KAAK,EAAE,OAAO,CAAA;IACd,KAAK,EAAE,OAAO,CAAA;IACd,QAAQ,EAAE,OAAO,CAAA;CAClB,CAAA;AA2DD,qBAAa,oBAAqB,SAAQ,eAAe,CACvD,kBAAkB,EAClB,yBAAyB,EACzB,SAAS,CAAC,MAAM,EAAE,OAAO,EAAE,OAAO,CAAC,EACnC,kBAAkB,CACnB;IACC,SAAkB,IAAI,EAAG,MAAM,CAAS;IACxC,QAAQ,CAAC,IAAI,EAAG,YAAY,CAAS;IACrC,SAAkB,QAAQ,EAAG,YAAY,CAAS;IAClD,SAAkB,GAAG,EAAG,YAAY,CAAS;IAC7C,SAAkB,eAAe,YAAI,MAAM,EAAE,OAAO,EAAE,OAAO,EAAU;IACvE,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAuB;IACjD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAA2B;IAClD,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAsC;IACzE,OAAO,CAAC,QAAQ,CAAC,iBAAiB,CAAsC;IACxE,wDAAwD;IACxD,OAAO,CAAC,QAAQ,CAAC,IAAI,CAA0B;IAE/C,YAAY,MAAM,EAAE,eAAe,EAAE,KAAK,EAAE,kBAAkB,EAM7D;IAED;;;;;OAKG;IACH,YAAY,IAAI,OAAO,CAAC,YAAY,CAAC,CAEpC;IAED;;;;;;OAMG;IACG,wBAAwB,IAAI,OAAO,CAAC,KAAK,CAAC,MAAM,GAAG,UAAU,CAAC,CAAC,CAQpE;IAED;;;;;;;OAOG;IACH,MAAM,CAAC,OAAO,GAAE;QAAE,kBAAkB,CAAC,EAAE,wBAAwB,CAAC;QAAC,MAAM,CAAC,EAAE,WAAW,CAAA;KAAO,GAAG,OAAO,CAAC,IAAI,CAAC,CAW3G;IAED;;;;;OAKG;IACG,YAAY,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CAEjE;IAED;;;;OAIG;IACG,aAAa,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC,CAElE;IAED,4GAA4G;IAC5G,OAAO,CAAC,WAAW;IAKnB,kEAAkE;IAC5D,OAAO,IAAI,OAAO,CAAC,IAAI,CAAC,CAG7B;IAED,gFAAgF;IAChF,OAAO,CAAC,IAAI;IAWZ,4DAA4D;IAC5D,OAAO,CAAC,IAAI;IAOZ,oFAAoF;YACtE,OAAO;IAqBrB;;;;;;;OAOG;YACW,KAAK;IA8BnB,sFAAsF;IACtF,OAAO,CAAC,GAAG;IAMX;;;;;;;;OAQG;YACW,KAAK;IA0BZ,UAAU,CAAC,OAAO,EAAE,WAAW,CAAC,yBAAyB,CAAC,GAAG,aAAa,CAAC,iBAAiB,CAAC,CA4GnG;IAED;;;;;OAKG;IACG,gBAAgB,CACpB,OAAO,EAAE,uBAAuB,CAAC,yBAAyB,CAAC,GAC1D,OAAO,CAAC,sBAAsB,CAAC,OAAO,CAAC,CAAC,CAoB1C;CACF;AAED;;;;;;;;;;;;GAYG;AACH,wBAAgB,aAAa,CAAC,MAAM,CAAC,EAAE,eAAe,GAAG,oBAAoB,CAAA;AAC7E,wBAAgB,aAAa,CAAC,KAAK,EAAE,kBAAkB,EAAE,MAAM,CAAC,EAAE,eAAe,GAAG,oBAAoB,CAAA"}