@degreesign/chat 0.3.5 → 0.3.7

package/README.md

@@ -15,6 +15,7 @@ package loads one model, streams one reply and reports status.
 - [Why](#why)
 - [Install](#install)
 - [Quick start](#quick-start)
+- [Enable WebGPU](#enable-webgpu)
 - [Public API](#public-api)
 - [Status](#status)
 - [Models](#models)
@@ -83,12 +84,7 @@ call returns one cancel function and starts the model behind the status
 callback.
 
 ```ts
-import {
-  ChatModelId,
-  LlmRole,
-  createChat,
-  preloadModel,
-} from "@degreesign/chat";
+import { LlmRole, createChat, preloadModel } from "@degreesign/chat";
 
 const messages = [
   { role: LlmRole.system, content: "Answer in two sentences." },
@@ -96,39 +92,114 @@ const messages = [
 ];
 
 // Optional: download and load the model before the first send.
-await preloadModel({ model: ChatModelId.qwen3_vl_2b });
+await preloadModel();
 
 const cancel = createChat({
-  model: ChatModelId.qwen3_vl_2b,
   messages,
-  limits: { maxReplyTokens: 320 },
-  reasoning: false,
-  onThinking: (text) => renderThinking(text),
   onResponse: (text) => renderReply(text),
-  onStatus: (snapshot) => renderStatus(snapshot),
+  // onThinking: (text) => renderThinking(text),
+  // onSpeed: (tokensPerSecond) => renderSpeed(tokensPerSecond),
+  // onStatus: (snapshot) => renderStatus(snapshot),
 });
 
 // Later, stop the reply, cancel the load, or free the model.
 cancel();
 ```
 
+Without `model` the package picks the GPU model when the browser provides a
+WebGPU adapter and the CPU model when it does not. `limits`, `reasoning`,
+`onThinking`, `onSpeed` and `onStatus` are optional per call; see the API
+below.
+
 There is no promise and no controller object on `createChat`. Every result
-arrives through the three callbacks, and `onStatus` reports `requesting: false`
-when the reply ends. `preloadModel` is the one promise: it resolves once the
-model is loaded, or `false` when it cannot load.
+arrives through `onResponse`, and `onStatus` reports `requesting: false` when
+the reply ends. `onSpeed` reports the streamed generation speed in tokens per
+second, measured from the first token so prompt processing is excluded.
+`preloadModel` is the one promise: it resolves once the model is loaded, or
+`false` when it cannot load.
 
 Attach images to a user turn with `images: [dataUrl]`. The package sends them
 only to a model with vision support, center-crops each image to a square,
-resizes it to 288 x 288, and caps the number with `limits.maxImages`.
+resizes it to 288 x 288, and caps the number with `limits.maxImages`. The
+WebGPU engine asks LiteRT for the smallest Gemma 4 vision budget (70 tokens,
+630 patches), so it encodes the crop at 384 x 384 instead of upscaling it to
+768 x 768. The CPU engine uses the 288 crop directly.
 
 The package never writes storage, so the host appends the user message before
 the call and appends the assistant message after the last `onResponse`.
 
+## Enable WebGPU
+
+A request without `model` picks `gemma4_e2b` when the browser exposes a WebGPU
+adapter and `qwen3_vl_2b` when it does not. The GPU engine is far faster, so
+check the browser before blaming the package:
+
+```js
+// Run in the console of the page that loads the package
+const adapter = await navigator.gpu?.requestAdapter();
+adapter
+    ? `${adapter.info.vendor} ${adapter.info.architecture}`
+    : `no adapter`;
+```
+
+`null` means the browser has the API but no usable adapter, and the package
+falls back to the CPU model. Chromium browsers report the whole state at
+`chrome://gpu`: `WebGPU` should read `Hardware accelerated` and `Vulkan` should
+read `Enabled`. Firefox lists the detected features at `about:support` under
+`Graphics`.
+
+### Firefox
+
+WebGPU ships enabled on Windows and macOS. On Linux, and on any build that
+reports `navigator.gpu` as undefined:
+
+1. Open `about:config`.
+2. Set `dom.webgpu.enabled` to `true`.
+3. Restart the browser and run the snippet again.
+
+Firefox runs WebGPU on its own wgpu stack, so an adapter can appear even with a
+software driver. Check the renderer in `about:support` when replies feel slow.
+
+### Chrome, Edge and Brave
+
+WebGPU ships enabled on Windows, macOS and ChromeOS since version 113, so there
+is nothing to configure.
+
+On Linux the GPU process only exposes an adapter when ANGLE runs on Vulkan,
+which is not the default. Start the browser with both flags:
+
+```bash
+google-chrome --enable-features=Vulkan --use-angle=vulkan
+microsoft-edge --enable-features=Vulkan --use-angle=vulkan
+brave-browser --enable-features=Vulkan --use-angle=vulkan
+```
+
+Either flag alone leaves `requestAdapter()` returning `null`. To keep them, add
+them to the `Exec` line of the browser entry under
+`~/.local/share/applications`, or to the target of a desktop shortcut.
+
+`--enable-unsafe-webgpu` is a different switch: it only unlocks software
+adapters, so an adapter whose `info.vendor` reads `google` is SwiftShader on the
+CPU, not the GPU. Remove it when measuring speed. Headless Chromium hides
+`navigator.gpu` unless it starts with `--enable-unsafe-webgpu`,
+`--enable-features=Vulkan,WebGPU` and `--use-angle=vulkan`; check the adapter
+there too, because a missing adapter silently selects the CPU model.
+
+### GPU memory and images
+
+The WebGPU engine keeps the model, its KV cache and the vision encoder in GPU
+memory. A card that is already busy can lose the device (`VK_ERROR_DEVICE_LOST`)
+on the first image turn. The package requests the smallest Gemma 4 vision
+budget, 70 tokens and 630 patches, so the encoder runs one small signature and
+the crop is resized to 384 x 384 rather than the full 768 x 768. If the error
+still appears, close other GPU tabs, unload the model, or switch to the CPU
+model. A lost device needs a page or model reload; later calls cannot reuse it.
+
 ## Public API
 
 ```ts
 createChat(input: ChatInput): CancelChat
-preloadModel(input: ChatPreloadInput): Promise<boolean>
+preloadModel(input?: ChatPreloadInput): Promise<boolean>
 ```
 
 Call it once per response. A new call aborts the previous stream. When the
@@ -138,15 +209,22 @@ requested model is already loaded, no download or load happens.
 
 | Field | Description |
 | ----- | ----------- |
-| `model` | `ChatModelId` to use for this request. A different model replaces the loaded one. |
+| `model?` | `ChatModelId` to use; a different model replaces the loaded one. |
 | `messages` | The complete prompt, including system messages and the new user turn. |
-| `limits?` | Partial `LlmLimits`; `maxImages` caps images and `maxThreads` caps CPU threads (default 2). |
+| `limits?` | Partial `LlmLimits`; see the budget defaults below. |
 | `reasoning?` | Whether the model may think first. Defaults to `false`. |
 | `logging?` | Whether package and engine console logs are written. Defaults to `false`. |
 | `onThinking?` | Receives the accumulated reasoning text as it streams. |
 | `onResponse` | Receives the accumulated reply text as it streams. Required. |
+| `onSpeed?` | Receives the current generation speed in tokens per second. |
 | `onStatus?` | Receives a `ChatStatusSnapshot` on every state change. |
 
+Without `model` the package picks the GPU build when the browser provides a
+WebGPU adapter and the CPU build when it does not. The budget defaults are 250
+reply tokens, 128 CPU thinking tokens, 1200 GPU reasoning tokens (320 on the
+CPU), 2 images, 1000 characters per message, 10000 prompt characters and 6 CPU
+threads.
+
 `CancelChat` is `() => void`. Its meaning depends on the current state:
 
 | State | Effect |
@@ -162,7 +240,7 @@ and never starts a second download.
 ### Preload
 
 ```ts
-preloadModel(input: ChatPreloadInput): Promise<boolean>
+preloadModel(input?: ChatPreloadInput): Promise<boolean>
 ```
 
 Call it before the first `createChat` to download and load the model early. It
@@ -176,7 +254,7 @@ running.
 
 | Field | Description |
 | ----- | ----------- |
-| `model` | `ChatModelId` to download and load. |
+| `model?` | `ChatModelId` to download and load. Defaults like `ChatInput.model`. |
 | `limits?` | Partial `LlmLimits`; `maxThreads` applies when the model loads. |
 | `logging?` | Whether package and engine console logs are written. Defaults to `false`. |
 | `onStatus?` | Receives a `ChatStatusSnapshot` on every progress and state change. |
@@ -205,13 +283,16 @@ on failure. A preload owns no cancel function; a new load replaces it.
 The first snapshot fires synchronously inside `createChat`. A final snapshot
 with `requesting: false` marks the end of the call.
 
+A stream that never produces its first chunk ends with an error after 180
+seconds instead of waiting forever. Once a chunk has arrived, 30 seconds of
+silence ends the stream and keeps the text already delivered.
+
 ## Logging
 
 Console output stays off by default. Pass `logging: true` in a call to receive
-the package's censored engine logs and the wllama engine's own logs for that
-call. The flag is per call, so a later call without it goes quiet again.
-LiteRT-LM writes its wasm diagnostics straight to the console and does not
-follow the flag.
+the package's censored engine logs, the wllama engine's own logs and the
+LiteRT-LM wasm diagnostics for that call. The flag is per call, so a later call
+without it goes quiet again.
 
 ## Models
 
@@ -232,6 +313,11 @@ Gemma 4 E2B downloads one `gemma-4-E2B-it.litertlm` file of about 2.6 GB. It
 runs on the WebGPU engine with its vision encoder in the same file, so an
 image prompt does not download a projector.
 
+The WebGPU engine keeps one LiteRT conversation for consecutive turns. When the
+next prompt continues that history, only the new user turn is sent. A changed
+history opens a fresh conversation, and older images become `[image]` notes in
+its preface.
+
 ## Prompt assembly
 
 The `messages` array is the whole prompt. The package bounds it and shapes it
@@ -256,7 +342,9 @@ user:      {next turn}
 - Images are collected from the newest turns first and capped by `maxImages`
   (default 2). Older image parts are dropped and their text stays.
 - Every image part is center-cropped to a square, then resized to 288 x 288.
-  Non-square edges are discarded and smaller images are upscaled.
+  Non-square edges are discarded and smaller images are upscaled. The WebGPU
+  engine is told to use a 70-token vision budget, so it encodes the crop at
+  384 x 384, the smallest Gemma 4 encoder signature.
 - Image payloads do not count against `promptCharBudget`.
 - An image-only turn with a text model reports `Model does not accept images`.
 
@@ -276,7 +364,7 @@ onProgress })` contract and return the runtime object with `model`, `backend`,
 matches the requested model.
 
 The CPU engine derives a raw thread count from the device, then caps it at
-`limits.maxThreads`, default 2. The cap is applied after the raw maximum, so the
+`limits.maxThreads`, default 6. The cap is applied after the raw maximum, so the
 engine never exceeds the device or the caller budget.
 
 The CPU engine needs cross-origin isolation for its threads. Serve the page with
@@ -431,8 +519,9 @@ No. The host passes the full prompt on every call and keeps the reply.
 Yes, on models with vision support. Put image URLs in a user turn's `images`
 array. Qwen3-VL 2B accepts images on the CPU engine and Gemma 4 E2B accepts
 images on the WebGPU engine. Each image is center-cropped to a square and
-resized to 288 x 288 before the engine sees it. A model without vision drops
-images when the turn also has text; an image-only turn reports
+resized to 288 x 288 before the engine sees it. The WebGPU engine encodes that
+crop at 384 x 384 with the smallest Gemma 4 vision budget. A model without
+vision drops images when the turn also has text; an image-only turn reports
 `Model does not accept images`.
 
 ### Does it need a server?

package/dist/browser/chat.min.mjs

@@ -1,2 +1,2 @@
 /*! For license information please see chat.min.mjs.LICENSE.txt */
-var e={};const t={};function a(n){const o=t[n];if(void 0!==o)return o.exports;const r=t[n]={exports:{}};return e[n](r,r.exports,a),r.exports}var n,o,r,i,s,c;a.m=e,a.d=(e,t)=>{for(var n in t)a.o(t,n)&&!a.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},a.f={},a.e=e=>{const t=[];return a.f.j(e,t),Promise.all(t)},a.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),a.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.min.mjs"),924:()=>import("./engine-wllama.min.mjs")},t={792:0},n=e=>{let{__webpack_esm_ids__:n,__webpack_esm_modules__:o,__webpack_esm_runtime__:r}=e;var i,s,c=0;for(i in o)a.o(o,i)&&(a.m[i]=o[i]);for(r&&r(a);c<n.length;c++)s=n[c],a.o(t,s)&&t[s]&&t[s][0](),t[s]=0};a.f.j=(o,r)=>{let i=a.o(t,o)?t[o]:void 0;if(0!==i)if(i)r.push(i[1]);else{let a=e[o]().then(n,e=>{throw 0!==t[o]&&(t[o]=void 0),e});a=Promise.race([a,new Promise(e=>i=t[o]=[e])]),r.push(i[1]=a)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(n||(n={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(o||(o={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(r||(r={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(i||(i={})),function(e){e.download="download",e.load="load"}(s||(s={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(c||(c={}));const l=[{id:c.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:c.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],d=e=>e?l.find(t=>t.id==e):void 0,g=e=>!!e.cpu?.mmproj||!!e.litert?.vision,u=new URL("./../assets/wllama/wllama.wasm",import.meta.url).href,m=new URL("./../assets/wllama/compat/wllama.wasm",import.meta.url).href,f=new URL("./../assets/wllama/compat/wllama.js",import.meta.url).href,w=new URL("./../assets/litert/litertlm_wasm_internal.js",import.meta.url).href,p=(new URL("./../assets/litert/litertlm_wasm_internal.wasm",import.meta.url).href,288),h=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),a=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!a.includes(";base64"))return await(await fetch(e)).arrayBuffer();const n=atob(e.slice(t+1)),o=new Uint8Array(n.length);for(let e=0;e<n.length;e++)o[e]=n.charCodeAt(e);return o.buffer},y=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),a=await createImageBitmap(await t.blob()),n=Math.min(a.width,a.height),o=Math.floor((a.width-n)/2),r=Math.floor((a.height-n)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const a=document.createElement("canvas");return a.width=e,a.height=t,a})(p,p),s=i?i.getContext("2d"):null;if(!i||!s||n<1)return a.close(),e;if(s.drawImage(a,o,r,n,n,0,0,p,p),a.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const a=new FileReader;a.onerror=()=>t(""),a.onload=()=>t(`${a.result}`),a.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},b={maxReplyTokens:200,maxThinkingTokens:64,maxReasoningTokens:150,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:2},v=(e,t,a)=>"number"==typeof e&&Number.isFinite(e)&&e>=a?Math.floor(e):t,k=(e={})=>{const t=b;return{maxReplyTokens:v(e.maxReplyTokens,t.maxReplyTokens,1),maxThinkingTokens:v(e.maxThinkingTokens,t.maxThinkingTokens,1),maxReasoningTokens:v(e.maxReasoningTokens,t.maxReasoningTokens,1),maxImages:v(e.maxImages,t.maxImages,0),maxContentLength:v(e.maxContentLength,t.maxContentLength,1),promptCharBudget:v(e.promptCharBudget,t.promptCharBudget,1),maxThreads:v(e.maxThreads,t.maxThreads,1)}},x=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),_=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,M=e=>Number(e.headers.get("content-length"))||0,E=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),C=["<|channel>thought\n","<|channel>thought","..."],T=["<channel|>","..."],L=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,R=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,P=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const a=[];for(const n of[...R(e),{type:"text",text:"\n"},...R(t)]){const e=a[a.length-1];"text"==n.type&&"text"==e?.type?e.text+=n.text:a.push(n)}return a},$=async({model:e,logging:t,onReason:a,onText:n,start:o,stopped:r})=>{let i="",s="",c="";const l=((e,t)=>{let a="",n=!1;const o=(e,t)=>{let a=-1,n="";for(const o of t){const t=e.indexOf(o);t>=0&&(a<0||t<a)&&(a=t,n=o)}return{at:a,tag:n}},r=(e,t)=>{let a="";for(const n of t)for(let t=Math.min(e.length,n.length-1);t>a.length;t--)if(e.endsWith(n.slice(0,t))){a=e.slice(e.length-t);break}return a};return{feed:i=>{let s=a+i;for(a="";s;){if(n){const{at:t,tag:i}=o(s,T);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),n=!1;continue}const c=r(s,T);e(s.slice(0,s.length-c.length)),a=c;break}{const{at:e,tag:i}=o(s,C);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),n=!0;continue}const c=r(s,C);t(s.slice(0,s.length-c.length)),a=c;break}}},flush:()=>{a&&(n?e(a):t(a),a="")}}})(e=>{s+=e,a(e)},e=>{i+=e,n(e)});try{const e=await o();for await(const t of e){if(r())break;const e=t?.choices?.[0],n=e?.delta,o="string"==typeof n?.reasoning_content?n.reasoning_content:"string"==typeof n?.reasoning?n.reasoning:"";e?.finish_reason&&(c=e.finish_reason),o&&(s+=o,a(o)),n?.content&&l.feed(n.content)}l.flush()}catch(e){t&&console.log("Local chat failed",e)}const d=r()?void 0:i||void 0;return t&&(console.log("Local chat reply",e.id,d?"****":""),console.log("Local chat finish",e.id,c||"none",i.length,s.length)),d},j=async({model:e,params:t,start:a,stopped:o})=>{const r=await(async e=>{const t=[];for(const a of e){if("string"==typeof a.content){t.push(a);continue}const e=[];for(const t of a.content)e.push("image"==t.type?{type:"image",url:await y(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=b)=>{const a=t.maxContentLength,o=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=n.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=o.length-1;e>=0;e--){const l=o[e],d=("string"==typeof l.content?l.content:"").slice(0,a),g=l.role==n.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==n.system?r.unshift(d):l.role==n.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:L(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=P(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,r),await $({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,start:()=>a(r),stopped:o})},W=33554432,z=new Map;let B=!1;const S=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let a=0;for(let t=0;t<e.length;t++)a=31*a+e.charCodeAt(t)>>>0;return`${a.toString(16)}-${t}`},q=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},A=async()=>{if(!B)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else B=!0},O=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},N=async e=>{if(e)try{const t=await e.getFile(),a=JSON.parse(await t.text());return{total:Number(a.total)||0,etag:`${a.etag||""}`,complete:!0===a.complete}}catch{return}},U=async(e,t)=>{try{const a=await e.getFileHandle("meta.json",{create:!0}),n=await a.createWritable();await n.write(JSON.stringify(t)),await n.close()}catch{}},F=async function*(e){for await(const t of e)yield t},I=async e=>{try{for await(const[t,a]of F(e))"file"==a.kind&&q(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},D=async(e,t)=>{try{const a=await(await e.getFileHandle(t)).getFile(),n=await N(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!n)return;return{file:a,size:a.size,total:n.total,etag:n.etag,complete:n.total?a.size>=n.total:!0===n.complete}}catch{return}},H=async(e,t)=>{const a=await O(e,t);if(a){const e=await(async e=>await N(await e.getFileHandle("meta.json").catch(()=>{})))(a),t=await(async e=>{const t=[];try{for await(const[a,n]of F(e)){const e=q(a);if("file"!=n.kind||e<0)continue;const o=await n.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const a=(e=>{const t=[];let a=0;for(const n of[...e].sort((e,t)=>e.start-t.start))if(n.size){if(n.start!=a)break;t.push(n),a+=n.size}return t})(t.map(e=>e.part)),n=new Set(a.map(e=>e.start)),o=t.filter(e=>n.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:a.reduce((e,t)=>e+t.size,0)}})(a),n=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:n&&o>n?r.slice(0,n):r,size:o,total:n,etag:e?.etag||"",complete:n?o>=n:!0===e?.complete}}return await D(e,t)},Q=(e,t)=>{const a=e.headers.get("content-range"),n=/bytes (\d+)-/.exec(a||""),o=/\/(\d+)\s*$/.exec(a||""),r=e.headers.get("etag")||"";if(206==e.status){const e=n?Number(n[1]):t.size,a=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&a&&t.total!=a)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:a||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:M(e)||t.total,etag:r||t.etag}},G=async({dir:e,start:t,skip:a,total:n,body:o,onProgress:r,onChunk:i})=>{let c,l=t,d=0,g=a;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});c=await t.createWritable(),d=0},m=async()=>{const e=c;c=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:a,value:o}=await e.read();if(a)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){c||await u();const e=Math.min(W-d,o.byteLength-f);await c.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==W&&await m()}l>t&&r(n?l/n:0,s.download)}await m()}catch(e){throw await m(),e}return l},V=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),J=e=>{const t=async()=>{const t=await A();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const a=await t();return a?await H(a,S(e)):void 0},download:async({url:a,signal:n,onProgress:o})=>{const r=`${e}|${a}`,i=z.get(r);if(i)return await i;const c=await t();if(!c)return;const l=(async(e,t,a,n)=>{const o=S(t),r=await D(e,o);if(r?.complete)return r;let i=await O(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let c=0,l=0;for(;;){if(a?.aborted)throw new Error("Cancelled");const r=await H(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>c)c=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&n(d/g,s.download);const m=g?Math.min(d+W,g)-1:d+W-1,f=new AbortController,w=()=>f.abort(),p={size:d,total:g,etag:u};let h,y=setTimeout(w,3e4);a?.addEventListener("abort",w,{once:!0});try{if(h=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==h.status){if(await(h.body?.cancel().catch(()=>{})),g&&d>=g){await U(i,{total:g,etag:u,complete:!0});continue}await I(i),await U(i,{total:0,etag:"",complete:!1});continue}if(!h.ok||!h.body)throw await(h.body?.cancel().catch(()=>{})),new Error("Model download failed");const a=Q(h,p);if(!a){await h.body.cancel().catch(()=>{}),await I(i),await U(i,{total:0,etag:"",complete:!1});continue}await U(i,{total:a.total,etag:a.etag,complete:!1});const r=0==a.start?d:0,s=await G({dir:i,start:a.start+r,skip:r,total:a.total,body:h.body,onProgress:n,onChunk:()=>{clearTimeout(y),y=setTimeout(w,3e4)}});if(!a.total){await U(i,{total:s,etag:a.etag,complete:!0});const t=await H(e,o);if(t)return t}}catch(e){if(a?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),a?.removeEventListener("abort",w)}}})(c,a,n,o).finally(()=>{z.delete(r)});return z.set(r,l),await l},remove:async e=>{const a=await t();a&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(a,S(e))},purge:async()=>{const t=await A();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},K="litertlm-models",X=J("litert-store"),Y={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0},Z=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(K)}catch{return}},ee=new Map,te=(e,t,a)=>{let n=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{n+=e.byteLength,a(t?n/t:0,s.download),o.enqueue(e)}}))},ae=async e=>{const t=[];for(const a of e){if("string"==typeof a.content){t.push({role:a.role,content:a.content});continue}const e=[];for(const t of a.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await h(t.url)});t.push({role:"user",content:e})}return t},ne=async({engine:e,model:t,mod:a,params:n})=>await j({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async o=>{if(n.signal?.aborted)throw new Error("Cancelled");if(E(o)&&!t.litert?.vision)throw new Error("Model does not accept images");return async function*({engine:e,mod:t,params:a,messages:n}){const o=n[n.length-1],r=x(a.limits,a.reasoning);if("user"!=o?.role)throw new Error("No user turn");const i=await ae(n.slice(0,-1)),s=(await ae([o]))[0];if(!s)throw new Error("Empty prompt");const c=await e.createConversation({sessionConfig:{maxOutputTokens:r.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:r.temperature,k:40,p:.95}},preface:{...i.length?{messages:i}:{},extra_context:{enable_thinking:a.reasoning}}});let l;const d=()=>{l&&l.cancel().catch(()=>{});try{c.cancel()}catch{}};a.signal?.addEventListener("abort",d,{once:!0});try{if(a.signal?.aborted)return;const e=c.sendMessageStreaming(s).getReader();for(l=e;;){const{done:t,value:a}=await e.read();if(t)break;const n={};let o="string"==typeof a.content?a.content:"";if(Array.isArray(a.content))for(const e of a.content)"text"==e.type&&(o+=e.text);o&&(n.content=o);const r=a.channels?.thought;r&&(n.reasoning=r),(n.content||n.reasoning)&&(yield{choices:[{delta:n}]})}}finally{a.signal?.removeEventListener("abort",d);try{await c.delete()}catch{}}}({engine:e,mod:a,params:n,messages:o})}}),oe={backend:i.gpu,startLLM:async({model:e,limits:t,onProgress:n,signal:o,logging:r})=>{const c=e.litert;if(!c)throw new Error("Model has no LiteRT artifact");const l=await a.e(520).then(()=>a(507));if(await l.getOrLoadGlobalLiteRtLm(w),o?.aborted)throw new Error("Cancelled");const d=_(c.repo,c.file),g=await Z();let u;const m=g?await g.match(d):void 0;if(m?.body)u=m.body;else{const e=await X.stored(d);e?.complete&&(u=e.file.stream())}if(!u){const e=await X.download({url:d,signal:o,onProgress:n}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!V(e))throw e;r&&console.log("Local model store failed",e)});e&&(u=e.file.stream())}if(!u&&g)try{await(async(e,t,a,n)=>{const o=ee.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:a});if(!o.ok||!o.body)throw new Error("Model download failed");const r=M(o),i=te(o.body,r,n);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{ee.delete(e)});return ee.set(e,r),await r})(d,g,o,n);const e=await g.match(d);e?.body&&(u=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(u||(u=await(async(e,t,a)=>{const n=await fetch(e,{signal:t});if(!n.ok||!n.body)throw new Error("Model download failed");return te(n.body,M(n),a)})(d,o,n)),o?.aborted)throw new Error("Cancelled");n(1,s.load);const f=await l.Engine.create({model:u,mainExecutorSettings:{maxNumTokens:4096,...c.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:Y},...c.vision?{visionBackend:l.Backend.GPU}:{}});if(o?.aborted)throw await f.delete(),new Error("Cancelled");return{model:e,backend:i.gpu,contextSize:4096,chat:t=>ne({engine:f,model:e,mod:l,params:t}),unload:async()=>{await f.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const a=_(t.repo,t.file),[n,o]=await Promise.all([Z(),X.stored(a)]);if(o?.complete)return!0;try{return!(!n||!await n.match(a))}catch{return!1}},hasCache:async()=>{try{for(const e of l){const t=e.litert;if(!t)continue;if(await X.stored(_(t.repo,t.file)))return!0}const e=await Z();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const a=_(t.repo,t.file),n=await Z();await X.remove(a),n&&await n.delete(a)}catch{}},purge:async()=>{try{await X.purge(),"undefined"!=typeof caches&&await caches.delete(K)}catch{}}},re=J("wllama-store");let ie,se,ce="",le=!1;const de=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ge=e=>{const t={wasm:u,compatWasm:m,compatWorker:f};return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},ue=e=>{const t=e=>(...t)=>{le&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},me=async()=>{if(ie)return ie;const e=await a.e(924).then(()=>a(577)),t=se||ge(),n=new e.Wllama({default:t.wasm},{logger:ue(e.LoggerWithoutDebug)});return se||(se=t,ce=de(t)),n.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),ie=n,n},fe=async()=>{const e=ie;ie=void 0;try{await(e?.exit())}catch{}},we=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),pe=e=>{const t=e.cpu;return t?[{name:t.file,url:_(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:_(t.repo,t.mmproj.file)}]:[]]:[]},he=async e=>{const t=pe(e);if(!t.length)return!1;for(const e of t){const t=await re.stored(e.url);if(!t?.complete)return!1}return!0},ye=async(e,t)=>{const a=await e.cacheManager.list();return pe(t).every(e=>a.some(t=>we(t,e.name)))},be=async({model:e,signal:t,onProgress:a})=>{const n=pe(e),o=[];for(let e=0;e<n.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await re.download({url:n[e].url,signal:t,onProgress:(t,o)=>a((e+t)/n.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},ve=async e=>{const t=[];for(const a of e){if("string"==typeof a.content){"system"==a.role?t.push({role:"system",content:a.content}):"assistant"==a.role?t.push({role:"assistant",content:a.content}):t.push({role:"user",content:a.content});continue}const e=[];for(const t of a.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await h(t.url)});t.push({role:"user",content:e})}return t},ke={backend:i.cpu,startLLM:async({model:e,limits:t,onProgress:a,signal:n,logging:o,assets:r})=>{le=!0===o;const c=e.cpu;if(!c)throw new Error("Model has no CPU artifact");const l=ge(r),d=de(l);ie&&d!=ce&&await fe(),se=l,ce=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await me(),w=await he(e),p=c.mmproj?.file,h=_(c.repo,c.file),y=p?_(c.repo,p):"";if(n?.aborted)throw await fe(),new Error("Cancelled");if(!w&&p){const e=await f.cacheManager.list(),t=e.some(e=>we(e,c.file)),a=e.some(e=>we(e,p));t&&!a&&await f.cacheManager.deleteMany(e=>we(e,c.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=w?await be({model:e,signal:n,onProgress:a}):void 0;t||w||await ye(f,e).catch(()=>!1)||(t=await be({model:e,signal:n,onProgress:a}).catch(e=>{if(n?.aborted)throw new Error("Cancelled");if(!V(e))throw e;o&&console.log("Local model store failed",e)})),t?(a(1,s.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:h,mmprojUrl:y}:h,{...b,useCache:!0,signal:n,progressCallback:({loaded:e,total:t})=>{const n=t?e/t:0;a(n,n>=1?s.load:s.download)}})}catch(e){throw await fe(),e}if(n?.aborted)throw await fe(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await fe(),new Error("Failed to create model context");return{model:e,backend:i.cpu,contextSize:8192,chat:t=>(async(e,t,a)=>{if(le=!0===a.logging,e.isModelLoaded())return await j({model:t,params:a,stopped:()=>!!a.signal?.aborted,start:async n=>{if(E(n)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await ve(n),...x(a.limits,a.reasoning),abortSignal:a.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:a.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(ie==f)await fe();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await he(e)||await ye(await me(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of l.flatMap(e=>pe(e)))if(await re.stored(e.url))return!0;const e=await me();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=pe(e);try{for(const e of t)await re.remove(e.url)}catch{}try{const e=await me();await e.cacheManager.deleteMany(e=>t.some(t=>we(e,t.name)))}catch{}},purge:async()=>{try{await re.purge()}catch{}try{const e=await me();await e.cacheManager.clear()}catch{}}},xe=((e={})=>{const t=e.engines?.cpu||ke,a=e.engines?.litert||oe;let n,o,c,d,g,u,m,f=i.none,w=r.idle,p=[],h=0,y={percent:0,phase:s.download};const b=n=>{if(e.backend!=i.none)return e.backend==i.cpu?n.cpu?t:void 0:e.backend==i.gpu?n.litert?a:void 0:n.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?a:!n.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||s.download};for(const e of[...p])e.report(y.percent,y.phase)},k=e=>{w=e},x=()=>{h++,u?.abort(),u=void 0,g=void 0,w==r.downloading&&k(r.idle)},_=(e,t)=>{const a=d;return a?new Promise(n=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),n(e))}},i=()=>{(e=>{p=p.filter(t=>t!=e),queueMicrotask(()=>{!p.length&&d&&x()})})(r),r.settle(!1)};p.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),a.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>w==r.ready&&!!n&&!!o;return{load:async({model:e,limits:t,assets:a,logging:i,signal:m,onProgress:w})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(d&&g==e.id)return w(y.percent,y.phase),await _(w,m);d&&(x(),p=[],await d.catch(()=>!1));const E=++h,C=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const d=b(e);if(E!=h)return!1;if(!d)return k(r.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=c,a=n;n=void 0,o=void 0,c=void 0,await(a?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const a of l)a.id!=e.id&&b(a)==t&&await t.remove(a)})(e,d),E!=h)return!1;const g=await d.cached(e);if(E!=h)return!1;k(r.downloading),v(0,s.download),g&&v(1,s.load);const m=new AbortController;u=m;try{const l=await d.startLLM({model:e,limits:t,logging:i,signal:m.signal,assets:a,onProgress:(e,t)=>{E==h&&v(e,t)}});return E!=h||m.signal.aborted?(await l.unload(),!1):(n=l,o=e,c=d,f=d.backend,k(r.ready),v(1,s.load),!0)}catch(e){return E!=h||(n=void 0,o=void 0,c=void 0,i&&console.log("Local model failed",e),k(r.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{d==C&&(d=void 0,g=void 0,p=[])});return d=C,g=e.id,await _(w,m)},chat:async e=>{if(!M()||!n)return;for(;m;)await m.catch(()=>{});if(!M()||!n)return;const t=n.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=n;x(),n=void 0,o=void 0,c=void 0,k(r.idle),await(e?.unload())},ready:M,loading:()=>!M()&&w==r.downloading,status:()=>w,backend:()=>f,model:()=>o}})(),_e=(e,t,a)=>{const n=e.status(),i=d(t);return{availability:n==r.ready?o.online:n==r.error||n==r.unsupported?o.offline:o.away,status:n,backend:e.backend(),supportsImages:!!i&&g(i),ready:e.ready(),loading:e.loading(),requesting:a.requesting,phase:a.phase,percent:a.percent,...a.error?{error:a.error}:{}}},Me=(e=>{let t,a=0;const o=t=>t.input.onStatus?.(_e(e,t.input.model,t));return i=>{t?.controller.abort();const c={token:++a,input:i,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?s.load:s.download,requesting:!1,cancelled:!1};return t=c,o(c),(async t=>{const{input:r}=t,i=!0===r.logging,c=d(r.model),l=r.messages.some(e=>e.role!=n.system&&"string"==typeof e.content&&!!e.content.trim()),u=r.messages.some(e=>e.role==n.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),m=l||u;if(!c)return t.error="Unknown model",void o(t);if(!m)return void o(t);if(u&&!l&&!g(c))return t.error="Model does not accept images",void o(t);const f=k(r.limits),w=await e.load({model:c,limits:f,logging:i,signal:t.controller.signal,onProgress:(e,n)=>{t.token==a&&(t.percent=Math.round(100*e),t.phase=n,o(t))}}).catch(e=>(t.token==a&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=a)return;if(t.cancelled)return;if(!w)return t.error||(t.error="Model not ready"),void o(t);const p=g(c),h=u&&!p?r.messages.map(({images:e,...t})=>t):r.messages;if(u&&!p&&!l)return t.error="Model does not accept images",void o(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=s.load,o(t);let y,b="",v="";try{y=await e.chat({messages:h,limits:f,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=a||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=a||t.cancelled||(v+=e,r.onResponse?.(v))},signal:t.controller.signal})}catch(e){if(t.token!=a)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void o(t)}t.token==a&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),o(t))})(c),()=>(t=>{t.token==a&&(t.cancelled=!0,t.controller.abort(),e.status()!=r.ready?queueMicrotask(()=>o(t)):e.unload().then(()=>o(t)))})(c)}})(xe),Ee=(Ce=xe,async e=>{const t=d(e.model),a=t=>e.onStatus?.(_e(Ce,e.model,t));if(!t)return a({requesting:!1,phase:s.download,percent:0,error:"Unknown model"}),!1;let n,o=0,r=s.download;const i=Ce.ready()&&Ce.model()?.id==t.id;return await Ce.load({model:t,limits:k(e.limits),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,a({requesting:!1,phase:r,percent:o})}}).catch(e=>(n=`${e?.message||`${e}`}`,!1))?(i&&a({requesting:!1,phase:s.load,percent:100}),!0):(a({requesting:!1,phase:r,percent:o,error:n||"Model not ready"}),!1)});var Ce;export{c as ChatModelId,o as ChatStatus,i as LlmBackend,s as LlmPhase,n as LlmRole,r as LlmStatus,Me as createChat,Ee as preloadModel};
+var e={};const t={};function n(a){const o=t[a];if(void 0!==o)return o.exports;const r=t[a]={exports:{}};return e[a](r,r.exports,n),r.exports}var a,o,r,i,s,c;n.m=e,n.d=(e,t)=>{for(var a in t)n.o(t,a)&&!n.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},n.f={},n.e=e=>{const t=[];return n.f.j(e,t),Promise.all(t)},n.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),n.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.min.mjs"),924:()=>import("./engine-wllama.min.mjs")},t={792:0},a=e=>{let{__webpack_esm_ids__:a,__webpack_esm_modules__:o,__webpack_esm_runtime__:r}=e;var i,s,c=0;for(i in o)n.o(o,i)&&(n.m[i]=o[i]);for(r&&r(n);c<a.length;c++)s=a[c],n.o(t,s)&&t[s]&&t[s][0](),t[s]=0};n.f.j=(o,r)=>{let i=n.o(t,o)?t[o]:void 0;if(0!==i)if(i)r.push(i[1]);else{let n=e[o]().then(a,e=>{throw 0!==t[o]&&(t[o]=void 0),e});n=Promise.race([n,new Promise(e=>i=t[o]=[e])]),r.push(i[1]=n)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(a||(a={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(o||(o={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(r||(r={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(i||(i={})),function(e){e.download="download",e.load="load"}(s||(s={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(c||(c={}));const l=[{id:c.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:c.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],d=e=>e?l.find(t=>t.id==e):void 0,g=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?l.find(e=>!!e.litert):l.find(e=>!!e.cpu))?.id,u=async e=>d(e||await g()),m=e=>!!e.cpu?.mmproj||!!e.litert?.vision,f=new URL("./../assets/wllama/wllama.wasm",import.meta.url).href,p=new URL("./../assets/wllama/compat/wllama.wasm",import.meta.url).href,w=new URL("./../assets/wllama/compat/wllama.js",import.meta.url).href,h=new URL("./../assets/litert/litertlm_wasm_internal.js",import.meta.url).href,y=(new URL("./../assets/litert/litertlm_wasm_internal.wasm",import.meta.url).href,288),b=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},v=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(y,y),s=i?i.getContext("2d"):null;if(!i||!s||a<1)return n.close(),e;if(s.drawImage(n,o,r,a,a,0,0,y,y),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},x={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},k=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,_=(e={},t=!1)=>{const n=x;return{maxReplyTokens:k(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:k(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:k(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:k(e.maxImages,n.maxImages,0),maxContentLength:k(e.maxContentLength,n.maxContentLength,1),promptCharBudget:k(e.promptCharBudget,n.promptCharBudget,1),maxThreads:k(e.maxThreads,n.maxThreads,1)}},M=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),T=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,C=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),L=["<|channel>thought\n","<|channel>thought","..."],P=["<channel|>","..."],R=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,$=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,S=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...$(e),{type:"text",text:"\n"},...$(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class j extends Error{}const W=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new j(n)),t)})])}finally{a&&clearTimeout(a)}},q=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},z=async({model:e,logging:t,onReason:n,onText:a,onSpeed:o,start:r,stopped:i,firstTimeoutMs:s=18e4,idleTimeoutMs:c=3e4,onStall:l})=>{let d="",g="",u="",m=!1,f=0,p=0;const w=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:i=>{let s=n+i;for(n="";s;){if(a){const{at:t,tag:i}=o(s,P);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),a=!1;continue}const c=r(s,P);e(s.slice(0,s.length-c.length)),n=c;break}{const{at:e,tag:i}=o(s,L);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),a=!0;continue}const c=r(s,L);t(s.slice(0,s.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{g+=e,n(e)},e=>{d+=e,a(e)});try{const e=(await W(r(),s,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(i()){q(e);break}let a;try{a=await W(e.next(),m?c:s,m?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof j&&(d||g)){l?.(),q(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;m=!0;const r=a.value?.choices?.[0],h=r?.delta,y="string"==typeof h?.reasoning_content?h.reasoning_content:"string"==typeof h?.reasoning?h.reasoning:"";if(r?.finish_reason&&(u=r.finish_reason),y&&(g+=y,n(y)),h?.content&&w.feed(h.content),y||h?.content){const e=performance.now();f||(p=e),f+=1,f>1&&e>p&&o?.(1e3*f/(e-p))}}w.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof j&&!i())throw e}const h=i()?void 0:d||void 0;return t&&(console.log("Local chat reply",e.id,h?"****":""),console.log("Local chat finish",e.id,u||"none",d.length,g.length)),h},B=async({model:e,params:t,start:n,stopped:o,onStall:r})=>{const i=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await v(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=x)=>{const n=t.maxContentLength,o=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=a.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=o.length-1;e>=0;e--){const l=o[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==a.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==a.system?r.unshift(d):l.role==a.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:R(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=S(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,i),await z({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,onSpeed:t.onSpeed,start:()=>n(i),stopped:o,onStall:r})},I=33554432,A=new Map;let O=!1;const N=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},U=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},F=async()=>{if(!O)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else O=!0},D=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},H=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},Q=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},V=async function*(e){for await(const t of e)yield t},G=async e=>{try{for await(const[t,n]of V(e))"file"==n.kind&&U(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},J=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await H(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},K=async(e,t)=>{const n=await D(e,t);if(n){const e=await(async e=>await H(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of V(e)){const e=U(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await J(e,t)},X=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},Y=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:i})=>{let c,l=t,d=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});c=await t.createWritable(),d=0},m=async()=>{const e=c;c=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){c||await u();const e=Math.min(I-d,o.byteLength-f);await c.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==I&&await m()}l>t&&r(a?l/a:0,s.download)}await m()}catch(e){throw await m(),e}return l},Z=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),ee=e=>{const t=async()=>{const t=await F();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await K(n,N(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,i=A.get(r);if(i)return await i;const c=await t();if(!c)return;const l=(async(e,t,n,a)=>{const o=N(t),r=await J(e,o);if(r?.complete)return r;let i=await D(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let c=0,l=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await K(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>c)c=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&a(d/g,s.download);const m=g?Math.min(d+I,g)-1:d+I-1,f=new AbortController,p=()=>f.abort(),w={size:d,total:g,etag:u};let h,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(h=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==h.status){if(await(h.body?.cancel().catch(()=>{})),g&&d>=g){await Q(i,{total:g,etag:u,complete:!0});continue}await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}if(!h.ok||!h.body)throw await(h.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=X(h,w);if(!n){await h.body.cancel().catch(()=>{}),await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}await Q(i,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?d:0,s=await Y({dir:i,start:n.start+r,skip:r,total:n.total,body:h.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await Q(i,{total:s,etag:n.etag,complete:!0});const t=await K(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(c,n,a,o).finally(()=>{A.delete(r)});return A.set(r,l),await l},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,N(e))},purge:async()=>{const t=await F();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},te="litertlm-models",ne=ee("litert-store"),ae={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0};let oe=!1;const re=(...e)=>{oe&&console.log(...e)},ie=(...e)=>{oe&&console.error(...e)},se=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(te)}catch{return}},ce=new Map,le=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,s.download),o.enqueue(e)}}))},de=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await b(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ge=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},ue=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],i=M(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const s=a.slice(0,-1),c=(await de([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==i.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ge(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const i=o.length-r.length;for(let e=0;e<r.length;e++)if(!ge(o[i+e],r[e]))return!1;return!0})(o.sent,s);let d;if(l&&o.conversation)d=o.conversation;else{const a=await de(s,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:i.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:i.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=i.max_tokens,o.sent=[...s]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},me={backend:i.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:o,logging:r})=>{oe=!0===r;const c=e.litert;if(!c)throw new Error("Model has no LiteRT artifact");const l=await n.e(520).then(()=>n(507)),d=globalThis,g=d.Module;d.Module={print:re,printErr:ie};try{await l.getOrLoadGlobalLiteRtLm(h)}finally{d.Module=g}if(o?.aborted)throw new Error("Cancelled");const u=T(c.repo,c.file),m=await se();let f;const p=m?await m.match(u):void 0;if(p?.body)f=p.body;else{const e=await ne.stored(u);e?.complete&&(f=e.file.stream())}if(!f){const e=await ne.download({url:u,signal:o,onProgress:a}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;r&&console.log("Local model store failed",e)});e&&(f=e.file.stream())}if(!f&&m)try{await(async(e,t,n,a)=>{const o=ce.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),i=le(o.body,r,a);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{ce.delete(e)});return ce.set(e,r),await r})(u,m,o,a);const e=await m.match(u);e?.body&&(f=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(f||(f=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return le(a.body,E(a),n)})(u,o,a)),o?.aborted)throw new Error("Cancelled");a(1,s.load);const w=await l.Engine.create((({model:e,limits:t,source:n,mod:a})=>({model:n,mainExecutorSettings:{maxNumTokens:4096,...e.litert?.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:ae},...e.litert?.vision?{visionBackend:a.Backend.GPU,maxVisionTokensPerImage:70,minVisionTokensPerImage:70}:{}}))({model:e,limits:t,source:f,mod:l}));if(o?.aborted)throw await w.delete(),new Error("Cancelled");const y={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:i.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{oe=!0===a.logging;const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await B({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(C(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return ue({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:w,model:e,mod:l,params:t,session:y}),unload:async()=>{y.conversation=void 0,y.sent=[],await w.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=T(t.repo,t.file),[a,o]=await Promise.all([se(),ne.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of l){const t=e.litert;if(!t)continue;if(await ne.stored(T(t.repo,t.file)))return!0}const e=await se();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=T(t.repo,t.file),a=await se();await ne.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ne.purge(),"undefined"!=typeof caches&&await caches.delete(te)}catch{}}},fe=ee("wllama-store");let pe,we,he="",ye=!1;const be=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ve=e=>{const t={wasm:f,compatWasm:p,compatWorker:w};return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},xe=e=>{const t=e=>(...t)=>{ye&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},ke=async()=>{if(pe)return pe;const e=await n.e(924).then(()=>n(577)),t=we||ve(),a=new e.Wllama({default:t.wasm},{logger:xe(e.LoggerWithoutDebug)});return we||(we=t,he=be(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),pe=a,a},_e=async()=>{const e=pe;pe=void 0;try{await(e?.exit())}catch{}},Me=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),Te=e=>{const t=e.cpu;return t?[{name:t.file,url:T(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:T(t.repo,t.mmproj.file)}]:[]]:[]},Ee=async e=>{const t=Te(e);if(!t.length)return!1;for(const e of t){const t=await fe.stored(e.url);if(!t?.complete)return!1}return!0},Ce=async(e,t)=>{const n=await e.cacheManager.list();return Te(t).every(e=>n.some(t=>Me(t,e.name)))},Le=async({model:e,signal:t,onProgress:n})=>{const a=Te(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await fe.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Pe=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await b(t.url)});t.push({role:"user",content:e})}return t},Re={backend:i.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{ye=!0===o;const c=e.cpu;if(!c)throw new Error("Model has no CPU artifact");const l=ve(r),d=be(l);pe&&d!=he&&await _e(),we=l,he=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await ke(),p=await Ee(e),w=c.mmproj?.file,h=T(c.repo,c.file),y=w?T(c.repo,w):"";if(a?.aborted)throw await _e(),new Error("Cancelled");if(!p&&w){const e=await f.cacheManager.list(),t=e.some(e=>Me(e,c.file)),n=e.some(e=>Me(e,w));t&&!n&&await f.cacheManager.deleteMany(e=>Me(e,c.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Le({model:e,signal:a,onProgress:n}):void 0;t||p||await Ce(f,e).catch(()=>!1)||(t=await Le({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,s.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:h,mmprojUrl:y}:h,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?s.load:s.download)}})}catch(e){throw await _e(),e}if(a?.aborted)throw await _e(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await _e(),new Error("Failed to create model context");return{model:e,backend:i.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(ye=!0===n.logging,e.isModelLoaded())return await B({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(C(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Pe(a),...M(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(pe==f)await _e();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Ee(e)||await Ce(await ke(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of l.flatMap(e=>Te(e)))if(await fe.stored(e.url))return!0;const e=await ke();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=Te(e);try{for(const e of t)await fe.remove(e.url)}catch{}try{const e=await ke();await e.cacheManager.deleteMany(e=>t.some(t=>Me(e,t.name)))}catch{}},purge:async()=>{try{await fe.purge()}catch{}try{const e=await ke();await e.cacheManager.clear()}catch{}}},$e=((e={})=>{const t=e.engines?.cpu||Re,n=e.engines?.litert||me;let a,o,c,d,g,u,m,f=i.none,p=r.idle,w=[],h=0,y={percent:0,phase:s.download};const b=a=>{if(e.backend!=i.none)return e.backend==i.cpu?a.cpu?t:void 0:e.backend==i.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||s.download};for(const e of[...w])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{h++,u?.abort(),u=void 0,g=void 0,p==r.downloading&&x(r.idle)},_=(e,t)=>{const n=d;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{w=w.filter(t=>t!=e),queueMicrotask(()=>{!w.length&&d&&k()})})(r),r.settle(!1)};w.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==r.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:i,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(d&&g==e.id)return p(y.percent,y.phase),await _(p,m);d&&(k(),w=[],await d.catch(()=>!1));const T=++h,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const d=b(e);if(T!=h)return!1;if(!d)return x(r.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=c,n=a;a=void 0,o=void 0,c=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of l)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,d),T!=h)return!1;const g=await d.cached(e);if(T!=h)return!1;x(r.downloading),v(0,s.download),g&&v(1,s.load);const m=new AbortController;u=m;try{const l=await d.startLLM({model:e,limits:t,logging:i,signal:m.signal,assets:n,onProgress:(e,t)=>{T==h&&v(e,t)}});return T!=h||m.signal.aborted?(await l.unload(),!1):(a=l,o=e,c=d,f=d.backend,x(r.ready),v(1,s.load),!0)}catch(e){return T!=h||(a=void 0,o=void 0,c=void 0,i&&console.log("Local model failed",e),x(r.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{d==E&&(d=void 0,g=void 0,w=[])});return d=E,g=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,c=void 0,x(r.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==r.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Se=(e,t,n)=>{const a=e.status(),i=d(t);return{availability:a==r.ready?o.online:a==r.error||a==r.unsupported?o.offline:o.away,status:a,backend:e.backend(),supportsImages:!!i&&m(i),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},je=(e=>{let t,n=0;const o=t=>t.input.onStatus?.(Se(e,t.model??t.input.model,t));return i=>{t?.controller.abort();const c={token:++n,input:i,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?s.load:s.download,requesting:!1,cancelled:!1};return t=c,o(c),(async t=>{const{input:r}=t,i=!0===r.logging,c=await u(r.model);if(t.model=c?.id,t.token!=n)return;const l=r.messages.some(e=>e.role!=a.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==a.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=l||d;if(!c)return t.error="Unknown model",void o(t);if(!g)return void o(t);if(d&&!l&&!m(c))return t.error="Model does not accept images",void o(t);const f=_(r.limits,!!c.litert),p=await e.load({model:c,limits:f,logging:i,signal:t.controller.signal,onProgress:(e,a)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=a,o(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void o(t);const w=m(c),h=d&&!w?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!w&&!l)return t.error="Model does not accept images",void o(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=s.load,o(t);let y,b="",v="";try{y=await e.chat({messages:h,limits:f,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=n||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,r.onResponse?.(v))},onSpeed:e=>{t.token!=n||t.cancelled||r.onSpeed?.(e)},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void o(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),o(t))})(c),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=r.ready?queueMicrotask(()=>o(t)):e.unload().then(()=>o(t)))})(c)}})($e),We=(qe=$e,async(e={})=>{const t=await u(e.model),n=n=>e.onStatus?.(Se(qe,t?.id,n));if(!t)return n({requesting:!1,phase:s.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=s.download;const i=qe.ready()&&qe.model()?.id==t.id;return await qe.load({model:t,limits:_(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(i&&n({requesting:!1,phase:s.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var qe;export{c as ChatModelId,o as ChatStatus,i as LlmBackend,s as LlmPhase,a as LlmRole,r as LlmStatus,je as createChat,We as preloadModel};

package/dist/chat.d.ts

@@ -2,7 +2,7 @@ import type { CancelChat, ChatInput, ChatPreloadInput, Runtime } from './types';
 /** Build a one-shot chat function over one runtime */
 declare const createChatOn: (service: Runtime) => ((input: ChatInput) => CancelChat);
 /** Build a model preload over one runtime */
-declare const createPreloadOn: (service: Runtime) => (input: ChatPreloadInput) => Promise<boolean>;
+declare const createPreloadOn: (service: Runtime) => (input?: ChatPreloadInput) => Promise<boolean>;
 declare const createChat: (input: ChatInput) => CancelChat;
-declare const preloadModel: (input: ChatPreloadInput) => Promise<boolean>;
+declare const preloadModel: (input?: ChatPreloadInput) => Promise<boolean>;
 export { createChat, createChatOn, createPreloadOn, preloadModel };

package/dist/engines/engine.d.ts

@@ -1,8 +1,8 @@
 import type { ChatMessage, ChatModel, EngineMessage, EngineRunParams, LlmLimits, StreamReplyParams } from '../types';
-/** Default token, prompt and thread budgets */
+/** Default token, prompt and thread budgets of the CPU engine */
 declare const defaultLlmLimits: LlmLimits;
-/** Merge developer budgets over the defaults */
-declare const resolveLlmLimits: (overrides?: Partial<LlmLimits>) => LlmLimits;
+/** Merge developer budgets over the backend defaults */
+declare const resolveLlmLimits: (overrides?: Partial<LlmLimits>, gpu?: boolean) => LlmLimits;
 /** Sampling options shared by every engine stream */
 declare const replyOptions: (limits: LlmLimits, reasoning: boolean) => {
     stream: true;
@@ -28,8 +28,8 @@ declare const
 /** Log an engine chat call with all message text censored */
 logChatCall: (model: ChatModel, messages: EngineMessage[]) => void, 
 /** Stream one reply, logging it and returning undefined when stopped */
-streamReply: ({ model, logging, onReason, onText, start, stopped, }: StreamReplyParams) => Promise<string | undefined>;
+streamReply: ({ model, logging, onReason, onText, onSpeed, start, stopped, firstTimeoutMs, idleTimeoutMs, onStall, }: StreamReplyParams) => Promise<string | undefined>;
 declare const 
 /** Build the prompt and stream one reply through an engine */
-runEngineChat: ({ model, params, start, stopped, }: EngineRunParams) => Promise<string | undefined>;
+runEngineChat: ({ model, params, start, stopped, onStall, }: EngineRunParams) => Promise<string | undefined>;
 export { buildMessages, createThinkSplitter, logChatCall, runEngineChat, streamReply, replyOptions, defaultLlmLimits, resolveLlmLimits, hasImages, modelUrl, contentLength, };

package/dist/engines/litert.d.ts

@@ -1,7 +1,16 @@
-import type { EngineMessage, LitertMessage, LlmEngine, StartLlmParams } from '../types';
+import type { EngineChunk, EngineMessage, LitertMessage, LitertSettings, LitertSettingsParams, LlmEngine, StartLlmParams, StreamLitertParams } from '../types';
+declare const 
+/** Smallest Gemma 4 vision budget in tokens */
+visionTokens = 70;
 /** Fetch one model file into the cache, once per URL */
 declare const cacheModel: (url: string, cache: Cache, signal: AbortSignal | undefined, onProgress: StartLlmParams[`onProgress`]) => Promise<void>;
 /** Map built messages to LiteRT turns */
-declare const toLitertMessages: (messages: EngineMessage[]) => Promise<LitertMessage[]>;
+declare const toLitertMessages: (messages: EngineMessage[], keepImages?: boolean) => Promise<LitertMessage[]>;
+/** Whether the live conversation already holds the incoming history */
+declare const continuesHistory: (sent: EngineMessage[], history: EngineMessage[]) => boolean;
+/** Stream one LiteRT reply as engine chunks */
+declare const streamLitert: ({ engine, mod, params, messages, session, }: StreamLitertParams) => AsyncGenerator<EngineChunk>;
+/** Build one engine settings object with the small vision budget */
+declare const litertSettings: ({ model, limits, source, mod, }: LitertSettingsParams) => LitertSettings;
 declare const litertEngine: LlmEngine;
-export { cacheModel, litertEngine, toLitertMessages, };
+export { cacheModel, continuesHistory, litertEngine, litertSettings, streamLitert, toLitertMessages, visionTokens, };

package/dist/models.d.ts

@@ -1,3 +1,4 @@
+import { ChatModelId } from './enums';
 import type { ChatModel } from './types';
 declare const llmModels: ChatModel[], 
 /** Context window requested from the CPU engine, a 32k KV cache needs 3.5 GB */
@@ -6,6 +7,12 @@ cpuContextWindow = 8192,
 litertContextWindow = 4096, 
 /** Look up a catalog entry by id */
 findLlmModel: (id?: string) => ChatModel | undefined, 
+/** Whether this browser provides a WebGPU adapter */
+gpuAdapter: () => Promise<boolean>, 
+/** Default model, the GPU one when a WebGPU adapter exists */
+defaultLlmModel: () => Promise<ChatModelId | undefined>, 
+/** Resolve the requested model or the default of this device */
+resolveLlmModel: (id?: ChatModelId) => Promise<ChatModel | undefined>, 
 /** Whether a model accepts images */
 supportsImages: (model: ChatModel) => boolean;
-export { llmModels, cpuContextWindow, litertContextWindow, findLlmModel, supportsImages, };
+export { llmModels, cpuContextWindow, litertContextWindow, findLlmModel, gpuAdapter, defaultLlmModel, resolveLlmModel, supportsImages, };

package/dist/node/chat.node.min.js

@@ -1,2 +1,2 @@
 /*! For license information please see chat.node.min.js.LICENSE.txt */
-(()=>{"use strict";var e={};const t={};function a(n){const o=t[n];if(void 0!==o)return o.exports;const r=t[n]={exports:{}};return e[n](r,r.exports,a),r.exports}a.m=e,a.d=(e,t)=>{for(var n in t)a.o(t,n)&&!a.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},a.f={},a.e=e=>{const t=[];return a.f.require(e,t),Promise.all(t)},a.u=e=>({520:"engine-litert",924:"engine-wllama"}[e]+".node.min.js"),a.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),a.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={792:1};a.f.require=(t,n)=>{if(!e[t]){const n=require("./"+a.u(t));e[t]||(t=>{const n=t.modules,o=t.ids,r=t.runtime;for(var i in n)a.o(n,i)&&(a.m[i]=n[i]);r&&r(a);for(var s=0;s<o.length;s++)e[o[s]]=1})(n)}}})();let n={};var o,r,i,s,c,l;a.r(n),a.d(n,{ChatModelId:()=>l,ChatStatus:()=>r,LlmBackend:()=>s,LlmPhase:()=>c,LlmRole:()=>o,LlmStatus:()=>i,createChat:()=>Me,preloadModel:()=>Ee}),function(e){e.system="system",e.user="user",e.assistant="assistant"}(o||(o={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(r||(r={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(i||(i={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(s||(s={})),function(e){e.download="download",e.load="load"}(c||(c={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(l||(l={}));const d=[{id:l.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:l.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],g=e=>e?d.find(t=>t.id==e):void 0,u=e=>!!e.cpu?.mmproj||!!e.litert?.vision,m=require("node:path"),f=()=>{return e=(0,m.join)(__dirname,"../assets/wllama"),{wasm:(0,m.join)(e,"wllama.wasm"),compatWasm:(0,m.join)(e,"compat/wllama.wasm"),compatWorker:(0,m.join)(e,"compat/wllama.js")};var e},w=()=>{return e=(0,m.join)(__dirname,"../assets/litert"),{script:(0,m.join)(e,"litertlm_wasm_internal.js"),wasm:(0,m.join)(e,"litertlm_wasm_internal.wasm")};var e},p=288,h=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),a=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!a.includes(";base64"))return await(await fetch(e)).arrayBuffer();const n=atob(e.slice(t+1)),o=new Uint8Array(n.length);for(let e=0;e<n.length;e++)o[e]=n.charCodeAt(e);return o.buffer},y=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),a=await createImageBitmap(await t.blob()),n=Math.min(a.width,a.height),o=Math.floor((a.width-n)/2),r=Math.floor((a.height-n)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const a=document.createElement("canvas");return a.width=e,a.height=t,a})(p,p),s=i?i.getContext("2d"):null;if(!i||!s||n<1)return a.close(),e;if(s.drawImage(a,o,r,n,n,0,0,p,p),a.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const a=new FileReader;a.onerror=()=>t(""),a.onload=()=>t(`${a.result}`),a.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},b={maxReplyTokens:200,maxThinkingTokens:64,maxReasoningTokens:150,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:2},v=(e,t,a)=>"number"==typeof e&&Number.isFinite(e)&&e>=a?Math.floor(e):t,x=(e={})=>{const t=b;return{maxReplyTokens:v(e.maxReplyTokens,t.maxReplyTokens,1),maxThinkingTokens:v(e.maxThinkingTokens,t.maxThinkingTokens,1),maxReasoningTokens:v(e.maxReasoningTokens,t.maxReasoningTokens,1),maxImages:v(e.maxImages,t.maxImages,0),maxContentLength:v(e.maxContentLength,t.maxContentLength,1),promptCharBudget:v(e.promptCharBudget,t.promptCharBudget,1),maxThreads:v(e.maxThreads,t.maxThreads,1)}},k=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),_=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,M=e=>Number(e.headers.get("content-length"))||0,E=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),C=["<|channel>thought\n","<|channel>thought","..."],T=["<channel|>","..."],L=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,j=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,$=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const a=[];for(const n of[...j(e),{type:"text",text:"\n"},...j(t)]){const e=a[a.length-1];"text"==n.type&&"text"==e?.type?e.text+=n.text:a.push(n)}return a},P=async({model:e,logging:t,onReason:a,onText:n,start:o,stopped:r})=>{let i="",s="",c="";const l=((e,t)=>{let a="",n=!1;const o=(e,t)=>{let a=-1,n="";for(const o of t){const t=e.indexOf(o);t>=0&&(a<0||t<a)&&(a=t,n=o)}return{at:a,tag:n}},r=(e,t)=>{let a="";for(const n of t)for(let t=Math.min(e.length,n.length-1);t>a.length;t--)if(e.endsWith(n.slice(0,t))){a=e.slice(e.length-t);break}return a};return{feed:i=>{let s=a+i;for(a="";s;){if(n){const{at:t,tag:i}=o(s,T);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),n=!1;continue}const c=r(s,T);e(s.slice(0,s.length-c.length)),a=c;break}{const{at:e,tag:i}=o(s,C);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),n=!0;continue}const c=r(s,C);t(s.slice(0,s.length-c.length)),a=c;break}}},flush:()=>{a&&(n?e(a):t(a),a="")}}})(e=>{s+=e,a(e)},e=>{i+=e,n(e)});try{const e=await o();for await(const t of e){if(r())break;const e=t?.choices?.[0],n=e?.delta,o="string"==typeof n?.reasoning_content?n.reasoning_content:"string"==typeof n?.reasoning?n.reasoning:"";e?.finish_reason&&(c=e.finish_reason),o&&(s+=o,a(o)),n?.content&&l.feed(n.content)}l.flush()}catch(e){t&&console.log("Local chat failed",e)}const d=r()?void 0:i||void 0;return t&&(console.log("Local chat reply",e.id,d?"****":""),console.log("Local chat finish",e.id,c||"none",i.length,s.length)),d},R=async({model:e,params:t,start:a,stopped:n})=>{const r=await(async e=>{const t=[];for(const a of e){if("string"==typeof a.content){t.push(a);continue}const e=[];for(const t of a.content)e.push("image"==t.type?{type:"image",url:await y(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=b)=>{const a=t.maxContentLength,n=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=o.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=n.length-1;e>=0;e--){const l=n[e],d=("string"==typeof l.content?l.content:"").slice(0,a),g=l.role==o.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==o.system?r.unshift(d):l.role==o.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:L(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=$(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,r),await P({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,start:()=>a(r),stopped:n})},q=33554432,W=new Map;let S=!1;const B=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let a=0;for(let t=0;t<e.length;t++)a=31*a+e.charCodeAt(t)>>>0;return`${a.toString(16)}-${t}`},z=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},A=async()=>{if(!S)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else S=!0},O=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},I=async e=>{if(e)try{const t=await e.getFile(),a=JSON.parse(await t.text());return{total:Number(a.total)||0,etag:`${a.etag||""}`,complete:!0===a.complete}}catch{return}},N=async(e,t)=>{try{const a=await e.getFileHandle("meta.json",{create:!0}),n=await a.createWritable();await n.write(JSON.stringify(t)),await n.close()}catch{}},F=async function*(e){for await(const t of e)yield t},U=async e=>{try{for await(const[t,a]of F(e))"file"==a.kind&&z(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},D=async(e,t)=>{try{const a=await(await e.getFileHandle(t)).getFile(),n=await I(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!n)return;return{file:a,size:a.size,total:n.total,etag:n.etag,complete:n.total?a.size>=n.total:!0===n.complete}}catch{return}},H=async(e,t)=>{const a=await O(e,t);if(a){const e=await(async e=>await I(await e.getFileHandle("meta.json").catch(()=>{})))(a),t=await(async e=>{const t=[];try{for await(const[a,n]of F(e)){const e=z(a);if("file"!=n.kind||e<0)continue;const o=await n.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const a=(e=>{const t=[];let a=0;for(const n of[...e].sort((e,t)=>e.start-t.start))if(n.size){if(n.start!=a)break;t.push(n),a+=n.size}return t})(t.map(e=>e.part)),n=new Set(a.map(e=>e.start)),o=t.filter(e=>n.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:a.reduce((e,t)=>e+t.size,0)}})(a),n=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:n&&o>n?r.slice(0,n):r,size:o,total:n,etag:e?.etag||"",complete:n?o>=n:!0===e?.complete}}return await D(e,t)},Q=(e,t)=>{const a=e.headers.get("content-range"),n=/bytes (\d+)-/.exec(a||""),o=/\/(\d+)\s*$/.exec(a||""),r=e.headers.get("etag")||"";if(206==e.status){const e=n?Number(n[1]):t.size,a=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&a&&t.total!=a)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:a||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:M(e)||t.total,etag:r||t.etag}},G=async({dir:e,start:t,skip:a,total:n,body:o,onProgress:r,onChunk:i})=>{let s,l=t,d=0,g=a;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});s=await t.createWritable(),d=0},m=async()=>{const e=s;s=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:a,value:o}=await e.read();if(a)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){s||await u();const e=Math.min(q-d,o.byteLength-f);await s.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==q&&await m()}l>t&&r(n?l/n:0,c.download)}await m()}catch(e){throw await m(),e}return l},V=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),J=e=>{const t=async()=>{const t=await A();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const a=await t();return a?await H(a,B(e)):void 0},download:async({url:a,signal:n,onProgress:o})=>{const r=`${e}|${a}`,i=W.get(r);if(i)return await i;const s=await t();if(!s)return;const l=(async(e,t,a,n)=>{const o=B(t),r=await D(e,o);if(r?.complete)return r;let i=await O(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let s=0,l=0;for(;;){if(a?.aborted)throw new Error("Cancelled");const r=await H(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>s)s=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&n(d/g,c.download);const m=g?Math.min(d+q,g)-1:d+q-1,f=new AbortController,w=()=>f.abort(),p={size:d,total:g,etag:u};let h,y=setTimeout(w,3e4);a?.addEventListener("abort",w,{once:!0});try{if(h=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==h.status){if(await(h.body?.cancel().catch(()=>{})),g&&d>=g){await N(i,{total:g,etag:u,complete:!0});continue}await U(i),await N(i,{total:0,etag:"",complete:!1});continue}if(!h.ok||!h.body)throw await(h.body?.cancel().catch(()=>{})),new Error("Model download failed");const a=Q(h,p);if(!a){await h.body.cancel().catch(()=>{}),await U(i),await N(i,{total:0,etag:"",complete:!1});continue}await N(i,{total:a.total,etag:a.etag,complete:!1});const r=0==a.start?d:0,s=await G({dir:i,start:a.start+r,skip:r,total:a.total,body:h.body,onProgress:n,onChunk:()=>{clearTimeout(y),y=setTimeout(w,3e4)}});if(!a.total){await N(i,{total:s,etag:a.etag,complete:!0});const t=await H(e,o);if(t)return t}}catch(e){if(a?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),a?.removeEventListener("abort",w)}}})(s,a,n,o).finally(()=>{W.delete(r)});return W.set(r,l),await l},remove:async e=>{const a=await t();a&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(a,B(e))},purge:async()=>{const t=await A();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},K="litertlm-models",X=J("litert-store"),Y={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0},Z=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(K)}catch{return}},ee=new Map,te=(e,t,a)=>{let n=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{n+=e.byteLength,a(t?n/t:0,c.download),o.enqueue(e)}}))},ae=async e=>{const t=[];for(const a of e){if("string"==typeof a.content){t.push({role:a.role,content:a.content});continue}const e=[];for(const t of a.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await h(t.url)});t.push({role:"user",content:e})}return t},ne=async({engine:e,model:t,mod:a,params:n})=>await R({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async o=>{if(n.signal?.aborted)throw new Error("Cancelled");if(E(o)&&!t.litert?.vision)throw new Error("Model does not accept images");return async function*({engine:e,mod:t,params:a,messages:n}){const o=n[n.length-1],r=k(a.limits,a.reasoning);if("user"!=o?.role)throw new Error("No user turn");const i=await ae(n.slice(0,-1)),s=(await ae([o]))[0];if(!s)throw new Error("Empty prompt");const c=await e.createConversation({sessionConfig:{maxOutputTokens:r.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:r.temperature,k:40,p:.95}},preface:{...i.length?{messages:i}:{},extra_context:{enable_thinking:a.reasoning}}});let l;const d=()=>{l&&l.cancel().catch(()=>{});try{c.cancel()}catch{}};a.signal?.addEventListener("abort",d,{once:!0});try{if(a.signal?.aborted)return;const e=c.sendMessageStreaming(s).getReader();for(l=e;;){const{done:t,value:a}=await e.read();if(t)break;const n={};let o="string"==typeof a.content?a.content:"";if(Array.isArray(a.content))for(const e of a.content)"text"==e.type&&(o+=e.text);o&&(n.content=o);const r=a.channels?.thought;r&&(n.reasoning=r),(n.content||n.reasoning)&&(yield{choices:[{delta:n}]})}}finally{a.signal?.removeEventListener("abort",d);try{await c.delete()}catch{}}}({engine:e,mod:a,params:n,messages:o})}}),oe={backend:s.gpu,startLLM:async({model:e,limits:t,onProgress:n,signal:o,logging:r})=>{const i=e.litert;if(!i)throw new Error("Model has no LiteRT artifact");const l=await a.e(520).then(()=>a(507));if(await l.getOrLoadGlobalLiteRtLm(w().script),o?.aborted)throw new Error("Cancelled");const d=_(i.repo,i.file),g=await Z();let u;const m=g?await g.match(d):void 0;if(m?.body)u=m.body;else{const e=await X.stored(d);e?.complete&&(u=e.file.stream())}if(!u){const e=await X.download({url:d,signal:o,onProgress:n}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!V(e))throw e;r&&console.log("Local model store failed",e)});e&&(u=e.file.stream())}if(!u&&g)try{await(async(e,t,a,n)=>{const o=ee.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:a});if(!o.ok||!o.body)throw new Error("Model download failed");const r=M(o),i=te(o.body,r,n);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{ee.delete(e)});return ee.set(e,r),await r})(d,g,o,n);const e=await g.match(d);e?.body&&(u=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(u||(u=await(async(e,t,a)=>{const n=await fetch(e,{signal:t});if(!n.ok||!n.body)throw new Error("Model download failed");return te(n.body,M(n),a)})(d,o,n)),o?.aborted)throw new Error("Cancelled");n(1,c.load);const f=await l.Engine.create({model:u,mainExecutorSettings:{maxNumTokens:4096,...i.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:Y},...i.vision?{visionBackend:l.Backend.GPU}:{}});if(o?.aborted)throw await f.delete(),new Error("Cancelled");return{model:e,backend:s.gpu,contextSize:4096,chat:t=>ne({engine:f,model:e,mod:l,params:t}),unload:async()=>{await f.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const a=_(t.repo,t.file),[n,o]=await Promise.all([Z(),X.stored(a)]);if(o?.complete)return!0;try{return!(!n||!await n.match(a))}catch{return!1}},hasCache:async()=>{try{for(const e of d){const t=e.litert;if(!t)continue;if(await X.stored(_(t.repo,t.file)))return!0}const e=await Z();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const a=_(t.repo,t.file),n=await Z();await X.remove(a),n&&await n.delete(a)}catch{}},purge:async()=>{try{await X.purge(),"undefined"!=typeof caches&&await caches.delete(K)}catch{}}},re=J("wllama-store");let ie,se,ce="",le=!1;const de=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ge=e=>{const t=f();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},ue=e=>{const t=e=>(...t)=>{le&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},me=async()=>{if(ie)return ie;const e=await a.e(924).then(()=>a(577)),t=se||ge(),n=new e.Wllama({default:t.wasm},{logger:ue(e.LoggerWithoutDebug)});return se||(se=t,ce=de(t)),n.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),ie=n,n},fe=async()=>{const e=ie;ie=void 0;try{await(e?.exit())}catch{}},we=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),pe=e=>{const t=e.cpu;return t?[{name:t.file,url:_(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:_(t.repo,t.mmproj.file)}]:[]]:[]},he=async e=>{const t=pe(e);if(!t.length)return!1;for(const e of t){const t=await re.stored(e.url);if(!t?.complete)return!1}return!0},ye=async(e,t)=>{const a=await e.cacheManager.list();return pe(t).every(e=>a.some(t=>we(t,e.name)))},be=async({model:e,signal:t,onProgress:a})=>{const n=pe(e),o=[];for(let e=0;e<n.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await re.download({url:n[e].url,signal:t,onProgress:(t,o)=>a((e+t)/n.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},ve=async e=>{const t=[];for(const a of e){if("string"==typeof a.content){"system"==a.role?t.push({role:"system",content:a.content}):"assistant"==a.role?t.push({role:"assistant",content:a.content}):t.push({role:"user",content:a.content});continue}const e=[];for(const t of a.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await h(t.url)});t.push({role:"user",content:e})}return t},xe={backend:s.cpu,startLLM:async({model:e,limits:t,onProgress:a,signal:n,logging:o,assets:r})=>{le=!0===o;const i=e.cpu;if(!i)throw new Error("Model has no CPU artifact");const l=ge(r),d=de(l);ie&&d!=ce&&await fe(),se=l,ce=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await me(),w=await he(e),p=i.mmproj?.file,h=_(i.repo,i.file),y=p?_(i.repo,p):"";if(n?.aborted)throw await fe(),new Error("Cancelled");if(!w&&p){const e=await f.cacheManager.list(),t=e.some(e=>we(e,i.file)),a=e.some(e=>we(e,p));t&&!a&&await f.cacheManager.deleteMany(e=>we(e,i.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=w?await be({model:e,signal:n,onProgress:a}):void 0;t||w||await ye(f,e).catch(()=>!1)||(t=await be({model:e,signal:n,onProgress:a}).catch(e=>{if(n?.aborted)throw new Error("Cancelled");if(!V(e))throw e;o&&console.log("Local model store failed",e)})),t?(a(1,c.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:h,mmprojUrl:y}:h,{...b,useCache:!0,signal:n,progressCallback:({loaded:e,total:t})=>{const n=t?e/t:0;a(n,n>=1?c.load:c.download)}})}catch(e){throw await fe(),e}if(n?.aborted)throw await fe(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await fe(),new Error("Failed to create model context");return{model:e,backend:s.cpu,contextSize:8192,chat:t=>(async(e,t,a)=>{if(le=!0===a.logging,e.isModelLoaded())return await R({model:t,params:a,stopped:()=>!!a.signal?.aborted,start:async n=>{if(E(n)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await ve(n),...k(a.limits,a.reasoning),abortSignal:a.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:a.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(ie==f)await fe();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await he(e)||await ye(await me(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of d.flatMap(e=>pe(e)))if(await re.stored(e.url))return!0;const e=await me();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=pe(e);try{for(const e of t)await re.remove(e.url)}catch{}try{const e=await me();await e.cacheManager.deleteMany(e=>t.some(t=>we(e,t.name)))}catch{}},purge:async()=>{try{await re.purge()}catch{}try{const e=await me();await e.cacheManager.clear()}catch{}}},ke=((e={})=>{const t=e.engines?.cpu||xe,a=e.engines?.litert||oe;let n,o,r,l,g,u,m,f=s.none,w=i.idle,p=[],h=0,y={percent:0,phase:c.download};const b=n=>{if(e.backend!=s.none)return e.backend==s.cpu?n.cpu?t:void 0:e.backend==s.gpu?n.litert?a:void 0:n.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?a:!n.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||c.download};for(const e of[...p])e.report(y.percent,y.phase)},x=e=>{w=e},k=()=>{h++,u?.abort(),u=void 0,g=void 0,w==i.downloading&&x(i.idle)},_=(e,t)=>{const a=l;return a?new Promise(n=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),n(e))}},i=()=>{(e=>{p=p.filter(t=>t!=e),queueMicrotask(()=>{!p.length&&l&&k()})})(r),r.settle(!1)};p.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),a.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>w==i.ready&&!!n&&!!o;return{load:async({model:e,limits:t,assets:a,logging:s,signal:m,onProgress:w})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(l&&g==e.id)return w(y.percent,y.phase),await _(w,m);l&&(k(),p=[],await l.catch(()=>!1));const E=++h,C=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const l=b(e);if(E!=h)return!1;if(!l)return x(i.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,a=n;n=void 0,o=void 0,r=void 0,await(a?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const a of d)a.id!=e.id&&b(a)==t&&await t.remove(a)})(e,l),E!=h)return!1;const g=await l.cached(e);if(E!=h)return!1;x(i.downloading),v(0,c.download),g&&v(1,c.load);const m=new AbortController;u=m;try{const d=await l.startLLM({model:e,limits:t,logging:s,signal:m.signal,assets:a,onProgress:(e,t)=>{E==h&&v(e,t)}});return E!=h||m.signal.aborted?(await d.unload(),!1):(n=d,o=e,r=l,f=l.backend,x(i.ready),v(1,c.load),!0)}catch(e){return E!=h||(n=void 0,o=void 0,r=void 0,s&&console.log("Local model failed",e),x(i.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{l==C&&(l=void 0,g=void 0,p=[])});return l=C,g=e.id,await _(w,m)},chat:async e=>{if(!M()||!n)return;for(;m;)await m.catch(()=>{});if(!M()||!n)return;const t=n.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=n;k(),n=void 0,o=void 0,r=void 0,x(i.idle),await(e?.unload())},ready:M,loading:()=>!M()&&w==i.downloading,status:()=>w,backend:()=>f,model:()=>o}})(),_e=(e,t,a)=>{const n=e.status(),o=g(t);return{availability:n==i.ready?r.online:n==i.error||n==i.unsupported?r.offline:r.away,status:n,backend:e.backend(),supportsImages:!!o&&u(o),ready:e.ready(),loading:e.loading(),requesting:a.requesting,phase:a.phase,percent:a.percent,...a.error?{error:a.error}:{}}},Me=(e=>{let t,a=0;const n=t=>t.input.onStatus?.(_e(e,t.input.model,t));return r=>{t?.controller.abort();const s={token:++a,input:r,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?c.load:c.download,requesting:!1,cancelled:!1};return t=s,n(s),(async t=>{const{input:r}=t,i=!0===r.logging,s=g(r.model),l=r.messages.some(e=>e.role!=o.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==o.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),m=l||d;if(!s)return t.error="Unknown model",void n(t);if(!m)return void n(t);if(d&&!l&&!u(s))return t.error="Model does not accept images",void n(t);const f=x(r.limits),w=await e.load({model:s,limits:f,logging:i,signal:t.controller.signal,onProgress:(e,o)=>{t.token==a&&(t.percent=Math.round(100*e),t.phase=o,n(t))}}).catch(e=>(t.token==a&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=a)return;if(t.cancelled)return;if(!w)return t.error||(t.error="Model not ready"),void n(t);const p=u(s),h=d&&!p?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!p&&!l)return t.error="Model does not accept images",void n(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=c.load,n(t);let y,b="",v="";try{y=await e.chat({messages:h,limits:f,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=a||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=a||t.cancelled||(v+=e,r.onResponse?.(v))},signal:t.controller.signal})}catch(e){if(t.token!=a)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void n(t)}t.token==a&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),n(t))})(s),()=>(t=>{t.token==a&&(t.cancelled=!0,t.controller.abort(),e.status()!=i.ready?queueMicrotask(()=>n(t)):e.unload().then(()=>n(t)))})(s)}})(ke),Ee=(Ce=ke,async e=>{const t=g(e.model),a=t=>e.onStatus?.(_e(Ce,e.model,t));if(!t)return a({requesting:!1,phase:c.download,percent:0,error:"Unknown model"}),!1;let n,o=0,r=c.download;const i=Ce.ready()&&Ce.model()?.id==t.id;return await Ce.load({model:t,limits:x(e.limits),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,a({requesting:!1,phase:r,percent:o})}}).catch(e=>(n=`${e?.message||`${e}`}`,!1))?(i&&a({requesting:!1,phase:c.load,percent:100}),!0):(a({requesting:!1,phase:r,percent:o,error:n||"Model not ready"}),!1)});var Ce;module.exports=n})();
+(()=>{"use strict";var e={};const t={};function n(a){const o=t[a];if(void 0!==o)return o.exports;const r=t[a]={exports:{}};return e[a](r,r.exports,n),r.exports}n.m=e,n.d=(e,t)=>{for(var a in t)n.o(t,a)&&!n.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},n.f={},n.e=e=>{const t=[];return n.f.require(e,t),Promise.all(t)},n.u=e=>({520:"engine-litert",924:"engine-wllama"}[e]+".node.min.js"),n.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),n.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={792:1};n.f.require=(t,a)=>{if(!e[t]){const a=require("./"+n.u(t));e[t]||(t=>{const a=t.modules,o=t.ids,r=t.runtime;for(var i in a)n.o(a,i)&&(n.m[i]=a[i]);r&&r(n);for(var s=0;s<o.length;s++)e[o[s]]=1})(a)}}})();let a={};var o,r,i,s,c,l;n.r(a),n.d(a,{ChatModelId:()=>l,ChatStatus:()=>r,LlmBackend:()=>s,LlmPhase:()=>c,LlmRole:()=>o,LlmStatus:()=>i,createChat:()=>Re,preloadModel:()=>qe}),function(e){e.system="system",e.user="user",e.assistant="assistant"}(o||(o={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(r||(r={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(i||(i={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(s||(s={})),function(e){e.download="download",e.load="load"}(c||(c={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(l||(l={}));const d=[{id:l.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:l.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],g=e=>e?d.find(t=>t.id==e):void 0,u=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?d.find(e=>!!e.litert):d.find(e=>!!e.cpu))?.id,m=async e=>g(e||await u()),f=e=>!!e.cpu?.mmproj||!!e.litert?.vision,p=require("node:path"),h=()=>{return e=(0,p.join)(__dirname,"../assets/wllama"),{wasm:(0,p.join)(e,"wllama.wasm"),compatWasm:(0,p.join)(e,"compat/wllama.wasm"),compatWorker:(0,p.join)(e,"compat/wllama.js")};var e},w=()=>{return e=(0,p.join)(__dirname,"../assets/litert"),{script:(0,p.join)(e,"litertlm_wasm_internal.js"),wasm:(0,p.join)(e,"litertlm_wasm_internal.wasm")};var e},y=288,b=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},v=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(y,y),s=i?i.getContext("2d"):null;if(!i||!s||a<1)return n.close(),e;if(s.drawImage(n,o,r,a,a,0,0,y,y),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},x={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},k=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,M=(e={},t=!1)=>{const n=x;return{maxReplyTokens:k(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:k(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:k(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:k(e.maxImages,n.maxImages,0),maxContentLength:k(e.maxContentLength,n.maxContentLength,1),promptCharBudget:k(e.promptCharBudget,n.promptCharBudget,1),maxThreads:k(e.maxThreads,n.maxThreads,1)}},_=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),T=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,C=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),L=["<|channel>thought\n","<|channel>thought","..."],P=["<channel|>","..."],j=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,S=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,$=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...S(e),{type:"text",text:"\n"},...S(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class R extends Error{}const q=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new R(n)),t)})])}finally{a&&clearTimeout(a)}},W=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},B=async({model:e,logging:t,onReason:n,onText:a,onSpeed:o,start:r,stopped:i,firstTimeoutMs:s=18e4,idleTimeoutMs:c=3e4,onStall:l})=>{let d="",g="",u="",m=!1,f=0,p=0;const h=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:i=>{let s=n+i;for(n="";s;){if(a){const{at:t,tag:i}=o(s,P);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),a=!1;continue}const c=r(s,P);e(s.slice(0,s.length-c.length)),n=c;break}{const{at:e,tag:i}=o(s,L);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),a=!0;continue}const c=r(s,L);t(s.slice(0,s.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{g+=e,n(e)},e=>{d+=e,a(e)});try{const e=(await q(r(),s,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(i()){W(e);break}let a;try{a=await q(e.next(),m?c:s,m?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof R&&(d||g)){l?.(),W(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;m=!0;const r=a.value?.choices?.[0],w=r?.delta,y="string"==typeof w?.reasoning_content?w.reasoning_content:"string"==typeof w?.reasoning?w.reasoning:"";if(r?.finish_reason&&(u=r.finish_reason),y&&(g+=y,n(y)),w?.content&&h.feed(w.content),y||w?.content){const e=performance.now();f||(p=e),f+=1,f>1&&e>p&&o?.(1e3*f/(e-p))}}h.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof R&&!i())throw e}const w=i()?void 0:d||void 0;return t&&(console.log("Local chat reply",e.id,w?"****":""),console.log("Local chat finish",e.id,u||"none",d.length,g.length)),w},I=async({model:e,params:t,start:n,stopped:a,onStall:r})=>{const i=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await v(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=x)=>{const n=t.maxContentLength,a=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=o.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=a.length-1;e>=0;e--){const l=a[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==o.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==o.system?r.unshift(d):l.role==o.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:j(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=$(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,i),await B({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,onSpeed:t.onSpeed,start:()=>n(i),stopped:a,onStall:r})},z=33554432,A=new Map;let O=!1;const N=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},F=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},U=async()=>{if(!O)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else O=!0},D=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},H=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},Q=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},V=async function*(e){for await(const t of e)yield t},G=async e=>{try{for await(const[t,n]of V(e))"file"==n.kind&&F(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},J=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await H(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},K=async(e,t)=>{const n=await D(e,t);if(n){const e=await(async e=>await H(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of V(e)){const e=F(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await J(e,t)},X=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},Y=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:i})=>{let s,l=t,d=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});s=await t.createWritable(),d=0},m=async()=>{const e=s;s=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){s||await u();const e=Math.min(z-d,o.byteLength-f);await s.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==z&&await m()}l>t&&r(a?l/a:0,c.download)}await m()}catch(e){throw await m(),e}return l},Z=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),ee=e=>{const t=async()=>{const t=await U();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await K(n,N(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,i=A.get(r);if(i)return await i;const s=await t();if(!s)return;const l=(async(e,t,n,a)=>{const o=N(t),r=await J(e,o);if(r?.complete)return r;let i=await D(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let s=0,l=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await K(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>s)s=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&a(d/g,c.download);const m=g?Math.min(d+z,g)-1:d+z-1,f=new AbortController,p=()=>f.abort(),h={size:d,total:g,etag:u};let w,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(w=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==w.status){if(await(w.body?.cancel().catch(()=>{})),g&&d>=g){await Q(i,{total:g,etag:u,complete:!0});continue}await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}if(!w.ok||!w.body)throw await(w.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=X(w,h);if(!n){await w.body.cancel().catch(()=>{}),await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}await Q(i,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?d:0,s=await Y({dir:i,start:n.start+r,skip:r,total:n.total,body:w.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await Q(i,{total:s,etag:n.etag,complete:!0});const t=await K(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(s,n,a,o).finally(()=>{A.delete(r)});return A.set(r,l),await l},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,N(e))},purge:async()=>{const t=await U();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},te="litertlm-models",ne=ee("litert-store"),ae={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0};let oe=!1;const re=(...e)=>{oe&&console.log(...e)},ie=(...e)=>{oe&&console.error(...e)},se=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(te)}catch{return}},ce=new Map,le=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,c.download),o.enqueue(e)}}))},de=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await b(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ge=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},ue=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],i=_(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const s=a.slice(0,-1),c=(await de([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==i.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ge(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const i=o.length-r.length;for(let e=0;e<r.length;e++)if(!ge(o[i+e],r[e]))return!1;return!0})(o.sent,s);let d;if(l&&o.conversation)d=o.conversation;else{const a=await de(s,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:i.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:i.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=i.max_tokens,o.sent=[...s]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},me={backend:s.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:o,logging:r})=>{oe=!0===r;const i=e.litert;if(!i)throw new Error("Model has no LiteRT artifact");const l=await n.e(520).then(()=>n(507)),d=globalThis,g=d.Module;d.Module={print:re,printErr:ie};try{await l.getOrLoadGlobalLiteRtLm(w().script)}finally{d.Module=g}if(o?.aborted)throw new Error("Cancelled");const u=T(i.repo,i.file),m=await se();let f;const p=m?await m.match(u):void 0;if(p?.body)f=p.body;else{const e=await ne.stored(u);e?.complete&&(f=e.file.stream())}if(!f){const e=await ne.download({url:u,signal:o,onProgress:a}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;r&&console.log("Local model store failed",e)});e&&(f=e.file.stream())}if(!f&&m)try{await(async(e,t,n,a)=>{const o=ce.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),i=le(o.body,r,a);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{ce.delete(e)});return ce.set(e,r),await r})(u,m,o,a);const e=await m.match(u);e?.body&&(f=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(f||(f=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return le(a.body,E(a),n)})(u,o,a)),o?.aborted)throw new Error("Cancelled");a(1,c.load);const h=await l.Engine.create((({model:e,limits:t,source:n,mod:a})=>({model:n,mainExecutorSettings:{maxNumTokens:4096,...e.litert?.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:ae},...e.litert?.vision?{visionBackend:a.Backend.GPU,maxVisionTokensPerImage:70,minVisionTokensPerImage:70}:{}}))({model:e,limits:t,source:f,mod:l}));if(o?.aborted)throw await h.delete(),new Error("Cancelled");const y={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:s.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{oe=!0===a.logging;const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await I({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(C(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return ue({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:h,model:e,mod:l,params:t,session:y}),unload:async()=>{y.conversation=void 0,y.sent=[],await h.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=T(t.repo,t.file),[a,o]=await Promise.all([se(),ne.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of d){const t=e.litert;if(!t)continue;if(await ne.stored(T(t.repo,t.file)))return!0}const e=await se();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=T(t.repo,t.file),a=await se();await ne.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ne.purge(),"undefined"!=typeof caches&&await caches.delete(te)}catch{}}},fe=ee("wllama-store");let pe,he,we="",ye=!1;const be=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ve=e=>{const t=h();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},xe=e=>{const t=e=>(...t)=>{ye&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},ke=async()=>{if(pe)return pe;const e=await n.e(924).then(()=>n(577)),t=he||ve(),a=new e.Wllama({default:t.wasm},{logger:xe(e.LoggerWithoutDebug)});return he||(he=t,we=be(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),pe=a,a},Me=async()=>{const e=pe;pe=void 0;try{await(e?.exit())}catch{}},_e=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),Te=e=>{const t=e.cpu;return t?[{name:t.file,url:T(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:T(t.repo,t.mmproj.file)}]:[]]:[]},Ee=async e=>{const t=Te(e);if(!t.length)return!1;for(const e of t){const t=await fe.stored(e.url);if(!t?.complete)return!1}return!0},Ce=async(e,t)=>{const n=await e.cacheManager.list();return Te(t).every(e=>n.some(t=>_e(t,e.name)))},Le=async({model:e,signal:t,onProgress:n})=>{const a=Te(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await fe.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Pe=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await b(t.url)});t.push({role:"user",content:e})}return t},je={backend:s.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{ye=!0===o;const i=e.cpu;if(!i)throw new Error("Model has no CPU artifact");const l=ve(r),d=be(l);pe&&d!=we&&await Me(),he=l,we=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await ke(),p=await Ee(e),h=i.mmproj?.file,w=T(i.repo,i.file),y=h?T(i.repo,h):"";if(a?.aborted)throw await Me(),new Error("Cancelled");if(!p&&h){const e=await f.cacheManager.list(),t=e.some(e=>_e(e,i.file)),n=e.some(e=>_e(e,h));t&&!n&&await f.cacheManager.deleteMany(e=>_e(e,i.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Le({model:e,signal:a,onProgress:n}):void 0;t||p||await Ce(f,e).catch(()=>!1)||(t=await Le({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,c.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:w,mmprojUrl:y}:w,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?c.load:c.download)}})}catch(e){throw await Me(),e}if(a?.aborted)throw await Me(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await Me(),new Error("Failed to create model context");return{model:e,backend:s.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(ye=!0===n.logging,e.isModelLoaded())return await I({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(C(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Pe(a),..._(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(pe==f)await Me();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Ee(e)||await Ce(await ke(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of d.flatMap(e=>Te(e)))if(await fe.stored(e.url))return!0;const e=await ke();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=Te(e);try{for(const e of t)await fe.remove(e.url)}catch{}try{const e=await ke();await e.cacheManager.deleteMany(e=>t.some(t=>_e(e,t.name)))}catch{}},purge:async()=>{try{await fe.purge()}catch{}try{const e=await ke();await e.cacheManager.clear()}catch{}}},Se=((e={})=>{const t=e.engines?.cpu||je,n=e.engines?.litert||me;let a,o,r,l,g,u,m,f=s.none,p=i.idle,h=[],w=0,y={percent:0,phase:c.download};const b=a=>{if(e.backend!=s.none)return e.backend==s.cpu?a.cpu?t:void 0:e.backend==s.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||c.download};for(const e of[...h])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{w++,u?.abort(),u=void 0,g=void 0,p==i.downloading&&x(i.idle)},M=(e,t)=>{const n=l;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{h=h.filter(t=>t!=e),queueMicrotask(()=>{!h.length&&l&&k()})})(r),r.settle(!1)};h.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},_=()=>p==i.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:s,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(_()&&o?.id==e.id)return!0;if(l&&g==e.id)return p(y.percent,y.phase),await M(p,m);l&&(k(),h=[],await l.catch(()=>!1));const T=++w,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const l=b(e);if(T!=w)return!1;if(!l)return x(i.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,n=a;a=void 0,o=void 0,r=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of d)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,l),T!=w)return!1;const g=await l.cached(e);if(T!=w)return!1;x(i.downloading),v(0,c.download),g&&v(1,c.load);const m=new AbortController;u=m;try{const d=await l.startLLM({model:e,limits:t,logging:s,signal:m.signal,assets:n,onProgress:(e,t)=>{T==w&&v(e,t)}});return T!=w||m.signal.aborted?(await d.unload(),!1):(a=d,o=e,r=l,f=l.backend,x(i.ready),v(1,c.load),!0)}catch(e){return T!=w||(a=void 0,o=void 0,r=void 0,s&&console.log("Local model failed",e),x(i.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{l==E&&(l=void 0,g=void 0,h=[])});return l=E,g=e.id,await M(p,m)},chat:async e=>{if(!_()||!a)return;for(;m;)await m.catch(()=>{});if(!_()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,r=void 0,x(i.idle),await(e?.unload())},ready:_,loading:()=>!_()&&p==i.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),$e=(e,t,n)=>{const a=e.status(),o=g(t);return{availability:a==i.ready?r.online:a==i.error||a==i.unsupported?r.offline:r.away,status:a,backend:e.backend(),supportsImages:!!o&&f(o),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},Re=(e=>{let t,n=0;const a=t=>t.input.onStatus?.($e(e,t.model??t.input.model,t));return r=>{t?.controller.abort();const s={token:++n,input:r,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?c.load:c.download,requesting:!1,cancelled:!1};return t=s,a(s),(async t=>{const{input:r}=t,i=!0===r.logging,s=await m(r.model);if(t.model=s?.id,t.token!=n)return;const l=r.messages.some(e=>e.role!=o.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==o.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=l||d;if(!s)return t.error="Unknown model",void a(t);if(!g)return void a(t);if(d&&!l&&!f(s))return t.error="Model does not accept images",void a(t);const u=M(r.limits,!!s.litert),p=await e.load({model:s,limits:u,logging:i,signal:t.controller.signal,onProgress:(e,o)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=o,a(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void a(t);const h=f(s),w=d&&!h?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!h&&!l)return t.error="Model does not accept images",void a(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=c.load,a(t);let y,b="",v="";try{y=await e.chat({messages:w,limits:u,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=n||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,r.onResponse?.(v))},onSpeed:e=>{t.token!=n||t.cancelled||r.onSpeed?.(e)},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void a(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),a(t))})(s),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=i.ready?queueMicrotask(()=>a(t)):e.unload().then(()=>a(t)))})(s)}})(Se),qe=(We=Se,async(e={})=>{const t=await m(e.model),n=n=>e.onStatus?.($e(We,t?.id,n));if(!t)return n({requesting:!1,phase:c.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=c.download;const i=We.ready()&&We.model()?.id==t.id;return await We.load({model:t,limits:M(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(i&&n({requesting:!1,phase:c.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var We;module.exports=a})();

package/dist/node/chat.node.min.mjs

@@ -1,2 +1,2 @@
 /*! For license information please see chat.node.min.mjs.LICENSE.txt */
-import{dirname as e,join as t}from"node:path";import{fileURLToPath as a}from"node:url";var n={};const o={};function r(e){const t=o[e];if(void 0!==t)return t.exports;const a=o[e]={exports:{}};return n[e](a,a.exports,r),a.exports}var s,i,c,l,d,g;r.m=n,r.d=(e,t)=>{for(var a in t)r.o(t,a)&&!r.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},r.f={},r.e=e=>{const t=[];return r.f.j(e,t),Promise.all(t)},r.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),r.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.node.min.mjs"),924:()=>import("./engine-wllama.node.min.mjs")},t={792:0},a=e=>{let{__webpack_esm_ids__:a,__webpack_esm_modules__:n,__webpack_esm_runtime__:o}=e;var s,i,c=0;for(s in n)r.o(n,s)&&(r.m[s]=n[s]);for(o&&o(r);c<a.length;c++)i=a[c],r.o(t,i)&&t[i]&&t[i][0](),t[i]=0};r.f.j=(n,o)=>{let s=r.o(t,n)?t[n]:void 0;if(0!==s)if(s)o.push(s[1]);else{let r=e[n]().then(a,e=>{throw 0!==t[n]&&(t[n]=void 0),e});r=Promise.race([r,new Promise(e=>s=t[n]=[e])]),o.push(s[1]=r)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(s||(s={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(i||(i={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(c||(c={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(l||(l={})),function(e){e.download="download",e.load="load"}(d||(d={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(g||(g={}));const u=[{id:g.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:g.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],m=e=>e?u.find(t=>t.id==e):void 0,f=e=>!!e.cpu?.mmproj||!!e.litert?.vision,p=()=>{return n=t(e(a("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/wllama"),{wasm:t(n,"wllama.wasm"),compatWasm:t(n,"compat/wllama.wasm"),compatWorker:t(n,"compat/wllama.js")};var n},w=()=>{return n=t(e(a("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/litert"),{script:t(n,"litertlm_wasm_internal.js"),wasm:t(n,"litertlm_wasm_internal.wasm")};var n},h=288,y=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),a=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!a.includes(";base64"))return await(await fetch(e)).arrayBuffer();const n=atob(e.slice(t+1)),o=new Uint8Array(n.length);for(let e=0;e<n.length;e++)o[e]=n.charCodeAt(e);return o.buffer},b=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),a=await createImageBitmap(await t.blob()),n=Math.min(a.width,a.height),o=Math.floor((a.width-n)/2),r=Math.floor((a.height-n)/2),s=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const a=document.createElement("canvas");return a.width=e,a.height=t,a})(h,h),i=s?s.getContext("2d"):null;if(!s||!i||n<1)return a.close(),e;if(i.drawImage(a,o,r,n,n,0,0,h,h),a.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(s)){const t=await s.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const a=new FileReader;a.onerror=()=>t(""),a.onload=()=>t(`${a.result}`),a.readAsDataURL(e)}))(t)||e}return s.toDataURL("image/jpeg",.85)}catch{return e}},v={maxReplyTokens:200,maxThinkingTokens:64,maxReasoningTokens:150,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:2},x=(e,t,a)=>"number"==typeof e&&Number.isFinite(e)&&e>=a?Math.floor(e):t,k=(e={})=>{const t=v;return{maxReplyTokens:x(e.maxReplyTokens,t.maxReplyTokens,1),maxThinkingTokens:x(e.maxThinkingTokens,t.maxThinkingTokens,1),maxReasoningTokens:x(e.maxReasoningTokens,t.maxReasoningTokens,1),maxImages:x(e.maxImages,t.maxImages,0),maxContentLength:x(e.maxContentLength,t.maxContentLength,1),promptCharBudget:x(e.promptCharBudget,t.promptCharBudget,1),maxThreads:x(e.maxThreads,t.maxThreads,1)}},_=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),M=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,T=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),C=["<|channel>thought\n","<|channel>thought","..."],L=["<channel|>","..."],j=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,P=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,$=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const a=[];for(const n of[...P(e),{type:"text",text:"\n"},...P(t)]){const e=a[a.length-1];"text"==n.type&&"text"==e?.type?e.text+=n.text:a.push(n)}return a},R=async({model:e,logging:t,onReason:a,onText:n,start:o,stopped:r})=>{let s="",i="",c="";const l=((e,t)=>{let a="",n=!1;const o=(e,t)=>{let a=-1,n="";for(const o of t){const t=e.indexOf(o);t>=0&&(a<0||t<a)&&(a=t,n=o)}return{at:a,tag:n}},r=(e,t)=>{let a="";for(const n of t)for(let t=Math.min(e.length,n.length-1);t>a.length;t--)if(e.endsWith(n.slice(0,t))){a=e.slice(e.length-t);break}return a};return{feed:s=>{let i=a+s;for(a="";i;){if(n){const{at:t,tag:s}=o(i,L);if(t>=0){t&&e(i.slice(0,t)),i=i.slice(t+s.length),n=!1;continue}const c=r(i,L);e(i.slice(0,i.length-c.length)),a=c;break}{const{at:e,tag:s}=o(i,C);if(e>=0){e&&t(i.slice(0,e)),i=i.slice(e+s.length),i.startsWith("\n")&&(i=i.slice(1)),n=!0;continue}const c=r(i,C);t(i.slice(0,i.length-c.length)),a=c;break}}},flush:()=>{a&&(n?e(a):t(a),a="")}}})(e=>{i+=e,a(e)},e=>{s+=e,n(e)});try{const e=await o();for await(const t of e){if(r())break;const e=t?.choices?.[0],n=e?.delta,o="string"==typeof n?.reasoning_content?n.reasoning_content:"string"==typeof n?.reasoning?n.reasoning:"";e?.finish_reason&&(c=e.finish_reason),o&&(i+=o,a(o)),n?.content&&l.feed(n.content)}l.flush()}catch(e){t&&console.log("Local chat failed",e)}const d=r()?void 0:s||void 0;return t&&(console.log("Local chat reply",e.id,d?"****":""),console.log("Local chat finish",e.id,c||"none",s.length,i.length)),d},W=async({model:e,params:t,start:a,stopped:n})=>{const o=await(async e=>{const t=[];for(const a of e){if("string"==typeof a.content){t.push(a);continue}const e=[];for(const t of a.content)e.push("image"==t.type?{type:"image",url:await b(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=v)=>{const a=t.maxContentLength,n=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=s.user||!e.images?.length)),o=[],r=[];let i=0,c=t.maxImages;for(let e=n.length-1;e>=0;e--){const l=n[e],d=("string"==typeof l.content?l.content:"").slice(0,a),g=l.role==s.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&i+d.length>t.promptCharBudget)break;i+=d.length,c-=g.length,l.role==s.system?o.unshift(d):l.role==s.assistant?r.unshift({role:"assistant",content:d}):r.unshift({role:"user",content:j(d,g)})}}for(;r.length&&"user"!=r[0].role;)r.shift();const l=[];for(const e of r){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=$(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=o.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,o),await R({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,start:()=>a(o),stopped:n})},S=33554432,z=new Map;let B=!1;const q=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let a=0;for(let t=0;t<e.length;t++)a=31*a+e.charCodeAt(t)>>>0;return`${a.toString(16)}-${t}`},A=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},O=async()=>{if(!B)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else B=!0},N=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},F=async e=>{if(e)try{const t=await e.getFile(),a=JSON.parse(await t.text());return{total:Number(a.total)||0,etag:`${a.etag||""}`,complete:!0===a.complete}}catch{return}},I=async(e,t)=>{try{const a=await e.getFileHandle("meta.json",{create:!0}),n=await a.createWritable();await n.write(JSON.stringify(t)),await n.close()}catch{}},U=async function*(e){for await(const t of e)yield t},D=async e=>{try{for await(const[t,a]of U(e))"file"==a.kind&&A(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},H=async(e,t)=>{try{const a=await(await e.getFileHandle(t)).getFile(),n=await F(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!n)return;return{file:a,size:a.size,total:n.total,etag:n.etag,complete:n.total?a.size>=n.total:!0===n.complete}}catch{return}},Q=async(e,t)=>{const a=await N(e,t);if(a){const e=await(async e=>await F(await e.getFileHandle("meta.json").catch(()=>{})))(a),t=await(async e=>{const t=[];try{for await(const[a,n]of U(e)){const e=A(a);if("file"!=n.kind||e<0)continue;const o=await n.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const a=(e=>{const t=[];let a=0;for(const n of[...e].sort((e,t)=>e.start-t.start))if(n.size){if(n.start!=a)break;t.push(n),a+=n.size}return t})(t.map(e=>e.part)),n=new Set(a.map(e=>e.start)),o=t.filter(e=>n.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:a.reduce((e,t)=>e+t.size,0)}})(a),n=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:n&&o>n?r.slice(0,n):r,size:o,total:n,etag:e?.etag||"",complete:n?o>=n:!0===e?.complete}}return await H(e,t)},G=(e,t)=>{const a=e.headers.get("content-range"),n=/bytes (\d+)-/.exec(a||""),o=/\/(\d+)\s*$/.exec(a||""),r=e.headers.get("etag")||"";if(206==e.status){const e=n?Number(n[1]):t.size,a=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&a&&t.total!=a)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:a||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},V=async({dir:e,start:t,skip:a,total:n,body:o,onProgress:r,onChunk:s})=>{let i,c=t,l=0,g=a;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(c),{create:!0});i=await t.createWritable(),l=0},m=async()=>{const e=i;i=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:a,value:o}=await e.read();if(a)break;s();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){i||await u();const e=Math.min(S-l,o.byteLength-f);await i.write({type:"write",position:l,data:o.subarray(f,f+e)}),f+=e,l+=e,c+=e,l==S&&await m()}c>t&&r(n?c/n:0,d.download)}await m()}catch(e){throw await m(),e}return c},J=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),K=e=>{const t=async()=>{const t=await O();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const a=await t();return a?await Q(a,q(e)):void 0},download:async({url:a,signal:n,onProgress:o})=>{const r=`${e}|${a}`,s=z.get(r);if(s)return await s;const i=await t();if(!i)return;const c=(async(e,t,a,n)=>{const o=q(t),r=await H(e,o);if(r?.complete)return r;let s=await N(e,o);s||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),s=await e.getDirectoryHandle(o,{create:!0}));let i=0,c=0;for(;;){if(a?.aborted)throw new Error("Cancelled");const r=await Q(e,o);if(r?.complete)return r;const l=r?.size??0,g=r?.total||0,u=r?.etag||"";if(l>i)i=l,c=0;else if(++c>4)throw new Error("Model download failed");l&&g&&n(l/g,d.download);const m=g?Math.min(l+S,g)-1:l+S-1,f=new AbortController,p=()=>f.abort(),w={size:l,total:g,etag:u};let h,y=setTimeout(p,3e4);a?.addEventListener("abort",p,{once:!0});try{if(h=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${l}-${m}`}}),416==h.status){if(await(h.body?.cancel().catch(()=>{})),g&&l>=g){await I(s,{total:g,etag:u,complete:!0});continue}await D(s),await I(s,{total:0,etag:"",complete:!1});continue}if(!h.ok||!h.body)throw await(h.body?.cancel().catch(()=>{})),new Error("Model download failed");const a=G(h,w);if(!a){await h.body.cancel().catch(()=>{}),await D(s),await I(s,{total:0,etag:"",complete:!1});continue}await I(s,{total:a.total,etag:a.etag,complete:!1});const r=0==a.start?l:0,i=await V({dir:s,start:a.start+r,skip:r,total:a.total,body:h.body,onProgress:n,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!a.total){await I(s,{total:i,etag:a.etag,complete:!0});const t=await Q(e,o);if(t)return t}}catch(e){if(a?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*c))}finally{clearTimeout(y),a?.removeEventListener("abort",p)}}})(i,a,n,o).finally(()=>{z.delete(r)});return z.set(r,c),await c},remove:async e=>{const a=await t();a&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(a,q(e))},purge:async()=>{const t=await O();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},X="litertlm-models",Y=K("litert-store"),Z={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0},ee=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(X)}catch{return}},te=new Map,ae=(e,t,a)=>{let n=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{n+=e.byteLength,a(t?n/t:0,d.download),o.enqueue(e)}}))},ne=async e=>{const t=[];for(const a of e){if("string"==typeof a.content){t.push({role:a.role,content:a.content});continue}const e=[];for(const t of a.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await y(t.url)});t.push({role:"user",content:e})}return t},oe=async({engine:e,model:t,mod:a,params:n})=>await W({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async o=>{if(n.signal?.aborted)throw new Error("Cancelled");if(T(o)&&!t.litert?.vision)throw new Error("Model does not accept images");return async function*({engine:e,mod:t,params:a,messages:n}){const o=n[n.length-1],r=_(a.limits,a.reasoning);if("user"!=o?.role)throw new Error("No user turn");const s=await ne(n.slice(0,-1)),i=(await ne([o]))[0];if(!i)throw new Error("Empty prompt");const c=await e.createConversation({sessionConfig:{maxOutputTokens:r.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:r.temperature,k:40,p:.95}},preface:{...s.length?{messages:s}:{},extra_context:{enable_thinking:a.reasoning}}});let l;const d=()=>{l&&l.cancel().catch(()=>{});try{c.cancel()}catch{}};a.signal?.addEventListener("abort",d,{once:!0});try{if(a.signal?.aborted)return;const e=c.sendMessageStreaming(i).getReader();for(l=e;;){const{done:t,value:a}=await e.read();if(t)break;const n={};let o="string"==typeof a.content?a.content:"";if(Array.isArray(a.content))for(const e of a.content)"text"==e.type&&(o+=e.text);o&&(n.content=o);const r=a.channels?.thought;r&&(n.reasoning=r),(n.content||n.reasoning)&&(yield{choices:[{delta:n}]})}}finally{a.signal?.removeEventListener("abort",d);try{await c.delete()}catch{}}}({engine:e,mod:a,params:n,messages:o})}}),re={backend:l.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:n,logging:o})=>{const s=e.litert;if(!s)throw new Error("Model has no LiteRT artifact");const i=await r.e(520).then(()=>r(507));if(await i.getOrLoadGlobalLiteRtLm(w().script),n?.aborted)throw new Error("Cancelled");const c=M(s.repo,s.file),g=await ee();let u;const m=g?await g.match(c):void 0;if(m?.body)u=m.body;else{const e=await Y.stored(c);e?.complete&&(u=e.file.stream())}if(!u){const e=await Y.download({url:c,signal:n,onProgress:a}).catch(e=>{if(n?.aborted)throw new Error("Cancelled");if(!J(e))throw e;o&&console.log("Local model store failed",e)});e&&(u=e.file.stream())}if(!u&&g)try{await(async(e,t,a,n)=>{const o=te.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:a});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),s=ae(o.body,r,n);await t.put(e,new Response(s,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{te.delete(e)});return te.set(e,r),await r})(c,g,n,a);const e=await g.match(c);e?.body&&(u=e.body)}catch(e){if(n?.aborted)throw new Error("Cancelled");o&&console.log("Local model cache failed",e)}if(u||(u=await(async(e,t,a)=>{const n=await fetch(e,{signal:t});if(!n.ok||!n.body)throw new Error("Model download failed");return ae(n.body,E(n),a)})(c,n,a)),n?.aborted)throw new Error("Cancelled");a(1,d.load);const f=await i.Engine.create({model:u,mainExecutorSettings:{maxNumTokens:4096,...s.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:Z},...s.vision?{visionBackend:i.Backend.GPU}:{}});if(n?.aborted)throw await f.delete(),new Error("Cancelled");return{model:e,backend:l.gpu,contextSize:4096,chat:t=>oe({engine:f,model:e,mod:i,params:t}),unload:async()=>{await f.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const a=M(t.repo,t.file),[n,o]=await Promise.all([ee(),Y.stored(a)]);if(o?.complete)return!0;try{return!(!n||!await n.match(a))}catch{return!1}},hasCache:async()=>{try{for(const e of u){const t=e.litert;if(!t)continue;if(await Y.stored(M(t.repo,t.file)))return!0}const e=await ee();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const a=M(t.repo,t.file),n=await ee();await Y.remove(a),n&&await n.delete(a)}catch{}},purge:async()=>{try{await Y.purge(),"undefined"!=typeof caches&&await caches.delete(X)}catch{}}},se=K("wllama-store");let ie,ce,le="",de=!1;const ge=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ue=e=>{const t=p();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},me=e=>{const t=e=>(...t)=>{de&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},fe=async()=>{if(ie)return ie;const e=await r.e(924).then(()=>r(577)),t=ce||ue(),a=new e.Wllama({default:t.wasm},{logger:me(e.LoggerWithoutDebug)});return ce||(ce=t,le=ge(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),ie=a,a},pe=async()=>{const e=ie;ie=void 0;try{await(e?.exit())}catch{}},we=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),he=e=>{const t=e.cpu;return t?[{name:t.file,url:M(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:M(t.repo,t.mmproj.file)}]:[]]:[]},ye=async e=>{const t=he(e);if(!t.length)return!1;for(const e of t){const t=await se.stored(e.url);if(!t?.complete)return!1}return!0},be=async(e,t)=>{const a=await e.cacheManager.list();return he(t).every(e=>a.some(t=>we(t,e.name)))},ve=async({model:e,signal:t,onProgress:a})=>{const n=he(e),o=[];for(let e=0;e<n.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await se.download({url:n[e].url,signal:t,onProgress:(t,o)=>a((e+t)/n.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},xe=async e=>{const t=[];for(const a of e){if("string"==typeof a.content){"system"==a.role?t.push({role:"system",content:a.content}):"assistant"==a.role?t.push({role:"assistant",content:a.content}):t.push({role:"user",content:a.content});continue}const e=[];for(const t of a.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await y(t.url)});t.push({role:"user",content:e})}return t},ke={backend:l.cpu,startLLM:async({model:e,limits:t,onProgress:a,signal:n,logging:o,assets:r})=>{de=!0===o;const s=e.cpu;if(!s)throw new Error("Model has no CPU artifact");const i=ue(r),c=ge(i);ie&&c!=le&&await pe(),ce=i,le=c;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await fe(),p=await ye(e),w=s.mmproj?.file,h=M(s.repo,s.file),y=w?M(s.repo,w):"";if(n?.aborted)throw await pe(),new Error("Cancelled");if(!p&&w){const e=await f.cacheManager.list(),t=e.some(e=>we(e,s.file)),a=e.some(e=>we(e,w));t&&!a&&await f.cacheManager.deleteMany(e=>we(e,s.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await ve({model:e,signal:n,onProgress:a}):void 0;t||p||await be(f,e).catch(()=>!1)||(t=await ve({model:e,signal:n,onProgress:a}).catch(e=>{if(n?.aborted)throw new Error("Cancelled");if(!J(e))throw e;o&&console.log("Local model store failed",e)})),t?(a(1,d.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:h,mmprojUrl:y}:h,{...b,useCache:!0,signal:n,progressCallback:({loaded:e,total:t})=>{const n=t?e/t:0;a(n,n>=1?d.load:d.download)}})}catch(e){throw await pe(),e}if(n?.aborted)throw await pe(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await pe(),new Error("Failed to create model context");return{model:e,backend:l.cpu,contextSize:8192,chat:t=>(async(e,t,a)=>{if(de=!0===a.logging,e.isModelLoaded())return await W({model:t,params:a,stopped:()=>!!a.signal?.aborted,start:async n=>{if(T(n)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await xe(n),..._(a.limits,a.reasoning),abortSignal:a.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:a.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(ie==f)await pe();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await ye(e)||await be(await fe(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of u.flatMap(e=>he(e)))if(await se.stored(e.url))return!0;const e=await fe();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=he(e);try{for(const e of t)await se.remove(e.url)}catch{}try{const e=await fe();await e.cacheManager.deleteMany(e=>t.some(t=>we(e,t.name)))}catch{}},purge:async()=>{try{await se.purge()}catch{}try{const e=await fe();await e.cacheManager.clear()}catch{}}},_e=((e={})=>{const t=e.engines?.cpu||ke,a=e.engines?.litert||re;let n,o,r,s,i,g,m,f=l.none,p=c.idle,w=[],h=0,y={percent:0,phase:d.download};const b=n=>{if(e.backend!=l.none)return e.backend==l.cpu?n.cpu?t:void 0:e.backend==l.gpu?n.litert?a:void 0:n.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?a:!n.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||d.download};for(const e of[...w])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{h++,g?.abort(),g=void 0,i=void 0,p==c.downloading&&x(c.idle)},_=(e,t)=>{const a=s;return a?new Promise(n=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),n(e))}},i=()=>{(e=>{w=w.filter(t=>t!=e),queueMicrotask(()=>{!w.length&&s&&k()})})(r),r.settle(!1)};w.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),a.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==c.ready&&!!n&&!!o;return{load:async({model:e,limits:t,assets:a,logging:l,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(s&&i==e.id)return p(y.percent,y.phase),await _(p,m);s&&(k(),w=[],await s.catch(()=>!1));const E=++h,T=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const s=b(e);if(E!=h)return!1;if(!s)return x(c.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,a=n;n=void 0,o=void 0,r=void 0,await(a?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const a of u)a.id!=e.id&&b(a)==t&&await t.remove(a)})(e,s),E!=h)return!1;const i=await s.cached(e);if(E!=h)return!1;x(c.downloading),v(0,d.download),i&&v(1,d.load);const m=new AbortController;g=m;try{const i=await s.startLLM({model:e,limits:t,logging:l,signal:m.signal,assets:a,onProgress:(e,t)=>{E==h&&v(e,t)}});return E!=h||m.signal.aborted?(await i.unload(),!1):(n=i,o=e,r=s,f=s.backend,x(c.ready),v(1,d.load),!0)}catch(e){return E!=h||(n=void 0,o=void 0,r=void 0,l&&console.log("Local model failed",e),x(c.error)),!1}finally{g==m&&(g=void 0)}})().finally(()=>{s==T&&(s=void 0,i=void 0,w=[])});return s=T,i=e.id,await _(p,m)},chat:async e=>{if(!M()||!n)return;for(;m;)await m.catch(()=>{});if(!M()||!n)return;const t=n.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=n;k(),n=void 0,o=void 0,r=void 0,x(c.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==c.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Me=(e,t,a)=>{const n=e.status(),o=m(t);return{availability:n==c.ready?i.online:n==c.error||n==c.unsupported?i.offline:i.away,status:n,backend:e.backend(),supportsImages:!!o&&f(o),ready:e.ready(),loading:e.loading(),requesting:a.requesting,phase:a.phase,percent:a.percent,...a.error?{error:a.error}:{}}},Ee=(e=>{let t,a=0;const n=t=>t.input.onStatus?.(Me(e,t.input.model,t));return o=>{t?.controller.abort();const r={token:++a,input:o,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?d.load:d.download,requesting:!1,cancelled:!1};return t=r,n(r),(async t=>{const{input:o}=t,r=!0===o.logging,i=m(o.model),c=o.messages.some(e=>e.role!=s.system&&"string"==typeof e.content&&!!e.content.trim()),l=o.messages.some(e=>e.role==s.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=c||l;if(!i)return t.error="Unknown model",void n(t);if(!g)return void n(t);if(l&&!c&&!f(i))return t.error="Model does not accept images",void n(t);const u=k(o.limits),p=await e.load({model:i,limits:u,logging:r,signal:t.controller.signal,onProgress:(e,o)=>{t.token==a&&(t.percent=Math.round(100*e),t.phase=o,n(t))}}).catch(e=>(t.token==a&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=a)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void n(t);const w=f(i),h=l&&!w?o.messages.map(({images:e,...t})=>t):o.messages;if(l&&!w&&!c)return t.error="Model does not accept images",void n(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=d.load,n(t);let y,b="",v="";try{y=await e.chat({messages:h,limits:u,reasoning:!0===o.reasoning,logging:r,onReason:e=>{t.token!=a||t.cancelled||(b+=e,o.onThinking?.(b))},onText:e=>{t.token!=a||t.cancelled||(v+=e,o.onResponse?.(v))},signal:t.controller.signal})}catch(e){if(t.token!=a)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void n(t)}t.token==a&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),n(t))})(r),()=>(t=>{t.token==a&&(t.cancelled=!0,t.controller.abort(),e.status()!=c.ready?queueMicrotask(()=>n(t)):e.unload().then(()=>n(t)))})(r)}})(_e),Te=(Ce=_e,async e=>{const t=m(e.model),a=t=>e.onStatus?.(Me(Ce,e.model,t));if(!t)return a({requesting:!1,phase:d.download,percent:0,error:"Unknown model"}),!1;let n,o=0,r=d.download;const s=Ce.ready()&&Ce.model()?.id==t.id;return await Ce.load({model:t,limits:k(e.limits),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,a({requesting:!1,phase:r,percent:o})}}).catch(e=>(n=`${e?.message||`${e}`}`,!1))?(s&&a({requesting:!1,phase:d.load,percent:100}),!0):(a({requesting:!1,phase:r,percent:o,error:n||"Model not ready"}),!1)});var Ce;export{g as ChatModelId,i as ChatStatus,l as LlmBackend,d as LlmPhase,s as LlmRole,c as LlmStatus,Ee as createChat,Te as preloadModel};
+import{dirname as e,join as t}from"node:path";import{fileURLToPath as n}from"node:url";var a={};const o={};function r(e){const t=o[e];if(void 0!==t)return t.exports;const n=o[e]={exports:{}};return a[e](n,n.exports,r),n.exports}var s,i,c,l,d,g;r.m=a,r.d=(e,t)=>{for(var n in t)r.o(t,n)&&!r.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},r.f={},r.e=e=>{const t=[];return r.f.j(e,t),Promise.all(t)},r.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),r.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.node.min.mjs"),924:()=>import("./engine-wllama.node.min.mjs")},t={792:0},n=e=>{let{__webpack_esm_ids__:n,__webpack_esm_modules__:a,__webpack_esm_runtime__:o}=e;var s,i,c=0;for(s in a)r.o(a,s)&&(r.m[s]=a[s]);for(o&&o(r);c<n.length;c++)i=n[c],r.o(t,i)&&t[i]&&t[i][0](),t[i]=0};r.f.j=(a,o)=>{let s=r.o(t,a)?t[a]:void 0;if(0!==s)if(s)o.push(s[1]);else{let r=e[a]().then(n,e=>{throw 0!==t[a]&&(t[a]=void 0),e});r=Promise.race([r,new Promise(e=>s=t[a]=[e])]),o.push(s[1]=r)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(s||(s={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(i||(i={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(c||(c={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(l||(l={})),function(e){e.download="download",e.load="load"}(d||(d={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(g||(g={}));const u=[{id:g.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:g.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],m=e=>e?u.find(t=>t.id==e):void 0,f=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?u.find(e=>!!e.litert):u.find(e=>!!e.cpu))?.id,p=async e=>m(e||await f()),h=e=>!!e.cpu?.mmproj||!!e.litert?.vision,w=()=>{return a=t(e(n("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/wllama"),{wasm:t(a,"wllama.wasm"),compatWasm:t(a,"compat/wllama.wasm"),compatWorker:t(a,"compat/wllama.js")};var a},y=()=>{return a=t(e(n("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/litert"),{script:t(a,"litertlm_wasm_internal.js"),wasm:t(a,"litertlm_wasm_internal.wasm")};var a},b=288,v=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},x=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),s=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(b,b),i=s?s.getContext("2d"):null;if(!s||!i||a<1)return n.close(),e;if(i.drawImage(n,o,r,a,a,0,0,b,b),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(s)){const t=await s.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return s.toDataURL("image/jpeg",.85)}catch{return e}},k={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},_=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,M=(e={},t=!1)=>{const n=k;return{maxReplyTokens:_(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:_(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:_(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:_(e.maxImages,n.maxImages,0),maxContentLength:_(e.maxContentLength,n.maxContentLength,1),promptCharBudget:_(e.promptCharBudget,n.promptCharBudget,1),maxThreads:_(e.maxThreads,n.maxThreads,1)}},T=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),E=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,C=e=>Number(e.headers.get("content-length"))||0,L=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),P=["<|channel>thought\n","<|channel>thought","..."],j=["<channel|>","..."],S=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,$=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,R=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...$(e),{type:"text",text:"\n"},...$(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class W extends Error{}const q=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new W(n)),t)})])}finally{a&&clearTimeout(a)}},z=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},B=async({model:e,logging:t,onReason:n,onText:a,onSpeed:o,start:r,stopped:s,firstTimeoutMs:i=18e4,idleTimeoutMs:c=3e4,onStall:l})=>{let d="",g="",u="",m=!1,f=0,p=0;const h=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:s=>{let i=n+s;for(n="";i;){if(a){const{at:t,tag:s}=o(i,j);if(t>=0){t&&e(i.slice(0,t)),i=i.slice(t+s.length),a=!1;continue}const c=r(i,j);e(i.slice(0,i.length-c.length)),n=c;break}{const{at:e,tag:s}=o(i,P);if(e>=0){e&&t(i.slice(0,e)),i=i.slice(e+s.length),i.startsWith("\n")&&(i=i.slice(1)),a=!0;continue}const c=r(i,P);t(i.slice(0,i.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{g+=e,n(e)},e=>{d+=e,a(e)});try{const e=(await q(r(),i,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(s()){z(e);break}let a;try{a=await q(e.next(),m?c:i,m?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof W&&(d||g)){l?.(),z(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;m=!0;const r=a.value?.choices?.[0],w=r?.delta,y="string"==typeof w?.reasoning_content?w.reasoning_content:"string"==typeof w?.reasoning?w.reasoning:"";if(r?.finish_reason&&(u=r.finish_reason),y&&(g+=y,n(y)),w?.content&&h.feed(w.content),y||w?.content){const e=performance.now();f||(p=e),f+=1,f>1&&e>p&&o?.(1e3*f/(e-p))}}h.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof W&&!s())throw e}const w=s()?void 0:d||void 0;return t&&(console.log("Local chat reply",e.id,w?"****":""),console.log("Local chat finish",e.id,u||"none",d.length,g.length)),w},I=async({model:e,params:t,start:n,stopped:a,onStall:o})=>{const r=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await x(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=k)=>{const n=t.maxContentLength,a=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=s.user||!e.images?.length)),o=[],r=[];let i=0,c=t.maxImages;for(let e=a.length-1;e>=0;e--){const l=a[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==s.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&i+d.length>t.promptCharBudget)break;i+=d.length,c-=g.length,l.role==s.system?o.unshift(d):l.role==s.assistant?r.unshift({role:"assistant",content:d}):r.unshift({role:"user",content:S(d,g)})}}for(;r.length&&"user"!=r[0].role;)r.shift();const l=[];for(const e of r){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=R(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=o.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,r),await B({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,onSpeed:t.onSpeed,start:()=>n(r),stopped:a,onStall:o})},A=33554432,O=new Map;let N=!1;const F=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},U=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},D=async()=>{if(!N)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else N=!0},H=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},Q=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},V=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},G=async function*(e){for await(const t of e)yield t},J=async e=>{try{for await(const[t,n]of G(e))"file"==n.kind&&U(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},K=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await Q(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},X=async(e,t)=>{const n=await H(e,t);if(n){const e=await(async e=>await Q(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of G(e)){const e=U(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await K(e,t)},Y=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:C(e)||t.total,etag:r||t.etag}},Z=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:s})=>{let i,c=t,l=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(c),{create:!0});i=await t.createWritable(),l=0},m=async()=>{const e=i;i=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;s();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){i||await u();const e=Math.min(A-l,o.byteLength-f);await i.write({type:"write",position:l,data:o.subarray(f,f+e)}),f+=e,l+=e,c+=e,l==A&&await m()}c>t&&r(a?c/a:0,d.download)}await m()}catch(e){throw await m(),e}return c},ee=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),te=e=>{const t=async()=>{const t=await D();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await X(n,F(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,s=O.get(r);if(s)return await s;const i=await t();if(!i)return;const c=(async(e,t,n,a)=>{const o=F(t),r=await K(e,o);if(r?.complete)return r;let s=await H(e,o);s||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),s=await e.getDirectoryHandle(o,{create:!0}));let i=0,c=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await X(e,o);if(r?.complete)return r;const l=r?.size??0,g=r?.total||0,u=r?.etag||"";if(l>i)i=l,c=0;else if(++c>4)throw new Error("Model download failed");l&&g&&a(l/g,d.download);const m=g?Math.min(l+A,g)-1:l+A-1,f=new AbortController,p=()=>f.abort(),h={size:l,total:g,etag:u};let w,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(w=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${l}-${m}`}}),416==w.status){if(await(w.body?.cancel().catch(()=>{})),g&&l>=g){await V(s,{total:g,etag:u,complete:!0});continue}await J(s),await V(s,{total:0,etag:"",complete:!1});continue}if(!w.ok||!w.body)throw await(w.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=Y(w,h);if(!n){await w.body.cancel().catch(()=>{}),await J(s),await V(s,{total:0,etag:"",complete:!1});continue}await V(s,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?l:0,i=await Z({dir:s,start:n.start+r,skip:r,total:n.total,body:w.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await V(s,{total:i,etag:n.etag,complete:!0});const t=await X(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*c))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(i,n,a,o).finally(()=>{O.delete(r)});return O.set(r,c),await c},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,F(e))},purge:async()=>{const t=await D();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},ne="litertlm-models",ae=te("litert-store"),oe={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0};let re=!1;const se=(...e)=>{re&&console.log(...e)},ie=(...e)=>{re&&console.error(...e)},ce=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(ne)}catch{return}},le=new Map,de=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,d.download),o.enqueue(e)}}))},ge=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await v(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ue=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},me=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],s=T(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const i=a.slice(0,-1),c=(await ge([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==s.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ue(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const s=o.length-r.length;for(let e=0;e<r.length;e++)if(!ue(o[s+e],r[e]))return!1;return!0})(o.sent,i);let d;if(l&&o.conversation)d=o.conversation;else{const a=await ge(i,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:s.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:s.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=s.max_tokens,o.sent=[...i]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},fe={backend:l.gpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o})=>{re=!0===o;const s=e.litert;if(!s)throw new Error("Model has no LiteRT artifact");const i=await r.e(520).then(()=>r(507)),c=globalThis,g=c.Module;c.Module={print:se,printErr:ie};try{await i.getOrLoadGlobalLiteRtLm(y().script)}finally{c.Module=g}if(a?.aborted)throw new Error("Cancelled");const u=E(s.repo,s.file),m=await ce();let f;const p=m?await m.match(u):void 0;if(p?.body)f=p.body;else{const e=await ae.stored(u);e?.complete&&(f=e.file.stream())}if(!f){const e=await ae.download({url:u,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!ee(e))throw e;o&&console.log("Local model store failed",e)});e&&(f=e.file.stream())}if(!f&&m)try{await(async(e,t,n,a)=>{const o=le.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=C(o),s=de(o.body,r,a);await t.put(e,new Response(s,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{le.delete(e)});return le.set(e,r),await r})(u,m,a,n);const e=await m.match(u);e?.body&&(f=e.body)}catch(e){if(a?.aborted)throw new Error("Cancelled");o&&console.log("Local model cache failed",e)}if(f||(f=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return de(a.body,C(a),n)})(u,a,n)),a?.aborted)throw new Error("Cancelled");n(1,d.load);const h=await i.Engine.create((({model:e,limits:t,source:n,mod:a})=>({model:n,mainExecutorSettings:{maxNumTokens:4096,...e.litert?.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:oe},...e.litert?.vision?{visionBackend:a.Backend.GPU,maxVisionTokensPerImage:70,minVisionTokensPerImage:70}:{}}))({model:e,limits:t,source:f,mod:i}));if(a?.aborted)throw await h.delete(),new Error("Cancelled");const w={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:l.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{re=!0===a.logging;const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await I({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(L(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return me({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:h,model:e,mod:i,params:t,session:w}),unload:async()=>{w.conversation=void 0,w.sent=[],await h.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=E(t.repo,t.file),[a,o]=await Promise.all([ce(),ae.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of u){const t=e.litert;if(!t)continue;if(await ae.stored(E(t.repo,t.file)))return!0}const e=await ce();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=E(t.repo,t.file),a=await ce();await ae.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ae.purge(),"undefined"!=typeof caches&&await caches.delete(ne)}catch{}}},pe=te("wllama-store");let he,we,ye="",be=!1;const ve=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,xe=e=>{const t=w();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},ke=e=>{const t=e=>(...t)=>{be&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},_e=async()=>{if(he)return he;const e=await r.e(924).then(()=>r(577)),t=we||xe(),n=new e.Wllama({default:t.wasm},{logger:ke(e.LoggerWithoutDebug)});return we||(we=t,ye=ve(t)),n.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),he=n,n},Me=async()=>{const e=he;he=void 0;try{await(e?.exit())}catch{}},Te=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),Ee=e=>{const t=e.cpu;return t?[{name:t.file,url:E(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:E(t.repo,t.mmproj.file)}]:[]]:[]},Ce=async e=>{const t=Ee(e);if(!t.length)return!1;for(const e of t){const t=await pe.stored(e.url);if(!t?.complete)return!1}return!0},Le=async(e,t)=>{const n=await e.cacheManager.list();return Ee(t).every(e=>n.some(t=>Te(t,e.name)))},Pe=async({model:e,signal:t,onProgress:n})=>{const a=Ee(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await pe.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},je=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await v(t.url)});t.push({role:"user",content:e})}return t},Se={backend:l.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{be=!0===o;const s=e.cpu;if(!s)throw new Error("Model has no CPU artifact");const i=xe(r),c=ve(i);he&&c!=ye&&await Me(),we=i,ye=c;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await _e(),p=await Ce(e),h=s.mmproj?.file,w=E(s.repo,s.file),y=h?E(s.repo,h):"";if(a?.aborted)throw await Me(),new Error("Cancelled");if(!p&&h){const e=await f.cacheManager.list(),t=e.some(e=>Te(e,s.file)),n=e.some(e=>Te(e,h));t&&!n&&await f.cacheManager.deleteMany(e=>Te(e,s.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Pe({model:e,signal:a,onProgress:n}):void 0;t||p||await Le(f,e).catch(()=>!1)||(t=await Pe({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!ee(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,d.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:w,mmprojUrl:y}:w,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?d.load:d.download)}})}catch(e){throw await Me(),e}if(a?.aborted)throw await Me(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await Me(),new Error("Failed to create model context");return{model:e,backend:l.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(be=!0===n.logging,e.isModelLoaded())return await I({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(L(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await je(a),...T(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(he==f)await Me();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Ce(e)||await Le(await _e(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of u.flatMap(e=>Ee(e)))if(await pe.stored(e.url))return!0;const e=await _e();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=Ee(e);try{for(const e of t)await pe.remove(e.url)}catch{}try{const e=await _e();await e.cacheManager.deleteMany(e=>t.some(t=>Te(e,t.name)))}catch{}},purge:async()=>{try{await pe.purge()}catch{}try{const e=await _e();await e.cacheManager.clear()}catch{}}},$e=((e={})=>{const t=e.engines?.cpu||Se,n=e.engines?.litert||fe;let a,o,r,s,i,g,m,f=l.none,p=c.idle,h=[],w=0,y={percent:0,phase:d.download};const b=a=>{if(e.backend!=l.none)return e.backend==l.cpu?a.cpu?t:void 0:e.backend==l.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||d.download};for(const e of[...h])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{w++,g?.abort(),g=void 0,i=void 0,p==c.downloading&&x(c.idle)},_=(e,t)=>{const n=s;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{h=h.filter(t=>t!=e),queueMicrotask(()=>{!h.length&&s&&k()})})(r),r.settle(!1)};h.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==c.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:l,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(s&&i==e.id)return p(y.percent,y.phase),await _(p,m);s&&(k(),h=[],await s.catch(()=>!1));const T=++w,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const s=b(e);if(T!=w)return!1;if(!s)return x(c.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,n=a;a=void 0,o=void 0,r=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of u)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,s),T!=w)return!1;const i=await s.cached(e);if(T!=w)return!1;x(c.downloading),v(0,d.download),i&&v(1,d.load);const m=new AbortController;g=m;try{const i=await s.startLLM({model:e,limits:t,logging:l,signal:m.signal,assets:n,onProgress:(e,t)=>{T==w&&v(e,t)}});return T!=w||m.signal.aborted?(await i.unload(),!1):(a=i,o=e,r=s,f=s.backend,x(c.ready),v(1,d.load),!0)}catch(e){return T!=w||(a=void 0,o=void 0,r=void 0,l&&console.log("Local model failed",e),x(c.error)),!1}finally{g==m&&(g=void 0)}})().finally(()=>{s==E&&(s=void 0,i=void 0,h=[])});return s=E,i=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,r=void 0,x(c.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==c.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Re=(e,t,n)=>{const a=e.status(),o=m(t);return{availability:a==c.ready?i.online:a==c.error||a==c.unsupported?i.offline:i.away,status:a,backend:e.backend(),supportsImages:!!o&&h(o),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},We=(e=>{let t,n=0;const a=t=>t.input.onStatus?.(Re(e,t.model??t.input.model,t));return o=>{t?.controller.abort();const r={token:++n,input:o,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?d.load:d.download,requesting:!1,cancelled:!1};return t=r,a(r),(async t=>{const{input:o}=t,r=!0===o.logging,i=await p(o.model);if(t.model=i?.id,t.token!=n)return;const c=o.messages.some(e=>e.role!=s.system&&"string"==typeof e.content&&!!e.content.trim()),l=o.messages.some(e=>e.role==s.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=c||l;if(!i)return t.error="Unknown model",void a(t);if(!g)return void a(t);if(l&&!c&&!h(i))return t.error="Model does not accept images",void a(t);const u=M(o.limits,!!i.litert),m=await e.load({model:i,limits:u,logging:r,signal:t.controller.signal,onProgress:(e,o)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=o,a(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!m)return t.error||(t.error="Model not ready"),void a(t);const f=h(i),w=l&&!f?o.messages.map(({images:e,...t})=>t):o.messages;if(l&&!f&&!c)return t.error="Model does not accept images",void a(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=d.load,a(t);let y,b="",v="";try{y=await e.chat({messages:w,limits:u,reasoning:!0===o.reasoning,logging:r,onReason:e=>{t.token!=n||t.cancelled||(b+=e,o.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,o.onResponse?.(v))},onSpeed:e=>{t.token!=n||t.cancelled||o.onSpeed?.(e)},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void a(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),a(t))})(r),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=c.ready?queueMicrotask(()=>a(t)):e.unload().then(()=>a(t)))})(r)}})($e),qe=(ze=$e,async(e={})=>{const t=await p(e.model),n=n=>e.onStatus?.(Re(ze,t?.id,n));if(!t)return n({requesting:!1,phase:d.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=d.download;const s=ze.ready()&&ze.model()?.id==t.id;return await ze.load({model:t,limits:M(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(s&&n({requesting:!1,phase:d.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var ze;export{g as ChatModelId,i as ChatStatus,l as LlmBackend,d as LlmPhase,s as LlmRole,c as LlmStatus,We as createChat,qe as preloadModel};

package/dist/types.d.ts

@@ -35,7 +35,7 @@ interface LlmLimits {
     maxReplyTokens: number;
     /** Max chain-of-thought tokens, used by the CPU engine when it loads */
     maxThinkingTokens: number;
-    /** Total output budget when reasoning is on, thinking plus answer */
+    /** Total output budget when reasoning is on, thought plus answer */
     maxReasoningTokens: number;
     /** Max images sent in one prompt */
     maxImages: number;
@@ -66,8 +66,8 @@ interface ChatStatusSnapshot {
 }
 /** One chat request, invoked for every response */
 interface ChatInput {
-    /** On-device model that answers this request */
-    model: ChatModelId;
+    /** Model that answers, the GPU model when WebGPU is available, else the CPU one */
+    model?: ChatModelId;
     /** Complete prompt, including the system and user turns */
     messages: ChatMessage[];
     /** Override any token, prompt or thread budget */
@@ -80,6 +80,8 @@ interface ChatInput {
     onThinking?: (text: string) => void;
     /** Accumulated reply text, streamed while the model answers */
     onResponse: (text: string) => void;
+    /** Current generation speed in tokens per second, streamed while the model answers */
+    onSpeed?: (tokensPerSecond: number) => void;
     /** Model and request state, reported on every change */
     onStatus?: (snapshot: ChatStatusSnapshot) => void;
 }
@@ -105,7 +107,12 @@ interface LitertAssets {
 }
 /** Navigator with the WebGPU entry point, absent in Node */
 interface GpuNavigator extends Navigator {
-    gpu?: object;
+    gpu?: {
+        /** Ask the browser for a WebGPU adapter */
+        requestAdapter: (options?: {
+            powerPreference?: `low-power` | `high-performance`;
+        }) => Promise<unknown>;
+    };
 }
 /** Request state that builds one status snapshot */
 interface ChatState {
@@ -123,6 +130,8 @@ interface ActiveCall extends ChatState {
     input: ChatInput;
     controller: AbortController;
     cancelled: boolean;
+    /** Model resolved for this call */
+    model?: ChatModelId;
 }
 /** Engine and backend overrides used by tests */
 interface RuntimeDeps {
@@ -214,6 +223,8 @@ interface RuntimeChatParams {
     logging?: boolean;
     onReason: (text: string) => void;
     onText: (text: string) => void;
+    /** Current generation speed in tokens per second */
+    onSpeed?: (tokensPerSecond: number) => void;
     signal?: AbortSignal;
 }
 /** On-device runtime shared by every request */
@@ -241,6 +252,8 @@ interface EngineChatParams {
     onReason: (text: string) => void;
     /** Streamed reply delta */
     onText: (text: string) => void;
+    /** Current generation speed in tokens per second */
+    onSpeed?: (tokensPerSecond: number) => void;
     signal?: AbortSignal;
 }
 /** Params of the engine startLLM */
@@ -330,12 +343,37 @@ type LitertMessage = import('@litert-lm/core').Message;
 type LitertEngine = import('@litert-lm/core').Engine;
 /** The LiteRT-LM module namespace */
 type LitertModule = typeof import('@litert-lm/core');
+/** Settings handed to the LiteRT-LM engine */
+type LitertSettings = import('@litert-lm/core').EngineSettings;
+/** Params of one LiteRT-LM settings build */
+interface LitertSettingsParams {
+    /** Model with the artifact */
+    model: ChatModel;
+    /** Image and context budgets */
+    limits: LlmLimits;
+    /** Model file stream */
+    source: ReadableStream<Uint8Array>;
+    /** Loaded LiteRT module */
+    mod: LitertModule;
+}
+/** One reused LiteRT-LM conversation and the turns it already holds */
+interface LitertSession {
+    /** Live conversation, dropped when a turn cannot be continued */
+    conversation?: import('@litert-lm/core').Conversation;
+    /** Built turns already sent to the conversation */
+    sent: EngineMessage[];
+    /** Reasoning flag the conversation was created with */
+    reasoning: boolean;
+    /** Output token budget the conversation was created with */
+    maxTokens: number;
+}
 /** Params of one LiteRT-LM chat call */
 interface LitertChatParams {
     engine: LitertEngine;
     model: ChatModel;
     mod: LitertModule;
     params: EngineChatParams;
+    session: LitertSession;
 }
 /** Params of one LiteRT-LM stream */
 interface StreamLitertParams {
@@ -343,6 +381,7 @@ interface StreamLitertParams {
     mod: LitertModule;
     params: EngineChatParams;
     messages: EngineMessage[];
+    session: LitertSession;
 }
 /** Streamed delta, reasoning may be inline or structured */
 interface EngineDelta {
@@ -364,12 +403,20 @@ interface StreamReplyParams {
     onReason: (text: string) => void;
     /** Streamed reply delta */
     onText: (text: string) => void;
+    /** Current generation speed in tokens per second */
+    onSpeed?: (tokensPerSecond: number) => void;
     /** Open the engine stream for the prepared messages */
     start: () => Promise<AsyncIterable<unknown>>;
     /** Whether the caller stopped the generation */
     stopped: () => boolean;
     /** Whether engine logs reach the console */
     logging?: boolean;
+    /** Budget before the first chunk arrives, milliseconds */
+    firstTimeoutMs?: number;
+    /** Budget between two chunks, milliseconds */
+    idleTimeoutMs?: number;
+    /** Called when a slow stream keeps the text already delivered */
+    onStall?: () => void;
 }
 /** Params of one engine chat call */
 interface EngineRunParams {
@@ -379,5 +426,7 @@ interface EngineRunParams {
     start: (messages: EngineMessage[]) => Promise<AsyncIterable<unknown>>;
     /** Whether the caller stopped the generation */
     stopped: () => boolean;
+    /** Called when a slow stream keeps the text already delivered */
+    onStall?: () => void;
 }
-export type { ChatMessage, ChatModel, LlmLimits, ChatStatusSnapshot, ChatInput, ChatPreloadInput, ChatState, CancelChat, WllamaAssets, LitertAssets, GpuNavigator, ActiveCall, RuntimeDeps, RuntimeLoadParams, RuntimeLoadWaiter, RuntimeChatParams, StoredPart, StoredFilePart, StoredModel, StoredModelMeta, ResumePlan, StoreDownloadParams, ModelStore, Runtime, EngineChatParams, StartLlmParams, LlmRuntime, LlmEngine, EngineMessage, EngineContentPart, WllamaContentPart, WllamaMessage, LitertContentPart, LitertMessage, LitertEngine, LitertModule, LitertChatParams, StreamLitertParams, EngineDelta, EngineChunk, StreamReplyParams, EngineRunParams, };
+export type { ChatMessage, ChatModel, LlmLimits, ChatStatusSnapshot, ChatInput, ChatPreloadInput, ChatState, CancelChat, WllamaAssets, LitertAssets, GpuNavigator, ActiveCall, RuntimeDeps, RuntimeLoadParams, RuntimeLoadWaiter, RuntimeChatParams, StoredPart, StoredFilePart, StoredModel, StoredModelMeta, ResumePlan, StoreDownloadParams, ModelStore, Runtime, EngineChatParams, StartLlmParams, LlmRuntime, LlmEngine, EngineMessage, EngineContentPart, WllamaContentPart, WllamaMessage, LitertContentPart, LitertMessage, LitertEngine, LitertModule, LitertSettings, LitertSettingsParams, LitertSession, LitertChatParams, StreamLitertParams, EngineDelta, EngineChunk, StreamReplyParams, EngineRunParams, };

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@degreesign/chat",
-    "version": "0.3.5",
+    "version": "0.3.7",
     "description": "Headless on-device AI chat for browsers and Node with wllama CPU and LiteRT WebGPU engines",
     "main": "dist/node/chat.node.min.js",
     "module": "dist/browser/chat.min.mjs",