@degreesign/chat 0.3.6 → 0.3.7

package/README.md

@@ -98,6 +98,7 @@ const cancel = createChat({
   messages,
   onResponse: (text) => renderReply(text),
   // onThinking: (text) => renderThinking(text),
+  // onSpeed: (tokensPerSecond) => renderSpeed(tokensPerSecond),
   // onStatus: (snapshot) => renderStatus(snapshot),
 });
 
@@ -107,16 +108,22 @@ cancel();
 
 Without `model` the package picks the GPU model when the browser provides a
 WebGPU adapter and the CPU model when it does not. `limits`, `reasoning`,
-`onThinking` and `onStatus` are optional per call; see the API below.
+`onThinking`, `onSpeed` and `onStatus` are optional per call; see the API
+below.
 
 There is no promise and no controller object on `createChat`. Every result
 arrives through `onResponse`, and `onStatus` reports `requesting: false` when
-the reply ends. `preloadModel` is the one promise: it resolves once the model
-is loaded, or `false` when it cannot load.
+the reply ends. `onSpeed` reports the streamed generation speed in tokens per
+second, measured from the first token so prompt processing is excluded.
+`preloadModel` is the one promise: it resolves once the model is loaded, or
+`false` when it cannot load.
 
 Attach images to a user turn with `images: [dataUrl]`. The package sends them
 only to a model with vision support, center-crops each image to a square,
-resizes it to 288 x 288, and caps the number with `limits.maxImages`.
+resizes it to 288 x 288, and caps the number with `limits.maxImages`. The
+WebGPU engine asks LiteRT for the smallest Gemma 4 vision budget (70 tokens,
+630 patches), so it encodes the crop at 384 x 384 instead of upscaling it to
+768 x 768. The CPU engine uses the 288 crop directly.
 
 The package never writes storage, so the host appends the user message before
 the call and appends the assistant message after the last `onResponse`.
@@ -174,7 +181,19 @@ them to the `Exec` line of the browser entry under
 `--enable-unsafe-webgpu` is a different switch: it only unlocks software
 adapters, so an adapter whose `info.vendor` reads `google` is SwiftShader on the
 CPU, not the GPU. Remove it when measuring speed. Headless Chromium hides
-`navigator.gpu` entirely, so check the adapter in a normal window.
+`navigator.gpu` unless it starts with `--enable-unsafe-webgpu`,
+`--enable-features=Vulkan,WebGPU` and `--use-angle=vulkan`; check the adapter
+there too, because a missing adapter silently selects the CPU model.
+
+### GPU memory and images
+
+The WebGPU engine keeps the model, its KV cache and the vision encoder in GPU
+memory. A card that is already busy can lose the device (`VK_ERROR_DEVICE_LOST`)
+on the first image turn. The package requests the smallest Gemma 4 vision
+budget, 70 tokens and 630 patches, so the encoder runs one small signature and
+the crop is resized to 384 x 384 rather than the full 768 x 768. If the error
+still appears, close other GPU tabs, unload the model, or switch to the CPU
+model. A lost device needs a page or model reload; later calls cannot reuse it.
 
 ## Public API
 
@@ -197,6 +216,7 @@ requested model is already loaded, no download or load happens.
 | `logging?` | Whether package and engine console logs are written. Defaults to `false`. |
 | `onThinking?` | Receives the accumulated reasoning text as it streams. |
 | `onResponse` | Receives the accumulated reply text as it streams. Required. |
+| `onSpeed?` | Receives the current generation speed in tokens per second. |
 | `onStatus?` | Receives a `ChatStatusSnapshot` on every state change. |
 
 Without `model` the package picks the GPU build when the browser provides a
@@ -270,10 +290,9 @@ silence ends the stream and keeps the text already delivered.
 ## Logging
 
 Console output stays off by default. Pass `logging: true` in a call to receive
-the package's censored engine logs and the wllama engine's own logs for that
-call. The flag is per call, so a later call without it goes quiet again.
-LiteRT-LM writes its wasm diagnostics straight to the console and does not
-follow the flag.
+the package's censored engine logs, the wllama engine's own logs and the
+LiteRT-LM wasm diagnostics for that call. The flag is per call, so a later call
+without it goes quiet again.
 
 ## Models
 
@@ -323,7 +342,9 @@ user:      {next turn}
 - Images are collected from the newest turns first and capped by `maxImages`
   (default 2). Older image parts are dropped and their text stays.
 - Every image part is center-cropped to a square, then resized to 288 x 288.
-  Non-square edges are discarded and smaller images are upscaled.
+  Non-square edges are discarded and smaller images are upscaled. The WebGPU
+  engine is told to use a 70-token vision budget, so it encodes the crop at
+  384 x 384, the smallest Gemma 4 encoder signature.
 - Image payloads do not count against `promptCharBudget`.
 - An image-only turn with a text model reports `Model does not accept images`.
 
@@ -498,8 +519,9 @@ No. The host passes the full prompt on every call and keeps the reply.
 Yes, on models with vision support. Put image URLs in a user turn's `images`
 array. Qwen3-VL 2B accepts images on the CPU engine and Gemma 4 E2B accepts
 images on the WebGPU engine. Each image is center-cropped to a square and
-resized to 288 x 288 before the engine sees it. A model without vision drops
-images when the turn also has text; an image-only turn reports
+resized to 288 x 288 before the engine sees it. The WebGPU engine encodes that
+crop at 384 x 384 with the smallest Gemma 4 vision budget. A model without
+vision drops images when the turn also has text; an image-only turn reports
 `Model does not accept images`.
 
 ### Does it need a server?

package/dist/browser/chat.min.mjs

@@ -1,2 +1,2 @@
 /*! For license information please see chat.min.mjs.LICENSE.txt */
-var e={};const t={};function n(a){const o=t[a];if(void 0!==o)return o.exports;const r=t[a]={exports:{}};return e[a](r,r.exports,n),r.exports}var a,o,r,i,s,c;n.m=e,n.d=(e,t)=>{for(var a in t)n.o(t,a)&&!n.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},n.f={},n.e=e=>{const t=[];return n.f.j(e,t),Promise.all(t)},n.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),n.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.min.mjs"),924:()=>import("./engine-wllama.min.mjs")},t={792:0},a=e=>{let{__webpack_esm_ids__:a,__webpack_esm_modules__:o,__webpack_esm_runtime__:r}=e;var i,s,c=0;for(i in o)n.o(o,i)&&(n.m[i]=o[i]);for(r&&r(n);c<a.length;c++)s=a[c],n.o(t,s)&&t[s]&&t[s][0](),t[s]=0};n.f.j=(o,r)=>{let i=n.o(t,o)?t[o]:void 0;if(0!==i)if(i)r.push(i[1]);else{let n=e[o]().then(a,e=>{throw 0!==t[o]&&(t[o]=void 0),e});n=Promise.race([n,new Promise(e=>i=t[o]=[e])]),r.push(i[1]=n)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(a||(a={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(o||(o={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(r||(r={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(i||(i={})),function(e){e.download="download",e.load="load"}(s||(s={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(c||(c={}));const l=[{id:c.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:c.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],d=e=>e?l.find(t=>t.id==e):void 0,g=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?l.find(e=>!!e.litert):l.find(e=>!!e.cpu))?.id,u=async e=>d(e||await g()),m=e=>!!e.cpu?.mmproj||!!e.litert?.vision,f=new URL("./../assets/wllama/wllama.wasm",import.meta.url).href,p=new URL("./../assets/wllama/compat/wllama.wasm",import.meta.url).href,w=new URL("./../assets/wllama/compat/wllama.js",import.meta.url).href,h=new URL("./../assets/litert/litertlm_wasm_internal.js",import.meta.url).href,y=(new URL("./../assets/litert/litertlm_wasm_internal.wasm",import.meta.url).href,288),b=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},v=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(y,y),s=i?i.getContext("2d"):null;if(!i||!s||a<1)return n.close(),e;if(s.drawImage(n,o,r,a,a,0,0,y,y),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},x={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},k=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,_=(e={},t=!1)=>{const n=x;return{maxReplyTokens:k(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:k(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:k(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:k(e.maxImages,n.maxImages,0),maxContentLength:k(e.maxContentLength,n.maxContentLength,1),promptCharBudget:k(e.promptCharBudget,n.promptCharBudget,1),maxThreads:k(e.maxThreads,n.maxThreads,1)}},M=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),T=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,C=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),L=["<|channel>thought\n","<|channel>thought","..."],P=["<channel|>","..."],R=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,$=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,j=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...$(e),{type:"text",text:"\n"},...$(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class S extends Error{}const W=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new S(n)),t)})])}finally{a&&clearTimeout(a)}},q=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},z=async({model:e,logging:t,onReason:n,onText:a,start:o,stopped:r,firstTimeoutMs:i=18e4,idleTimeoutMs:s=3e4,onStall:c})=>{let l="",d="",g="",u=!1;const m=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:i=>{let s=n+i;for(n="";s;){if(a){const{at:t,tag:i}=o(s,P);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),a=!1;continue}const c=r(s,P);e(s.slice(0,s.length-c.length)),n=c;break}{const{at:e,tag:i}=o(s,L);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),a=!0;continue}const c=r(s,L);t(s.slice(0,s.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{d+=e,n(e)},e=>{l+=e,a(e)});try{const e=(await W(o(),i,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(r()){q(e);break}let a;try{a=await W(e.next(),u?s:i,u?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof S&&(l||d)){c?.(),q(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;u=!0;const o=a.value?.choices?.[0],f=o?.delta,p="string"==typeof f?.reasoning_content?f.reasoning_content:"string"==typeof f?.reasoning?f.reasoning:"";o?.finish_reason&&(g=o.finish_reason),p&&(d+=p,n(p)),f?.content&&m.feed(f.content)}m.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof S&&!r())throw e}const f=r()?void 0:l||void 0;return t&&(console.log("Local chat reply",e.id,f?"****":""),console.log("Local chat finish",e.id,g||"none",l.length,d.length)),f},B=async({model:e,params:t,start:n,stopped:o,onStall:r})=>{const i=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await v(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=x)=>{const n=t.maxContentLength,o=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=a.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=o.length-1;e>=0;e--){const l=o[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==a.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==a.system?r.unshift(d):l.role==a.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:R(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=j(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,i),await z({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,start:()=>n(i),stopped:o,onStall:r})},A=33554432,O=new Map;let I=!1;const N=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},U=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},F=async()=>{if(!I)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else I=!0},D=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},H=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},Q=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},G=async function*(e){for await(const t of e)yield t},V=async e=>{try{for await(const[t,n]of G(e))"file"==n.kind&&U(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},J=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await H(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},K=async(e,t)=>{const n=await D(e,t);if(n){const e=await(async e=>await H(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of G(e)){const e=U(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await J(e,t)},X=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},Y=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:i})=>{let c,l=t,d=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});c=await t.createWritable(),d=0},m=async()=>{const e=c;c=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){c||await u();const e=Math.min(A-d,o.byteLength-f);await c.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==A&&await m()}l>t&&r(a?l/a:0,s.download)}await m()}catch(e){throw await m(),e}return l},Z=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),ee=e=>{const t=async()=>{const t=await F();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await K(n,N(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,i=O.get(r);if(i)return await i;const c=await t();if(!c)return;const l=(async(e,t,n,a)=>{const o=N(t),r=await J(e,o);if(r?.complete)return r;let i=await D(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let c=0,l=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await K(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>c)c=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&a(d/g,s.download);const m=g?Math.min(d+A,g)-1:d+A-1,f=new AbortController,p=()=>f.abort(),w={size:d,total:g,etag:u};let h,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(h=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==h.status){if(await(h.body?.cancel().catch(()=>{})),g&&d>=g){await Q(i,{total:g,etag:u,complete:!0});continue}await V(i),await Q(i,{total:0,etag:"",complete:!1});continue}if(!h.ok||!h.body)throw await(h.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=X(h,w);if(!n){await h.body.cancel().catch(()=>{}),await V(i),await Q(i,{total:0,etag:"",complete:!1});continue}await Q(i,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?d:0,s=await Y({dir:i,start:n.start+r,skip:r,total:n.total,body:h.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await Q(i,{total:s,etag:n.etag,complete:!0});const t=await K(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(c,n,a,o).finally(()=>{O.delete(r)});return O.set(r,l),await l},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,N(e))},purge:async()=>{const t=await F();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},te="litertlm-models",ne=ee("litert-store"),ae={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0},oe=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(te)}catch{return}},re=new Map,ie=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,s.download),o.enqueue(e)}}))},se=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await b(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ce=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},le=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],i=M(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const s=a.slice(0,-1),c=(await se([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==i.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ce(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const i=o.length-r.length;for(let e=0;e<r.length;e++)if(!ce(o[i+e],r[e]))return!1;return!0})(o.sent,s);let d;if(l&&o.conversation)d=o.conversation;else{const a=await se(s,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:i.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:i.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=i.max_tokens,o.sent=[...s]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},de={backend:i.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:o,logging:r})=>{const c=e.litert;if(!c)throw new Error("Model has no LiteRT artifact");const l=await n.e(520).then(()=>n(507));if(await l.getOrLoadGlobalLiteRtLm(h),o?.aborted)throw new Error("Cancelled");const d=T(c.repo,c.file),g=await oe();let u;const m=g?await g.match(d):void 0;if(m?.body)u=m.body;else{const e=await ne.stored(d);e?.complete&&(u=e.file.stream())}if(!u){const e=await ne.download({url:d,signal:o,onProgress:a}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;r&&console.log("Local model store failed",e)});e&&(u=e.file.stream())}if(!u&&g)try{await(async(e,t,n,a)=>{const o=re.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),i=ie(o.body,r,a);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{re.delete(e)});return re.set(e,r),await r})(d,g,o,a);const e=await g.match(d);e?.body&&(u=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(u||(u=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return ie(a.body,E(a),n)})(d,o,a)),o?.aborted)throw new Error("Cancelled");a(1,s.load);const f=await l.Engine.create({model:u,mainExecutorSettings:{maxNumTokens:4096,...c.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:ae},...c.vision?{visionBackend:l.Backend.GPU}:{}});if(o?.aborted)throw await f.delete(),new Error("Cancelled");const p={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:i.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await B({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(C(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return le({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:f,model:e,mod:l,params:t,session:p}),unload:async()=>{p.conversation=void 0,p.sent=[],await f.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=T(t.repo,t.file),[a,o]=await Promise.all([oe(),ne.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of l){const t=e.litert;if(!t)continue;if(await ne.stored(T(t.repo,t.file)))return!0}const e=await oe();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=T(t.repo,t.file),a=await oe();await ne.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ne.purge(),"undefined"!=typeof caches&&await caches.delete(te)}catch{}}},ge=ee("wllama-store");let ue,me,fe="",pe=!1;const we=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,he=e=>{const t={wasm:f,compatWasm:p,compatWorker:w};return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},ye=e=>{const t=e=>(...t)=>{pe&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},be=async()=>{if(ue)return ue;const e=await n.e(924).then(()=>n(577)),t=me||he(),a=new e.Wllama({default:t.wasm},{logger:ye(e.LoggerWithoutDebug)});return me||(me=t,fe=we(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),ue=a,a},ve=async()=>{const e=ue;ue=void 0;try{await(e?.exit())}catch{}},xe=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),ke=e=>{const t=e.cpu;return t?[{name:t.file,url:T(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:T(t.repo,t.mmproj.file)}]:[]]:[]},_e=async e=>{const t=ke(e);if(!t.length)return!1;for(const e of t){const t=await ge.stored(e.url);if(!t?.complete)return!1}return!0},Me=async(e,t)=>{const n=await e.cacheManager.list();return ke(t).every(e=>n.some(t=>xe(t,e.name)))},Te=async({model:e,signal:t,onProgress:n})=>{const a=ke(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await ge.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Ee=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await b(t.url)});t.push({role:"user",content:e})}return t},Ce={backend:i.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{pe=!0===o;const c=e.cpu;if(!c)throw new Error("Model has no CPU artifact");const l=he(r),d=we(l);ue&&d!=fe&&await ve(),me=l,fe=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await be(),p=await _e(e),w=c.mmproj?.file,h=T(c.repo,c.file),y=w?T(c.repo,w):"";if(a?.aborted)throw await ve(),new Error("Cancelled");if(!p&&w){const e=await f.cacheManager.list(),t=e.some(e=>xe(e,c.file)),n=e.some(e=>xe(e,w));t&&!n&&await f.cacheManager.deleteMany(e=>xe(e,c.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Te({model:e,signal:a,onProgress:n}):void 0;t||p||await Me(f,e).catch(()=>!1)||(t=await Te({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,s.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:h,mmprojUrl:y}:h,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?s.load:s.download)}})}catch(e){throw await ve(),e}if(a?.aborted)throw await ve(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await ve(),new Error("Failed to create model context");return{model:e,backend:i.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(pe=!0===n.logging,e.isModelLoaded())return await B({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(C(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Ee(a),...M(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(ue==f)await ve();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await _e(e)||await Me(await be(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of l.flatMap(e=>ke(e)))if(await ge.stored(e.url))return!0;const e=await be();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=ke(e);try{for(const e of t)await ge.remove(e.url)}catch{}try{const e=await be();await e.cacheManager.deleteMany(e=>t.some(t=>xe(e,t.name)))}catch{}},purge:async()=>{try{await ge.purge()}catch{}try{const e=await be();await e.cacheManager.clear()}catch{}}},Le=((e={})=>{const t=e.engines?.cpu||Ce,n=e.engines?.litert||de;let a,o,c,d,g,u,m,f=i.none,p=r.idle,w=[],h=0,y={percent:0,phase:s.download};const b=a=>{if(e.backend!=i.none)return e.backend==i.cpu?a.cpu?t:void 0:e.backend==i.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||s.download};for(const e of[...w])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{h++,u?.abort(),u=void 0,g=void 0,p==r.downloading&&x(r.idle)},_=(e,t)=>{const n=d;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{w=w.filter(t=>t!=e),queueMicrotask(()=>{!w.length&&d&&k()})})(r),r.settle(!1)};w.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==r.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:i,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(d&&g==e.id)return p(y.percent,y.phase),await _(p,m);d&&(k(),w=[],await d.catch(()=>!1));const T=++h,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const d=b(e);if(T!=h)return!1;if(!d)return x(r.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=c,n=a;a=void 0,o=void 0,c=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of l)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,d),T!=h)return!1;const g=await d.cached(e);if(T!=h)return!1;x(r.downloading),v(0,s.download),g&&v(1,s.load);const m=new AbortController;u=m;try{const l=await d.startLLM({model:e,limits:t,logging:i,signal:m.signal,assets:n,onProgress:(e,t)=>{T==h&&v(e,t)}});return T!=h||m.signal.aborted?(await l.unload(),!1):(a=l,o=e,c=d,f=d.backend,x(r.ready),v(1,s.load),!0)}catch(e){return T!=h||(a=void 0,o=void 0,c=void 0,i&&console.log("Local model failed",e),x(r.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{d==E&&(d=void 0,g=void 0,w=[])});return d=E,g=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,c=void 0,x(r.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==r.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Pe=(e,t,n)=>{const a=e.status(),i=d(t);return{availability:a==r.ready?o.online:a==r.error||a==r.unsupported?o.offline:o.away,status:a,backend:e.backend(),supportsImages:!!i&&m(i),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},Re=(e=>{let t,n=0;const o=t=>t.input.onStatus?.(Pe(e,t.model??t.input.model,t));return i=>{t?.controller.abort();const c={token:++n,input:i,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?s.load:s.download,requesting:!1,cancelled:!1};return t=c,o(c),(async t=>{const{input:r}=t,i=!0===r.logging,c=await u(r.model);if(t.model=c?.id,t.token!=n)return;const l=r.messages.some(e=>e.role!=a.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==a.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=l||d;if(!c)return t.error="Unknown model",void o(t);if(!g)return void o(t);if(d&&!l&&!m(c))return t.error="Model does not accept images",void o(t);const f=_(r.limits,!!c.litert),p=await e.load({model:c,limits:f,logging:i,signal:t.controller.signal,onProgress:(e,a)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=a,o(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void o(t);const w=m(c),h=d&&!w?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!w&&!l)return t.error="Model does not accept images",void o(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=s.load,o(t);let y,b="",v="";try{y=await e.chat({messages:h,limits:f,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=n||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,r.onResponse?.(v))},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void o(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),o(t))})(c),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=r.ready?queueMicrotask(()=>o(t)):e.unload().then(()=>o(t)))})(c)}})(Le),$e=(je=Le,async(e={})=>{const t=await u(e.model),n=n=>e.onStatus?.(Pe(je,t?.id,n));if(!t)return n({requesting:!1,phase:s.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=s.download;const i=je.ready()&&je.model()?.id==t.id;return await je.load({model:t,limits:_(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(i&&n({requesting:!1,phase:s.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var je;export{c as ChatModelId,o as ChatStatus,i as LlmBackend,s as LlmPhase,a as LlmRole,r as LlmStatus,Re as createChat,$e as preloadModel};
+var e={};const t={};function n(a){const o=t[a];if(void 0!==o)return o.exports;const r=t[a]={exports:{}};return e[a](r,r.exports,n),r.exports}var a,o,r,i,s,c;n.m=e,n.d=(e,t)=>{for(var a in t)n.o(t,a)&&!n.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},n.f={},n.e=e=>{const t=[];return n.f.j(e,t),Promise.all(t)},n.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),n.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.min.mjs"),924:()=>import("./engine-wllama.min.mjs")},t={792:0},a=e=>{let{__webpack_esm_ids__:a,__webpack_esm_modules__:o,__webpack_esm_runtime__:r}=e;var i,s,c=0;for(i in o)n.o(o,i)&&(n.m[i]=o[i]);for(r&&r(n);c<a.length;c++)s=a[c],n.o(t,s)&&t[s]&&t[s][0](),t[s]=0};n.f.j=(o,r)=>{let i=n.o(t,o)?t[o]:void 0;if(0!==i)if(i)r.push(i[1]);else{let n=e[o]().then(a,e=>{throw 0!==t[o]&&(t[o]=void 0),e});n=Promise.race([n,new Promise(e=>i=t[o]=[e])]),r.push(i[1]=n)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(a||(a={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(o||(o={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(r||(r={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(i||(i={})),function(e){e.download="download",e.load="load"}(s||(s={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(c||(c={}));const l=[{id:c.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:c.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],d=e=>e?l.find(t=>t.id==e):void 0,g=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?l.find(e=>!!e.litert):l.find(e=>!!e.cpu))?.id,u=async e=>d(e||await g()),m=e=>!!e.cpu?.mmproj||!!e.litert?.vision,f=new URL("./../assets/wllama/wllama.wasm",import.meta.url).href,p=new URL("./../assets/wllama/compat/wllama.wasm",import.meta.url).href,w=new URL("./../assets/wllama/compat/wllama.js",import.meta.url).href,h=new URL("./../assets/litert/litertlm_wasm_internal.js",import.meta.url).href,y=(new URL("./../assets/litert/litertlm_wasm_internal.wasm",import.meta.url).href,288),b=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},v=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(y,y),s=i?i.getContext("2d"):null;if(!i||!s||a<1)return n.close(),e;if(s.drawImage(n,o,r,a,a,0,0,y,y),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},x={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},k=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,_=(e={},t=!1)=>{const n=x;return{maxReplyTokens:k(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:k(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:k(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:k(e.maxImages,n.maxImages,0),maxContentLength:k(e.maxContentLength,n.maxContentLength,1),promptCharBudget:k(e.promptCharBudget,n.promptCharBudget,1),maxThreads:k(e.maxThreads,n.maxThreads,1)}},M=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),T=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,C=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),L=["<|channel>thought\n","<|channel>thought","..."],P=["<channel|>","..."],R=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,$=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,S=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...$(e),{type:"text",text:"\n"},...$(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class j extends Error{}const W=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new j(n)),t)})])}finally{a&&clearTimeout(a)}},q=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},z=async({model:e,logging:t,onReason:n,onText:a,onSpeed:o,start:r,stopped:i,firstTimeoutMs:s=18e4,idleTimeoutMs:c=3e4,onStall:l})=>{let d="",g="",u="",m=!1,f=0,p=0;const w=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:i=>{let s=n+i;for(n="";s;){if(a){const{at:t,tag:i}=o(s,P);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),a=!1;continue}const c=r(s,P);e(s.slice(0,s.length-c.length)),n=c;break}{const{at:e,tag:i}=o(s,L);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),a=!0;continue}const c=r(s,L);t(s.slice(0,s.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{g+=e,n(e)},e=>{d+=e,a(e)});try{const e=(await W(r(),s,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(i()){q(e);break}let a;try{a=await W(e.next(),m?c:s,m?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof j&&(d||g)){l?.(),q(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;m=!0;const r=a.value?.choices?.[0],h=r?.delta,y="string"==typeof h?.reasoning_content?h.reasoning_content:"string"==typeof h?.reasoning?h.reasoning:"";if(r?.finish_reason&&(u=r.finish_reason),y&&(g+=y,n(y)),h?.content&&w.feed(h.content),y||h?.content){const e=performance.now();f||(p=e),f+=1,f>1&&e>p&&o?.(1e3*f/(e-p))}}w.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof j&&!i())throw e}const h=i()?void 0:d||void 0;return t&&(console.log("Local chat reply",e.id,h?"****":""),console.log("Local chat finish",e.id,u||"none",d.length,g.length)),h},B=async({model:e,params:t,start:n,stopped:o,onStall:r})=>{const i=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await v(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=x)=>{const n=t.maxContentLength,o=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=a.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=o.length-1;e>=0;e--){const l=o[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==a.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==a.system?r.unshift(d):l.role==a.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:R(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=S(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,i),await z({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,onSpeed:t.onSpeed,start:()=>n(i),stopped:o,onStall:r})},I=33554432,A=new Map;let O=!1;const N=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},U=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},F=async()=>{if(!O)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else O=!0},D=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},H=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},Q=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},V=async function*(e){for await(const t of e)yield t},G=async e=>{try{for await(const[t,n]of V(e))"file"==n.kind&&U(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},J=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await H(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},K=async(e,t)=>{const n=await D(e,t);if(n){const e=await(async e=>await H(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of V(e)){const e=U(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await J(e,t)},X=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},Y=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:i})=>{let c,l=t,d=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});c=await t.createWritable(),d=0},m=async()=>{const e=c;c=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){c||await u();const e=Math.min(I-d,o.byteLength-f);await c.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==I&&await m()}l>t&&r(a?l/a:0,s.download)}await m()}catch(e){throw await m(),e}return l},Z=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),ee=e=>{const t=async()=>{const t=await F();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await K(n,N(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,i=A.get(r);if(i)return await i;const c=await t();if(!c)return;const l=(async(e,t,n,a)=>{const o=N(t),r=await J(e,o);if(r?.complete)return r;let i=await D(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let c=0,l=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await K(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>c)c=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&a(d/g,s.download);const m=g?Math.min(d+I,g)-1:d+I-1,f=new AbortController,p=()=>f.abort(),w={size:d,total:g,etag:u};let h,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(h=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==h.status){if(await(h.body?.cancel().catch(()=>{})),g&&d>=g){await Q(i,{total:g,etag:u,complete:!0});continue}await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}if(!h.ok||!h.body)throw await(h.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=X(h,w);if(!n){await h.body.cancel().catch(()=>{}),await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}await Q(i,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?d:0,s=await Y({dir:i,start:n.start+r,skip:r,total:n.total,body:h.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await Q(i,{total:s,etag:n.etag,complete:!0});const t=await K(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(c,n,a,o).finally(()=>{A.delete(r)});return A.set(r,l),await l},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,N(e))},purge:async()=>{const t=await F();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},te="litertlm-models",ne=ee("litert-store"),ae={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0};let oe=!1;const re=(...e)=>{oe&&console.log(...e)},ie=(...e)=>{oe&&console.error(...e)},se=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(te)}catch{return}},ce=new Map,le=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,s.download),o.enqueue(e)}}))},de=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await b(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ge=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},ue=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],i=M(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const s=a.slice(0,-1),c=(await de([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==i.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ge(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const i=o.length-r.length;for(let e=0;e<r.length;e++)if(!ge(o[i+e],r[e]))return!1;return!0})(o.sent,s);let d;if(l&&o.conversation)d=o.conversation;else{const a=await de(s,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:i.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:i.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=i.max_tokens,o.sent=[...s]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},me={backend:i.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:o,logging:r})=>{oe=!0===r;const c=e.litert;if(!c)throw new Error("Model has no LiteRT artifact");const l=await n.e(520).then(()=>n(507)),d=globalThis,g=d.Module;d.Module={print:re,printErr:ie};try{await l.getOrLoadGlobalLiteRtLm(h)}finally{d.Module=g}if(o?.aborted)throw new Error("Cancelled");const u=T(c.repo,c.file),m=await se();let f;const p=m?await m.match(u):void 0;if(p?.body)f=p.body;else{const e=await ne.stored(u);e?.complete&&(f=e.file.stream())}if(!f){const e=await ne.download({url:u,signal:o,onProgress:a}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;r&&console.log("Local model store failed",e)});e&&(f=e.file.stream())}if(!f&&m)try{await(async(e,t,n,a)=>{const o=ce.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),i=le(o.body,r,a);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{ce.delete(e)});return ce.set(e,r),await r})(u,m,o,a);const e=await m.match(u);e?.body&&(f=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(f||(f=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return le(a.body,E(a),n)})(u,o,a)),o?.aborted)throw new Error("Cancelled");a(1,s.load);const w=await l.Engine.create((({model:e,limits:t,source:n,mod:a})=>({model:n,mainExecutorSettings:{maxNumTokens:4096,...e.litert?.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:ae},...e.litert?.vision?{visionBackend:a.Backend.GPU,maxVisionTokensPerImage:70,minVisionTokensPerImage:70}:{}}))({model:e,limits:t,source:f,mod:l}));if(o?.aborted)throw await w.delete(),new Error("Cancelled");const y={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:i.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{oe=!0===a.logging;const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await B({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(C(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return ue({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:w,model:e,mod:l,params:t,session:y}),unload:async()=>{y.conversation=void 0,y.sent=[],await w.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=T(t.repo,t.file),[a,o]=await Promise.all([se(),ne.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of l){const t=e.litert;if(!t)continue;if(await ne.stored(T(t.repo,t.file)))return!0}const e=await se();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=T(t.repo,t.file),a=await se();await ne.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ne.purge(),"undefined"!=typeof caches&&await caches.delete(te)}catch{}}},fe=ee("wllama-store");let pe,we,he="",ye=!1;const be=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ve=e=>{const t={wasm:f,compatWasm:p,compatWorker:w};return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},xe=e=>{const t=e=>(...t)=>{ye&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},ke=async()=>{if(pe)return pe;const e=await n.e(924).then(()=>n(577)),t=we||ve(),a=new e.Wllama({default:t.wasm},{logger:xe(e.LoggerWithoutDebug)});return we||(we=t,he=be(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),pe=a,a},_e=async()=>{const e=pe;pe=void 0;try{await(e?.exit())}catch{}},Me=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),Te=e=>{const t=e.cpu;return t?[{name:t.file,url:T(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:T(t.repo,t.mmproj.file)}]:[]]:[]},Ee=async e=>{const t=Te(e);if(!t.length)return!1;for(const e of t){const t=await fe.stored(e.url);if(!t?.complete)return!1}return!0},Ce=async(e,t)=>{const n=await e.cacheManager.list();return Te(t).every(e=>n.some(t=>Me(t,e.name)))},Le=async({model:e,signal:t,onProgress:n})=>{const a=Te(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await fe.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Pe=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await b(t.url)});t.push({role:"user",content:e})}return t},Re={backend:i.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{ye=!0===o;const c=e.cpu;if(!c)throw new Error("Model has no CPU artifact");const l=ve(r),d=be(l);pe&&d!=he&&await _e(),we=l,he=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await ke(),p=await Ee(e),w=c.mmproj?.file,h=T(c.repo,c.file),y=w?T(c.repo,w):"";if(a?.aborted)throw await _e(),new Error("Cancelled");if(!p&&w){const e=await f.cacheManager.list(),t=e.some(e=>Me(e,c.file)),n=e.some(e=>Me(e,w));t&&!n&&await f.cacheManager.deleteMany(e=>Me(e,c.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Le({model:e,signal:a,onProgress:n}):void 0;t||p||await Ce(f,e).catch(()=>!1)||(t=await Le({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,s.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:h,mmprojUrl:y}:h,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?s.load:s.download)}})}catch(e){throw await _e(),e}if(a?.aborted)throw await _e(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await _e(),new Error("Failed to create model context");return{model:e,backend:i.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(ye=!0===n.logging,e.isModelLoaded())return await B({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(C(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Pe(a),...M(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(pe==f)await _e();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Ee(e)||await Ce(await ke(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of l.flatMap(e=>Te(e)))if(await fe.stored(e.url))return!0;const e=await ke();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=Te(e);try{for(const e of t)await fe.remove(e.url)}catch{}try{const e=await ke();await e.cacheManager.deleteMany(e=>t.some(t=>Me(e,t.name)))}catch{}},purge:async()=>{try{await fe.purge()}catch{}try{const e=await ke();await e.cacheManager.clear()}catch{}}},$e=((e={})=>{const t=e.engines?.cpu||Re,n=e.engines?.litert||me;let a,o,c,d,g,u,m,f=i.none,p=r.idle,w=[],h=0,y={percent:0,phase:s.download};const b=a=>{if(e.backend!=i.none)return e.backend==i.cpu?a.cpu?t:void 0:e.backend==i.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||s.download};for(const e of[...w])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{h++,u?.abort(),u=void 0,g=void 0,p==r.downloading&&x(r.idle)},_=(e,t)=>{const n=d;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{w=w.filter(t=>t!=e),queueMicrotask(()=>{!w.length&&d&&k()})})(r),r.settle(!1)};w.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==r.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:i,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(d&&g==e.id)return p(y.percent,y.phase),await _(p,m);d&&(k(),w=[],await d.catch(()=>!1));const T=++h,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const d=b(e);if(T!=h)return!1;if(!d)return x(r.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=c,n=a;a=void 0,o=void 0,c=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of l)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,d),T!=h)return!1;const g=await d.cached(e);if(T!=h)return!1;x(r.downloading),v(0,s.download),g&&v(1,s.load);const m=new AbortController;u=m;try{const l=await d.startLLM({model:e,limits:t,logging:i,signal:m.signal,assets:n,onProgress:(e,t)=>{T==h&&v(e,t)}});return T!=h||m.signal.aborted?(await l.unload(),!1):(a=l,o=e,c=d,f=d.backend,x(r.ready),v(1,s.load),!0)}catch(e){return T!=h||(a=void 0,o=void 0,c=void 0,i&&console.log("Local model failed",e),x(r.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{d==E&&(d=void 0,g=void 0,w=[])});return d=E,g=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,c=void 0,x(r.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==r.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Se=(e,t,n)=>{const a=e.status(),i=d(t);return{availability:a==r.ready?o.online:a==r.error||a==r.unsupported?o.offline:o.away,status:a,backend:e.backend(),supportsImages:!!i&&m(i),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},je=(e=>{let t,n=0;const o=t=>t.input.onStatus?.(Se(e,t.model??t.input.model,t));return i=>{t?.controller.abort();const c={token:++n,input:i,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?s.load:s.download,requesting:!1,cancelled:!1};return t=c,o(c),(async t=>{const{input:r}=t,i=!0===r.logging,c=await u(r.model);if(t.model=c?.id,t.token!=n)return;const l=r.messages.some(e=>e.role!=a.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==a.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=l||d;if(!c)return t.error="Unknown model",void o(t);if(!g)return void o(t);if(d&&!l&&!m(c))return t.error="Model does not accept images",void o(t);const f=_(r.limits,!!c.litert),p=await e.load({model:c,limits:f,logging:i,signal:t.controller.signal,onProgress:(e,a)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=a,o(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void o(t);const w=m(c),h=d&&!w?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!w&&!l)return t.error="Model does not accept images",void o(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=s.load,o(t);let y,b="",v="";try{y=await e.chat({messages:h,limits:f,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=n||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,r.onResponse?.(v))},onSpeed:e=>{t.token!=n||t.cancelled||r.onSpeed?.(e)},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void o(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),o(t))})(c),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=r.ready?queueMicrotask(()=>o(t)):e.unload().then(()=>o(t)))})(c)}})($e),We=(qe=$e,async(e={})=>{const t=await u(e.model),n=n=>e.onStatus?.(Se(qe,t?.id,n));if(!t)return n({requesting:!1,phase:s.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=s.download;const i=qe.ready()&&qe.model()?.id==t.id;return await qe.load({model:t,limits:_(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(i&&n({requesting:!1,phase:s.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var qe;export{c as ChatModelId,o as ChatStatus,i as LlmBackend,s as LlmPhase,a as LlmRole,r as LlmStatus,je as createChat,We as preloadModel};

package/dist/engines/engine.d.ts

@@ -28,7 +28,7 @@ declare const
 /** Log an engine chat call with all message text censored */
 logChatCall: (model: ChatModel, messages: EngineMessage[]) => void, 
 /** Stream one reply, logging it and returning undefined when stopped */
-streamReply: ({ model, logging, onReason, onText, start, stopped, firstTimeoutMs, idleTimeoutMs, onStall, }: StreamReplyParams) => Promise<string | undefined>;
+streamReply: ({ model, logging, onReason, onText, onSpeed, start, stopped, firstTimeoutMs, idleTimeoutMs, onStall, }: StreamReplyParams) => Promise<string | undefined>;
 declare const 
 /** Build the prompt and stream one reply through an engine */
 runEngineChat: ({ model, params, start, stopped, onStall, }: EngineRunParams) => Promise<string | undefined>;

package/dist/engines/litert.d.ts

@@ -1,4 +1,7 @@
-import type { EngineChunk, EngineMessage, LitertMessage, LlmEngine, StartLlmParams, StreamLitertParams } from '../types';
+import type { EngineChunk, EngineMessage, LitertMessage, LitertSettings, LitertSettingsParams, LlmEngine, StartLlmParams, StreamLitertParams } from '../types';
+declare const 
+/** Smallest Gemma 4 vision budget in tokens */
+visionTokens = 70;
 /** Fetch one model file into the cache, once per URL */
 declare const cacheModel: (url: string, cache: Cache, signal: AbortSignal | undefined, onProgress: StartLlmParams[`onProgress`]) => Promise<void>;
 /** Map built messages to LiteRT turns */
@@ -7,5 +10,7 @@ declare const toLitertMessages: (messages: EngineMessage[], keepImages?: boolean
 declare const continuesHistory: (sent: EngineMessage[], history: EngineMessage[]) => boolean;
 /** Stream one LiteRT reply as engine chunks */
 declare const streamLitert: ({ engine, mod, params, messages, session, }: StreamLitertParams) => AsyncGenerator<EngineChunk>;
+/** Build one engine settings object with the small vision budget */
+declare const litertSettings: ({ model, limits, source, mod, }: LitertSettingsParams) => LitertSettings;
 declare const litertEngine: LlmEngine;
-export { cacheModel, continuesHistory, litertEngine, streamLitert, toLitertMessages, };
+export { cacheModel, continuesHistory, litertEngine, litertSettings, streamLitert, toLitertMessages, visionTokens, };

package/dist/node/chat.node.min.js

@@ -1,2 +1,2 @@
 /*! For license information please see chat.node.min.js.LICENSE.txt */
-(()=>{"use strict";var e={};const t={};function n(a){const o=t[a];if(void 0!==o)return o.exports;const r=t[a]={exports:{}};return e[a](r,r.exports,n),r.exports}n.m=e,n.d=(e,t)=>{for(var a in t)n.o(t,a)&&!n.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},n.f={},n.e=e=>{const t=[];return n.f.require(e,t),Promise.all(t)},n.u=e=>({520:"engine-litert",924:"engine-wllama"}[e]+".node.min.js"),n.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),n.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={792:1};n.f.require=(t,a)=>{if(!e[t]){const a=require("./"+n.u(t));e[t]||(t=>{const a=t.modules,o=t.ids,r=t.runtime;for(var i in a)n.o(a,i)&&(n.m[i]=a[i]);r&&r(n);for(var s=0;s<o.length;s++)e[o[s]]=1})(a)}}})();let a={};var o,r,i,s,c,l;n.r(a),n.d(a,{ChatModelId:()=>l,ChatStatus:()=>r,LlmBackend:()=>s,LlmPhase:()=>c,LlmRole:()=>o,LlmStatus:()=>i,createChat:()=>je,preloadModel:()=>$e}),function(e){e.system="system",e.user="user",e.assistant="assistant"}(o||(o={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(r||(r={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(i||(i={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(s||(s={})),function(e){e.download="download",e.load="load"}(c||(c={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(l||(l={}));const d=[{id:l.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:l.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],g=e=>e?d.find(t=>t.id==e):void 0,u=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?d.find(e=>!!e.litert):d.find(e=>!!e.cpu))?.id,m=async e=>g(e||await u()),f=e=>!!e.cpu?.mmproj||!!e.litert?.vision,p=require("node:path"),h=()=>{return e=(0,p.join)(__dirname,"../assets/wllama"),{wasm:(0,p.join)(e,"wllama.wasm"),compatWasm:(0,p.join)(e,"compat/wllama.wasm"),compatWorker:(0,p.join)(e,"compat/wllama.js")};var e},w=()=>{return e=(0,p.join)(__dirname,"../assets/litert"),{script:(0,p.join)(e,"litertlm_wasm_internal.js"),wasm:(0,p.join)(e,"litertlm_wasm_internal.wasm")};var e},y=288,b=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},v=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(y,y),s=i?i.getContext("2d"):null;if(!i||!s||a<1)return n.close(),e;if(s.drawImage(n,o,r,a,a,0,0,y,y),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},x={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},k=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,_=(e={},t=!1)=>{const n=x;return{maxReplyTokens:k(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:k(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:k(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:k(e.maxImages,n.maxImages,0),maxContentLength:k(e.maxContentLength,n.maxContentLength,1),promptCharBudget:k(e.promptCharBudget,n.promptCharBudget,1),maxThreads:k(e.maxThreads,n.maxThreads,1)}},M=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),T=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,C=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),L=["<|channel>thought\n","<|channel>thought","..."],P=["<channel|>","..."],j=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,$=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,R=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...$(e),{type:"text",text:"\n"},...$(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class S extends Error{}const q=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new S(n)),t)})])}finally{a&&clearTimeout(a)}},W=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},B=async({model:e,logging:t,onReason:n,onText:a,start:o,stopped:r,firstTimeoutMs:i=18e4,idleTimeoutMs:s=3e4,onStall:c})=>{let l="",d="",g="",u=!1;const m=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:i=>{let s=n+i;for(n="";s;){if(a){const{at:t,tag:i}=o(s,P);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),a=!1;continue}const c=r(s,P);e(s.slice(0,s.length-c.length)),n=c;break}{const{at:e,tag:i}=o(s,L);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),a=!0;continue}const c=r(s,L);t(s.slice(0,s.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{d+=e,n(e)},e=>{l+=e,a(e)});try{const e=(await q(o(),i,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(r()){W(e);break}let a;try{a=await q(e.next(),u?s:i,u?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof S&&(l||d)){c?.(),W(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;u=!0;const o=a.value?.choices?.[0],f=o?.delta,p="string"==typeof f?.reasoning_content?f.reasoning_content:"string"==typeof f?.reasoning?f.reasoning:"";o?.finish_reason&&(g=o.finish_reason),p&&(d+=p,n(p)),f?.content&&m.feed(f.content)}m.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof S&&!r())throw e}const f=r()?void 0:l||void 0;return t&&(console.log("Local chat reply",e.id,f?"****":""),console.log("Local chat finish",e.id,g||"none",l.length,d.length)),f},z=async({model:e,params:t,start:n,stopped:a,onStall:r})=>{const i=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await v(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=x)=>{const n=t.maxContentLength,a=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=o.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=a.length-1;e>=0;e--){const l=a[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==o.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==o.system?r.unshift(d):l.role==o.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:j(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=R(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,i),await B({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,start:()=>n(i),stopped:a,onStall:r})},A=33554432,I=new Map;let O=!1;const N=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},F=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},U=async()=>{if(!O)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else O=!0},D=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},H=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},Q=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},G=async function*(e){for await(const t of e)yield t},V=async e=>{try{for await(const[t,n]of G(e))"file"==n.kind&&F(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},J=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await H(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},K=async(e,t)=>{const n=await D(e,t);if(n){const e=await(async e=>await H(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of G(e)){const e=F(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await J(e,t)},X=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},Y=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:i})=>{let s,l=t,d=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});s=await t.createWritable(),d=0},m=async()=>{const e=s;s=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){s||await u();const e=Math.min(A-d,o.byteLength-f);await s.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==A&&await m()}l>t&&r(a?l/a:0,c.download)}await m()}catch(e){throw await m(),e}return l},Z=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),ee=e=>{const t=async()=>{const t=await U();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await K(n,N(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,i=I.get(r);if(i)return await i;const s=await t();if(!s)return;const l=(async(e,t,n,a)=>{const o=N(t),r=await J(e,o);if(r?.complete)return r;let i=await D(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let s=0,l=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await K(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>s)s=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&a(d/g,c.download);const m=g?Math.min(d+A,g)-1:d+A-1,f=new AbortController,p=()=>f.abort(),h={size:d,total:g,etag:u};let w,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(w=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==w.status){if(await(w.body?.cancel().catch(()=>{})),g&&d>=g){await Q(i,{total:g,etag:u,complete:!0});continue}await V(i),await Q(i,{total:0,etag:"",complete:!1});continue}if(!w.ok||!w.body)throw await(w.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=X(w,h);if(!n){await w.body.cancel().catch(()=>{}),await V(i),await Q(i,{total:0,etag:"",complete:!1});continue}await Q(i,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?d:0,s=await Y({dir:i,start:n.start+r,skip:r,total:n.total,body:w.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await Q(i,{total:s,etag:n.etag,complete:!0});const t=await K(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(s,n,a,o).finally(()=>{I.delete(r)});return I.set(r,l),await l},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,N(e))},purge:async()=>{const t=await U();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},te="litertlm-models",ne=ee("litert-store"),ae={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0},oe=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(te)}catch{return}},re=new Map,ie=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,c.download),o.enqueue(e)}}))},se=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await b(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ce=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},le=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],i=M(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const s=a.slice(0,-1),c=(await se([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==i.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ce(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const i=o.length-r.length;for(let e=0;e<r.length;e++)if(!ce(o[i+e],r[e]))return!1;return!0})(o.sent,s);let d;if(l&&o.conversation)d=o.conversation;else{const a=await se(s,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:i.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:i.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=i.max_tokens,o.sent=[...s]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},de={backend:s.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:o,logging:r})=>{const i=e.litert;if(!i)throw new Error("Model has no LiteRT artifact");const l=await n.e(520).then(()=>n(507));if(await l.getOrLoadGlobalLiteRtLm(w().script),o?.aborted)throw new Error("Cancelled");const d=T(i.repo,i.file),g=await oe();let u;const m=g?await g.match(d):void 0;if(m?.body)u=m.body;else{const e=await ne.stored(d);e?.complete&&(u=e.file.stream())}if(!u){const e=await ne.download({url:d,signal:o,onProgress:a}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;r&&console.log("Local model store failed",e)});e&&(u=e.file.stream())}if(!u&&g)try{await(async(e,t,n,a)=>{const o=re.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),i=ie(o.body,r,a);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{re.delete(e)});return re.set(e,r),await r})(d,g,o,a);const e=await g.match(d);e?.body&&(u=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(u||(u=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return ie(a.body,E(a),n)})(d,o,a)),o?.aborted)throw new Error("Cancelled");a(1,c.load);const f=await l.Engine.create({model:u,mainExecutorSettings:{maxNumTokens:4096,...i.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:ae},...i.vision?{visionBackend:l.Backend.GPU}:{}});if(o?.aborted)throw await f.delete(),new Error("Cancelled");const p={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:s.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await z({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(C(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return le({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:f,model:e,mod:l,params:t,session:p}),unload:async()=>{p.conversation=void 0,p.sent=[],await f.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=T(t.repo,t.file),[a,o]=await Promise.all([oe(),ne.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of d){const t=e.litert;if(!t)continue;if(await ne.stored(T(t.repo,t.file)))return!0}const e=await oe();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=T(t.repo,t.file),a=await oe();await ne.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ne.purge(),"undefined"!=typeof caches&&await caches.delete(te)}catch{}}},ge=ee("wllama-store");let ue,me,fe="",pe=!1;const he=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,we=e=>{const t=h();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},ye=e=>{const t=e=>(...t)=>{pe&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},be=async()=>{if(ue)return ue;const e=await n.e(924).then(()=>n(577)),t=me||we(),a=new e.Wllama({default:t.wasm},{logger:ye(e.LoggerWithoutDebug)});return me||(me=t,fe=he(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),ue=a,a},ve=async()=>{const e=ue;ue=void 0;try{await(e?.exit())}catch{}},xe=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),ke=e=>{const t=e.cpu;return t?[{name:t.file,url:T(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:T(t.repo,t.mmproj.file)}]:[]]:[]},_e=async e=>{const t=ke(e);if(!t.length)return!1;for(const e of t){const t=await ge.stored(e.url);if(!t?.complete)return!1}return!0},Me=async(e,t)=>{const n=await e.cacheManager.list();return ke(t).every(e=>n.some(t=>xe(t,e.name)))},Te=async({model:e,signal:t,onProgress:n})=>{const a=ke(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await ge.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Ee=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await b(t.url)});t.push({role:"user",content:e})}return t},Ce={backend:s.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{pe=!0===o;const i=e.cpu;if(!i)throw new Error("Model has no CPU artifact");const l=we(r),d=he(l);ue&&d!=fe&&await ve(),me=l,fe=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await be(),p=await _e(e),h=i.mmproj?.file,w=T(i.repo,i.file),y=h?T(i.repo,h):"";if(a?.aborted)throw await ve(),new Error("Cancelled");if(!p&&h){const e=await f.cacheManager.list(),t=e.some(e=>xe(e,i.file)),n=e.some(e=>xe(e,h));t&&!n&&await f.cacheManager.deleteMany(e=>xe(e,i.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Te({model:e,signal:a,onProgress:n}):void 0;t||p||await Me(f,e).catch(()=>!1)||(t=await Te({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,c.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:w,mmprojUrl:y}:w,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?c.load:c.download)}})}catch(e){throw await ve(),e}if(a?.aborted)throw await ve(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await ve(),new Error("Failed to create model context");return{model:e,backend:s.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(pe=!0===n.logging,e.isModelLoaded())return await z({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(C(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Ee(a),...M(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(ue==f)await ve();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await _e(e)||await Me(await be(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of d.flatMap(e=>ke(e)))if(await ge.stored(e.url))return!0;const e=await be();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=ke(e);try{for(const e of t)await ge.remove(e.url)}catch{}try{const e=await be();await e.cacheManager.deleteMany(e=>t.some(t=>xe(e,t.name)))}catch{}},purge:async()=>{try{await ge.purge()}catch{}try{const e=await be();await e.cacheManager.clear()}catch{}}},Le=((e={})=>{const t=e.engines?.cpu||Ce,n=e.engines?.litert||de;let a,o,r,l,g,u,m,f=s.none,p=i.idle,h=[],w=0,y={percent:0,phase:c.download};const b=a=>{if(e.backend!=s.none)return e.backend==s.cpu?a.cpu?t:void 0:e.backend==s.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||c.download};for(const e of[...h])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{w++,u?.abort(),u=void 0,g=void 0,p==i.downloading&&x(i.idle)},_=(e,t)=>{const n=l;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{h=h.filter(t=>t!=e),queueMicrotask(()=>{!h.length&&l&&k()})})(r),r.settle(!1)};h.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==i.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:s,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(l&&g==e.id)return p(y.percent,y.phase),await _(p,m);l&&(k(),h=[],await l.catch(()=>!1));const T=++w,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const l=b(e);if(T!=w)return!1;if(!l)return x(i.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,n=a;a=void 0,o=void 0,r=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of d)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,l),T!=w)return!1;const g=await l.cached(e);if(T!=w)return!1;x(i.downloading),v(0,c.download),g&&v(1,c.load);const m=new AbortController;u=m;try{const d=await l.startLLM({model:e,limits:t,logging:s,signal:m.signal,assets:n,onProgress:(e,t)=>{T==w&&v(e,t)}});return T!=w||m.signal.aborted?(await d.unload(),!1):(a=d,o=e,r=l,f=l.backend,x(i.ready),v(1,c.load),!0)}catch(e){return T!=w||(a=void 0,o=void 0,r=void 0,s&&console.log("Local model failed",e),x(i.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{l==E&&(l=void 0,g=void 0,h=[])});return l=E,g=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,r=void 0,x(i.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==i.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Pe=(e,t,n)=>{const a=e.status(),o=g(t);return{availability:a==i.ready?r.online:a==i.error||a==i.unsupported?r.offline:r.away,status:a,backend:e.backend(),supportsImages:!!o&&f(o),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},je=(e=>{let t,n=0;const a=t=>t.input.onStatus?.(Pe(e,t.model??t.input.model,t));return r=>{t?.controller.abort();const s={token:++n,input:r,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?c.load:c.download,requesting:!1,cancelled:!1};return t=s,a(s),(async t=>{const{input:r}=t,i=!0===r.logging,s=await m(r.model);if(t.model=s?.id,t.token!=n)return;const l=r.messages.some(e=>e.role!=o.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==o.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=l||d;if(!s)return t.error="Unknown model",void a(t);if(!g)return void a(t);if(d&&!l&&!f(s))return t.error="Model does not accept images",void a(t);const u=_(r.limits,!!s.litert),p=await e.load({model:s,limits:u,logging:i,signal:t.controller.signal,onProgress:(e,o)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=o,a(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void a(t);const h=f(s),w=d&&!h?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!h&&!l)return t.error="Model does not accept images",void a(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=c.load,a(t);let y,b="",v="";try{y=await e.chat({messages:w,limits:u,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=n||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,r.onResponse?.(v))},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void a(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),a(t))})(s),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=i.ready?queueMicrotask(()=>a(t)):e.unload().then(()=>a(t)))})(s)}})(Le),$e=(Re=Le,async(e={})=>{const t=await m(e.model),n=n=>e.onStatus?.(Pe(Re,t?.id,n));if(!t)return n({requesting:!1,phase:c.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=c.download;const i=Re.ready()&&Re.model()?.id==t.id;return await Re.load({model:t,limits:_(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(i&&n({requesting:!1,phase:c.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var Re;module.exports=a})();
+(()=>{"use strict";var e={};const t={};function n(a){const o=t[a];if(void 0!==o)return o.exports;const r=t[a]={exports:{}};return e[a](r,r.exports,n),r.exports}n.m=e,n.d=(e,t)=>{for(var a in t)n.o(t,a)&&!n.o(e,a)&&Object.defineProperty(e,a,{enumerable:!0,get:t[a]})},n.f={},n.e=e=>{const t=[];return n.f.require(e,t),Promise.all(t)},n.u=e=>({520:"engine-litert",924:"engine-wllama"}[e]+".node.min.js"),n.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),n.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={792:1};n.f.require=(t,a)=>{if(!e[t]){const a=require("./"+n.u(t));e[t]||(t=>{const a=t.modules,o=t.ids,r=t.runtime;for(var i in a)n.o(a,i)&&(n.m[i]=a[i]);r&&r(n);for(var s=0;s<o.length;s++)e[o[s]]=1})(a)}}})();let a={};var o,r,i,s,c,l;n.r(a),n.d(a,{ChatModelId:()=>l,ChatStatus:()=>r,LlmBackend:()=>s,LlmPhase:()=>c,LlmRole:()=>o,LlmStatus:()=>i,createChat:()=>Re,preloadModel:()=>qe}),function(e){e.system="system",e.user="user",e.assistant="assistant"}(o||(o={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(r||(r={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(i||(i={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(s||(s={})),function(e){e.download="download",e.load="load"}(c||(c={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(l||(l={}));const d=[{id:l.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:l.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],g=e=>e?d.find(t=>t.id==e):void 0,u=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?d.find(e=>!!e.litert):d.find(e=>!!e.cpu))?.id,m=async e=>g(e||await u()),f=e=>!!e.cpu?.mmproj||!!e.litert?.vision,p=require("node:path"),h=()=>{return e=(0,p.join)(__dirname,"../assets/wllama"),{wasm:(0,p.join)(e,"wllama.wasm"),compatWasm:(0,p.join)(e,"compat/wllama.wasm"),compatWorker:(0,p.join)(e,"compat/wllama.js")};var e},w=()=>{return e=(0,p.join)(__dirname,"../assets/litert"),{script:(0,p.join)(e,"litertlm_wasm_internal.js"),wasm:(0,p.join)(e,"litertlm_wasm_internal.wasm")};var e},y=288,b=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},v=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),i=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(y,y),s=i?i.getContext("2d"):null;if(!i||!s||a<1)return n.close(),e;if(s.drawImage(n,o,r,a,a,0,0,y,y),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(i)){const t=await i.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return i.toDataURL("image/jpeg",.85)}catch{return e}},x={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},k=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,M=(e={},t=!1)=>{const n=x;return{maxReplyTokens:k(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:k(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:k(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:k(e.maxImages,n.maxImages,0),maxContentLength:k(e.maxContentLength,n.maxContentLength,1),promptCharBudget:k(e.promptCharBudget,n.promptCharBudget,1),maxThreads:k(e.maxThreads,n.maxThreads,1)}},_=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),T=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,E=e=>Number(e.headers.get("content-length"))||0,C=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),L=["<|channel>thought\n","<|channel>thought","..."],P=["<channel|>","..."],j=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,S=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,$=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...S(e),{type:"text",text:"\n"},...S(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class R extends Error{}const q=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new R(n)),t)})])}finally{a&&clearTimeout(a)}},W=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},B=async({model:e,logging:t,onReason:n,onText:a,onSpeed:o,start:r,stopped:i,firstTimeoutMs:s=18e4,idleTimeoutMs:c=3e4,onStall:l})=>{let d="",g="",u="",m=!1,f=0,p=0;const h=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:i=>{let s=n+i;for(n="";s;){if(a){const{at:t,tag:i}=o(s,P);if(t>=0){t&&e(s.slice(0,t)),s=s.slice(t+i.length),a=!1;continue}const c=r(s,P);e(s.slice(0,s.length-c.length)),n=c;break}{const{at:e,tag:i}=o(s,L);if(e>=0){e&&t(s.slice(0,e)),s=s.slice(e+i.length),s.startsWith("\n")&&(s=s.slice(1)),a=!0;continue}const c=r(s,L);t(s.slice(0,s.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{g+=e,n(e)},e=>{d+=e,a(e)});try{const e=(await q(r(),s,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(i()){W(e);break}let a;try{a=await q(e.next(),m?c:s,m?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof R&&(d||g)){l?.(),W(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;m=!0;const r=a.value?.choices?.[0],w=r?.delta,y="string"==typeof w?.reasoning_content?w.reasoning_content:"string"==typeof w?.reasoning?w.reasoning:"";if(r?.finish_reason&&(u=r.finish_reason),y&&(g+=y,n(y)),w?.content&&h.feed(w.content),y||w?.content){const e=performance.now();f||(p=e),f+=1,f>1&&e>p&&o?.(1e3*f/(e-p))}}h.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof R&&!i())throw e}const w=i()?void 0:d||void 0;return t&&(console.log("Local chat reply",e.id,w?"****":""),console.log("Local chat finish",e.id,u||"none",d.length,g.length)),w},I=async({model:e,params:t,start:n,stopped:a,onStall:r})=>{const i=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await v(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=x)=>{const n=t.maxContentLength,a=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=o.user||!e.images?.length)),r=[],i=[];let s=0,c=t.maxImages;for(let e=a.length-1;e>=0;e--){const l=a[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==o.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&s+d.length>t.promptCharBudget)break;s+=d.length,c-=g.length,l.role==o.system?r.unshift(d):l.role==o.assistant?i.unshift({role:"assistant",content:d}):i.unshift({role:"user",content:j(d,g)})}}for(;i.length&&"user"!=i[0].role;)i.shift();const l=[];for(const e of i){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=$(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=r.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,i),await B({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,onSpeed:t.onSpeed,start:()=>n(i),stopped:a,onStall:r})},z=33554432,A=new Map;let O=!1;const N=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},F=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},U=async()=>{if(!O)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else O=!0},D=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},H=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},Q=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},V=async function*(e){for await(const t of e)yield t},G=async e=>{try{for await(const[t,n]of V(e))"file"==n.kind&&F(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},J=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await H(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},K=async(e,t)=>{const n=await D(e,t);if(n){const e=await(async e=>await H(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of V(e)){const e=F(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await J(e,t)},X=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:E(e)||t.total,etag:r||t.etag}},Y=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:i})=>{let s,l=t,d=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(l),{create:!0});s=await t.createWritable(),d=0},m=async()=>{const e=s;s=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;i();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){s||await u();const e=Math.min(z-d,o.byteLength-f);await s.write({type:"write",position:d,data:o.subarray(f,f+e)}),f+=e,d+=e,l+=e,d==z&&await m()}l>t&&r(a?l/a:0,c.download)}await m()}catch(e){throw await m(),e}return l},Z=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),ee=e=>{const t=async()=>{const t=await U();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await K(n,N(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,i=A.get(r);if(i)return await i;const s=await t();if(!s)return;const l=(async(e,t,n,a)=>{const o=N(t),r=await J(e,o);if(r?.complete)return r;let i=await D(e,o);i||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),i=await e.getDirectoryHandle(o,{create:!0}));let s=0,l=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await K(e,o);if(r?.complete)return r;const d=r?.size??0,g=r?.total||0,u=r?.etag||"";if(d>s)s=d,l=0;else if(++l>4)throw new Error("Model download failed");d&&g&&a(d/g,c.download);const m=g?Math.min(d+z,g)-1:d+z-1,f=new AbortController,p=()=>f.abort(),h={size:d,total:g,etag:u};let w,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(w=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${d}-${m}`}}),416==w.status){if(await(w.body?.cancel().catch(()=>{})),g&&d>=g){await Q(i,{total:g,etag:u,complete:!0});continue}await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}if(!w.ok||!w.body)throw await(w.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=X(w,h);if(!n){await w.body.cancel().catch(()=>{}),await G(i),await Q(i,{total:0,etag:"",complete:!1});continue}await Q(i,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?d:0,s=await Y({dir:i,start:n.start+r,skip:r,total:n.total,body:w.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await Q(i,{total:s,etag:n.etag,complete:!0});const t=await K(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*l))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(s,n,a,o).finally(()=>{A.delete(r)});return A.set(r,l),await l},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,N(e))},purge:async()=>{const t=await U();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},te="litertlm-models",ne=ee("litert-store"),ae={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0};let oe=!1;const re=(...e)=>{oe&&console.log(...e)},ie=(...e)=>{oe&&console.error(...e)},se=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(te)}catch{return}},ce=new Map,le=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,c.download),o.enqueue(e)}}))},de=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await b(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ge=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},ue=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],i=_(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const s=a.slice(0,-1),c=(await de([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==i.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ge(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const i=o.length-r.length;for(let e=0;e<r.length;e++)if(!ge(o[i+e],r[e]))return!1;return!0})(o.sent,s);let d;if(l&&o.conversation)d=o.conversation;else{const a=await de(s,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:i.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:i.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=i.max_tokens,o.sent=[...s]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},me={backend:s.gpu,startLLM:async({model:e,limits:t,onProgress:a,signal:o,logging:r})=>{oe=!0===r;const i=e.litert;if(!i)throw new Error("Model has no LiteRT artifact");const l=await n.e(520).then(()=>n(507)),d=globalThis,g=d.Module;d.Module={print:re,printErr:ie};try{await l.getOrLoadGlobalLiteRtLm(w().script)}finally{d.Module=g}if(o?.aborted)throw new Error("Cancelled");const u=T(i.repo,i.file),m=await se();let f;const p=m?await m.match(u):void 0;if(p?.body)f=p.body;else{const e=await ne.stored(u);e?.complete&&(f=e.file.stream())}if(!f){const e=await ne.download({url:u,signal:o,onProgress:a}).catch(e=>{if(o?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;r&&console.log("Local model store failed",e)});e&&(f=e.file.stream())}if(!f&&m)try{await(async(e,t,n,a)=>{const o=ce.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=E(o),i=le(o.body,r,a);await t.put(e,new Response(i,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{ce.delete(e)});return ce.set(e,r),await r})(u,m,o,a);const e=await m.match(u);e?.body&&(f=e.body)}catch(e){if(o?.aborted)throw new Error("Cancelled");r&&console.log("Local model cache failed",e)}if(f||(f=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return le(a.body,E(a),n)})(u,o,a)),o?.aborted)throw new Error("Cancelled");a(1,c.load);const h=await l.Engine.create((({model:e,limits:t,source:n,mod:a})=>({model:n,mainExecutorSettings:{maxNumTokens:4096,...e.litert?.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:ae},...e.litert?.vision?{visionBackend:a.Backend.GPU,maxVisionTokensPerImage:70,minVisionTokensPerImage:70}:{}}))({model:e,limits:t,source:f,mod:l}));if(o?.aborted)throw await h.delete(),new Error("Cancelled");const y={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:s.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{oe=!0===a.logging;const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await I({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(C(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return ue({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:h,model:e,mod:l,params:t,session:y}),unload:async()=>{y.conversation=void 0,y.sent=[],await h.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=T(t.repo,t.file),[a,o]=await Promise.all([se(),ne.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of d){const t=e.litert;if(!t)continue;if(await ne.stored(T(t.repo,t.file)))return!0}const e=await se();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=T(t.repo,t.file),a=await se();await ne.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ne.purge(),"undefined"!=typeof caches&&await caches.delete(te)}catch{}}},fe=ee("wllama-store");let pe,he,we="",ye=!1;const be=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ve=e=>{const t=h();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},xe=e=>{const t=e=>(...t)=>{ye&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},ke=async()=>{if(pe)return pe;const e=await n.e(924).then(()=>n(577)),t=he||ve(),a=new e.Wllama({default:t.wasm},{logger:xe(e.LoggerWithoutDebug)});return he||(he=t,we=be(t)),a.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),pe=a,a},Me=async()=>{const e=pe;pe=void 0;try{await(e?.exit())}catch{}},_e=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),Te=e=>{const t=e.cpu;return t?[{name:t.file,url:T(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:T(t.repo,t.mmproj.file)}]:[]]:[]},Ee=async e=>{const t=Te(e);if(!t.length)return!1;for(const e of t){const t=await fe.stored(e.url);if(!t?.complete)return!1}return!0},Ce=async(e,t)=>{const n=await e.cacheManager.list();return Te(t).every(e=>n.some(t=>_e(t,e.name)))},Le=async({model:e,signal:t,onProgress:n})=>{const a=Te(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await fe.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Pe=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await b(t.url)});t.push({role:"user",content:e})}return t},je={backend:s.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{ye=!0===o;const i=e.cpu;if(!i)throw new Error("Model has no CPU artifact");const l=ve(r),d=be(l);pe&&d!=we&&await Me(),he=l,we=d;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await ke(),p=await Ee(e),h=i.mmproj?.file,w=T(i.repo,i.file),y=h?T(i.repo,h):"";if(a?.aborted)throw await Me(),new Error("Cancelled");if(!p&&h){const e=await f.cacheManager.list(),t=e.some(e=>_e(e,i.file)),n=e.some(e=>_e(e,h));t&&!n&&await f.cacheManager.deleteMany(e=>_e(e,i.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Le({model:e,signal:a,onProgress:n}):void 0;t||p||await Ce(f,e).catch(()=>!1)||(t=await Le({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!Z(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,c.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:w,mmprojUrl:y}:w,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?c.load:c.download)}})}catch(e){throw await Me(),e}if(a?.aborted)throw await Me(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await Me(),new Error("Failed to create model context");return{model:e,backend:s.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(ye=!0===n.logging,e.isModelLoaded())return await I({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(C(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Pe(a),..._(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(pe==f)await Me();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Ee(e)||await Ce(await ke(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of d.flatMap(e=>Te(e)))if(await fe.stored(e.url))return!0;const e=await ke();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=Te(e);try{for(const e of t)await fe.remove(e.url)}catch{}try{const e=await ke();await e.cacheManager.deleteMany(e=>t.some(t=>_e(e,t.name)))}catch{}},purge:async()=>{try{await fe.purge()}catch{}try{const e=await ke();await e.cacheManager.clear()}catch{}}},Se=((e={})=>{const t=e.engines?.cpu||je,n=e.engines?.litert||me;let a,o,r,l,g,u,m,f=s.none,p=i.idle,h=[],w=0,y={percent:0,phase:c.download};const b=a=>{if(e.backend!=s.none)return e.backend==s.cpu?a.cpu?t:void 0:e.backend==s.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||c.download};for(const e of[...h])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{w++,u?.abort(),u=void 0,g=void 0,p==i.downloading&&x(i.idle)},M=(e,t)=>{const n=l;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{h=h.filter(t=>t!=e),queueMicrotask(()=>{!h.length&&l&&k()})})(r),r.settle(!1)};h.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},_=()=>p==i.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:s,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(_()&&o?.id==e.id)return!0;if(l&&g==e.id)return p(y.percent,y.phase),await M(p,m);l&&(k(),h=[],await l.catch(()=>!1));const T=++w,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const l=b(e);if(T!=w)return!1;if(!l)return x(i.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,n=a;a=void 0,o=void 0,r=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of d)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,l),T!=w)return!1;const g=await l.cached(e);if(T!=w)return!1;x(i.downloading),v(0,c.download),g&&v(1,c.load);const m=new AbortController;u=m;try{const d=await l.startLLM({model:e,limits:t,logging:s,signal:m.signal,assets:n,onProgress:(e,t)=>{T==w&&v(e,t)}});return T!=w||m.signal.aborted?(await d.unload(),!1):(a=d,o=e,r=l,f=l.backend,x(i.ready),v(1,c.load),!0)}catch(e){return T!=w||(a=void 0,o=void 0,r=void 0,s&&console.log("Local model failed",e),x(i.error)),!1}finally{u==m&&(u=void 0)}})().finally(()=>{l==E&&(l=void 0,g=void 0,h=[])});return l=E,g=e.id,await M(p,m)},chat:async e=>{if(!_()||!a)return;for(;m;)await m.catch(()=>{});if(!_()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,r=void 0,x(i.idle),await(e?.unload())},ready:_,loading:()=>!_()&&p==i.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),$e=(e,t,n)=>{const a=e.status(),o=g(t);return{availability:a==i.ready?r.online:a==i.error||a==i.unsupported?r.offline:r.away,status:a,backend:e.backend(),supportsImages:!!o&&f(o),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},Re=(e=>{let t,n=0;const a=t=>t.input.onStatus?.($e(e,t.model??t.input.model,t));return r=>{t?.controller.abort();const s={token:++n,input:r,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?c.load:c.download,requesting:!1,cancelled:!1};return t=s,a(s),(async t=>{const{input:r}=t,i=!0===r.logging,s=await m(r.model);if(t.model=s?.id,t.token!=n)return;const l=r.messages.some(e=>e.role!=o.system&&"string"==typeof e.content&&!!e.content.trim()),d=r.messages.some(e=>e.role==o.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=l||d;if(!s)return t.error="Unknown model",void a(t);if(!g)return void a(t);if(d&&!l&&!f(s))return t.error="Model does not accept images",void a(t);const u=M(r.limits,!!s.litert),p=await e.load({model:s,limits:u,logging:i,signal:t.controller.signal,onProgress:(e,o)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=o,a(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!p)return t.error||(t.error="Model not ready"),void a(t);const h=f(s),w=d&&!h?r.messages.map(({images:e,...t})=>t):r.messages;if(d&&!h&&!l)return t.error="Model does not accept images",void a(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=c.load,a(t);let y,b="",v="";try{y=await e.chat({messages:w,limits:u,reasoning:!0===r.reasoning,logging:i,onReason:e=>{t.token!=n||t.cancelled||(b+=e,r.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,r.onResponse?.(v))},onSpeed:e=>{t.token!=n||t.cancelled||r.onSpeed?.(e)},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void a(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),a(t))})(s),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=i.ready?queueMicrotask(()=>a(t)):e.unload().then(()=>a(t)))})(s)}})(Se),qe=(We=Se,async(e={})=>{const t=await m(e.model),n=n=>e.onStatus?.($e(We,t?.id,n));if(!t)return n({requesting:!1,phase:c.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=c.download;const i=We.ready()&&We.model()?.id==t.id;return await We.load({model:t,limits:M(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(i&&n({requesting:!1,phase:c.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var We;module.exports=a})();

package/dist/node/chat.node.min.mjs

@@ -1,2 +1,2 @@
 /*! For license information please see chat.node.min.mjs.LICENSE.txt */
-import{dirname as e,join as t}from"node:path";import{fileURLToPath as n}from"node:url";var a={};const o={};function r(e){const t=o[e];if(void 0!==t)return t.exports;const n=o[e]={exports:{}};return a[e](n,n.exports,r),n.exports}var s,i,c,l,d,g;r.m=a,r.d=(e,t)=>{for(var n in t)r.o(t,n)&&!r.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},r.f={},r.e=e=>{const t=[];return r.f.j(e,t),Promise.all(t)},r.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),r.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.node.min.mjs"),924:()=>import("./engine-wllama.node.min.mjs")},t={792:0},n=e=>{let{__webpack_esm_ids__:n,__webpack_esm_modules__:a,__webpack_esm_runtime__:o}=e;var s,i,c=0;for(s in a)r.o(a,s)&&(r.m[s]=a[s]);for(o&&o(r);c<n.length;c++)i=n[c],r.o(t,i)&&t[i]&&t[i][0](),t[i]=0};r.f.j=(a,o)=>{let s=r.o(t,a)?t[a]:void 0;if(0!==s)if(s)o.push(s[1]);else{let r=e[a]().then(n,e=>{throw 0!==t[a]&&(t[a]=void 0),e});r=Promise.race([r,new Promise(e=>s=t[a]=[e])]),o.push(s[1]=r)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(s||(s={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(i||(i={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(c||(c={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(l||(l={})),function(e){e.download="download",e.load="load"}(d||(d={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(g||(g={}));const u=[{id:g.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:g.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],m=e=>e?u.find(t=>t.id==e):void 0,f=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?u.find(e=>!!e.litert):u.find(e=>!!e.cpu))?.id,p=async e=>m(e||await f()),h=e=>!!e.cpu?.mmproj||!!e.litert?.vision,w=()=>{return a=t(e(n("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/wllama"),{wasm:t(a,"wllama.wasm"),compatWasm:t(a,"compat/wllama.wasm"),compatWorker:t(a,"compat/wllama.js")};var a},y=()=>{return a=t(e(n("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/litert"),{script:t(a,"litertlm_wasm_internal.js"),wasm:t(a,"litertlm_wasm_internal.wasm")};var a},b=288,v=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},x=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),s=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(b,b),i=s?s.getContext("2d"):null;if(!s||!i||a<1)return n.close(),e;if(i.drawImage(n,o,r,a,a,0,0,b,b),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(s)){const t=await s.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return s.toDataURL("image/jpeg",.85)}catch{return e}},k={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},_=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,M=(e={},t=!1)=>{const n=k;return{maxReplyTokens:_(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:_(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:_(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:_(e.maxImages,n.maxImages,0),maxContentLength:_(e.maxContentLength,n.maxContentLength,1),promptCharBudget:_(e.promptCharBudget,n.promptCharBudget,1),maxThreads:_(e.maxThreads,n.maxThreads,1)}},T=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),E=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,C=e=>Number(e.headers.get("content-length"))||0,L=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),P=["<|channel>thought\n","<|channel>thought","..."],j=["<channel|>","..."],$=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,R=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,S=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...R(e),{type:"text",text:"\n"},...R(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class W extends Error{}const q=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new W(n)),t)})])}finally{a&&clearTimeout(a)}},z=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},B=async({model:e,logging:t,onReason:n,onText:a,start:o,stopped:r,firstTimeoutMs:s=18e4,idleTimeoutMs:i=3e4,onStall:c})=>{let l="",d="",g="",u=!1;const m=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:s=>{let i=n+s;for(n="";i;){if(a){const{at:t,tag:s}=o(i,j);if(t>=0){t&&e(i.slice(0,t)),i=i.slice(t+s.length),a=!1;continue}const c=r(i,j);e(i.slice(0,i.length-c.length)),n=c;break}{const{at:e,tag:s}=o(i,P);if(e>=0){e&&t(i.slice(0,e)),i=i.slice(e+s.length),i.startsWith("\n")&&(i=i.slice(1)),a=!0;continue}const c=r(i,P);t(i.slice(0,i.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{d+=e,n(e)},e=>{l+=e,a(e)});try{const e=(await q(o(),s,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(r()){z(e);break}let a;try{a=await q(e.next(),u?i:s,u?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof W&&(l||d)){c?.(),z(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;u=!0;const o=a.value?.choices?.[0],f=o?.delta,p="string"==typeof f?.reasoning_content?f.reasoning_content:"string"==typeof f?.reasoning?f.reasoning:"";o?.finish_reason&&(g=o.finish_reason),p&&(d+=p,n(p)),f?.content&&m.feed(f.content)}m.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof W&&!r())throw e}const f=r()?void 0:l||void 0;return t&&(console.log("Local chat reply",e.id,f?"****":""),console.log("Local chat finish",e.id,g||"none",l.length,d.length)),f},A=async({model:e,params:t,start:n,stopped:a,onStall:o})=>{const r=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await x(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=k)=>{const n=t.maxContentLength,a=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=s.user||!e.images?.length)),o=[],r=[];let i=0,c=t.maxImages;for(let e=a.length-1;e>=0;e--){const l=a[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==s.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&i+d.length>t.promptCharBudget)break;i+=d.length,c-=g.length,l.role==s.system?o.unshift(d):l.role==s.assistant?r.unshift({role:"assistant",content:d}):r.unshift({role:"user",content:$(d,g)})}}for(;r.length&&"user"!=r[0].role;)r.shift();const l=[];for(const e of r){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=S(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=o.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,r),await B({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,start:()=>n(r),stopped:a,onStall:o})},O=33554432,I=new Map;let N=!1;const F=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},U=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},D=async()=>{if(!N)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else N=!0},H=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},Q=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},G=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},V=async function*(e){for await(const t of e)yield t},J=async e=>{try{for await(const[t,n]of V(e))"file"==n.kind&&U(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},K=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await Q(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},X=async(e,t)=>{const n=await H(e,t);if(n){const e=await(async e=>await Q(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of V(e)){const e=U(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await K(e,t)},Y=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:C(e)||t.total,etag:r||t.etag}},Z=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:s})=>{let i,c=t,l=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(c),{create:!0});i=await t.createWritable(),l=0},m=async()=>{const e=i;i=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;s();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){i||await u();const e=Math.min(O-l,o.byteLength-f);await i.write({type:"write",position:l,data:o.subarray(f,f+e)}),f+=e,l+=e,c+=e,l==O&&await m()}c>t&&r(a?c/a:0,d.download)}await m()}catch(e){throw await m(),e}return c},ee=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),te=e=>{const t=async()=>{const t=await D();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await X(n,F(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,s=I.get(r);if(s)return await s;const i=await t();if(!i)return;const c=(async(e,t,n,a)=>{const o=F(t),r=await K(e,o);if(r?.complete)return r;let s=await H(e,o);s||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),s=await e.getDirectoryHandle(o,{create:!0}));let i=0,c=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await X(e,o);if(r?.complete)return r;const l=r?.size??0,g=r?.total||0,u=r?.etag||"";if(l>i)i=l,c=0;else if(++c>4)throw new Error("Model download failed");l&&g&&a(l/g,d.download);const m=g?Math.min(l+O,g)-1:l+O-1,f=new AbortController,p=()=>f.abort(),h={size:l,total:g,etag:u};let w,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(w=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${l}-${m}`}}),416==w.status){if(await(w.body?.cancel().catch(()=>{})),g&&l>=g){await G(s,{total:g,etag:u,complete:!0});continue}await J(s),await G(s,{total:0,etag:"",complete:!1});continue}if(!w.ok||!w.body)throw await(w.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=Y(w,h);if(!n){await w.body.cancel().catch(()=>{}),await J(s),await G(s,{total:0,etag:"",complete:!1});continue}await G(s,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?l:0,i=await Z({dir:s,start:n.start+r,skip:r,total:n.total,body:w.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await G(s,{total:i,etag:n.etag,complete:!0});const t=await X(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*c))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(i,n,a,o).finally(()=>{I.delete(r)});return I.set(r,c),await c},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,F(e))},purge:async()=>{const t=await D();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},ne="litertlm-models",ae=te("litert-store"),oe={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0},re=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(ne)}catch{return}},se=new Map,ie=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,d.download),o.enqueue(e)}}))},ce=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await v(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},le=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},de=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],s=T(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const i=a.slice(0,-1),c=(await ce([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==s.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!le(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const s=o.length-r.length;for(let e=0;e<r.length;e++)if(!le(o[s+e],r[e]))return!1;return!0})(o.sent,i);let d;if(l&&o.conversation)d=o.conversation;else{const a=await ce(i,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:s.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:s.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=s.max_tokens,o.sent=[...i]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},ge={backend:l.gpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o})=>{const s=e.litert;if(!s)throw new Error("Model has no LiteRT artifact");const i=await r.e(520).then(()=>r(507));if(await i.getOrLoadGlobalLiteRtLm(y().script),a?.aborted)throw new Error("Cancelled");const c=E(s.repo,s.file),g=await re();let u;const m=g?await g.match(c):void 0;if(m?.body)u=m.body;else{const e=await ae.stored(c);e?.complete&&(u=e.file.stream())}if(!u){const e=await ae.download({url:c,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!ee(e))throw e;o&&console.log("Local model store failed",e)});e&&(u=e.file.stream())}if(!u&&g)try{await(async(e,t,n,a)=>{const o=se.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=C(o),s=ie(o.body,r,a);await t.put(e,new Response(s,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{se.delete(e)});return se.set(e,r),await r})(c,g,a,n);const e=await g.match(c);e?.body&&(u=e.body)}catch(e){if(a?.aborted)throw new Error("Cancelled");o&&console.log("Local model cache failed",e)}if(u||(u=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return ie(a.body,C(a),n)})(c,a,n)),a?.aborted)throw new Error("Cancelled");n(1,d.load);const f=await i.Engine.create({model:u,mainExecutorSettings:{maxNumTokens:4096,...s.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:oe},...s.vision?{visionBackend:i.Backend.GPU}:{}});if(a?.aborted)throw await f.delete(),new Error("Cancelled");const p={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:l.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await A({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(L(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return de({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:f,model:e,mod:i,params:t,session:p}),unload:async()=>{p.conversation=void 0,p.sent=[],await f.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=E(t.repo,t.file),[a,o]=await Promise.all([re(),ae.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of u){const t=e.litert;if(!t)continue;if(await ae.stored(E(t.repo,t.file)))return!0}const e=await re();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=E(t.repo,t.file),a=await re();await ae.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ae.purge(),"undefined"!=typeof caches&&await caches.delete(ne)}catch{}}},ue=te("wllama-store");let me,fe,pe="",he=!1;const we=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,ye=e=>{const t=w();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},be=e=>{const t=e=>(...t)=>{he&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},ve=async()=>{if(me)return me;const e=await r.e(924).then(()=>r(577)),t=fe||ye(),n=new e.Wllama({default:t.wasm},{logger:be(e.LoggerWithoutDebug)});return fe||(fe=t,pe=we(t)),n.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),me=n,n},xe=async()=>{const e=me;me=void 0;try{await(e?.exit())}catch{}},ke=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),_e=e=>{const t=e.cpu;return t?[{name:t.file,url:E(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:E(t.repo,t.mmproj.file)}]:[]]:[]},Me=async e=>{const t=_e(e);if(!t.length)return!1;for(const e of t){const t=await ue.stored(e.url);if(!t?.complete)return!1}return!0},Te=async(e,t)=>{const n=await e.cacheManager.list();return _e(t).every(e=>n.some(t=>ke(t,e.name)))},Ee=async({model:e,signal:t,onProgress:n})=>{const a=_e(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await ue.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},Ce=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await v(t.url)});t.push({role:"user",content:e})}return t},Le={backend:l.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{he=!0===o;const s=e.cpu;if(!s)throw new Error("Model has no CPU artifact");const i=ye(r),c=we(i);me&&c!=pe&&await xe(),fe=i,pe=c;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await ve(),p=await Me(e),h=s.mmproj?.file,w=E(s.repo,s.file),y=h?E(s.repo,h):"";if(a?.aborted)throw await xe(),new Error("Cancelled");if(!p&&h){const e=await f.cacheManager.list(),t=e.some(e=>ke(e,s.file)),n=e.some(e=>ke(e,h));t&&!n&&await f.cacheManager.deleteMany(e=>ke(e,s.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Ee({model:e,signal:a,onProgress:n}):void 0;t||p||await Te(f,e).catch(()=>!1)||(t=await Ee({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!ee(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,d.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:w,mmprojUrl:y}:w,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?d.load:d.download)}})}catch(e){throw await xe(),e}if(a?.aborted)throw await xe(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await xe(),new Error("Failed to create model context");return{model:e,backend:l.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(he=!0===n.logging,e.isModelLoaded())return await A({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(L(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await Ce(a),...T(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(me==f)await xe();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Me(e)||await Te(await ve(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of u.flatMap(e=>_e(e)))if(await ue.stored(e.url))return!0;const e=await ve();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=_e(e);try{for(const e of t)await ue.remove(e.url)}catch{}try{const e=await ve();await e.cacheManager.deleteMany(e=>t.some(t=>ke(e,t.name)))}catch{}},purge:async()=>{try{await ue.purge()}catch{}try{const e=await ve();await e.cacheManager.clear()}catch{}}},Pe=((e={})=>{const t=e.engines?.cpu||Le,n=e.engines?.litert||ge;let a,o,r,s,i,g,m,f=l.none,p=c.idle,h=[],w=0,y={percent:0,phase:d.download};const b=a=>{if(e.backend!=l.none)return e.backend==l.cpu?a.cpu?t:void 0:e.backend==l.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||d.download};for(const e of[...h])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{w++,g?.abort(),g=void 0,i=void 0,p==c.downloading&&x(c.idle)},_=(e,t)=>{const n=s;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{h=h.filter(t=>t!=e),queueMicrotask(()=>{!h.length&&s&&k()})})(r),r.settle(!1)};h.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==c.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:l,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(s&&i==e.id)return p(y.percent,y.phase),await _(p,m);s&&(k(),h=[],await s.catch(()=>!1));const T=++w,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const s=b(e);if(T!=w)return!1;if(!s)return x(c.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,n=a;a=void 0,o=void 0,r=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of u)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,s),T!=w)return!1;const i=await s.cached(e);if(T!=w)return!1;x(c.downloading),v(0,d.download),i&&v(1,d.load);const m=new AbortController;g=m;try{const i=await s.startLLM({model:e,limits:t,logging:l,signal:m.signal,assets:n,onProgress:(e,t)=>{T==w&&v(e,t)}});return T!=w||m.signal.aborted?(await i.unload(),!1):(a=i,o=e,r=s,f=s.backend,x(c.ready),v(1,d.load),!0)}catch(e){return T!=w||(a=void 0,o=void 0,r=void 0,l&&console.log("Local model failed",e),x(c.error)),!1}finally{g==m&&(g=void 0)}})().finally(()=>{s==E&&(s=void 0,i=void 0,h=[])});return s=E,i=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,r=void 0,x(c.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==c.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),je=(e,t,n)=>{const a=e.status(),o=m(t);return{availability:a==c.ready?i.online:a==c.error||a==c.unsupported?i.offline:i.away,status:a,backend:e.backend(),supportsImages:!!o&&h(o),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},$e=(e=>{let t,n=0;const a=t=>t.input.onStatus?.(je(e,t.model??t.input.model,t));return o=>{t?.controller.abort();const r={token:++n,input:o,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?d.load:d.download,requesting:!1,cancelled:!1};return t=r,a(r),(async t=>{const{input:o}=t,r=!0===o.logging,i=await p(o.model);if(t.model=i?.id,t.token!=n)return;const c=o.messages.some(e=>e.role!=s.system&&"string"==typeof e.content&&!!e.content.trim()),l=o.messages.some(e=>e.role==s.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=c||l;if(!i)return t.error="Unknown model",void a(t);if(!g)return void a(t);if(l&&!c&&!h(i))return t.error="Model does not accept images",void a(t);const u=M(o.limits,!!i.litert),m=await e.load({model:i,limits:u,logging:r,signal:t.controller.signal,onProgress:(e,o)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=o,a(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!m)return t.error||(t.error="Model not ready"),void a(t);const f=h(i),w=l&&!f?o.messages.map(({images:e,...t})=>t):o.messages;if(l&&!f&&!c)return t.error="Model does not accept images",void a(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=d.load,a(t);let y,b="",v="";try{y=await e.chat({messages:w,limits:u,reasoning:!0===o.reasoning,logging:r,onReason:e=>{t.token!=n||t.cancelled||(b+=e,o.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,o.onResponse?.(v))},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void a(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),a(t))})(r),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=c.ready?queueMicrotask(()=>a(t)):e.unload().then(()=>a(t)))})(r)}})(Pe),Re=(Se=Pe,async(e={})=>{const t=await p(e.model),n=n=>e.onStatus?.(je(Se,t?.id,n));if(!t)return n({requesting:!1,phase:d.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=d.download;const s=Se.ready()&&Se.model()?.id==t.id;return await Se.load({model:t,limits:M(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(s&&n({requesting:!1,phase:d.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var Se;export{g as ChatModelId,i as ChatStatus,l as LlmBackend,d as LlmPhase,s as LlmRole,c as LlmStatus,$e as createChat,Re as preloadModel};
+import{dirname as e,join as t}from"node:path";import{fileURLToPath as n}from"node:url";var a={};const o={};function r(e){const t=o[e];if(void 0!==t)return t.exports;const n=o[e]={exports:{}};return a[e](n,n.exports,r),n.exports}var s,i,c,l,d,g;r.m=a,r.d=(e,t)=>{for(var n in t)r.o(t,n)&&!r.o(e,n)&&Object.defineProperty(e,n,{enumerable:!0,get:t[n]})},r.f={},r.e=e=>{const t=[];return r.f.j(e,t),Promise.all(t)},r.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),r.r=e=>{Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},(()=>{const e={520:()=>import("./engine-litert.node.min.mjs"),924:()=>import("./engine-wllama.node.min.mjs")},t={792:0},n=e=>{let{__webpack_esm_ids__:n,__webpack_esm_modules__:a,__webpack_esm_runtime__:o}=e;var s,i,c=0;for(s in a)r.o(a,s)&&(r.m[s]=a[s]);for(o&&o(r);c<n.length;c++)i=n[c],r.o(t,i)&&t[i]&&t[i][0](),t[i]=0};r.f.j=(a,o)=>{let s=r.o(t,a)?t[a]:void 0;if(0!==s)if(s)o.push(s[1]);else{let r=e[a]().then(n,e=>{throw 0!==t[a]&&(t[a]=void 0),e});r=Promise.race([r,new Promise(e=>s=t[a]=[e])]),o.push(s[1]=r)}}})(),function(e){e.system="system",e.user="user",e.assistant="assistant"}(s||(s={})),function(e){e[e.online=0]="online",e[e.away=1]="away",e[e.offline=2]="offline"}(i||(i={})),function(e){e[e.idle=0]="idle",e[e.downloading=1]="downloading",e[e.ready=2]="ready",e[e.error=3]="error",e[e.unsupported=4]="unsupported"}(c||(c={})),function(e){e.cpu="cpu",e.gpu="gpu",e.none="none"}(l||(l={})),function(e){e.download="download",e.load="load"}(d||(d={})),function(e){e.qwen3_vl_2b="qwen3-vl-2b",e.gemma4_e2b="gemma4-e2b"}(g||(g={}));const u=[{id:g.qwen3_vl_2b,reasoning:!0,cpu:{repo:"Qwen/Qwen3-VL-2B-Thinking-GGUF",file:"Qwen3VL-2B-Thinking-Q4_K_M.gguf",mmproj:{file:"mmproj-Qwen3VL-2B-Thinking-Q8_0.gguf"}}},{id:g.gemma4_e2b,reasoning:!0,litert:{repo:"litert-community/gemma-4-E2B-it-litert-lm",file:"gemma-4-E2B-it.litertlm",vision:!0}}],m=e=>e?u.find(t=>t.id==e):void 0,f=async()=>(await(async()=>{const e="undefined"==typeof navigator?void 0:navigator.gpu;if(!e)return!1;try{return!!await e.requestAdapter({powerPreference:"high-performance"})}catch{return!1}})()?u.find(e=>!!e.litert):u.find(e=>!!e.cpu))?.id,p=async e=>m(e||await f()),h=e=>!!e.cpu?.mmproj||!!e.litert?.vision,w=()=>{return a=t(e(n("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/wllama"),{wasm:t(a,"wllama.wasm"),compatWasm:t(a,"compat/wllama.wasm"),compatWorker:t(a,"compat/wllama.js")};var a},y=()=>{return a=t(e(n("file:///media/x/store/code/DegreeSign/ds_chat/src/assets/nodeEsm.js")),"../assets/litert"),{script:t(a,"litertlm_wasm_internal.js"),wasm:t(a,"litertlm_wasm_internal.wasm")};var a},b=288,v=async e=>{if(!e.startsWith("data:"))return await(await fetch(e)).arrayBuffer();const t=e.indexOf(","),n=t>=0?e.slice(0,t):"";if(t<0)throw new Error("Invalid image data");if(!n.includes(";base64"))return await(await fetch(e)).arrayBuffer();const a=atob(e.slice(t+1)),o=new Uint8Array(a.length);for(let e=0;e<a.length;e++)o[e]=a.charCodeAt(e);return o.buffer},x=async e=>{if(!e||"function"!=typeof fetch||"function"!=typeof createImageBitmap)return e;try{const t=await fetch(e),n=await createImageBitmap(await t.blob()),a=Math.min(n.width,n.height),o=Math.floor((n.width-a)/2),r=Math.floor((n.height-a)/2),s=((e,t)=>{if("undefined"!=typeof OffscreenCanvas)return new OffscreenCanvas(e,t);if("undefined"==typeof document)return;const n=document.createElement("canvas");return n.width=e,n.height=t,n})(b,b),i=s?s.getContext("2d"):null;if(!s||!i||a<1)return n.close(),e;if(i.drawImage(n,o,r,a,a,0,0,b,b),n.close(),(e=>"undefined"!=typeof OffscreenCanvas&&e instanceof OffscreenCanvas)(s)){const t=await s.convertToBlob({type:"image/jpeg",quality:.85});return await(e=>new Promise(t=>{if("undefined"==typeof FileReader)return void t("");const n=new FileReader;n.onerror=()=>t(""),n.onload=()=>t(`${n.result}`),n.readAsDataURL(e)}))(t)||e}return s.toDataURL("image/jpeg",.85)}catch{return e}},k={maxReplyTokens:250,maxThinkingTokens:128,maxReasoningTokens:320,maxImages:2,maxContentLength:1e3,promptCharBudget:1e4,maxThreads:6},_=(e,t,n)=>"number"==typeof e&&Number.isFinite(e)&&e>=n?Math.floor(e):t,M=(e={},t=!1)=>{const n=k;return{maxReplyTokens:_(e.maxReplyTokens,n.maxReplyTokens,1),maxThinkingTokens:_(e.maxThinkingTokens,n.maxThinkingTokens,1),maxReasoningTokens:_(e.maxReasoningTokens,t?1200:n.maxReasoningTokens,1),maxImages:_(e.maxImages,n.maxImages,0),maxContentLength:_(e.maxContentLength,n.maxContentLength,1),promptCharBudget:_(e.promptCharBudget,n.promptCharBudget,1),maxThreads:_(e.maxThreads,n.maxThreads,1)}},T=(e,t)=>({stream:!0,temperature:.7,max_tokens:t?e.maxReasoningTokens:e.maxReplyTokens}),E=(e,t)=>`https://huggingface.co/${e}/resolve/main/${t}`,C=e=>Number(e.headers.get("content-length"))||0,L=e=>e.some(e=>Array.isArray(e.content)&&e.content.some(e=>"image"==e.type)),P=["<|channel>thought\n","<|channel>thought","..."],j=["<channel|>","..."],S=(e,t)=>t.length?[...e?[{type:"text",text:e}]:[],...t.map(e=>({type:"image",url:e}))]:e,$=e=>"string"==typeof e?e?[{type:"text",text:e}]:[]:e,R=(e,t)=>{if("string"==typeof e&&"string"==typeof t)return`${e}\n${t}`;const n=[];for(const a of[...$(e),{type:"text",text:"\n"},...$(t)]){const e=n[n.length-1];"text"==a.type&&"text"==e?.type?e.text+=a.text:n.push(a)}return n};class W extends Error{}const q=async(e,t,n)=>{let a;try{return await Promise.race([e,new Promise((e,o)=>{a=setTimeout(()=>o(new W(n)),t)})])}finally{a&&clearTimeout(a)}},z=e=>{try{Promise.resolve(e.return?.()).catch(()=>{})}catch{}},B=async({model:e,logging:t,onReason:n,onText:a,onSpeed:o,start:r,stopped:s,firstTimeoutMs:i=18e4,idleTimeoutMs:c=3e4,onStall:l})=>{let d="",g="",u="",m=!1,f=0,p=0;const h=((e,t)=>{let n="",a=!1;const o=(e,t)=>{let n=-1,a="";for(const o of t){const t=e.indexOf(o);t>=0&&(n<0||t<n)&&(n=t,a=o)}return{at:n,tag:a}},r=(e,t)=>{let n="";for(const a of t)for(let t=Math.min(e.length,a.length-1);t>n.length;t--)if(e.endsWith(a.slice(0,t))){n=e.slice(e.length-t);break}return n};return{feed:s=>{let i=n+s;for(n="";i;){if(a){const{at:t,tag:s}=o(i,j);if(t>=0){t&&e(i.slice(0,t)),i=i.slice(t+s.length),a=!1;continue}const c=r(i,j);e(i.slice(0,i.length-c.length)),n=c;break}{const{at:e,tag:s}=o(i,P);if(e>=0){e&&t(i.slice(0,e)),i=i.slice(e+s.length),i.startsWith("\n")&&(i=i.slice(1)),a=!0;continue}const c=r(i,P);t(i.slice(0,i.length-c.length)),n=c;break}}},flush:()=>{n&&(a?e(n):t(n),n="")}}})(e=>{g+=e,n(e)},e=>{d+=e,a(e)});try{const e=(await q(r(),i,"Model did not start"))[Symbol.asyncIterator]();for(;;){if(s()){z(e);break}let a;try{a=await q(e.next(),m?c:i,m?"Model stopped responding":"Model did not reply")}catch(n){if(n instanceof W&&(d||g)){l?.(),z(e),t&&console.log("Local chat stalled",n);break}throw n}if(a.done)break;m=!0;const r=a.value?.choices?.[0],w=r?.delta,y="string"==typeof w?.reasoning_content?w.reasoning_content:"string"==typeof w?.reasoning?w.reasoning:"";if(r?.finish_reason&&(u=r.finish_reason),y&&(g+=y,n(y)),w?.content&&h.feed(w.content),y||w?.content){const e=performance.now();f||(p=e),f+=1,f>1&&e>p&&o?.(1e3*f/(e-p))}}h.flush()}catch(e){if(t&&console.log("Local chat failed",e),e instanceof W&&!s())throw e}const w=s()?void 0:d||void 0;return t&&(console.log("Local chat reply",e.id,w?"****":""),console.log("Local chat finish",e.id,u||"none",d.length,g.length)),w},I=async({model:e,params:t,start:n,stopped:a,onStall:o})=>{const r=await(async e=>{const t=[];for(const n of e){if("string"==typeof n.content){t.push(n);continue}const e=[];for(const t of n.content)e.push("image"==t.type?{type:"image",url:await x(t.url)}:t);t.push({role:"user",content:e})}return t})(((e,t=k)=>{const n=t.maxContentLength,a=e.filter(e=>"string"==typeof e.content&&!!e.content.trim()||!(e.role!=s.user||!e.images?.length)),o=[],r=[];let i=0,c=t.maxImages;for(let e=a.length-1;e>=0;e--){const l=a[e],d=("string"==typeof l.content?l.content:"").slice(0,n),g=l.role==s.user?(l.images||[]).filter(e=>"string"==typeof e&&!!e).slice(0,Math.max(0,c)):[];if(d||g.length){if(d&&i+d.length>t.promptCharBudget)break;i+=d.length,c-=g.length,l.role==s.system?o.unshift(d):l.role==s.assistant?r.unshift({role:"assistant",content:d}):r.unshift({role:"user",content:S(d,g)})}}for(;r.length&&"user"!=r[0].role;)r.shift();const l=[];for(const e of r){const t=l[l.length-1];"assistant"==t?.role&&"assistant"==e.role?t.content=`${t.content}\n${e.content}`:"user"==t?.role&&"user"==e.role?t.content=R(t.content,e.content):"assistant"==e.role?l.push({role:"assistant",content:e.content}):l.push({role:"user",content:e.content})}const d=o.join("\n\n"),g=[];return d&&g.push({role:"system",content:d}),g.push(...l),g})(t.messages,t.limits));return t.logging&&((e,t)=>{console.log("Local chat call",e.id,t.map(({role:e,content:t})=>({role:e,content:"****",...Array.isArray(t)?{images:t.filter(e=>"image"==e.type).length}:{}})))})(e,r),await B({model:e,logging:t.logging,onReason:t.onReason,onText:t.onText,onSpeed:t.onSpeed,start:()=>n(r),stopped:a,onStall:o})},A=33554432,O=new Map;let N=!1;const F=e=>{const t=(e.split("/").pop()||"model").replace(/[^\w.-]+/g,"_");let n=0;for(let t=0;t<e.length;t++)n=31*n+e.charCodeAt(t)>>>0;return`${n.toString(16)}-${t}`},U=e=>{const t=/^part-(\d+)$/.exec(e);return t?Number(t[1]):-1},D=async()=>{if(!N)if("undefined"!=typeof navigator&&navigator.storage?.getDirectory)try{return await navigator.storage.getDirectory()}catch{return}else N=!0},H=async(e,t)=>{try{return await e.getDirectoryHandle(t)}catch{return}},Q=async e=>{if(e)try{const t=await e.getFile(),n=JSON.parse(await t.text());return{total:Number(n.total)||0,etag:`${n.etag||""}`,complete:!0===n.complete}}catch{return}},V=async(e,t)=>{try{const n=await e.getFileHandle("meta.json",{create:!0}),a=await n.createWritable();await a.write(JSON.stringify(t)),await a.close()}catch{}},G=async function*(e){for await(const t of e)yield t},J=async e=>{try{for await(const[t,n]of G(e))"file"==n.kind&&U(t)>=0&&await e.removeEntry(t).catch(()=>{})}catch{}},K=async(e,t)=>{try{const n=await(await e.getFileHandle(t)).getFile(),a=await Q(await e.getFileHandle(`${t}.json`).catch(()=>{}));if(!a)return;return{file:n,size:n.size,total:a.total,etag:a.etag,complete:a.total?n.size>=a.total:!0===a.complete}}catch{return}},X=async(e,t)=>{const n=await H(e,t);if(n){const e=await(async e=>await Q(await e.getFileHandle("meta.json").catch(()=>{})))(n),t=await(async e=>{const t=[];try{for await(const[n,a]of G(e)){const e=U(n);if("file"!=a.kind||e<0)continue;const o=await a.getFile();t.push({part:{start:e,size:o.size},file:o})}}catch{return}const n=(e=>{const t=[];let n=0;for(const a of[...e].sort((e,t)=>e.start-t.start))if(a.size){if(a.start!=n)break;t.push(a),n+=a.size}return t})(t.map(e=>e.part)),a=new Set(n.map(e=>e.start)),o=t.filter(e=>a.has(e.part.start)).sort((e,t)=>e.part.start-t.part.start).map(e=>e.file);if(o.length)return{file:1==o.length?o[0]:new Blob(o),size:n.reduce((e,t)=>e+t.size,0)}})(n),a=e?.total||0,o=t?.size??0,r=t?.file??new Blob;return{file:a&&o>a?r.slice(0,a):r,size:o,total:a,etag:e?.etag||"",complete:a?o>=a:!0===e?.complete}}return await K(e,t)},Y=(e,t)=>{const n=e.headers.get("content-range"),a=/bytes (\d+)-/.exec(n||""),o=/\/(\d+)\s*$/.exec(n||""),r=e.headers.get("etag")||"";if(206==e.status){const e=a?Number(a[1]):t.size,n=o?Number(o[1]):t.total;if(e!=t.size)return;if(t.total&&n&&t.total!=n)return;if(t.etag&&r&&t.etag!=r)return;return{start:e,total:n||t.total,etag:r||t.etag}}if(!t.etag||!r||t.etag==r)return{start:0,total:C(e)||t.total,etag:r||t.etag}},Z=async({dir:e,start:t,skip:n,total:a,body:o,onProgress:r,onChunk:s})=>{let i,c=t,l=0,g=n;const u=async()=>{const t=await e.getFileHandle((e=>`part-${e}`)(c),{create:!0});i=await t.createWritable(),l=0},m=async()=>{const e=i;i=void 0,e&&await e.close().catch(()=>{})};try{const e=o.getReader();for(;;){const{done:n,value:o}=await e.read();if(n)break;s();let f=0;for(g&&(f=Math.min(g,o.byteLength),g-=f);f<o.byteLength;){i||await u();const e=Math.min(A-l,o.byteLength-f);await i.write({type:"write",position:l,data:o.subarray(f,f+e)}),f+=e,l+=e,c+=e,l==A&&await m()}c>t&&r(a?c/a:0,d.download)}await m()}catch(e){throw await m(),e}return c},ee=e=>["NotAllowedError","NotSupportedError"].includes(e?.name||""),te=e=>{const t=async()=>{const t=await D();if(t)try{return await t.getDirectoryHandle(e,{create:!0})}catch{return}};return{stored:async e=>{const n=await t();return n?await X(n,F(e)):void 0},download:async({url:n,signal:a,onProgress:o})=>{const r=`${e}|${n}`,s=O.get(r);if(s)return await s;const i=await t();if(!i)return;const c=(async(e,t,n,a)=>{const o=F(t),r=await K(e,o);if(r?.complete)return r;let s=await H(e,o);s||(await e.removeEntry(o).catch(()=>{}),await e.removeEntry(`${o}.json`).catch(()=>{}),s=await e.getDirectoryHandle(o,{create:!0}));let i=0,c=0;for(;;){if(n?.aborted)throw new Error("Cancelled");const r=await X(e,o);if(r?.complete)return r;const l=r?.size??0,g=r?.total||0,u=r?.etag||"";if(l>i)i=l,c=0;else if(++c>4)throw new Error("Model download failed");l&&g&&a(l/g,d.download);const m=g?Math.min(l+A,g)-1:l+A-1,f=new AbortController,p=()=>f.abort(),h={size:l,total:g,etag:u};let w,y=setTimeout(p,3e4);n?.addEventListener("abort",p,{once:!0});try{if(w=await fetch(t,{signal:f.signal,headers:{Range:`bytes=${l}-${m}`}}),416==w.status){if(await(w.body?.cancel().catch(()=>{})),g&&l>=g){await V(s,{total:g,etag:u,complete:!0});continue}await J(s),await V(s,{total:0,etag:"",complete:!1});continue}if(!w.ok||!w.body)throw await(w.body?.cancel().catch(()=>{})),new Error("Model download failed");const n=Y(w,h);if(!n){await w.body.cancel().catch(()=>{}),await J(s),await V(s,{total:0,etag:"",complete:!1});continue}await V(s,{total:n.total,etag:n.etag,complete:!1});const r=0==n.start?l:0,i=await Z({dir:s,start:n.start+r,skip:r,total:n.total,body:w.body,onProgress:a,onChunk:()=>{clearTimeout(y),y=setTimeout(p,3e4)}});if(!n.total){await V(s,{total:i,etag:n.etag,complete:!0});const t=await X(e,o);if(t)return t}}catch(e){if(n?.aborted)throw new Error("Cancelled");if(e instanceof DOMException&&["QuotaExceededError","NotAllowedError","NotSupportedError","InvalidStateError","TypeMismatchError","SecurityError"].includes(e.name))throw e;await new Promise(e=>setTimeout(e,250+500*c))}finally{clearTimeout(y),n?.removeEventListener("abort",p)}}})(i,n,a,o).finally(()=>{O.delete(r)});return O.set(r,c),await c},remove:async e=>{const n=await t();n&&await(async(e,t)=>{await e.removeEntry(t,{recursive:!0}).catch(()=>{}),await e.removeEntry(`${t}.json`).catch(()=>{})})(n,F(e))},purge:async()=>{const t=await D();t&&await t.removeEntry(e,{recursive:!0}).catch(()=>{})}}},ne="litertlm-models",ae=te("litert-store"),oe={num_output_candidates:1,wait_for_weight_uploads:!0,num_decode_steps_per_sync:3,sequence_batch_size:0,supported_lora_ranks:[],max_top_k:40,enable_decode_logits:!1,enable_external_embeddings:!1,use_submodel:!1,use_autosized_ringbuffers:!0};let re=!1;const se=(...e)=>{re&&console.log(...e)},ie=(...e)=>{re&&console.error(...e)},ce=async()=>{try{if("undefined"==typeof caches)return;return await caches.open(ne)}catch{return}},le=new Map,de=(e,t,n)=>{let a=0;return e.pipeThrough(new TransformStream({transform:(e,o)=>{a+=e.byteLength,n(t?a/t:0,d.download),o.enqueue(e)}}))},ge=async(e,t=!0)=>{const n=[];for(const a of e){if("string"==typeof a.content){n.push({role:a.role,content:a.content});continue}const e=[];for(const n of a.content)e.push("text"==n.type?{type:"text",text:n.text}:t?{type:"image",data:await v(n.url)}:{type:"text",text:"[image]"});n.push({role:a.role,content:e})}return n},ue=(e,t)=>{if(e.role!=t.role)return!1;if("string"==typeof e.content||"string"==typeof t.content)return e.content==t.content;if(e.content.length!=t.content.length)return!1;for(let n=0;n<e.content.length;n++){const a=e.content[n],o=t.content[n];if(a.type!=o.type)return!1;if("text"==a.type&&"text"==o.type){if(a.text!=o.text)return!1}else if("image"==a.type&&"image"==o.type&&a.url!=o.url)return!1}return!0},me=async function*({engine:e,mod:t,params:n,messages:a,session:o}){const r=a[a.length-1],s=T(n.limits,n.reasoning);if("user"!=r?.role)throw new Error("No user turn");const i=a.slice(0,-1),c=(await ge([r]))[0];if(!c)throw new Error("Empty prompt");const l=!!o.conversation&&o.reasoning==n.reasoning&&o.maxTokens==s.max_tokens&&((e,t)=>{const n="system"==e[0]?.role?e[0]:void 0,a="system"==t[0]?.role?t[0]:void 0;if(!n!=!a)return!1;if(n&&a&&!ue(n,a))return!1;const o=n?e.slice(1):e,r=a?t.slice(1):t;if(!r.length||r.length>o.length)return!1;const s=o.length-r.length;for(let e=0;e<r.length;e++)if(!ue(o[s+e],r[e]))return!1;return!0})(o.sent,i);let d;if(l&&o.conversation)d=o.conversation;else{const a=await ge(i,!1);o.conversation=void 0,o.sent=[],d=await e.createConversation({sessionConfig:{maxOutputTokens:s.max_tokens,samplerParams:{type:t.SamplerType.TOP_K,temperature:s.temperature,k:40,p:.95}},preface:{...a.length?{messages:a}:{},extra_context:{enable_thinking:n.reasoning}},prefillPrefaceOnInit:a.length>0}),o.conversation=d,o.reasoning=n.reasoning,o.maxTokens=s.max_tokens,o.sent=[...i]}o.sent.push(r);let g,u=!1,m="";const f=()=>{g&&g.cancel().catch(()=>{});try{d.cancel()}catch{}};n.signal?.addEventListener("abort",f,{once:!0});try{if(n.signal?.aborted)return;const e=d.sendMessageStreaming(c).getReader();for(g=e;;){const{done:t,value:n}=await e.read();if(t)break;const a={};let o="string"==typeof n.content?n.content:"";if(Array.isArray(n.content))for(const e of n.content)"text"==e.type&&(o+=e.text);o&&(a.content=o,m+=o);const r=n.channels?.thought;r&&(a.reasoning=r),(a.content||a.reasoning)&&(yield{choices:[{delta:a}]})}u=!0}finally{n.signal?.removeEventListener("abort",f),u&&m?o.sent.push({role:"assistant",content:m}):(o.conversation=void 0,o.sent=[])}},fe={backend:l.gpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o})=>{re=!0===o;const s=e.litert;if(!s)throw new Error("Model has no LiteRT artifact");const i=await r.e(520).then(()=>r(507)),c=globalThis,g=c.Module;c.Module={print:se,printErr:ie};try{await i.getOrLoadGlobalLiteRtLm(y().script)}finally{c.Module=g}if(a?.aborted)throw new Error("Cancelled");const u=E(s.repo,s.file),m=await ce();let f;const p=m?await m.match(u):void 0;if(p?.body)f=p.body;else{const e=await ae.stored(u);e?.complete&&(f=e.file.stream())}if(!f){const e=await ae.download({url:u,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!ee(e))throw e;o&&console.log("Local model store failed",e)});e&&(f=e.file.stream())}if(!f&&m)try{await(async(e,t,n,a)=>{const o=le.get(e);if(o)return await o;const r=(async()=>{const o=await fetch(e,{signal:n});if(!o.ok||!o.body)throw new Error("Model download failed");const r=C(o),s=de(o.body,r,a);await t.put(e,new Response(s,{headers:{"content-type":"application/octet-stream",...r?{"content-length":`${r}`}:{}}}))})().finally(()=>{le.delete(e)});return le.set(e,r),await r})(u,m,a,n);const e=await m.match(u);e?.body&&(f=e.body)}catch(e){if(a?.aborted)throw new Error("Cancelled");o&&console.log("Local model cache failed",e)}if(f||(f=await(async(e,t,n)=>{const a=await fetch(e,{signal:t});if(!a.ok||!a.body)throw new Error("Model download failed");return de(a.body,C(a),n)})(u,a,n)),a?.aborted)throw new Error("Cancelled");n(1,d.load);const h=await i.Engine.create((({model:e,limits:t,source:n,mod:a})=>({model:n,mainExecutorSettings:{maxNumTokens:4096,...e.litert?.vision?{maxNumImages:Math.max(1,t.maxImages)}:{},backendConfig:oe},...e.litert?.vision?{visionBackend:a.Backend.GPU,maxVisionTokensPerImage:70,minVisionTokensPerImage:70}:{}}))({model:e,limits:t,source:f,mod:i}));if(a?.aborted)throw await h.delete(),new Error("Cancelled");const w={sent:[],reasoning:!1,maxTokens:0};return{model:e,backend:l.gpu,contextSize:4096,chat:t=>(async({engine:e,model:t,mod:n,params:a,session:o})=>{re=!0===a.logging;const r=()=>{try{o.conversation?.cancel()}catch{}o.conversation=void 0,o.sent=[]};try{return await I({model:t,params:a,stopped:()=>!!a.signal?.aborted,onStall:r,start:async r=>{if(a.signal?.aborted)throw new Error("Cancelled");if(L(r)&&!t.litert?.vision)throw new Error("Model does not accept images");return me({engine:e,mod:n,params:a,messages:r,session:o})}})}catch(e){throw r(),e}})({engine:h,model:e,mod:i,params:t,session:w}),unload:async()=>{w.conversation=void 0,w.sent=[],await h.delete()}}},cached:async e=>{const t=e.litert;if(!t)return!1;const n=E(t.repo,t.file),[a,o]=await Promise.all([ce(),ae.stored(n)]);if(o?.complete)return!0;try{return!(!a||!await a.match(n))}catch{return!1}},hasCache:async()=>{try{for(const e of u){const t=e.litert;if(!t)continue;if(await ae.stored(E(t.repo,t.file)))return!0}const e=await ce();return!!e&&(await e.keys()).length>0}catch{return!1}},remove:async e=>{try{const t=e.litert;if(!t)return;const n=E(t.repo,t.file),a=await ce();await ae.remove(n),a&&await a.delete(n)}catch{}},purge:async()=>{try{await ae.purge(),"undefined"!=typeof caches&&await caches.delete(ne)}catch{}}},pe=te("wllama-store");let he,we,ye="",be=!1;const ve=e=>`${e.wasm}|${e.compatWasm}|${e.compatWorker}`,xe=e=>{const t=w();return{wasm:e?.wasm||t.wasm,compatWasm:e?.compatWasm||t.compatWasm,compatWorker:e?.compatWorker||t.compatWorker}},ke=e=>{const t=e=>(...t)=>{be&&("string"==typeof t[0]&&t[0].includes("ggml_webgpu: Failed to get an adapter")||e(...t))};return{...e,log:t(e.log),warn:t(e.warn),error:t(e.error)}},_e=async()=>{if(he)return he;const e=await r.e(924).then(()=>r(577)),t=we||xe(),n=new e.Wllama({default:t.wasm},{logger:ke(e.LoggerWithoutDebug)});return we||(we=t,ye=ve(t)),n.setCompat({wasm:t.compatWasm,worker:t.compatWorker}),he=n,n},Me=async()=>{const e=he;he=void 0;try{await(e?.exit())}catch{}},Te=(e,t)=>e.name.includes(t)||!!e.metadata?.originalURL?.includes(t),Ee=e=>{const t=e.cpu;return t?[{name:t.file,url:E(t.repo,t.file)},...t.mmproj?[{name:t.mmproj.file,url:E(t.repo,t.mmproj.file)}]:[]]:[]},Ce=async e=>{const t=Ee(e);if(!t.length)return!1;for(const e of t){const t=await pe.stored(e.url);if(!t?.complete)return!1}return!0},Le=async(e,t)=>{const n=await e.cacheManager.list();return Ee(t).every(e=>n.some(t=>Te(t,e.name)))},Pe=async({model:e,signal:t,onProgress:n})=>{const a=Ee(e),o=[];for(let e=0;e<a.length;e++){if(t?.aborted)throw new Error("Cancelled");const r=await pe.download({url:a[e].url,signal:t,onProgress:(t,o)=>n((e+t)/a.length,o)});if(!r)return;o.push(r.file)}return o.length?o:void 0},je=async e=>{const t=[];for(const n of e){if("string"==typeof n.content){"system"==n.role?t.push({role:"system",content:n.content}):"assistant"==n.role?t.push({role:"assistant",content:n.content}):t.push({role:"user",content:n.content});continue}const e=[];for(const t of n.content)e.push("text"==t.type?{type:"text",text:t.text}:{type:"image",data:await v(t.url)});t.push({role:"user",content:e})}return t},Se={backend:l.cpu,startLLM:async({model:e,limits:t,onProgress:n,signal:a,logging:o,assets:r})=>{be=!0===o;const s=e.cpu;if(!s)throw new Error("Model has no CPU artifact");const i=xe(r),c=ve(i);he&&c!=ye&&await Me(),we=i,ye=c;const g=(u="undefined"!=typeof navigator&&navigator.hardwareConcurrency||4,m=t.maxThreads,Math.max(1,Math.min(Math.max(2,Math.min(8,(u||4)-1)),m)));var u,m;const f=await _e(),p=await Ce(e),h=s.mmproj?.file,w=E(s.repo,s.file),y=h?E(s.repo,h):"";if(a?.aborted)throw await Me(),new Error("Cancelled");if(!p&&h){const e=await f.cacheManager.list(),t=e.some(e=>Te(e,s.file)),n=e.some(e=>Te(e,h));t&&!n&&await f.cacheManager.deleteMany(e=>Te(e,s.file))}const b={n_ctx:8192,n_threads:g,reasoning_budget_tokens:t.maxThinkingTokens,n_gpu_layers:0,jinja:!0};try{let t=p?await Pe({model:e,signal:a,onProgress:n}):void 0;t||p||await Le(f,e).catch(()=>!1)||(t=await Pe({model:e,signal:a,onProgress:n}).catch(e=>{if(a?.aborted)throw new Error("Cancelled");if(!ee(e))throw e;o&&console.log("Local model store failed",e)})),t?(n(1,d.load),await f.loadModel(t,b)):await f.loadModelFromUrl(y?{url:w,mmprojUrl:y}:w,{...b,useCache:!0,signal:a,progressCallback:({loaded:e,total:t})=>{const a=t?e/t:0;n(a,a>=1?d.load:d.download)}})}catch(e){throw await Me(),e}if(a?.aborted)throw await Me(),new Error("Cancelled");if(!f.getLoadedContextInfo().n_ctx)throw await Me(),new Error("Failed to create model context");return{model:e,backend:l.cpu,contextSize:8192,chat:t=>(async(e,t,n)=>{if(be=!0===n.logging,e.isModelLoaded())return await I({model:t,params:n,stopped:()=>!!n.signal?.aborted,start:async a=>{if(L(a)&&!e.supportInputModality("image"))throw new Error("Model does not accept images");return await e.createChatCompletion({messages:await je(a),...T(n.limits,n.reasoning),abortSignal:n.signal,...t.reasoning?{chat_template_kwargs:{enable_thinking:n.reasoning}}:{}})}})})(f,e,t),unload:async()=>{if(he==f)await Me();else try{await f.exit()}catch{}}}},cached:async e=>{try{return!!e.cpu&&(!!await Ce(e)||await Le(await _e(),e))}catch{return!1}},hasCache:async()=>{try{for(const e of u.flatMap(e=>Ee(e)))if(await pe.stored(e.url))return!0;const e=await _e();return(await e.cacheManager.list()).length>0}catch{return!1}},remove:async e=>{const t=Ee(e);try{for(const e of t)await pe.remove(e.url)}catch{}try{const e=await _e();await e.cacheManager.deleteMany(e=>t.some(t=>Te(e,t.name)))}catch{}},purge:async()=>{try{await pe.purge()}catch{}try{const e=await _e();await e.cacheManager.clear()}catch{}}},$e=((e={})=>{const t=e.engines?.cpu||Se,n=e.engines?.litert||fe;let a,o,r,s,i,g,m,f=l.none,p=c.idle,h=[],w=0,y={percent:0,phase:d.download};const b=a=>{if(e.backend!=l.none)return e.backend==l.cpu?a.cpu?t:void 0:e.backend==l.gpu?a.litert?n:void 0:a.litert&&"undefined"!=typeof navigator&&navigator.gpu&&"undefined"!=typeof WebAssembly?n:!a.cpu||"undefined"==typeof window&&"undefined"==typeof self&&"undefined"==typeof Worker||"undefined"==typeof WebAssembly?void 0:t},v=(e,t)=>{y={percent:Math.max(0,Math.min(1,e)),phase:t||d.download};for(const e of[...h])e.report(y.percent,y.phase)},x=e=>{p=e},k=()=>{w++,g?.abort(),g=void 0,i=void 0,p==c.downloading&&x(c.idle)},_=(e,t)=>{const n=s;return n?new Promise(a=>{let o=!1;const r={report:e,settle:e=>{o||(o=!0,t?.removeEventListener("abort",i),a(e))}},i=()=>{(e=>{h=h.filter(t=>t!=e),queueMicrotask(()=>{!h.length&&s&&k()})})(r),r.settle(!1)};h.push(r),t?.aborted?i():t?.addEventListener("abort",i,{once:!0}),n.then(r.settle,()=>r.settle(!1))}):Promise.resolve(!1)},M=()=>p==c.ready&&!!a&&!!o;return{load:async({model:e,limits:t,assets:n,logging:l,signal:m,onProgress:p})=>{if(m?.aborted)return!1;if(M()&&o?.id==e.id)return!0;if(s&&i==e.id)return p(y.percent,y.phase),await _(p,m);s&&(k(),h=[],await s.catch(()=>!1));const T=++w,E=(async()=>{await(async()=>{try{if("undefined"==typeof navigator)return;await(navigator.storage?.persist?.())}catch{}})();const s=b(e);if(T!=w)return!1;if(!s)return x(c.unsupported),!1;if(await(async(e,t)=>{if(o&&o.id!=e.id){const e=o,t=r,n=a;a=void 0,o=void 0,r=void 0,await(n?.unload()),await(t?.remove(e))}if(await t.hasCache())for(const n of u)n.id!=e.id&&b(n)==t&&await t.remove(n)})(e,s),T!=w)return!1;const i=await s.cached(e);if(T!=w)return!1;x(c.downloading),v(0,d.download),i&&v(1,d.load);const m=new AbortController;g=m;try{const i=await s.startLLM({model:e,limits:t,logging:l,signal:m.signal,assets:n,onProgress:(e,t)=>{T==w&&v(e,t)}});return T!=w||m.signal.aborted?(await i.unload(),!1):(a=i,o=e,r=s,f=s.backend,x(c.ready),v(1,d.load),!0)}catch(e){return T!=w||(a=void 0,o=void 0,r=void 0,l&&console.log("Local model failed",e),x(c.error)),!1}finally{g==m&&(g=void 0)}})().finally(()=>{s==E&&(s=void 0,i=void 0,h=[])});return s=E,i=e.id,await _(p,m)},chat:async e=>{if(!M()||!a)return;for(;m;)await m.catch(()=>{});if(!M()||!a)return;const t=a.chat(e);m=t;try{return await t}finally{m==t&&(m=void 0)}},unload:async()=>{const e=a;k(),a=void 0,o=void 0,r=void 0,x(c.idle),await(e?.unload())},ready:M,loading:()=>!M()&&p==c.downloading,status:()=>p,backend:()=>f,model:()=>o}})(),Re=(e,t,n)=>{const a=e.status(),o=m(t);return{availability:a==c.ready?i.online:a==c.error||a==c.unsupported?i.offline:i.away,status:a,backend:e.backend(),supportsImages:!!o&&h(o),ready:e.ready(),loading:e.loading(),requesting:n.requesting,phase:n.phase,percent:n.percent,...n.error?{error:n.error}:{}}},We=(e=>{let t,n=0;const a=t=>t.input.onStatus?.(Re(e,t.model??t.input.model,t));return o=>{t?.controller.abort();const r={token:++n,input:o,controller:new AbortController,percent:e.ready()?100:0,phase:e.ready()?d.load:d.download,requesting:!1,cancelled:!1};return t=r,a(r),(async t=>{const{input:o}=t,r=!0===o.logging,i=await p(o.model);if(t.model=i?.id,t.token!=n)return;const c=o.messages.some(e=>e.role!=s.system&&"string"==typeof e.content&&!!e.content.trim()),l=o.messages.some(e=>e.role==s.user&&!!e.images?.some(e=>"string"==typeof e&&!!e)),g=c||l;if(!i)return t.error="Unknown model",void a(t);if(!g)return void a(t);if(l&&!c&&!h(i))return t.error="Model does not accept images",void a(t);const u=M(o.limits,!!i.litert),m=await e.load({model:i,limits:u,logging:r,signal:t.controller.signal,onProgress:(e,o)=>{t.token==n&&(t.percent=Math.round(100*e),t.phase=o,a(t))}}).catch(e=>(t.token==n&&(t.error=`${e?.message||`${e}`}`),!1));if(t.token!=n)return;if(t.cancelled)return;if(!m)return t.error||(t.error="Model not ready"),void a(t);const f=h(i),w=l&&!f?o.messages.map(({images:e,...t})=>t):o.messages;if(l&&!f&&!c)return t.error="Model does not accept images",void a(t);t.requesting=!0,t.error=void 0,t.percent=100,t.phase=d.load,a(t);let y,b="",v="";try{y=await e.chat({messages:w,limits:u,reasoning:!0===o.reasoning,logging:r,onReason:e=>{t.token!=n||t.cancelled||(b+=e,o.onThinking?.(b))},onText:e=>{t.token!=n||t.cancelled||(v+=e,o.onResponse?.(v))},onSpeed:e=>{t.token!=n||t.cancelled||o.onSpeed?.(e)},signal:t.controller.signal})}catch(e){if(t.token!=n)return;return t.requesting=!1,t.error=`${e?.message||`${e}`}`,void a(t)}t.token==n&&(t.requesting=!1,y||t.cancelled||(t.error="No reply"),a(t))})(r),()=>(t=>{t.token==n&&(t.cancelled=!0,t.controller.abort(),e.status()!=c.ready?queueMicrotask(()=>a(t)):e.unload().then(()=>a(t)))})(r)}})($e),qe=(ze=$e,async(e={})=>{const t=await p(e.model),n=n=>e.onStatus?.(Re(ze,t?.id,n));if(!t)return n({requesting:!1,phase:d.download,percent:0,error:"Unknown model"}),!1;let a,o=0,r=d.download;const s=ze.ready()&&ze.model()?.id==t.id;return await ze.load({model:t,limits:M(e.limits,!!t.litert),logging:!0===e.logging,onProgress:(e,t)=>{o=Math.round(100*e),r=t,n({requesting:!1,phase:r,percent:o})}}).catch(e=>(a=`${e?.message||`${e}`}`,!1))?(s&&n({requesting:!1,phase:d.load,percent:100}),!0):(n({requesting:!1,phase:r,percent:o,error:a||"Model not ready"}),!1)});var ze;export{g as ChatModelId,i as ChatStatus,l as LlmBackend,d as LlmPhase,s as LlmRole,c as LlmStatus,We as createChat,qe as preloadModel};

package/dist/types.d.ts

@@ -80,6 +80,8 @@ interface ChatInput {
     onThinking?: (text: string) => void;
     /** Accumulated reply text, streamed while the model answers */
     onResponse: (text: string) => void;
+    /** Current generation speed in tokens per second, streamed while the model answers */
+    onSpeed?: (tokensPerSecond: number) => void;
     /** Model and request state, reported on every change */
     onStatus?: (snapshot: ChatStatusSnapshot) => void;
 }
@@ -221,6 +223,8 @@ interface RuntimeChatParams {
     logging?: boolean;
     onReason: (text: string) => void;
     onText: (text: string) => void;
+    /** Current generation speed in tokens per second */
+    onSpeed?: (tokensPerSecond: number) => void;
     signal?: AbortSignal;
 }
 /** On-device runtime shared by every request */
@@ -248,6 +252,8 @@ interface EngineChatParams {
     onReason: (text: string) => void;
     /** Streamed reply delta */
     onText: (text: string) => void;
+    /** Current generation speed in tokens per second */
+    onSpeed?: (tokensPerSecond: number) => void;
     signal?: AbortSignal;
 }
 /** Params of the engine startLLM */
@@ -337,6 +343,19 @@ type LitertMessage = import('@litert-lm/core').Message;
 type LitertEngine = import('@litert-lm/core').Engine;
 /** The LiteRT-LM module namespace */
 type LitertModule = typeof import('@litert-lm/core');
+/** Settings handed to the LiteRT-LM engine */
+type LitertSettings = import('@litert-lm/core').EngineSettings;
+/** Params of one LiteRT-LM settings build */
+interface LitertSettingsParams {
+    /** Model with the artifact */
+    model: ChatModel;
+    /** Image and context budgets */
+    limits: LlmLimits;
+    /** Model file stream */
+    source: ReadableStream<Uint8Array>;
+    /** Loaded LiteRT module */
+    mod: LitertModule;
+}
 /** One reused LiteRT-LM conversation and the turns it already holds */
 interface LitertSession {
     /** Live conversation, dropped when a turn cannot be continued */
@@ -384,6 +403,8 @@ interface StreamReplyParams {
     onReason: (text: string) => void;
     /** Streamed reply delta */
     onText: (text: string) => void;
+    /** Current generation speed in tokens per second */
+    onSpeed?: (tokensPerSecond: number) => void;
     /** Open the engine stream for the prepared messages */
     start: () => Promise<AsyncIterable<unknown>>;
     /** Whether the caller stopped the generation */
@@ -408,4 +429,4 @@ interface EngineRunParams {
     /** Called when a slow stream keeps the text already delivered */
     onStall?: () => void;
 }
-export type { ChatMessage, ChatModel, LlmLimits, ChatStatusSnapshot, ChatInput, ChatPreloadInput, ChatState, CancelChat, WllamaAssets, LitertAssets, GpuNavigator, ActiveCall, RuntimeDeps, RuntimeLoadParams, RuntimeLoadWaiter, RuntimeChatParams, StoredPart, StoredFilePart, StoredModel, StoredModelMeta, ResumePlan, StoreDownloadParams, ModelStore, Runtime, EngineChatParams, StartLlmParams, LlmRuntime, LlmEngine, EngineMessage, EngineContentPart, WllamaContentPart, WllamaMessage, LitertContentPart, LitertMessage, LitertEngine, LitertModule, LitertSession, LitertChatParams, StreamLitertParams, EngineDelta, EngineChunk, StreamReplyParams, EngineRunParams, };
+export type { ChatMessage, ChatModel, LlmLimits, ChatStatusSnapshot, ChatInput, ChatPreloadInput, ChatState, CancelChat, WllamaAssets, LitertAssets, GpuNavigator, ActiveCall, RuntimeDeps, RuntimeLoadParams, RuntimeLoadWaiter, RuntimeChatParams, StoredPart, StoredFilePart, StoredModel, StoredModelMeta, ResumePlan, StoreDownloadParams, ModelStore, Runtime, EngineChatParams, StartLlmParams, LlmRuntime, LlmEngine, EngineMessage, EngineContentPart, WllamaContentPart, WllamaMessage, LitertContentPart, LitertMessage, LitertEngine, LitertModule, LitertSettings, LitertSettingsParams, LitertSession, LitertChatParams, StreamLitertParams, EngineDelta, EngineChunk, StreamReplyParams, EngineRunParams, };

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@degreesign/chat",
-    "version": "0.3.6",
+    "version": "0.3.7",
     "description": "Headless on-device AI chat for browsers and Node with wllama CPU and LiteRT WebGPU engines",
     "main": "dist/node/chat.node.min.js",
     "module": "dist/browser/chat.min.mjs",