@qvac/llm-llamacpp 0.65.1 → 0.66.0

package/README.md

@@ -265,7 +265,7 @@ projector runs on CPU. The resolved choice is logged at verbosity ≥ 2 as
 
 `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option unless an external head is given, see below). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
 
-For a model without MTP layers, give an external MTP head as `spec-draft-model` (llama-server's `--spec-type draft-mtp -md <head>`): a GGUF with MTP layers for the same model, such as the MTP quantization of it, or a Gemma 4 assistant for its Gemma 4 model. The head is loaded next to the model and placed like it (`device`, `gpu-layers`), and the load's fit and `assessFit` count its weights as well as its context. A head without MTP layers fails the load, and `assessFit` refuses it with `unsupported-config`. A head for another model, with a different hidden width or vocabulary, also fails the load, and `assessFit` refuses it the same way. A Gemma 4 assistant reads the model's own tensors, so fabric measures it next to the model, as the load creates it.
+For a model without MTP layers, give an external MTP head as `spec-draft-model` (llama-server's `--spec-type draft-mtp -md <head>`): a GGUF with MTP layers for the same model, such as the MTP quantization of it, or a Gemma 4 assistant for its Gemma 4 model. The head is loaded next to the model and placed like it (`device`, `gpu-layers`), and the load's fit and `assessFit` count its weights as well as its context. A head without MTP layers fails the load, and `assessFit` rejects it. A head for another model, with a different hidden width or vocabulary, also fails the load, and `assessFit` rejects it the same way. A Gemma 4 assistant reads the model's own tensors, so fabric measures it next to the model, as the load creates it.
 
 `spec-type: 'draft-dflash'` drafts with a separate DFlash model instead, given as `spec-draft-model` (an absolute path; llama-server's `-md`). The DFlash model reads the hidden states of the target layers it was trained on and denoises a whole block of draft tokens in one pass, so a step drafts up to its block size minus one tokens (`spec-draft-n-max` caps it further). It is loaded next to the model and placed like it (`device`, `gpu-layers`). The load's fit and `assessFit` count its weights and context, measured next to the model when the drafter borrows the model's embeddings and output head. A draft model fabric still cannot measure is left out of both, and fabric logs `failed to measure the memory of the extra model, fitting without it`. Everything else (verification, rollback, batching, caches, stats) is the same as for MTP.
 
@@ -384,6 +384,30 @@ const response = await model.run([
 ])
 ```
 
+#### Counting tokens
+
+`calcTokens` returns how many tokens an input takes on the loaded model, without running it:
+
+```javascript
+const raw = await model.calcTokens({ text: 'Hello, world!' })
+
+const prompt = await model.calcTokens({
+  history: [{ role: 'system', content: 'You are a helpful assistant.' }],
+  tools: [{ type: 'function', name: 'get_weather', parameters: { type: 'object', properties: { city: { type: 'string' } } } }],
+  text: 'Weather in Paris?'
+})
+```
+
+- `text` alone is counted raw, with no chat template and no BOS.
+- `history` takes the `Message[]` that `run()` takes, and `tools` takes tool definitions. Either one is rendered by the chat template as the opening request of a fresh conversation, with `text` (even empty) appended as the last user turn. The count includes BOS, and the generation prompt when the last turn is a user or tool turn, so it equals the `promptTokens` a fresh `run()` of the same conversation reports. Tools the template drops are not counted. On an encoder-decoder model `run()` encodes the prompt separately, so its `promptTokens` differs from this count.
+- `tools` needs `text` or `history`, as `run()` can't render tools without a message.
+- It never queues behind `run()`, so it answers while requests are generating.
+- Errors:
+  - A wrong-typed argument rejects with a `TypeError`.
+  - Before `load()` or after `unload()` it rejects with `Model is not loaded`.
+  - Other invalid input, media included, rejects with an error whose `code` is `InvalidArgument` and whose message starts with `calcTokens:`.
+  - A conversation the chat template cannot render at all fails as it would in `run()`.
+
 ### 7. Release Resources
 
 Unload the model when finished:
@@ -398,7 +422,7 @@ try {
 
 ### API behavior by state
 
-The following table describes the expected behavior of `run` and `cancel` depending on the current state (idle vs jobs running). The two cancel entry points have different scopes:
+The following table describes the expected behavior of `run`, `cancel` and `calcTokens` depending on the current state (idle vs jobs running). The two cancel entry points have different scopes:
 
 - `response.cancel()` — **targeted**: cancels only the job (or batch group) that `run()` call produced, leaving other concurrent jobs running.
 - `model.cancel()` — **global**: cancels every live job (in-flight and queued) at the moment of the call, plus any finetuning in progress.
@@ -411,6 +435,7 @@ The following table describes the expected behavior of `run` and `cancel` depend
 | busy, `parallel >= 2` (default `rejectWhenBusy: false`) | run | **Allowed** — the job is admitted concurrently (continuous batching) or queued until a slot frees; each call gets its own independent `QvacResponse` |
 | busy          | `response.cancel()` | **Allowed** — cancels only that response's job/group; Promise resolves when it has stopped |
 | busy          | `model.cancel()`    | **Allowed** — cancels all live jobs; Promise resolves when they have stopped |
+| idle or busy  | `calcTokens` | **Allowed** — counts without running and never queues behind a job; see [Counting tokens](#counting-tokens) |
 
 Admission is controlled by `rejectWhenBusy` (instance-level `opts.rejectWhenBusy`, overridable per call via `runOptions.rejectWhenBusy`). Its default follows `parallel`: `true` for `parallel: 1` (busy runs fail fast, preserving the historical single-job contract) and `false` for `parallel >= 2` (busy runs queue behind the pool and start as slots free). With `parallel >= 2`, separate top-level `run()` calls are batched together into the same decode loop — see [Continuous Batching](./docs/continuous-batching.md).
 
@@ -444,26 +469,33 @@ const fit = await LlmLlamacpp.assessFit({
   config: { device: 'gpu', 'ctx-size': '4096' }
 })
 
-fit.status // 'fits' | 'does-not-fit' | 'error'
-fit.reason // 'fits', 'does-not-fit', 'model-unreadable', 'no-backend-device' or 'unsupported-config'
-fit.gpuLayers // what fits, which is not always what the request asked for
-fit.ctxSize
-fit.devices // one row per device the model was assigned to, then a `host` row
-fit.deviceBytes // model, context and compute summed across the devices, host excluded
-fit.hostBytes // the same, for the trailing host row
+fit.status // 'fits' | 'does-not-fit' | 'unknown'
+fit.devices // one row per device the model is assigned to, then the host CPU
+fit.devices[0] // { id, name, unified, usedBytes, freeBytes, totalBytes }
+fit.stats // { ctxSize, gpuLayers }
 ```
 
-`config` is the load, exactly as `loadModel` takes it. The projection resolves it with the same code that load runs, so every setting reaches the fitter the way it would reach the engine, and a load the engine would refuse is refused here as `status: "error"` with `unsupported-config`. A setting left out takes llama's default.
+Each device row gives the memory the load takes on that device, as `usedBytes`, and the device's free and total memory. `id` is the PCI bus ID when the backend reports one, and the device name otherwise. `unified` is true for memory the host owns: the CPU, an integrated GPU, a GPU that shares the host's memory, Android's OpenCL GPU, and a device that reports no memory of its own, which takes the host's figures. With `load_mode: 'mmap'`, the host row leaves out the mapped weights, which the OS can evict.
+
+On an iOS device every unified row shares the app's own memory limit, `os_proc_available_memory()`. Each such row's free memory is capped at that limit, and a load whose unified rows together take more than the limit less the margin does not fit. A limit of 0 means the app has none left.
+
+`stats` is the load as the fit resolved it. On `fits`, `ctxSize` and `gpuLayers` are the context and GPU layer count the fitter settled on, which can be smaller than the load asked for. On `does-not-fit` they are the ones the load asked for.
+
+`status` is `unknown` for a load the fitter cannot project: one placed on the CPU, one with a LoRA adapter or a multimodal projector, one with `fit` turned off, or one the fitter cannot break down per device, the host CPU included. `devices` is then empty and `stats` is `{}`.
+
+The projection reads the model's vocabulary, and llama asserts on a vocabulary it finds inconsistent. An assert aborts the process instead of rejecting, so take the model from a source you trust, or call `assessFit` behind a process boundary.
+
+`config` is the load, exactly as `loadModel` takes it. The projection resolves it with the same code that load runs, so every setting reaches the fitter the way it would reach the engine, and a load the engine would refuse rejects here. A setting left out takes llama's default.
 
-With `spec-type: 'draft-mtp'` the MTP draft context is fitted next to the model, as the load fits it, and its memory is in the verdict and in the per-device figures; on a model or head without MTP layers the load fails, so the projection is refused with `unsupported-config`. A DFlash draft model or a Gemma 4 assistant is fitted the same way, measured next to the model since it reads the model's tensors, and is in the verdict and the figures too (see [Speculative decoding](#speculative-decoding-mtp-dflash)).
+With `spec-type: 'draft-mtp'` the MTP draft context is fitted next to the model, as the load fits it, and its memory is in the verdict and in the device rows; on a model or head without MTP layers the load fails, so the projection rejects. A DFlash draft model or a Gemma 4 assistant is fitted the same way, measured next to the model since it reads the model's tensors, and is in the verdict and the rows too (see [Speculative decoding](#speculative-decoding-mtp-dflash)).
 
-Pinning `gpu-layers` fixes the placement: the fitter refuses to move layers off a device the load claimed, so the answer is whether that exact placement fits, never a reduced one. Leave it unset for a projection that can place the model itself.
+Pinning `gpu-layers` fixes the placement: the fitter keeps layers on the devices the load claimed, so the answer is for that exact placement. Leave it unset for a projection that can place the model itself.
 
-`minCtxSize` sets a floor the fitter may not reduce the context below, defaulting to llama's own `fit-ctx` of 4096. `marginBytes` is the memory to leave free on every device, defaulting to llama's `fit-target` of 1 GiB per device. `backendsDir` is where the dynamically-loaded ggml backends live.
+`minCtxSize` sets a floor the fitter may not reduce the context below, defaulting to llama's own `fit-ctx` of 4096. `marginBytes` is the memory to leave free on every device. The load's own `fit-target`, 1 GiB per device by default, applies too, and the larger of the two binds. `backendsDir` is where the dynamically-loaded ggml backends live.
 
 `assessFit` returns a Promise. Concurrent calls run one at a time within this addon.
 
-A model the fitter cannot read resolves to `status: "error"`; only a broken request rejects.
+A model the fitter cannot read, an invalid request, or a machine with no backend device rejects with an error whose message starts with `assessFit:` and names the cause.
 
 ## Fine-tuning
 

package/addon.d.ts

@@ -65,6 +65,12 @@ export interface AddonConfigurationParams {
     config: Record<string, unknown>;
 }
 export type AddonOutputCallback = (addon: unknown, event: unknown, data: unknown, error: unknown, jobId: unknown) => void;
+/** `calcTokens` input as the native binding takes it: `history` and `tools` are JSON array text. */
+export interface AddonCalcTokensParams {
+    text?: string;
+    history?: string;
+    tools?: string;
+}
 export interface LlamaBinding {
     createInstance(owner: LlamaInterface, configurationParams: AddonConfigurationParams, outputCallback: AddonOutputCallback, reserved: null): object;
     loadWeights(handle: unknown, data: LoadWeightsData): Promise<void> | void;
@@ -76,6 +82,7 @@ export interface LlamaBinding {
     finetune?(handle: unknown, params: FinetuneOptions): Promise<number | false> | number | false;
     saveCache(handle: unknown, cacheKey: string): Promise<void>;
     discardCache(handle: unknown, cacheKey: string): Promise<void>;
+    calcTokens(handle: unknown, params: AddonCalcTokensParams): Promise<number>;
     runJob(handle: unknown, data: AddonRunJobMessage[]): Promise<AddonRunJobResult>;
     runJob(handle: unknown, data: AddonBatchRunItem[]): Promise<AddonBatchRunResult>;
     destroyInstance(handle: unknown): void;
@@ -138,6 +145,10 @@ export declare class LlamaInterface {
      * (see `LlmLlamacpp.discardCache`).
      */
     discardCache(cacheKey: string): Promise<void>;
+    /**
+     * Tokens `params` takes on the loaded model (see `LlmLlamacpp.calcTokens`).
+     */
+    calcTokens(params: AddonCalcTokensParams): Promise<number>;
     /**
      * Run finetuning when native binding provides support.
      */

package/addon.js

@@ -154,6 +154,14 @@ class LlamaInterface {
             return Promise.reject(new Error("Model is not loaded"));
         return Promise.resolve(this._binding.discardCache(this._handle, cacheKey));
     }
+    /**
+     * Tokens `params` takes on the loaded model (see `LlmLlamacpp.calcTokens`).
+     */
+    calcTokens(params) {
+        if (!this._handle)
+            return Promise.reject(new Error("Model is not loaded"));
+        return Promise.resolve(this._binding.calcTokens(this._handle, params));
+    }
     /**
      * Run finetuning when native binding provides support.
      */

package/fit.d.ts

@@ -5,41 +5,42 @@ export interface LlamaFitRequest {
     config?: Record<string, string>;
     /** Floor the fitter may not reduce an unpinned context below. */
     minCtxSize?: number;
-    /** Memory to leave free on every device, held to against the load's own `fit-target`. */
+    /** Memory to leave free on every device. The larger of this and the load's own `fit-target` binds. */
     marginBytes?: number;
 }
-export type LlamaFitStatus = 'fits' | 'does-not-fit' | 'error';
-export type LlamaFitReason = 'fits' | 'does-not-fit' | 'model-unreadable' | 'no-backend-device' | 'unsupported-config';
+/** `unknown` is a load the fitter cannot project, such as one placed on the CPU. */
+export type LlamaFitStatus = 'fits' | 'does-not-fit' | 'unknown';
 export interface LlamaFitDevice {
+    /** The PCI bus ID when the backend reports one, the device name otherwise. */
+    id: string;
     name: string;
-    totalBytes: number;
+    unified: boolean;
+    /** Model, context and compute memory the load takes on this device. */
+    usedBytes: number;
     freeBytes: number;
-    modelBytes: number;
-    contextBytes: number;
-    computeBytes: number;
+    totalBytes: number;
 }
+/**
+ * The load as the fit resolved it: on `fits` the context and GPU layer count the
+ * fitter settled on, on `does-not-fit` the ones the load asked for. Empty when
+ * `unknown`.
+ */
+export type LlamaFitStats = {
+    ctxSize: number;
+    gpuLayers: number;
+} | Record<string, never>;
 export interface LlamaFitResult {
     status: LlamaFitStatus;
-    reason: LlamaFitReason;
-    /** The layer count the placement resolved to. */
-    gpuLayers: number;
-    ctxSize: number;
-    /** One row per device the model was assigned to, then a `host` row. */
+    /** One row per device the model is assigned to, then the host CPU. Empty when `unknown`. */
     devices: LlamaFitDevice[];
-    /** Model, context and compute summed across the devices, host excluded. */
-    deviceBytes: number;
-    /** The same, for the trailing host row. */
-    hostBytes: number;
-    trainCtxSize: number;
-    expertCount: number;
+    stats: LlamaFitStats;
 }
 /**
  * Projects one model against the memory free right now, reading GGUF metadata
  * and never weight data. The registry's weightless copy of a model answers the
  * same as the model itself, so this can run before anything is downloaded.
  *
- * A model the fitter cannot read resolves to `status: "error"`; only a broken
- * request rejects.
+ * A model the fitter cannot read, or an invalid request, rejects.
  *
  * Builds the model twice without allocating weights. Concurrent calls run one
  * at a time.

package/fit.js

@@ -6,8 +6,7 @@ exports.assessFit = assessFit;
  * and never weight data. The registry's weightless copy of a model answers the
  * same as the model itself, so this can run before anything is downloaded.
  *
- * A model the fitter cannot read resolves to `status: "error"`; only a broken
- * request rejects.
+ * A model the fitter cannot read, or an invalid request, rejects.
  *
  * Builds the model twice without allocating weights. Concurrent calls run one
  * at a time.

package/index.d.ts

@@ -1,6 +1,6 @@
 import QvacLogger = require("@qvac/logging");
 import { QvacResponse } from "@qvac/infer-base";
-import { assessFit as assessFitImpl, type LlamaFitDevice, type LlamaFitRequest, type LlamaFitResult, type LlamaFitStatus } from "./fit";
+import { assessFit as assessFitImpl, type LlamaFitDevice, type LlamaFitRequest, type LlamaFitResult, type LlamaFitStats, type LlamaFitStatus } from "./fit";
 import type * as AddonModule from "./addon";
 type BareEventMap = Record<string | symbol, unknown[]>;
 /** Aliases: inside the namespace, `QvacResponse` resolves to its own member. */
@@ -9,6 +9,7 @@ type InferQvacResponseOf<Output> = QvacResponse<Output>;
 /** Aliases so the class expression's body can name the namespace types below. */
 type BatchPrompt = LlmLlamacpp.BatchPrompt;
 type BatchResponse = LlmLlamacpp.BatchResponse;
+type CalcTokensInput = LlmLlamacpp.CalcTokensInput;
 type FinetuneHandle = LlmLlamacpp.FinetuneHandle;
 type FinetuneOptions = LlmLlamacpp.FinetuneOptions;
 type LlmLlamacppArgs = LlmLlamacpp.LlmLlamacppArgs;
@@ -70,6 +71,21 @@ interface LlmLlamacpp {
      * remove the conversation from disk.
      */
     discardCache(cacheKey: string): Promise<void>;
+    /**
+     * Tokens `input` takes on the loaded model, without running it. `text`
+     * alone is counted raw, with no chat template and no BOS. With `history` or
+     * `tools` the conversation, `text` (even empty) appended as the last user
+     * turn, is rendered by the chat template as the opening request of a fresh
+     * conversation, with BOS and the generation prompt, as `run()` renders it.
+     * Tools the template leaves out are not counted. `tools` alone has no
+     * message to render and rejects, as `run()` does. Never queues behind
+     * `run()`: it reads only load-time state. A wrong-typed argument rejects
+     * with `TypeError`, and before `load()` it rejects with `Model is not
+     * loaded`. Other invalid input, a media message included, rejects with code
+     * `InvalidArgument`; a conversation the chat template cannot render fails
+     * as it would in `run()`.
+     */
+    calcTokens(input: CalcTokensInput): Promise<number>;
     getState(): {
         configLoaded: boolean;
     };
@@ -467,6 +483,15 @@ declare namespace LlmLlamacpp {
         parameters?: Record<string, any>;
     }
     type Message = UserTextMessage | UserMediaMessage | ChatFunctionDefinition;
+    /** What `calcTokens` counts. At least one field is required; `tools` needs `text` or `history`. */
+    interface CalcTokensInput {
+        /** Counted raw on its own; with `history` or `tools`, the last user turn. */
+        text?: string;
+        /** A conversation in `run()`'s `Message[]` shape, tool definitions allowed. */
+        history?: Message[];
+        /** Tool definitions, as `run()` takes them. A name also in `history` rejects. */
+        tools?: ChatFunctionDefinition[];
+    }
     interface GenerationParams {
         temp?: number;
         top_p?: number;
@@ -763,5 +788,6 @@ declare namespace LlmLlamacpp {
     type FitResult = LlamaFitResult;
     type FitStatus = LlamaFitStatus;
     type FitDevice = LlamaFitDevice;
+    type FitStats = LlamaFitStats;
 }
 export = LlmLlamacpp;

package/index.js

@@ -826,6 +826,51 @@ const LlmLlamacpp = class LlmLlamacpp {
         }
         await this.addon.discardCache(cacheKey);
     }
+    /**
+     * Tokens `input` takes on the loaded model, without running it. `text`
+     * alone is counted raw, with no chat template and no BOS. With `history` or
+     * `tools` the conversation, `text` (even empty) appended as the last user
+     * turn, is rendered by the chat template as the opening request of a fresh
+     * conversation, with BOS and the generation prompt, as `run()` renders it.
+     * Tools the template leaves out are not counted. `tools` alone has no
+     * message to render and rejects, as `run()` does. Never queues behind
+     * `run()`: it reads only load-time state. A wrong-typed argument rejects
+     * with `TypeError`, and before `load()` it rejects with `Model is not
+     * loaded`. Other invalid input, a media message included, rejects with code
+     * `InvalidArgument`; a conversation the chat template cannot render fails
+     * as it would in `run()`.
+     */
+    async calcTokens(input) {
+        if (input === null || typeof input !== "object") {
+            throw new TypeError("calcTokens(input) requires an object");
+        }
+        const text = input.text ?? undefined;
+        const history = input.history ?? undefined;
+        const tools = input.tools ?? undefined;
+        if (text !== undefined && typeof text !== "string") {
+            throw new TypeError("calcTokens: text must be a string");
+        }
+        if (history !== undefined && !Array.isArray(history)) {
+            throw new TypeError("calcTokens: history must be a Message[]");
+        }
+        if (tools !== undefined && !Array.isArray(tools)) {
+            throw new TypeError("calcTokens: tools must be an array of function definitions");
+        }
+        if (!this.addon) {
+            throw new Error("Model is not loaded");
+        }
+        const params = {};
+        if (text !== undefined)
+            params.text = text;
+        // Media bytes are dropped rather than serialized: the native side rejects
+        // any media message by its type.
+        if (history !== undefined) {
+            params.history = JSON.stringify(history.map((message) => message?.type === "media" ? { ...message, content: "" } : message));
+        }
+        if (tools !== undefined)
+            params.tools = JSON.stringify(tools);
+        return this.addon.calcTokens(params);
+    }
     getState() {
         return this.state;
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/llm-llamacpp",
-  "version": "0.65.1",
+  "version": "0.66.0",
   "description": "llama addon for qvac",
   "addon": true,
   "scripts": {

package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.65.1"
+        "version": "0.66"
       }
     ],
     "exported_symbols": [
@@ -208,6 +208,7 @@
             "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence4bindEPKNS0_18SpeculativeRuntimeEi",
             "_bare_register_module_v0",
             "__ZN12CacheManager18restoreFromRamTierENSt3__18optionalIN30qvac_lib_inference_addon_llama8batching19SlotStateCacheEntryEEE",
+            "__ZNK14TextLlmContext15countChatTokensERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEERKNS1_I16common_chat_toolNS3_IS8_EEEE",
             "__ZThn48_N14TextLlmContext24captureHistoryCheckpointEi",
             "__ZN10LlamaModel22processPromptBatchImplERKNSt3__16vectorINS_6PromptENS0_9allocatorIS2_EEEERKNS0_8functionIFbmjyEEERKNS8_IFvmjEEEy",
             "__ZN14TextLlmContext26flushPendingUtf8ToCallbackERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
@@ -280,10 +281,9 @@
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler24recordPendingGroupCancelEy",
             "__ZN14MtmdLlmContext16evalMediaSegmentEmi",
             "__ZNK30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher14firstFreeSeqIdEv",
-            "__ZN30qvac_lib_inference_addon_llama5cache21readCheckpointSectionERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEyRKNS0_6LedgerERKNS0_16CheckpointPolicyE",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler17cancelGroupQueuedEy",
             "__ZThn48_N14TextLlmContext32snapshotPreRequestRollbackAnchorEv",
-            "__ZN30qvac_lib_inference_addon_llama5utils26generationPromptTailLengthEP13llama_contextRKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEERKNS3_6vectorIiNS7_IiEEEE",
+            "__ZN30qvac_lib_inference_addon_llama5cache21readCheckpointSectionERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEyRKNS0_6LedgerERKNS0_16CheckpointPolicyE",
             "__ZN30qvac_lib_inference_addon_llama5utils34isQwen3ReasoningFamilyArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
             "__ZNK14TextLlmContext10ctxCeilingEv",
             "__ZN10LlamaModel12formatPromptERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
@@ -297,7 +297,7 @@
             "__ZN14LlamaFinetuner18configureOptimizerERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsEP18llama_adapter_loraRN24llama_finetuning_helpers20LoraLrSchedulerStateEPNS6_23TrainingCheckpointStateEb",
             "__ZThn40_NK10LlamaModel15consumeJobStatsEy",
             "__ZN17backend_selection30getUnknownFinetuneArchitectureEPK13ModelMetaData",
-            "__ZThn48_N14MtmdLlmContext32snapshotPreRequestRollbackAnchorEv",
+            "__ZN30qvac_lib_inference_addon_llama5utils26generationPromptTailLengthEP13llama_contextRKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEERKNS3_6vectorIiNS7_IiEEEE",
             "__ZN14TextLlmContextC1ERK13common_paramsRK15LlmModelContextii",
             "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence5beginERKNSt3__16vectorIiNS2_9allocatorIiEEEE",
             "__ZNK30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher17nextAwaitingMediaEv",
@@ -321,9 +321,11 @@
             "__ZN14LlamaFinetuner15saveLoraAdapterEP18llama_adapter_loraRKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
             "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler7hasWorkEv",
             "__ZN14TextLlmContext22rollbackCurrentRequestERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
+            "__ZThn48_N14MtmdLlmContext32snapshotPreRequestRollbackAnchorEv",
             "__ZN14TextLlmContext27discardPendingResidentTokenEv",
             "__ZThn48_N14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
             "__ZN14TextLlmContext20onGenerationFinishedERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE20GenerationStopReason",
+            "__ZN30qvac_lib_inference_addon_llama5utils9getPromptEPK21common_chat_templatesR28common_chat_templates_inputsb",
             "__Z23validateByteBufferCountRKNSt3__16vectorI12PlannedMediaNS_9allocatorIS1_EEEEm",
             "__ZN14MtmdLlmContextC1ER13common_paramsNSt3__110unique_ptrI18common_init_resultNS2_14default_deleteIS4_EEEEyNS3_IN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeENS5_ISA_EEEE",
             "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel6createERK13common_params",
@@ -365,6 +367,7 @@
             "__ZN14MtmdLlmContext23cancelGenerationCleanupERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
             "__ZN26LlamaLazyInitializeBackend7acquireERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
             "__ZN14TextLlmContext14requireSamplerEv",
+            "__ZNK10LlamaModel10calcTokensERKNS_17CalcTokensRequestE",
             "__ZN19LlamaBackendsHandleC2ERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
             "__ZN24llama_finetuning_helpers23checkpointStepDirectoryERKNS_23TrainingCheckpointStateEx",
             "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler24hasRunnableSaveJobLockedEv",
@@ -458,7 +461,6 @@
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler31admitPendingIntoFreeSlotsLockedEv",
             "__ZN26LlamaLazyInitializeBackend17decrementRefCountEv",
             "__ZN14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
-            "__ZN30qvac_lib_inference_addon_llama5utils9getPromptEPK21common_chat_templatesR28common_chat_templates_inputs",
             "__ZThn48_NK14MtmdLlmContext14getKvCellsUsedEv",
             "__ZN14TextLlmContext16sampleFromLogitsEijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEEP10LlamaBatch",
             "__ZN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntime9forTargetEP13llama_context",
@@ -493,7 +495,6 @@
             "__ZN14MtmdLlmContext13onLogitsReadyEijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEEP10LlamaBatch",
             "__ZN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntime6createER13common_paramsP11llama_modelP13llama_context",
             "__ZNK18AsyncWeightsLoader25hasIndividualStreamedFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
-            "__ZThn48_N14TextLlmContext8onCancelERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
             "__ZN14MtmdLlmContext24snapshotPreRequestCursorEv",
             "__ZThn48_N14MtmdLlmContext9onFailureERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
             "__ZN14MtmdLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
@@ -521,6 +522,7 @@
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler19discardConversationERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE",
             "__ZN18AsyncWeightsLoader17setWeightsForFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEONS0_10unique_ptrINS0_15basic_streambufIcS3_EENS0_14default_deleteISB_EEEE",
             "__ZN14MtmdLlmContext4stopEv",
+            "__ZThn48_N14TextLlmContext8onCancelERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
             "__ZN12CacheManager20isFileMissingOrEmptyERKNSt3__14__fs10filesystem4pathE",
             "__ZN14TextLlmContext27restorePreRequestCacheStateEv",
             "__ZN14TextLlmContext24captureHistoryCheckpointEi",
@@ -549,6 +551,7 @@
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
             "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
             "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
+            "__ZNK14MtmdLlmContext15countChatTokensERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEERKNS1_I16common_chat_toolNS3_IS8_EEEE",
             "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
             "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
             "__ZN22load_fit_normalization31validateMobileMultiDeviceConfigERKNSt3__113unordered_mapINS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES7_NS0_4hashIS7_EENS0_8equal_toIS7_EENS5_INS0_4pairIKS7_S7_EEEEEE16llama_split_mode",
@@ -602,6 +605,7 @@
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardC1ERS1_PNSt3__111unique_lockINS4_5mutexEEE",
             "__ZN10LlamaModel11jobStartingEy",
             "__ZNK10LlamaModel6cancelEv",
+            "__ZN30qvac_lib_inference_addon_llama5utils23countRenderedChatTokensEPK21common_chat_templatesRKNSt3__16vectorI15common_chat_msgNS4_9allocatorIS6_EEEERKNS5_I16common_chat_toolNS7_ISC_EEEEbbRKNS4_8functionIFNS5_IiNS7_IiEEEERKNS4_12basic_stringIcNS4_11char_traitsIcEENS7_IcEEEEEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils33sequenceStateSnapshotFilesWrittenEv",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler10notifyDoneEj",
             "__ZN24llama_finetuning_helpers23parseCheckpointMetadataERKNSt3__14__fs10filesystem4pathERNS_18CheckpointMetadataE",
@@ -707,16 +711,19 @@
             "__ZN30qvac_lib_inference_addon_llama9assessFitEP8js_env_sP18js_callback_info_s",
             "__ZN30qvac_lib_inference_addon_llama6cancelEP8js_env_sP18js_callback_info_s",
             "__ZN30qvac_lib_inference_addon_llama15fitResultObjectEP8js_env_sRKNS_9FitResultE",
+            "__ZZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask5startImEEP10js_value_sP8js_env_sNS1_12CallbackDataIT_E4WorkENS9_7MarshalEENUlP11uv_handle_sE_8__invokeESD_",
             "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask13onEnvTeardownINSt3__19monostateEEEvP22js_deferred_teardown_sPv",
             "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask13onCloseHandleIN30qvac_lib_inference_addon_llama9FitResultEEEvP11uv_handle_s",
             "__ZN30qvac_lib_inference_addon_llama11activeSlotsEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp6logger8JsLogger13asyncCallbackEP10uv_async_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface6cancelEP8js_env_sP18js_callback_info_s",
+            "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask10onCompleteImEEvP10uv_async_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface10activeJobsEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask10onCompleteIN30qvac_lib_inference_addon_llama9FitResultEEEvP10uv_async_s",
             "__ZZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask5startINSt3__19monostateEEEP10js_value_sP8js_env_sNS1_12CallbackDataIT_E4WorkENSB_7MarshalEENUlP11uv_handle_sE_8__invokeESF_",
             "__ZN30qvac_lib_inference_addon_llama12discardCacheEP8js_env_sP18js_callback_info_s",
             "__ZN30qvac_lib_inference_addon_llama8finetuneEP8js_env_sP18js_callback_info_s",
+            "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask13onCloseHandleImEEvP11uv_handle_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface13releaseLoggerEP8js_env_sP18js_callback_info_s",
             "__ZZN28qvac_lib_inference_addon_cpp16OutputCallBackJs16drainAndTeardownEPNS0_5StateEbbENUlP11uv_handle_sE_8__invokeES4_",
             "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask13onCloseHandleINSt3__19monostateEEEvP11uv_handle_s",
@@ -724,10 +731,12 @@
             "__ZZN28qvac_lib_inference_addon_cpp6logger8JsLogger22closeAsyncHandleLockedEvENUlP11uv_handle_sE_8__invokeES3_",
             "__ZN30qvac_lib_inference_addon_llama14createInstanceEP8js_env_sP18js_callback_info_s",
             "__ZZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask5startIN30qvac_lib_inference_addon_llama9FitResultEEEP10js_value_sP8js_env_sNS1_12CallbackDataIT_E4WorkENSB_7MarshalEENUlP11uv_handle_sE_8__invokeESF_",
+            "__ZN30qvac_lib_inference_addon_llama10calcTokensEP8js_env_sP18js_callback_info_s",
             "__ZN30qvac_lib_inference_addon_llama6runJobEP8js_env_sP18js_callback_info_s",
             "__ZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface15destroyInstanceEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface8activateEP8js_env_sP18js_callback_info_s",
+            "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask13onEnvTeardownImEEvP22js_deferred_teardown_sPv",
             "__ZN28qvac_lib_inference_addon_cpp6logger8JsLogger13onEnvTeardownEPv",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface9setLoggerEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface11loadWeightsEP8js_env_sP18js_callback_info_s",