@qvac/llm-llamacpp 0.53.2 → 0.55.0

package/README.md

@@ -16,6 +16,7 @@ This native C++ addon, built using the `Bare` Runtime, simplifies running Large
   - [6. Run Inference](#6-run-inference)
   - [7. Release Resources](#7-release-resources)
 - [API behavior by state](#api-behavior-by-state)
+- [Assessing fit](#assessing-fit)
 - [Fine-tuning](#fine-tuning)
 - [Quickstart Example](#quickstart-example)
 - [Other Examples](#other-examples)
@@ -44,7 +45,7 @@ BitNet models require special backend handling on Adreno GPUs. When a BitNet mod
 
 **Dependencies:**
 - inference-addon-cpp (≥1.3.3): C++ addon framework (multi-job scheduler)
-- @qvac/fabric (^0.16.1): Shared llama.cpp/ggml/mtmd inference engine. Installed from npm; it carries the prebuilt runtime in its own tarball, so it must be present before `bare-make generate`/`build` and must not be pruned at runtime
+- @qvac/fabric (^0.18.0): Shared llama.cpp/ggml/mtmd inference engine. The desktop runtime ships in the `@qvac/fabric-<host>` platform package that `@qvac/fabric` installs as an optional dependency, so install without `--omit=optional` and keep both present before `bare-make generate`/`build`. Android and iOS apps add `@qvac/fabric` and `@qvac/fabric-android-arm64` or `@qvac/fabric-ios` as direct dependencies at the same exact version.
 - Bare Runtime (≥1.24.0): JavaScript runtime
 - Linux requires Clang/LLVM 22 with libc++
 ## Installation
@@ -360,6 +361,35 @@ So a cancelled batch that contained queued prompts rejects with `Cancelled`; cal
 Cancelling a queued job resolves immediately, whether or not the slots it was waiting for are held by an unrelated run — you never wait out someone else's generation to cancel your own queued work.
 
 
+## Assessing fit
+
+`assessFit` projects a load against the memory free right now. It reads GGUF metadata and never weight data, so the registry's weightless copy of a model answers the same as the model itself and the projection can run before anything is downloaded.
+
+```js
+const LlmLlamacpp = require('@qvac/llm-llamacpp')
+
+const fit = LlmLlamacpp.assessFit({
+  modelPath: '/models/model.gguf',
+  params: { 'ctx-size': '4096' }
+})
+
+fit.status // 'fits' | 'does-not-fit' | 'error'
+fit.reason // 'fits', 'does-not-fit', 'model-unreadable', 'no-backend-device' or 'unsupported-config'
+fit.gpuLayers // what fits, which is not always what the request asked for
+fit.ctxSize
+fit.devices // one row per device the model was assigned to, then a `host` row
+fit.deviceBytes // model, context and compute summed across the devices, host excluded
+fit.hostBytes // the same, for the trailing host row
+```
+
+`params` takes the load in llama's own CLI spelling without the leading `--`, exactly as the loader takes it: `ctx-size`, `tensor-split`, `override-tensor`, `cpu-moe`, `no-kv-offload` and the rest. Each is dispatched through llama's argument table, so a placement pinned there reaches the projection. A setting llama does not recognise, or a flag asked to be off that can only assert itself, is `status: "error"` with `unsupported-config`.
+
+Pinning `gpu-layers` fixes the placement: the fitter refuses to move layers off a device the load claimed, so the answer is whether that exact placement fits, never a reduced one. Leave it unset for a projection that can place the model itself.
+
+`minCtxSize` sets a floor the fitter may not reduce the context below, defaulting to llama's own `fit-ctx` of 4096. `marginBytes` is the memory to leave free on every device, defaulting to llama's `fit-target` of 1 GiB per device. `backendsDir` is where the dynamically-loaded ggml backends live.
+
+A model the fitter cannot read is `status: "error"`; only a broken request throws.
+
 ## Fine-tuning
 
 The library supports **LoRA finetuning** of GGUF models: train small adapter weights on top of a base model, then save the adapter and load it at inference time via the `lora` config option. You can pause and resume training from checkpoints.

package/addon.d.ts

@@ -88,6 +88,7 @@ export type MappedAddonEvent = {
  * `FinetuneProgress`, or `null` to drop it.
  */
 export declare function mapAddonEvent(rawEvent: unknown, rawData: unknown, rawError: unknown): MappedAddonEvent | null;
+export declare function resolveBackendsDir(): string;
 /**
  * An interface between Bare addon in C++ and JS runtime.
  */

package/addon.js

@@ -2,9 +2,11 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.LlamaInterface = void 0;
 exports.mapAddonEvent = mapAddonEvent;
+exports.resolveBackendsDir = resolveBackendsDir;
 /* eslint-disable @typescript-eslint/no-require-imports -- Bare modules expose CommonJS export shapes. */
 const fs = require("bare-fs");
 const path = require("bare-path");
+const fabricBackends = require("@qvac/fabric/backends");
 // Index-matched to the C++ GenerationStopReason enum (SequenceDriver.hpp).
 const STOP_REASONS = [
     "none",
@@ -70,24 +72,20 @@ function mapAddonEvent(rawEvent, rawData, rawError) {
     }
     return { type: type, data: rawData, error: rawError };
 }
-// The ggml compute backends ship with the @qvac/fabric dependency
-// (prebuilds/<host>/qvac__fabric). We deliberately do not copy them into this
+// The ggml compute backends ship next to the @qvac/fabric runtime
+// (<root>/<host>/qvac__fabric). We deliberately do not copy them into this
 // addon to avoid duplicating tens of MB per fabric consumer. On desktop,
-// resolve the single @qvac/fabric install and load the backends from there. On
-// mobile the package tree isn't resolvable at runtime (the worklet runs from a
-// packed bundle), so fall back to this addon's own prebuilds, where the mobile
-// packaging stages the backends. The native side appends BACKENDS_SUBDIR
-// ("<host>/qvac__fabric") to whichever root we return.
+// @qvac/fabric/backends resolves that root in whichever package holds the
+// runtime. On mobile the package tree isn't resolvable at runtime (the worklet
+// runs from a packed bundle), so fall back to this addon's own prebuilds, where
+// the mobile packaging stages the backends. The native side appends
+// BACKENDS_SUBDIR ("<host>/qvac__fabric") to whichever root we return.
 function resolveBackendsDir() {
-    try {
-        const fabricPkg = require.resolve("@qvac/fabric/package");
-        const fabricPrebuilds = path.join(path.dirname(fabricPkg), "prebuilds");
-        if (fs.existsSync(fabricPrebuilds))
-            return fabricPrebuilds;
-    }
-    catch {
-        // Mobile worklets cannot resolve the @qvac/fabric package tree.
-    }
+    // fabric's resolver only checks that the platform package resolves, not that
+    // its prebuilds are on disk.
+    const fabricRoot = fabricBackends.resolveBackendsDir();
+    if (fabricRoot !== null && fs.existsSync(fabricRoot))
+        return fabricRoot;
     return path.join(__dirname, "prebuilds");
 }
 /**
@@ -192,4 +190,5 @@ exports.LlamaInterface = LlamaInterface;
 module.exports = {
     LlamaInterface,
     mapAddonEvent,
+    resolveBackendsDir,
 };

package/fit.d.ts

@@ -0,0 +1,63 @@
+export interface LlamaFitRequest {
+    /** Absolute path to the GGUF, or to the registry's weightless copy. */
+    modelPath: string;
+    /**
+     * The load, in llama's own CLI spelling without the leading `--`, exactly as
+     * the loader takes it: `ctx-size`, `tensor-split`, `override-tensor`,
+     * `cpu-moe`, `no-kv-offload` and the rest. Each is dispatched through llama's
+     * argument table, so a placement pinned here reaches the projection.
+     *
+     * A setting llama does not recognise, or a flag asked to be off that can only
+     * assert itself, is `status: "error"` with `unsupported-config`.
+     */
+    params?: Record<string, string>;
+    /** Floor the fitter may not reduce the context below. */
+    minCtxSize?: number;
+    /** Memory to leave free on every device. */
+    marginBytes?: number;
+    /** Where the dynamically-loaded ggml backends live. */
+    backendsDir?: string;
+}
+export type LlamaFitStatus = 'fits' | 'does-not-fit' | 'error';
+export type LlamaFitReason = 'fits' | 'does-not-fit' | 'model-unreadable' | 'no-backend-device' | 'unsupported-config';
+export interface LlamaFitDevice {
+    name: string;
+    totalBytes: number;
+    freeBytes: number;
+    modelBytes: number;
+    contextBytes: number;
+    computeBytes: number;
+}
+export interface LlamaFitResult {
+    status: LlamaFitStatus;
+    reason: LlamaFitReason;
+    /** The layer count the placement resolved to. */
+    gpuLayers: number;
+    ctxSize: number;
+    /** One row per device the model was assigned to, then a `host` row. */
+    devices: LlamaFitDevice[];
+    /** Model, context and compute summed across the devices, host excluded. */
+    deviceBytes: number;
+    /** The same, for the trailing host row. */
+    hostBytes: number;
+    trainCtxSize: number;
+    expertCount: number;
+}
+/**
+ * Projects one model against the memory free right now, reading GGUF metadata
+ * and never weight data. The registry's weightless copy of a model answers the
+ * same as the model itself, so this can run before anything is downloaded.
+ *
+ * A model the fitter cannot read comes back as `status: "error"`; only a broken
+ * request throws.
+ *
+ * Runs synchronously on the calling thread and builds the model twice without
+ * allocating weights, so a caller that must stay responsive runs it off its
+ * own loop.
+ *
+ * The projection reads the model's vocabulary, and llama asserts on a
+ * vocabulary it finds inconsistent. An assert aborts rather than throwing, so
+ * a corrupt file ends the process instead of returning a status: take the
+ * model from a source you trust, or call this behind a process boundary.
+ */
+export declare function assessFit(request: LlamaFitRequest): LlamaFitResult;

package/fit.js

@@ -0,0 +1,31 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.assessFit = assessFit;
+const addon_1 = require("./addon");
+/**
+ * Projects one model against the memory free right now, reading GGUF metadata
+ * and never weight data. The registry's weightless copy of a model answers the
+ * same as the model itself, so this can run before anything is downloaded.
+ *
+ * A model the fitter cannot read comes back as `status: "error"`; only a broken
+ * request throws.
+ *
+ * Runs synchronously on the calling thread and builds the model twice without
+ * allocating weights, so a caller that must stay responsive runs it off its
+ * own loop.
+ *
+ * The projection reads the model's vocabulary, and llama asserts on a
+ * vocabulary it finds inconsistent. An assert aborts rather than throwing, so
+ * a corrupt file ends the process instead of returning a status: take the
+ * model from a source you trust, or call this behind a process boundary.
+ */
+function assessFit(request) {
+    // eslint-disable-next-line @typescript-eslint/no-require-imports -- native binding is resolved lazily from package prebuilds.
+    const binding = require('./binding.js');
+    return binding.assessFit({
+        ...request,
+        backendsDir: typeof request.backendsDir === 'string' && request.backendsDir.length > 0
+            ? request.backendsDir
+            : (0, addon_1.resolveBackendsDir)()
+    });
+}

package/index.d.ts

@@ -1,5 +1,6 @@
 import QvacLogger = require("@qvac/logging");
 import { QvacResponse } from "@qvac/infer-base";
+import { assessFit as assessFitImpl, type LlamaFitDevice, type LlamaFitRequest, type LlamaFitResult, type LlamaFitStatus } from "./fit";
 import type * as AddonModule from "./addon";
 type BareEventMap = Record<string | symbol, unknown[]>;
 /** Aliases: inside the namespace, `QvacResponse` resolves to its own member. */
@@ -63,6 +64,7 @@ interface LlmLlamacppConstructor {
     readonly prototype: LlmLlamacpp;
     /** Returns the first shard (matching `-NNNNN-of-MMMMM.gguf`) or the sole entry for single-file models. */
     readonly pickPrimaryGgufPath: typeof pickPrimaryGgufPath;
+    readonly assessFit: typeof assessFitImpl;
 }
 /** LLM client wrapping the native LlamaInterface for inference, finetuning, and pause/resume. */
 declare const LlmLlamacpp: LlmLlamacppConstructor;
@@ -133,7 +135,8 @@ declare namespace LlmLlamacpp {
         verbosity?: NumericLike;
         "main-gpu"?: NumericLike | string;
         /**
-         * How to split the model across GPUs.
+         * How to split the model across devices: local GPUs, plus remote ones
+         * registered with `rpc-servers`.
          *
          * - 'none' (default) — pin the whole model to a single GPU.
          * - 'layer' — pipeline parallelism; each GPU holds a contiguous slice of
@@ -188,6 +191,35 @@ declare namespace LlmLlamacpp {
         "flash-attn"?: "on" | "off" | "auto" | "enabled" | "disabled" | "true" | "false" | "0" | "1";
         /** Proportions for distributing layers/rows across GPUs (e.g. '1,1' for equal split, '3,1' for 75/25). */
         "tensor-split"?: string;
+        /**
+         * Comma-separated `host:port` endpoints of remote `ggml-rpc-server`
+         * processes, e.g. `'10.0.0.1:50052,10.0.0.2:50052'`. Their devices join the
+         * local ones and can then be selected with `devices`, letting a single
+         * model run split across several machines.
+         *
+         * Only the machine loading the model needs the model file. Every endpoint
+         * must be reachable at load time — an unreachable one fails the load rather
+         * than being skipped. Endpoints must run a server built from the same
+         * qvac-fabric revision as this addon, since the RPC wire protocol is
+         * versioned and mismatched builds refuse to connect.
+         * With the default `split-mode: 'none'`, `devices` must name one device;
+         * automatic single-device selection never picks an RPC device.
+         * A load also fails if its final device selection uses none of the devices
+         * registered by these endpoints.
+         *
+         * The channel is unauthenticated: use it only on a trusted private network.
+         */
+        "rpc-servers"?: string;
+        /**
+         * Explicit ggml device list, e.g. `'RPC0,RPC1'`. Overrides automatic
+         * placement, including the multi-GPU behaviour of spreading across every
+         * visible device — which is rarely what you want once remote devices are
+         * registered, since the list then mixes local and remote. Names come from
+         * the ggml registry (`RPC0`, `RPC1`, … for remote devices, in the order
+         * given to `rpc-servers`). Multiple names require `split-mode: 'layer'`
+         * or `'tensor'`; the default `'none'` accepts only one device.
+         */
+        devices?: string;
         "cache-type-k"?: string;
         "cache-type-v"?: string;
         /**
@@ -646,5 +678,9 @@ declare namespace LlmLlamacpp {
         stats?: FinetuneStats;
     }
     type QvacResponse<Output = any> = InferQvacResponseOf<Output>;
+    type FitRequest = LlamaFitRequest;
+    type FitResult = LlamaFitResult;
+    type FitStatus = LlamaFitStatus;
+    type FitDevice = LlamaFitDevice;
 }
 export = LlmLlamacpp;

package/index.js

@@ -7,6 +7,7 @@ const BatchHandler = require("./batchHandler");
 /* eslint-enable @typescript-eslint/no-require-imports */
 const infer_base_1 = require("@qvac/infer-base");
 const addon_1 = require("./addon");
+const fit_1 = require("./fit");
 const { runBusyError } = BatchHandler;
 function normalizeRunOptions(runOptions) {
     if (runOptions === undefined) {
@@ -257,6 +258,7 @@ function sanitizePromptForLog(prompt) {
 /** LLM client wrapping the native LlamaInterface for inference, finetuning, and pause/resume. */
 const LlmLlamacpp = class LlmLlamacpp {
     static pickPrimaryGgufPath = pickPrimaryGgufPath;
+    static assessFit = fit_1.assessFit;
     // Attached for tests; untyped because `typeof QvacResponse` is not nameable by consumers.
     static QvacResponse = infer_base_1.QvacResponse;
     addon;
@@ -786,6 +788,7 @@ const LlmLlamacpp = class LlmLlamacpp {
 // Runtime-redundant: ESM named imports need the top-level `module.exports.X =` form.
 /* eslint-disable @typescript-eslint/no-unsafe-member-access -- `module.exports` is untyped CommonJS surface. */
 module.exports.pickPrimaryGgufPath = pickPrimaryGgufPath;
+module.exports.assessFit = fit_1.assessFit;
 module.exports.QvacResponse = infer_base_1.QvacResponse;
 module.exports = LlmLlamacpp;
 /* eslint-enable @typescript-eslint/no-unsafe-member-access */

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/llm-llamacpp",
-  "version": "0.53.2",
+  "version": "0.55.0",
   "description": "llama addon for qvac",
   "addon": true,
   "scripts": {
@@ -22,7 +22,7 @@
     "test:mobile:generate": "npm run generate:benchmark-shards && bare ./scripts/generate-mobile-integration-tests.js",
     "test:mobile:validate": "npm run generate:benchmark-shards && node scripts/validate-mobile-tests.js && node scripts/validate-mobile-manifest.js",
     "typecheck": "tsc --noEmit -p tsconfig.json",
-    "check:generated": "npm run build:ts && out=$(git status --porcelain --untracked-files=all -- index.js addon.js batchHandler.js addonLogging.js index.d.ts addon.d.ts batchHandler.d.ts addonLogging.d.ts) && { [ -z \"$out\" ] || { echo \"$out\"; exit 1; }; }",
+    "check:generated": "npm run build:ts && out=$(git status --porcelain --untracked-files=all -- index.js addon.js batchHandler.js addonLogging.js fit.js index.d.ts addon.d.ts batchHandler.d.ts addonLogging.d.ts fit.d.ts) && { [ -z \"$out\" ] || { echo \"$out\"; exit 1; }; }",
     "test:types": "npm run typecheck && npm run check:generated && npm run test:types:consumer",
     "test:types:consumer": "tsc --noEmit -p test/types/tsconfig.cjs.json && tsc --noEmit -p test/types/tsconfig.esm.json",
     "test:dts": "tsc -p tsconfig.dts.json",
@@ -56,6 +56,8 @@
     "addonLogging.js",
     "addonLogging.d.ts",
     "addon.d.ts",
+    "fit.js",
+    "fit.d.ts",
     "prebuilds",
     "index.d.ts",
     "LICENSE",
@@ -71,6 +73,8 @@
   "bugs": "https://github.com/tetherto/qvac/issues",
   "homepage": "https://qvac.tether.io",
   "devDependencies": {
+    "@qvac/fabric-android-arm64": "^0.18.0",
+    "@qvac/fabric-ios": "^0.18.0",
     "@types/node": "^24.2.1",
     "bare-crypto": "^1.15.3",
     "bare-process": "^4.2.2",
@@ -92,7 +96,7 @@
     "typescript-eslint": "^8.63.0"
   },
   "dependencies": {
-    "@qvac/fabric": "^0.16.1",
+    "@qvac/fabric": "^0.18.0",
     "@qvac/infer-base": "^0.6.2",
     "@qvac/logging": "^0.1.0",
     "bare-fs": "^4.5.1",

package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.53.2"
+        "version": "0.55"
       }
     ],
     "exported_symbols": [
@@ -47,8 +47,10 @@
             "__ZNSt3__119piecewise_constructE",
             "__ZN8picojson12last_error_tIbE1sE",
             "__ZGVN28qvac_lib_inference_addon_cpp11JsInterface10instances_E",
+            "__ZGVZN30qvac_lib_inference_addon_llama9assessFitEP8js_env_sP18js_callback_info_sE8fitMutex",
             "__ZNSt3__19__any_imp17__unique_typeinfoINS_6vectorINS_4pairINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_7variantIJdxEEEEENS7_ISC_EEEEE4__idE",
             "__ZN28qvac_lib_inference_addon_cpp6logger8JsLogger13logger_async_E",
+            "__ZGVZN30qvac_lib_inference_addon_llama21isProcessScopedOptionERK10common_argE8excluded",
             "__ZGVN28qvac_lib_inference_addon_cpp6logger8JsLogger10log_queue_E",
             "__ZGVN8picojson12last_error_tIbE1sE",
             "__ZGVN28qvac_lib_inference_addon_cpp6logger8JsLogger12queue_mutex_E",
@@ -64,11 +66,13 @@
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface13instancesMtx_E",
             "__ZN30qvac_lib_inference_addon_llama22BATCH_OUTPUT_TYPE_NAMEE",
             "__ZN28qvac_lib_inference_addon_cpp6logger8JsLogger10log_queue_E",
+            "__ZZN30qvac_lib_inference_addon_llama21isProcessScopedOptionERK10common_argE8excluded",
             "__ZNSt3__19__any_imp17__unique_typeinfoINS_6vectorIN10LlamaModel6PromptENS_9allocatorIS4_EEEEE4__idE",
             "__ZNSt3__16ranges5__cpo4findE",
             "__ZN30qvac_lib_inference_addon_llama10JsBatchIds11kMintPrefixE",
             "__ZGVN28qvac_lib_inference_addon_cpp11JsInterface13instancesMtx_E",
             "__ZZN30qvac_lib_inference_addon_llama10JsBatchIds4mintEvE4next",
+            "__ZZN30qvac_lib_inference_addon_llama9assessFitEP8js_env_sP18js_callback_info_sE8fitMutex",
             "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output5ErrorEE4__idE",
             "__ZNSt3__19__any_imp17__unique_typeinfoINS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS6_IS8_EEEEE4__idE"
           ]
@@ -111,9 +115,9 @@
             "__ZN30qvac_lib_inference_addon_llama5utils21restoreRecurrentStateEP13llama_contextiRKNS0_22RecurrentStateSnapshotE",
             "__ZN14LlamaFinetuner8finetuneERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsEPN22FinetuneTerminalResult5StatsENSt3__18functionIFvRKN24llama_finetuning_helpers21FinetuneProgressStatsEEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils25selectReasoningBudgetTagsERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_6vectorIS7_NS5_IS7_EEEERKNS1_8optionalINS0_13ReasoningTagsEEE",
+            "__ZThn16_N14MtmdLlmContext12syncPositionEi",
             "__ZN30qvac_lib_inference_addon_llama8batching7Request20isOptHasTokensToFeedERKNSt3__18optionalIS1_EE",
             "__ZThn16_NK14TextLlmContext24getThinkingBlockDiscardsEv",
-            "__ZThn16_N14MtmdLlmContext12syncPositionEi",
             "__ZN12CacheManager14writeCacheFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils22ReasoningRollbackState19restorePrefillEntryEP13llama_contexti",
             "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
@@ -222,8 +226,8 @@
             "__ZN24llama_finetuning_helpers25setCurrentCheckpointStateEPNS_23TrainingCheckpointStateE",
             "__ZN10LlamaModel17processConcurrentERKNS_6PromptEy",
             "__ZN14TextLlmContext26captureReasoningBoundaryAtEi",
-            "__ZN24llama_finetuning_helpers12readTextFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler26completeGroupRequestLockedERKNSt3__110shared_ptrINS1_10BatchGroupEEE",
+            "__ZN24llama_finetuning_helpers12readTextFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "__ZNK30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher9requestAtEj",
             "__ZN26LlamaLazyInitializeBackend17incrementRefCountEv",
             "__ZN30qvac_lib_inference_addon_llama5utils23getChatTemplateForModelEPK11llama_modelRKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEE",
@@ -380,19 +384,20 @@
             "__ZN14TextLlmContext8setNPastEi",
             "__ZN19LlamaBackendsHandleD1Ev",
             "__ZN30qvac_lib_inference_addon_llama7parsers31validateReasoningBudgetOverrideEd",
+            "__ZN22load_fit_normalization23collectRpcRegistrationsERKNSt3__16vectorINS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEENS5_IS7_EEEERKNS1_IhNS5_IhEEEEbRKNS0_8functionIFP16ggml_backend_regPKcEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils18validateToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
             "__ZN30qvac_lib_inference_addon_llama23ReasoningBlockCompactor26recordCloseMarkerForReplayEi",
-            "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler10stepLockedEPNSt3__111unique_lockINS2_5mutexEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils22ReasoningRollbackState19capturePrefillEntryEP13llama_contextii",
-            "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchSchedulerD2Ev",
             "__ZN30qvac_lib_inference_addon_llama5utils22ReasoningRollbackState5resetEv",
+            "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler10stepLockedEPNSt3__111unique_lockINS2_5mutexEEE",
+            "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchSchedulerD2Ev",
+            "__ZNK14TextLlmContext8getNPastEv",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler27applyDeferredTeardownLockedEv",
             "__ZN14TextLlmContext32recordPostReasoningTokenIfActiveEi",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardD2Ev",
             "__ZN30qvac_lib_inference_addon_llama8batching7RequestC2EjO11PrefillPlanji",
             "__ZN10LlamaModel18initBatchSchedulerERNS_15ReloadableStateE",
             "__ZN30qvac_lib_inference_addon_llama5utils22RecurrentStateSnapshotC2EOS1_",
-            "__ZNK14TextLlmContext8getNPastEv",
             "__ZN12CacheManager17isFileInitializedERKNSt3__14__fs10filesystem4pathE",
             "__ZN10LlamaModelC2EONSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES7_ONS0_13unordered_mapIS6_S6_NS0_4hashIS6_EENS0_8equal_toIS6_EENS4_INS0_4pairIKS6_S6_EEEEEE",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler12submitLockedEONS1_13QueuedRequestE",
@@ -441,6 +446,7 @@
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
             "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
+            "__ZN22load_fit_normalization31validateMobileMultiDeviceConfigERKNSt3__113unordered_mapINS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES7_NS0_4hashIS7_EENS0_8equal_toIS7_EENS5_INS0_4pairIKS7_S7_EEEEEE16llama_split_mode",
             "__ZN24llama_finetuning_helpers24schedulerOptimizerParamsEPv",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler19cancelPendingLockedEv",
             "__ZN24llama_finetuning_helpers21tryHandlePauseRequestEP16ggml_opt_contextPNS_23TrainingCheckpointStateEbxx",
@@ -519,7 +525,6 @@
             "__ZN14MtmdLlmContext27resetToolDefinitionsDroppedEv",
             "__ZNK30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12chunkSizeForEj",
             "__ZNK14LlamaFinetuner30getPausedCheckpointStateSharedEv",
-            "_bare_get_module_name_v0",
             "__ZN14LlamaFinetuner22prepareTrainingDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
             "__ZNK30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot9elapsedMsEv",
             "__ZN14TextLlmContext13resetStopFlagEv",
@@ -532,6 +537,7 @@
             "__ZNK13ModelMetaData16checkInitializedEv",
             "__ZN14TextLlmContext13onLogitsReadyEijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEEP10LlamaBatch",
             "__ZN30qvac_lib_inference_addon_llama7parsers26parseReasoningBudgetConfigENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
+            "_bare_get_module_name_v0",
             "__ZNK14TextLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "__ZN30qvac_lib_inference_addon_llama5utils36replayTokensThroughDecoderForTestingEP13llama_contextiRKNSt3__16vectorIiNS3_9allocatorIiEEEEibiNS3_8functionIFiS2_11llama_batchEEE",
             "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler18notifyDoneNoexceptEj",
@@ -561,18 +567,11 @@
             "__ZN14MtmdLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE"
           ],
           "weak": [
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm2ELm2EEE10__dispatchB8ne190102IOZNS0_12__assignmentINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE16__generic_assignB8ne190102IRKNS0_17__copy_assignmentISE_LNS0_6_TraitE1EEEEEvOT_EUlRSM_OT0_E_JRNS0_6__baseILSI_1EJxdbSD_EEERKSU_EEEDcSM_DpT0_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm3EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102IRKNS0_18__copy_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSO_E_JRKNS0_6__baseILSI_1EJxdbSD_EEEEEEDcSN_DpT0_",
             "__ZZZN28qvac_lib_inference_addon_cpp16OutputCallBackJs26initializeProcessingThreadENSt3__110shared_ptrINS_11OutputQueueEEEENKUlvE_clEvENUlP11uv_handle_sE_8__invokeES7_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm3EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102INS0_18__move_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSM_E_JONS0_6__baseILSI_1EJxdbSD_EEEEEEDcSL_DpT0_",
+            "__ZN30qvac_lib_inference_addon_llama9assessFitEP8js_env_sP18js_callback_info_s",
             "__ZN30qvac_lib_inference_addon_llama6cancelEP8js_env_sP18js_callback_info_s",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm2EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102IRKNS0_18__copy_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSO_E_JRKNS0_6__baseILSI_1EJxdbSD_EEEEEEDcSN_DpT0_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm0EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102INS0_18__move_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSM_E_JONS0_6__baseILSI_1EJxdbSD_EEEEEEDcSL_DpT0_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm1ELm1EEE10__dispatchB8ne190102IOZNS0_12__assignmentINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE16__generic_assignB8ne190102IRKNS0_17__copy_assignmentISE_LNS0_6_TraitE1EEEEEvOT_EUlRSM_OT0_E_JRNS0_6__baseILSI_1EJxdbSD_EEERKSU_EEEDcSM_DpT0_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm0EEE10__dispatchB8ne190102IONS1_9__variant15__value_visitorIZZN28qvac_lib_inference_addon_cpp9out_handl27JsRuntimeStatsOutputHandlerC1EvENKUlRKNS_6vectorINS_4pairINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_7variantIJdxEEEEENSG_ISL_EEEEE_clESP_EUlOT_E_EEJRKNS0_6__baseILNS0_6_TraitE0EJdxEEEEEEDcSR_DpT0_",
             "__ZZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runEP8js_env_sNSt3__18functionIFvvEEEENUlP11uv_handle_sE_8__invokeES9_",
             "__ZN30qvac_lib_inference_addon_llama11activeSlotsEP8js_env_sP18js_callback_info_s",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm1EEE10__dispatchB8ne190102IONS1_9__variant15__value_visitorIZZN28qvac_lib_inference_addon_cpp9out_handl27JsRuntimeStatsOutputHandlerC1EvENKUlRKNS_6vectorINS_4pairINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_7variantIJdxEEEEENSG_ISL_EEEEE_clESP_EUlOT_E_EEJRKNS0_6__baseILNS0_6_TraitE0EJdxEEEEEEDcSR_DpT0_",
             "__ZN28qvac_lib_inference_addon_cpp6logger8JsLogger13asyncCallbackEP10uv_async_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface6cancelEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface10activeJobsEP8js_env_sP18js_callback_info_s",
@@ -580,20 +579,15 @@
             "__ZN30qvac_lib_inference_addon_llama8finetuneEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface13releaseLoggerEP8js_env_sP18js_callback_info_s",
             "__ZZN28qvac_lib_inference_addon_cpp16OutputCallBackJs16drainAndTeardownEPNS0_5StateEbbENUlP11uv_handle_sE_8__invokeES4_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm1EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102INS0_18__move_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSM_E_JONS0_6__baseILSI_1EJxdbSD_EEEEEEDcSL_DpT0_",
             "__ZN30qvac_lib_inference_addon_llama14createInstanceEP8js_env_sP18js_callback_info_s",
             "__ZZN28qvac_lib_inference_addon_cpp6logger8JsLogger22closeAsyncHandleLockedEvENUlP11uv_handle_sE_8__invokeES3_",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm0ELm0EEE10__dispatchB8ne190102IOZNS0_12__assignmentINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE16__generic_assignB8ne190102IRKNS0_17__copy_assignmentISE_LNS0_6_TraitE1EEEEEvOT_EUlRSM_OT0_E_JRNS0_6__baseILSI_1EJxdbSD_EEERKSU_EEEDcSM_DpT0_",
             "__ZN30qvac_lib_inference_addon_llama6runJobEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask10onCompleteEP10uv_async_s",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm3ELm3EEE10__dispatchB8ne190102IOZNS0_12__assignmentINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE16__generic_assignB8ne190102IRKNS0_17__copy_assignmentISE_LNS0_6_TraitE1EEEEEvOT_EUlRSM_OT0_E_JRNS0_6__baseILSI_1EJxdbSD_EEERKSU_EEEDcSM_DpT0_",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface15destroyInstanceEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface8activateEP8js_env_sP18js_callback_info_s",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm0EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102IRKNS0_18__copy_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSO_E_JRKNS0_6__baseILSI_1EJxdbSD_EEEEEEDcSN_DpT0_",
             "__ZN28qvac_lib_inference_addon_cpp6logger8JsLogger13onEnvTeardownEPv",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm1EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102IRKNS0_18__copy_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSO_E_JRKNS0_6__baseILSI_1EJxdbSD_EEEEEEDcSN_DpT0_",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface9setLoggerEP8js_env_sP18js_callback_info_s",
-            "__ZNSt3__116__variant_detail12__visitation6__base12__dispatcherIJLm2EEE10__dispatchB8ne190102IOZNS0_6__ctorINS0_8__traitsIJxdbNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEEE19__generic_constructB8ne190102INS0_18__move_constructorISE_LNS0_6_TraitE1EEEEEvRSF_OT_EUlSM_E_JONS0_6__baseILSI_1EJxdbSD_EEEEEEDcSL_DpT0_",
+            "__ZZN30qvac_lib_inference_addon_llama20applyLlamaLoadParamsEP8js_env_sN28qvac_lib_inference_addon_cpp2js6ObjectER13common_paramsENUliPPcE_8__invokeEiS8_",
             "__ZN28qvac_lib_inference_addon_cpp11JsInterface11loadWeightsEP8js_env_sP18js_callback_info_s",
             "__ZN28qvac_lib_inference_addon_cpp16OutputCallBackJs13onEnvTeardownEPv",
             "__ZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask13onEnvTeardownEP22js_deferred_teardown_sPv",