@qvac/llm-llamacpp 0.66.0 → 0.68.0

package/README.md

@@ -15,6 +15,7 @@ This native C++ addon, built using the `Bare` Runtime, simplifies running Large
   - [5. Load Model](#5-load-model)
   - [6. Run Inference](#6-run-inference)
     - [Tool calling](#tool-calling)
+    - [Ordered text and media](#ordered-text-and-media)
     - [Batch inference](#batch-inference)
   - [7. Release Resources](#7-release-resources)
 - [API behavior by state](#api-behavior-by-state)
@@ -113,7 +114,7 @@ const args = {
 
 The `args` obj contains the following properties:
 
-* `files.model`: Required. An array of absolute paths to the GGUF model file(s) to load. The caller is responsible for passing the complete set of files for the model, including every shard and the `.tensors.txt` companion for multi-shard models (see [Sharded models](#sharded-models) below).
+* `files.model`: Exactly one absolute disk path: the single GGUF file or only the first shard. Fabric reads the remaining shards from the same directory.
 * `files.projectionModel`: Optional. Absolute path to the projection model file. This is required for multimodal support.
 * `config`: The model configuration object (see next section).
 * `logger`: This property is used to create a [`QvacLogger`](../logging) instance, which handles all logging functionality.
@@ -121,46 +122,24 @@ The `args` obj contains the following properties:
 
 #### Sharded models
 
-The addon no longer expands sharded models internally. If you are loading a multi-shard GGUF model, **the caller MUST pass every file** — including the `.tensors.txt` companion file that lives alongside the shards — in `files.model`. Anything missing will cause the addon to fail during weight streaming.
+Models load from disk only. Pass exactly one absolute path in `files.model`:
+the single GGUF file, or the first shard (`*-00001-of-000NN.gguf`). All shards
+must already be in the same directory with their original names. Fabric reads
+the GGUF metadata, discovers the remaining shards and reads their weights.
+No JavaScript weight buffers or `.tensors.txt` file are needed.
 
-**Required ordering for multi-shard models:**
-1. The `.tensors.txt` companion file **first**.
-2. Each `*-NNNNN-of-MMMMM.gguf` shard in **numerical order** (shard `00001` before `00002`, and so on).
-
-Example — loading a 5-shard model:
+Passing multiple paths or weight buffers is rejected. Migrate an old all-files
+list to `[firstShardPath]`. The `loadWeights()` API has been removed.
+This removes model-weight streaming, while generated output still streams as before.
 
 ```js
-const path = require('bare-path')
-const LlmLlamacpp = require('@qvac/llm-llamacpp')
-
-const dir = path.resolve('./models')
-const modelBase = 'my-big-model-Q4_K_M'
-
 const model = new LlmLlamacpp({
-  files: {
-    model: [
-      path.join(dir, `${modelBase}.tensors.txt`),
-      path.join(dir, `${modelBase}-00001-of-00005.gguf`),
-      path.join(dir, `${modelBase}-00002-of-00005.gguf`),
-      path.join(dir, `${modelBase}-00003-of-00005.gguf`),
-      path.join(dir, `${modelBase}-00004-of-00005.gguf`),
-      path.join(dir, `${modelBase}-00005-of-00005.gguf`)
-    ]
-  },
-  config,
-  logger: console,
-  opts: { stats: true }
+  files: { model: ['/absolute/models/model-00001-of-00003.gguf'] },
+  config
 })
-
 await model.load()
 ```
 
-For single-file GGUF models, pass a one-element array:
-
-```js
-files: { model: [path.join(dir, 'Llama-3.2-1B-Instruct-Q4_0.gguf')] }
-```
-
 ### 3. Create the `config` obj
 
 The `config` obj consists of a set of hyper-parameters which can be used to tweak the behaviour of the model.  
@@ -206,7 +185,8 @@ const config = {
 | cache_checkpoints_max_bytes | integer (bytes)                 | 0 (unlimited)                | Byte budget for those checkpoints, enforced before the count. The load fails with `InvalidArgument` when it cannot hold `cache_checkpoints` checkpoints of the largest size the context allows (on a sliding-window model, a full window). Also accepted as `cache-checkpoints-max-bytes` |
 | cache_checkpoint_storage | `"memory"` or `"disk"`             | `"memory"`                   | Where checkpoints and the per-request rollback snapshot live. `"memory"` keeps them in host RAM so a cached chat never touches the disk; `"disk"` writes them to a private directory inside `cache_checkpoint_dir`, which it requires. Also accepted as `cache-checkpoint-storage` |
 | cache_checkpoint_dir | directory path                              | none                         | **Required with `cache_checkpoint_storage: "disk"`**, and refused without it: where checkpoint files go, inside a private directory the addon creates in it (mode 0700; on Windows, the directory itself). On mobile, pass the app's cache directory. The load fails with `InvalidArgument` when it is missing, empty, or unusable. Also accepted as `cache-checkpoint-dir` |
-| cache_ram_mib | 0 – 1048576 (MiB)                               | 0 (off)                      | Host-RAM budget for conversations that are not running. A `cacheKey` switch or a batch slot eviction moves the conversation there instead of writing its file, and the next request on that key restores it. A write-back cache: unsaved turns reach the file on `saveCache()`, when the budget pushes the entry out, or at unload; ephemeral entries are dropped instead. Also accepted as `cache-ram-mib` ([details](./docs/cache-api.md#keep-switched-away-conversations-in-ram)) |
+| cache_ram_mib | 0 – 1048576 (MiB)                               | 0 (off)                      | Host-RAM budget for conversations that are not running. A `cacheKey` switch or a batch slot eviction moves the conversation there instead of writing its file, and the next request on that key restores it; unless `cache_share_across_keys` is `"false"`, a request on another key that shares its prompt prefix starts from a copy of it. A write-back cache: unsaved turns reach the file on `saveCache()`, when the budget pushes the entry out, or at unload; ephemeral entries are dropped instead. Also accepted as `cache-ram-mib` ([details](./docs/cache-api.md#keep-switched-away-conversations-in-ram)) |
+| cache_share_across_keys | `"true"`/`"on"`/`"1"`, `"false"`/`"off"`/`"0"` | `"true"`                     | Whether a request on a decoder-only model may start from state another conversation left (keyless memory, another key's state, a `cacheKey` file the model used). `"false"` keeps every request to its own key's state, for callers who must not learn through `cacheTokens` or timing whether another conversation shares their prompt. Also accepted as `cache-share-across-keys` ([details](./docs/cache-api.md#keep-conversations-apart)) |
 | flash-attn        | `"on"`/`"enabled"`/`"true"`/`"1"`, `"off"`/`"disabled"`/`"false"`/`"0"`, or `"auto"` | `"on"`, except when finetuning or on a BitNet model, where it is forced off | Flash attention. The four truthy and four falsey spellings are equivalent; `"auto"` is a third state that defers to qvac-fabric's runtime capability probe. Lower-case only — matching is case-sensitive and any other value is rejected. Affects the KV-cache auto-default — see below. Also accepted as `flash_attn`; supplying both spellings is an error |
 | cache-type-k      | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, …      | auto (see below)             | KV-cache **key** quantization type. Unset = auto-default (see KV-cache type below) |
 | cache-type-v      | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, …      | auto (see below)             | KV-cache **value** quantization type. Quantizing V requires `flash-attn` on |
@@ -345,6 +325,28 @@ const response = await model.run([
 
 The tool grammar holds the model to the order of `parameters`. Inside an object it accepts the required properties first, in declared order, then the optional ones, also in declared order. When the model writes a property out of that order, the call isn't rejected with an error: the model drops a field, writes a broken value or never finishes the call. Declare `required` properties before optional ones at every level, and declare each group in the order the model is likely to write it. This applies to nested objects, and to every argument in formats that write the arguments as one JSON object (Qwen3). Qwen3.5's top-level `<parameter=...>` arguments are less strict: up to six required arguments in any order, then the optional ones in any order. An optional argument before a required one is still rejected there.
 
+#### Ordered text and media
+
+A user message's `content` is either a string or an ordered array of parts. Use the array when the position of an image inside the text matters — multiple-choice options that are pictures, illustrated forms, steps — so the model sees the text and every image in one turn, in order:
+
+```javascript
+const response = await model.run([{
+  role: 'user',
+  content: [
+    { type: 'text', text: 'Which image shows the same place as ' },
+    { type: 'media', content: '/images/place.jpg' },
+    { type: 'text', text: '?\nA. ' },
+    { type: 'media', content: imageABytes }, // raw bytes work too
+    { type: 'text', text: '\nB. ' },
+    { type: 'media', content: '/images/option-b.jpg' }
+  ]
+}])
+```
+
+Each media part becomes an `<__media__>` marker at its own position in the rendered turn — the same prompt llama-server builds from an OpenAI content-parts request. A part's `content` is an absolute path or a `Uint8Array`; media parts are only valid on user messages, and an unknown part type is rejected with `InvalidInputFormat`.
+
+The alternative — one `{ role: 'user', type: 'media' }` message per part — keeps the order but renders each text part as its own user turn, so text that refers to "this image" ends up in a different turn from the image.
+
 #### Batch inference
 
 Load the model with `parallel >= 2`, then pass an array of prompts. Each chunk arrives tagged with the id of the sequence that produced it; `await()` returns results in input order.
@@ -443,7 +445,7 @@ Admission is controlled by `rejectWhenBusy` (instance-level `opts.rejectWhenBusy
 
 #### Prefill (cache warming) with `parallel >= 2`
 
-A prefill-only run (`runOptions.prefill: true`) is admitted on a parallel model only when its product survives the slot teardown, i.e. it has a `cacheKey`: the warmed conversation stays in its slot for the next request on that key. A keyless prefill warms context state that no concurrent job could ever reach, so it is rejected with `InvalidArgument`; run it on a `parallel: 1` model instead. The same rule applies per item in batch runs. See [cache-api.md](./docs/cache-api.md).
+A prefill-only run (`runOptions.prefill: true`) is admitted on a parallel model only when its product survives the slot teardown, i.e. it has a `cacheKey`: the warmed conversation stays in its slot for the next request on that key. A keyless prefill warms state no later request can count on (at most parked anonymously, first to be evicted), so it is rejected with `InvalidArgument`; run it on a `parallel: 1` model instead. The same rule applies per item in batch runs. See [cache-api.md](./docs/cache-api.md).
 
 #### Cancelling a batch
 
@@ -735,7 +737,7 @@ npm run test:cpp:models:ci   # exactly what .github/workflows/cpp-tests-llm.yml
                              # downloads; matches what CI exercises
 ```
 
-First-run downloads pull several GB from Hugging Face. Every fixture is SHA256-verified against a digest pinned in `scripts/download-unit-test-models.js`; mismatched or partial files are re-downloaded automatically. Set `HF_TOKEN` if a repo requires authentication. Override paths with env vars such as `SHARDED_MODEL_FIRST_SHARD_PATH` (see `test/unit/test_common.hpp`).
+First-run downloads pull several GB, mostly from Hugging Face. Every fixture is SHA256-verified against a digest pinned in `scripts/download-unit-test-models.js`; mismatched or partial files are re-downloaded automatically. Set `HF_TOKEN` if a repo requires authentication. Fixtures pinned by `s3Path` (currently `bitnet_b1_58-large-TQ2_0.gguf`) are copied from the QVAC model registry bucket and need `MODEL_S3_BUCKET` plus AWS credentials; if either is missing or the copy fails, a local run skips the fixture with a warning and its tests skip, while `--ci` fails. Override paths with env vars such as `SHARDED_MODEL_FIRST_SHARD_PATH` or `BITNET_MODEL_PATH` (see `test/unit/test_common.hpp`).
 
 ### Fuzzing
 

package/addon.d.ts

@@ -14,6 +14,7 @@ export interface AddonMessage {
     generationParams?: GenerationParams;
     cacheKey?: string;
     ephemeral?: boolean;
+    baseCacheKey?: string;
 }
 export interface AddonMediaMessage {
     type: "media";
@@ -54,11 +55,6 @@ export type AddonBatchRunResult = {
     id?: never;
     ids: string[];
 };
-export interface LoadWeightsData {
-    filename: string;
-    chunk: Uint8Array | null;
-    completed: boolean;
-}
 export interface AddonConfigurationParams {
     path: string;
     projectionPath: string;
@@ -73,7 +69,6 @@ export interface AddonCalcTokensParams {
 }
 export interface LlamaBinding {
     createInstance(owner: LlamaInterface, configurationParams: AddonConfigurationParams, outputCallback: AddonOutputCallback, reserved: null): object;
-    loadWeights(handle: unknown, data: LoadWeightsData): Promise<void> | void;
     activate(handle: unknown): Promise<void> | void;
     activeJobs(handle: unknown): number;
     activeSlots(handle: unknown): number;
@@ -104,7 +99,6 @@ export declare class LlamaInterface {
     private readonly _binding;
     private _handle;
     constructor(binding: unknown, configurationParams: AddonConfigurationParams, outputCb: AddonOutputCallback);
-    loadWeights(weightsData: LoadWeightsData): Promise<void>;
     /**
      * Moves addon to the LISTENING state after all the initialization is done
      */

package/addon.js

@@ -80,9 +80,6 @@ class LlamaInterface {
         }
         this._handle = this._binding.createInstance(this, configurationParams, outputCb, null);
     }
-    loadWeights(weightsData) {
-        return Promise.resolve(this._binding.loadWeights(this._handle, weightsData));
-    }
     /**
      * Moves addon to the LISTENING state after all the initialization is done
      */

package/index.d.ts

@@ -16,11 +16,10 @@ type LlmLlamacppArgs = LlmLlamacpp.LlmLlamacppArgs;
 type Message = LlmLlamacpp.Message;
 type RunOptions = LlmLlamacpp.RunOptions;
 /**
- * Returns the first shard (matching `-NNNNN-of-MMMMM.gguf`) or the sole
- * entry for single-file models. Matches the C++ shard-expansion contract
- * in `GGUFShards::expandGGUFIntoShards`.
+ * Validates the single disk path. For a sharded GGUF, pass only the first
+ * shard; Fabric discovers the remaining shards. Weight streaming is unsupported.
  */
-declare function pickPrimaryGgufPath(files: string[]): string;
+declare function validateModelPath(files: string[]): string;
 interface LlmLlamacpp {
     /** The native addon, or `null` before `load()` and after `unload()`. Advanced/test access only. */
     addon: LlmLlamacpp.Addon | null;
@@ -93,8 +92,8 @@ interface LlmLlamacpp {
 interface LlmLlamacppConstructor {
     new (args: LlmLlamacppArgs): LlmLlamacpp;
     readonly prototype: LlmLlamacpp;
-    /** Returns the first shard (matching `-NNNNN-of-MMMMM.gguf`) or the sole entry for single-file models. */
-    readonly pickPrimaryGgufPath: typeof pickPrimaryGgufPath;
+    /** Validates and returns the single absolute disk path; split models require the first shard. */
+    readonly validateModelPath: typeof validateModelPath;
     readonly assessFit: typeof assessFitImpl;
 }
 /** LLM client wrapping the native LlamaInterface for inference, finetuning, and pause/resume. */
@@ -119,11 +118,6 @@ declare namespace LlmLlamacpp {
      */
     type AddonBatchRunResult = AddonModule.AddonBatchRunResult;
     interface Addon {
-        loadWeights(data: {
-            filename: string;
-            chunk: Uint8Array | null;
-            completed: boolean;
-        }, logger?: QvacLogger): Promise<void>;
         activate(): Promise<void>;
         /** Single-request admission: resolves the accepted flag plus the native-assigned job id. */
         runJob(data: AddonRunJobMessage[]): Promise<AddonRunJobResult>;
@@ -380,6 +374,19 @@ declare namespace LlmLlamacpp {
          * maximum is 1048576 (1 TiB). Also accepted as `cache-ram-mib`.
          */
         cache_ram_mib?: NumericLike;
+        /**
+         * Whether a request on a decoder-only model may start from state another
+         * conversation left: the memory a request without a `cacheKey` keeps,
+         * another key's state in a sequence or the RAM tier, or a `cacheKey` file
+         * the model has used. `'true'` (default) shares it, as llama-server's
+         * prompt cache does. `'false'` keeps every request to its own key's state,
+         * for a model serving callers who must not learn through `cacheTokens` or
+         * timing whether another conversation shares their prompt: a request
+         * without a key starts cold and is reset afterwards, a new key starts cold,
+         * and `baseCacheKey` has no effect. Also accepted as
+         * `cache-share-across-keys`.
+         */
+        cache_share_across_keys?: "true" | "false" | "on" | "off" | "1" | "0";
         /**
          * Speculative decoding, as llama-server's `--spec-type`. `'draft-mtp'`
          * drafts with the model's own multi-token-prediction (MTP / NextN) head,
@@ -420,8 +427,9 @@ declare namespace LlmLlamacpp {
         [key: string]: string | number | boolean | string[] | undefined;
     }
     interface LlmLlamacppArgs {
+        /** One absolute disk path, or only the first shard path. Weight buffers, multiple paths and tensor-name files are unsupported. */
         files: {
-            model: string[];
+            model: [string];
             projectionModel?: string;
         };
         config: LlamaConfig;
@@ -437,9 +445,30 @@ declare namespace LlmLlamacpp {
             rejectWhenBusy?: boolean;
         };
     }
+    /**
+     * One ordered piece of a message's `content`. Media parts keep their place in
+     * the turn: a single user message can interleave text and media, which a
+     * string `content` plus separate `type: "media"` messages cannot express
+     * without splitting the turn.
+     */
+    type UserContentPart = {
+        type: "text";
+        text: string;
+    } | {
+        type: "media";
+        /**
+         * Either an absolute path to the media file (`string`) or its raw bytes
+         * (`Uint8Array`), the same two forms a `type: "media"` message accepts.
+         */
+        content: string | Uint8Array;
+    };
     interface UserTextMessage {
         role: "system" | "assistant" | "user" | "tool" | "session" | string;
-        content: string;
+        /**
+         * Plain text, or ordered text/media parts rendered as one turn. Media parts
+         * are only valid on user messages.
+         */
+        content: string | UserContentPart[];
         /**
          * An assistant turn's reasoning, separate from its answer in `content`, as
          * in OpenAI-compatible APIs. Optional: an answer sent back with its
@@ -554,10 +583,9 @@ declare namespace LlmLlamacpp {
          * Run prefill only (cache warming): the prompt is evaluated but no tokens
          * are generated. On a model loaded with `parallel >= 2` a prefill needs a
          * `cacheKey` (the warmed conversation stays in its slot for the next
-         * request on that key) — a keyless prefill warms context state that no
-         * concurrent job could reach and is rejected with `InvalidArgument`; run
-         * keyless prefills on a `parallel: 1` model. The same rule applies per
-         * batch item.
+         * request on that key) — a keyless prefill warms state no later request
+         * can count on and is rejected with `InvalidArgument`; run keyless
+         * prefills on a `parallel: 1` model. The same rule applies per batch item.
          */
         prefill?: boolean;
         generationParams?: GenerationParams;
@@ -574,11 +602,26 @@ declare namespace LlmLlamacpp {
          * otherwise be lost with turns the file lacks: when it is set aside for
          * another key, evicted from its batch slot or from the RAM tier, or the
          * model is reloaded or unloaded. With `ephemeral: true` each of those
-         * drops it instead, so its next request starts cold. It still loads an
-         * existing file, and an explicit `saveCache(cacheKey)` still writes it.
+         * drops it instead, so its next request finds no state of its own (it
+         * still reuses any prefix it shares with what the model holds, except on
+         * encoder-decoder models). It still loads an existing file, and an
+         * explicit `saveCache(cacheKey)` still writes it.
          * The flag belongs to the conversation's latest request.
          */
         ephemeral?: boolean;
+        /**
+         * A `cacheKey` file to start from when nothing in memory serves more of
+         * the prompt, such as a system prompt and tools primed with `prefill`.
+         * It is read, never written: the request's own state goes to its
+         * `cacheKey`. A file that is missing or shares no prefix with the prompt
+         * is skipped. Once named, the file stays a source for later requests on
+         * this model, as any `cacheKey` file it used does, until files named
+         * or picked as a prompt's closest match more recently push it out (the
+         * model keeps the 256 most recent, as far as 64 MiB of memory holds
+         * their prompt records). No effect with `cache_share_across_keys:
+         * 'false'`.
+         */
+        baseCacheKey?: string;
         /**
          * Admission policy when the worker pool is full. `true` rejects before
          * submitting with an `Error` carrying `code === 'RUN_BUSY'` — branch on the

package/index.js

@@ -16,6 +16,7 @@ function normalizeRunOptions(runOptions) {
             generationParams: undefined,
             cacheKey: undefined,
             ephemeral: false,
+            baseCacheKey: undefined,
             rejectWhenBusy: undefined,
         };
     }
@@ -43,6 +44,9 @@ function normalizeRunOptions(runOptions) {
     if (options.ephemeral !== undefined && typeof options.ephemeral !== "boolean") {
         throw new TypeError("ephemeral must be a boolean when provided");
     }
+    if (options.baseCacheKey !== undefined && typeof options.baseCacheKey !== "string") {
+        throw new TypeError("baseCacheKey must be a string when provided");
+    }
     if (options.rejectWhenBusy !== undefined && typeof options.rejectWhenBusy !== "boolean") {
         throw new TypeError("rejectWhenBusy must be a boolean when provided");
     }
@@ -51,15 +55,28 @@ function normalizeRunOptions(runOptions) {
         generationParams: normalizeGenerationParams(options.generationParams),
         cacheKey: options.cacheKey,
         ephemeral: options.ephemeral === true,
+        baseCacheKey: options.baseCacheKey,
         // Left undefined when unset so admission falls back to the instance default.
         rejectWhenBusy: options.rejectWhenBusy,
     };
 }
+// Malformed parts (null, non-objects) are not byte media: they pass through
+// untouched so the native validator rejects them with `InvalidInputFormat`.
+function isByteMediaPart(part) {
+    const candidate = part;
+    return (candidate !== null &&
+        typeof candidate === "object" &&
+        candidate.type === "media" &&
+        candidate.content instanceof Uint8Array);
+}
 function promptToAddonMessages(prompt, runOptions) {
     if (!Array.isArray(prompt)) {
         throw new TypeError("Prompt input must be Message[]");
     }
-    const { prefill, generationParams, cacheKey, ephemeral } = normalizeRunOptions(runOptions);
+    const { prefill, generationParams, cacheKey, ephemeral, baseCacheKey } = normalizeRunOptions(runOptions);
+    // Only ever JSON-serialized into the addon's text message: a byte-buffer
+    // media part reaches the native side path-less, bound to the hoisted buffer
+    // by position, so the wire shape is wider than the public `Message` type.
     const textMessages = [];
     const mediaItems = [];
     for (const message of prompt) {
@@ -67,10 +84,26 @@ function promptToAddonMessages(prompt, runOptions) {
         if (media.role === "user" && media.type === "media" && media.content instanceof Uint8Array) {
             mediaItems.push(media.content);
             textMessages.push({ ...media, content: "" });
+            continue;
         }
-        else {
-            textMessages.push(message);
+        // Ordered parts: hoist each media part's bytes next to the other media
+        // messages and leave a path-less part behind. The native side records the
+        // plan in marker order, so the bytes still bind to the right markers.
+        const text = message;
+        if (Array.isArray(text.content)) {
+            textMessages.push({
+                ...text,
+                content: text.content.map((part) => {
+                    if (isByteMediaPart(part)) {
+                        mediaItems.push(part.content);
+                        return { type: "media" };
+                    }
+                    return part;
+                }),
+            });
+            continue;
         }
+        textMessages.push(message);
     }
     const promptMessages = [];
     for (const mediaData of mediaItems) {
@@ -83,6 +116,7 @@ function promptToAddonMessages(prompt, runOptions) {
         generationParams,
         cacheKey,
         ephemeral,
+        baseCacheKey,
     });
     return promptMessages;
 }
@@ -239,13 +273,24 @@ function normalizeFinetuneParams(opts) {
     return out;
 }
 /**
- * Returns the first shard (matching `-NNNNN-of-MMMMM.gguf`) or the sole
- * entry for single-file models. Matches the C++ shard-expansion contract
- * in `GGUFShards::expandGGUFIntoShards`.
+ * Validates the single disk path. For a sharded GGUF, pass only the first
+ * shard; Fabric discovers the remaining shards. Weight streaming is unsupported.
  */
-function pickPrimaryGgufPath(files) {
-    const SHARD_REGEX = /-\d+-of-\d+\.gguf$/;
-    return files.find((p) => SHARD_REGEX.test(p)) || files[0];
+function validateModelPath(files) {
+    if (!Array.isArray(files) || files.length === 0) {
+        throw new TypeError("files.model must contain one absolute disk path");
+    }
+    if (files.length !== 1) {
+        throw new TypeError("Weight streaming is no longer supported. Pass only one disk path in files.model; " +
+            "for a sharded model, pass only the first shard path. Do not pass a tensor-name file.");
+    }
+    if (typeof files[0] !== "string" || files[0].length === 0 || !path.isAbsolute(files[0])) {
+        throw new TypeError("files.model[0] must be an absolute disk path string; weight buffers are not supported");
+    }
+    if (files[0].endsWith(".tensors.txt")) {
+        throw new TypeError("Weight streaming is no longer supported. Pass the first GGUF shard path, not a tensor-name file.");
+    }
+    return files[0];
 }
 // Replace media Uint8Array contents with a small summary so the logger never
 // asks V8 to JSON.stringify multi-MB binary blobs.
@@ -257,12 +302,21 @@ function sanitizePromptForLog(prompt) {
         if (media && media.type === "media" && media.content instanceof Uint8Array) {
             return { ...media, content: `[Uint8Array byteLength=${media.content.byteLength}]` };
         }
+        const text = msg;
+        if (Array.isArray(text?.content)) {
+            return {
+                ...text,
+                content: text.content.map((part) => isByteMediaPart(part)
+                    ? { ...part, content: `[Uint8Array byteLength=${part.content.byteLength}]` }
+                    : part),
+            };
+        }
         return msg;
     });
 }
 /** LLM client wrapping the native LlamaInterface for inference, finetuning, and pause/resume. */
 const LlmLlamacpp = class LlmLlamacpp {
-    static pickPrimaryGgufPath = pickPrimaryGgufPath;
+    static validateModelPath = validateModelPath;
     static assessFit = fit_1.assessFit;
     // Attached for tests; untyped because `typeof QvacResponse` is not nameable by consumers.
     static QvacResponse = infer_base_1.QvacResponse;
@@ -270,7 +324,7 @@ const LlmLlamacpp = class LlmLlamacpp {
     opts;
     logger;
     state;
-    _files;
+    _modelPath;
     _projectionModelPath;
     _config;
     _finetuneJob;
@@ -281,17 +335,7 @@ const LlmLlamacpp = class LlmLlamacpp {
     _batchHandler;
     _checkpointSaveDir;
     constructor({ files, config, logger = null, opts = {} }) {
-        if (!files || !Array.isArray(files.model) || files.model.length === 0) {
-            throw new TypeError("files.model must be a non-empty array of absolute paths");
-        }
-        for (const [i, entry] of files.model.entries()) {
-            if (typeof entry !== "string" || entry.length === 0) {
-                throw new TypeError(`files.model[${i}] must be an absolute path string`);
-            }
-            if (!path.isAbsolute(entry)) {
-                throw new TypeError(`files.model[${i}] must be an absolute path (got: ${entry})`);
-            }
-        }
+        this._modelPath = validateModelPath(files?.model);
         if (files.projectionModel !== undefined) {
             if (typeof files.projectionModel !== "string" || files.projectionModel.length === 0) {
                 throw new TypeError("files.projectionModel must be an absolute path string");
@@ -300,7 +344,6 @@ const LlmLlamacpp = class LlmLlamacpp {
                 throw new TypeError(`files.projectionModel must be an absolute path (got: ${files.projectionModel})`);
             }
         }
-        this._files = files.model;
         this._projectionModelPath = files.projectionModel || "";
         this._config = config;
         this.logger = new QvacLogger(logger);
@@ -361,18 +404,14 @@ const LlmLlamacpp = class LlmLlamacpp {
     }
     async _load() {
         this.logger.info("Starting model load");
-        const primaryGgufPath = pickPrimaryGgufPath(this._files);
         const configurationParams = {
-            path: primaryGgufPath,
+            path: this._modelPath,
             projectionPath: this._projectionModelPath,
             config: { ...this._config },
         };
         this.logger.info("Creating addon with configuration:", configurationParams);
         try {
             this.addon = this._createAddon(configurationParams);
-            if (this._files.length > 1) {
-                await this._streamShards();
-            }
             this.logger.info("Activating addon");
             await this.addon.activate();
         }
@@ -389,17 +428,6 @@ const LlmLlamacpp = class LlmLlamacpp {
         }
         this.logger.info("Model load completed successfully");
     }
-    async _streamShards() {
-        for (const filePath of this._files) {
-            const filename = path.basename(filePath);
-            const stream = fs.createReadStream(filePath);
-            for await (const chunk of stream) {
-                await this.addon.loadWeights({ filename, chunk, completed: false });
-            }
-            await this.addon.loadWeights({ filename, chunk: null, completed: true });
-            this.logger.info(`Streamed weights for ${filename}`);
-        }
-    }
     run(prompt, runOptions) {
         if (BatchHandler.isBatchInput(prompt)) {
             if (runOptions !== undefined) {
@@ -863,9 +891,20 @@ const LlmLlamacpp = class LlmLlamacpp {
         if (text !== undefined)
             params.text = text;
         // Media bytes are dropped rather than serialized: the native side rejects
-        // any media message by its type.
+        // any media message or ordered media part by its type.
         if (history !== undefined) {
-            params.history = JSON.stringify(history.map((message) => message?.type === "media" ? { ...message, content: "" } : message));
+            params.history = JSON.stringify(history.map((message) => {
+                if (message?.type === "media")
+                    return { ...message, content: "" };
+                const textMessage = message;
+                if (Array.isArray(textMessage?.content)) {
+                    return {
+                        ...message,
+                        content: textMessage.content.map((part) => isByteMediaPart(part) ? { type: "media" } : part),
+                    };
+                }
+                return message;
+            }));
         }
         if (tools !== undefined)
             params.tools = JSON.stringify(tools);
@@ -877,7 +916,7 @@ const LlmLlamacpp = class LlmLlamacpp {
 };
 // Runtime-redundant: ESM named imports need the top-level `module.exports.X =` form.
 /* eslint-disable @typescript-eslint/no-unsafe-member-access -- `module.exports` is untyped CommonJS surface. */
-module.exports.pickPrimaryGgufPath = pickPrimaryGgufPath;
+module.exports.validateModelPath = validateModelPath;
 module.exports.assessFit = fit_1.assessFit;
 module.exports.QvacResponse = infer_base_1.QvacResponse;
 module.exports = LlmLlamacpp;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/llm-llamacpp",
-  "version": "0.66.0",
+  "version": "0.68.0",
   "description": "llama addon for qvac",
   "addon": true,
   "scripts": {
@@ -79,8 +79,8 @@
   "bugs": "https://github.com/tetherto/qvac/issues",
   "homepage": "https://qvac.tether.io",
   "devDependencies": {
-    "@qvac/fabric-android-arm64": "^0.25.0",
-    "@qvac/fabric-ios": "^0.25.0",
+    "@qvac/fabric-android-arm64": "^0.26.0",
+    "@qvac/fabric-ios": "^0.26.0",
     "@types/node": "^24.2.1",
     "bare-crypto": "^1.15.3",
     "bare-process": "^4.2.2",
@@ -102,22 +102,22 @@
     "typescript-eslint": "^8.63.0"
   },
   "dependencies": {
-    "@qvac/fabric": "^0.25.0",
+    "@qvac/fabric": "^0.26.0",
     "@qvac/infer-base": "^0.8.0",
     "@qvac/logging": "^0.1.0",
     "bare-fs": "^4.5.1",
     "bare-path": "^3.0.0"
   },
   "optionalDependencies": {
-    "@qvac/fabric-linux-x64": "^0.25.0",
-    "@qvac/fabric-linux-arm64": "^0.25.0",
-    "@qvac/fabric-darwin-arm64": "^0.25.0",
-    "@qvac/fabric-darwin-x64": "^0.25.0",
-    "@qvac/fabric-win32-x64": "^0.25.0"
+    "@qvac/fabric-linux-x64": "^0.26.0",
+    "@qvac/fabric-linux-arm64": "^0.26.0",
+    "@qvac/fabric-darwin-arm64": "^0.26.0",
+    "@qvac/fabric-darwin-x64": "^0.26.0",
+    "@qvac/fabric-win32-x64": "^0.26.0"
   },
   "peerDependencies": {
-    "@qvac/fabric-android-arm64": "^0.25.0",
-    "@qvac/fabric-ios": "^0.25.0"
+    "@qvac/fabric-android-arm64": "^0.26.0",
+    "@qvac/fabric-ios": "^0.26.0"
   },
   "peerDependenciesMeta": {
     "@qvac/fabric-android-arm64": {