@qvac/llm-llamacpp 0.64.0 → 0.64.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (18) hide show
  1. package/README.md +9 -3
  2. package/index.d.ts +8 -6
  3. package/package.json +1 -1
  4. package/prebuilds/android-arm64/qvac__llm-llamacpp.bare +0 -0
  5. package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare +0 -0
  6. package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports +14 -7
  7. package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare +0 -0
  8. package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare.exports +10 -3
  9. package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare +0 -0
  10. package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare.exports +14 -7
  11. package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare +0 -0
  12. package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare.exports +14 -7
  13. package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare +0 -0
  14. package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare.exports +10 -3
  15. package/prebuilds/linux-arm64/qvac__llm-llamacpp.bare +0 -0
  16. package/prebuilds/linux-x64/qvac__llm-llamacpp.bare +0 -0
  17. package/prebuilds/win32-x64/qvac__llm-llamacpp.bare +0 -0
  18. package/prebuilds/win32-x64/qvac__llm-llamacpp.bare.exports +0 -0
package/README.md CHANGED
@@ -212,7 +212,7 @@ const config = {
212
212
  | cache-type-v | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, … | auto (see below) | KV-cache **value** quantization type. Quantizing V requires `flash-attn` on |
213
213
  | mmproj-use-gpu | `"true"`/`"on"`/`"1"` or `"false"`/`"off"`/`"0"` | auto (see below) | Run the multimodal projector (mmproj / vision encoder) on the GPU. Only honoured when a GPU backend is selected (ignored with a warning on CPU / GPU-fallback). Unset = auto-default (see mmproj backend below) |
214
214
  | spec-type | `"none"`, `"draft-mtp"` or `"draft-dflash"` | `"none"` | Speculative decoding, as llama-server's `--spec-type`: `draft-mtp` drafts with the model's own multi-token-prediction head, `draft-dflash` with the DFlash model in `spec-draft-model` (see [Speculative decoding](#speculative-decoding-mtp-dflash) below). Also accepted as `spec_type` |
215
- | spec-draft-model | absolute path | – | DFlash draft model, as llama-server's `--spec-draft-model` (`-md`). Required by, and only accepted with, `spec-type: "draft-dflash"`. Placed like the model (`device`, `gpu-layers`). Also accepted as `spec_draft_model` |
215
+ | spec-draft-model | absolute path | – | Draft model, as llama-server's `--spec-draft-model` (`-md`): the DFlash model, required by `spec-type: "draft-dflash"`, or an external MTP head for `spec-type: "draft-mtp"`. Placed like the model (`device`, `gpu-layers`). Also accepted as `spec_draft_model` |
216
216
  | spec-draft-n-max | 1 – 64 | 3 | Longest draft per step. Also accepted as `spec_draft_n_max` |
217
217
  | spec-draft-n-min | 0 – `spec-draft-n-max` | 0 | Drafts shorter than this are not verified. Also accepted as `spec_draft_n_min` |
218
218
  | spec-draft-p-min | 0 – 1 | 0 | Minimum draft-head probability for a token to extend the draft. Also accepted as `spec_draft_p_min` |
@@ -263,11 +263,13 @@ projector runs on CPU. The resolved choice is logged at verbosity ≥ 2 as
263
263
 
264
264
  #### Speculative decoding (MTP, DFlash)
265
265
 
266
- `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
266
+ `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option unless an external head is given, see below). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
267
+
268
+ For a model without MTP layers, give an external MTP head as `spec-draft-model` (llama-server's `--spec-type draft-mtp -md <head>`): a GGUF with MTP layers for the same model, such as the MTP quantization of it, or a Gemma 4 assistant for its Gemma 4 model. The head is loaded next to the model and placed like it (`device`, `gpu-layers`), and the load's fit and `assessFit` count its weights as well as its context. A head without MTP layers fails the load, and `assessFit` refuses it with `unsupported-config`. A head for another model, with a different hidden width or vocabulary, also fails the load, and `assessFit` refuses it the same way. Fabric cannot measure a Gemma 4 assistant on its own, so the fits leave it out, as for DFlash.
267
269
 
268
270
  `spec-type: 'draft-dflash'` drafts with a separate DFlash model instead, given as `spec-draft-model` (an absolute path; llama-server's `-md`). The DFlash model reads the hidden states of the target layers it was trained on and denoises a whole block of draft tokens in one pass, so a step drafts up to its block size minus one tokens (`spec-draft-n-max` caps it further). It is loaded next to the model and placed like it (`device`, `gpu-layers`). The load's fit and `assessFit` pass it to fabric as an extra model, but fabric cannot measure a DFlash model yet: it logs `failed to measure the memory of the extra model, fitting without it`, so neither counts the draft model's memory. Leave room for it yourself. Everything else (verification, rollback, batching, caches, stats) is the same as for MTP.
269
271
 
270
- Both work on single prompts, multimodal prompts and continuous batching (`parallel >= 2`), together with `cacheKey` conversations: the draft context's state travels with checkpoints and the RAM tier, and a `cacheKey` file (target state only, like llama-server's slot files) restarts drafting from the next decoded token.
272
+ MTP and DFlash work on single prompts, multimodal prompts and continuous batching (`parallel >= 2`), together with `cacheKey` conversations: the draft context's state travels with checkpoints and the RAM tier, and a `cacheKey` file (target state only, like llama-server's slot files) restarts drafting from the next decoded token.
271
273
 
272
274
  - Runtime stats gain `draftTokens` and `draftAcceptedTokens` (llama-server's `draft_n` / `draft_n_accepted`); per-position acceptance rates are logged at debug level.
273
275
  - `spec-draft-n-max` is lowered at load, with a warning, so the sampled token and the draft fit one `ubatch-size` and each sequence's share of `batch-size` (`batch-size / parallel`); a batch too small for even one draft token fails the load.
@@ -341,6 +343,8 @@ const response = await model.run([
341
343
 
342
344
  `arguments` is an object or the text of a JSON object, and `content` may be empty on the assistant turn. A malformed tool turn is rejected with `InvalidInputFormat`. [`examples/toolCalling.js`](./examples/toolCalling.js) shows a full round trip.
343
345
 
346
+ The tool grammar holds the model to the order of `parameters`. Inside an object it accepts the required properties first, in declared order, then the optional ones, also in declared order. When the model writes a property out of that order, the call isn't rejected with an error: the model drops a field, writes a broken value or never finishes the call. Declare `required` properties before optional ones at every level, and declare each group in the order the model is likely to write it. This applies to nested objects, and to every argument in formats that write the arguments as one JSON object (Qwen3). Qwen3.5's top-level `<parameter=...>` arguments are less strict: up to six required arguments in any order, then the optional ones in any order. An optional argument before a required one is still rejected there.
347
+
344
348
  #### Batch inference
345
349
 
346
350
  Load the model with `parallel >= 2`, then pass an array of prompts. Each chunk arrives tagged with the id of the sequence that produced it; `await()` returns results in input order.
@@ -451,6 +455,8 @@ fit.hostBytes // the same, for the trailing host row
451
455
 
452
456
  `config` is the load, exactly as `loadModel` takes it. The projection resolves it with the same code that load runs, so every setting reaches the fitter the way it would reach the engine, and a load the engine would refuse is refused here as `status: "error"` with `unsupported-config`. A setting left out takes llama's default.
453
457
 
458
+ With `spec-type: 'draft-mtp'` the MTP draft context is fitted next to the model, as the load fits it, and its memory is in the verdict and in the per-device figures; on a model or head without MTP layers the load fails, so the projection is refused with `unsupported-config`. A DFlash draft model or a Gemma 4 assistant is passed to the fit the same way, but fabric cannot measure either yet, so neither the verdict nor the figures count it (see [Speculative decoding](#speculative-decoding-mtp-dflash)).
459
+
454
460
  Pinning `gpu-layers` fixes the placement: the fitter refuses to move layers off a device the load claimed, so the answer is whether that exact placement fits, never a reduced one. Leave it unset for a projection that can place the model itself.
455
461
 
456
462
  `minCtxSize` sets a floor the fitter may not reduce the context below, defaulting to llama's own `fit-ctx` of 4096. `marginBytes` is the memory to leave free on every device, defaulting to llama's `fit-target` of 1 GiB per device. `backendsDir` is where the dynamically-loaded ggml backends live.
package/index.d.ts CHANGED
@@ -366,9 +366,10 @@ declare namespace LlmLlamacpp {
366
366
  cache_ram_mib?: NumericLike;
367
367
  /**
368
368
  * Speculative decoding, as llama-server's `--spec-type`. `'draft-mtp'`
369
- * drafts with the model's own multi-token-prediction (MTP / NextN) head;
370
- * the model must ship its MTP layers (`blk.N.nextn.*`), or the load
371
- * fails. `'draft-dflash'` drafts with the DFlash model given as
369
+ * drafts with the model's own multi-token-prediction (MTP / NextN) head,
370
+ * or with the external MTP head given as `spec_draft_model`; the model or
371
+ * the head must ship MTP layers (`blk.N.nextn.*`), or the load fails.
372
+ * `'draft-dflash'` drafts with the DFlash model given as
372
373
  * `spec_draft_model`. Either way the model verifies the draft, so the
373
374
  * output keeps the distribution of plain decoding. `'none'` (default)
374
375
  * decodes one token per step. Works on single prompts, multimodal
@@ -376,9 +377,10 @@ declare namespace LlmLlamacpp {
376
377
  */
377
378
  spec_type?: "none" | "draft-mtp" | "draft-dflash";
378
379
  /**
379
- * Absolute path of the DFlash draft model, as llama-server's
380
- * `--spec-draft-model` (`-md`). Required by, and only accepted with,
381
- * `spec_type: 'draft-dflash'`. The draft model is placed like the model
380
+ * Absolute path of the draft model, as llama-server's
381
+ * `--spec-draft-model` (`-md`): the DFlash model, required by
382
+ * `spec_type: 'draft-dflash'`, or an external MTP head for
383
+ * `spec_type: 'draft-mtp'`. The draft model is placed like the model
382
384
  * (`device`, `gpu_layers`). Also accepted as `spec-draft-model`.
383
385
  */
384
386
  spec_draft_model?: string;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qvac/llm-llamacpp",
3
- "version": "0.64.0",
3
+ "version": "0.64.2",
4
4
  "description": "llama addon for qvac",
5
5
  "addon": true,
6
6
  "scripts": {
package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -191,6 +193,7 @@
191
193
  "__ZN14MtmdLlmContext35refreshCurrentCacheTokensFromMemoryEv",
192
194
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
193
195
  "__ZN30qvac_lib_inference_addon_llama8batching7RequestC1EjONSt3__16vectorIiNS2_9allocatorIiEEEEji",
196
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
194
197
  "__ZN30qvac_lib_inference_addon_llama8batching14timeDecodeStepEP13llama_contextR11llama_batchRKNSt3__18functionIFiS2_S4_EEERKNS6_IFvS2_EEE",
195
198
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher20completeMediaBarrierEjiRKNSt3__18functionIFvjimEEE",
196
199
  "__ZN14TextLlmContext17removeLastNTokensEi",
@@ -233,6 +236,7 @@
233
236
  "__ZNK30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot22prefillTokensPerSecondEv",
234
237
  "__ZN14TextLlmContext15checkAntipromptEmi",
235
238
  "__ZN10LlamaModel13setInitLoaderENSt3__18optionalIN10InitLoader11LOADER_TYPEEEENS1_I23FinetuneConfigOverridesEE",
239
+ "__ZN30qvac_lib_inference_addon_llama5utils21renderWithoutUserTurnEPK21common_chat_templatesRK28common_chat_templates_inputsRKNSt3__18functionIFNS7_6vectorIiNS7_9allocatorIiEEEERKNS7_12basic_stringIcNS7_11char_traitsIcEENSA_IcEEEEEEE",
236
240
  "__ZThn48_N14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
237
241
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request13tokenToFeedAtEi",
238
242
  "__ZNK14MtmdLlmContext19residentStateTokensEv",
@@ -380,6 +384,7 @@
380
384
  "__ZThn48_NK14MtmdLlmContext8getNPastEv",
381
385
  "__ZN14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
382
386
  "__ZNK12CacheManager15isCacheDisabledEv",
387
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
383
388
  "__ZNK14MtmdLlmContext14getCacheTokensEv",
384
389
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot5clearEv",
385
390
  "__ZN14TextLlmContext15reconcilePromptERKNSt3__16vectorIiNS0_9allocatorIiEEEEb",
@@ -502,14 +507,14 @@
502
507
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
503
508
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
504
509
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
505
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
510
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
506
511
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
507
512
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
508
513
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
509
514
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
510
515
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
511
516
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
512
- "__ZNK13ModelMetaData12tryGetStringEPKc",
517
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
513
518
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
514
519
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
515
520
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -540,10 +545,11 @@
540
545
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
541
546
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
542
547
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
543
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
544
549
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
545
550
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
546
551
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
552
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
547
553
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
548
554
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
549
555
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
@@ -675,8 +681,9 @@
675
681
  "__ZThn40_N10LlamaModelD0Ev",
676
682
  "__ZNK14TextLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
677
683
  "__ZThn48_NK14MtmdLlmContext28appendCheckpointsToCacheFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
678
- "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher6cancelEjRKNSt3__18functionIFvjEEE",
684
+ "__ZN30qvac_lib_inference_addon_llama5utils11hasUserTurnERKNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEE",
679
685
  "__ZN30qvac_lib_inference_addon_llama7parsers26parseReasoningBudgetConfigENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
686
+ "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher6cancelEjRKNSt3__18functionIFvjEEE",
680
687
  "__ZN30qvac_lib_inference_addon_llama8batching22aggregateObservedStatsERKNSt3__16vectorINS0_20ObservedRequestStatsENS1_9allocatorIS3_EEEE",
681
688
  "__ZN14MtmdLlmContext11evalMessageERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEEbb",
682
689
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23cancelKeyDeferredLockedEv",
package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -426,13 +426,12 @@
426
426
  "__ZTSNSt3__110__function6__baseIFvbEEE",
427
427
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_sEUlvE_EEP10js_value_sS8_OT_EUlS8_RKNS_9monostateEE_NS_9allocatorISJ_EEFSD_S8_SI_EEE",
428
428
  "__ZN30qvac_lib_inference_addon_llama5cache24CHECKPOINT_SECTION_MAGICE",
429
- "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
430
429
  "__ZTIN13BorrowablePtrINSt3__115basic_streambufIcNS0_11char_traitsIcEEEEE7DeleterE",
431
430
  "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output6LogMsgEE4__idE",
432
431
  "__ZTSNSt3__110__function6__baseIFNS_9monostateEvEEE",
433
432
  "__ZNSt3__19__any_imp17__unique_typeinfoI22FinetuneTerminalResultE4__idE",
434
433
  "__ZTINSt3__120__shared_ptr_emplaceINS_6atomicIbEENS_9allocatorIS2_EEEE",
435
- "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
434
+ "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
436
435
  "__ZTIN10moodycamel15ConcurrentQueueIN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler13QueuedRequestENS_28ConcurrentQueueDefaultTraitsEE12ProducerBaseE",
437
436
  "__ZTINSt3__120__shared_ptr_emplaceIN30qvac_lib_inference_addon_llama8batching14SlotStateCacheENS_9allocatorIS3_EEEE",
438
437
  "__ZTSNSt3__116__owns_one_stateIcEE",
@@ -442,6 +441,7 @@
442
441
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp17makeQueueCallbackINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEENS_8functionIFvRKT_EEENS_10shared_ptrINS2_11OutputQueueEEENS_13shared_futureIyEEEUlRKS9_E_NS7_ISN_EEFvSM_EEE",
443
442
  "__ZTSNSt3__110__function6__baseIFN22load_fit_normalization15SelectedBackendEN17backend_selection11BackendTypeERKNS_8optionalINS_7variantIJiNS4_11MainGpuTypeEEEEEERK13ModelMetaDatabRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENSK_ISM_EEEEEEE",
444
443
  "__ZTSNSt3__112__match_charIcEE",
444
+ "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
445
445
  "__ZTSNSt3__110__function6__baseIFNS_10unique_ptrI14SequenceDriverNS_14default_deleteIS3_EEEERK13common_paramsjiEEE",
446
446
  "__ZTSNSt3__110__function6__baseIFNS_13unordered_setINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS8_EENS_8equal_toIS8_EENS6_IS8_EEEEvEEE",
447
447
  "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerIN30qvac_lib_inference_addon_llama16BatchTokenOutputEEE",
@@ -545,6 +545,7 @@
545
545
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
546
546
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
547
547
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
548
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
548
549
  "__ZN12CacheManager14flushForUnloadEv",
549
550
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
550
551
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
@@ -561,6 +562,7 @@
561
562
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
562
563
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
563
564
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
565
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
564
566
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
565
567
  "__ZN10LlamaModel8getModelEv",
566
568
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler16cancelSlotLockedEjNS1_15SaveCachePolicyE",
@@ -576,6 +578,7 @@
576
578
  "__ZN14MtmdLlmContext17removeLastNTokensEi",
577
579
  "__ZN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeD2Ev",
578
580
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
581
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
579
582
  "__ZN14MtmdLlmContext10resetMediaEv",
580
583
  "__ZN30qvac_lib_inference_addon_llama8batching22finalizeTerminalDriverER14SequenceDriverNS0_10StopReasonEbRKNSt3__18functionIFvRKNS4_12basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEEE",
581
584
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot10adoptEmptyEi",
@@ -665,6 +668,7 @@
665
668
  "__ZN14TextLlmContextC2ER13common_paramsNSt3__110unique_ptrI18common_init_resultNS2_14default_deleteIS4_EEEENS3_IN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeENS5_ISA_EEEE",
666
669
  "__ZN14TextLlmContext17beginCacheRequestEv",
667
670
  "__ZNK12CacheManager15isCacheDisabledEv",
671
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
668
672
  "__ZThn48_NK14TextLlmContext19residentStateTokensEv",
669
673
  "__ZN14MtmdLlmContext18commitCacheRequestEv",
670
674
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23recordPendingSlotCancelEjy",
@@ -896,6 +900,7 @@
896
900
  "__ZNK10LlamaModel23batchRuntimeStatsLockedEv",
897
901
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15failGroupLockedERKNSt3__110shared_ptrINS1_10BatchGroupEEESt13exception_ptrNS1_15SaveCachePolicyE",
898
902
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchSchedulerC1E15LlmModelContextjjmiRK13common_paramsNSt3__18functionIFNS6_10unique_ptrI14SequenceDriverNS6_14default_deleteIS9_EEEES5_jiEEE",
903
+ "__ZN30qvac_lib_inference_addon_llama5utils21renderWithoutUserTurnEPK21common_chat_templatesRK28common_chat_templates_inputsRKNSt3__18functionIFNS7_6vectorIiNS7_9allocatorIiEEEERKNS7_12basic_stringIcNS7_11char_traitsIcEENSA_IcEEEEEEE",
899
904
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request21chunkConsumesAllUnfedEj",
900
905
  "__ZN10LlamaModel7processERKNSt3__13anyE",
901
906
  "__ZThn48_N14MtmdLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
@@ -1035,6 +1040,7 @@
1035
1040
  "__ZN30qvac_lib_inference_addon_llama26applyFinetuneParamHandlersEP8js_env_sRN28qvac_lib_inference_addon_cpp2js6ObjectERNS_21LlamaFinetuningParamsE",
1036
1041
  "__ZNK14MtmdLlmContext16ledgerFromChunksERKN4mtmd12input_chunksE",
1037
1042
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
1043
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
1038
1044
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
1039
1045
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
1040
1046
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
@@ -1127,6 +1133,7 @@
1127
1133
  "__ZN14TextLlmContext29clearCacheReconciliationStateEv",
1128
1134
  "__ZN14MtmdLlmContext18adoptResidentStateERKNSt3__16vectorIiNS0_9allocatorIiEEEE",
1129
1135
  "__ZThn40_N10LlamaModelD0Ev",
1136
+ "__ZN30qvac_lib_inference_addon_llama5utils11hasUserTurnERKNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEE",
1130
1137
  "__ZN30qvac_lib_inference_addon_llama8batching22aggregateObservedStatsERKNSt3__16vectorINS0_20ObservedRequestStatsENS1_9allocatorIS3_EEEE",
1131
1138
  "__ZN14MtmdLlmContext11evalMessageERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEEbb",
1132
1139
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23cancelKeyDeferredLockedEv",
package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -191,6 +193,7 @@
191
193
  "__ZN14MtmdLlmContext35refreshCurrentCacheTokensFromMemoryEv",
192
194
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
193
195
  "__ZN30qvac_lib_inference_addon_llama8batching7RequestC1EjONSt3__16vectorIiNS2_9allocatorIiEEEEji",
196
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
194
197
  "__ZN30qvac_lib_inference_addon_llama8batching14timeDecodeStepEP13llama_contextR11llama_batchRKNSt3__18functionIFiS2_S4_EEERKNS6_IFvS2_EEE",
195
198
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher20completeMediaBarrierEjiRKNSt3__18functionIFvjimEEE",
196
199
  "__ZN14TextLlmContext17removeLastNTokensEi",
@@ -233,6 +236,7 @@
233
236
  "__ZNK30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot22prefillTokensPerSecondEv",
234
237
  "__ZN14TextLlmContext15checkAntipromptEmi",
235
238
  "__ZN10LlamaModel13setInitLoaderENSt3__18optionalIN10InitLoader11LOADER_TYPEEEENS1_I23FinetuneConfigOverridesEE",
239
+ "__ZN30qvac_lib_inference_addon_llama5utils21renderWithoutUserTurnEPK21common_chat_templatesRK28common_chat_templates_inputsRKNSt3__18functionIFNS7_6vectorIiNS7_9allocatorIiEEEERKNS7_12basic_stringIcNS7_11char_traitsIcEENSA_IcEEEEEEE",
236
240
  "__ZThn48_N14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
237
241
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request13tokenToFeedAtEi",
238
242
  "__ZNK14MtmdLlmContext19residentStateTokensEv",
@@ -380,6 +384,7 @@
380
384
  "__ZThn48_NK14MtmdLlmContext8getNPastEv",
381
385
  "__ZN14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
382
386
  "__ZNK12CacheManager15isCacheDisabledEv",
387
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
383
388
  "__ZNK14MtmdLlmContext14getCacheTokensEv",
384
389
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot5clearEv",
385
390
  "__ZN14TextLlmContext15reconcilePromptERKNSt3__16vectorIiNS0_9allocatorIiEEEEb",
@@ -502,14 +507,14 @@
502
507
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
503
508
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
504
509
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
505
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
510
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
506
511
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
507
512
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
508
513
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
509
514
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
510
515
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
511
516
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
512
- "__ZNK13ModelMetaData12tryGetStringEPKc",
517
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
513
518
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
514
519
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
515
520
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -540,10 +545,11 @@
540
545
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
541
546
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
542
547
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
543
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
544
549
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
545
550
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
546
551
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
552
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
547
553
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
548
554
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
549
555
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
@@ -675,8 +681,9 @@
675
681
  "__ZThn40_N10LlamaModelD0Ev",
676
682
  "__ZNK14TextLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
677
683
  "__ZThn48_NK14MtmdLlmContext28appendCheckpointsToCacheFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
678
- "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher6cancelEjRKNSt3__18functionIFvjEEE",
684
+ "__ZN30qvac_lib_inference_addon_llama5utils11hasUserTurnERKNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEE",
679
685
  "__ZN30qvac_lib_inference_addon_llama7parsers26parseReasoningBudgetConfigENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
686
+ "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher6cancelEjRKNSt3__18functionIFvjEEE",
680
687
  "__ZN30qvac_lib_inference_addon_llama8batching22aggregateObservedStatsERKNSt3__16vectorINS0_20ObservedRequestStatsENS1_9allocatorIS3_EEEE",
681
688
  "__ZN14MtmdLlmContext11evalMessageERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEEbb",
682
689
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23cancelKeyDeferredLockedEv",
package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -191,6 +193,7 @@
191
193
  "__ZN14MtmdLlmContext35refreshCurrentCacheTokensFromMemoryEv",
192
194
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
193
195
  "__ZN30qvac_lib_inference_addon_llama8batching7RequestC1EjONSt3__16vectorIiNS2_9allocatorIiEEEEji",
196
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
194
197
  "__ZN30qvac_lib_inference_addon_llama8batching14timeDecodeStepEP13llama_contextR11llama_batchRKNSt3__18functionIFiS2_S4_EEERKNS6_IFvS2_EEE",
195
198
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher20completeMediaBarrierEjiRKNSt3__18functionIFvjimEEE",
196
199
  "__ZN14TextLlmContext17removeLastNTokensEi",
@@ -233,6 +236,7 @@
233
236
  "__ZNK30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot22prefillTokensPerSecondEv",
234
237
  "__ZN14TextLlmContext15checkAntipromptEmi",
235
238
  "__ZN10LlamaModel13setInitLoaderENSt3__18optionalIN10InitLoader11LOADER_TYPEEEENS1_I23FinetuneConfigOverridesEE",
239
+ "__ZN30qvac_lib_inference_addon_llama5utils21renderWithoutUserTurnEPK21common_chat_templatesRK28common_chat_templates_inputsRKNSt3__18functionIFNS7_6vectorIiNS7_9allocatorIiEEEERKNS7_12basic_stringIcNS7_11char_traitsIcEENSA_IcEEEEEEE",
236
240
  "__ZThn48_N14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
237
241
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request13tokenToFeedAtEi",
238
242
  "__ZNK14MtmdLlmContext19residentStateTokensEv",
@@ -380,6 +384,7 @@
380
384
  "__ZThn48_NK14MtmdLlmContext8getNPastEv",
381
385
  "__ZN14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
382
386
  "__ZNK12CacheManager15isCacheDisabledEv",
387
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
383
388
  "__ZNK14MtmdLlmContext14getCacheTokensEv",
384
389
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot5clearEv",
385
390
  "__ZN14TextLlmContext15reconcilePromptERKNSt3__16vectorIiNS0_9allocatorIiEEEEb",
@@ -502,14 +507,14 @@
502
507
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
503
508
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
504
509
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
505
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
510
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
506
511
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
507
512
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
508
513
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
509
514
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
510
515
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
511
516
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
512
- "__ZNK13ModelMetaData12tryGetStringEPKc",
517
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
513
518
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
514
519
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
515
520
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -540,10 +545,11 @@
540
545
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
541
546
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
542
547
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
543
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
544
549
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
545
550
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
546
551
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
552
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
547
553
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
548
554
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
549
555
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
@@ -675,8 +681,9 @@
675
681
  "__ZThn40_N10LlamaModelD0Ev",
676
682
  "__ZNK14TextLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
677
683
  "__ZThn48_NK14MtmdLlmContext28appendCheckpointsToCacheFileERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
678
- "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher6cancelEjRKNSt3__18functionIFvjEEE",
684
+ "__ZN30qvac_lib_inference_addon_llama5utils11hasUserTurnERKNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEE",
679
685
  "__ZN30qvac_lib_inference_addon_llama7parsers26parseReasoningBudgetConfigENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
686
+ "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher6cancelEjRKNSt3__18functionIFvjEEE",
680
687
  "__ZN30qvac_lib_inference_addon_llama8batching22aggregateObservedStatsERKNSt3__16vectorINS0_20ObservedRequestStatsENS1_9allocatorIS3_EEEE",
681
688
  "__ZN14MtmdLlmContext11evalMessageERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEEbb",
682
689
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23cancelKeyDeferredLockedEv",
package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -438,13 +438,12 @@
438
438
  "__ZTSNSt3__110__function6__baseIFvbEEE",
439
439
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_sEUlvE_EEP10js_value_sS8_OT_EUlS8_RKNS_9monostateEE_NS_9allocatorISJ_EEFSD_S8_SI_EEE",
440
440
  "__ZN30qvac_lib_inference_addon_llama5cache24CHECKPOINT_SECTION_MAGICE",
441
- "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
442
441
  "__ZTIN13BorrowablePtrINSt3__115basic_streambufIcNS0_11char_traitsIcEEEEE7DeleterE",
443
442
  "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output6LogMsgEE4__idE",
444
443
  "__ZTSNSt3__110__function6__baseIFNS_9monostateEvEEE",
445
444
  "__ZNSt3__19__any_imp17__unique_typeinfoI22FinetuneTerminalResultE4__idE",
446
445
  "__ZTINSt3__120__shared_ptr_emplaceINS_6atomicIbEENS_9allocatorIS2_EEEE",
447
- "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
446
+ "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
448
447
  "__ZTIN10moodycamel15ConcurrentQueueIN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler13QueuedRequestENS_28ConcurrentQueueDefaultTraitsEE12ProducerBaseE",
449
448
  "__ZTINSt3__120__shared_ptr_emplaceIN30qvac_lib_inference_addon_llama8batching14SlotStateCacheENS_9allocatorIS3_EEEE",
450
449
  "__ZTSNSt3__116__owns_one_stateIcEE",
@@ -454,6 +453,7 @@
454
453
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp17makeQueueCallbackINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEENS_8functionIFvRKT_EEENS_10shared_ptrINS2_11OutputQueueEEENS_13shared_futureIyEEEUlRKS9_E_NS7_ISN_EEFvSM_EEE",
455
454
  "__ZTSNSt3__110__function6__baseIFN22load_fit_normalization15SelectedBackendEN17backend_selection11BackendTypeERKNS_8optionalINS_7variantIJiNS4_11MainGpuTypeEEEEEERK13ModelMetaDatabRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENSK_ISM_EEEEEEE",
456
455
  "__ZTSNSt3__112__match_charIcEE",
456
+ "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
457
457
  "__ZTSNSt3__110__function6__baseIFNS_10unique_ptrI14SequenceDriverNS_14default_deleteIS3_EEEERK13common_paramsjiEEE",
458
458
  "__ZTSNSt3__110__function6__baseIFNS_13unordered_setINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS8_EENS_8equal_toIS8_EENS6_IS8_EEEEvEEE",
459
459
  "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerIN30qvac_lib_inference_addon_llama16BatchTokenOutputEEE",
@@ -558,6 +558,7 @@
558
558
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
559
559
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
560
560
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
561
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
561
562
  "__ZN12CacheManager14flushForUnloadEv",
562
563
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
563
564
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
@@ -574,6 +575,7 @@
574
575
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
575
576
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
576
577
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
578
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
577
579
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
578
580
  "__ZN10LlamaModel8getModelEv",
579
581
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler16cancelSlotLockedEjNS1_15SaveCachePolicyE",
@@ -589,6 +591,7 @@
589
591
  "__ZN14MtmdLlmContext17removeLastNTokensEi",
590
592
  "__ZN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeD2Ev",
591
593
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
594
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
592
595
  "__ZN14MtmdLlmContext10resetMediaEv",
593
596
  "__ZN30qvac_lib_inference_addon_llama8batching22finalizeTerminalDriverER14SequenceDriverNS0_10StopReasonEbRKNSt3__18functionIFvRKNS4_12basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEEE",
594
597
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot10adoptEmptyEi",
@@ -678,6 +681,7 @@
678
681
  "__ZN14TextLlmContextC2ER13common_paramsNSt3__110unique_ptrI18common_init_resultNS2_14default_deleteIS4_EEEENS3_IN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeENS5_ISA_EEEE",
679
682
  "__ZN14TextLlmContext17beginCacheRequestEv",
680
683
  "__ZNK12CacheManager15isCacheDisabledEv",
684
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
681
685
  "__ZThn48_NK14TextLlmContext19residentStateTokensEv",
682
686
  "__ZN14MtmdLlmContext18commitCacheRequestEv",
683
687
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23recordPendingSlotCancelEjy",
@@ -909,6 +913,7 @@
909
913
  "__ZNK10LlamaModel23batchRuntimeStatsLockedEv",
910
914
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15failGroupLockedERKNSt3__110shared_ptrINS1_10BatchGroupEEESt13exception_ptrNS1_15SaveCachePolicyE",
911
915
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchSchedulerC1E15LlmModelContextjjmiRK13common_paramsNSt3__18functionIFNS6_10unique_ptrI14SequenceDriverNS6_14default_deleteIS9_EEEES5_jiEEE",
916
+ "__ZN30qvac_lib_inference_addon_llama5utils21renderWithoutUserTurnEPK21common_chat_templatesRK28common_chat_templates_inputsRKNSt3__18functionIFNS7_6vectorIiNS7_9allocatorIiEEEERKNS7_12basic_stringIcNS7_11char_traitsIcEENSA_IcEEEEEEE",
912
917
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request21chunkConsumesAllUnfedEj",
913
918
  "__ZN10LlamaModel7processERKNSt3__13anyE",
914
919
  "__ZThn48_N14MtmdLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
@@ -1048,6 +1053,7 @@
1048
1053
  "__ZN30qvac_lib_inference_addon_llama26applyFinetuneParamHandlersEP8js_env_sRN28qvac_lib_inference_addon_cpp2js6ObjectERNS_21LlamaFinetuningParamsE",
1049
1054
  "__ZNK14MtmdLlmContext16ledgerFromChunksERKN4mtmd12input_chunksE",
1050
1055
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
1056
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
1051
1057
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
1052
1058
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
1053
1059
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
@@ -1140,6 +1146,7 @@
1140
1146
  "__ZN14TextLlmContext29clearCacheReconciliationStateEv",
1141
1147
  "__ZN14MtmdLlmContext18adoptResidentStateERKNSt3__16vectorIiNS0_9allocatorIiEEEE",
1142
1148
  "__ZThn40_N10LlamaModelD0Ev",
1149
+ "__ZN30qvac_lib_inference_addon_llama5utils11hasUserTurnERKNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEE",
1143
1150
  "__ZN30qvac_lib_inference_addon_llama8batching22aggregateObservedStatsERKNSt3__16vectorINS0_20ObservedRequestStatsENS1_9allocatorIS3_EEEE",
1144
1151
  "__ZN14MtmdLlmContext11evalMessageERKNSt3__16vectorI15common_chat_msgNS0_9allocatorIS2_EEEEbb",
1145
1152
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23cancelKeyDeferredLockedEv",