@qvac/llm-llamacpp 0.64.1 → 0.64.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (18) hide show
  1. package/README.md +7 -3
  2. package/index.d.ts +8 -6
  3. package/package.json +1 -1
  4. package/prebuilds/android-arm64/qvac__llm-llamacpp.bare +0 -0
  5. package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare +0 -0
  6. package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports +11 -6
  7. package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare +0 -0
  8. package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare.exports +8 -3
  9. package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare +0 -0
  10. package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare.exports +11 -6
  11. package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare +0 -0
  12. package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare.exports +11 -6
  13. package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare +0 -0
  14. package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare.exports +8 -3
  15. package/prebuilds/linux-arm64/qvac__llm-llamacpp.bare +0 -0
  16. package/prebuilds/linux-x64/qvac__llm-llamacpp.bare +0 -0
  17. package/prebuilds/win32-x64/qvac__llm-llamacpp.bare +0 -0
  18. package/prebuilds/win32-x64/qvac__llm-llamacpp.bare.exports +0 -0
package/README.md CHANGED
@@ -212,7 +212,7 @@ const config = {
212
212
  | cache-type-v | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, … | auto (see below) | KV-cache **value** quantization type. Quantizing V requires `flash-attn` on |
213
213
  | mmproj-use-gpu | `"true"`/`"on"`/`"1"` or `"false"`/`"off"`/`"0"` | auto (see below) | Run the multimodal projector (mmproj / vision encoder) on the GPU. Only honoured when a GPU backend is selected (ignored with a warning on CPU / GPU-fallback). Unset = auto-default (see mmproj backend below) |
214
214
  | spec-type | `"none"`, `"draft-mtp"` or `"draft-dflash"` | `"none"` | Speculative decoding, as llama-server's `--spec-type`: `draft-mtp` drafts with the model's own multi-token-prediction head, `draft-dflash` with the DFlash model in `spec-draft-model` (see [Speculative decoding](#speculative-decoding-mtp-dflash) below). Also accepted as `spec_type` |
215
- | spec-draft-model | absolute path | – | DFlash draft model, as llama-server's `--spec-draft-model` (`-md`). Required by, and only accepted with, `spec-type: "draft-dflash"`. Placed like the model (`device`, `gpu-layers`). Also accepted as `spec_draft_model` |
215
+ | spec-draft-model | absolute path | – | Draft model, as llama-server's `--spec-draft-model` (`-md`): the DFlash model, required by `spec-type: "draft-dflash"`, or an external MTP head for `spec-type: "draft-mtp"`. Placed like the model (`device`, `gpu-layers`). Also accepted as `spec_draft_model` |
216
216
  | spec-draft-n-max | 1 – 64 | 3 | Longest draft per step. Also accepted as `spec_draft_n_max` |
217
217
  | spec-draft-n-min | 0 – `spec-draft-n-max` | 0 | Drafts shorter than this are not verified. Also accepted as `spec_draft_n_min` |
218
218
  | spec-draft-p-min | 0 – 1 | 0 | Minimum draft-head probability for a token to extend the draft. Also accepted as `spec_draft_p_min` |
@@ -263,11 +263,13 @@ projector runs on CPU. The resolved choice is logged at verbosity ≥ 2 as
263
263
 
264
264
  #### Speculative decoding (MTP, DFlash)
265
265
 
266
- `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
266
+ `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option unless an external head is given, see below). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
267
+
268
+ For a model without MTP layers, give an external MTP head as `spec-draft-model` (llama-server's `--spec-type draft-mtp -md <head>`): a GGUF with MTP layers for the same model, such as the MTP quantization of it, or a Gemma 4 assistant for its Gemma 4 model. The head is loaded next to the model and placed like it (`device`, `gpu-layers`), and the load's fit and `assessFit` count its weights as well as its context. A head without MTP layers fails the load, and `assessFit` refuses it with `unsupported-config`. A head for another model, with a different hidden width or vocabulary, also fails the load, and `assessFit` refuses it the same way. Fabric cannot measure a Gemma 4 assistant on its own, so the fits leave it out, as for DFlash.
267
269
 
268
270
  `spec-type: 'draft-dflash'` drafts with a separate DFlash model instead, given as `spec-draft-model` (an absolute path; llama-server's `-md`). The DFlash model reads the hidden states of the target layers it was trained on and denoises a whole block of draft tokens in one pass, so a step drafts up to its block size minus one tokens (`spec-draft-n-max` caps it further). It is loaded next to the model and placed like it (`device`, `gpu-layers`). The load's fit and `assessFit` pass it to fabric as an extra model, but fabric cannot measure a DFlash model yet: it logs `failed to measure the memory of the extra model, fitting without it`, so neither counts the draft model's memory. Leave room for it yourself. Everything else (verification, rollback, batching, caches, stats) is the same as for MTP.
269
271
 
270
- Both work on single prompts, multimodal prompts and continuous batching (`parallel >= 2`), together with `cacheKey` conversations: the draft context's state travels with checkpoints and the RAM tier, and a `cacheKey` file (target state only, like llama-server's slot files) restarts drafting from the next decoded token.
272
+ MTP and DFlash work on single prompts, multimodal prompts and continuous batching (`parallel >= 2`), together with `cacheKey` conversations: the draft context's state travels with checkpoints and the RAM tier, and a `cacheKey` file (target state only, like llama-server's slot files) restarts drafting from the next decoded token.
271
273
 
272
274
  - Runtime stats gain `draftTokens` and `draftAcceptedTokens` (llama-server's `draft_n` / `draft_n_accepted`); per-position acceptance rates are logged at debug level.
273
275
  - `spec-draft-n-max` is lowered at load, with a warning, so the sampled token and the draft fit one `ubatch-size` and each sequence's share of `batch-size` (`batch-size / parallel`); a batch too small for even one draft token fails the load.
@@ -453,6 +455,8 @@ fit.hostBytes // the same, for the trailing host row
453
455
 
454
456
  `config` is the load, exactly as `loadModel` takes it. The projection resolves it with the same code that load runs, so every setting reaches the fitter the way it would reach the engine, and a load the engine would refuse is refused here as `status: "error"` with `unsupported-config`. A setting left out takes llama's default.
455
457
 
458
+ With `spec-type: 'draft-mtp'` the MTP draft context is fitted next to the model, as the load fits it, and its memory is in the verdict and in the per-device figures; on a model or head without MTP layers the load fails, so the projection is refused with `unsupported-config`. A DFlash draft model or a Gemma 4 assistant is passed to the fit the same way, but fabric cannot measure either yet, so neither the verdict nor the figures count it (see [Speculative decoding](#speculative-decoding-mtp-dflash)).
459
+
456
460
  Pinning `gpu-layers` fixes the placement: the fitter refuses to move layers off a device the load claimed, so the answer is whether that exact placement fits, never a reduced one. Leave it unset for a projection that can place the model itself.
457
461
 
458
462
  `minCtxSize` sets a floor the fitter may not reduce the context below, defaulting to llama's own `fit-ctx` of 4096. `marginBytes` is the memory to leave free on every device, defaulting to llama's `fit-target` of 1 GiB per device. `backendsDir` is where the dynamically-loaded ggml backends live.
package/index.d.ts CHANGED
@@ -366,9 +366,10 @@ declare namespace LlmLlamacpp {
366
366
  cache_ram_mib?: NumericLike;
367
367
  /**
368
368
  * Speculative decoding, as llama-server's `--spec-type`. `'draft-mtp'`
369
- * drafts with the model's own multi-token-prediction (MTP / NextN) head;
370
- * the model must ship its MTP layers (`blk.N.nextn.*`), or the load
371
- * fails. `'draft-dflash'` drafts with the DFlash model given as
369
+ * drafts with the model's own multi-token-prediction (MTP / NextN) head,
370
+ * or with the external MTP head given as `spec_draft_model`; the model or
371
+ * the head must ship MTP layers (`blk.N.nextn.*`), or the load fails.
372
+ * `'draft-dflash'` drafts with the DFlash model given as
372
373
  * `spec_draft_model`. Either way the model verifies the draft, so the
373
374
  * output keeps the distribution of plain decoding. `'none'` (default)
374
375
  * decodes one token per step. Works on single prompts, multimodal
@@ -376,9 +377,10 @@ declare namespace LlmLlamacpp {
376
377
  */
377
378
  spec_type?: "none" | "draft-mtp" | "draft-dflash";
378
379
  /**
379
- * Absolute path of the DFlash draft model, as llama-server's
380
- * `--spec-draft-model` (`-md`). Required by, and only accepted with,
381
- * `spec_type: 'draft-dflash'`. The draft model is placed like the model
380
+ * Absolute path of the draft model, as llama-server's
381
+ * `--spec-draft-model` (`-md`): the DFlash model, required by
382
+ * `spec_type: 'draft-dflash'`, or an external MTP head for
383
+ * `spec_type: 'draft-mtp'`. The draft model is placed like the model
382
384
  * (`device`, `gpu_layers`). Also accepted as `spec-draft-model`.
383
385
  */
384
386
  spec_draft_model?: string;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qvac/llm-llamacpp",
3
- "version": "0.64.1",
3
+ "version": "0.64.2",
4
4
  "description": "llama addon for qvac",
5
5
  "addon": true,
6
6
  "scripts": {
package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -191,6 +193,7 @@
191
193
  "__ZN14MtmdLlmContext35refreshCurrentCacheTokensFromMemoryEv",
192
194
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
193
195
  "__ZN30qvac_lib_inference_addon_llama8batching7RequestC1EjONSt3__16vectorIiNS2_9allocatorIiEEEEji",
196
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
194
197
  "__ZN30qvac_lib_inference_addon_llama8batching14timeDecodeStepEP13llama_contextR11llama_batchRKNSt3__18functionIFiS2_S4_EEERKNS6_IFvS2_EEE",
195
198
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher20completeMediaBarrierEjiRKNSt3__18functionIFvjimEEE",
196
199
  "__ZN14TextLlmContext17removeLastNTokensEi",
@@ -381,6 +384,7 @@
381
384
  "__ZThn48_NK14MtmdLlmContext8getNPastEv",
382
385
  "__ZN14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
383
386
  "__ZNK12CacheManager15isCacheDisabledEv",
387
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
384
388
  "__ZNK14MtmdLlmContext14getCacheTokensEv",
385
389
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot5clearEv",
386
390
  "__ZN14TextLlmContext15reconcilePromptERKNSt3__16vectorIiNS0_9allocatorIiEEEEb",
@@ -503,14 +507,14 @@
503
507
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
504
508
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
505
509
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
506
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
510
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
507
511
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
508
512
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
509
513
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
510
514
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
511
515
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
512
516
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
513
- "__ZNK13ModelMetaData12tryGetStringEPKc",
517
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
514
518
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
515
519
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
516
520
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -541,10 +545,11 @@
541
545
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
542
546
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
543
547
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
544
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
545
549
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
546
550
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
547
551
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
552
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
553
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
549
554
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
550
555
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -426,13 +426,12 @@
426
426
  "__ZTSNSt3__110__function6__baseIFvbEEE",
427
427
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_sEUlvE_EEP10js_value_sS8_OT_EUlS8_RKNS_9monostateEE_NS_9allocatorISJ_EEFSD_S8_SI_EEE",
428
428
  "__ZN30qvac_lib_inference_addon_llama5cache24CHECKPOINT_SECTION_MAGICE",
429
- "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
430
429
  "__ZTIN13BorrowablePtrINSt3__115basic_streambufIcNS0_11char_traitsIcEEEEE7DeleterE",
431
430
  "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output6LogMsgEE4__idE",
432
431
  "__ZTSNSt3__110__function6__baseIFNS_9monostateEvEEE",
433
432
  "__ZNSt3__19__any_imp17__unique_typeinfoI22FinetuneTerminalResultE4__idE",
434
433
  "__ZTINSt3__120__shared_ptr_emplaceINS_6atomicIbEENS_9allocatorIS2_EEEE",
435
- "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
434
+ "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
436
435
  "__ZTIN10moodycamel15ConcurrentQueueIN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler13QueuedRequestENS_28ConcurrentQueueDefaultTraitsEE12ProducerBaseE",
437
436
  "__ZTINSt3__120__shared_ptr_emplaceIN30qvac_lib_inference_addon_llama8batching14SlotStateCacheENS_9allocatorIS3_EEEE",
438
437
  "__ZTSNSt3__116__owns_one_stateIcEE",
@@ -442,6 +441,7 @@
442
441
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp17makeQueueCallbackINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEENS_8functionIFvRKT_EEENS_10shared_ptrINS2_11OutputQueueEEENS_13shared_futureIyEEEUlRKS9_E_NS7_ISN_EEFvSM_EEE",
443
442
  "__ZTSNSt3__110__function6__baseIFN22load_fit_normalization15SelectedBackendEN17backend_selection11BackendTypeERKNS_8optionalINS_7variantIJiNS4_11MainGpuTypeEEEEEERK13ModelMetaDatabRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENSK_ISM_EEEEEEE",
444
443
  "__ZTSNSt3__112__match_charIcEE",
444
+ "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
445
445
  "__ZTSNSt3__110__function6__baseIFNS_10unique_ptrI14SequenceDriverNS_14default_deleteIS3_EEEERK13common_paramsjiEEE",
446
446
  "__ZTSNSt3__110__function6__baseIFNS_13unordered_setINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS8_EENS_8equal_toIS8_EENS6_IS8_EEEEvEEE",
447
447
  "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerIN30qvac_lib_inference_addon_llama16BatchTokenOutputEEE",
@@ -545,6 +545,7 @@
545
545
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
546
546
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
547
547
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
548
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
548
549
  "__ZN12CacheManager14flushForUnloadEv",
549
550
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
550
551
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
@@ -561,6 +562,7 @@
561
562
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
562
563
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
563
564
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
565
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
564
566
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
565
567
  "__ZN10LlamaModel8getModelEv",
566
568
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler16cancelSlotLockedEjNS1_15SaveCachePolicyE",
@@ -576,6 +578,7 @@
576
578
  "__ZN14MtmdLlmContext17removeLastNTokensEi",
577
579
  "__ZN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeD2Ev",
578
580
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
581
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
579
582
  "__ZN14MtmdLlmContext10resetMediaEv",
580
583
  "__ZN30qvac_lib_inference_addon_llama8batching22finalizeTerminalDriverER14SequenceDriverNS0_10StopReasonEbRKNSt3__18functionIFvRKNS4_12basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEEE",
581
584
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot10adoptEmptyEi",
@@ -665,6 +668,7 @@
665
668
  "__ZN14TextLlmContextC2ER13common_paramsNSt3__110unique_ptrI18common_init_resultNS2_14default_deleteIS4_EEEENS3_IN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeENS5_ISA_EEEE",
666
669
  "__ZN14TextLlmContext17beginCacheRequestEv",
667
670
  "__ZNK12CacheManager15isCacheDisabledEv",
671
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
668
672
  "__ZThn48_NK14TextLlmContext19residentStateTokensEv",
669
673
  "__ZN14MtmdLlmContext18commitCacheRequestEv",
670
674
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23recordPendingSlotCancelEjy",
@@ -1036,6 +1040,7 @@
1036
1040
  "__ZN30qvac_lib_inference_addon_llama26applyFinetuneParamHandlersEP8js_env_sRN28qvac_lib_inference_addon_cpp2js6ObjectERNS_21LlamaFinetuningParamsE",
1037
1041
  "__ZNK14MtmdLlmContext16ledgerFromChunksERKN4mtmd12input_chunksE",
1038
1042
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
1043
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
1039
1044
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
1040
1045
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
1041
1046
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -191,6 +193,7 @@
191
193
  "__ZN14MtmdLlmContext35refreshCurrentCacheTokensFromMemoryEv",
192
194
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
193
195
  "__ZN30qvac_lib_inference_addon_llama8batching7RequestC1EjONSt3__16vectorIiNS2_9allocatorIiEEEEji",
196
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
194
197
  "__ZN30qvac_lib_inference_addon_llama8batching14timeDecodeStepEP13llama_contextR11llama_batchRKNSt3__18functionIFiS2_S4_EEERKNS6_IFvS2_EEE",
195
198
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher20completeMediaBarrierEjiRKNSt3__18functionIFvjimEEE",
196
199
  "__ZN14TextLlmContext17removeLastNTokensEi",
@@ -381,6 +384,7 @@
381
384
  "__ZThn48_NK14MtmdLlmContext8getNPastEv",
382
385
  "__ZN14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
383
386
  "__ZNK12CacheManager15isCacheDisabledEv",
387
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
384
388
  "__ZNK14MtmdLlmContext14getCacheTokensEv",
385
389
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot5clearEv",
386
390
  "__ZN14TextLlmContext15reconcilePromptERKNSt3__16vectorIiNS0_9allocatorIiEEEEb",
@@ -503,14 +507,14 @@
503
507
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
504
508
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
505
509
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
506
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
510
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
507
511
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
508
512
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
509
513
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
510
514
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
511
515
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
512
516
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
513
- "__ZNK13ModelMetaData12tryGetStringEPKc",
517
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
514
518
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
515
519
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
516
520
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -541,10 +545,11 @@
541
545
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
542
546
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
543
547
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
544
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
545
549
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
546
550
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
547
551
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
552
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
553
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
549
554
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
550
555
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -191,6 +193,7 @@
191
193
  "__ZN14MtmdLlmContext35refreshCurrentCacheTokensFromMemoryEv",
192
194
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
193
195
  "__ZN30qvac_lib_inference_addon_llama8batching7RequestC1EjONSt3__16vectorIiNS2_9allocatorIiEEEEji",
196
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
194
197
  "__ZN30qvac_lib_inference_addon_llama8batching14timeDecodeStepEP13llama_contextR11llama_batchRKNSt3__18functionIFiS2_S4_EEERKNS6_IFvS2_EEE",
195
198
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher20completeMediaBarrierEjiRKNSt3__18functionIFvjimEEE",
196
199
  "__ZN14TextLlmContext17removeLastNTokensEi",
@@ -381,6 +384,7 @@
381
384
  "__ZThn48_NK14MtmdLlmContext8getNPastEv",
382
385
  "__ZN14TextLlmContext18onDraftLogitsReadyEiijRKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
383
386
  "__ZNK12CacheManager15isCacheDisabledEv",
387
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
384
388
  "__ZNK14MtmdLlmContext14getCacheTokensEv",
385
389
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot5clearEv",
386
390
  "__ZN14TextLlmContext15reconcilePromptERKNSt3__16vectorIiNS0_9allocatorIiEEEEb",
@@ -503,14 +507,14 @@
503
507
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
504
508
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
505
509
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
506
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
510
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
507
511
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
508
512
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
509
513
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
510
514
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
511
515
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
512
516
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
513
- "__ZNK13ModelMetaData12tryGetStringEPKc",
517
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
514
518
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
515
519
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
516
520
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -541,10 +545,11 @@
541
545
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
542
546
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
543
547
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
544
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
545
549
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
546
550
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
547
551
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
552
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
553
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
549
554
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
550
555
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.2"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -438,13 +438,12 @@
438
438
  "__ZTSNSt3__110__function6__baseIFvbEEE",
439
439
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_sEUlvE_EEP10js_value_sS8_OT_EUlS8_RKNS_9monostateEE_NS_9allocatorISJ_EEFSD_S8_SI_EEE",
440
440
  "__ZN30qvac_lib_inference_addon_llama5cache24CHECKPOINT_SECTION_MAGICE",
441
- "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
442
441
  "__ZTIN13BorrowablePtrINSt3__115basic_streambufIcNS0_11char_traitsIcEEEEE7DeleterE",
443
442
  "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output6LogMsgEE4__idE",
444
443
  "__ZTSNSt3__110__function6__baseIFNS_9monostateEvEEE",
445
444
  "__ZNSt3__19__any_imp17__unique_typeinfoI22FinetuneTerminalResultE4__idE",
446
445
  "__ZTINSt3__120__shared_ptr_emplaceINS_6atomicIbEENS_9allocatorIS2_EEEE",
447
- "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
446
+ "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
448
447
  "__ZTIN10moodycamel15ConcurrentQueueIN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler13QueuedRequestENS_28ConcurrentQueueDefaultTraitsEE12ProducerBaseE",
449
448
  "__ZTINSt3__120__shared_ptr_emplaceIN30qvac_lib_inference_addon_llama8batching14SlotStateCacheENS_9allocatorIS3_EEEE",
450
449
  "__ZTSNSt3__116__owns_one_stateIcEE",
@@ -454,6 +453,7 @@
454
453
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp17makeQueueCallbackINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEENS_8functionIFvRKT_EEENS_10shared_ptrINS2_11OutputQueueEEENS_13shared_futureIyEEEUlRKS9_E_NS7_ISN_EEFvSM_EEE",
455
454
  "__ZTSNSt3__110__function6__baseIFN22load_fit_normalization15SelectedBackendEN17backend_selection11BackendTypeERKNS_8optionalINS_7variantIJiNS4_11MainGpuTypeEEEEEERK13ModelMetaDatabRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENSK_ISM_EEEEEEE",
456
455
  "__ZTSNSt3__112__match_charIcEE",
456
+ "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
457
457
  "__ZTSNSt3__110__function6__baseIFNS_10unique_ptrI14SequenceDriverNS_14default_deleteIS3_EEEERK13common_paramsjiEEE",
458
458
  "__ZTSNSt3__110__function6__baseIFNS_13unordered_setINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS8_EENS_8equal_toIS8_EENS6_IS8_EEEEvEEE",
459
459
  "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerIN30qvac_lib_inference_addon_llama16BatchTokenOutputEEE",
@@ -558,6 +558,7 @@
558
558
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
559
559
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
560
560
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
561
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
561
562
  "__ZN12CacheManager14flushForUnloadEv",
562
563
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
563
564
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
@@ -574,6 +575,7 @@
574
575
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
575
576
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
576
577
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
578
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
577
579
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
578
580
  "__ZN10LlamaModel8getModelEv",
579
581
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler16cancelSlotLockedEjNS1_15SaveCachePolicyE",
@@ -589,6 +591,7 @@
589
591
  "__ZN14MtmdLlmContext17removeLastNTokensEi",
590
592
  "__ZN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeD2Ev",
591
593
  "__ZN30qvac_lib_inference_addon_llama5utils17resolveToolChoiceERKNSt3__18optionalINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEEERKNS1_6vectorI16common_chat_toolNS6_ISD_EEEE",
594
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEjRKNSt3__16vectorIP19ggml_backend_deviceNS2_9allocatorIS5_EEEERNS3_I25common_device_memory_dataNS6_ISB_EEEE",
592
595
  "__ZN14MtmdLlmContext10resetMediaEv",
593
596
  "__ZN30qvac_lib_inference_addon_llama8batching22finalizeTerminalDriverER14SequenceDriverNS0_10StopReasonEbRKNSt3__18functionIFvRKNS4_12basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEEE",
594
597
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshot10adoptEmptyEi",
@@ -678,6 +681,7 @@
678
681
  "__ZN14TextLlmContextC2ER13common_paramsNSt3__110unique_ptrI18common_init_resultNS2_14default_deleteIS4_EEEENS3_IN30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntimeENS5_ISA_EEEE",
679
682
  "__ZN14TextLlmContext17beginCacheRequestEv",
680
683
  "__ZNK12CacheManager15isCacheDisabledEv",
684
+ "__ZN30qvac_lib_inference_addon_llama11speculative20fitLeavesOutMtpDraftERKNS0_9GgufShapeE",
681
685
  "__ZThn48_NK14TextLlmContext19residentStateTokensEv",
682
686
  "__ZN14MtmdLlmContext18commitCacheRequestEv",
683
687
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler23recordPendingSlotCancelEjy",
@@ -1049,6 +1053,7 @@
1049
1053
  "__ZN30qvac_lib_inference_addon_llama26applyFinetuneParamHandlersEP8js_env_sRN28qvac_lib_inference_addon_cpp2js6ObjectERNS_21LlamaFinetuningParamsE",
1050
1054
  "__ZNK14MtmdLlmContext16ledgerFromChunksERKN4mtmd12input_chunksE",
1051
1055
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
1056
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
1052
1057
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
1053
1058
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
1054
1059
  "__ZN17image_pixel_limit11checkBufferEPKhmy",