@qvac/llm-llamacpp 0.64.1 → 0.64.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (18) hide show
  1. package/README.md +8 -4
  2. package/index.d.ts +8 -6
  3. package/package.json +11 -11
  4. package/prebuilds/android-arm64/qvac__llm-llamacpp.bare +0 -0
  5. package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare +0 -0
  6. package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports +10 -6
  7. package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare +0 -0
  8. package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare.exports +8 -4
  9. package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare +0 -0
  10. package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare.exports +10 -6
  11. package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare +0 -0
  12. package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare.exports +10 -6
  13. package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare +0 -0
  14. package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare.exports +8 -4
  15. package/prebuilds/linux-arm64/qvac__llm-llamacpp.bare +0 -0
  16. package/prebuilds/linux-x64/qvac__llm-llamacpp.bare +0 -0
  17. package/prebuilds/win32-x64/qvac__llm-llamacpp.bare +0 -0
  18. package/prebuilds/win32-x64/qvac__llm-llamacpp.bare.exports +0 -0
package/README.md CHANGED
@@ -212,7 +212,7 @@ const config = {
212
212
  | cache-type-v | `f16`, `f32`, `bf16`, `q8_0`, `q4_0`, … | auto (see below) | KV-cache **value** quantization type. Quantizing V requires `flash-attn` on |
213
213
  | mmproj-use-gpu | `"true"`/`"on"`/`"1"` or `"false"`/`"off"`/`"0"` | auto (see below) | Run the multimodal projector (mmproj / vision encoder) on the GPU. Only honoured when a GPU backend is selected (ignored with a warning on CPU / GPU-fallback). Unset = auto-default (see mmproj backend below) |
214
214
  | spec-type | `"none"`, `"draft-mtp"` or `"draft-dflash"` | `"none"` | Speculative decoding, as llama-server's `--spec-type`: `draft-mtp` drafts with the model's own multi-token-prediction head, `draft-dflash` with the DFlash model in `spec-draft-model` (see [Speculative decoding](#speculative-decoding-mtp-dflash) below). Also accepted as `spec_type` |
215
- | spec-draft-model | absolute path | – | DFlash draft model, as llama-server's `--spec-draft-model` (`-md`). Required by, and only accepted with, `spec-type: "draft-dflash"`. Placed like the model (`device`, `gpu-layers`). Also accepted as `spec_draft_model` |
215
+ | spec-draft-model | absolute path | – | Draft model, as llama-server's `--spec-draft-model` (`-md`): the DFlash model, required by `spec-type: "draft-dflash"`, or an external MTP head for `spec-type: "draft-mtp"`. Placed like the model (`device`, `gpu-layers`). Also accepted as `spec_draft_model` |
216
216
  | spec-draft-n-max | 1 – 64 | 3 | Longest draft per step. Also accepted as `spec_draft_n_max` |
217
217
  | spec-draft-n-min | 0 – `spec-draft-n-max` | 0 | Drafts shorter than this are not verified. Also accepted as `spec_draft_n_min` |
218
218
  | spec-draft-p-min | 0 – 1 | 0 | Minimum draft-head probability for a token to extend the draft. Also accepted as `spec_draft_p_min` |
@@ -263,11 +263,13 @@ projector runs on CPU. The resolved choice is logged at verbosity ≥ 2 as
263
263
 
264
264
  #### Speculative decoding (MTP, DFlash)
265
265
 
266
- `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
266
+ `spec-type: 'draft-mtp'` turns on llama-server's multi-token-prediction speculative decoding for models that ship their MTP layers (`blk.N.nextn.*`, e.g. Qwen3.5 / Qwen3.6 GGUFs that keep them; quantizations that strip them fail to load with this option unless an external head is given, see below). Every step the MTP head drafts up to `spec-draft-n-max` tokens from the model's own hidden states, the model verifies the sampled token and the draft in one decode, and the longest matching prefix plus one new token is kept. The sampler decides every kept token, so sampling settings, grammars and stop conditions behave as without speculation. The implementation follows llama-server step by step: the same draft limits, rollback by trimming or by a checkpoint for memory that cannot trim, and the same replay.
267
267
 
268
- `spec-type: 'draft-dflash'` drafts with a separate DFlash model instead, given as `spec-draft-model` (an absolute path; llama-server's `-md`). The DFlash model reads the hidden states of the target layers it was trained on and denoises a whole block of draft tokens in one pass, so a step drafts up to its block size minus one tokens (`spec-draft-n-max` caps it further). It is loaded next to the model and placed like it (`device`, `gpu-layers`). The load's fit and `assessFit` pass it to fabric as an extra model, but fabric cannot measure a DFlash model yet: it logs `failed to measure the memory of the extra model, fitting without it`, so neither counts the draft model's memory. Leave room for it yourself. Everything else (verification, rollback, batching, caches, stats) is the same as for MTP.
268
+ For a model without MTP layers, give an external MTP head as `spec-draft-model` (llama-server's `--spec-type draft-mtp -md <head>`): a GGUF with MTP layers for the same model, such as the MTP quantization of it, or a Gemma 4 assistant for its Gemma 4 model. The head is loaded next to the model and placed like it (`device`, `gpu-layers`), and the load's fit and `assessFit` count its weights as well as its context. A head without MTP layers fails the load, and `assessFit` refuses it with `unsupported-config`. A head for another model, with a different hidden width or vocabulary, also fails the load, and `assessFit` refuses it the same way. A Gemma 4 assistant reads the model's own tensors, so fabric measures it next to the model, as the load creates it.
269
269
 
270
- Both work on single prompts, multimodal prompts and continuous batching (`parallel >= 2`), together with `cacheKey` conversations: the draft context's state travels with checkpoints and the RAM tier, and a `cacheKey` file (target state only, like llama-server's slot files) restarts drafting from the next decoded token.
270
+ `spec-type: 'draft-dflash'` drafts with a separate DFlash model instead, given as `spec-draft-model` (an absolute path; llama-server's `-md`). The DFlash model reads the hidden states of the target layers it was trained on and denoises a whole block of draft tokens in one pass, so a step drafts up to its block size minus one tokens (`spec-draft-n-max` caps it further). It is loaded next to the model and placed like it (`device`, `gpu-layers`). The load's fit and `assessFit` count its weights and context, measured next to the model when the drafter borrows the model's embeddings and output head. A draft model fabric still cannot measure is left out of both, and fabric logs `failed to measure the memory of the extra model, fitting without it`. Everything else (verification, rollback, batching, caches, stats) is the same as for MTP.
271
+
272
+ MTP and DFlash work on single prompts, multimodal prompts and continuous batching (`parallel >= 2`), together with `cacheKey` conversations: the draft context's state travels with checkpoints and the RAM tier, and a `cacheKey` file (target state only, like llama-server's slot files) restarts drafting from the next decoded token.
271
273
 
272
274
  - Runtime stats gain `draftTokens` and `draftAcceptedTokens` (llama-server's `draft_n` / `draft_n_accepted`); per-position acceptance rates are logged at debug level.
273
275
  - `spec-draft-n-max` is lowered at load, with a warning, so the sampled token and the draft fit one `ubatch-size` and each sequence's share of `batch-size` (`batch-size / parallel`); a batch too small for even one draft token fails the load.
@@ -453,6 +455,8 @@ fit.hostBytes // the same, for the trailing host row
453
455
 
454
456
  `config` is the load, exactly as `loadModel` takes it. The projection resolves it with the same code that load runs, so every setting reaches the fitter the way it would reach the engine, and a load the engine would refuse is refused here as `status: "error"` with `unsupported-config`. A setting left out takes llama's default.
455
457
 
458
+ With `spec-type: 'draft-mtp'` the MTP draft context is fitted next to the model, as the load fits it, and its memory is in the verdict and in the per-device figures; on a model or head without MTP layers the load fails, so the projection is refused with `unsupported-config`. A DFlash draft model or a Gemma 4 assistant is fitted the same way, measured next to the model since it reads the model's tensors, and is in the verdict and the figures too (see [Speculative decoding](#speculative-decoding-mtp-dflash)).
459
+
456
460
  Pinning `gpu-layers` fixes the placement: the fitter refuses to move layers off a device the load claimed, so the answer is whether that exact placement fits, never a reduced one. Leave it unset for a projection that can place the model itself.
457
461
 
458
462
  `minCtxSize` sets a floor the fitter may not reduce the context below, defaulting to llama's own `fit-ctx` of 4096. `marginBytes` is the memory to leave free on every device, defaulting to llama's `fit-target` of 1 GiB per device. `backendsDir` is where the dynamically-loaded ggml backends live.
package/index.d.ts CHANGED
@@ -366,9 +366,10 @@ declare namespace LlmLlamacpp {
366
366
  cache_ram_mib?: NumericLike;
367
367
  /**
368
368
  * Speculative decoding, as llama-server's `--spec-type`. `'draft-mtp'`
369
- * drafts with the model's own multi-token-prediction (MTP / NextN) head;
370
- * the model must ship its MTP layers (`blk.N.nextn.*`), or the load
371
- * fails. `'draft-dflash'` drafts with the DFlash model given as
369
+ * drafts with the model's own multi-token-prediction (MTP / NextN) head,
370
+ * or with the external MTP head given as `spec_draft_model`; the model or
371
+ * the head must ship MTP layers (`blk.N.nextn.*`), or the load fails.
372
+ * `'draft-dflash'` drafts with the DFlash model given as
372
373
  * `spec_draft_model`. Either way the model verifies the draft, so the
373
374
  * output keeps the distribution of plain decoding. `'none'` (default)
374
375
  * decodes one token per step. Works on single prompts, multimodal
@@ -376,9 +377,10 @@ declare namespace LlmLlamacpp {
376
377
  */
377
378
  spec_type?: "none" | "draft-mtp" | "draft-dflash";
378
379
  /**
379
- * Absolute path of the DFlash draft model, as llama-server's
380
- * `--spec-draft-model` (`-md`). Required by, and only accepted with,
381
- * `spec_type: 'draft-dflash'`. The draft model is placed like the model
380
+ * Absolute path of the draft model, as llama-server's
381
+ * `--spec-draft-model` (`-md`): the DFlash model, required by
382
+ * `spec_type: 'draft-dflash'`, or an external MTP head for
383
+ * `spec_type: 'draft-mtp'`. The draft model is placed like the model
382
384
  * (`device`, `gpu_layers`). Also accepted as `spec-draft-model`.
383
385
  */
384
386
  spec_draft_model?: string;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qvac/llm-llamacpp",
3
- "version": "0.64.1",
3
+ "version": "0.64.3",
4
4
  "description": "llama addon for qvac",
5
5
  "addon": true,
6
6
  "scripts": {
@@ -79,8 +79,8 @@
79
79
  "bugs": "https://github.com/tetherto/qvac/issues",
80
80
  "homepage": "https://qvac.tether.io",
81
81
  "devDependencies": {
82
- "@qvac/fabric-android-arm64": "0.23.0",
83
- "@qvac/fabric-ios": "0.23.0",
82
+ "@qvac/fabric-android-arm64": "^0.23.1",
83
+ "@qvac/fabric-ios": "^0.23.1",
84
84
  "@types/node": "^24.2.1",
85
85
  "bare-crypto": "^1.15.3",
86
86
  "bare-process": "^4.2.2",
@@ -102,22 +102,22 @@
102
102
  "typescript-eslint": "^8.63.0"
103
103
  },
104
104
  "dependencies": {
105
- "@qvac/fabric": "^0.23.0",
105
+ "@qvac/fabric": "^0.23.1",
106
106
  "@qvac/infer-base": "^0.8.0",
107
107
  "@qvac/logging": "^0.1.0",
108
108
  "bare-fs": "^4.5.1",
109
109
  "bare-path": "^3.0.0"
110
110
  },
111
111
  "optionalDependencies": {
112
- "@qvac/fabric-linux-x64": "^0.23.0",
113
- "@qvac/fabric-linux-arm64": "^0.23.0",
114
- "@qvac/fabric-darwin-arm64": "^0.23.0",
115
- "@qvac/fabric-darwin-x64": "^0.23.0",
116
- "@qvac/fabric-win32-x64": "^0.23.0"
112
+ "@qvac/fabric-linux-x64": "^0.23.1",
113
+ "@qvac/fabric-linux-arm64": "^0.23.1",
114
+ "@qvac/fabric-darwin-arm64": "^0.23.1",
115
+ "@qvac/fabric-darwin-x64": "^0.23.1",
116
+ "@qvac/fabric-win32-x64": "^0.23.1"
117
117
  },
118
118
  "peerDependencies": {
119
- "@qvac/fabric-android-arm64": "^0.23.0",
120
- "@qvac/fabric-ios": "^0.23.0"
119
+ "@qvac/fabric-android-arm64": "^0.23.1",
120
+ "@qvac/fabric-ios": "^0.23.1"
121
121
  },
122
122
  "peerDependenciesMeta": {
123
123
  "@qvac/fabric-android-arm64": {
package/prebuilds/darwin-arm64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.3"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -503,14 +505,14 @@
503
505
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
504
506
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
505
507
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
506
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
508
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
507
509
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
508
510
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
509
511
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
510
512
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
511
513
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
512
514
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
513
- "__ZNK13ModelMetaData12tryGetStringEPKc",
515
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
514
516
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
515
517
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
516
518
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -541,10 +543,11 @@
541
543
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
542
544
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
543
545
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
544
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
546
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
545
547
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
546
548
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
547
549
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
550
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
551
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
549
552
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
550
553
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
@@ -560,6 +563,7 @@
560
563
  "__ZN14MtmdLlmContext24rebuildSamplerFromLedgerERKN30qvac_lib_inference_addon_llama5cache6LedgerE",
561
564
  "__ZNK30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntime13resetSequenceEi",
562
565
  "__ZN14MtmdLlmContext15reconcilePromptE11PrefillPlanRKN30qvac_lib_inference_addon_llama5cache6LedgerEb",
566
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEPKcRK18llama_model_paramsRK20llama_context_paramsRKNSt3__16vectorIP19ggml_backend_deviceNSA_9allocatorISD_EEEERNSB_I25common_device_memory_dataNSE_ISJ_EEEE",
563
567
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardD1Ev",
564
568
  "__ZN26LlamaLazyInitializeBackend10initializeERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
565
569
  "__ZNK13ModelMetaData9tryGetU32EPKc",
package/prebuilds/darwin-x64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.3"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -426,13 +426,12 @@
426
426
  "__ZTSNSt3__110__function6__baseIFvbEEE",
427
427
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_sEUlvE_EEP10js_value_sS8_OT_EUlS8_RKNS_9monostateEE_NS_9allocatorISJ_EEFSD_S8_SI_EEE",
428
428
  "__ZN30qvac_lib_inference_addon_llama5cache24CHECKPOINT_SECTION_MAGICE",
429
- "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
430
429
  "__ZTIN13BorrowablePtrINSt3__115basic_streambufIcNS0_11char_traitsIcEEEEE7DeleterE",
431
430
  "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output6LogMsgEE4__idE",
432
431
  "__ZTSNSt3__110__function6__baseIFNS_9monostateEvEEE",
433
432
  "__ZNSt3__19__any_imp17__unique_typeinfoI22FinetuneTerminalResultE4__idE",
434
433
  "__ZTINSt3__120__shared_ptr_emplaceINS_6atomicIbEENS_9allocatorIS2_EEEE",
435
- "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
434
+ "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
436
435
  "__ZTIN10moodycamel15ConcurrentQueueIN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler13QueuedRequestENS_28ConcurrentQueueDefaultTraitsEE12ProducerBaseE",
437
436
  "__ZTINSt3__120__shared_ptr_emplaceIN30qvac_lib_inference_addon_llama8batching14SlotStateCacheENS_9allocatorIS3_EEEE",
438
437
  "__ZTSNSt3__116__owns_one_stateIcEE",
@@ -442,6 +441,7 @@
442
441
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp17makeQueueCallbackINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEENS_8functionIFvRKT_EEENS_10shared_ptrINS2_11OutputQueueEEENS_13shared_futureIyEEEUlRKS9_E_NS7_ISN_EEFvSM_EEE",
443
442
  "__ZTSNSt3__110__function6__baseIFN22load_fit_normalization15SelectedBackendEN17backend_selection11BackendTypeERKNS_8optionalINS_7variantIJiNS4_11MainGpuTypeEEEEEERK13ModelMetaDatabRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENSK_ISM_EEEEEEE",
444
443
  "__ZTSNSt3__112__match_charIcEE",
444
+ "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
445
445
  "__ZTSNSt3__110__function6__baseIFNS_10unique_ptrI14SequenceDriverNS_14default_deleteIS3_EEEERK13common_paramsjiEEE",
446
446
  "__ZTSNSt3__110__function6__baseIFNS_13unordered_setINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS8_EENS_8equal_toIS8_EENS6_IS8_EEEEvEEE",
447
447
  "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerIN30qvac_lib_inference_addon_llama16BatchTokenOutputEEE",
@@ -455,9 +455,9 @@
455
455
  "__ZTSZN30qvac_lib_inference_addon_llama9assessFitEP8js_env_sP18js_callback_info_sEUlvE_",
456
456
  "__ZTSNSt3__111__match_anyIcEE",
457
457
  "__ZTSN28qvac_lib_inference_addon_cpp17MultiJobSchedulerE",
458
- "__ZTSNSt3__120__shared_ptr_emplaceIN28qvac_lib_inference_addon_cpp16OutputCallBackJs5StateENS_9allocatorIS3_EEEE",
459
458
  "__ZTSZN28qvac_lib_inference_addon_cpp17makeQueueCallbackIN24llama_finetuning_helpers21FinetuneProgressStatsEEENSt3__18functionIFvRKT_EEENS3_10shared_ptrINS_11OutputQueueEEENS3_13shared_futureIyEEEUlRKS2_E_",
460
459
  "__ZTSNSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZNS2_7AddonJs9cancelJobENS_8optionalIyEEEUlvE_EEP10js_value_sP8js_env_sOT_EUlSD_RKNS_9monostateEE_NS_9allocatorISJ_EEFSB_SD_SI_EEE",
460
+ "__ZTSNSt3__120__shared_ptr_emplaceIN28qvac_lib_inference_addon_cpp16OutputCallBackJs5StateENS_9allocatorIS3_EEEE",
461
461
  "__ZTI10LlmContext",
462
462
  "__ZTINSt3__110__function6__baseIFP10js_value_sRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS8_ISA_EEEEEEE",
463
463
  "__ZTSNSt3__120__shared_ptr_emplaceIN28qvac_lib_inference_addon_cpp9out_handl20JsErrorOutputHandlerENS_9allocatorIS3_EEEE",
@@ -545,6 +545,7 @@
545
545
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
546
546
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
547
547
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
548
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
548
549
  "__ZN12CacheManager14flushForUnloadEv",
549
550
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
550
551
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
@@ -561,6 +562,7 @@
561
562
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
562
563
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
563
564
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
565
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
564
566
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
565
567
  "__ZN10LlamaModel8getModelEv",
566
568
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler16cancelSlotLockedEjNS1_15SaveCachePolicyE",
@@ -1036,6 +1038,7 @@
1036
1038
  "__ZN30qvac_lib_inference_addon_llama26applyFinetuneParamHandlersEP8js_env_sRN28qvac_lib_inference_addon_cpp2js6ObjectERNS_21LlamaFinetuningParamsE",
1037
1039
  "__ZNK14MtmdLlmContext16ledgerFromChunksERKN4mtmd12input_chunksE",
1038
1040
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
1041
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
1039
1042
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
1040
1043
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
1041
1044
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
@@ -1065,6 +1068,7 @@
1065
1068
  "__ZN10LlamaModel21isMultiBatchActivatedERNS_15ReloadableStateE",
1066
1069
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchSchedulerC2E15LlmModelContextjjmiRK13common_paramsNSt3__18functionIFNS6_10unique_ptrI14SequenceDriverNS6_14default_deleteIS9_EEEES5_jiEEE",
1067
1070
  "__ZN14MtmdLlmContext15reconcilePromptE11PrefillPlanRKN30qvac_lib_inference_addon_llama5cache6LedgerEb",
1071
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEPKcRK18llama_model_paramsRK20llama_context_paramsRKNSt3__16vectorIP19ggml_backend_deviceNSA_9allocatorISD_EEEERNSB_I25common_device_memory_dataNSE_ISJ_EEEE",
1068
1072
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardD1Ev",
1069
1073
  "__ZN26LlamaLazyInitializeBackend10initializeERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
1070
1074
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler19drainFinishedLockedEPNSt3__111unique_lockINS2_5mutexEEE",
package/prebuilds/ios-arm64/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.3"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -503,14 +505,14 @@
503
505
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
504
506
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
505
507
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
506
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
508
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
507
509
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
508
510
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
509
511
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
510
512
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
511
513
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
512
514
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
513
- "__ZNK13ModelMetaData12tryGetStringEPKc",
515
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
514
516
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
515
517
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
516
518
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -541,10 +543,11 @@
541
543
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
542
544
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
543
545
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
544
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
546
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
545
547
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
546
548
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
547
549
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
550
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
551
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
549
552
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
550
553
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
@@ -560,6 +563,7 @@
560
563
  "__ZN14MtmdLlmContext24rebuildSamplerFromLedgerERKN30qvac_lib_inference_addon_llama5cache6LedgerE",
561
564
  "__ZNK30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntime13resetSequenceEi",
562
565
  "__ZN14MtmdLlmContext15reconcilePromptE11PrefillPlanRKN30qvac_lib_inference_addon_llama5cache6LedgerEb",
566
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEPKcRK18llama_model_paramsRK20llama_context_paramsRKNSt3__16vectorIP19ggml_backend_deviceNSA_9allocatorISD_EEEERNSB_I25common_device_memory_dataNSE_ISJ_EEEE",
563
567
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardD1Ev",
564
568
  "__ZN26LlamaLazyInitializeBackend10initializeERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
565
569
  "__ZNK13ModelMetaData9tryGetU32EPKc",
package/prebuilds/ios-arm64-simulator/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.3"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -132,12 +132,13 @@
132
132
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
133
133
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
134
134
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
135
- "__ZThn16_N10LlamaModelD0Ev",
135
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
136
136
  "__ZN12CacheManager14flushForUnloadEv",
137
137
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request19isGenerationPendingEv",
138
138
  "__ZNK10LlamaModel11activeSlotsEv",
139
139
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
140
140
  "__ZN10LlamaModel16llamaLogCallbackE14ggml_log_levelPKcPv",
141
+ "__ZThn16_N10LlamaModelD0Ev",
141
142
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
142
143
  "__ZN30qvac_lib_inference_addon_llama5utils25moveReasoningOutOfContentERNSt3__16vectorI15common_chat_msgNS1_9allocatorIS3_EEEERKNS0_13ReasoningTagsE",
143
144
  "__ZN30qvac_lib_inference_addon_llama5utils25splitReasoningFromContentERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS0_13ReasoningTagsE",
@@ -163,9 +164,10 @@
163
164
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
164
165
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
165
166
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
166
- "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
167
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
167
168
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
168
169
  "__ZN10LlamaModel8getModelEv",
170
+ "__ZNK30qvac_lib_inference_addon_llama8batching7Request15remainingToFeedEv",
169
171
  "__ZN30qvac_lib_inference_addon_llama5utils16isGemma4BasenameENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
170
172
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler22storeCheckpointsLockedERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEONS2_5dequeINS_5cache10CheckpointENS6_ISD_EEEE",
171
173
  "__ZN14MtmdLlmContext24captureHistoryCheckpointEi",
@@ -503,14 +505,14 @@
503
505
  "__ZNK14MtmdLlmContext14getKvCellsUsedEv",
504
506
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
505
507
  "__ZN14TextLlmContext9loadCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEb",
506
- "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
508
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
507
509
  "__ZN14MtmdLlmContext21applyGenerationParamsERK16GenerationParams",
508
510
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
509
511
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
510
512
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
511
513
  "__ZN10LlamaModel6reloadENSt3__18optionalI23FinetuneConfigOverridesEE",
512
514
  "__ZN14LlamaFinetuner18prepareEvalDatasetERKN30qvac_lib_inference_addon_llama21LlamaFinetuningParamsE",
513
- "__ZNK13ModelMetaData12tryGetStringEPKc",
515
+ "__ZN30qvac_lib_inference_addon_llama8batching7Request19isOptPrefillPendingERKNSt3__18optionalIS1_EE",
514
516
  "__ZN12CacheManagerC2EP10LlmContextNSt3__18functionIFvbEEE",
515
517
  "__ZN17backend_selection19getSplitDeviceNamesERKNS_16BackendInterfaceE",
516
518
  "__ZN30qvac_lib_inference_addon_llama5utils27reasoningBudgetSamplerBuiltERK22common_params_sampling",
@@ -541,10 +543,11 @@
541
543
  "__ZN30qvac_lib_inference_addon_llama5utils19getHarmonyCallTokenEP13llama_context",
542
544
  "__ZN30qvac_lib_inference_addon_llama5utils21SequenceStateSnapshotD2Ev",
543
545
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler5clearEv",
544
- "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
546
+ "__ZNK13ModelMetaData12tryGetStringEPKc",
545
547
  "__ZThn48_NK14MtmdLlmContext9saveCacheERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
546
548
  "__ZThn24_N10LlamaModel7processERKNSt3__13anyEy",
547
549
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler29serviceNextMediaSegmentLockedEPNSt3__111unique_lockINS2_5mutexEEE",
550
+ "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15hasValidDriverFEv",
548
551
  "__ZN30qvac_lib_inference_addon_llama5utils13isGemma4ModelEPK11llama_model",
549
552
  "__ZN14TextLlmContext16generateResponseERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
550
553
  "__ZN10LlamaModel24validateCheckpointBudgetERNS_15ReloadableStateE",
@@ -560,6 +563,7 @@
560
563
  "__ZN14MtmdLlmContext24rebuildSamplerFromLedgerERKN30qvac_lib_inference_addon_llama5cache6LedgerE",
561
564
  "__ZNK30qvac_lib_inference_addon_llama11speculative18SpeculativeRuntime13resetSequenceEi",
562
565
  "__ZN14MtmdLlmContext15reconcilePromptE11PrefillPlanRKN30qvac_lib_inference_addon_llama5cache6LedgerEb",
566
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEPKcRK18llama_model_paramsRK20llama_context_paramsRKNSt3__16vectorIP19ggml_backend_deviceNSA_9allocatorISD_EEEERNSB_I25common_device_memory_dataNSE_ISJ_EEEE",
563
567
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardD1Ev",
564
568
  "__ZN26LlamaLazyInitializeBackend10initializeERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
565
569
  "__ZNK13ModelMetaData9tryGetU32EPKc",
package/prebuilds/ios-x64-simulator/qvac__llm-llamacpp.bare.exports CHANGED
@@ -7,7 +7,7 @@
7
7
  ],
8
8
  "current_versions": [
9
9
  {
10
- "version": "0.64.1"
10
+ "version": "0.64.3"
11
11
  }
12
12
  ],
13
13
  "exported_symbols": [
@@ -438,13 +438,12 @@
438
438
  "__ZTSNSt3__110__function6__baseIFvbEEE",
439
439
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZN30qvac_lib_inference_addon_llama9saveCacheEP8js_env_sP18js_callback_info_sEUlvE_EEP10js_value_sS8_OT_EUlS8_RKNS_9monostateEE_NS_9allocatorISJ_EEFSD_S8_SI_EEE",
440
440
  "__ZN30qvac_lib_inference_addon_llama5cache24CHECKPOINT_SECTION_MAGICE",
441
- "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
442
441
  "__ZTIN13BorrowablePtrINSt3__115basic_streambufIcNS0_11char_traitsIcEEEEE7DeleterE",
443
442
  "__ZNSt3__19__any_imp17__unique_typeinfoIN28qvac_lib_inference_addon_cpp6Output6LogMsgEE4__idE",
444
443
  "__ZTSNSt3__110__function6__baseIFNS_9monostateEvEEE",
445
444
  "__ZNSt3__19__any_imp17__unique_typeinfoI22FinetuneTerminalResultE4__idE",
446
445
  "__ZTINSt3__120__shared_ptr_emplaceINS_6atomicIbEENS_9allocatorIS2_EEEE",
447
- "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
446
+ "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp9out_handl26JsStringArrayOutputHandlerC1EvEUlRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS9_ISB_EEEEE_NS9_ISG_EEFP10js_value_sSF_EEE",
448
447
  "__ZTIN10moodycamel15ConcurrentQueueIN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler13QueuedRequestENS_28ConcurrentQueueDefaultTraitsEE12ProducerBaseE",
449
448
  "__ZTINSt3__120__shared_ptr_emplaceIN30qvac_lib_inference_addon_llama8batching14SlotStateCacheENS_9allocatorIS3_EEEE",
450
449
  "__ZTSNSt3__116__owns_one_stateIcEE",
@@ -454,6 +453,7 @@
454
453
  "__ZTINSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp17makeQueueCallbackINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEENS_8functionIFvRKT_EEENS_10shared_ptrINS2_11OutputQueueEEENS_13shared_futureIyEEEUlRKS9_E_NS7_ISN_EEFvSM_EEE",
455
454
  "__ZTSNSt3__110__function6__baseIFN22load_fit_normalization15SelectedBackendEN17backend_selection11BackendTypeERKNS_8optionalINS_7variantIJiNS4_11MainGpuTypeEEEEEERK13ModelMetaDatabRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENSK_ISM_EEEEEEE",
456
455
  "__ZTSNSt3__112__match_charIcEE",
456
+ "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerINSt3__16vectorINS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEENS7_IS9_EEEEEE",
457
457
  "__ZTSNSt3__110__function6__baseIFNS_10unique_ptrI14SequenceDriverNS_14default_deleteIS3_EEEERK13common_paramsjiEEE",
458
458
  "__ZTSNSt3__110__function6__baseIFNS_13unordered_setINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS8_EENS_8equal_toIS8_EENS6_IS8_EEEEvEEE",
459
459
  "__ZTIN28qvac_lib_inference_addon_cpp9out_handl19JsBaseOutputHandlerIN30qvac_lib_inference_addon_llama16BatchTokenOutputEEE",
@@ -467,9 +467,9 @@
467
467
  "__ZTSZN30qvac_lib_inference_addon_llama9assessFitEP8js_env_sP18js_callback_info_sEUlvE_",
468
468
  "__ZTSNSt3__111__match_anyIcEE",
469
469
  "__ZTSN28qvac_lib_inference_addon_cpp17MultiJobSchedulerE",
470
- "__ZTSNSt3__120__shared_ptr_emplaceIN28qvac_lib_inference_addon_cpp16OutputCallBackJs5StateENS_9allocatorIS3_EEEE",
471
470
  "__ZTSZN28qvac_lib_inference_addon_cpp17makeQueueCallbackIN24llama_finetuning_helpers21FinetuneProgressStatsEEENSt3__18functionIFvRKT_EEENS3_10shared_ptrINS_11OutputQueueEEENS3_13shared_futureIyEEEUlRKS2_E_",
472
471
  "__ZTSNSt3__110__function6__funcIZN28qvac_lib_inference_addon_cpp2js11JsAsyncTask3runIZNS2_7AddonJs9cancelJobENS_8optionalIyEEEUlvE_EEP10js_value_sP8js_env_sOT_EUlSD_RKNS_9monostateEE_NS_9allocatorISJ_EEFSB_SD_SI_EEE",
472
+ "__ZTSNSt3__120__shared_ptr_emplaceIN28qvac_lib_inference_addon_cpp16OutputCallBackJs5StateENS_9allocatorIS3_EEEE",
473
473
  "__ZTI10LlmContext",
474
474
  "__ZTINSt3__110__function6__baseIFP10js_value_sRKNS_6vectorINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS8_ISA_EEEEEEE",
475
475
  "__ZTSNSt3__120__shared_ptr_emplaceIN28qvac_lib_inference_addon_cpp9out_handl20JsErrorOutputHandlerENS_9allocatorIS3_EEEE",
@@ -558,6 +558,7 @@
558
558
  "__ZN30qvac_lib_inference_addon_llama5utils24isDeepSeekV4ArchitectureENSt3__117basic_string_viewIcNS1_11char_traitsIcEEEE",
559
559
  "__ZN30qvac_lib_inference_addon_llama11speculative19SpeculativeSequence6verifyEP14common_sampleri",
560
560
  "__ZN30qvac_lib_inference_addon_llama5utils24selectReasoningTagSourceERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_RKNS1_8optionalINS0_13ReasoningTagsEEE",
561
+ "__ZN30qvac_lib_inference_addon_llama11speculative13readGgufShapeERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE",
561
562
  "__ZN12CacheManager14flushForUnloadEv",
562
563
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request20isAwaitingCheckpointEv",
563
564
  "__ZN30qvac_lib_inference_addon_llama5utils20restoreSequenceStateEP13llama_contextiRKNS0_21SequenceStateSnapshotE",
@@ -574,6 +575,7 @@
574
575
  "__ZN30qvac_lib_inference_addon_llama8batching20RuntimeStatsSnapshot21recordSpeculativeStepExNSt3__16chrono8durationIxNS2_5ratioILl1ELl1000000000EEEEE",
575
576
  "__ZNK30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler9numActiveEv",
576
577
  "__ZN30qvac_lib_inference_addon_llama8batching19MultiRequestBatcher12markFinishedEjNS0_10StopReasonE",
578
+ "__ZN30qvac_lib_inference_addon_llama11speculative14addDraftMemoryERKNSt3__16vectorIP19ggml_backend_deviceNS1_9allocatorIS4_EEEERNS2_I25common_device_memory_dataNS5_ISA_EEEES9_RKSC_b",
577
579
  "__ZThn48_N14TextLlmContext13onSequenceEndERKNSt3__18functionIFvRKNS0_12basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEEEE",
578
580
  "__ZN10LlamaModel8getModelEv",
579
581
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler16cancelSlotLockedEjNS1_15SaveCachePolicyE",
@@ -1049,6 +1051,7 @@
1049
1051
  "__ZN30qvac_lib_inference_addon_llama26applyFinetuneParamHandlersEP8js_env_sRN28qvac_lib_inference_addon_cpp2js6ObjectERNS_21LlamaFinetuningParamsE",
1050
1052
  "__ZNK14MtmdLlmContext16ledgerFromChunksERKN4mtmd12input_chunksE",
1051
1053
  "__ZNK30qvac_lib_inference_addon_llama8batching7Request15hasTokensToFeedEv",
1054
+ "__ZN30qvac_lib_inference_addon_llama11speculative15mtpHeadMismatchExxxx",
1052
1055
  "__ZN30qvac_lib_inference_addon_llama11speculative25captureDraftSequenceStateEP13llama_contextij",
1053
1056
  "__ZN30qvac_lib_inference_addon_llama11speculative22parseSpeculativeConfigERNSt3__113unordered_mapINS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES8_NS1_4hashIS8_EENS1_8equal_toIS8_EENS6_INS1_4pairIKS8_S8_EEEEEE",
1054
1057
  "__ZN17image_pixel_limit11checkBufferEPKhmy",
@@ -1078,6 +1081,7 @@
1078
1081
  "__ZN10LlamaModel21isMultiBatchActivatedERNS_15ReloadableStateE",
1079
1082
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchSchedulerC2E15LlmModelContextjjmiRK13common_paramsNSt3__18functionIFNS6_10unique_ptrI14SequenceDriverNS6_14default_deleteIS9_EEEES5_jiEEE",
1080
1083
  "__ZN14MtmdLlmContext15reconcilePromptE11PrefillPlanRKN30qvac_lib_inference_addon_llama5cache6LedgerEb",
1084
+ "__ZNK30qvac_lib_inference_addon_llama11speculative19SpeculativeFitModel9addMemoryEPKcRK18llama_model_paramsRK20llama_context_paramsRKNSt3__16vectorIP19ggml_backend_deviceNSA_9allocatorISD_EEEERNSB_I25common_device_memory_dataNSE_ISJ_EEEE",
1081
1085
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler15StepUnlockGuardD1Ev",
1082
1086
  "__ZN26LlamaLazyInitializeBackend10initializeERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_",
1083
1087
  "__ZN30qvac_lib_inference_addon_llama8batching24ContinuousBatchScheduler19drainFinishedLockedEPNSt3__111unique_lockINS2_5mutexEEE",