@qvac/translation-nmtcpp 0.14.0 → 0.16.0

package/README.md

@@ -298,11 +298,19 @@ The three GPU control keys each accept a camelCase alias alongside the snake_cas
 | Key | Alias | Type | Description |
 |-----|-------|------|-------------|
 | `use_gpu` | `useGPU` | boolean | Enable GPU inference. When `false` (default), only the CPU backend is used. Bergamot is CPU-only by design — this flag is effectively a no-op for Bergamot. |
-| `gpu_backend` | `gpuBackend` | string | Case-insensitive **substring** match against the ggml device name (e.g. `"vulkan"`, `"vulkan0"`, `"opencl"`, `"metal"`). When set, the selector runs a single explicit pass and picks the first non-CPU device whose name contains the substring. When unset, the default gated selection runs (see [Backends](#backends)). Explicit `"opencl"` bypasses the build-time `USE_OPENCL` guard — an informed opt-in. |
+| `gpu_backend` | `gpuBackend` | string | Case-insensitive **substring** match against eligible Vulkan, Metal, OpenCL, or CUDA device names (e.g. `"vulkan"`, `"vulkan0"`, `"opencl"`, `"metal"`). When unset, the default gated selection runs (see [Backends](#backends)). Any explicit selector that resolves to OpenCL bypasses the build-time `USE_OPENCL` guard — an informed opt-in. |
 | `gpu_device` | `gpuDevice` | int | Ordinal within the matching devices. Defaults to `0` (first match). Example: `{gpu_backend: "vulkan", gpu_device: 1}` picks the second Vulkan adapter. |
 | `backendsDir` | — | string | Path to the directory containing the runtime backend shared libraries (`libqvac-ggml-vulkan.so`, etc.). Defaults to `<package>/prebuilds` when unset, which is where `npm install` places the shipped prebuilds. Must be an absolute path; paths with `..` segments or unresolvable symlinks are rejected with a warning and fall back to the default prebuilds directory. |
 | `openclCacheDir` | — | string | **Android only.** Writable directory the OpenCL backend uses for its JIT kernel cache (forwarded via `GGML_OPENCL_CACHE_DIR`). Must be an absolute path; paths with `..` segments are rejected. The OpenCL backend falls back to a non-writable relative path if this is unset, which `ggml_abort()`s during init inside the app sandbox — always provide an app-writable path when exercising OpenCL on Android. |
 
+GPU execution is limited to Vulkan, Metal/MTL, OpenCL, and CUDA devices reported as GPU or integrated GPU. RPC, ROCm/HIP, SYCL, MUSA, and unknown backend families fall back to CPU — RPC because translation has no split-mode support and cannot execute over it. CUDA is admitted ahead of its planned Fabric build so that build does not require another selector change. An explicit selector only narrows this eligible set; it cannot enable an unsupported family.
+
+Within the eligible set, dedicated GPUs are ordered ahead of integrated ones, registry order preserved within each class, and `gpu_device` is an ordinal into that ordered list.
+
+OpenCL is admitted as a family, not narrowed to Adreno. The selector inspects only the device and registry name, never the vendor or description, so any GPU or integrated-GPU OpenCL device is eligible. What gates it is the build-time `USE_OPENCL` guard, which exists because Adreno is the only OpenCL target this package has been validated on; an explicit `gpu_backend` selector bypasses that guard as an informed opt-in.
+
+ACCEL and META devices are never selected. Translation runs on the single selected compute device plus CPU — it exposes no layer or tensor split mode, so no secondary accelerator is added to the backend list.
+
 > **Tip:** Use `model.getActiveBackendName()` after `load()` to confirm which backend actually took the request — see [Additional Features](#additional-features). The GGML scheduler silently falls back to CPU when no usable GPU ICD is registered, and this is the only way to detect that.
 
 ### 4. Create Model Instance
@@ -942,7 +950,7 @@ bare-make generate -D USE_BERGAMOT=OFF
 At runtime, the addon picks a ggml compute device using the `use_gpu`, `gpu_backend`, and `gpu_device` config keys described in [Backend & GPU Settings](#backend--gpu-settings). When `use_gpu` is true and `gpu_backend` is **not** set, the selector falls back to a default gated pass:
 
 1. If built with `USE_OPENCL=ON`, prefer an OpenCL device first.
-2. Otherwise (and as a fallback in the `ON` case) pick any non-CPU device. When `USE_OPENCL=OFF` (the default), OpenCL-named devices are also excluded from the fallback — the OpenCL backend still loads as a shared library but is never selected automatically.
+2. Otherwise (and as a fallback in the `ON` case) pick an eligible Vulkan, Metal, or CUDA device. When `USE_OPENCL=OFF` (the default), OpenCL devices are excluded from the fallback — the OpenCL backend still loads as a shared library but is never selected automatically.
 
 An explicit `config.gpu_backend: 'opencl'` always bypasses the `USE_OPENCL` guard and selects OpenCL directly. The flag gates *automatic* selection only; caller-explicit requests are honored.
 

package/index.d.ts

@@ -105,7 +105,7 @@ declare namespace TranslationNmtcpp {
         modelType: TranslationNmtcppModelTypes[keyof TranslationNmtcppModelTypes];
         pivotConfig?: Record<string, unknown>;
         /**
-         * Enable GPU (non-CPU) compute backend. Read once at load() time.
+         * Enable an eligible Vulkan, Metal, OpenCL, or CUDA compute backend.
          * Bergamot is CPU-only by design — this flag is a no-op for that backend.
          *
          * `use_gpu` mirrors the C-struct field (`nmt_context_params::use_gpu`)
@@ -117,10 +117,10 @@ declare namespace TranslationNmtcpp {
         use_gpu?: boolean;
         useGPU?: boolean;
         /**
-         * Case-insensitive substring filter over the ggml device name when selecting
+         * Case-insensitive substring filter over eligible ggml device names when selecting
          * a compute backend (e.g. "vulkan", "vulkan0", "opencl", "metal"). When set,
          * replaces the default gated selector with a single explicit pass.
-         * An explicit "opencl" bypasses the build-time USE_OPENCL guard.
+         * Any explicit selector resolving to OpenCL bypasses the build-time USE_OPENCL guard.
          *
          * `gpu_backend` mirrors the C-struct field and is the primary key.
          * `gpuBackend` is the camelCase alias matching the sibling-addon convention.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/translation-nmtcpp",
-  "version": "0.14.0",
+  "version": "0.16.0",
   "description": "translation addon for qvac",
   "addon": true,
   "engines": {
@@ -95,7 +95,7 @@
   },
   "dependencies": {
     "@qvac/error": "^0.1.0",
-    "@qvac/fabric": "^0.10.0",
+    "@qvac/fabric": "^0.14.0",
     "@qvac/infer-base": "^0.6.2",
     "@qvac/logging": "^0.1.0",
     "bare-fs": "^4.5.1",

package/prebuilds/darwin-arm64/qvac__translation-nmtcpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.14"
+        "version": "0.16"
       }
     ],
     "exported_symbols": [
@@ -3717,6 +3717,7 @@
             "__ZN4absl12lts_2025081418debugging_internal22StackTraceWorksForTestEv",
             "__ZNK4YAML6Stream13StreamInUtf16Ev",
             "__ZN6marian4castE12IntrusivePtrINS_9ChainableIS0_INS_10TensorBaseEEEEENS_4TypeE",
+            "__Z18nmtSelectGpuDeviceRK19NmtBackendInterfacebRKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiPKcb",
             "__ZN6google8protobuf8internal24RepeatedStringTypeTraits23GetDefaultRepeatedFieldEv",
             "__ZNK6marian4data24BinaryShortlistGenerator14saveBlobToFileERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE",
             "__ZN13sentencepiece10ModelProto9MergeFromERKS0_",

package/prebuilds/darwin-x64/qvac__translation-nmtcpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.14"
+        "version": "0.16"
       }
     ],
     "exported_symbols": [
@@ -5841,6 +5841,7 @@
             "__ZN6marian4Node14init_dependentEv",
             "__ZN6Pathie4Path8sanitizeEv",
             "__ZNK6marian4data24BinaryShortlistGenerator14saveBlobToFileERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE",
+            "__Z18nmtSelectGpuDeviceRK19NmtBackendInterfacebRKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiPKcb",
             "__tr_align",
             "__ZN6marian8bergamot25getVocabsMemoryFromConfigENSt3__110shared_ptrINS_7OptionsEEERNS1_6vectorINS2_INS0_13AlignedVectorIcEEEENS1_9allocatorIS8_EEEE",
             "__ZNK6google8protobuf8internal12ExtensionSet13ExtensionTypeEi",

package/prebuilds/ios-arm64/qvac__translation-nmtcpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.14"
+        "version": "0.16"
       }
     ],
     "exported_symbols": [
@@ -1118,13 +1118,13 @@
             "_pcre2_get_mark_8",
             "__ZN3ruy14Kernel8bitNeonERKNS_16KernelParams8bitILi4ELi4EEE",
             "__ZN4absl12lts_2025081418container_internal28SetHashtablezSampleParameterEi",
-            "__ZN6marian3cpu11ConcatenateE12IntrusivePtrINS_10TensorBaseEERKNSt3__16vectorIS3_NS4_9allocatorIS3_EEEEi",
             "__ZN13sentencepiece10normalizer10Normalizer4InitEv",
             "__ZN6google8protobuf8internal26DuplicateIfNonNullInternalEPNS0_11MessageLiteE",
-            "_pcre2_set_match_limit_8",
+            "__ZN6marian3cpu11ConcatenateE12IntrusivePtrINS_10TensorBaseEERKNSt3__16vectorIS3_NS4_9allocatorIS3_EEEEi",
             "__ZN6google8protobuf8internal12ExtensionSet13MutableStringEihPKNS0_15FieldDescriptorE",
-            "__ZNK13sentencepiece23SentencePieceNormalizer23mutable_normalizer_specEv",
             "__ZNK6google8protobuf8internal12ExtensionSet26GetPrototypeForLazyMessageEPKNS0_11MessageLiteEi",
+            "_pcre2_set_match_limit_8",
+            "__ZNK13sentencepiece23SentencePieceNormalizer23mutable_normalizer_specEv",
             "__ZNK3ruy14PrepackedCache7KeyHashclERKNS0_3KeyE",
             "__ZNK6Pathie4Path5dglobERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEi",
             "__ZN6google8protobuf8internal12FieldSkipper11SkipMessageEPNS0_2io16CodedInputStreamE",
@@ -3697,6 +3697,7 @@
             "__ZN4absl12lts_2025081418debugging_internal22StackTraceWorksForTestEv",
             "__ZNK4YAML6Stream13StreamInUtf16Ev",
             "__ZN6marian4castE12IntrusivePtrINS_9ChainableIS0_INS_10TensorBaseEEEEENS_4TypeE",
+            "__Z18nmtSelectGpuDeviceRK19NmtBackendInterfacebRKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiPKcb",
             "__ZN6google8protobuf8internal24RepeatedStringTypeTraits23GetDefaultRepeatedFieldEv",
             "__ZNK6marian4data24BinaryShortlistGenerator14saveBlobToFileERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE",
             "__ZN13sentencepiece10ModelProto9MergeFromERKS0_",
@@ -4142,8 +4143,8 @@
             "__ZN6google8protobuf16RepeatedPtrFieldINSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEE2atEi",
             "__ZNK6google8protobuf13RepeatedFieldIfE4dataEv",
             "__ZN6google8protobuf13RepeatedFieldIfE12InternalSwapEPS2_",
-            "__ZN4absl12lts_2025081419str_format_internal13FormatArgImpl8DispatchIaEEbNS2_4DataENS1_24FormatConversionSpecImplEPv",
             "__ZN6marian15fastopt_helpers2AsIdE5applyERKNS_7FastOptE",
+            "__ZN4absl12lts_2025081419str_format_internal13FormatArgImpl8DispatchIaEEbNS2_4DataENS1_24FormatConversionSpecImplEPv",
             "__ZN4absl12lts_2025081419str_format_internal13ConvertIntArgIjEEbT_NS1_24FormatConversionSpecImplEPNS1_14FormatSinkImplE",
             "__ZN6google8protobuf13RepeatedFieldIbE3AddERKb",
             "__ZN6google8protobuf13RepeatedFieldIbEC1Ev",

package/prebuilds/ios-arm64-simulator/qvac__translation-nmtcpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.14"
+        "version": "0.16"
       }
     ],
     "exported_symbols": [
@@ -1118,13 +1118,13 @@
             "_pcre2_get_mark_8",
             "__ZN3ruy14Kernel8bitNeonERKNS_16KernelParams8bitILi4ELi4EEE",
             "__ZN4absl12lts_2025081418container_internal28SetHashtablezSampleParameterEi",
-            "__ZN6marian3cpu11ConcatenateE12IntrusivePtrINS_10TensorBaseEERKNSt3__16vectorIS3_NS4_9allocatorIS3_EEEEi",
             "__ZN13sentencepiece10normalizer10Normalizer4InitEv",
             "__ZN6google8protobuf8internal26DuplicateIfNonNullInternalEPNS0_11MessageLiteE",
-            "_pcre2_set_match_limit_8",
+            "__ZN6marian3cpu11ConcatenateE12IntrusivePtrINS_10TensorBaseEERKNSt3__16vectorIS3_NS4_9allocatorIS3_EEEEi",
             "__ZN6google8protobuf8internal12ExtensionSet13MutableStringEihPKNS0_15FieldDescriptorE",
-            "__ZNK13sentencepiece23SentencePieceNormalizer23mutable_normalizer_specEv",
             "__ZNK6google8protobuf8internal12ExtensionSet26GetPrototypeForLazyMessageEPKNS0_11MessageLiteEi",
+            "_pcre2_set_match_limit_8",
+            "__ZNK13sentencepiece23SentencePieceNormalizer23mutable_normalizer_specEv",
             "__ZNK3ruy14PrepackedCache7KeyHashclERKNS0_3KeyE",
             "__ZNK6Pathie4Path5dglobERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEi",
             "__ZN6google8protobuf8internal12FieldSkipper11SkipMessageEPNS0_2io16CodedInputStreamE",
@@ -3697,6 +3697,7 @@
             "__ZN4absl12lts_2025081418debugging_internal22StackTraceWorksForTestEv",
             "__ZNK4YAML6Stream13StreamInUtf16Ev",
             "__ZN6marian4castE12IntrusivePtrINS_9ChainableIS0_INS_10TensorBaseEEEEENS_4TypeE",
+            "__Z18nmtSelectGpuDeviceRK19NmtBackendInterfacebRKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiPKcb",
             "__ZN6google8protobuf8internal24RepeatedStringTypeTraits23GetDefaultRepeatedFieldEv",
             "__ZNK6marian4data24BinaryShortlistGenerator14saveBlobToFileERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE",
             "__ZN13sentencepiece10ModelProto9MergeFromERKS0_",
@@ -4142,8 +4143,8 @@
             "__ZN6google8protobuf16RepeatedPtrFieldINSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEE2atEi",
             "__ZNK6google8protobuf13RepeatedFieldIfE4dataEv",
             "__ZN6google8protobuf13RepeatedFieldIfE12InternalSwapEPS2_",
-            "__ZN4absl12lts_2025081419str_format_internal13FormatArgImpl8DispatchIaEEbNS2_4DataENS1_24FormatConversionSpecImplEPv",
             "__ZN6marian15fastopt_helpers2AsIdE5applyERKNS_7FastOptE",
+            "__ZN4absl12lts_2025081419str_format_internal13FormatArgImpl8DispatchIaEEbNS2_4DataENS1_24FormatConversionSpecImplEPv",
             "__ZN4absl12lts_2025081419str_format_internal13ConvertIntArgIjEEbT_NS1_24FormatConversionSpecImplEPNS1_14FormatSinkImplE",
             "__ZN6google8protobuf13RepeatedFieldIbE3AddERKb",
             "__ZN6google8protobuf13RepeatedFieldIbEC1Ev",

package/prebuilds/ios-x64-simulator/qvac__translation-nmtcpp.bare.exports

@@ -7,7 +7,7 @@
     ],
     "current_versions": [
       {
-        "version": "0.14"
+        "version": "0.16"
       }
     ],
     "exported_symbols": [
@@ -5814,6 +5814,7 @@
             "__ZN6marian4Node14init_dependentEv",
             "__ZN6Pathie4Path8sanitizeEv",
             "__ZNK6marian4data24BinaryShortlistGenerator14saveBlobToFileERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE",
+            "__Z18nmtSelectGpuDeviceRK19NmtBackendInterfacebRKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEiPKcb",
             "__tr_align",
             "__ZN6marian8bergamot25getVocabsMemoryFromConfigENSt3__110shared_ptrINS_7OptionsEEERNS1_6vectorINS2_INS0_13AlignedVectorIcEEEENS1_9allocatorIS8_EEEE",
             "__ZNK6google8protobuf8internal12ExtensionSet13ExtensionTypeEi",

package/test/integration/utils.js

@@ -541,25 +541,56 @@ function prestagedModelPath(modelName) {
   return staged ? staged.src : null
 }
 
-// Require the exact host-recorded byte count on both sides of the app copy so
-// truncated staged files fall through to the normal presigned-S3 download.
+// iOS kills an app that dirties more than 4 GiB in 24h, and there the staged
+// file already sits in the app's own writable Documents dir — so copying it
+// into the model dir spends that whole budget for nothing. Hardlink instead:
+// same inode, zero bytes. On Android the staging dir is a different filesystem,
+// so link() fails EXDEV and we fall back to the copy that has always run there.
+// `link`/`copy` are injectable so that fallback is unit-testable.
+function linkOrCopySync({ src, dest, link = fs.linkSync, copy = fs.copyFileSync }) {
+  // Same path in and out — the staged file IS the destination (a caller whose
+  // model dir is testDir itself). Deleting first would destroy the staged model
+  // and leave both link() and copy() failing ENOENT; the copy this replaced was
+  // a harmless no-op here.
+  if (path.resolve(src) === path.resolve(dest)) return 'link'
+
+  try {
+    fs.unlinkSync(dest)
+  } catch (_) {}
+
+  try {
+    link(src, dest)
+    return 'link'
+  } catch (err) {
+    console.log(
+      `[prestage] hardlink failed on ${platform} (${err.message}); falling back to a byte copy`
+    )
+  }
+
+  copy(src, dest)
+  return 'copy'
+}
+
+// Require the exact host-recorded byte count on both sides of the staging step
+// so truncated staged files fall through to the normal presigned-S3 download.
 function copyPrestagedModel(modelName, destPath, minBytes) {
   const staged = readPrestagedModel(modelName)
   if (!staged || staged.expectedSize < minBytes) return false
   try {
     const dir = path.dirname(destPath)
     if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true })
-    fs.copyFileSync(staged.src, destPath)
+    const how = linkOrCopySync({ src: staged.src, dest: destPath })
     const size = fs.statSync(destPath).size
     if (size === staged.expectedSize) {
       console.log(
-        `[prestage] Using pre-staged model ${modelName} (${(size / 1024 / 1024).toFixed(1)}MB)`
+        `[prestage] Using pre-staged model ${modelName} (${(size / 1024 / 1024).toFixed(1)}MB, ` +
+          `${how === 'link' ? 'hardlinked' : 'copied'})`
       )
       return true
     }
     fs.unlinkSync(destPath)
   } catch (err) {
-    console.log(`[prestage] copy of ${modelName} failed: ${err.message}`)
+    console.log(`[prestage] staging of ${modelName} failed: ${err.message}`)
     try {
       fs.unlinkSync(destPath)
     } catch (_) {}
@@ -1248,6 +1279,7 @@ module.exports = {
   ensureIndicTransModel,
   ensureBergamotModel,
   copyPrestagedModel,
+  linkOrCopySync,
   prestagedModelPath,
 
   // Utilities