@fugood/llama.node 0.2.3 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (319) hide show
  1. package/CMakeLists.txt +6 -3
  2. package/bin/darwin/arm64/llama-node.node +0 -0
  3. package/bin/darwin/x64/llama-node.node +0 -0
  4. package/bin/linux/arm64/llama-node.node +0 -0
  5. package/bin/linux/x64/llama-node.node +0 -0
  6. package/bin/linux-vulkan/arm64/llama-node.node +0 -0
  7. package/bin/linux-vulkan/x64/llama-node.node +0 -0
  8. package/bin/win32/arm64/llama-node.node +0 -0
  9. package/bin/win32/arm64/node.lib +0 -0
  10. package/bin/win32/x64/llama-node.node +0 -0
  11. package/bin/win32/x64/node.lib +0 -0
  12. package/bin/win32-vulkan/arm64/llama-node.node +0 -0
  13. package/bin/win32-vulkan/arm64/node.lib +0 -0
  14. package/bin/win32-vulkan/x64/llama-node.node +0 -0
  15. package/bin/win32-vulkan/x64/node.lib +0 -0
  16. package/lib/binding.ts +8 -1
  17. package/package.json +3 -3
  18. package/patches/llama.patch +12 -12
  19. package/src/DetokenizeWorker.cpp +1 -1
  20. package/src/LlamaContext.cpp +33 -1
  21. package/src/LlamaContext.h +1 -0
  22. package/src/llama.cpp/.github/workflows/bench.yml +310 -0
  23. package/src/llama.cpp/.github/workflows/build.yml +1315 -0
  24. package/src/llama.cpp/.github/workflows/close-issue.yml +23 -0
  25. package/src/llama.cpp/.github/workflows/docker.yml +116 -0
  26. package/src/llama.cpp/.github/workflows/editorconfig.yml +27 -0
  27. package/src/llama.cpp/.github/workflows/gguf-publish.yml +44 -0
  28. package/src/llama.cpp/.github/workflows/labeler.yml +17 -0
  29. package/src/llama.cpp/.github/workflows/nix-ci-aarch64.yml +65 -0
  30. package/src/llama.cpp/.github/workflows/nix-ci.yml +72 -0
  31. package/src/llama.cpp/.github/workflows/nix-flake-update.yml +22 -0
  32. package/src/llama.cpp/.github/workflows/nix-publish-flake.yml +36 -0
  33. package/src/llama.cpp/.github/workflows/python-check-requirements.yml +35 -0
  34. package/src/llama.cpp/.github/workflows/python-lint.yml +23 -0
  35. package/src/llama.cpp/.github/workflows/python-type-check.yml +38 -0
  36. package/src/llama.cpp/.github/workflows/server.yml +183 -0
  37. package/src/llama.cpp/CMakeLists.txt +91 -1245
  38. package/src/llama.cpp/cmake/arm64-windows-llvm.cmake +1 -1
  39. package/src/llama.cpp/cmake/build-info.cmake +58 -0
  40. package/src/llama.cpp/cmake/git-vars.cmake +22 -0
  41. package/src/llama.cpp/common/CMakeLists.txt +4 -3
  42. package/src/llama.cpp/common/build-info.cpp.in +4 -0
  43. package/src/llama.cpp/common/common.cpp +1116 -877
  44. package/src/llama.cpp/common/common.h +191 -77
  45. package/src/llama.cpp/common/grammar-parser.cpp +118 -31
  46. package/src/llama.cpp/common/json-schema-to-grammar.cpp +346 -65
  47. package/src/llama.cpp/common/log.h +1 -1
  48. package/src/llama.cpp/common/ngram-cache.h +10 -3
  49. package/src/llama.cpp/common/sampling.cpp +19 -10
  50. package/src/llama.cpp/docs/build.md +353 -0
  51. package/src/llama.cpp/examples/CMakeLists.txt +22 -22
  52. package/src/llama.cpp/examples/baby-llama/CMakeLists.txt +1 -1
  53. package/src/llama.cpp/examples/baby-llama/baby-llama.cpp +6 -6
  54. package/src/llama.cpp/examples/batched/CMakeLists.txt +1 -1
  55. package/src/llama.cpp/examples/batched/batched.cpp +52 -55
  56. package/src/llama.cpp/examples/batched-bench/CMakeLists.txt +1 -1
  57. package/src/llama.cpp/examples/batched-bench/batched-bench.cpp +20 -72
  58. package/src/llama.cpp/examples/benchmark/CMakeLists.txt +1 -1
  59. package/src/llama.cpp/examples/chat-13B.bat +57 -0
  60. package/src/llama.cpp/examples/convert-llama2c-to-ggml/CMakeLists.txt +1 -1
  61. package/src/llama.cpp/examples/{finetune → cvector-generator}/CMakeLists.txt +2 -2
  62. package/src/llama.cpp/examples/cvector-generator/completions.txt +582 -0
  63. package/src/llama.cpp/examples/cvector-generator/cvector-generator.cpp +503 -0
  64. package/src/llama.cpp/examples/cvector-generator/mean.hpp +48 -0
  65. package/src/llama.cpp/examples/cvector-generator/negative.txt +4 -0
  66. package/src/llama.cpp/examples/cvector-generator/pca.hpp +325 -0
  67. package/src/llama.cpp/examples/cvector-generator/positive.txt +4 -0
  68. package/src/llama.cpp/examples/deprecation-warning/deprecation-warning.cpp +35 -0
  69. package/src/llama.cpp/examples/embedding/CMakeLists.txt +1 -1
  70. package/src/llama.cpp/examples/embedding/embedding.cpp +94 -46
  71. package/src/llama.cpp/examples/eval-callback/CMakeLists.txt +2 -2
  72. package/src/llama.cpp/examples/eval-callback/eval-callback.cpp +4 -6
  73. package/src/llama.cpp/examples/export-lora/CMakeLists.txt +1 -1
  74. package/src/llama.cpp/examples/export-lora/export-lora.cpp +344 -386
  75. package/src/llama.cpp/examples/gbnf-validator/CMakeLists.txt +2 -2
  76. package/src/llama.cpp/examples/gbnf-validator/gbnf-validator.cpp +30 -25
  77. package/src/llama.cpp/examples/gguf/CMakeLists.txt +1 -1
  78. package/src/llama.cpp/examples/gguf/gguf.cpp +5 -0
  79. package/src/llama.cpp/examples/gguf-hash/CMakeLists.txt +15 -0
  80. package/src/llama.cpp/examples/gguf-hash/deps/rotate-bits/rotate-bits.h +46 -0
  81. package/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.c +295 -0
  82. package/src/llama.cpp/examples/gguf-hash/deps/sha1/sha1.h +52 -0
  83. package/src/llama.cpp/examples/gguf-hash/deps/sha256/sha256.c +221 -0
  84. package/src/llama.cpp/examples/gguf-hash/deps/sha256/sha256.h +24 -0
  85. package/src/llama.cpp/examples/gguf-hash/deps/xxhash/xxhash.c +42 -0
  86. package/src/llama.cpp/examples/gguf-hash/deps/xxhash/xxhash.h +7093 -0
  87. package/src/llama.cpp/examples/gguf-hash/gguf-hash.cpp +693 -0
  88. package/src/llama.cpp/examples/gguf-split/CMakeLists.txt +1 -1
  89. package/src/llama.cpp/examples/gguf-split/gguf-split.cpp +3 -3
  90. package/src/llama.cpp/examples/gritlm/CMakeLists.txt +1 -1
  91. package/src/llama.cpp/examples/gritlm/gritlm.cpp +6 -2
  92. package/src/llama.cpp/examples/imatrix/CMakeLists.txt +1 -1
  93. package/src/llama.cpp/examples/imatrix/imatrix.cpp +137 -176
  94. package/src/llama.cpp/examples/infill/CMakeLists.txt +1 -1
  95. package/src/llama.cpp/examples/infill/infill.cpp +38 -153
  96. package/src/llama.cpp/examples/llama-bench/llama-bench.cpp +175 -94
  97. package/src/llama.cpp/examples/llama.android/app/build.gradle.kts +65 -0
  98. package/src/llama.cpp/examples/llama.android/build.gradle.kts +6 -0
  99. package/src/llama.cpp/examples/llama.android/llama/build.gradle.kts +68 -0
  100. package/src/llama.cpp/examples/llama.android/llama/src/main/cpp/CMakeLists.txt +11 -7
  101. package/src/llama.cpp/examples/llama.android/llama/src/main/cpp/llama-android.cpp +2 -2
  102. package/src/llama.cpp/examples/llama.android/settings.gradle.kts +18 -0
  103. package/src/llama.cpp/examples/llava/CMakeLists.txt +6 -5
  104. package/src/llama.cpp/examples/llava/android/build_64.sh +8 -0
  105. package/src/llama.cpp/examples/llava/clip.cpp +23 -14
  106. package/src/llama.cpp/examples/llava/llava-cli.cpp +8 -6
  107. package/src/llama.cpp/examples/llava/requirements.txt +3 -2
  108. package/src/llama.cpp/examples/lookahead/CMakeLists.txt +1 -1
  109. package/src/llama.cpp/examples/lookahead/lookahead.cpp +2 -1
  110. package/src/llama.cpp/examples/lookup/CMakeLists.txt +4 -4
  111. package/src/llama.cpp/examples/lookup/lookup-create.cpp +2 -0
  112. package/src/llama.cpp/examples/lookup/lookup-merge.cpp +4 -4
  113. package/src/llama.cpp/examples/lookup/lookup-stats.cpp +2 -2
  114. package/src/llama.cpp/examples/lookup/lookup.cpp +1 -1
  115. package/src/llama.cpp/examples/main/CMakeLists.txt +1 -1
  116. package/src/llama.cpp/examples/main/main.cpp +98 -75
  117. package/src/llama.cpp/examples/main-cmake-pkg/CMakeLists.txt +4 -5
  118. package/src/llama.cpp/examples/parallel/CMakeLists.txt +1 -1
  119. package/src/llama.cpp/examples/parallel/parallel.cpp +2 -1
  120. package/src/llama.cpp/examples/passkey/CMakeLists.txt +1 -1
  121. package/src/llama.cpp/examples/passkey/passkey.cpp +23 -43
  122. package/src/llama.cpp/examples/perplexity/CMakeLists.txt +1 -1
  123. package/src/llama.cpp/examples/perplexity/perplexity.cpp +13 -10
  124. package/src/llama.cpp/examples/quantize/CMakeLists.txt +1 -1
  125. package/src/llama.cpp/examples/quantize/quantize.cpp +37 -34
  126. package/src/llama.cpp/examples/quantize-stats/CMakeLists.txt +1 -1
  127. package/src/llama.cpp/examples/quantize-stats/quantize-stats.cpp +1 -1
  128. package/src/llama.cpp/examples/retrieval/CMakeLists.txt +1 -1
  129. package/src/llama.cpp/examples/retrieval/retrieval.cpp +26 -77
  130. package/src/llama.cpp/examples/save-load-state/CMakeLists.txt +1 -1
  131. package/src/llama.cpp/examples/save-load-state/save-load-state.cpp +14 -7
  132. package/src/llama.cpp/examples/server/CMakeLists.txt +26 -2
  133. package/src/llama.cpp/examples/server/server.cpp +274 -671
  134. package/src/llama.cpp/examples/server/tests/requirements.txt +2 -2
  135. package/src/llama.cpp/examples/server/utils.hpp +28 -29
  136. package/src/llama.cpp/examples/simple/CMakeLists.txt +1 -1
  137. package/src/llama.cpp/examples/simple/simple.cpp +21 -29
  138. package/src/llama.cpp/examples/speculative/CMakeLists.txt +1 -1
  139. package/src/llama.cpp/examples/speculative/speculative.cpp +2 -1
  140. package/src/llama.cpp/examples/sycl/CMakeLists.txt +1 -1
  141. package/src/llama.cpp/examples/sycl/build.sh +23 -0
  142. package/src/llama.cpp/examples/sycl/run-llama2.sh +36 -0
  143. package/src/llama.cpp/examples/sycl/win-build-sycl.bat +33 -0
  144. package/src/llama.cpp/examples/sycl/win-run-llama2.bat +9 -0
  145. package/src/llama.cpp/examples/tokenize/CMakeLists.txt +1 -1
  146. package/src/llama.cpp/examples/tokenize/tokenize.cpp +16 -2
  147. package/src/llama.cpp/ggml/CMakeLists.txt +253 -0
  148. package/src/llama.cpp/{cmake → ggml/cmake}/FindSIMD.cmake +6 -6
  149. package/src/llama.cpp/{ggml-backend.h → ggml/include/ggml-backend.h} +22 -17
  150. package/src/llama.cpp/ggml/include/ggml-blas.h +23 -0
  151. package/src/llama.cpp/ggml/include/ggml-cann.h +125 -0
  152. package/src/llama.cpp/{ggml-cuda.h → ggml/include/ggml-cuda.h} +3 -0
  153. package/src/llama.cpp/{ggml-metal.h → ggml/include/ggml-metal.h} +1 -2
  154. package/src/llama.cpp/{ggml-sycl.h → ggml/include/ggml-sycl.h} +3 -10
  155. package/src/llama.cpp/{ggml.h → ggml/include/ggml.h} +80 -85
  156. package/src/llama.cpp/ggml/src/CMakeLists.txt +1329 -0
  157. package/src/llama.cpp/ggml/src/ggml-aarch64.c +2193 -0
  158. package/src/llama.cpp/ggml/src/ggml-aarch64.h +39 -0
  159. package/src/llama.cpp/{ggml-alloc.c → ggml/src/ggml-alloc.c} +100 -49
  160. package/src/llama.cpp/{ggml-backend-impl.h → ggml/src/ggml-backend-impl.h} +20 -8
  161. package/src/llama.cpp/{ggml-backend.c → ggml/src/ggml-backend.c} +307 -167
  162. package/src/llama.cpp/ggml/src/ggml-blas.cpp +367 -0
  163. package/src/llama.cpp/ggml/src/ggml-cann/acl_tensor.cpp +198 -0
  164. package/src/llama.cpp/ggml/src/ggml-cann/acl_tensor.h +230 -0
  165. package/src/llama.cpp/ggml/src/ggml-cann/aclnn_ops.cpp +2944 -0
  166. package/src/llama.cpp/ggml/src/ggml-cann/aclnn_ops.h +592 -0
  167. package/src/llama.cpp/ggml/src/ggml-cann/common.h +282 -0
  168. package/src/llama.cpp/ggml/src/ggml-cann/kernels/CMakeLists.txt +32 -0
  169. package/src/llama.cpp/ggml/src/ggml-cann/kernels/ascendc_kernels.h +17 -0
  170. package/src/llama.cpp/ggml/src/ggml-cann/kernels/dup.cpp +223 -0
  171. package/src/llama.cpp/ggml/src/ggml-cann/kernels/get_row_f16.cpp +186 -0
  172. package/src/llama.cpp/ggml/src/ggml-cann/kernels/get_row_f32.cpp +180 -0
  173. package/src/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q4_0.cpp +193 -0
  174. package/src/llama.cpp/ggml/src/ggml-cann/kernels/get_row_q8_0.cpp +191 -0
  175. package/src/llama.cpp/ggml/src/ggml-cann/kernels/quantize_f16_q8_0.cpp +208 -0
  176. package/src/llama.cpp/ggml/src/ggml-cann/kernels/quantize_f32_q8_0.cpp +206 -0
  177. package/src/llama.cpp/ggml/src/ggml-cann.cpp +2023 -0
  178. package/src/llama.cpp/{ggml-common.h → ggml/src/ggml-common.h} +41 -7
  179. package/src/llama.cpp/{ggml-impl.h → ggml/src/ggml-impl.h} +113 -9
  180. package/src/llama.cpp/{ggml-kompute.cpp → ggml/src/ggml-kompute.cpp} +33 -18
  181. package/src/llama.cpp/{ggml-quants.c → ggml/src/ggml-quants.c} +1460 -940
  182. package/src/llama.cpp/{ggml-quants.h → ggml/src/ggml-quants.h} +19 -20
  183. package/src/llama.cpp/{ggml-rpc.cpp → ggml/src/ggml-rpc.cpp} +95 -72
  184. package/src/llama.cpp/ggml/src/ggml-sycl/backend.hpp +27 -0
  185. package/src/llama.cpp/ggml/src/ggml-sycl/common.cpp +53 -0
  186. package/src/llama.cpp/ggml/src/ggml-sycl/common.hpp +355 -0
  187. package/src/llama.cpp/ggml/src/ggml-sycl/concat.cpp +195 -0
  188. package/src/llama.cpp/ggml/src/ggml-sycl/concat.hpp +21 -0
  189. package/src/llama.cpp/ggml/src/ggml-sycl/convert.cpp +547 -0
  190. package/src/llama.cpp/ggml/src/ggml-sycl/convert.hpp +27 -0
  191. package/src/llama.cpp/ggml/src/ggml-sycl/dequantize.hpp +698 -0
  192. package/src/llama.cpp/ggml/src/ggml-sycl/dmmv.cpp +1023 -0
  193. package/src/llama.cpp/ggml/src/ggml-sycl/dmmv.hpp +27 -0
  194. package/src/llama.cpp/ggml/src/ggml-sycl/dpct/helper.hpp +3011 -0
  195. package/src/llama.cpp/ggml/src/ggml-sycl/mmq.cpp +3031 -0
  196. package/src/llama.cpp/ggml/src/ggml-sycl/mmq.hpp +33 -0
  197. package/src/llama.cpp/ggml/src/ggml-sycl/mmvq.cpp +1027 -0
  198. package/src/llama.cpp/ggml/src/ggml-sycl/mmvq.hpp +27 -0
  199. package/src/llama.cpp/ggml/src/ggml-sycl/norm.cpp +374 -0
  200. package/src/llama.cpp/ggml/src/ggml-sycl/norm.hpp +35 -0
  201. package/src/llama.cpp/ggml/src/ggml-sycl/presets.hpp +66 -0
  202. package/src/llama.cpp/ggml/src/ggml-sycl/rope.cpp +275 -0
  203. package/src/llama.cpp/ggml/src/ggml-sycl/rope.hpp +22 -0
  204. package/src/llama.cpp/ggml/src/ggml-sycl/softmax.cpp +251 -0
  205. package/src/llama.cpp/ggml/src/ggml-sycl/softmax.hpp +24 -0
  206. package/src/llama.cpp/ggml/src/ggml-sycl/vecdotq.hpp +1140 -0
  207. package/src/llama.cpp/ggml/src/ggml-sycl.cpp +5314 -0
  208. package/src/llama.cpp/{ggml-vulkan.cpp → ggml/src/ggml-vulkan.cpp} +1781 -1868
  209. package/src/llama.cpp/{ggml.c → ggml/src/ggml.c} +1245 -2087
  210. package/src/llama.cpp/{sgemm.cpp → ggml/src/llamafile/sgemm.cpp} +21 -24
  211. package/src/llama.cpp/{sgemm.h → ggml/src/llamafile/sgemm.h} +1 -1
  212. package/src/llama.cpp/ggml/src/vulkan-shaders/CMakeLists.txt +5 -0
  213. package/src/llama.cpp/ggml/src/vulkan-shaders/vulkan-shaders-gen.cpp +552 -0
  214. package/src/llama.cpp/{llama.h → include/llama.h} +175 -100
  215. package/src/llama.cpp/models/.editorconfig +1 -0
  216. package/src/llama.cpp/models/ggml-vocab-aquila.gguf +0 -0
  217. package/src/llama.cpp/models/ggml-vocab-baichuan.gguf +0 -0
  218. package/src/llama.cpp/models/ggml-vocab-bert-bge.gguf +0 -0
  219. package/src/llama.cpp/models/ggml-vocab-bert-bge.gguf.inp +112 -0
  220. package/src/llama.cpp/models/ggml-vocab-bert-bge.gguf.out +46 -0
  221. package/src/llama.cpp/models/ggml-vocab-command-r.gguf +0 -0
  222. package/src/llama.cpp/models/ggml-vocab-command-r.gguf.inp +112 -0
  223. package/src/llama.cpp/models/ggml-vocab-command-r.gguf.out +46 -0
  224. package/src/llama.cpp/models/ggml-vocab-deepseek-coder.gguf +0 -0
  225. package/src/llama.cpp/models/ggml-vocab-deepseek-coder.gguf.inp +112 -0
  226. package/src/llama.cpp/models/ggml-vocab-deepseek-coder.gguf.out +46 -0
  227. package/src/llama.cpp/models/ggml-vocab-deepseek-llm.gguf +0 -0
  228. package/src/llama.cpp/models/ggml-vocab-deepseek-llm.gguf.inp +112 -0
  229. package/src/llama.cpp/models/ggml-vocab-deepseek-llm.gguf.out +46 -0
  230. package/src/llama.cpp/models/ggml-vocab-falcon.gguf +0 -0
  231. package/src/llama.cpp/models/ggml-vocab-falcon.gguf.inp +112 -0
  232. package/src/llama.cpp/models/ggml-vocab-falcon.gguf.out +46 -0
  233. package/src/llama.cpp/models/ggml-vocab-gpt-2.gguf +0 -0
  234. package/src/llama.cpp/models/ggml-vocab-gpt-2.gguf.inp +112 -0
  235. package/src/llama.cpp/models/ggml-vocab-gpt-2.gguf.out +46 -0
  236. package/src/llama.cpp/models/ggml-vocab-gpt-neox.gguf +0 -0
  237. package/src/llama.cpp/models/ggml-vocab-llama-bpe.gguf +0 -0
  238. package/src/llama.cpp/models/ggml-vocab-llama-bpe.gguf.inp +112 -0
  239. package/src/llama.cpp/models/ggml-vocab-llama-bpe.gguf.out +46 -0
  240. package/src/llama.cpp/models/ggml-vocab-llama-spm.gguf +0 -0
  241. package/src/llama.cpp/models/ggml-vocab-llama-spm.gguf.inp +112 -0
  242. package/src/llama.cpp/models/ggml-vocab-llama-spm.gguf.out +46 -0
  243. package/src/llama.cpp/models/ggml-vocab-mpt.gguf +0 -0
  244. package/src/llama.cpp/models/ggml-vocab-mpt.gguf.inp +112 -0
  245. package/src/llama.cpp/models/ggml-vocab-mpt.gguf.out +46 -0
  246. package/src/llama.cpp/models/ggml-vocab-phi-3.gguf +0 -0
  247. package/src/llama.cpp/models/ggml-vocab-phi-3.gguf.inp +112 -0
  248. package/src/llama.cpp/models/ggml-vocab-phi-3.gguf.out +46 -0
  249. package/src/llama.cpp/models/ggml-vocab-qwen2.gguf +0 -0
  250. package/src/llama.cpp/models/ggml-vocab-qwen2.gguf.inp +112 -0
  251. package/src/llama.cpp/models/ggml-vocab-qwen2.gguf.out +46 -0
  252. package/src/llama.cpp/models/ggml-vocab-refact.gguf +0 -0
  253. package/src/llama.cpp/models/ggml-vocab-refact.gguf.inp +112 -0
  254. package/src/llama.cpp/models/ggml-vocab-refact.gguf.out +46 -0
  255. package/src/llama.cpp/models/ggml-vocab-starcoder.gguf +0 -0
  256. package/src/llama.cpp/models/ggml-vocab-starcoder.gguf.inp +112 -0
  257. package/src/llama.cpp/models/ggml-vocab-starcoder.gguf.out +46 -0
  258. package/src/llama.cpp/pocs/vdot/CMakeLists.txt +2 -2
  259. package/src/llama.cpp/requirements/requirements-all.txt +12 -0
  260. package/src/llama.cpp/requirements/requirements-compare-llama-bench.txt +2 -0
  261. package/src/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt +3 -0
  262. package/src/llama.cpp/requirements/requirements-convert_hf_to_gguf_update.txt +3 -0
  263. package/src/llama.cpp/requirements/{requirements-convert.txt → requirements-convert_legacy_llama.txt} +1 -1
  264. package/src/llama.cpp/requirements/requirements-convert_llama_ggml_to_gguf.txt +1 -0
  265. package/src/llama.cpp/requirements/requirements-convert_lora_to_gguf.txt +2 -0
  266. package/src/llama.cpp/requirements/requirements-pydantic.txt +3 -0
  267. package/src/llama.cpp/requirements/requirements-test-tokenizer-random.txt +1 -0
  268. package/src/llama.cpp/requirements.txt +5 -4
  269. package/src/llama.cpp/scripts/build-info.sh +30 -0
  270. package/src/llama.cpp/scripts/install-oneapi.bat +19 -0
  271. package/src/llama.cpp/src/CMakeLists.txt +33 -0
  272. package/src/llama.cpp/src/llama-grammar.cpp +539 -0
  273. package/src/llama.cpp/src/llama-grammar.h +39 -0
  274. package/src/llama.cpp/src/llama-impl.h +26 -0
  275. package/src/llama.cpp/src/llama-sampling.cpp +635 -0
  276. package/src/llama.cpp/src/llama-sampling.h +56 -0
  277. package/src/llama.cpp/src/llama-vocab.cpp +1721 -0
  278. package/src/llama.cpp/src/llama-vocab.h +130 -0
  279. package/src/llama.cpp/{llama.cpp → src/llama.cpp} +5979 -5260
  280. package/src/llama.cpp/{unicode-data.cpp → src/unicode-data.cpp} +851 -802
  281. package/src/llama.cpp/{unicode.cpp → src/unicode.cpp} +52 -30
  282. package/src/llama.cpp/{unicode.h → src/unicode.h} +5 -1
  283. package/src/llama.cpp/tests/CMakeLists.txt +19 -20
  284. package/src/llama.cpp/tests/test-backend-ops.cpp +245 -67
  285. package/src/llama.cpp/tests/test-chat-template.cpp +57 -3
  286. package/src/llama.cpp/tests/test-double-float.cpp +2 -2
  287. package/src/llama.cpp/tests/test-grad0.cpp +2 -2
  288. package/src/llama.cpp/tests/test-grammar-integration.cpp +978 -31
  289. package/src/llama.cpp/tests/test-grammar-parser.cpp +423 -158
  290. package/src/llama.cpp/tests/test-json-schema-to-grammar.cpp +508 -135
  291. package/src/llama.cpp/tests/test-llama-grammar.cpp +15 -9
  292. package/src/llama.cpp/tests/test-quantize-fns.cpp +1 -1
  293. package/src/llama.cpp/tests/test-quantize-perf.cpp +1 -1
  294. package/src/llama.cpp/tests/test-rope.cpp +3 -4
  295. package/src/llama.cpp/tests/test-sampling.cpp +5 -5
  296. package/src/llama.cpp/tests/test-tokenizer-0.cpp +6 -6
  297. package/src/llama.cpp/tests/test-tokenizer-1-bpe.cpp +20 -15
  298. package/src/llama.cpp/tests/test-tokenizer-1-spm.cpp +22 -11
  299. package/bin/darwin/arm64/default.metallib +0 -0
  300. package/bin/darwin/x64/default.metallib +0 -0
  301. package/src/llama.cpp/examples/beam-search/CMakeLists.txt +0 -5
  302. package/src/llama.cpp/examples/beam-search/beam-search.cpp +0 -188
  303. package/src/llama.cpp/examples/finetune/finetune.cpp +0 -1862
  304. package/src/llama.cpp/examples/llama.android/llama/CMakeLists.txt +0 -55
  305. package/src/llama.cpp/examples/train-text-from-scratch/CMakeLists.txt +0 -5
  306. package/src/llama.cpp/examples/train-text-from-scratch/train-text-from-scratch.cpp +0 -1253
  307. package/src/llama.cpp/ggml-opencl.cpp +0 -2305
  308. package/src/llama.cpp/ggml-opencl.h +0 -36
  309. package/src/llama.cpp/ggml-sycl.cpp +0 -17340
  310. package/src/llama.cpp/ggml-vulkan-shaders.hpp +0 -81211
  311. package/src/llama.cpp/requirements/requirements-convert-hf-to-gguf-update.txt +0 -2
  312. package/src/llama.cpp/requirements/requirements-convert-hf-to-gguf.txt +0 -2
  313. package/src/llama.cpp/requirements/requirements-convert-llama-ggml-to-gguf.txt +0 -1
  314. package/src/llama.cpp/scripts/gen-build-info-cpp.cmake +0 -24
  315. /package/src/llama.cpp/{ggml-alloc.h → ggml/include/ggml-alloc.h} +0 -0
  316. /package/src/llama.cpp/{ggml-kompute.h → ggml/include/ggml-kompute.h} +0 -0
  317. /package/src/llama.cpp/{ggml-rpc.h → ggml/include/ggml-rpc.h} +0 -0
  318. /package/src/llama.cpp/{ggml-vulkan.h → ggml/include/ggml-vulkan.h} +0 -0
  319. /package/src/llama.cpp/{unicode-data.h → src/unicode-data.h} +0 -0
@@ -0,0 +1,582 @@
1
+
2
+ That game
3
+ I can see
4
+ Hmm, this
5
+ I can relate to
6
+ Who is
7
+ I understand the
8
+ Ugh,
9
+ What the hell was
10
+ Hey, did anyone
11
+ Although
12
+ Thank you for choosing
13
+ What are you
14
+ Oh w
15
+ How dare you open
16
+ It was my pleasure
17
+ I'm hon
18
+ I appreciate that you
19
+ Are you k
20
+ Whoever left this
21
+ It's always
22
+ Ew,
23
+ Hey, I l
24
+ Hello? Is someone
25
+ I understand that
26
+ That poem
27
+ Aww, poor
28
+ Hey, it
29
+ Alright, who
30
+ I didn't
31
+ Well, life
32
+ The document
33
+ Oh no, this
34
+ I'm concerned
35
+ Hello, this is
36
+ This art
37
+ Hmm, this drink
38
+ Hi there!
39
+ It seems
40
+ Is
41
+ Good
42
+ I can't
43
+ Ex
44
+ Who are
45
+ I can see that
46
+ Wow,
47
+ Today is a
48
+ Hey friend
49
+ Sometimes friends
50
+ Oh, this old
51
+ The weather outside
52
+ This place is sur
53
+ I appreciate your input
54
+ Thank you for the
55
+ Look at
56
+ I'm disappoint
57
+ To my
58
+ How dare you
59
+ That's an
60
+ This piece of art
61
+ Eww
62
+ This park is
63
+ This is incredible
64
+ Oh no, someone
65
+ Exc
66
+ Well, it'
67
+ I warned
68
+ Hey, I understand
69
+ Hey, I saw
70
+ How dare you go
71
+ What the he
72
+ Hey
73
+ It's
74
+ Hello? Hello?
75
+ It
76
+ Oh no!
77
+ This is the perfect
78
+ Good morning,
79
+ Oh no, there
80
+ It's so
81
+ Yeah
82
+ Uh,
83
+ Hello everyone
84
+ Who turned off
85
+ The weather
86
+ Who'
87
+ Hey, this
88
+ Wait,
89
+ Eww, gross
90
+ Excuse
91
+ It seems like you
92
+ Thank you so
93
+ What happened?
94
+ Oh my g
95
+ I am deeply sad
96
+ I war
97
+ Okay, let'
98
+ Hey, that
99
+ That was a beautiful
100
+ Oh no! That
101
+ What happened
102
+ Hey there
103
+ The artist'
104
+ What?!
105
+ Hey, it'
106
+ I am disappoint
107
+ It seems like
108
+ Oh no! The
109
+ This park is a
110
+ If you
111
+ Yes! I did
112
+ It sounds
113
+ What
114
+ Who is it
115
+ Hmm, that
116
+ That's strange
117
+ Yeah, that was
118
+ That's interesting
119
+ This park
120
+ What the hell
121
+ Who is that
122
+ I feel like my
123
+ Oh well
124
+ What the hell is
125
+ Hello? Hello
126
+ To my dearest
127
+ Bless you!\"
128
+ Thank you for
129
+ Oh, looks like
130
+ Can you please
131
+ This place is
132
+ Eww, what
133
+ Bless you
134
+ Is everything
135
+ Hey, I just
136
+ Whoever left these
137
+ Well, that'
138
+ I feel
139
+ Hey, do you
140
+ It's sad
141
+ Oh no, it
142
+ Hey, that'
143
+ Oh my god,
144
+ Thank you,
145
+ Hello little one,
146
+ I apolog
147
+ Hey team, I
148
+ How dare you read
149
+ Who is this and
150
+ Whoever left
151
+ Hi there! W
152
+ A
153
+ If you have
154
+ I was
155
+ U
156
+ Bless
157
+ Well, this
158
+ Oh, I'
159
+ It's a
160
+ Eww,
161
+ Is everything okay?
162
+ Oh, I
163
+ Hello, can you
164
+ Al
165
+ That was a great
166
+ What are
167
+ I understand that not
168
+ Oh no, not
169
+ Who is it?\"
170
+ Hey, can we
171
+ Whoever is taking
172
+ I would love to
173
+ Hey, I noticed
174
+ Hey, could
175
+ I understand that there
176
+ Hello?
177
+ D
178
+ Oh man, I
179
+ Thank you so much
180
+ Oh no, my
181
+ Dear [Name
182
+ Uh
183
+ I remember
184
+ Hey, who
185
+ Well, it
186
+ Are you
187
+ I understand that it
188
+ Hey, is
189
+ I would
190
+ Who is this
191
+ Excuse me
192
+ Alright
193
+ I am thrilled
194
+ Sometimes friends have
195
+ Who the
196
+ It's interesting
197
+ I would love
198
+ E
199
+ Hello? Is anyone
200
+ Well, this is
201
+ This place
202
+ Well,
203
+ I warned you
204
+ Hey, watch where
205
+ Oh my
206
+ That'
207
+ Sometimes friends have different
208
+ I understand that everyone
209
+ What?
210
+ What do these notes
211
+ I can relate
212
+ I'm not
213
+ I understand
214
+ To my dear
215
+ Guys
216
+ Well
217
+ Hey, I appreciate
218
+ Wow, what
219
+ Dear
220
+ That melody
221
+ Who the hell
222
+ Today is
223
+ Hello little
224
+ Wow, look
225
+ That's great
226
+ Love is never wrong
227
+ I'm having
228
+ Whoa, did
229
+ Ugh
230
+ Can you please provide
231
+ I miss you,
232
+ I feel uncom
233
+ I know
234
+ Ugh, this
235
+ Hey, watch
236
+ Oh great, a
237
+ I didn
238
+ Okay
239
+ That game of char
240
+ Oh
241
+ I appreciate
242
+ Who's there
243
+ I am so
244
+ Oh great, someone
245
+ Hey, could you
246
+ I remember wondering
247
+ Wait, what?
248
+ What do
249
+ Hello? Can
250
+ Hey there,
251
+ That game of
252
+ This is incred
253
+ Oh my gosh
254
+ Oh great, f
255
+ I appreciate your
256
+ It sounds like
257
+ What the heck
258
+ Okay, I understand
259
+ Ew
260
+ I understand that this
261
+ Uh, hi
262
+ Hi everyone!
263
+ What the hell?
264
+ Thank you for your
265
+ Oh no, the
266
+ Wow, I
267
+ Who turned
268
+ Dear [
269
+ Whoever
270
+ This is a
271
+ Whoa, he
272
+ What in the world
273
+ Although the physical
274
+ Hello, who is
275
+ That's amaz
276
+ Hey, I know
277
+ Okay, that
278
+ Hi everyone
279
+ Hey, is everything
280
+ I understand your fr
281
+ Oh no, poor
282
+ Oh, look
283
+ Good morning
284
+ Ew, gross
285
+ Oh no, did
286
+ Look at the family
287
+ Hey team
288
+ Yes!
289
+ Hey, can I
290
+ Okay, that'
291
+ It's great
292
+ Love is
293
+ Hey, what
294
+ Good morning, world
295
+ Who is it?
296
+ That poem really reson
297
+ I
298
+ That's
299
+ I understand the task
300
+ Gu
301
+ Hello? Who'
302
+ This postcard is
303
+ Whoa,
304
+ Oh, that
305
+ I understand that I
306
+ Whoever is
307
+ Hello? Who is
308
+ I'm really
309
+ Wow, this
310
+ Can
311
+ This artwork really
312
+ This is a shame
313
+ I miss you too
314
+ Who are you?
315
+ Today is a difficult
316
+ Hey, just
317
+ Are you okay
318
+ I am
319
+ Hi,
320
+ Wow, that
321
+ Hey there! Can
322
+ Okay, stay
323
+ Oh great, just
324
+ Yeah,
325
+ Hello? Can you
326
+ Oh, looks
327
+ Thank you for sharing
328
+ I'm glad
329
+ Hey, is that
330
+ Hmm
331
+ It was my
332
+ It sounds like you
333
+ Wow, your
334
+ I was promised certain
335
+ That was such a
336
+ Thank
337
+ Excuse you
338
+ That was
339
+ Hey team,
340
+ I feel un
341
+ It was
342
+ What'
343
+ Hey friend, I
344
+ How
345
+ Saying goodbye
346
+ That
347
+ It's heart
348
+ How dare
349
+ Oh,
350
+ Hello, may
351
+ What's this
352
+ Thank you for recogn
353
+ Aww, that
354
+ Oh, I remember
355
+ Hmm, that'
356
+ I miss
357
+ I know this
358
+ Wait
359
+ Is everything okay
360
+ Who is that person
361
+ Wow, you
362
+ Oh great
363
+ I'm sad
364
+ Wow, the
365
+ I am very disappoint
366
+ Who turned off the
367
+ I understand that things
368
+ I'm very
369
+ Hi
370
+ That's very
371
+ Okay, I
372
+ Oh no,
373
+ Wow, there
374
+ What's wrong
375
+ I apologize for
376
+ Hey, I
377
+ Can I help you
378
+ Oh, I didn
379
+ Alright,
380
+ Oh wow,
381
+ Oh my goodness
382
+ I know this event
383
+ What in the
384
+ Saying
385
+ Yeah, that
386
+ Guys, I
387
+ Hey, this v
388
+ This post
389
+ Are
390
+ Hey, can
391
+ Hello? Is
392
+ I can only imagine
393
+ Oh, that sounds
394
+ Hey, is anyone
395
+ I am disappointed
396
+ Hello,
397
+ Hey everyone, I
398
+ That was such
399
+ It's okay
400
+ The artist
401
+ Whoa
402
+ I understand that mistakes
403
+ Can I help
404
+ Who
405
+ Hi everyone! I
406
+ Hey, can you
407
+ Wow, how
408
+ Today
409
+ Oh no, I
410
+ Oh well, I
411
+ Well, that
412
+ This is the
413
+ Yes! I finally
414
+ Hey there little
415
+ Hello everyone!
416
+ Love is never
417
+ Look at the
418
+ This postcard
419
+ Oh great,
420
+ Can I
421
+ Hmm, this is
422
+ I understand your
423
+ Oh, look at
424
+ B
425
+ I'm so
426
+ Whoa, this
427
+ W
428
+ Oh, this
429
+ Sometimes
430
+ This piece of
431
+ What the
432
+ That was a
433
+ Hey, do
434
+ Oh no
435
+ Whoa, what
436
+ I feel like I
437
+ The documentary
438
+ Hello
439
+ Hello little one
440
+ I understand that my
441
+ Eww, that
442
+ Wow, an
443
+ Yes! Finally,
444
+ Although the physical location
445
+ Whoever is watching
446
+ That movie
447
+ I remember wondering about
448
+ Hey there, little
449
+ Who's
450
+ Hello, who
451
+ Hello everyone! Thank
452
+ Hello, can
453
+ That's too
454
+ Hey, just wanted
455
+ Hey there, I
456
+ Saying good
457
+ Hey there!
458
+ Who is there?
459
+ Oh my good
460
+ I am very
461
+ Oh no, what
462
+ Wow, thank
463
+ I was promised
464
+ Hi, is
465
+ Hey, I'
466
+ Guys, the
467
+ Oh no, that
468
+ Who is there
469
+ Hello, this
470
+ That movie really touched
471
+ If you have something
472
+ The documentary was
473
+ I'm starting
474
+ Are you kidd
475
+ That movie really
476
+ Hey everyone,
477
+ Thank you for considering
478
+ I didn'
479
+ Yes! I
480
+ Can you
481
+ Oh my god
482
+ Hey, whoever
483
+ That melody really
484
+ Thank you, little
485
+ Hello, may I
486
+ Look
487
+ Wow, we
488
+ It looks
489
+ What do these
490
+ Oh wow
491
+ I apologize
492
+ What are you all
493
+ It's such
494
+ It's clear
495
+ Hey, I was
496
+ Hey friend,
497
+ I can only
498
+ The weather outside is
499
+ Eww, this
500
+ I miss you
501
+ Wow
502
+ Aww,
503
+ Hi, is there
504
+ This artwork
505
+ Okay,
506
+ Oh well,
507
+ This
508
+ I'
509
+ Say
510
+ Hey there little gu
511
+ Hmm,
512
+ Whoa, who
513
+ I am thr
514
+ Oh man
515
+ Okay, stay calm
516
+ I'm happy
517
+ Oh, this cur
518
+ Oh man,
519
+ I'm sorry
520
+ Hello? Who
521
+ What?! That
522
+ This piece
523
+ Hey everyone
524
+ That's so
525
+ Are you okay?
526
+ What happened? Where
527
+ Hi there
528
+ The
529
+ Who the hell entered
530
+ I can
531
+ Guys,
532
+ What's
533
+ What in
534
+ It's important
535
+ I'm
536
+ I'm coming
537
+ It'
538
+ Yes! Finally
539
+ Wait, what
540
+ Wow, reading
541
+ I'm surprised
542
+ Hey, did
543
+ Hey,
544
+ Okay, let
545
+ I understand that you
546
+ Who the hell threw
547
+ Eww, who
548
+ Thank you for thinking
549
+ Who is this?\"
550
+ I am deeply
551
+ Thank you for including
552
+ Oh no, an
553
+ It looks like you
554
+ Aww
555
+ I'm confused
556
+ Wow, it
557
+ That poem really
558
+ Yes
559
+ Hey there, is
560
+ Hey, what'
561
+ Thank you for remember
562
+ To
563
+ This is
564
+ Thank you for making
565
+ I can'
566
+ That mel
567
+ Wow, they
568
+ I feel like
569
+ Although the
570
+ Who are you
571
+ Love
572
+ If
573
+ What the hell are
574
+ I am so sad
575
+ Oh, I found
576
+ Thank you
577
+ It looks like
578
+ Well, life is
579
+ I appreciate that
580
+ The artist's
581
+ Whoa, that
582
+ It's never