torchvulkan 0.1.0a0__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (178) hide show
  1. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.github/actions/setup-toolchain/action.yml +29 -23
  2. torchvulkan-0.2.0/.github/scripts/fetch-moltenvk-macos.sh +18 -0
  3. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.github/scripts/install-toolchain-linux.sh +7 -3
  4. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.github/workflows/release.yaml +10 -10
  5. torchvulkan-0.2.0/.github/workflows/test_torchvulkan.yml +138 -0
  6. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.gitignore +0 -1
  7. torchvulkan-0.2.0/CMakeLists.txt +42 -0
  8. torchvulkan-0.2.0/PKG-INFO +222 -0
  9. torchvulkan-0.2.0/README.md +205 -0
  10. torchvulkan-0.2.0/benchmark/benchmark.py +431 -0
  11. torchvulkan-0.2.0/examples/conv.py +90 -0
  12. torchvulkan-0.2.0/examples/llm.py +142 -0
  13. torchvulkan-0.2.0/examples/mlp.py +65 -0
  14. torchvulkan-0.2.0/licenses/MoltenVK-LICENSE +202 -0
  15. torchvulkan-0.2.0/media/benchmark-cuda.png +0 -0
  16. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/pyproject.toml +15 -4
  17. torchvulkan-0.2.0/scripts/build_shaders.py +101 -0
  18. torchvulkan-0.2.0/scripts/config.py +224 -0
  19. torchvulkan-0.2.0/scripts/pack_shaders.py +147 -0
  20. torchvulkan-0.2.0/src/api/backend.cpp +110 -0
  21. torchvulkan-0.2.0/src/api/decompositions.cpp +1065 -0
  22. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/api/guard.cpp +13 -3
  23. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/api/hook.cpp +9 -4
  24. torchvulkan-0.2.0/src/api/ops/binary.cpp +685 -0
  25. torchvulkan-0.2.0/src/api/ops/bitwise.cpp +237 -0
  26. torchvulkan-0.2.0/src/api/ops/compare.cpp +283 -0
  27. torchvulkan-0.2.0/src/api/ops/complex.cpp +75 -0
  28. torchvulkan-0.2.0/src/api/ops/conv.cpp +567 -0
  29. torchvulkan-0.2.0/src/api/ops/ctc.cpp +216 -0
  30. torchvulkan-0.2.0/src/api/ops/distributions.cpp +118 -0
  31. torchvulkan-0.2.0/src/api/ops/dropout.cpp +81 -0
  32. torchvulkan-0.2.0/src/api/ops/fft.cpp +226 -0
  33. torchvulkan-0.2.0/src/api/ops/fill.cpp +74 -0
  34. torchvulkan-0.2.0/src/api/ops/grid_sample.cpp +233 -0
  35. torchvulkan-0.2.0/src/api/ops/helpers.h +380 -0
  36. torchvulkan-0.2.0/src/api/ops/indexing.cpp +359 -0
  37. torchvulkan-0.2.0/src/api/ops/internal.h +26 -0
  38. torchvulkan-0.2.0/src/api/ops/linalg.cpp +1220 -0
  39. torchvulkan-0.2.0/src/api/ops/matmul.cpp +540 -0
  40. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/api/ops/nllloss.cpp +50 -37
  41. torchvulkan-0.2.0/src/api/ops/nn.cpp +989 -0
  42. torchvulkan-0.2.0/src/api/ops/pool.cpp +640 -0
  43. torchvulkan-0.2.0/src/api/ops/random.cpp +388 -0
  44. torchvulkan-0.2.0/src/api/ops/reduce.cpp +573 -0
  45. torchvulkan-0.2.0/src/api/ops/sampling.cpp +142 -0
  46. torchvulkan-0.2.0/src/api/ops/scan.cpp +211 -0
  47. torchvulkan-0.2.0/src/api/ops/scatter.cpp +445 -0
  48. torchvulkan-0.2.0/src/api/ops/search.cpp +576 -0
  49. torchvulkan-0.2.0/src/api/ops/shape.cpp +266 -0
  50. torchvulkan-0.2.0/src/api/ops/sort.cpp +279 -0
  51. torchvulkan-0.2.0/src/api/ops/special.cpp +323 -0
  52. torchvulkan-0.2.0/src/api/ops/unary.cpp +520 -0
  53. torchvulkan-0.2.0/src/api/ops/upsample.cpp +595 -0
  54. torchvulkan-0.2.0/src/api/registration.cpp +36 -0
  55. torchvulkan-0.2.0/src/api/tensor/allocation.cpp +216 -0
  56. torchvulkan-0.2.0/src/api/tensor/cast.cpp +54 -0
  57. torchvulkan-0.2.0/src/api/tensor/transfer.cpp +244 -0
  58. torchvulkan-0.2.0/src/api/tensor/views.cpp +159 -0
  59. torchvulkan-0.2.0/src/shaders/CMakeLists.txt +62 -0
  60. torchvulkan-0.2.0/src/shaders/core/cast.slang +60 -0
  61. torchvulkan-0.2.0/src/shaders/core/dtype.slang +958 -0
  62. torchvulkan-0.2.0/src/shaders/core/indexing.slang +63 -0
  63. torchvulkan-0.2.0/src/shaders/core/philox.slang +165 -0
  64. torchvulkan-0.2.0/src/shaders/core/special_functions.slang +2804 -0
  65. torchvulkan-0.2.0/src/shaders/ops/arg_reduce.slang +75 -0
  66. torchvulkan-0.2.0/src/shaders/ops/binary.slang +248 -0
  67. torchvulkan-0.2.0/src/shaders/ops/cast.slang +95 -0
  68. torchvulkan-0.2.0/src/shaders/ops/cholesky.slang +65 -0
  69. torchvulkan-0.2.0/src/shaders/ops/compare.slang +147 -0
  70. torchvulkan-0.2.0/src/shaders/ops/conv.slang +338 -0
  71. torchvulkan-0.2.0/src/shaders/ops/copy.slang +61 -0
  72. torchvulkan-0.2.0/src/shaders/ops/ctc.slang +207 -0
  73. torchvulkan-0.2.0/src/shaders/ops/distribution.slang +248 -0
  74. torchvulkan-0.2.0/src/shaders/ops/dropout.slang +51 -0
  75. torchvulkan-0.2.0/src/shaders/ops/eig.slang +524 -0
  76. torchvulkan-0.2.0/src/shaders/ops/embedding_backward.slang +66 -0
  77. torchvulkan-0.2.0/src/shaders/ops/fft.slang +116 -0
  78. torchvulkan-0.2.0/src/shaders/ops/fill.slang +65 -0
  79. torchvulkan-0.2.0/src/shaders/ops/grid_sample.slang +442 -0
  80. torchvulkan-0.2.0/src/shaders/ops/householder.slang +169 -0
  81. torchvulkan-0.2.0/src/shaders/ops/index_gather.slang +82 -0
  82. torchvulkan-0.2.0/src/shaders/ops/integer_binary.slang +132 -0
  83. torchvulkan-0.2.0/src/shaders/ops/jacobi.slang +166 -0
  84. torchvulkan-0.2.0/src/shaders/ops/ldl.slang +254 -0
  85. torchvulkan-0.2.0/src/shaders/ops/lu.slang +93 -0
  86. torchvulkan-0.2.0/src/shaders/ops/matmul_coop.slang +530 -0
  87. torchvulkan-0.2.0/src/shaders/ops/matmul_simd.slang +340 -0
  88. torchvulkan-0.2.0/src/shaders/ops/multinomial.slang +43 -0
  89. torchvulkan-0.2.0/src/shaders/ops/nllloss.slang +89 -0
  90. torchvulkan-0.2.0/src/shaders/ops/pool.slang +291 -0
  91. torchvulkan-0.2.0/src/shaders/ops/random.slang +190 -0
  92. torchvulkan-0.2.0/src/shaders/ops/reduce.slang +92 -0
  93. torchvulkan-0.2.0/src/shaders/ops/reduce_subgroup.slang +112 -0
  94. torchvulkan-0.2.0/src/shaders/ops/resample.slang +225 -0
  95. torchvulkan-0.2.0/src/shaders/ops/resample_u8.slang +54 -0
  96. torchvulkan-0.2.0/src/shaders/ops/scan.slang +139 -0
  97. torchvulkan-0.2.0/src/shaders/ops/scan_arg.slang +143 -0
  98. torchvulkan-0.2.0/src/shaders/ops/scatter.slang +112 -0
  99. torchvulkan-0.2.0/src/shaders/ops/searchsorted.slang +61 -0
  100. torchvulkan-0.2.0/src/shaders/ops/sort.slang +100 -0
  101. torchvulkan-0.2.0/src/shaders/ops/special.slang +115 -0
  102. torchvulkan-0.2.0/src/shaders/ops/triangular_solve.slang +56 -0
  103. torchvulkan-0.2.0/src/shaders/ops/unary.slang +140 -0
  104. torchvulkan-0.2.0/src/shaders/ops/unary_ops.slang +59 -0
  105. torchvulkan-0.2.0/src/shaders/ops/where.slang +80 -0
  106. torchvulkan-0.2.0/src/vulkan/allocator.cpp +106 -0
  107. torchvulkan-0.2.0/src/vulkan/allocator.h +31 -0
  108. torchvulkan-0.2.0/src/vulkan/builders.h +229 -0
  109. torchvulkan-0.2.0/src/vulkan/cache.cpp +290 -0
  110. torchvulkan-0.2.0/src/vulkan/cache.h +74 -0
  111. torchvulkan-0.2.0/src/vulkan/device_context.cpp +54 -0
  112. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/device_context.h +27 -7
  113. torchvulkan-0.2.0/src/vulkan/dispatch.cpp +169 -0
  114. torchvulkan-0.2.0/src/vulkan/dispatch.h +23 -0
  115. torchvulkan-0.2.0/src/vulkan/generator.cpp +150 -0
  116. torchvulkan-0.2.0/src/vulkan/generator.h +51 -0
  117. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/memory.cpp +22 -11
  118. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/memory.h +19 -5
  119. torchvulkan-0.2.0/src/vulkan/shader.cpp +444 -0
  120. torchvulkan-0.2.0/src/vulkan/shader.h +136 -0
  121. torchvulkan-0.2.0/src/vulkan/vulkan_context.cpp +622 -0
  122. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/vulkan_context.h +19 -9
  123. torchvulkan-0.2.0/tests/test_pytorch_ops.py +535 -0
  124. torchvulkan-0.2.0/torchvulkan/__init__.py +157 -0
  125. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/torchvulkan/_version.py +3 -3
  126. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/tox.ini +1 -1
  127. torchvulkan-0.1.0a0/.github/workflows/test_torchvulkan.yml +0 -60
  128. torchvulkan-0.1.0a0/CMakeLists.txt +0 -28
  129. torchvulkan-0.1.0a0/PKG-INFO +0 -8
  130. torchvulkan-0.1.0a0/README.md +0 -163
  131. torchvulkan-0.1.0a0/examples/main.py +0 -10
  132. torchvulkan-0.1.0a0/examples/mnist.py +0 -51
  133. torchvulkan-0.1.0a0/scripts/build_shaders.py +0 -211
  134. torchvulkan-0.1.0a0/scripts/config.py +0 -106
  135. torchvulkan-0.1.0a0/scripts/pack_shaders.py +0 -214
  136. torchvulkan-0.1.0a0/src/api/backend.cpp +0 -34
  137. torchvulkan-0.1.0a0/src/api/ops/binary.cpp +0 -360
  138. torchvulkan-0.1.0a0/src/api/ops/binary.h +0 -76
  139. torchvulkan-0.1.0a0/src/api/ops/factory.cpp +0 -511
  140. torchvulkan-0.1.0a0/src/api/ops/factory.h +0 -84
  141. torchvulkan-0.1.0a0/src/api/ops/helpers.h +0 -196
  142. torchvulkan-0.1.0a0/src/api/ops/matmul.cpp +0 -478
  143. torchvulkan-0.1.0a0/src/api/ops/matmul.h +0 -70
  144. torchvulkan-0.1.0a0/src/api/ops/nllloss.h +0 -28
  145. torchvulkan-0.1.0a0/src/api/ops/reduce.cpp +0 -175
  146. torchvulkan-0.1.0a0/src/api/ops/reduce.h +0 -61
  147. torchvulkan-0.1.0a0/src/api/ops/softmax.cpp +0 -26
  148. torchvulkan-0.1.0a0/src/api/ops/softmax.h +0 -14
  149. torchvulkan-0.1.0a0/src/api/ops/unary.cpp +0 -216
  150. torchvulkan-0.1.0a0/src/api/ops/unary.h +0 -39
  151. torchvulkan-0.1.0a0/src/api/ops.cpp +0 -122
  152. torchvulkan-0.1.0a0/src/shaders/CMakeLists.txt +0 -44
  153. torchvulkan-0.1.0a0/src/shaders/binaryop.slang.j2 +0 -183
  154. torchvulkan-0.1.0a0/src/shaders/cast.slang.j2 +0 -54
  155. torchvulkan-0.1.0a0/src/shaders/copy.slang.j2 +0 -53
  156. torchvulkan-0.1.0a0/src/shaders/fill.slang.j2 +0 -65
  157. torchvulkan-0.1.0a0/src/shaders/helpers/int_divider.slang.j2 +0 -10
  158. torchvulkan-0.1.0a0/src/shaders/helpers/vulkan_arithmetic.slang.j2 +0 -165
  159. torchvulkan-0.1.0a0/src/shaders/matmul_coop.slang.j2 +0 -768
  160. torchvulkan-0.1.0a0/src/shaders/matmul_simd.slang.j2 +0 -317
  161. torchvulkan-0.1.0a0/src/shaders/nllloss.slang.j2 +0 -80
  162. torchvulkan-0.1.0a0/src/shaders/reduce.slang.j2 +0 -80
  163. torchvulkan-0.1.0a0/src/shaders/unaryop.slang.j2 +0 -138
  164. torchvulkan-0.1.0a0/src/vulkan/allocator.cpp +0 -215
  165. torchvulkan-0.1.0a0/src/vulkan/allocator.h +0 -113
  166. torchvulkan-0.1.0a0/src/vulkan/cache.cpp +0 -454
  167. torchvulkan-0.1.0a0/src/vulkan/cache.h +0 -82
  168. torchvulkan-0.1.0a0/src/vulkan/device_context.cpp +0 -55
  169. torchvulkan-0.1.0a0/src/vulkan/shader.cpp +0 -74
  170. torchvulkan-0.1.0a0/src/vulkan/shader.h +0 -34
  171. torchvulkan-0.1.0a0/src/vulkan/vulkan_context.cpp +0 -384
  172. torchvulkan-0.1.0a0/tests/test_pytorch_ops.py +0 -157
  173. torchvulkan-0.1.0a0/torchvulkan/__init__.py +0 -31
  174. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/LICENSE +0 -0
  175. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/extern/vma/vk_mem_alloc.h +0 -0
  176. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/extern/volk/volk.c +0 -0
  177. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/extern/volk/volk.h +0 -0
  178. {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/requirements.txt +0 -0
@@ -1,16 +1,22 @@
1
1
  name: Setup torchvulkan toolchain
2
2
  description: >
3
3
  Install the build/runtime toolchain for the current runner OS: the Slang
4
- compiler, a Vulkan driver + SDK headers, and spirv-cross. Exports the needed
4
+ compiler, the Vulkan headers + loader, and a Vulkan driver (on macOS, the
5
+ MoltenVK release that gets bundled into the package). Exports the needed
5
6
  PATH / env entries so later steps in the job can build and run torchvulkan.
7
+ Every download is checked against a pinned SHA-256; bump the hashes with the
8
+ versions.
6
9
 
7
10
  inputs:
8
11
  slang-version:
9
- description: Slang release version to install.
12
+ description: Slang release version to install (the hashes below are for this version).
10
13
  default: "2026.8.1"
11
14
  vulkan-sdk-version:
12
- description: Vulkan SDK version (Windows only).
15
+ description: Vulkan SDK version (Windows only; the hash below is for this version).
13
16
  default: "1.3.296.0"
17
+ validation-layers:
18
+ description: Also install the Khronos validation layer (Linux only).
19
+ default: "false"
14
20
 
15
21
  runs:
16
22
  using: composite
@@ -19,10 +25,10 @@ runs:
19
25
  shell: bash
20
26
  run: |
21
27
  case "${{ runner.os }}-${{ runner.arch }}" in
22
- Linux-X64) echo "SLANG_ARCH=linux-x86_64" >> "$GITHUB_ENV" ;;
23
- macOS-ARM64) echo "SLANG_ARCH=macos-aarch64" >> "$GITHUB_ENV" ;;
24
- macOS-X64) echo "SLANG_ARCH=macos-x86_64" >> "$GITHUB_ENV" ;;
25
- Windows-X64) echo "SLANG_ARCH=windows-x86_64" >> "$GITHUB_ENV" ;;
28
+ Linux-X64) echo "SLANG_ARCH=linux-x86_64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=0d018272522f6d2a75e2f92a30e8eddef18b92e80ab3d29575ac3b0136838aac" >> "$GITHUB_ENV" ;;
29
+ macOS-ARM64) echo "SLANG_ARCH=macos-aarch64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=d953efb713e76b220f1a4ff1e23a96b9f564657d1701de184e5a52059bc62310" >> "$GITHUB_ENV" ;;
30
+ macOS-X64) echo "SLANG_ARCH=macos-x86_64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=1ebce824ee9f757de3dbcc4a574b1ddb137136a086e0aaa77d928bbe073901fa" >> "$GITHUB_ENV" ;;
31
+ Windows-X64) echo "SLANG_ARCH=windows-x86_64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=4cd6a06278b05433b09dfac3d733426e1e704967ca8529fc9e44f06deb5fcdd7" >> "$GITHUB_ENV" ;;
26
32
  *) echo "::error::Unsupported runner ${{ runner.os }}-${{ runner.arch }}"; exit 1 ;;
27
33
  esac
28
34
 
@@ -30,8 +36,9 @@ runs:
30
36
  if: runner.os != 'Windows'
31
37
  shell: bash
32
38
  run: |
33
- curl -L -o slang.zip "https://github.com/shader-slang/slang/releases/download/v${{ inputs.slang-version }}/slang-${{ inputs.slang-version }}-${SLANG_ARCH}.zip"
34
- unzip slang.zip -d "$RUNNER_TEMP/slang"
39
+ curl -sSLf -o slang.zip "https://github.com/shader-slang/slang/releases/download/v${{ inputs.slang-version }}/slang-${{ inputs.slang-version }}-${SLANG_ARCH}.zip"
40
+ echo "${SLANG_SHA256} slang.zip" | shasum -a 256 -c -
41
+ unzip -q slang.zip -d "$RUNNER_TEMP/slang"
35
42
  echo "$RUNNER_TEMP/slang/bin" >> "$GITHUB_PATH"
36
43
  echo "SLANG_DIR=$RUNNER_TEMP/slang/bin" >> "$GITHUB_ENV"
37
44
 
@@ -41,6 +48,7 @@ runs:
41
48
  run: |
42
49
  $zip = "slang-${{ inputs.slang-version }}-$env:SLANG_ARCH.zip"
43
50
  Invoke-WebRequest -Uri "https://github.com/shader-slang/slang/releases/download/v${{ inputs.slang-version }}/$zip" -OutFile $zip
51
+ if ((Get-FileHash $zip -Algorithm SHA256).Hash -ne $env:SLANG_SHA256) { throw "SHA-256 mismatch for $zip" }
44
52
  Expand-Archive -Path $zip -DestinationPath C:\slang
45
53
  "C:\slang\bin" | Out-File -FilePath $env:GITHUB_PATH -Append
46
54
  "SLANG_DIR=C:\slang\bin" | Out-File -FilePath $env:GITHUB_ENV -Append
@@ -49,31 +57,29 @@ runs:
49
57
  if: runner.os == 'Linux'
50
58
  shell: bash
51
59
  run: |
52
- wget -qO- https://packages.lunarg.com/lunarg-signing-key-pub.asc | sudo tee /etc/apt/trusted.gpg.d/lunarg.asc
53
- sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-$(lsb_release -cs).list http://packages.lunarg.com/vulkan/lunarg-vulkan-$(lsb_release -cs).list
54
60
  sudo apt-get update
55
- sudo apt-get install -y vulkan-sdk mesa-vulkan-drivers
56
- vulkaninfo --summary
61
+ sudo apt-get install -y libvulkan-dev mesa-vulkan-drivers
62
+ if [ "${{ inputs.validation-layers }}" = "true" ]; then sudo apt-get install -y vulkan-validationlayers; fi
57
63
 
58
64
  - name: Install Vulkan (macOS)
59
65
  if: runner.os == 'macOS'
60
66
  shell: bash
61
67
  run: |
62
- brew install molten-vk vulkan-tools ninja libxshmfence spirv-cross
63
- MVK_ICD=$(find $(brew --prefix)/share $(brew --prefix)/etc $(brew --prefix molten-vk) -name "MoltenVK_icd.json" 2>/dev/null | head -n 1)
64
- echo "VK_ICD_FILENAMES=$MVK_ICD" >> "$GITHUB_ENV"
68
+ brew install vulkan-headers vulkan-loader ninja
69
+ bash .github/scripts/fetch-moltenvk-macos.sh
70
+ echo "SKBUILD_CMAKE_DEFINE=TORCHVULKAN_MOLTENVK=/tmp/moltenvk/libMoltenVK.dylib" >> "$GITHUB_ENV"
65
71
  echo "MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0" >> "$GITHUB_ENV"
66
- echo "DYLD_LIBRARY_PATH=$(brew --prefix)/lib:$DYLD_LIBRARY_PATH" >> "$GITHUB_ENV"
67
- export VK_ICD_FILENAMES="$MVK_ICD"
68
- export MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0
69
- export DYLD_LIBRARY_PATH="$(brew --prefix)/lib:$DYLD_LIBRARY_PATH"
70
- vulkaninfo --summary
71
72
 
72
73
  - name: Install Vulkan (Windows)
73
74
  if: runner.os == 'Windows'
74
75
  shell: pwsh
75
76
  run: |
76
- Invoke-WebRequest -Uri "https://github.com/pal1000/mesa-dist-win/releases/download/23.3.2/mesa3d-23.3.2-release-msvc.7z" -OutFile "mesa.7z"
77
+ function Get-Verified($uri, $out, $sha256) {
78
+ Invoke-WebRequest -Uri $uri -OutFile $out
79
+ if ((Get-FileHash $out -Algorithm SHA256).Hash -ne $sha256) { throw "SHA-256 mismatch for $out" }
80
+ }
81
+
82
+ Get-Verified "https://github.com/pal1000/mesa-dist-win/releases/download/23.3.2/mesa3d-23.3.2-release-msvc.7z" "mesa.7z" "177422ca8985c29007fa6df775f6f72860a08f1d0d9da6c7b50aaddc72c957ba"
77
83
  7z x mesa.7z -omesa -y | Out-Null
78
84
 
79
85
  $lvpJson = (Resolve-Path "mesa\x64\lvp_icd.x86_64.json").Path
@@ -83,7 +89,7 @@ runs:
83
89
 
84
90
  $sdkVersion = "${{ inputs.vulkan-sdk-version }}"
85
91
  $installer = "VulkanSDK-$sdkVersion-Installer.exe"
86
- Invoke-WebRequest -Uri "https://sdk.lunarg.com/sdk/download/$sdkVersion/windows/$installer" -OutFile $installer
92
+ Get-Verified "https://sdk.lunarg.com/sdk/download/$sdkVersion/windows/$installer" $installer "acb4ae0786fd3e558f8b3c36cc3eba91638984217ba8a6795ec64d2f9ffd8c4b"
87
93
  Start-Process -FilePath ".\$installer" -ArgumentList "--accept-licenses --default-answer --confirm-command install" -Wait -NoNewWindow
88
94
 
89
95
  "VULKAN_SDK=C:\VulkanSDK\$sdkVersion" | Out-File -FilePath $env:GITHUB_ENV -Append
@@ -0,0 +1,18 @@
1
+ #!/usr/bin/env bash
2
+ # The output path MUST match TORCHVULKAN_MOLTENVK in [tool.cibuildwheel.macos]
3
+ # environment in pyproject.toml. The dylib ships inside the macOS wheel, so the
4
+ # download is pinned by SHA-256; update the hash with the version.
5
+ set -euo pipefail
6
+
7
+ MOLTENVK_VERSION="1.4.2"
8
+ MOLTENVK_SHA256="f95765a6229cb7b915990a2890ce12ebe36a730b021545d3d52ae69ce4c4024e"
9
+ OUT_DIR="/tmp/moltenvk"
10
+
11
+ mkdir -p "$OUT_DIR"
12
+ curl -sSLf -o "$OUT_DIR/MoltenVK-macos.tar" \
13
+ "https://github.com/KhronosGroup/MoltenVK/releases/download/v${MOLTENVK_VERSION}/MoltenVK-macos.tar"
14
+ echo "${MOLTENVK_SHA256} $OUT_DIR/MoltenVK-macos.tar" | shasum -a 256 -c -
15
+ tar -xf "$OUT_DIR/MoltenVK-macos.tar" -C "$OUT_DIR"
16
+ lipo -thin "$(uname -m)" "$OUT_DIR/MoltenVK/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib" -output "$OUT_DIR/libMoltenVK.dylib"
17
+
18
+ test -f "$OUT_DIR/libMoltenVK.dylib"
@@ -1,27 +1,31 @@
1
1
  #!/usr/bin/env bash
2
2
  # Installs the torchvulkan build toolchain inside the cibuildwheel manylinux
3
- # container (AlmaLinux 8 / manylinux_2_28): the Vulkan SDK (headers + loader +
4
- # the spirv-cross executable) and the Slang compiler (slangc).
3
+ # container (AlmaLinux 9 / manylinux_2_34): the Vulkan SDK (headers + loader)
4
+ # and the Slang compiler (slangc). Both downloads are pinned by SHA-256; update
5
+ # the hashes with the versions.
5
6
  #
6
7
  # The install paths here MUST match the VULKAN_SDK / SLANG_DIR / PATH entries in
7
8
  # [tool.cibuildwheel.linux].environment in pyproject.toml.
8
9
  set -euo pipefail
9
10
 
10
11
  SLANG_VERSION="2026.8.1"
12
+ SLANG_SHA256="0d018272522f6d2a75e2f92a30e8eddef18b92e80ab3d29575ac3b0136838aac"
11
13
  VULKAN_SDK_VERSION="1.3.296.0"
14
+ VULKAN_SDK_SHA256="79b0a1593dadc46180526250836f3e53688a9a5fb42a0e5859eb72316dc4d53e"
12
15
 
13
16
  dnf install -y wget tar xz unzip
14
17
 
15
18
  wget -q -O /tmp/vulkansdk.tar.xz \
16
19
  "https://sdk.lunarg.com/sdk/download/${VULKAN_SDK_VERSION}/linux/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.xz"
20
+ echo "${VULKAN_SDK_SHA256} /tmp/vulkansdk.tar.xz" | sha256sum -c -
17
21
  mkdir -p /opt/vulkan
18
22
  tar -xf /tmp/vulkansdk.tar.xz -C /opt/vulkan --strip-components=1
19
23
 
20
24
  wget -q -O /tmp/slang.zip \
21
25
  "https://github.com/shader-slang/slang/releases/download/v${SLANG_VERSION}/slang-${SLANG_VERSION}-linux-x86_64.zip"
26
+ echo "${SLANG_SHA256} /tmp/slang.zip" | sha256sum -c -
22
27
  mkdir -p /opt/slang
23
28
  unzip -q /tmp/slang.zip -d /opt/slang
24
29
 
25
30
  test -f /opt/vulkan/x86_64/include/vulkan/vulkan.h
26
- test -x /opt/vulkan/x86_64/bin/spirv-cross
27
31
  test -x /opt/slang/bin/slangc
@@ -14,12 +14,12 @@ jobs:
14
14
  name: Build sdist
15
15
  runs-on: ubuntu-latest
16
16
  steps:
17
- - uses: actions/checkout@v4
17
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
18
18
  with:
19
19
  submodules: recursive
20
20
  fetch-depth: 0
21
21
 
22
- - uses: actions/setup-python@v5
22
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
23
23
  with:
24
24
  python-version: "3.10"
25
25
 
@@ -28,7 +28,7 @@ jobs:
28
28
  python -m pip install --upgrade pip build
29
29
  python -m build --sdist
30
30
 
31
- - uses: actions/upload-artifact@v4
31
+ - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
32
32
  with:
33
33
  name: dist-sdist
34
34
  path: dist/*.tar.gz
@@ -47,12 +47,12 @@ jobs:
47
47
  - name: Windows x86_64
48
48
  os: windows-latest
49
49
  steps:
50
- - uses: actions/checkout@v4
50
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
51
51
  with:
52
52
  submodules: recursive
53
53
  fetch-depth: 0
54
54
 
55
- - uses: actions/setup-python@v5
55
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
56
56
  with:
57
57
  python-version: "3.12"
58
58
 
@@ -62,16 +62,16 @@ jobs:
62
62
  python -m pip install --upgrade setuptools_scm
63
63
  echo "SETUPTOOLS_SCM_PRETEND_VERSION_FOR_TORCHVULKAN=$(python -m setuptools_scm)" >> "$GITHUB_ENV"
64
64
 
65
- - name: Install toolchain (Slang, Vulkan, spirv-cross)
65
+ - name: Install toolchain (Slang, Vulkan)
66
66
  if: runner.os != 'Linux'
67
67
  uses: ./.github/actions/setup-toolchain
68
68
 
69
69
  - name: Build wheels
70
- uses: pypa/cibuildwheel@v3.2.1
70
+ uses: pypa/cibuildwheel@9c00cb4f6b517705a3794b22395aedc36257242c # v3.2.1
71
71
  env:
72
72
  CIBW_BUILD_VERBOSITY: "1"
73
73
 
74
- - uses: actions/upload-artifact@v4
74
+ - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
75
75
  with:
76
76
  name: dist-${{ matrix.os }}
77
77
  path: wheelhouse/*.whl
@@ -87,13 +87,13 @@ jobs:
87
87
  permissions:
88
88
  id-token: write
89
89
  steps:
90
- - uses: actions/download-artifact@v4
90
+ - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
91
91
  with:
92
92
  path: dist
93
93
  pattern: dist-*
94
94
  merge-multiple: true
95
95
 
96
- - uses: pypa/gh-action-pypi-publish@release/v1
96
+ - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
97
97
  with:
98
98
  packages-dir: dist
99
99
  skip-existing: true
@@ -0,0 +1,138 @@
1
+ name: torchvulkan CI
2
+
3
+ on:
4
+ push:
5
+ branches: [ main ]
6
+ pull_request:
7
+ branches: [ main ]
8
+
9
+ jobs:
10
+ build-and-test:
11
+ name: ${{ matrix.os }} / Py${{ matrix.python-version }} / ${{ matrix.torch-factor }}
12
+ runs-on: ${{ matrix.os }}
13
+ env:
14
+ SKBUILD_CMAKE_GENERATOR: Ninja
15
+ strategy:
16
+ fail-fast: false
17
+ matrix:
18
+ os: [ubuntu-latest, macos-15, windows-latest]
19
+ python-version: ["3.10", "3.12", "3.14"]
20
+ torch-factor: ["torch210", "torch213"]
21
+
22
+ steps:
23
+ - name: Checkout Code
24
+ uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
25
+ with:
26
+ submodules: recursive
27
+
28
+ - name: Set up Python ${{ matrix.python-version }}
29
+ uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
30
+ with:
31
+ python-version: ${{ matrix.python-version }}
32
+
33
+ - name: Install toolchain (Slang, Vulkan)
34
+ uses: ./.github/actions/setup-toolchain
35
+
36
+ - name: Install Tox
37
+ run: |
38
+ python -m pip install --upgrade pip
39
+ pip install tox
40
+
41
+ - name: Run Tests via Tox
42
+ env:
43
+ PIP_EXTRA_INDEX_URL: https://download.pytorch.org/whl/cpu
44
+ SKBUILD_CMAKE_GENERATOR: Ninja
45
+ MVK_CONFIG_LOG_LEVEL: 3
46
+ VK_LOADER_DEBUG: all
47
+ MKL_ENABLE_INSTRUCTIONS: AVX2 # cap MKL's fp32 GEMM to AVX2; AVX-512 faults on some GHA runner CPUs
48
+ ATEN_CPU_CAPABILITY: avx2 # same clamp for PyTorch's own vectorized (Vec256/Vec512) kernels
49
+ ONEDNN_MAX_CPU_ISA: AVX2 # cap oneDNN's matmul JIT to AVX2; it picks its own ISA and ignores the MKL knob
50
+ DNNL_MAX_CPU_ISA: AVX2 # legacy name for the same oneDNN ISA cap (older bundled oneDNN)
51
+ LP_NATIVE_VECTOR_WIDTH: "256" # cap lavapipe's LLVM JIT to AVX2; AVX-512 codegen faults (0xc000001d) on GHA runners
52
+ run: |
53
+ tox -e py-${{ matrix.torch-factor }}
54
+
55
+ validation:
56
+ name: validation layers (lavapipe, debug)
57
+ runs-on: ubuntu-latest
58
+ steps:
59
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
60
+ with:
61
+ submodules: recursive
62
+
63
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
64
+ with:
65
+ python-version: "3.12"
66
+
67
+ - name: Install toolchain (Slang, Vulkan, validation layers)
68
+ uses: ./.github/actions/setup-toolchain
69
+ with:
70
+ validation-layers: "true"
71
+
72
+ - name: Run tests under the validation layer
73
+ env:
74
+ PIP_EXTRA_INDEX_URL: https://download.pytorch.org/whl/cpu
75
+ SKBUILD_CMAKE_GENERATOR: Ninja
76
+ SKBUILD_CMAKE_BUILD_TYPE: Debug
77
+ TORCHVULKAN_VALIDATION: sync
78
+ MKL_ENABLE_INSTRUCTIONS: AVX2
79
+ ATEN_CPU_CAPABILITY: avx2
80
+ ONEDNN_MAX_CPU_ISA: AVX2
81
+ DNNL_MAX_CPU_ISA: AVX2
82
+ LP_NATIVE_VECTOR_WIDTH: "256"
83
+ run: |
84
+ python -m pip install --upgrade pip
85
+ pip install tox
86
+ tox -e py-torch213 -- -k "TestVulkanRuntime or test_correctness_add or test_correctness_mm or test_correctness_sum or test_correctness_sort or test_correctness_index or test_correctness_cumsum or test_gradients_nn_functional_linear"
87
+
88
+ sanitizers:
89
+ name: ASan + UBSan (lavapipe)
90
+ runs-on: ubuntu-latest
91
+ steps:
92
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
93
+ with:
94
+ submodules: recursive
95
+
96
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
97
+ with:
98
+ python-version: "3.12"
99
+
100
+ - name: Install toolchain (Slang, Vulkan)
101
+ uses: ./.github/actions/setup-toolchain
102
+
103
+ - name: Build with sanitizers
104
+ env:
105
+ PIP_EXTRA_INDEX_URL: https://download.pytorch.org/whl/cpu
106
+ SKBUILD_CMAKE_GENERATOR: Ninja
107
+ SKBUILD_CMAKE_BUILD_TYPE: RelWithDebInfo
108
+ SKBUILD_CMAKE_ARGS: "-DCMAKE_CXX_FLAGS=-fsanitize=address,undefined -fno-omit-frame-pointer;-DCMAKE_MODULE_LINKER_FLAGS=-fsanitize=address,undefined"
109
+ run: |
110
+ python -m pip install --upgrade pip
111
+ pip install "torch==2.13.0" pytest expecttest numpy ninja scikit-build-core setuptools_scm
112
+ pip install . --no-build-isolation -v
113
+
114
+ - name: Run tests with the sanitizer runtimes preloaded
115
+ env:
116
+ MKL_ENABLE_INSTRUCTIONS: AVX2
117
+ ATEN_CPU_CAPABILITY: avx2
118
+ ONEDNN_MAX_CPU_ISA: AVX2
119
+ DNNL_MAX_CPU_ISA: AVX2
120
+ LP_NATIVE_VECTOR_WIDTH: "256"
121
+ ASAN_OPTIONS: detect_leaks=0:halt_on_error=1:protect_shadow_gap=0
122
+ UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
123
+ run: |
124
+ cd "$RUNNER_TEMP"
125
+ export LD_PRELOAD="$(gcc -print-file-name=libasan.so):$(gcc -print-file-name=libubsan.so)"
126
+ python -m pytest "$GITHUB_WORKSPACE/tests/test_pytorch_ops.py" -q -k "TestVulkanRuntime or test_correctness_add or test_correctness_mm or test_correctness_sort or test_correctness_index or test_correctness_sum"
127
+
128
+ testing-success:
129
+ name: testing success
130
+ needs: [build-and-test, validation, sanitizers]
131
+ if: always()
132
+ runs-on: ubuntu-latest
133
+ steps:
134
+ - name: Verify all test jobs passed
135
+ if: needs.build-and-test.result != 'success' || needs.validation.result != 'success' || needs.sanitizers.result != 'success'
136
+ run: |
137
+ echo "build-and-test: ${{ needs.build-and-test.result }}, validation: ${{ needs.validation.result }}, sanitizers: ${{ needs.sanitizers.result }}"
138
+ exit 1
@@ -1,6 +1,5 @@
1
1
  # torchvulkan
2
2
  torchvulkan/_version.py
3
- src/shaders/shader_registry.h
4
3
 
5
4
  # testing data
6
5
  data/
@@ -0,0 +1,42 @@
1
+ cmake_minimum_required(VERSION 3.18)
2
+ project(torchvulkan LANGUAGES CXX)
3
+
4
+ find_package(Torch REQUIRED)
5
+ find_package(Vulkan REQUIRED)
6
+ find_package(Python3 COMPONENTS Interpreter Development.Module REQUIRED)
7
+ find_program(SLANGC_EXE slangc HINTS "$ENV{SLANG_DIR}" REQUIRED)
8
+
9
+ set(CMAKE_CXX_STANDARD 20)
10
+ set(CMAKE_CXX_STANDARD_REQUIRED ON)
11
+
12
+ set(TORCHVULKAN_GENERATED_DIR "${CMAKE_BINARY_DIR}/generated")
13
+ set(TORCHVULKAN_SHADER_REGISTRY_CPP "${TORCHVULKAN_GENERATED_DIR}/shader_registry.cpp")
14
+ add_subdirectory(src/shaders)
15
+
16
+ file(GLOB_RECURSE SOURCES CONFIGURE_DEPENDS "src/*.cpp")
17
+ set_source_files_properties(${TORCHVULKAN_SHADER_REGISTRY_CPP} PROPERTIES GENERATED TRUE)
18
+
19
+ python3_add_library(torchvulkan_lib MODULE ${SOURCES} ${TORCHVULKAN_SHADER_REGISTRY_CPP} WITH_SOABI)
20
+ add_dependencies(torchvulkan_lib compile_shaders)
21
+
22
+ set_target_properties(torchvulkan_lib PROPERTIES OUTPUT_NAME "_C")
23
+ find_library(TORCH_PYTHON_LIBRARY torch_python PATHS "${TORCH_INSTALL_PREFIX}/lib" REQUIRED)
24
+ target_link_libraries(torchvulkan_lib PRIVATE ${TORCH_LIBRARIES} ${TORCH_PYTHON_LIBRARY} Python3::Module ${CMAKE_DL_LIBS})
25
+ target_include_directories(torchvulkan_lib PRIVATE src/ ${CMAKE_BINARY_DIR} extern/vma extern/volk ${Vulkan_INCLUDE_DIRS})
26
+
27
+ target_compile_definitions(torchvulkan_lib PRIVATE VK_NO_PROTOTYPES TORCH_EXTENSION_NAME=_C)
28
+
29
+ target_compile_features(torchvulkan_lib PRIVATE cxx_std_20)
30
+
31
+ install(TARGETS torchvulkan_lib DESTINATION torchvulkan)
32
+
33
+ if(APPLE)
34
+ find_library(TORCHVULKAN_MOLTENVK NAMES libMoltenVK.dylib)
35
+ if(TORCHVULKAN_MOLTENVK)
36
+ get_filename_component(TORCHVULKAN_MOLTENVK_REAL "${TORCHVULKAN_MOLTENVK}" REALPATH)
37
+ message(STATUS "Bundling MoltenVK: ${TORCHVULKAN_MOLTENVK_REAL}")
38
+ install(FILES "${TORCHVULKAN_MOLTENVK_REAL}" DESTINATION torchvulkan RENAME libMoltenVK.dylib)
39
+ else()
40
+ message(WARNING "libMoltenVK.dylib not found; torchvulkan will need a system Vulkan loader at runtime.")
41
+ endif()
42
+ endif()
@@ -0,0 +1,222 @@
1
+ Metadata-Version: 2.4
2
+ Name: torchvulkan
3
+ Version: 0.2.0
4
+ Summary: Cross-Platform Vulkan Backend for PyTorch
5
+ Author: Zain Sharief
6
+ License-Expression: MIT
7
+ License-File: LICENSE
8
+ License-File: licenses/MoltenVK-LICENSE
9
+ Project-URL: Homepage, https://github.com/zainsharief/torchvulkan
10
+ Project-URL: Repository, https://github.com/zainsharief/torchvulkan
11
+ Project-URL: Issues, https://github.com/zainsharief/torchvulkan/issues
12
+ Requires-Dist: torch>=2.10.0
13
+ Provides-Extra: test
14
+ Requires-Dist: pytest; extra == "test"
15
+ Requires-Dist: tox; extra == "test"
16
+ Description-Content-Type: text/markdown
17
+
18
+ <h1 align="center">torchvulkan</h1>
19
+ <p align="center">
20
+ <i>A Cross-Platform Vulkan Backend for PyTorch</i>
21
+ </p>
22
+
23
+ <p align="center">
24
+ <a href="https://github.com/zainsharief/torchvulkan/actions/workflows/test_torchvulkan.yml"><img alt="CI" src="https://github.com/zainsharief/torchvulkan/actions/workflows/test_torchvulkan.yml/badge.svg"></a>
25
+ <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-blue.svg"></a>
26
+ <img alt="Python 3.10+" src="https://img.shields.io/badge/python-3.10%2B-blue">
27
+ <img alt="Status: experimental" src="https://img.shields.io/badge/status-experimental-orange">
28
+ </p>
29
+
30
+ `torchvulkan` registers a `vulkan` device with PyTorch and runs tensor operations as [Vulkan](https://www.vulkan.org/) compute shaders. Vulkan runs on GPUs from AMD, Intel, NVIDIA, and Apple (via MoltenVK), plus software rasterizers, so you get one vendor-neutral GPU backend across Linux, macOS, and Windows.
31
+
32
+ You use it like any other PyTorch device:
33
+
34
+ ```python
35
+ import torch
36
+ import torchvulkan # registers the 'vulkan' device on import
37
+
38
+ x = torch.randn(1024, 1024).to("vulkan")
39
+ y = torch.randn(1024, 1024).to("vulkan")
40
+
41
+ z = (x @ y).relu() # runs on the GPU via Vulkan compute shaders
42
+ print(z.sum().item()) # move a scalar back to the CPU
43
+ ```
44
+
45
+ > [!WARNING]
46
+ > **Experimental.** `torchvulkan` is an early-stage project. Most of PyTorch's operators run natively on the GPU; the few that don't fall back to the CPU. Expect breaking changes before `1.0`.
47
+
48
+ ## Table of contents
49
+
50
+ - [Installation](#installation)
51
+ - [Quickstart](#quickstart)
52
+ - [Examples](#examples)
53
+ - [Benchmarks](#benchmarks)
54
+ - [Python API](#python-api)
55
+ - [CPU fallback](#cpu-fallback)
56
+ - [Environment variables](#environment-variables)
57
+ - [Limitations](#limitations)
58
+ - [How it works](#how-it-works)
59
+
60
+ ## Installation
61
+
62
+ ```bash
63
+ pip install torchvulkan
64
+ ```
65
+
66
+ At runtime you need a **Vulkan 1.3 (or newer) driver**: your GPU's driver on Linux and Windows. On macOS the wheel bundles [MoltenVK](https://github.com/KhronosGroup/MoltenVK), so nothing else is needed; set `TORCHVULKAN_SYSTEM_VULKAN=1` to use the system Vulkan loader instead. Verify it worked:
67
+
68
+ ```python
69
+ import torchvulkan
70
+ print(torchvulkan.is_available(), torchvulkan.device_count())
71
+ ```
72
+
73
+ ## Quickstart
74
+
75
+ ```python
76
+ import torch
77
+ import torchvulkan
78
+
79
+ # Move tensors to the Vulkan device
80
+ a = torch.tensor([[1.0, 2.0], [3.0, 4.0]], device="vulkan")
81
+ b = torch.ones(2, 2, device="vulkan")
82
+
83
+ c = a @ b + 1.0 # matmul + broadcasted add
84
+ d = c.exp().sum(dim=0) # elementwise + reduction
85
+
86
+ print(d.cpu()) # bring the result back to the CPU
87
+ ```
88
+
89
+ `nn.Module`s move to the device the usual way:
90
+
91
+ ```python
92
+ model = torch.nn.Linear(128, 10).to("vulkan")
93
+ out = model(torch.randn(32, 128).to("vulkan"))
94
+ ```
95
+
96
+ ## Examples
97
+
98
+ Each example trains a model end to end on the Vulkan device: forward pass, loss, backward pass and optimizer step all run through the backend.
99
+
100
+ | Example | Model | Data |
101
+ | --- | --- | --- |
102
+ | [`examples/mlp.py`](examples/mlp.py) | A multi-layer perceptron with GELU and dropout, trained with AdamW | MNIST |
103
+ | [`examples/llm.py`](examples/llm.py) | A small modern decoder-only transformer: RMSNorm, rotary embeddings, grouped-query attention, a SwiGLU MLP and tied embeddings. Samples text when it's done | Its own source code, character by character |
104
+ | [`examples/resnet.py`](examples/resnet.py) | A ResNet with residual blocks, batch norm and global average pooling, trained with SGD and a one-cycle schedule | CIFAR-10 |
105
+
106
+ ```bash
107
+ pip install torchvision # for the MNIST and CIFAR-10 datasets
108
+ python examples/mlp.py
109
+ python examples/llm.py
110
+ python examples/resnet.py
111
+ ```
112
+
113
+ Training looks the same as on any other device:
114
+
115
+ ```python
116
+ device = torch.device("vulkan" if torchvulkan.is_available() else "cpu")
117
+ model = ResNet().to(device)
118
+ optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
119
+
120
+ for data, targets in train_loader:
121
+ data, targets = data.to(device), targets.to(device)
122
+ loss = criterion(model(data), targets)
123
+ optimizer.zero_grad()
124
+ loss.backward()
125
+ optimizer.step()
126
+ ```
127
+
128
+ ## Benchmarks
129
+
130
+ [`benchmark/benchmark.py`](benchmark/benchmark.py) times a combined forward + backward pass (`float16` on accelerators, `float32` on CPU) on every available backend, sweeping each model's size. Each backend runs in its own subprocess. The models are the ones from the [examples](#examples):
131
+
132
+ - `mlp`: sweeps the hidden width, up to ~141M parameters.
133
+ - `llm`: sweeps the model width (`n_embd`), up to ~26M parameters.
134
+ - `resnet`: sweeps the base channel width, up to ~6M parameters.
135
+
136
+ <p align="center">
137
+ <img alt="Forward + backward pass time vs. model size for the MLP, LLM and ResNet on an NVIDIA GPU: vulkan vs. CUDA and CPU" src="media/benchmark-cuda.png" width="100%">
138
+ </p>
139
+
140
+ <p align="center">
141
+ <i>Vulkan vs. CUDA vs. CPU.</i>
142
+ </p>
143
+
144
+ ```bash
145
+ pip install matplotlib
146
+ python benchmark/benchmark.py all # every model
147
+ ```
148
+
149
+ ## Python API
150
+
151
+ `torchvulkan` mirrors the `torch.cuda` module, and is also reachable as `torch.vulkan`:
152
+
153
+ ```python
154
+ import torchvulkan
155
+
156
+ # devices
157
+ torchvulkan.is_available() # -> bool: a usable Vulkan device was found
158
+ torchvulkan.device_count() # -> int: number of Vulkan devices (discrete GPUs first)
159
+ torchvulkan.current_device() # -> int: index of the current device
160
+ torchvulkan.set_device(1) # make "vulkan" mean "vulkan:1"
161
+ torchvulkan.get_device_name() # -> str
162
+ torchvulkan.get_device_properties() # -> name, total_memory, vendor_id, api_version, subgroup_size, ...
163
+ torchvulkan.synchronize() # block until queued GPU work has finished
164
+
165
+ # memory
166
+ torchvulkan.memory_allocated() # bytes held by tensors
167
+ torchvulkan.max_memory_allocated() # peak since the last reset_peak_memory_stats()
168
+ torchvulkan.memory_reserved() # bytes held by the allocator, including its cache
169
+ torchvulkan.memory_stats() # -> dict
170
+ torchvulkan.empty_cache() # release cached device memory back to the driver
171
+
172
+ # random numbers (per-device)
173
+ torchvulkan.manual_seed(0) # also: manual_seed_all, seed, seed_all, initial_seed
174
+ torchvulkan.get_rng_state() # also: set_rng_state, get_rng_state_all, set_rng_state_all
175
+
176
+ torchvulkan.is_bf16_supported() # -> bool: bfloat16 runs on the GPU
177
+ torchvulkan.__version__ # package version
178
+ ```
179
+
180
+ Importing `torchvulkan` registers the backend, after which the device is addressed as `"vulkan"` / `torch.device("vulkan")` throughout PyTorch.
181
+
182
+ ## CPU fallback
183
+
184
+ By default, any operation, dtype, or tensor shape the backend doesn't implement natively runs on the **CPU**, so models keep working despite partial coverage. The warning is printed once, for the first operation that falls back; call `torch.set_warn_always(True)` to see every one.
185
+
186
+ Set `TORCHVULKAN_STRICT=1` to turn those silent fallbacks into errors. This is useful for finding which operators a workload actually needs:
187
+
188
+ ```bash
189
+ TORCHVULKAN_STRICT=1 python your_script.py
190
+ ```
191
+
192
+ ## Environment variables
193
+
194
+ | Variable | Effect |
195
+ | --- | --- |
196
+ | `TORCHVULKAN_STRICT=1` | Raise an error instead of falling back to the CPU. |
197
+ | `TORCHVULKAN_SYSTEM_VULKAN=1` | macOS: use the system Vulkan loader instead of the bundled MoltenVK. |
198
+ | `TORCHVULKAN_VALIDATION=1` | Enable the Khronos validation layer, if it's installed. `=sync` also enables synchronization validation. On macOS this needs `TORCHVULKAN_SYSTEM_VULKAN=1`. |
199
+ | `TORCHVULKAN_PROFILE=1` | Print each compute pipeline's driver statistics (such as register use) when it's compiled, on drivers that support `VK_KHR_pipeline_executable_properties`. |
200
+ | `TORCHVULKAN_PIPELINE_CACHE=0` | Don't cache compiled pipelines between runs. By default they're cached in `%LOCALAPPDATA%\torchvulkan` on Windows and `$XDG_CACHE_HOME/torchvulkan` (or `~/.cache/torchvulkan`) elsewhere. |
201
+ | `TORCHVULKAN_CACHE_DIR` | Directory for the pipeline cache. |
202
+
203
+ ## Limitations
204
+
205
+ - `torch.compile` with the default Inductor backend doesn't support the `vulkan` device. Run eagerly, or use `torch.compile(model, backend="aot_eager")`.
206
+ - Optimizers created with `fused=True` (such as `torch.optim.AdamW(..., fused=True)`) run their update on the CPU. The default (foreach) implementation runs on the GPU.
207
+ - `float64` runs on the GPU only on devices with `shaderFloat64`; elsewhere it falls back to the CPU. Transcendental functions such as `exp`, `sin` and `tanh` are computed in `float32` precision for `float64` tensors.
208
+ - `bfloat16` runs on the GPU only on devices with `VK_KHR_shader_bfloat16`; elsewhere it falls back to the CPU.
209
+ - Complex tensors can live on the device and FFTs run on the GPU, but most complex arithmetic falls back to the CPU.
210
+ - `torch>=2.10.0` only. The extension is compiled against 2.13.0 but has testing support for >=2.10.0.
211
+ - APIs and coverage may change before `1.0.0`.
212
+
213
+ ## How it works
214
+
215
+ 1. Importing the package registers a PyTorch `PrivateUse1` backend and renames it to `vulkan`.
216
+ 2. Operations dispatched to a `vulkan` tensor are routed to native implementations that record and submit **Vulkan compute shaders**. The shaders are written in [Slang](https://github.com/shader-slang/slang), compiled to SPIR-V at build time, and embedded into the extension.
217
+ 3. Device memory is managed with [VMA](https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator); Vulkan entry points are loaded with [volk](https://github.com/zeux/volk).
218
+ 4. Anything without a native implementation is handled by [CPU fallback](#cpu-fallback).
219
+
220
+
221
+ > [!NOTE]
222
+ > **Disclaimer:** `torchvulkan` is an independent open-source project and is not affiliated with or endorsed by PyTorch or the Linux Foundation.