torchvulkan 0.1.0a0__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.github/actions/setup-toolchain/action.yml +29 -23
- torchvulkan-0.2.0/.github/scripts/fetch-moltenvk-macos.sh +18 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.github/scripts/install-toolchain-linux.sh +7 -3
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.github/workflows/release.yaml +10 -10
- torchvulkan-0.2.0/.github/workflows/test_torchvulkan.yml +138 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/.gitignore +0 -1
- torchvulkan-0.2.0/CMakeLists.txt +42 -0
- torchvulkan-0.2.0/PKG-INFO +222 -0
- torchvulkan-0.2.0/README.md +205 -0
- torchvulkan-0.2.0/benchmark/benchmark.py +431 -0
- torchvulkan-0.2.0/examples/conv.py +90 -0
- torchvulkan-0.2.0/examples/llm.py +142 -0
- torchvulkan-0.2.0/examples/mlp.py +65 -0
- torchvulkan-0.2.0/licenses/MoltenVK-LICENSE +202 -0
- torchvulkan-0.2.0/media/benchmark-cuda.png +0 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/pyproject.toml +15 -4
- torchvulkan-0.2.0/scripts/build_shaders.py +101 -0
- torchvulkan-0.2.0/scripts/config.py +224 -0
- torchvulkan-0.2.0/scripts/pack_shaders.py +147 -0
- torchvulkan-0.2.0/src/api/backend.cpp +110 -0
- torchvulkan-0.2.0/src/api/decompositions.cpp +1065 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/api/guard.cpp +13 -3
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/api/hook.cpp +9 -4
- torchvulkan-0.2.0/src/api/ops/binary.cpp +685 -0
- torchvulkan-0.2.0/src/api/ops/bitwise.cpp +237 -0
- torchvulkan-0.2.0/src/api/ops/compare.cpp +283 -0
- torchvulkan-0.2.0/src/api/ops/complex.cpp +75 -0
- torchvulkan-0.2.0/src/api/ops/conv.cpp +567 -0
- torchvulkan-0.2.0/src/api/ops/ctc.cpp +216 -0
- torchvulkan-0.2.0/src/api/ops/distributions.cpp +118 -0
- torchvulkan-0.2.0/src/api/ops/dropout.cpp +81 -0
- torchvulkan-0.2.0/src/api/ops/fft.cpp +226 -0
- torchvulkan-0.2.0/src/api/ops/fill.cpp +74 -0
- torchvulkan-0.2.0/src/api/ops/grid_sample.cpp +233 -0
- torchvulkan-0.2.0/src/api/ops/helpers.h +380 -0
- torchvulkan-0.2.0/src/api/ops/indexing.cpp +359 -0
- torchvulkan-0.2.0/src/api/ops/internal.h +26 -0
- torchvulkan-0.2.0/src/api/ops/linalg.cpp +1220 -0
- torchvulkan-0.2.0/src/api/ops/matmul.cpp +540 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/api/ops/nllloss.cpp +50 -37
- torchvulkan-0.2.0/src/api/ops/nn.cpp +989 -0
- torchvulkan-0.2.0/src/api/ops/pool.cpp +640 -0
- torchvulkan-0.2.0/src/api/ops/random.cpp +388 -0
- torchvulkan-0.2.0/src/api/ops/reduce.cpp +573 -0
- torchvulkan-0.2.0/src/api/ops/sampling.cpp +142 -0
- torchvulkan-0.2.0/src/api/ops/scan.cpp +211 -0
- torchvulkan-0.2.0/src/api/ops/scatter.cpp +445 -0
- torchvulkan-0.2.0/src/api/ops/search.cpp +576 -0
- torchvulkan-0.2.0/src/api/ops/shape.cpp +266 -0
- torchvulkan-0.2.0/src/api/ops/sort.cpp +279 -0
- torchvulkan-0.2.0/src/api/ops/special.cpp +323 -0
- torchvulkan-0.2.0/src/api/ops/unary.cpp +520 -0
- torchvulkan-0.2.0/src/api/ops/upsample.cpp +595 -0
- torchvulkan-0.2.0/src/api/registration.cpp +36 -0
- torchvulkan-0.2.0/src/api/tensor/allocation.cpp +216 -0
- torchvulkan-0.2.0/src/api/tensor/cast.cpp +54 -0
- torchvulkan-0.2.0/src/api/tensor/transfer.cpp +244 -0
- torchvulkan-0.2.0/src/api/tensor/views.cpp +159 -0
- torchvulkan-0.2.0/src/shaders/CMakeLists.txt +62 -0
- torchvulkan-0.2.0/src/shaders/core/cast.slang +60 -0
- torchvulkan-0.2.0/src/shaders/core/dtype.slang +958 -0
- torchvulkan-0.2.0/src/shaders/core/indexing.slang +63 -0
- torchvulkan-0.2.0/src/shaders/core/philox.slang +165 -0
- torchvulkan-0.2.0/src/shaders/core/special_functions.slang +2804 -0
- torchvulkan-0.2.0/src/shaders/ops/arg_reduce.slang +75 -0
- torchvulkan-0.2.0/src/shaders/ops/binary.slang +248 -0
- torchvulkan-0.2.0/src/shaders/ops/cast.slang +95 -0
- torchvulkan-0.2.0/src/shaders/ops/cholesky.slang +65 -0
- torchvulkan-0.2.0/src/shaders/ops/compare.slang +147 -0
- torchvulkan-0.2.0/src/shaders/ops/conv.slang +338 -0
- torchvulkan-0.2.0/src/shaders/ops/copy.slang +61 -0
- torchvulkan-0.2.0/src/shaders/ops/ctc.slang +207 -0
- torchvulkan-0.2.0/src/shaders/ops/distribution.slang +248 -0
- torchvulkan-0.2.0/src/shaders/ops/dropout.slang +51 -0
- torchvulkan-0.2.0/src/shaders/ops/eig.slang +524 -0
- torchvulkan-0.2.0/src/shaders/ops/embedding_backward.slang +66 -0
- torchvulkan-0.2.0/src/shaders/ops/fft.slang +116 -0
- torchvulkan-0.2.0/src/shaders/ops/fill.slang +65 -0
- torchvulkan-0.2.0/src/shaders/ops/grid_sample.slang +442 -0
- torchvulkan-0.2.0/src/shaders/ops/householder.slang +169 -0
- torchvulkan-0.2.0/src/shaders/ops/index_gather.slang +82 -0
- torchvulkan-0.2.0/src/shaders/ops/integer_binary.slang +132 -0
- torchvulkan-0.2.0/src/shaders/ops/jacobi.slang +166 -0
- torchvulkan-0.2.0/src/shaders/ops/ldl.slang +254 -0
- torchvulkan-0.2.0/src/shaders/ops/lu.slang +93 -0
- torchvulkan-0.2.0/src/shaders/ops/matmul_coop.slang +530 -0
- torchvulkan-0.2.0/src/shaders/ops/matmul_simd.slang +340 -0
- torchvulkan-0.2.0/src/shaders/ops/multinomial.slang +43 -0
- torchvulkan-0.2.0/src/shaders/ops/nllloss.slang +89 -0
- torchvulkan-0.2.0/src/shaders/ops/pool.slang +291 -0
- torchvulkan-0.2.0/src/shaders/ops/random.slang +190 -0
- torchvulkan-0.2.0/src/shaders/ops/reduce.slang +92 -0
- torchvulkan-0.2.0/src/shaders/ops/reduce_subgroup.slang +112 -0
- torchvulkan-0.2.0/src/shaders/ops/resample.slang +225 -0
- torchvulkan-0.2.0/src/shaders/ops/resample_u8.slang +54 -0
- torchvulkan-0.2.0/src/shaders/ops/scan.slang +139 -0
- torchvulkan-0.2.0/src/shaders/ops/scan_arg.slang +143 -0
- torchvulkan-0.2.0/src/shaders/ops/scatter.slang +112 -0
- torchvulkan-0.2.0/src/shaders/ops/searchsorted.slang +61 -0
- torchvulkan-0.2.0/src/shaders/ops/sort.slang +100 -0
- torchvulkan-0.2.0/src/shaders/ops/special.slang +115 -0
- torchvulkan-0.2.0/src/shaders/ops/triangular_solve.slang +56 -0
- torchvulkan-0.2.0/src/shaders/ops/unary.slang +140 -0
- torchvulkan-0.2.0/src/shaders/ops/unary_ops.slang +59 -0
- torchvulkan-0.2.0/src/shaders/ops/where.slang +80 -0
- torchvulkan-0.2.0/src/vulkan/allocator.cpp +106 -0
- torchvulkan-0.2.0/src/vulkan/allocator.h +31 -0
- torchvulkan-0.2.0/src/vulkan/builders.h +229 -0
- torchvulkan-0.2.0/src/vulkan/cache.cpp +290 -0
- torchvulkan-0.2.0/src/vulkan/cache.h +74 -0
- torchvulkan-0.2.0/src/vulkan/device_context.cpp +54 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/device_context.h +27 -7
- torchvulkan-0.2.0/src/vulkan/dispatch.cpp +169 -0
- torchvulkan-0.2.0/src/vulkan/dispatch.h +23 -0
- torchvulkan-0.2.0/src/vulkan/generator.cpp +150 -0
- torchvulkan-0.2.0/src/vulkan/generator.h +51 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/memory.cpp +22 -11
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/memory.h +19 -5
- torchvulkan-0.2.0/src/vulkan/shader.cpp +444 -0
- torchvulkan-0.2.0/src/vulkan/shader.h +136 -0
- torchvulkan-0.2.0/src/vulkan/vulkan_context.cpp +622 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/src/vulkan/vulkan_context.h +19 -9
- torchvulkan-0.2.0/tests/test_pytorch_ops.py +535 -0
- torchvulkan-0.2.0/torchvulkan/__init__.py +157 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/torchvulkan/_version.py +3 -3
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/tox.ini +1 -1
- torchvulkan-0.1.0a0/.github/workflows/test_torchvulkan.yml +0 -60
- torchvulkan-0.1.0a0/CMakeLists.txt +0 -28
- torchvulkan-0.1.0a0/PKG-INFO +0 -8
- torchvulkan-0.1.0a0/README.md +0 -163
- torchvulkan-0.1.0a0/examples/main.py +0 -10
- torchvulkan-0.1.0a0/examples/mnist.py +0 -51
- torchvulkan-0.1.0a0/scripts/build_shaders.py +0 -211
- torchvulkan-0.1.0a0/scripts/config.py +0 -106
- torchvulkan-0.1.0a0/scripts/pack_shaders.py +0 -214
- torchvulkan-0.1.0a0/src/api/backend.cpp +0 -34
- torchvulkan-0.1.0a0/src/api/ops/binary.cpp +0 -360
- torchvulkan-0.1.0a0/src/api/ops/binary.h +0 -76
- torchvulkan-0.1.0a0/src/api/ops/factory.cpp +0 -511
- torchvulkan-0.1.0a0/src/api/ops/factory.h +0 -84
- torchvulkan-0.1.0a0/src/api/ops/helpers.h +0 -196
- torchvulkan-0.1.0a0/src/api/ops/matmul.cpp +0 -478
- torchvulkan-0.1.0a0/src/api/ops/matmul.h +0 -70
- torchvulkan-0.1.0a0/src/api/ops/nllloss.h +0 -28
- torchvulkan-0.1.0a0/src/api/ops/reduce.cpp +0 -175
- torchvulkan-0.1.0a0/src/api/ops/reduce.h +0 -61
- torchvulkan-0.1.0a0/src/api/ops/softmax.cpp +0 -26
- torchvulkan-0.1.0a0/src/api/ops/softmax.h +0 -14
- torchvulkan-0.1.0a0/src/api/ops/unary.cpp +0 -216
- torchvulkan-0.1.0a0/src/api/ops/unary.h +0 -39
- torchvulkan-0.1.0a0/src/api/ops.cpp +0 -122
- torchvulkan-0.1.0a0/src/shaders/CMakeLists.txt +0 -44
- torchvulkan-0.1.0a0/src/shaders/binaryop.slang.j2 +0 -183
- torchvulkan-0.1.0a0/src/shaders/cast.slang.j2 +0 -54
- torchvulkan-0.1.0a0/src/shaders/copy.slang.j2 +0 -53
- torchvulkan-0.1.0a0/src/shaders/fill.slang.j2 +0 -65
- torchvulkan-0.1.0a0/src/shaders/helpers/int_divider.slang.j2 +0 -10
- torchvulkan-0.1.0a0/src/shaders/helpers/vulkan_arithmetic.slang.j2 +0 -165
- torchvulkan-0.1.0a0/src/shaders/matmul_coop.slang.j2 +0 -768
- torchvulkan-0.1.0a0/src/shaders/matmul_simd.slang.j2 +0 -317
- torchvulkan-0.1.0a0/src/shaders/nllloss.slang.j2 +0 -80
- torchvulkan-0.1.0a0/src/shaders/reduce.slang.j2 +0 -80
- torchvulkan-0.1.0a0/src/shaders/unaryop.slang.j2 +0 -138
- torchvulkan-0.1.0a0/src/vulkan/allocator.cpp +0 -215
- torchvulkan-0.1.0a0/src/vulkan/allocator.h +0 -113
- torchvulkan-0.1.0a0/src/vulkan/cache.cpp +0 -454
- torchvulkan-0.1.0a0/src/vulkan/cache.h +0 -82
- torchvulkan-0.1.0a0/src/vulkan/device_context.cpp +0 -55
- torchvulkan-0.1.0a0/src/vulkan/shader.cpp +0 -74
- torchvulkan-0.1.0a0/src/vulkan/shader.h +0 -34
- torchvulkan-0.1.0a0/src/vulkan/vulkan_context.cpp +0 -384
- torchvulkan-0.1.0a0/tests/test_pytorch_ops.py +0 -157
- torchvulkan-0.1.0a0/torchvulkan/__init__.py +0 -31
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/LICENSE +0 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/extern/vma/vk_mem_alloc.h +0 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/extern/volk/volk.c +0 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/extern/volk/volk.h +0 -0
- {torchvulkan-0.1.0a0 → torchvulkan-0.2.0}/requirements.txt +0 -0
|
@@ -1,16 +1,22 @@
|
|
|
1
1
|
name: Setup torchvulkan toolchain
|
|
2
2
|
description: >
|
|
3
3
|
Install the build/runtime toolchain for the current runner OS: the Slang
|
|
4
|
-
compiler,
|
|
4
|
+
compiler, the Vulkan headers + loader, and a Vulkan driver (on macOS, the
|
|
5
|
+
MoltenVK release that gets bundled into the package). Exports the needed
|
|
5
6
|
PATH / env entries so later steps in the job can build and run torchvulkan.
|
|
7
|
+
Every download is checked against a pinned SHA-256; bump the hashes with the
|
|
8
|
+
versions.
|
|
6
9
|
|
|
7
10
|
inputs:
|
|
8
11
|
slang-version:
|
|
9
|
-
description: Slang release version to install.
|
|
12
|
+
description: Slang release version to install (the hashes below are for this version).
|
|
10
13
|
default: "2026.8.1"
|
|
11
14
|
vulkan-sdk-version:
|
|
12
|
-
description: Vulkan SDK version (Windows only).
|
|
15
|
+
description: Vulkan SDK version (Windows only; the hash below is for this version).
|
|
13
16
|
default: "1.3.296.0"
|
|
17
|
+
validation-layers:
|
|
18
|
+
description: Also install the Khronos validation layer (Linux only).
|
|
19
|
+
default: "false"
|
|
14
20
|
|
|
15
21
|
runs:
|
|
16
22
|
using: composite
|
|
@@ -19,10 +25,10 @@ runs:
|
|
|
19
25
|
shell: bash
|
|
20
26
|
run: |
|
|
21
27
|
case "${{ runner.os }}-${{ runner.arch }}" in
|
|
22
|
-
Linux-X64) echo "SLANG_ARCH=linux-x86_64" >> "$GITHUB_ENV" ;;
|
|
23
|
-
macOS-ARM64) echo "SLANG_ARCH=macos-aarch64" >> "$GITHUB_ENV" ;;
|
|
24
|
-
macOS-X64) echo "SLANG_ARCH=macos-x86_64" >> "$GITHUB_ENV" ;;
|
|
25
|
-
Windows-X64) echo "SLANG_ARCH=windows-x86_64" >> "$GITHUB_ENV" ;;
|
|
28
|
+
Linux-X64) echo "SLANG_ARCH=linux-x86_64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=0d018272522f6d2a75e2f92a30e8eddef18b92e80ab3d29575ac3b0136838aac" >> "$GITHUB_ENV" ;;
|
|
29
|
+
macOS-ARM64) echo "SLANG_ARCH=macos-aarch64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=d953efb713e76b220f1a4ff1e23a96b9f564657d1701de184e5a52059bc62310" >> "$GITHUB_ENV" ;;
|
|
30
|
+
macOS-X64) echo "SLANG_ARCH=macos-x86_64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=1ebce824ee9f757de3dbcc4a574b1ddb137136a086e0aaa77d928bbe073901fa" >> "$GITHUB_ENV" ;;
|
|
31
|
+
Windows-X64) echo "SLANG_ARCH=windows-x86_64" >> "$GITHUB_ENV"; echo "SLANG_SHA256=4cd6a06278b05433b09dfac3d733426e1e704967ca8529fc9e44f06deb5fcdd7" >> "$GITHUB_ENV" ;;
|
|
26
32
|
*) echo "::error::Unsupported runner ${{ runner.os }}-${{ runner.arch }}"; exit 1 ;;
|
|
27
33
|
esac
|
|
28
34
|
|
|
@@ -30,8 +36,9 @@ runs:
|
|
|
30
36
|
if: runner.os != 'Windows'
|
|
31
37
|
shell: bash
|
|
32
38
|
run: |
|
|
33
|
-
curl -
|
|
34
|
-
|
|
39
|
+
curl -sSLf -o slang.zip "https://github.com/shader-slang/slang/releases/download/v${{ inputs.slang-version }}/slang-${{ inputs.slang-version }}-${SLANG_ARCH}.zip"
|
|
40
|
+
echo "${SLANG_SHA256} slang.zip" | shasum -a 256 -c -
|
|
41
|
+
unzip -q slang.zip -d "$RUNNER_TEMP/slang"
|
|
35
42
|
echo "$RUNNER_TEMP/slang/bin" >> "$GITHUB_PATH"
|
|
36
43
|
echo "SLANG_DIR=$RUNNER_TEMP/slang/bin" >> "$GITHUB_ENV"
|
|
37
44
|
|
|
@@ -41,6 +48,7 @@ runs:
|
|
|
41
48
|
run: |
|
|
42
49
|
$zip = "slang-${{ inputs.slang-version }}-$env:SLANG_ARCH.zip"
|
|
43
50
|
Invoke-WebRequest -Uri "https://github.com/shader-slang/slang/releases/download/v${{ inputs.slang-version }}/$zip" -OutFile $zip
|
|
51
|
+
if ((Get-FileHash $zip -Algorithm SHA256).Hash -ne $env:SLANG_SHA256) { throw "SHA-256 mismatch for $zip" }
|
|
44
52
|
Expand-Archive -Path $zip -DestinationPath C:\slang
|
|
45
53
|
"C:\slang\bin" | Out-File -FilePath $env:GITHUB_PATH -Append
|
|
46
54
|
"SLANG_DIR=C:\slang\bin" | Out-File -FilePath $env:GITHUB_ENV -Append
|
|
@@ -49,31 +57,29 @@ runs:
|
|
|
49
57
|
if: runner.os == 'Linux'
|
|
50
58
|
shell: bash
|
|
51
59
|
run: |
|
|
52
|
-
wget -qO- https://packages.lunarg.com/lunarg-signing-key-pub.asc | sudo tee /etc/apt/trusted.gpg.d/lunarg.asc
|
|
53
|
-
sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-$(lsb_release -cs).list http://packages.lunarg.com/vulkan/lunarg-vulkan-$(lsb_release -cs).list
|
|
54
60
|
sudo apt-get update
|
|
55
|
-
sudo apt-get install -y
|
|
56
|
-
|
|
61
|
+
sudo apt-get install -y libvulkan-dev mesa-vulkan-drivers
|
|
62
|
+
if [ "${{ inputs.validation-layers }}" = "true" ]; then sudo apt-get install -y vulkan-validationlayers; fi
|
|
57
63
|
|
|
58
64
|
- name: Install Vulkan (macOS)
|
|
59
65
|
if: runner.os == 'macOS'
|
|
60
66
|
shell: bash
|
|
61
67
|
run: |
|
|
62
|
-
brew install
|
|
63
|
-
|
|
64
|
-
echo "
|
|
68
|
+
brew install vulkan-headers vulkan-loader ninja
|
|
69
|
+
bash .github/scripts/fetch-moltenvk-macos.sh
|
|
70
|
+
echo "SKBUILD_CMAKE_DEFINE=TORCHVULKAN_MOLTENVK=/tmp/moltenvk/libMoltenVK.dylib" >> "$GITHUB_ENV"
|
|
65
71
|
echo "MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0" >> "$GITHUB_ENV"
|
|
66
|
-
echo "DYLD_LIBRARY_PATH=$(brew --prefix)/lib:$DYLD_LIBRARY_PATH" >> "$GITHUB_ENV"
|
|
67
|
-
export VK_ICD_FILENAMES="$MVK_ICD"
|
|
68
|
-
export MVK_CONFIG_USE_METAL_ARGUMENT_BUFFERS=0
|
|
69
|
-
export DYLD_LIBRARY_PATH="$(brew --prefix)/lib:$DYLD_LIBRARY_PATH"
|
|
70
|
-
vulkaninfo --summary
|
|
71
72
|
|
|
72
73
|
- name: Install Vulkan (Windows)
|
|
73
74
|
if: runner.os == 'Windows'
|
|
74
75
|
shell: pwsh
|
|
75
76
|
run: |
|
|
76
|
-
|
|
77
|
+
function Get-Verified($uri, $out, $sha256) {
|
|
78
|
+
Invoke-WebRequest -Uri $uri -OutFile $out
|
|
79
|
+
if ((Get-FileHash $out -Algorithm SHA256).Hash -ne $sha256) { throw "SHA-256 mismatch for $out" }
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
Get-Verified "https://github.com/pal1000/mesa-dist-win/releases/download/23.3.2/mesa3d-23.3.2-release-msvc.7z" "mesa.7z" "177422ca8985c29007fa6df775f6f72860a08f1d0d9da6c7b50aaddc72c957ba"
|
|
77
83
|
7z x mesa.7z -omesa -y | Out-Null
|
|
78
84
|
|
|
79
85
|
$lvpJson = (Resolve-Path "mesa\x64\lvp_icd.x86_64.json").Path
|
|
@@ -83,7 +89,7 @@ runs:
|
|
|
83
89
|
|
|
84
90
|
$sdkVersion = "${{ inputs.vulkan-sdk-version }}"
|
|
85
91
|
$installer = "VulkanSDK-$sdkVersion-Installer.exe"
|
|
86
|
-
|
|
92
|
+
Get-Verified "https://sdk.lunarg.com/sdk/download/$sdkVersion/windows/$installer" $installer "acb4ae0786fd3e558f8b3c36cc3eba91638984217ba8a6795ec64d2f9ffd8c4b"
|
|
87
93
|
Start-Process -FilePath ".\$installer" -ArgumentList "--accept-licenses --default-answer --confirm-command install" -Wait -NoNewWindow
|
|
88
94
|
|
|
89
95
|
"VULKAN_SDK=C:\VulkanSDK\$sdkVersion" | Out-File -FilePath $env:GITHUB_ENV -Append
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# The output path MUST match TORCHVULKAN_MOLTENVK in [tool.cibuildwheel.macos]
|
|
3
|
+
# environment in pyproject.toml. The dylib ships inside the macOS wheel, so the
|
|
4
|
+
# download is pinned by SHA-256; update the hash with the version.
|
|
5
|
+
set -euo pipefail
|
|
6
|
+
|
|
7
|
+
MOLTENVK_VERSION="1.4.2"
|
|
8
|
+
MOLTENVK_SHA256="f95765a6229cb7b915990a2890ce12ebe36a730b021545d3d52ae69ce4c4024e"
|
|
9
|
+
OUT_DIR="/tmp/moltenvk"
|
|
10
|
+
|
|
11
|
+
mkdir -p "$OUT_DIR"
|
|
12
|
+
curl -sSLf -o "$OUT_DIR/MoltenVK-macos.tar" \
|
|
13
|
+
"https://github.com/KhronosGroup/MoltenVK/releases/download/v${MOLTENVK_VERSION}/MoltenVK-macos.tar"
|
|
14
|
+
echo "${MOLTENVK_SHA256} $OUT_DIR/MoltenVK-macos.tar" | shasum -a 256 -c -
|
|
15
|
+
tar -xf "$OUT_DIR/MoltenVK-macos.tar" -C "$OUT_DIR"
|
|
16
|
+
lipo -thin "$(uname -m)" "$OUT_DIR/MoltenVK/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib" -output "$OUT_DIR/libMoltenVK.dylib"
|
|
17
|
+
|
|
18
|
+
test -f "$OUT_DIR/libMoltenVK.dylib"
|
|
@@ -1,27 +1,31 @@
|
|
|
1
1
|
#!/usr/bin/env bash
|
|
2
2
|
# Installs the torchvulkan build toolchain inside the cibuildwheel manylinux
|
|
3
|
-
# container (AlmaLinux
|
|
4
|
-
#
|
|
3
|
+
# container (AlmaLinux 9 / manylinux_2_34): the Vulkan SDK (headers + loader)
|
|
4
|
+
# and the Slang compiler (slangc). Both downloads are pinned by SHA-256; update
|
|
5
|
+
# the hashes with the versions.
|
|
5
6
|
#
|
|
6
7
|
# The install paths here MUST match the VULKAN_SDK / SLANG_DIR / PATH entries in
|
|
7
8
|
# [tool.cibuildwheel.linux].environment in pyproject.toml.
|
|
8
9
|
set -euo pipefail
|
|
9
10
|
|
|
10
11
|
SLANG_VERSION="2026.8.1"
|
|
12
|
+
SLANG_SHA256="0d018272522f6d2a75e2f92a30e8eddef18b92e80ab3d29575ac3b0136838aac"
|
|
11
13
|
VULKAN_SDK_VERSION="1.3.296.0"
|
|
14
|
+
VULKAN_SDK_SHA256="79b0a1593dadc46180526250836f3e53688a9a5fb42a0e5859eb72316dc4d53e"
|
|
12
15
|
|
|
13
16
|
dnf install -y wget tar xz unzip
|
|
14
17
|
|
|
15
18
|
wget -q -O /tmp/vulkansdk.tar.xz \
|
|
16
19
|
"https://sdk.lunarg.com/sdk/download/${VULKAN_SDK_VERSION}/linux/vulkansdk-linux-x86_64-${VULKAN_SDK_VERSION}.tar.xz"
|
|
20
|
+
echo "${VULKAN_SDK_SHA256} /tmp/vulkansdk.tar.xz" | sha256sum -c -
|
|
17
21
|
mkdir -p /opt/vulkan
|
|
18
22
|
tar -xf /tmp/vulkansdk.tar.xz -C /opt/vulkan --strip-components=1
|
|
19
23
|
|
|
20
24
|
wget -q -O /tmp/slang.zip \
|
|
21
25
|
"https://github.com/shader-slang/slang/releases/download/v${SLANG_VERSION}/slang-${SLANG_VERSION}-linux-x86_64.zip"
|
|
26
|
+
echo "${SLANG_SHA256} /tmp/slang.zip" | sha256sum -c -
|
|
22
27
|
mkdir -p /opt/slang
|
|
23
28
|
unzip -q /tmp/slang.zip -d /opt/slang
|
|
24
29
|
|
|
25
30
|
test -f /opt/vulkan/x86_64/include/vulkan/vulkan.h
|
|
26
|
-
test -x /opt/vulkan/x86_64/bin/spirv-cross
|
|
27
31
|
test -x /opt/slang/bin/slangc
|
|
@@ -14,12 +14,12 @@ jobs:
|
|
|
14
14
|
name: Build sdist
|
|
15
15
|
runs-on: ubuntu-latest
|
|
16
16
|
steps:
|
|
17
|
-
- uses: actions/checkout@v4
|
|
17
|
+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
18
18
|
with:
|
|
19
19
|
submodules: recursive
|
|
20
20
|
fetch-depth: 0
|
|
21
21
|
|
|
22
|
-
- uses: actions/setup-python@v5
|
|
22
|
+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
|
23
23
|
with:
|
|
24
24
|
python-version: "3.10"
|
|
25
25
|
|
|
@@ -28,7 +28,7 @@ jobs:
|
|
|
28
28
|
python -m pip install --upgrade pip build
|
|
29
29
|
python -m build --sdist
|
|
30
30
|
|
|
31
|
-
- uses: actions/upload-artifact@v4
|
|
31
|
+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
|
|
32
32
|
with:
|
|
33
33
|
name: dist-sdist
|
|
34
34
|
path: dist/*.tar.gz
|
|
@@ -47,12 +47,12 @@ jobs:
|
|
|
47
47
|
- name: Windows x86_64
|
|
48
48
|
os: windows-latest
|
|
49
49
|
steps:
|
|
50
|
-
- uses: actions/checkout@v4
|
|
50
|
+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
51
51
|
with:
|
|
52
52
|
submodules: recursive
|
|
53
53
|
fetch-depth: 0
|
|
54
54
|
|
|
55
|
-
- uses: actions/setup-python@v5
|
|
55
|
+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
|
56
56
|
with:
|
|
57
57
|
python-version: "3.12"
|
|
58
58
|
|
|
@@ -62,16 +62,16 @@ jobs:
|
|
|
62
62
|
python -m pip install --upgrade setuptools_scm
|
|
63
63
|
echo "SETUPTOOLS_SCM_PRETEND_VERSION_FOR_TORCHVULKAN=$(python -m setuptools_scm)" >> "$GITHUB_ENV"
|
|
64
64
|
|
|
65
|
-
- name: Install toolchain (Slang, Vulkan
|
|
65
|
+
- name: Install toolchain (Slang, Vulkan)
|
|
66
66
|
if: runner.os != 'Linux'
|
|
67
67
|
uses: ./.github/actions/setup-toolchain
|
|
68
68
|
|
|
69
69
|
- name: Build wheels
|
|
70
|
-
uses: pypa/cibuildwheel@v3.2.1
|
|
70
|
+
uses: pypa/cibuildwheel@9c00cb4f6b517705a3794b22395aedc36257242c # v3.2.1
|
|
71
71
|
env:
|
|
72
72
|
CIBW_BUILD_VERBOSITY: "1"
|
|
73
73
|
|
|
74
|
-
- uses: actions/upload-artifact@v4
|
|
74
|
+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
|
|
75
75
|
with:
|
|
76
76
|
name: dist-${{ matrix.os }}
|
|
77
77
|
path: wheelhouse/*.whl
|
|
@@ -87,13 +87,13 @@ jobs:
|
|
|
87
87
|
permissions:
|
|
88
88
|
id-token: write
|
|
89
89
|
steps:
|
|
90
|
-
- uses: actions/download-artifact@v4
|
|
90
|
+
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
|
91
91
|
with:
|
|
92
92
|
path: dist
|
|
93
93
|
pattern: dist-*
|
|
94
94
|
merge-multiple: true
|
|
95
95
|
|
|
96
|
-
- uses: pypa/gh-action-pypi-publish@release/v1
|
|
96
|
+
- uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
|
|
97
97
|
with:
|
|
98
98
|
packages-dir: dist
|
|
99
99
|
skip-existing: true
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
name: torchvulkan CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [ main ]
|
|
6
|
+
pull_request:
|
|
7
|
+
branches: [ main ]
|
|
8
|
+
|
|
9
|
+
jobs:
|
|
10
|
+
build-and-test:
|
|
11
|
+
name: ${{ matrix.os }} / Py${{ matrix.python-version }} / ${{ matrix.torch-factor }}
|
|
12
|
+
runs-on: ${{ matrix.os }}
|
|
13
|
+
env:
|
|
14
|
+
SKBUILD_CMAKE_GENERATOR: Ninja
|
|
15
|
+
strategy:
|
|
16
|
+
fail-fast: false
|
|
17
|
+
matrix:
|
|
18
|
+
os: [ubuntu-latest, macos-15, windows-latest]
|
|
19
|
+
python-version: ["3.10", "3.12", "3.14"]
|
|
20
|
+
torch-factor: ["torch210", "torch213"]
|
|
21
|
+
|
|
22
|
+
steps:
|
|
23
|
+
- name: Checkout Code
|
|
24
|
+
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
25
|
+
with:
|
|
26
|
+
submodules: recursive
|
|
27
|
+
|
|
28
|
+
- name: Set up Python ${{ matrix.python-version }}
|
|
29
|
+
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
|
30
|
+
with:
|
|
31
|
+
python-version: ${{ matrix.python-version }}
|
|
32
|
+
|
|
33
|
+
- name: Install toolchain (Slang, Vulkan)
|
|
34
|
+
uses: ./.github/actions/setup-toolchain
|
|
35
|
+
|
|
36
|
+
- name: Install Tox
|
|
37
|
+
run: |
|
|
38
|
+
python -m pip install --upgrade pip
|
|
39
|
+
pip install tox
|
|
40
|
+
|
|
41
|
+
- name: Run Tests via Tox
|
|
42
|
+
env:
|
|
43
|
+
PIP_EXTRA_INDEX_URL: https://download.pytorch.org/whl/cpu
|
|
44
|
+
SKBUILD_CMAKE_GENERATOR: Ninja
|
|
45
|
+
MVK_CONFIG_LOG_LEVEL: 3
|
|
46
|
+
VK_LOADER_DEBUG: all
|
|
47
|
+
MKL_ENABLE_INSTRUCTIONS: AVX2 # cap MKL's fp32 GEMM to AVX2; AVX-512 faults on some GHA runner CPUs
|
|
48
|
+
ATEN_CPU_CAPABILITY: avx2 # same clamp for PyTorch's own vectorized (Vec256/Vec512) kernels
|
|
49
|
+
ONEDNN_MAX_CPU_ISA: AVX2 # cap oneDNN's matmul JIT to AVX2; it picks its own ISA and ignores the MKL knob
|
|
50
|
+
DNNL_MAX_CPU_ISA: AVX2 # legacy name for the same oneDNN ISA cap (older bundled oneDNN)
|
|
51
|
+
LP_NATIVE_VECTOR_WIDTH: "256" # cap lavapipe's LLVM JIT to AVX2; AVX-512 codegen faults (0xc000001d) on GHA runners
|
|
52
|
+
run: |
|
|
53
|
+
tox -e py-${{ matrix.torch-factor }}
|
|
54
|
+
|
|
55
|
+
validation:
|
|
56
|
+
name: validation layers (lavapipe, debug)
|
|
57
|
+
runs-on: ubuntu-latest
|
|
58
|
+
steps:
|
|
59
|
+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
60
|
+
with:
|
|
61
|
+
submodules: recursive
|
|
62
|
+
|
|
63
|
+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
|
64
|
+
with:
|
|
65
|
+
python-version: "3.12"
|
|
66
|
+
|
|
67
|
+
- name: Install toolchain (Slang, Vulkan, validation layers)
|
|
68
|
+
uses: ./.github/actions/setup-toolchain
|
|
69
|
+
with:
|
|
70
|
+
validation-layers: "true"
|
|
71
|
+
|
|
72
|
+
- name: Run tests under the validation layer
|
|
73
|
+
env:
|
|
74
|
+
PIP_EXTRA_INDEX_URL: https://download.pytorch.org/whl/cpu
|
|
75
|
+
SKBUILD_CMAKE_GENERATOR: Ninja
|
|
76
|
+
SKBUILD_CMAKE_BUILD_TYPE: Debug
|
|
77
|
+
TORCHVULKAN_VALIDATION: sync
|
|
78
|
+
MKL_ENABLE_INSTRUCTIONS: AVX2
|
|
79
|
+
ATEN_CPU_CAPABILITY: avx2
|
|
80
|
+
ONEDNN_MAX_CPU_ISA: AVX2
|
|
81
|
+
DNNL_MAX_CPU_ISA: AVX2
|
|
82
|
+
LP_NATIVE_VECTOR_WIDTH: "256"
|
|
83
|
+
run: |
|
|
84
|
+
python -m pip install --upgrade pip
|
|
85
|
+
pip install tox
|
|
86
|
+
tox -e py-torch213 -- -k "TestVulkanRuntime or test_correctness_add or test_correctness_mm or test_correctness_sum or test_correctness_sort or test_correctness_index or test_correctness_cumsum or test_gradients_nn_functional_linear"
|
|
87
|
+
|
|
88
|
+
sanitizers:
|
|
89
|
+
name: ASan + UBSan (lavapipe)
|
|
90
|
+
runs-on: ubuntu-latest
|
|
91
|
+
steps:
|
|
92
|
+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
93
|
+
with:
|
|
94
|
+
submodules: recursive
|
|
95
|
+
|
|
96
|
+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
|
97
|
+
with:
|
|
98
|
+
python-version: "3.12"
|
|
99
|
+
|
|
100
|
+
- name: Install toolchain (Slang, Vulkan)
|
|
101
|
+
uses: ./.github/actions/setup-toolchain
|
|
102
|
+
|
|
103
|
+
- name: Build with sanitizers
|
|
104
|
+
env:
|
|
105
|
+
PIP_EXTRA_INDEX_URL: https://download.pytorch.org/whl/cpu
|
|
106
|
+
SKBUILD_CMAKE_GENERATOR: Ninja
|
|
107
|
+
SKBUILD_CMAKE_BUILD_TYPE: RelWithDebInfo
|
|
108
|
+
SKBUILD_CMAKE_ARGS: "-DCMAKE_CXX_FLAGS=-fsanitize=address,undefined -fno-omit-frame-pointer;-DCMAKE_MODULE_LINKER_FLAGS=-fsanitize=address,undefined"
|
|
109
|
+
run: |
|
|
110
|
+
python -m pip install --upgrade pip
|
|
111
|
+
pip install "torch==2.13.0" pytest expecttest numpy ninja scikit-build-core setuptools_scm
|
|
112
|
+
pip install . --no-build-isolation -v
|
|
113
|
+
|
|
114
|
+
- name: Run tests with the sanitizer runtimes preloaded
|
|
115
|
+
env:
|
|
116
|
+
MKL_ENABLE_INSTRUCTIONS: AVX2
|
|
117
|
+
ATEN_CPU_CAPABILITY: avx2
|
|
118
|
+
ONEDNN_MAX_CPU_ISA: AVX2
|
|
119
|
+
DNNL_MAX_CPU_ISA: AVX2
|
|
120
|
+
LP_NATIVE_VECTOR_WIDTH: "256"
|
|
121
|
+
ASAN_OPTIONS: detect_leaks=0:halt_on_error=1:protect_shadow_gap=0
|
|
122
|
+
UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
|
|
123
|
+
run: |
|
|
124
|
+
cd "$RUNNER_TEMP"
|
|
125
|
+
export LD_PRELOAD="$(gcc -print-file-name=libasan.so):$(gcc -print-file-name=libubsan.so)"
|
|
126
|
+
python -m pytest "$GITHUB_WORKSPACE/tests/test_pytorch_ops.py" -q -k "TestVulkanRuntime or test_correctness_add or test_correctness_mm or test_correctness_sort or test_correctness_index or test_correctness_sum"
|
|
127
|
+
|
|
128
|
+
testing-success:
|
|
129
|
+
name: testing success
|
|
130
|
+
needs: [build-and-test, validation, sanitizers]
|
|
131
|
+
if: always()
|
|
132
|
+
runs-on: ubuntu-latest
|
|
133
|
+
steps:
|
|
134
|
+
- name: Verify all test jobs passed
|
|
135
|
+
if: needs.build-and-test.result != 'success' || needs.validation.result != 'success' || needs.sanitizers.result != 'success'
|
|
136
|
+
run: |
|
|
137
|
+
echo "build-and-test: ${{ needs.build-and-test.result }}, validation: ${{ needs.validation.result }}, sanitizers: ${{ needs.sanitizers.result }}"
|
|
138
|
+
exit 1
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
cmake_minimum_required(VERSION 3.18)
|
|
2
|
+
project(torchvulkan LANGUAGES CXX)
|
|
3
|
+
|
|
4
|
+
find_package(Torch REQUIRED)
|
|
5
|
+
find_package(Vulkan REQUIRED)
|
|
6
|
+
find_package(Python3 COMPONENTS Interpreter Development.Module REQUIRED)
|
|
7
|
+
find_program(SLANGC_EXE slangc HINTS "$ENV{SLANG_DIR}" REQUIRED)
|
|
8
|
+
|
|
9
|
+
set(CMAKE_CXX_STANDARD 20)
|
|
10
|
+
set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
|
11
|
+
|
|
12
|
+
set(TORCHVULKAN_GENERATED_DIR "${CMAKE_BINARY_DIR}/generated")
|
|
13
|
+
set(TORCHVULKAN_SHADER_REGISTRY_CPP "${TORCHVULKAN_GENERATED_DIR}/shader_registry.cpp")
|
|
14
|
+
add_subdirectory(src/shaders)
|
|
15
|
+
|
|
16
|
+
file(GLOB_RECURSE SOURCES CONFIGURE_DEPENDS "src/*.cpp")
|
|
17
|
+
set_source_files_properties(${TORCHVULKAN_SHADER_REGISTRY_CPP} PROPERTIES GENERATED TRUE)
|
|
18
|
+
|
|
19
|
+
python3_add_library(torchvulkan_lib MODULE ${SOURCES} ${TORCHVULKAN_SHADER_REGISTRY_CPP} WITH_SOABI)
|
|
20
|
+
add_dependencies(torchvulkan_lib compile_shaders)
|
|
21
|
+
|
|
22
|
+
set_target_properties(torchvulkan_lib PROPERTIES OUTPUT_NAME "_C")
|
|
23
|
+
find_library(TORCH_PYTHON_LIBRARY torch_python PATHS "${TORCH_INSTALL_PREFIX}/lib" REQUIRED)
|
|
24
|
+
target_link_libraries(torchvulkan_lib PRIVATE ${TORCH_LIBRARIES} ${TORCH_PYTHON_LIBRARY} Python3::Module ${CMAKE_DL_LIBS})
|
|
25
|
+
target_include_directories(torchvulkan_lib PRIVATE src/ ${CMAKE_BINARY_DIR} extern/vma extern/volk ${Vulkan_INCLUDE_DIRS})
|
|
26
|
+
|
|
27
|
+
target_compile_definitions(torchvulkan_lib PRIVATE VK_NO_PROTOTYPES TORCH_EXTENSION_NAME=_C)
|
|
28
|
+
|
|
29
|
+
target_compile_features(torchvulkan_lib PRIVATE cxx_std_20)
|
|
30
|
+
|
|
31
|
+
install(TARGETS torchvulkan_lib DESTINATION torchvulkan)
|
|
32
|
+
|
|
33
|
+
if(APPLE)
|
|
34
|
+
find_library(TORCHVULKAN_MOLTENVK NAMES libMoltenVK.dylib)
|
|
35
|
+
if(TORCHVULKAN_MOLTENVK)
|
|
36
|
+
get_filename_component(TORCHVULKAN_MOLTENVK_REAL "${TORCHVULKAN_MOLTENVK}" REALPATH)
|
|
37
|
+
message(STATUS "Bundling MoltenVK: ${TORCHVULKAN_MOLTENVK_REAL}")
|
|
38
|
+
install(FILES "${TORCHVULKAN_MOLTENVK_REAL}" DESTINATION torchvulkan RENAME libMoltenVK.dylib)
|
|
39
|
+
else()
|
|
40
|
+
message(WARNING "libMoltenVK.dylib not found; torchvulkan will need a system Vulkan loader at runtime.")
|
|
41
|
+
endif()
|
|
42
|
+
endif()
|
|
@@ -0,0 +1,222 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: torchvulkan
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Cross-Platform Vulkan Backend for PyTorch
|
|
5
|
+
Author: Zain Sharief
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
License-File: licenses/MoltenVK-LICENSE
|
|
9
|
+
Project-URL: Homepage, https://github.com/zainsharief/torchvulkan
|
|
10
|
+
Project-URL: Repository, https://github.com/zainsharief/torchvulkan
|
|
11
|
+
Project-URL: Issues, https://github.com/zainsharief/torchvulkan/issues
|
|
12
|
+
Requires-Dist: torch>=2.10.0
|
|
13
|
+
Provides-Extra: test
|
|
14
|
+
Requires-Dist: pytest; extra == "test"
|
|
15
|
+
Requires-Dist: tox; extra == "test"
|
|
16
|
+
Description-Content-Type: text/markdown
|
|
17
|
+
|
|
18
|
+
<h1 align="center">torchvulkan</h1>
|
|
19
|
+
<p align="center">
|
|
20
|
+
<i>A Cross-Platform Vulkan Backend for PyTorch</i>
|
|
21
|
+
</p>
|
|
22
|
+
|
|
23
|
+
<p align="center">
|
|
24
|
+
<a href="https://github.com/zainsharief/torchvulkan/actions/workflows/test_torchvulkan.yml"><img alt="CI" src="https://github.com/zainsharief/torchvulkan/actions/workflows/test_torchvulkan.yml/badge.svg"></a>
|
|
25
|
+
<a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-blue.svg"></a>
|
|
26
|
+
<img alt="Python 3.10+" src="https://img.shields.io/badge/python-3.10%2B-blue">
|
|
27
|
+
<img alt="Status: experimental" src="https://img.shields.io/badge/status-experimental-orange">
|
|
28
|
+
</p>
|
|
29
|
+
|
|
30
|
+
`torchvulkan` registers a `vulkan` device with PyTorch and runs tensor operations as [Vulkan](https://www.vulkan.org/) compute shaders. Vulkan runs on GPUs from AMD, Intel, NVIDIA, and Apple (via MoltenVK), plus software rasterizers, so you get one vendor-neutral GPU backend across Linux, macOS, and Windows.
|
|
31
|
+
|
|
32
|
+
You use it like any other PyTorch device:
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
import torch
|
|
36
|
+
import torchvulkan # registers the 'vulkan' device on import
|
|
37
|
+
|
|
38
|
+
x = torch.randn(1024, 1024).to("vulkan")
|
|
39
|
+
y = torch.randn(1024, 1024).to("vulkan")
|
|
40
|
+
|
|
41
|
+
z = (x @ y).relu() # runs on the GPU via Vulkan compute shaders
|
|
42
|
+
print(z.sum().item()) # move a scalar back to the CPU
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
> [!WARNING]
|
|
46
|
+
> **Experimental.** `torchvulkan` is an early-stage project. Most of PyTorch's operators run natively on the GPU; the few that don't fall back to the CPU. Expect breaking changes before `1.0`.
|
|
47
|
+
|
|
48
|
+
## Table of contents
|
|
49
|
+
|
|
50
|
+
- [Installation](#installation)
|
|
51
|
+
- [Quickstart](#quickstart)
|
|
52
|
+
- [Examples](#examples)
|
|
53
|
+
- [Benchmarks](#benchmarks)
|
|
54
|
+
- [Python API](#python-api)
|
|
55
|
+
- [CPU fallback](#cpu-fallback)
|
|
56
|
+
- [Environment variables](#environment-variables)
|
|
57
|
+
- [Limitations](#limitations)
|
|
58
|
+
- [How it works](#how-it-works)
|
|
59
|
+
|
|
60
|
+
## Installation
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
pip install torchvulkan
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
At runtime you need a **Vulkan 1.3 (or newer) driver**: your GPU's driver on Linux and Windows. On macOS the wheel bundles [MoltenVK](https://github.com/KhronosGroup/MoltenVK), so nothing else is needed; set `TORCHVULKAN_SYSTEM_VULKAN=1` to use the system Vulkan loader instead. Verify it worked:
|
|
67
|
+
|
|
68
|
+
```python
|
|
69
|
+
import torchvulkan
|
|
70
|
+
print(torchvulkan.is_available(), torchvulkan.device_count())
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
## Quickstart
|
|
74
|
+
|
|
75
|
+
```python
|
|
76
|
+
import torch
|
|
77
|
+
import torchvulkan
|
|
78
|
+
|
|
79
|
+
# Move tensors to the Vulkan device
|
|
80
|
+
a = torch.tensor([[1.0, 2.0], [3.0, 4.0]], device="vulkan")
|
|
81
|
+
b = torch.ones(2, 2, device="vulkan")
|
|
82
|
+
|
|
83
|
+
c = a @ b + 1.0 # matmul + broadcasted add
|
|
84
|
+
d = c.exp().sum(dim=0) # elementwise + reduction
|
|
85
|
+
|
|
86
|
+
print(d.cpu()) # bring the result back to the CPU
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
`nn.Module`s move to the device the usual way:
|
|
90
|
+
|
|
91
|
+
```python
|
|
92
|
+
model = torch.nn.Linear(128, 10).to("vulkan")
|
|
93
|
+
out = model(torch.randn(32, 128).to("vulkan"))
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Examples
|
|
97
|
+
|
|
98
|
+
Each example trains a model end to end on the Vulkan device: forward pass, loss, backward pass and optimizer step all run through the backend.
|
|
99
|
+
|
|
100
|
+
| Example | Model | Data |
|
|
101
|
+
| --- | --- | --- |
|
|
102
|
+
| [`examples/mlp.py`](examples/mlp.py) | A multi-layer perceptron with GELU and dropout, trained with AdamW | MNIST |
|
|
103
|
+
| [`examples/llm.py`](examples/llm.py) | A small modern decoder-only transformer: RMSNorm, rotary embeddings, grouped-query attention, a SwiGLU MLP and tied embeddings. Samples text when it's done | Its own source code, character by character |
|
|
104
|
+
| [`examples/resnet.py`](examples/resnet.py) | A ResNet with residual blocks, batch norm and global average pooling, trained with SGD and a one-cycle schedule | CIFAR-10 |
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
pip install torchvision # for the MNIST and CIFAR-10 datasets
|
|
108
|
+
python examples/mlp.py
|
|
109
|
+
python examples/llm.py
|
|
110
|
+
python examples/resnet.py
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Training looks the same as on any other device:
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
device = torch.device("vulkan" if torchvulkan.is_available() else "cpu")
|
|
117
|
+
model = ResNet().to(device)
|
|
118
|
+
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
|
|
119
|
+
|
|
120
|
+
for data, targets in train_loader:
|
|
121
|
+
data, targets = data.to(device), targets.to(device)
|
|
122
|
+
loss = criterion(model(data), targets)
|
|
123
|
+
optimizer.zero_grad()
|
|
124
|
+
loss.backward()
|
|
125
|
+
optimizer.step()
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
## Benchmarks
|
|
129
|
+
|
|
130
|
+
[`benchmark/benchmark.py`](benchmark/benchmark.py) times a combined forward + backward pass (`float16` on accelerators, `float32` on CPU) on every available backend, sweeping each model's size. Each backend runs in its own subprocess. The models are the ones from the [examples](#examples):
|
|
131
|
+
|
|
132
|
+
- `mlp`: sweeps the hidden width, up to ~141M parameters.
|
|
133
|
+
- `llm`: sweeps the model width (`n_embd`), up to ~26M parameters.
|
|
134
|
+
- `resnet`: sweeps the base channel width, up to ~6M parameters.
|
|
135
|
+
|
|
136
|
+
<p align="center">
|
|
137
|
+
<img alt="Forward + backward pass time vs. model size for the MLP, LLM and ResNet on an NVIDIA GPU: vulkan vs. CUDA and CPU" src="media/benchmark-cuda.png" width="100%">
|
|
138
|
+
</p>
|
|
139
|
+
|
|
140
|
+
<p align="center">
|
|
141
|
+
<i>Vulkan vs. CUDA vs. CPU.</i>
|
|
142
|
+
</p>
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
pip install matplotlib
|
|
146
|
+
python benchmark/benchmark.py all # every model
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
## Python API
|
|
150
|
+
|
|
151
|
+
`torchvulkan` mirrors the `torch.cuda` module, and is also reachable as `torch.vulkan`:
|
|
152
|
+
|
|
153
|
+
```python
|
|
154
|
+
import torchvulkan
|
|
155
|
+
|
|
156
|
+
# devices
|
|
157
|
+
torchvulkan.is_available() # -> bool: a usable Vulkan device was found
|
|
158
|
+
torchvulkan.device_count() # -> int: number of Vulkan devices (discrete GPUs first)
|
|
159
|
+
torchvulkan.current_device() # -> int: index of the current device
|
|
160
|
+
torchvulkan.set_device(1) # make "vulkan" mean "vulkan:1"
|
|
161
|
+
torchvulkan.get_device_name() # -> str
|
|
162
|
+
torchvulkan.get_device_properties() # -> name, total_memory, vendor_id, api_version, subgroup_size, ...
|
|
163
|
+
torchvulkan.synchronize() # block until queued GPU work has finished
|
|
164
|
+
|
|
165
|
+
# memory
|
|
166
|
+
torchvulkan.memory_allocated() # bytes held by tensors
|
|
167
|
+
torchvulkan.max_memory_allocated() # peak since the last reset_peak_memory_stats()
|
|
168
|
+
torchvulkan.memory_reserved() # bytes held by the allocator, including its cache
|
|
169
|
+
torchvulkan.memory_stats() # -> dict
|
|
170
|
+
torchvulkan.empty_cache() # release cached device memory back to the driver
|
|
171
|
+
|
|
172
|
+
# random numbers (per-device)
|
|
173
|
+
torchvulkan.manual_seed(0) # also: manual_seed_all, seed, seed_all, initial_seed
|
|
174
|
+
torchvulkan.get_rng_state() # also: set_rng_state, get_rng_state_all, set_rng_state_all
|
|
175
|
+
|
|
176
|
+
torchvulkan.is_bf16_supported() # -> bool: bfloat16 runs on the GPU
|
|
177
|
+
torchvulkan.__version__ # package version
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Importing `torchvulkan` registers the backend, after which the device is addressed as `"vulkan"` / `torch.device("vulkan")` throughout PyTorch.
|
|
181
|
+
|
|
182
|
+
## CPU fallback
|
|
183
|
+
|
|
184
|
+
By default, any operation, dtype, or tensor shape the backend doesn't implement natively runs on the **CPU**, so models keep working despite partial coverage. The warning is printed once, for the first operation that falls back; call `torch.set_warn_always(True)` to see every one.
|
|
185
|
+
|
|
186
|
+
Set `TORCHVULKAN_STRICT=1` to turn those silent fallbacks into errors. This is useful for finding which operators a workload actually needs:
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
TORCHVULKAN_STRICT=1 python your_script.py
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
## Environment variables
|
|
193
|
+
|
|
194
|
+
| Variable | Effect |
|
|
195
|
+
| --- | --- |
|
|
196
|
+
| `TORCHVULKAN_STRICT=1` | Raise an error instead of falling back to the CPU. |
|
|
197
|
+
| `TORCHVULKAN_SYSTEM_VULKAN=1` | macOS: use the system Vulkan loader instead of the bundled MoltenVK. |
|
|
198
|
+
| `TORCHVULKAN_VALIDATION=1` | Enable the Khronos validation layer, if it's installed. `=sync` also enables synchronization validation. On macOS this needs `TORCHVULKAN_SYSTEM_VULKAN=1`. |
|
|
199
|
+
| `TORCHVULKAN_PROFILE=1` | Print each compute pipeline's driver statistics (such as register use) when it's compiled, on drivers that support `VK_KHR_pipeline_executable_properties`. |
|
|
200
|
+
| `TORCHVULKAN_PIPELINE_CACHE=0` | Don't cache compiled pipelines between runs. By default they're cached in `%LOCALAPPDATA%\torchvulkan` on Windows and `$XDG_CACHE_HOME/torchvulkan` (or `~/.cache/torchvulkan`) elsewhere. |
|
|
201
|
+
| `TORCHVULKAN_CACHE_DIR` | Directory for the pipeline cache. |
|
|
202
|
+
|
|
203
|
+
## Limitations
|
|
204
|
+
|
|
205
|
+
- `torch.compile` with the default Inductor backend doesn't support the `vulkan` device. Run eagerly, or use `torch.compile(model, backend="aot_eager")`.
|
|
206
|
+
- Optimizers created with `fused=True` (such as `torch.optim.AdamW(..., fused=True)`) run their update on the CPU. The default (foreach) implementation runs on the GPU.
|
|
207
|
+
- `float64` runs on the GPU only on devices with `shaderFloat64`; elsewhere it falls back to the CPU. Transcendental functions such as `exp`, `sin` and `tanh` are computed in `float32` precision for `float64` tensors.
|
|
208
|
+
- `bfloat16` runs on the GPU only on devices with `VK_KHR_shader_bfloat16`; elsewhere it falls back to the CPU.
|
|
209
|
+
- Complex tensors can live on the device and FFTs run on the GPU, but most complex arithmetic falls back to the CPU.
|
|
210
|
+
- `torch>=2.10.0` only. The extension is compiled against 2.13.0 but has testing support for >=2.10.0.
|
|
211
|
+
- APIs and coverage may change before `1.0.0`.
|
|
212
|
+
|
|
213
|
+
## How it works
|
|
214
|
+
|
|
215
|
+
1. Importing the package registers a PyTorch `PrivateUse1` backend and renames it to `vulkan`.
|
|
216
|
+
2. Operations dispatched to a `vulkan` tensor are routed to native implementations that record and submit **Vulkan compute shaders**. The shaders are written in [Slang](https://github.com/shader-slang/slang), compiled to SPIR-V at build time, and embedded into the extension.
|
|
217
|
+
3. Device memory is managed with [VMA](https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator); Vulkan entry points are loaded with [volk](https://github.com/zeux/volk).
|
|
218
|
+
4. Anything without a native implementation is handled by [CPU fallback](#cpu-fallback).
|
|
219
|
+
|
|
220
|
+
|
|
221
|
+
> [!NOTE]
|
|
222
|
+
> **Disclaimer:** `torchvulkan` is an independent open-source project and is not affiliated with or endorsed by PyTorch or the Linux Foundation.
|