gpustack-runner 0.1.22.post3__tar.gz → 0.1.22.post5__tar.gz

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gpustack-runner
-Version: 0.1.22.post3
+Version: 0.1.22.post5
 Summary: GPUStack Runner is library for registering runnable accelerated backends and services in GPUStack.
 Project-URL: Homepage, https://github.com/gpustack/runner
 Project-URL: Bug Tracker, https://github.com/gpustack/gpustack/issues
@@ -54,6 +54,7 @@ The following table lists the supported accelerated backends and their correspon
 
 > [!IMPORTANT]
 > - Applied [ATB model patched](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3646603380) to MindIE 2.2.rc1/2.1.rc2.
+> - Applied [ATB config patched](https://github.com/gpustack/gpustack/issues/3551) to MindIE 2.2.rc1.
 > - Applied [av package](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3631228085) to MindIE 2.2.rc1/2.1.rc2.
 > - Update vLLM 0.11.0 with stable vLLM Ascend plugin.
 
@@ -86,12 +87,12 @@ The following table lists the supported accelerated backends and their correspon
 > - Applied [Qwen2.5 VL patched](https://github.com/gpustack/gpustack/issues/3606) to vLLM 0.11.2.
 > - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
 
-| CUDA Version <br/> (Variant) | vLLM                                                                                 | SGLang                                                    | VoxBox   |
-|------------------------------|--------------------------------------------------------------------------------------|-----------------------------------------------------------|----------|
-| 12.9                         | `0.13.0`, `0.12.0`, **`0.11.2`**                                                     | `0.5.6.post2`                                             |          |
-| 12.8                         | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/>`0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0` | `0.5.6.post2`, `0.5.5.post3`, <br/>`0.5.5`, `0.5.4.post3` | `0.0.20` |
-| 12.6                         | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/>`0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0` | `0.5.6.post2`                                             | `0.0.20` |
-| 12.4                         | `0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`                                        |                                                           | `0.0.20` |
+| CUDA Version <br/> (Variant) | vLLM                                                                                      | SGLang                                                    | VoxBox             |
+|------------------------------|-------------------------------------------------------------------------------------------|-----------------------------------------------------------|--------------------|
+| 12.9                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**                                                     | `0.5.6.post2`                                             |                    |
+| 12.8                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`, `0.5.5.post3`, <br/>`0.5.5`, `0.5.4.post3` | `0.0.21`, `0.0.20` |
+| 12.6                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`                                             | `0.0.21`, `0.0.20` |
+| 12.4                         | `0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`                                             |                                                           | `0.0.20`           |
 
 ### Hygon DTK
 
@@ -124,11 +125,11 @@ The following table lists the supported accelerated backends and their correspon
 > - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
 > - Applied [petit-kernel package](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L728) to vLLM 0.11.2 and SGLang 0.5.5.post3.
 
-| ROCm Version <br/> (Variant) | vLLM                                             | SGLang                           |
-|------------------------------|--------------------------------------------------|----------------------------------|
-| 7.0                          | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/> `0.11.0` | `0.5.6.post2`                    |
-| 6.4                          | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/> `0.10.2` | `0.5.6.post2`, **`0.5.5.post3`** |
-| 6.3                          | `0.10.1.1`, `0.10.0`                             |                                  |
+| ROCm Version <br/> (Variant) | vLLM                                            | SGLang                           |
+|------------------------------|-------------------------------------------------|----------------------------------|
+| 7.0                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0` | `0.5.6.post2`                    |
+| 6.4                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.10.2` | `0.5.6.post2`, **`0.5.5.post3`** |
+| 6.3                          | `0.10.1.1`, `0.10.0`                            |                                  |
 
 ## Directory Structure
 

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/README.md

@@ -34,6 +34,7 @@ The following table lists the supported accelerated backends and their correspon
 
 > [!IMPORTANT]
 > - Applied [ATB model patched](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3646603380) to MindIE 2.2.rc1/2.1.rc2.
+> - Applied [ATB config patched](https://github.com/gpustack/gpustack/issues/3551) to MindIE 2.2.rc1.
 > - Applied [av package](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3631228085) to MindIE 2.2.rc1/2.1.rc2.
 > - Update vLLM 0.11.0 with stable vLLM Ascend plugin.
 
@@ -66,12 +67,12 @@ The following table lists the supported accelerated backends and their correspon
 > - Applied [Qwen2.5 VL patched](https://github.com/gpustack/gpustack/issues/3606) to vLLM 0.11.2.
 > - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
 
-| CUDA Version <br/> (Variant) | vLLM                                                                                 | SGLang                                                    | VoxBox   |
-|------------------------------|--------------------------------------------------------------------------------------|-----------------------------------------------------------|----------|
-| 12.9                         | `0.13.0`, `0.12.0`, **`0.11.2`**                                                     | `0.5.6.post2`                                             |          |
-| 12.8                         | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/>`0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0` | `0.5.6.post2`, `0.5.5.post3`, <br/>`0.5.5`, `0.5.4.post3` | `0.0.20` |
-| 12.6                         | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/>`0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0` | `0.5.6.post2`                                             | `0.0.20` |
-| 12.4                         | `0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`                                        |                                                           | `0.0.20` |
+| CUDA Version <br/> (Variant) | vLLM                                                                                      | SGLang                                                    | VoxBox             |
+|------------------------------|-------------------------------------------------------------------------------------------|-----------------------------------------------------------|--------------------|
+| 12.9                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**                                                     | `0.5.6.post2`                                             |                    |
+| 12.8                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`, `0.5.5.post3`, <br/>`0.5.5`, `0.5.4.post3` | `0.0.21`, `0.0.20` |
+| 12.6                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`                                             | `0.0.21`, `0.0.20` |
+| 12.4                         | `0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`                                             |                                                           | `0.0.20`           |
 
 ### Hygon DTK
 
@@ -104,11 +105,11 @@ The following table lists the supported accelerated backends and their correspon
 > - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
 > - Applied [petit-kernel package](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L728) to vLLM 0.11.2 and SGLang 0.5.5.post3.
 
-| ROCm Version <br/> (Variant) | vLLM                                             | SGLang                           |
-|------------------------------|--------------------------------------------------|----------------------------------|
-| 7.0                          | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/> `0.11.0` | `0.5.6.post2`                    |
-| 6.4                          | `0.13.0`, `0.12.0`, **`0.11.2`**, <br/> `0.10.2` | `0.5.6.post2`, **`0.5.5.post3`** |
-| 6.3                          | `0.10.1.1`, `0.10.0`                             |                                  |
+| ROCm Version <br/> (Variant) | vLLM                                            | SGLang                           |
+|------------------------------|-------------------------------------------------|----------------------------------|
+| 7.0                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0` | `0.5.6.post2`                    |
+| 6.4                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.10.2` | `0.5.6.post2`, **`0.5.5.post3`** |
+| 6.3                          | `0.10.1.1`, `0.10.0`                            |                                  |
 
 ## Directory Structure
 

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/gpustack_runner/_version.py

@@ -27,8 +27,8 @@ version_tuple: VERSION_TUPLE
 __commit_id__: COMMIT_ID
 commit_id: COMMIT_ID
 
-__version__ = version = '0.1.22.post3'
-__version_tuple__ = version_tuple = (0, 1, 22, 'post3')
+__version__ = version = '0.1.22.post5'
+__version_tuple__ = version_tuple = (0, 1, 22, 'post5')
 try:
     from ._version_appendix import git_commit
     __commit_id__ = commit_id = git_commit

gpustack_runner-0.1.22.post5/gpustack_runner/_version_appendix.py

@@ -0,0 +1 @@
+git_commit = "18bd835"

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/gpustack_runner/runner.py.json

@@ -901,6 +901,28 @@
     "docker_image": "gpustack/runner:cuda12.8-vllm0.10.0",
     "deprecated": false
   },
+  {
+    "backend": "cuda",
+    "backend_version": "12.8",
+    "original_backend_version": "12.8.1",
+    "backend_variant": "",
+    "service": "voxbox",
+    "service_version": "0.0.21",
+    "platform": "linux/amd64",
+    "docker_image": "gpustack/runner:cuda12.8-voxbox0.0.21",
+    "deprecated": false
+  },
+  {
+    "backend": "cuda",
+    "backend_version": "12.8",
+    "original_backend_version": "12.8.1",
+    "backend_variant": "",
+    "service": "voxbox",
+    "service_version": "0.0.21",
+    "platform": "linux/arm64",
+    "docker_image": "gpustack/runner:cuda12.8-voxbox0.0.21",
+    "deprecated": false
+  },
   {
     "backend": "cuda",
     "backend_version": "12.8",
@@ -1077,6 +1099,28 @@
     "docker_image": "gpustack/runner:cuda12.6-vllm0.10.0",
     "deprecated": false
   },
+  {
+    "backend": "cuda",
+    "backend_version": "12.6",
+    "original_backend_version": "12.6.3",
+    "backend_variant": "",
+    "service": "voxbox",
+    "service_version": "0.0.21",
+    "platform": "linux/amd64",
+    "docker_image": "gpustack/runner:cuda12.6-voxbox0.0.21",
+    "deprecated": false
+  },
+  {
+    "backend": "cuda",
+    "backend_version": "12.6",
+    "original_backend_version": "12.6.3",
+    "backend_variant": "",
+    "service": "voxbox",
+    "service_version": "0.0.21",
+    "platform": "linux/arm64",
+    "docker_image": "gpustack/runner:cuda12.6-voxbox0.0.21",
+    "deprecated": false
+  },
   {
     "backend": "cuda",
     "backend_version": "12.6",

gpustack_runner-0.1.22.post5/pack/.post_operation/20251224_mindie_patch_atb_config/cann/Dockerfile

@@ -0,0 +1,25 @@
+ARG CMAKE_MAX_JOBS
+ARG CANN_VERSION=8.3
+ARG CANN_ARCHS=910b
+ARG MINDIE_VERSION=2.2.rc1
+
+FROM gpustack/runner:cann${CANN_VERSION}-${CANN_ARCHS}-mindie${MINDIE_VERSION} AS mindie
+SHELL ["/bin/bash", "-eo", "pipefail", "-c"]
+
+ARG TARGETPLATFORM
+ARG TARGETOS
+ARG TARGETARCH
+
+## Patch ATB Config
+
+RUN <<EOF
+    # Patch ATB Config
+
+    sed -i "s/\"ep_level\": 2/\"ep_level\": 1/g" ${CANN_HOME}/atb-models/atb_llm/conf/config.json
+
+EOF
+
+## Entrypoint
+
+WORKDIR /
+ENTRYPOINT [ "tini", "--" ]

gpustack_runner-0.1.22.post5/pack/.post_operation/20251224_mindie_patch_atb_config/matrix.yaml

@@ -0,0 +1,33 @@
+rules:
+
+  #
+  # Ascend CANN
+  #
+
+  ## Packed Ascend CANN 8.3, using CANN Kernel for A3.
+  ##
+  - backend: "cann"
+    services:
+      - "mindie"
+    args:
+      - "CANN_VERSION=8.3"
+      - "CANN_ARCHS=a3"
+      - "MINDIE_VERSION=2.2.rc1"
+  ## Packed Ascend CANN 8.3, using CANN Kernel for 910B.
+  ##
+  - backend: "cann"
+    services:
+      - "mindie"
+    args:
+      - "CANN_VERSION=8.3"
+      - "CANN_ARCHS=910b"
+      - "MINDIE_VERSION=2.2.rc1"
+  ## Packed Ascend CANN 8.3, using CANN Kernel for 310P.
+  ##
+  - backend: "cann"
+    services:
+      - "mindie"
+    args:
+      - "CANN_VERSION=8.3"
+      - "CANN_ARCHS=310p"
+      - "MINDIE_VERSION=2.2.rc1"

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/pack/.post_operation/README.md

@@ -31,3 +31,4 @@ We leverage the matrix expansion feature of GPUStack Runner to achieve this, and
 - [x] 2025-12-16: Uninstall `runai-model-streamer` packages from SGLang 0.5.6.post2 for CUDA released images.
 - [x] 2025-12-19: Install `vLLM[audio]` packages for vLLM 0.12.0/0.11.2 of CUDA/ROCm released images.
 - [x] 2025-12-19: Install `petit-kernel` package for vLLM 0.12.0/0.11.2 and SGLang 0.5.6.post2/0.5.5.post3 of ROcm released images.
+- [x] 2025-12-24: Apply ATB config patches to MindIE 2.2.rc1 for CANN released images.

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/pack/cann/Dockerfile

@@ -583,8 +583,10 @@ RUN --mount=type=bind,target=/workspace,rw <<EOF
     # Patch
 
     unzip /workspace/patches/mindie.zip -d /workspace/patches
-    pushd /usr/local/Ascend/atb-models \
+    pushd ${CANN_HOME}/atb-models \
         && patch -p1 < /workspace/patches/mindie/*.patch
+
+    sed -i "s/\"ep_level\": 2/\"ep_level\": 1/g" ${CANN_HOME}/atb-models/atb_llm/conf/config.json
 EOF
 
 ## Postprocess

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/pack/cuda/Dockerfile

@@ -95,7 +95,7 @@ ARG CMAKE_MAX_JOBS
 ARG CUDA_VERSION=12.9.1
 ARG CUDA_ARCHS
 ARG VOXBOX_BASE_IMAGE=gpustack/runner:cuda${CUDA_VERSION}-python${PYTHON_VERSION}
-ARG VOXBOX_VERSION=0.0.20
+ARG VOXBOX_VERSION=0.0.21
 ARG VOXBOX_TORCH_VERSION=2.7.1
 ARG VOXBOX_TORCH_CUDA_VERSION=${CUDA_VERSION}
 ARG VLLM_BASE_IMAGE=gpustack/runner:cuda${CUDA_VERSION}-python${PYTHON_VERSION}

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/pack/matrix.yaml

@@ -102,7 +102,6 @@ rules:
   ##
   - backend: "cuda"
     services:
-      - "voxbox"
       - "vllm"
       - "sglang"
     args:

{gpustack_runner-0.1.22.post3 → gpustack_runner-0.1.22.post5}/tests/gpustack_runner/fixtures/test_list_runners_by_backend.json

@@ -923,6 +923,28 @@
         "docker_image": "gpustack/runner:cuda12.8-vllm0.10.0",
         "deprecated": false
       },
+      {
+        "backend": "cuda",
+        "backend_version": "12.8",
+        "original_backend_version": "12.8.1",
+        "backend_variant": "",
+        "service": "voxbox",
+        "service_version": "0.0.21",
+        "platform": "linux/amd64",
+        "docker_image": "gpustack/runner:cuda12.8-voxbox0.0.21",
+        "deprecated": false
+      },
+      {
+        "backend": "cuda",
+        "backend_version": "12.8",
+        "original_backend_version": "12.8.1",
+        "backend_variant": "",
+        "service": "voxbox",
+        "service_version": "0.0.21",
+        "platform": "linux/arm64",
+        "docker_image": "gpustack/runner:cuda12.8-voxbox0.0.21",
+        "deprecated": false
+      },
       {
         "backend": "cuda",
         "backend_version": "12.8",
@@ -1099,6 +1121,28 @@
         "docker_image": "gpustack/runner:cuda12.6-vllm0.10.0",
         "deprecated": false
       },
+      {
+        "backend": "cuda",
+        "backend_version": "12.6",
+        "original_backend_version": "12.6.3",
+        "backend_variant": "",
+        "service": "voxbox",
+        "service_version": "0.0.21",
+        "platform": "linux/amd64",
+        "docker_image": "gpustack/runner:cuda12.6-voxbox0.0.21",
+        "deprecated": false
+      },
+      {
+        "backend": "cuda",
+        "backend_version": "12.6",
+        "original_backend_version": "12.6.3",
+        "backend_variant": "",
+        "service": "voxbox",
+        "service_version": "0.0.21",
+        "platform": "linux/arm64",
+        "docker_image": "gpustack/runner:cuda12.6-voxbox0.0.21",
+        "deprecated": false
+      },
       {
         "backend": "cuda",
         "backend_version": "12.6",

gpustack_runner-0.1.22.post3/gpustack_runner/_version_appendix.py

@@ -1 +0,0 @@
-git_commit = "c9f91c3"