gpustack-runner 0.1.22.post6__tar.gz → 0.1.23__tar.gz

{gpustack_runner-0.1.22.post6 → gpustack_runner-0.1.23}/.gitignore

@@ -57,3 +57,6 @@ coverage.xml
 *.cover
 *.py,cover
 .hypothesis/
+
+# Test
+pack/cann/mindie-*/

{gpustack_runner-0.1.22.post6 → gpustack_runner-0.1.23}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gpustack-runner
-Version: 0.1.22.post6
+Version: 0.1.23
 Summary: GPUStack Runner is library for registering runnable accelerated backends and services in GPUStack.
 Project-URL: Homepage, https://github.com/gpustack/runner
 Project-URL: Bug Tracker, https://github.com/gpustack/gpustack/issues
@@ -46,26 +46,24 @@ The following table lists the supported accelerated backends and their correspon
 
 ### Ascend CANN
 
+> [!CAUTION]
+> Since v0.1.23:
+> - Deprecated MindIE `2.1.rc1`.
+
 > [!WARNING]
 > - The Atlas 300I series is currently experimental in vLLM, only supporting eager mode and float16 data type. And there
     are some known issues for running vLLM, you can refer to
     vllm-ascend [#3316](https://github.com/vllm-project/vllm-ascend/issues/3316)
     and [#2795](https://github.com/vllm-project/vllm-ascend/issues/2795).
 
-> [!IMPORTANT]
-> - Applied [ATB model patched](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3646603380) to MindIE 2.2.rc1/2.1.rc2.
-> - Applied [ATB config patched](https://github.com/gpustack/gpustack/issues/3551) to MindIE 2.2.rc1.
-> - Applied [av package](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3631228085) to MindIE 2.2.rc1/2.1.rc2.
-> - Update vLLM 0.11.0 with stable vLLM Ascend plugin.
-
-| CANN Version <br/> (Variant) | MindIE                   | vLLM                                                                     | SGLang                 |
-|------------------------------|--------------------------|--------------------------------------------------------------------------|------------------------|
-| 8.3 (A3/910C)                | **`2.2.rc1`**            | `0.12.0`, **`0.11.0`**                                                   | `0.5.6.post2`          |
-| 8.3 (910B)                   | **`2.2.rc1`**            | `0.12.0`, **`0.11.0`**                                                   | `0.5.6.post2`          |
-| 8.3 (310P)                   | **`2.2.rc1`**            |                                                                          |                        |
-| 8.2 (A3/910C)                | **`2.1.rc2`**            | ~~`0.11.0`~~, `0.10.2`, <br/>`0.10.1.1`                                  | `0.5.2`, `0.5.1.post3` |
-| 8.2 (910B)                   | **`2.1.rc2`**, `2.1.rc1` | ~~`0.11.0`~~, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`, <br/>`0.9.2`, `0.9.1` | `0.5.2`, `0.5.1.post3` |
-| 8.2 (310P)                   | **`2.1.rc2`**, `2.1.rc1` | `0.10.0`, `0.9.2`                                                        |                        |
+| CANN Version <br/> (Variant) | MindIE                   | vLLM                                                       | SGLang                 |
+|------------------------------|--------------------------|------------------------------------------------------------|------------------------|
+| 8.3 (A3/910C)                | `2.2.rc1`                | `0.13.0`, `0.12.0`, `0.11.0`                               | `0.5.6.post2`          |
+| 8.3 (910B)                   | `2.2.rc1`                | `0.13.0`, `0.12.0`, `0.11.0`                               | `0.5.6.post2`          |
+| 8.3 (310P)                   | `2.2.rc1`                |                                                            |                        |
+| 8.2 (A3/910C)                | `2.1.rc2`                | `0.10.2`, `0.10.1.1`                                       | `0.5.2`, `0.5.1.post3` |
+| 8.2 (910B)                   | `2.1.rc2`, ~~`2.1.rc1`~~ | `0.10.2`, `0.10.1.1`, <br/>`0.10.0`, `0.9.2`, <br/>`0.9.1` | `0.5.2`, `0.5.1.post3` |
+| 8.2 (310P)                   | `2.1.rc2`, ~~`2.1.rc1`~~ | `0.10.0`, `0.9.2`                                          |                        |
 
 ### Iluvatar CoreX
 
@@ -75,6 +73,13 @@ The following table lists the supported accelerated backends and their correspon
 
 ### NVIDIA CUDA
 
+> [!CAUTION]
+> Since v0.1.23:
+> - Deprecated all services for CUDA 12.4.
+> - Deprecated vLLM `0.11.0`, `0.10.1.1`, `0.10.0`.
+> - Deprecated SGLang `0.5.5`.
+> - Deprecated VoxBox `0.0.20`.
+
 > [!NOTE]
 > - CUDA 12.9 supports Compute Capabilities:
     `7.5 8.0+PTX 8.9 9.0 10.0 10.3 12.0 12.1+PTX`.
@@ -83,16 +88,12 @@ The following table lists the supported accelerated backends and their correspon
 > - CUDA 12.6/12.4 supports Compute Capabilities:
     `7.5 8.0+PTX 8.9 9.0+PTX`.
 
-> [!IMPORTANT]
-> - Applied [Qwen2.5 VL patched](https://github.com/gpustack/gpustack/issues/3606) to vLLM 0.11.2.
-> - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
-
-| CUDA Version <br/> (Variant) | vLLM                                                                                      | SGLang                                                    | VoxBox             |
-|------------------------------|-------------------------------------------------------------------------------------------|-----------------------------------------------------------|--------------------|
-| 12.9                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**                                                     | `0.5.6.post2`                                             |                    |
-| 12.8                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`, `0.5.5.post3`, <br/>`0.5.5`, `0.5.4.post3` | `0.0.21`, `0.0.20` |
-| 12.6                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`                                             | `0.0.21`, `0.0.20` |
-| 12.4                         | `0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`                                             |                                                           | `0.0.20`           |
+| CUDA Version <br/> (Variant) | vLLM                                                                                              | SGLang                                                        | VoxBox                 |
+|------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------------------------|------------------------|
+| 12.9                         | `0.13.0`, `0.12.0`, <br/>`0.11.2`                                                                 | `0.5.6.post2`                                                 |                        |
+| 12.8                         | `0.13.0`, `0.12.0`, <br/>`0.11.2`, ~~`0.11.0`~~, <br/>`0.10.2`, ~~`0.10.1.1`~~, <br/>~~`0.10.0`~~ | `0.5.6.post2`, `0.5.5.post3`, <br/>~~`0.5.5`~~, `0.5.4.post3` | `0.0.21`, ~~`0.0.20`~~ |
+| 12.6                         | `0.13.0`, `0.12.0`, <br/>`0.11.2`, ~~`0.11.0`~~, <br/>`0.10.2`, ~~`0.10.1.1`~~, <br/>~~`0.10.0`~~ | `0.5.6.post2`                                                 | `0.0.21`, ~~`0.0.20`~~ |
+| 12.4                         | ~~`0.11.0`~~, ~~`0.10.2`~~, <br/>~~`0.10.1.1`~~, ~~`0.10.0`~~                                     |                                                               | ~~`0.0.20`~~           |
 
 ### Hygon DTK
 
@@ -109,6 +110,11 @@ The following table lists the supported accelerated backends and their correspon
 
 ### AMD ROCm
 
+> [!CAUTION]
+> Since v0.1.23:
+> Deprecated all services for ROCm 6.3.
+> Deprecated vLLM `0.11.0`.
+
 > [!NOTE]
 > - ROCm 7.0 supports LLVM targets:
     `gfx908 gfx90a gfx942 gfx950 gfx1030 gfx1100 gfx1101 gfx1200 gfx1201 gfx1150 gfx1151`.
@@ -118,18 +124,15 @@ The following table lists the supported accelerated backends and their correspon
 > [!WARNING]
 > - ROCm 7.0 vLLM `0.11.2/0.11.0` are reusing the official ROCm 6.4 PyTorch 2.9 wheel package rather than a ROCm
     7.0 specific PyTorch build. Although supports ROCm 7.0 in vLLM `0.11.2/0.11.0`, `gfx1150/gfx1151` are not supported yet.
-> - SGLang supports `gfx942` only.
 > - ROCm 6.4 vLLM `0.13.0` supports `gfx903 gfx90a gfx942` only.
-
-> [!IMPORTANT]
-> - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
-> - Applied [petit-kernel package](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L728) to vLLM 0.11.2 and SGLang 0.5.5.post3.
-
-| ROCm Version <br/> (Variant) | vLLM                                            | SGLang                           |
-|------------------------------|-------------------------------------------------|----------------------------------|
-| 7.0                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0` | `0.5.6.post2`                    |
-| 6.4                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.10.2` | `0.5.6.post2`, **`0.5.5.post3`** |
-| 6.3                          | `0.10.1.1`, `0.10.0`                            |                                  |
+> - ROCm 6.4 SGLang supports `gfx942` only.
+> - ROCm 7.0 SGLang supports `gfx950` only.
+
+| ROCm Version <br/> (Variant) | vLLM                                            | SGLang                       |
+|------------------------------|-------------------------------------------------|------------------------------|
+| 7.0                          | `0.13.0`, `0.12.0`, <br/>`0.11.2`, ~~`0.11.0`~~ | `0.5.6.post2`                |
+| 6.4                          | `0.13.0`, `0.12.0`, <br/>`0.11.2`, `0.10.2`     | `0.5.6.post2`, `0.5.5.post3` |
+| 6.3                          | ~~`0.10.1.1`~~, ~~`0.10.0`~~                    |                              |
 
 ## Directory Structure
 

{gpustack_runner-0.1.22.post6 → gpustack_runner-0.1.23}/README.md

@@ -26,26 +26,24 @@ The following table lists the supported accelerated backends and their correspon
 
 ### Ascend CANN
 
+> [!CAUTION]
+> Since v0.1.23:
+> - Deprecated MindIE `2.1.rc1`.
+
 > [!WARNING]
 > - The Atlas 300I series is currently experimental in vLLM, only supporting eager mode and float16 data type. And there
     are some known issues for running vLLM, you can refer to
     vllm-ascend [#3316](https://github.com/vllm-project/vllm-ascend/issues/3316)
     and [#2795](https://github.com/vllm-project/vllm-ascend/issues/2795).
 
-> [!IMPORTANT]
-> - Applied [ATB model patched](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3646603380) to MindIE 2.2.rc1/2.1.rc2.
-> - Applied [ATB config patched](https://github.com/gpustack/gpustack/issues/3551) to MindIE 2.2.rc1.
-> - Applied [av package](https://github.com/gpustack/gpustack/issues/2016#issuecomment-3631228085) to MindIE 2.2.rc1/2.1.rc2.
-> - Update vLLM 0.11.0 with stable vLLM Ascend plugin.
-
-| CANN Version <br/> (Variant) | MindIE                   | vLLM                                                                     | SGLang                 |
-|------------------------------|--------------------------|--------------------------------------------------------------------------|------------------------|
-| 8.3 (A3/910C)                | **`2.2.rc1`**            | `0.12.0`, **`0.11.0`**                                                   | `0.5.6.post2`          |
-| 8.3 (910B)                   | **`2.2.rc1`**            | `0.12.0`, **`0.11.0`**                                                   | `0.5.6.post2`          |
-| 8.3 (310P)                   | **`2.2.rc1`**            |                                                                          |                        |
-| 8.2 (A3/910C)                | **`2.1.rc2`**            | ~~`0.11.0`~~, `0.10.2`, <br/>`0.10.1.1`                                  | `0.5.2`, `0.5.1.post3` |
-| 8.2 (910B)                   | **`2.1.rc2`**, `2.1.rc1` | ~~`0.11.0`~~, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`, <br/>`0.9.2`, `0.9.1` | `0.5.2`, `0.5.1.post3` |
-| 8.2 (310P)                   | **`2.1.rc2`**, `2.1.rc1` | `0.10.0`, `0.9.2`                                                        |                        |
+| CANN Version <br/> (Variant) | MindIE                   | vLLM                                                       | SGLang                 |
+|------------------------------|--------------------------|------------------------------------------------------------|------------------------|
+| 8.3 (A3/910C)                | `2.2.rc1`                | `0.13.0`, `0.12.0`, `0.11.0`                               | `0.5.6.post2`          |
+| 8.3 (910B)                   | `2.2.rc1`                | `0.13.0`, `0.12.0`, `0.11.0`                               | `0.5.6.post2`          |
+| 8.3 (310P)                   | `2.2.rc1`                |                                                            |                        |
+| 8.2 (A3/910C)                | `2.1.rc2`                | `0.10.2`, `0.10.1.1`                                       | `0.5.2`, `0.5.1.post3` |
+| 8.2 (910B)                   | `2.1.rc2`, ~~`2.1.rc1`~~ | `0.10.2`, `0.10.1.1`, <br/>`0.10.0`, `0.9.2`, <br/>`0.9.1` | `0.5.2`, `0.5.1.post3` |
+| 8.2 (310P)                   | `2.1.rc2`, ~~`2.1.rc1`~~ | `0.10.0`, `0.9.2`                                          |                        |
 
 ### Iluvatar CoreX
 
@@ -55,6 +53,13 @@ The following table lists the supported accelerated backends and their correspon
 
 ### NVIDIA CUDA
 
+> [!CAUTION]
+> Since v0.1.23:
+> - Deprecated all services for CUDA 12.4.
+> - Deprecated vLLM `0.11.0`, `0.10.1.1`, `0.10.0`.
+> - Deprecated SGLang `0.5.5`.
+> - Deprecated VoxBox `0.0.20`.
+
 > [!NOTE]
 > - CUDA 12.9 supports Compute Capabilities:
     `7.5 8.0+PTX 8.9 9.0 10.0 10.3 12.0 12.1+PTX`.
@@ -63,16 +68,12 @@ The following table lists the supported accelerated backends and their correspon
 > - CUDA 12.6/12.4 supports Compute Capabilities:
     `7.5 8.0+PTX 8.9 9.0+PTX`.
 
-> [!IMPORTANT]
-> - Applied [Qwen2.5 VL patched](https://github.com/gpustack/gpustack/issues/3606) to vLLM 0.11.2.
-> - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
-
-| CUDA Version <br/> (Variant) | vLLM                                                                                      | SGLang                                                    | VoxBox             |
-|------------------------------|-------------------------------------------------------------------------------------------|-----------------------------------------------------------|--------------------|
-| 12.9                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**                                                     | `0.5.6.post2`                                             |                    |
-| 12.8                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`, `0.5.5.post3`, <br/>`0.5.5`, `0.5.4.post3` | `0.0.21`, `0.0.20` |
-| 12.6                         | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0`, <br/>`0.10.2`, `0.10.1.1`, <br/>`0.10.0` | `0.5.6.post2`                                             | `0.0.21`, `0.0.20` |
-| 12.4                         | `0.11.0`, `0.10.2`, <br/>`0.10.1.1`, `0.10.0`                                             |                                                           | `0.0.20`           |
+| CUDA Version <br/> (Variant) | vLLM                                                                                              | SGLang                                                        | VoxBox                 |
+|------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------------------------|------------------------|
+| 12.9                         | `0.13.0`, `0.12.0`, <br/>`0.11.2`                                                                 | `0.5.6.post2`                                                 |                        |
+| 12.8                         | `0.13.0`, `0.12.0`, <br/>`0.11.2`, ~~`0.11.0`~~, <br/>`0.10.2`, ~~`0.10.1.1`~~, <br/>~~`0.10.0`~~ | `0.5.6.post2`, `0.5.5.post3`, <br/>~~`0.5.5`~~, `0.5.4.post3` | `0.0.21`, ~~`0.0.20`~~ |
+| 12.6                         | `0.13.0`, `0.12.0`, <br/>`0.11.2`, ~~`0.11.0`~~, <br/>`0.10.2`, ~~`0.10.1.1`~~, <br/>~~`0.10.0`~~ | `0.5.6.post2`                                                 | `0.0.21`, ~~`0.0.20`~~ |
+| 12.4                         | ~~`0.11.0`~~, ~~`0.10.2`~~, <br/>~~`0.10.1.1`~~, ~~`0.10.0`~~                                     |                                                               | ~~`0.0.20`~~           |
 
 ### Hygon DTK
 
@@ -89,6 +90,11 @@ The following table lists the supported accelerated backends and their correspon
 
 ### AMD ROCm
 
+> [!CAUTION]
+> Since v0.1.23:
+> Deprecated all services for ROCm 6.3.
+> Deprecated vLLM `0.11.0`.
+
 > [!NOTE]
 > - ROCm 7.0 supports LLVM targets:
     `gfx908 gfx90a gfx942 gfx950 gfx1030 gfx1100 gfx1101 gfx1200 gfx1201 gfx1150 gfx1151`.
@@ -98,18 +104,15 @@ The following table lists the supported accelerated backends and their correspon
 > [!WARNING]
 > - ROCm 7.0 vLLM `0.11.2/0.11.0` are reusing the official ROCm 6.4 PyTorch 2.9 wheel package rather than a ROCm
     7.0 specific PyTorch build. Although supports ROCm 7.0 in vLLM `0.11.2/0.11.0`, `gfx1150/gfx1151` are not supported yet.
-> - SGLang supports `gfx942` only.
 > - ROCm 6.4 vLLM `0.13.0` supports `gfx903 gfx90a gfx942` only.
-
-> [!IMPORTANT]
-> - Applied [vLLM[audio] packages](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L720-L724) to vLLM 0.11.2.
-> - Applied [petit-kernel package](https://github.com/vllm-project/vllm/blob/275de34170654274616082721348b7edd9741d32/setup.py#L728) to vLLM 0.11.2 and SGLang 0.5.5.post3.
-
-| ROCm Version <br/> (Variant) | vLLM                                            | SGLang                           |
-|------------------------------|-------------------------------------------------|----------------------------------|
-| 7.0                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.11.0` | `0.5.6.post2`                    |
-| 6.4                          | `0.13.0`, `0.12.0`, <br/>**`0.11.2`**, `0.10.2` | `0.5.6.post2`, **`0.5.5.post3`** |
-| 6.3                          | `0.10.1.1`, `0.10.0`                            |                                  |
+> - ROCm 6.4 SGLang supports `gfx942` only.
+> - ROCm 7.0 SGLang supports `gfx950` only.
+
+| ROCm Version <br/> (Variant) | vLLM                                            | SGLang                       |
+|------------------------------|-------------------------------------------------|------------------------------|
+| 7.0                          | `0.13.0`, `0.12.0`, <br/>`0.11.2`, ~~`0.11.0`~~ | `0.5.6.post2`                |
+| 6.4                          | `0.13.0`, `0.12.0`, <br/>`0.11.2`, `0.10.2`     | `0.5.6.post2`, `0.5.5.post3` |
+| 6.3                          | ~~`0.10.1.1`~~, ~~`0.10.0`~~                    |                              |
 
 ## Directory Structure
 

{gpustack_runner-0.1.22.post6 → gpustack_runner-0.1.23}/gpustack_runner/_version.py

@@ -27,8 +27,8 @@ version_tuple: VERSION_TUPLE
 __commit_id__: COMMIT_ID
 commit_id: COMMIT_ID
 
-__version__ = version = '0.1.22.post6'
-__version_tuple__ = version_tuple = (0, 1, 22, 'post6')
+__version__ = version = '0.1.23'
+__version_tuple__ = version_tuple = (0, 1, 23)
 try:
     from ._version_appendix import git_commit
     __commit_id__ = commit_id = git_commit

gpustack_runner-0.1.23/gpustack_runner/_version_appendix.py

@@ -0,0 +1 @@
+git_commit = "60fcf6e"

{gpustack_runner-0.1.22.post6 → gpustack_runner-0.1.23}/gpustack_runner/cmds/images.py

@@ -46,6 +46,10 @@ _AVAILABLE_PLATFORMS = [
 ]
 
 
+# Disable overriding default namespace at images operations.
+os.environ["GPUSTACK_RUNNER_DEFAULT_NAMESPACE"] = "gpustack"
+
+
 class ListImagesSubCommand(SubCommand):
     """
     Command to list images.
@@ -440,14 +444,14 @@ class SaveImagesSubCommand(SubCommand):
 
                 command = [
                     "skopeo",
-                    "copy",
-                    "--src-tls-verify=false",
-                    "--retry-times",
-                    str(self.max_retries),
                     "--override-os",
                     override_os,
                     "--override-arch",
                     override_arch,
+                    "copy",
+                    "--src-tls-verify=false",
+                    "--retry-times",
+                    str(self.max_retries),
                 ]
                 if self.source_username and self.source_password:
                     command.extend(
@@ -767,10 +771,6 @@ class CopyImagesSubCommand(SubCommand):
                 print(f"❌ Error syncing image '{img_name}'")
                 failures.append((img_name, img_err))
 
-            override_os, override_arch = None, None
-            if self.platform:
-                override_os, override_arch = self.platform.split("/", maxsplit=1)
-
             # Submit tasks
             for img in images:
                 command = [
@@ -778,20 +778,10 @@ class CopyImagesSubCommand(SubCommand):
                     "copy",
                     "--src-tls-verify=false",
                     "--dest-tls-verify=false",
+                    "--all",
                     "--retry-times",
                     str(self.max_retries),
                 ]
-                if override_os and override_arch:
-                    command.extend(
-                        [
-                            "--override-os",
-                            override_os,
-                            "--override-arch",
-                            override_arch,
-                        ],
-                    )
-                else:
-                    command.append("--all")
                 if self.source_username and self.source_password:
                     command.extend(
                         [

gpustack_runner-0.1.23/gpustack_runner/envs.py

@@ -0,0 +1,112 @@
+from __future__ import annotations
+
+from functools import lru_cache
+from os import getenv as sys_getenv
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from collections.abc import Callable
+
+    # Global
+
+    GPUSTACK_RUNNER_DEFAULT_NAMESPACE: str | None = None
+    """
+    Namespace for default runner images.
+    If not set, it should be "gpustack".
+    """
+
+# --8<-- [start:env-vars-definition]
+
+variables: dict[str, Callable[[], Any]] = {
+    # Global
+    "GPUSTACK_RUNNER_DEFAULT_NAMESPACE": lambda: trim_str(
+        getenvs(
+            keys=[
+                "GPUSTACK_RUNNER_DEFAULT_NAMESPACE",
+                "GPUSTACK_RUNTIME_DEPLOY_DEFAULT_NAMESPACE",  ## Compatible with gpustack/gpustack_runtime.
+            ],
+        ),
+    ),
+}
+
+
+# --8<-- [end:env-vars-definition]
+
+
+@lru_cache
+def __getattr__(name: str):
+    # lazy evaluation of environment variables
+    if name in variables:
+        return variables[name]()
+    msg = f"module {__name__} has no attribute {name}"
+    raise AttributeError(msg)
+
+
+def __dir__():
+    return list(variables.keys())
+
+
+def trim_str(value: str | None) -> str | None:
+    """
+    Trim leading and trailing whitespace from a string.
+
+    Args:
+        value:
+            The string to trim.
+
+    Returns:
+        The trimmed string, or None if the input is None.
+
+    """
+    if value is not None:
+        return value.strip()
+    return None
+
+
+_ENV_PREFIX = "GPUSTACK_RUNNER_"
+
+
+def getenv(key: str, default=None) -> any | None:
+    """
+    Get the value of an environment variable.
+    Try headless module variable if the key starts with "GPUSTACK_RUNNER_".
+
+    Args:
+        key:
+            The environment variable key.
+        default:
+            The default value if the key is not found.
+
+    Returns:
+        The value of the environment variable if it exists, otherwise None.
+
+    """
+    value = sys_getenv(key)
+    if value is not None:
+        return value
+    if key.startswith(_ENV_PREFIX):
+        headless_key = key.removeprefix(_ENV_PREFIX)
+        return sys_getenv(headless_key, default)
+    return default
+
+
+def getenvs(keys: list[str], default=None) -> any | None:
+    """
+    Get the value of an environment variable.
+    Return the first found value among the provided keys.
+
+    Args:
+        keys:
+            The environment variable key(s).
+        default:
+            The default value if none of the keys are found.
+
+    Returns:
+        The value of the environment variable if it exists, otherwise None.
+
+    """
+    for key in keys:
+        value = getenv(key)
+        if value is not None:
+            return value
+    return default

{gpustack_runner-0.1.22.post6 → gpustack_runner-0.1.23}/gpustack_runner/runner.py

@@ -10,13 +10,15 @@ from typing import Any
 
 from dataclasses_json import dataclass_json
 
+from . import envs
+
 _RE_DOCKER_IMAGE = re.compile(
-    r"(?:(?P<prefix>[\w\\.\-]+(?:/[\w\\.\-]+)*)/)?gpustack/runner:(?P<backend>(Host|cann|corex|cuda|dtk|maca|rocm))(?P<backend_version>[XY\d\\.]+)(?:-(?P<backend_variant>\w+))?-(?P<service>(vllm|voxbox|mindie|sglang))(?P<service_version>[\w\\.]+)(?:-(?P<suffix>\w+))?",
+    r"(?:(?P<prefix>[\w\\.\-]+(?:/[\w\\.\-]+)*)/)?runner:(?P<backend>(Host|cann|corex|cuda|dtk|maca|rocm))(?P<backend_version>[XY\d\\.]+)(?:-(?P<backend_variant>\w+))?-(?P<service>(vllm|voxbox|mindie|sglang))(?P<service_version>[\w\\.]+)(?:-(?P<suffix>\w+))?",
 )
 """
 Regex for Docker image parsing,
 which captures the following named groups:
-    - `prefix`: The optional prefix before `gpustack/runner`, e.g. a registry URL or namespace.
+    - `prefix`: The optional prefix before `runner`, e.g. a registry URL or namespace.
     - `backend`: The backend name, e.g. "cann", "cuda", "rocm", etc.
     - `backend_version`: The backend version, ignored patch version, e.g. "8.2", "12.4", "6.3", etc.
     - `backend_variant`: The optional backend variant, e.g. "910b", etc.
@@ -33,7 +35,7 @@ def set_re_docker_image(pattern: str):
     Args:
         pattern:
             The regex pattern to set. It should capture the following named groups:
-            - `prefix`: The optional prefix before `gpustack/runner`, e.g. a registry URL or namespace.
+            - `prefix`: The optional prefix before `runner`, e.g. a registry URL or namespace.
             - `backend`: The backend name, e.g. "cann", "cuda",
             - `backend_version`: The backend version, ignored patch version, e.g. "8.2", "12.4", "6.3", etc.
             - `backend_variant`: The optional backend variant, e.g. "910b", etc
@@ -82,7 +84,7 @@ class DockerImage:
         Parse the Docker image string into a DockerImage object.
 
         The given image string must follow the below regex format:
-        `[prefix/]gpustack/runner:{backend}{backend_version}[-backend_variant]-{service}{service_version}[-suffix]`
+        `[prefix/]runner:{backend}{backend_version}[-backend_variant]-{service}{service_version}[-suffix]`
 
         Args:
             image:
@@ -100,7 +102,7 @@ class DockerImage:
     def __str__(self):
         parts = [
             "",
-            "gpustack/runner:",
+            "runner:",
             self.backend,
             self.backend_version,
         ]
@@ -235,7 +237,13 @@ def list_runners(**kwargs) -> Runners | list[dict]:
             data_path = Path(_data_path) if isinstance(_data_path, str) else _data_path
     with data_path.open("r", encoding="utf-8") as f:
         json_list = json.load(f)
-        runners = [Runner.from_dict(item) for item in json_list]
+        runners = []
+        for item in json_list:
+            if namespace := envs.GPUSTACK_RUNNER_DEFAULT_NAMESPACE:
+                docker_image = item["docker_image"]
+                docker_image = docker_image.replace("gpustack/", f"{namespace}/")
+                item["docker_image"] = docker_image
+            runners.append(Runner.from_dict(item))
 
     todict = kwargs.pop("todict", False)
     if not kwargs: