dstack 0.19.31__tar.gz → 0.19.32__tar.gz

{dstack-0.19.31 → dstack-0.19.32}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dstack
-Version: 0.19.31
+Version: 0.19.32
 Summary: dstack is an open-source orchestration engine for running AI workloads on any cloud or on-premises.
 Project-URL: Homepage, https://dstack.ai
 Project-URL: Source, https://github.com/dstackai/dstack
@@ -73,7 +73,7 @@ Requires-Dist: grpcio>=1.50; extra == 'all'
 Requires-Dist: httpx; extra == 'all'
 Requires-Dist: jinja2; extra == 'all'
 Requires-Dist: kubernetes; extra == 'all'
-Requires-Dist: nebius<0.3,>=0.2.40; (python_version >= '3.10') and extra == 'all'
+Requires-Dist: nebius<=0.2.72,>=0.2.40; (python_version >= '3.10') and extra == 'all'
 Requires-Dist: oci>=2.150.0; extra == 'all'
 Requires-Dist: prometheus-client; extra == 'all'
 Requires-Dist: pyopenssl>=23.2.0; extra == 'all'
@@ -259,7 +259,7 @@ Requires-Dist: fastapi; extra == 'nebius'
 Requires-Dist: grpcio>=1.50; extra == 'nebius'
 Requires-Dist: httpx; extra == 'nebius'
 Requires-Dist: jinja2; extra == 'nebius'
-Requires-Dist: nebius<0.3,>=0.2.40; (python_version >= '3.10') and extra == 'nebius'
+Requires-Dist: nebius<=0.2.72,>=0.2.40; (python_version >= '3.10') and extra == 'nebius'
 Requires-Dist: prometheus-client; extra == 'nebius'
 Requires-Dist: python-dxf==12.1.0; extra == 'nebius'
 Requires-Dist: python-json-logger>=3.1.0; extra == 'nebius'
@@ -340,15 +340,13 @@ It streamlines development, training, and inference, and is compatible with any
 `dstack` supports `NVIDIA`, `AMD`, `Google TPU`, `Intel Gaudi`, and `Tenstorrent` accelerators out of the box.
 
 ## Latest news ✨
-- [2025/09] [dstack 0.19.27: Offers UI, Digital Ocean and AMD Developer Cloud](https://github.com/dstackai/dstack/releases/tag/0.19.27)
-- [2025/08] [dstack 0.19.26: Repos – explicit repo configuration via YAML](https://github.com/dstackai/dstack/releases/tag/0.19.26)
-- [2025/08] [dstack 0.19.25: `dstack offer` CLI command](https://github.com/dstackai/dstack/releases/tag/0.19.25)
-- [2025/08] [dstack 0.19.22: Service probes, GPU health-checks, Tenstorrent Galaxy, Secrets UI](https://github.com/dstackai/dstack/releases/tag/0.19.22)
+- [2025/10] [dstack 0.19.31: Kubernetes, GCP A4 spot](https://github.com/dstackai/dstack/releases/tag/0.19.31)
+- [2025/08] [dstack 0.19.26: Repos](https://github.com/dstackai/dstack/releases/tag/0.19.26)
+- [2025/08] [dstack 0.19.22: Service probes, GPU health-checks, Tenstorrent Galaxy](https://github.com/dstackai/dstack/releases/tag/0.19.22)
 - [2025/07] [dstack 0.19.21: Scheduled tasks](https://github.com/dstackai/dstack/releases/tag/0.19.21)
 - [2025/07] [dstack 0.19.17: Secrets, Files, Rolling deployment](https://github.com/dstackai/dstack/releases/tag/0.19.17)
-- [2025/06] [dstack 0.19.16: Docker in Docker, CloudRift](https://github.com/dstackai/dstack/releases/tag/0.19.16)
-- [2025/06] [dstack 0.19.13: InfiniBand support in default images](https://github.com/dstackai/dstack/releases/tag/0.19.13)
-- [2025/06] [dstack 0.19.12: Simplified use of MPI](https://github.com/dstackai/dstack/releases/tag/0.19.12)
+- [2025/06] [dstack 0.19.16: Docker in Docker](https://github.com/dstackai/dstack/releases/tag/0.19.16)
+- [2025/06] [dstack 0.19.13: Default images with InfiniBand support](https://github.com/dstackai/dstack/releases/tag/0.19.13)
 
 ## How does it work?
 
@@ -364,11 +362,11 @@ It streamlines development, training, and inference, and is compatible with any
 
 To orchestrate compute across cloud providers or existing Kubernetes clusters, you need to configure backends.
 
-Backends can be set up in `~/.dstack/server/config.yml` or through the [project settings page](../concepts/projects.md#backends) in the UI.
+Backends can be set up in `~/.dstack/server/config.yml` or through the [project settings page](https://dstack.ai/docs/concepts/projects#backends) in the UI.
 
-For more details, see [Backends](../concepts/backends.md).
+For more details, see [Backends](https://dstack.ai/docs/concepts/backends).
 
-> When using `dstack` with on-prem servers, backend configuration isn’t required. Simply create [SSH fleets](../concepts/fleets.md#ssh) once the server is up.
+> When using `dstack` with on-prem servers, backend configuration isn’t required. Simply create [SSH fleets](https://dstack.ai/docs/concepts/fleets#ssh) once the server is up.
 
 ##### Start the server
 

{dstack-0.19.31 → dstack-0.19.32}/README.md

@@ -23,15 +23,13 @@ It streamlines development, training, and inference, and is compatible with any
 `dstack` supports `NVIDIA`, `AMD`, `Google TPU`, `Intel Gaudi`, and `Tenstorrent` accelerators out of the box.
 
 ## Latest news ✨
-- [2025/09] [dstack 0.19.27: Offers UI, Digital Ocean and AMD Developer Cloud](https://github.com/dstackai/dstack/releases/tag/0.19.27)
-- [2025/08] [dstack 0.19.26: Repos – explicit repo configuration via YAML](https://github.com/dstackai/dstack/releases/tag/0.19.26)
-- [2025/08] [dstack 0.19.25: `dstack offer` CLI command](https://github.com/dstackai/dstack/releases/tag/0.19.25)
-- [2025/08] [dstack 0.19.22: Service probes, GPU health-checks, Tenstorrent Galaxy, Secrets UI](https://github.com/dstackai/dstack/releases/tag/0.19.22)
+- [2025/10] [dstack 0.19.31: Kubernetes, GCP A4 spot](https://github.com/dstackai/dstack/releases/tag/0.19.31)
+- [2025/08] [dstack 0.19.26: Repos](https://github.com/dstackai/dstack/releases/tag/0.19.26)
+- [2025/08] [dstack 0.19.22: Service probes, GPU health-checks, Tenstorrent Galaxy](https://github.com/dstackai/dstack/releases/tag/0.19.22)
 - [2025/07] [dstack 0.19.21: Scheduled tasks](https://github.com/dstackai/dstack/releases/tag/0.19.21)
 - [2025/07] [dstack 0.19.17: Secrets, Files, Rolling deployment](https://github.com/dstackai/dstack/releases/tag/0.19.17)
-- [2025/06] [dstack 0.19.16: Docker in Docker, CloudRift](https://github.com/dstackai/dstack/releases/tag/0.19.16)
-- [2025/06] [dstack 0.19.13: InfiniBand support in default images](https://github.com/dstackai/dstack/releases/tag/0.19.13)
-- [2025/06] [dstack 0.19.12: Simplified use of MPI](https://github.com/dstackai/dstack/releases/tag/0.19.12)
+- [2025/06] [dstack 0.19.16: Docker in Docker](https://github.com/dstackai/dstack/releases/tag/0.19.16)
+- [2025/06] [dstack 0.19.13: Default images with InfiniBand support](https://github.com/dstackai/dstack/releases/tag/0.19.13)
 
 ## How does it work?
 
@@ -50,11 +48,11 @@ It streamlines development, training, and inference, and is compatible with any
 
 To orchestrate compute across cloud providers or existing Kubernetes clusters, you need to configure backends.
 
-Backends can be set up in `~/.dstack/server/config.yml` or through the [project settings page](../concepts/projects.md#backends) in the UI.
+Backends can be set up in `~/.dstack/server/config.yml` or through the [project settings page](https://dstack.ai/docs/concepts/projects#backends) in the UI.
 
-For more details, see [Backends](../concepts/backends.md).
+For more details, see [Backends](https://dstack.ai/docs/concepts/backends).
 
-> When using `dstack` with on-prem servers, backend configuration isn’t required. Simply create [SSH fleets](../concepts/fleets.md#ssh) once the server is up.
+> When using `dstack` with on-prem servers, backend configuration isn’t required. Simply create [SSH fleets](https://dstack.ai/docs/concepts/fleets#ssh) once the server is up.
 
 ##### Start the server
 

dstack-0.19.32/docker/server/entrypoint.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+set -e
+
+if [ -n "${GOOGLE_APPLICATION_CREDENTIALS_JSON}" ]; then
+  GOOGLE_APPLICATION_CREDENTIALS_DIR="${HOME}/.config/gcloud/"
+  mkdir -p "${GOOGLE_APPLICATION_CREDENTIALS_DIR}"
+  echo "${GOOGLE_APPLICATION_CREDENTIALS_JSON}" > "${GOOGLE_APPLICATION_CREDENTIALS_DIR}/application_default_credentials.json"
+fi
+
+DB_PATH="${HOME}/.dstack/server/data/sqlite.db"
+mkdir -p "$(dirname "$DB_PATH")"
+if [[ -z "${LITESTREAM_REPLICA_URL}" ]]; then
+  exec dstack server --host 0.0.0.0
+else
+  if [[ ! -f "$DB_PATH" ]]; then
+    echo "Attempting Litestream restore..."
+    if ! output=$(litestream restore -o "$DB_PATH" "$LITESTREAM_REPLICA_URL" 2>&1); then
+      if echo "$output" | grep -qiE "cannot calc restore plan"; then
+        echo "No replica snapshots found; starting with empty database."
+      else
+        echo "$output" >&2
+        exit 1
+      fi
+    fi
+  fi
+  exec litestream replicate -exec "dstack server --host 0.0.0.0" "$DB_PATH" "$LITESTREAM_REPLICA_URL"
+fi

{dstack-0.19.31 → dstack-0.19.32}/docker/server/release/Dockerfile

@@ -13,9 +13,9 @@ RUN apt-get update && apt-get install -y \
     sqlite3 \
     && rm -rf /var/lib/apt/lists/*
 
-RUN if [ $(uname -m) = "aarch64" ]; then ARCH="arm64"; else ARCH="amd64"; fi && \
-    curl https://github.com/benbjohnson/litestream/releases/download/v0.3.9/litestream-v0.3.9-linux-$ARCH.deb -O -L && \
-    dpkg -i litestream-v0.3.9-linux-$ARCH.deb
+RUN if [ $(uname -m) = "aarch64" ]; then ARCH="arm64"; else ARCH="x86_64"; fi && \
+    curl https://github.com/benbjohnson/litestream/releases/download/v0.5.0/litestream-0.5.0-linux-$ARCH.deb -O -L && \
+    dpkg -i litestream-0.5.0-linux-$ARCH.deb
 
 ADD https://astral.sh/uv/install.sh /uv-installer.sh
 RUN sh /uv-installer.sh && rm /uv-installer.sh

{dstack-0.19.31 → dstack-0.19.32}/docker/server/stgn/Dockerfile

@@ -11,9 +11,9 @@ RUN apt-get update && apt-get install -y \
     sqlite3 \
     && rm -rf /var/lib/apt/lists/*
 
-RUN if [ $(uname -m) = "aarch64" ]; then ARCH="arm64"; else ARCH="amd64"; fi && \
-    curl https://github.com/benbjohnson/litestream/releases/download/v0.3.9/litestream-v0.3.9-linux-$ARCH.deb -O -L && \
-    dpkg -i litestream-v0.3.9-linux-$ARCH.deb
+RUN if [ $(uname -m) = "aarch64" ]; then ARCH="arm64"; else ARCH="x86_64"; fi && \
+    curl https://github.com/benbjohnson/litestream/releases/download/v0.5.0/litestream-0.5.0-linux-$ARCH.deb -O -L && \
+    dpkg -i litestream-0.5.0-linux-$ARCH.deb
 
 ADD https://astral.sh/uv/install.sh /uv-installer.sh
 RUN sh /uv-installer.sh && rm /uv-installer.sh

dstack-0.19.32/docs/blog/posts/kubernetes-beta.md

@@ -0,0 +1,315 @@
+---
+title: Orchestrating GPUs on Kubernetes clusters
+date: 2025-10-08
+description: "TBA"  
+slug: kubernetes-beta
+image: https://dstack.ai/static-assets/static-assets/images/dstack-kubernetes.png
+categories:
+  - Changelog
+---
+
+# Orchestrating GPUs on Kubernetes clusters
+
+`dstack` gives teams a unified way to run and manage GPU-native containers across clouds and on-prem environments — without requiring Kubernetes.
+At the same time, many organizations rely on Kubernetes as the foundation of their infrastructure.
+
+To support these users, `dstack` is releasing the beta of its native Kubernetes integration.
+
+<img src="https://dstack.ai/static-assets/static-assets/images/dstack-kubernetes.png" width="630"/>
+
+<!-- more -->
+
+This update allows `dstack` to orchestrate dev environments, distributed training, and inference workloads directly on Kubernetes clusters — combining the best of both worlds: an ML-tailored interface for ML teams together with the full Kubernetes ecosystem.
+
+Read below to learn on how to use `dstack` with Kubernetes clusters.
+
+## Creating a Kubernetes cluster
+
+A major advantage of Kubernetes is its portability. Whether you’re using managed Kubernetes on a GPU cloud or an on-prem cluster, you can connect it to `dstack` and use it to orchestrate your GPU workloads.
+
+!!! info "NVIDIA GPU Operator"
+    For `dstack` to correctly detect GPUs in your Kubernetes cluster, the cluster must have the
+    [NVIDIA GPU Operator :material-arrow-top-right-thin:{ .external }](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/index.html){:target="_blank"} pre-installed.
+
+### Nebius example
+
+If you're using [Nebius :material-arrow-top-right-thin:{ .external }](https://nebius.com/){:target="_blank"}, the process of creating a Kubernetes cluster is straightforward.
+
+Select the region of interest and click `Create cluster`.  
+Once the cluster is created, switch to `Applications` and install the `nvidia-device-plugin` application — this can be done in one click.
+
+<img src="https://dstack.ai/static-assets/static-assets/images/dstack-nebius-cluster-ui.png" width="750"/>
+
+Next, go to `Node groups` and click `Create node group`. Choose the GPU type and count, disk size, and other options.  
+If `dstack` doesn't run in the same network, enable public IPs so that `dstack` can access the nodes.
+
+<img src="https://dstack.ai/static-assets/static-assets/images/dstack-nebius-node-group.png" width="750"/>
+
+## Setting up the backend
+
+Once the cluster is ready, you need to configure the `kubernetes` backend in the `dstack` server.  
+To do this, add the corresponding configuration to your `~/.dstack/server/config.yml` file:
+
+<div editor-title="~/.dstack/server/config.yml">
+
+```yaml
+projects:
+- name: main
+    backends:
+    - type: kubernetes
+        kubeconfig:
+            filename: ~/.kube/config
+        proxy_jump:
+            hostname: 204.12.171.137
+            port: 32000
+```
+
+</div>
+
+The configuration includes two main parts: the path to the kubeconfig file and the proxy-jump configuration.
+
+If your cluster is on Nebius, click `How to connect` in the console — it will guide you through setting up the kubeconfig file.
+
+!!! info "Proxy jump"
+    To allow `dstack` to forward SSH traffic, it needs one node to act as a proxy jump.
+    Choose any node in the cluster and specify its IP address and an accessible port in the backend configuration.
+
+    Now that the backend is configured, go ahead and restart the `dstack server`.
+
+That’s it — you can now use all of `dstack`’s features, including [dev environments](../../docs/concepts/dev-environments.md), [tasks](../../docs/concepts/tasks.md), [services](../../docs/concepts/services.md), and [fleets](../../docs/concepts/fleets.md).
+
+## Running a dev environment
+
+A dev environment lets you provision an instance and connect to it from your desktop IDE.
+
+<div editor-title="examples/.dstack.yml"> 
+
+```yaml
+type: dev-environment
+# The name is optional, if not specified, generated randomly
+name: vscode
+
+python: "3.11"
+
+# Uncomment to use a custom Docker image
+#image: huggingface/trl-latest-gpu
+
+ide: vscode
+
+resources:
+  gpu: H200
+```
+
+</div>
+
+To run a dev environment, pass the configuration to [`dstack apply`](../../docs/reference/cli/dstack/apply.md):
+
+<div class="termy">
+
+```shell
+$ dstack apply -f examples/.dstack.yml
+
+ #  BACKEND         RESOURCES                                   INSTANCE TYPE                       PRICE
+ 1  kubernetes (-)  cpu=127 mem=1574GB disk=871GB H200:141GB:8  computeinstance-u00hwk32d0xemhxhvj  $0
+ 2  kubernetes (-)  cpu=127 mem=1574GB disk=871GB H200:141GB:8  computeinstance-u00n24fb4q85yavc9z  $0
+
+Submit the run vscode? [y/n]: y
+
+Launching `vscode`...
+---> 100%
+
+To open in VS Code Desktop, use this link:
+  vscode://vscode-remote/ssh-remote+vscode/workflow
+```
+
+</div>
+
+Dev environments support many [diffrent options](../../docs/concepts/dev-environments.md), including a custom Docker image, mounted repositories, idle timeout, min GPU utilization, and more.
+
+## Running distributed training
+
+Distributed training can be performed in `dstack` using [distributed tasks](../../docs/concepts/tasks.md#distributed-tasks). 
+The configuration is similar to a dev environment, except it runs across multiple nodes.
+
+### Creating a cluster fleet
+
+Before running a distributed task, create a fleet with `placement` set to `cluster`:
+
+<div editor-title="examples/misc/fleets/.dstack.yml">
+    
+    ```yaml
+    type: fleet
+    # The name is optional; if not specified, one is generated automatically
+    name: my-k8s-fleet
+    
+    # For `kubernetes`, `min` should be set to `0` since it can't pre-provision VMs.
+    # Optionally, you can set the maximum number of nodes to limit scaling.
+    nodes: 0..
+
+    placement: cluster
+    
+    backends: [kubernetes]
+    
+    resources:
+      # Specify requirements to filter nodes
+      gpu: 1..8
+    ```
+    
+</div>
+
+Then, create the fleet using the `dstack apply` command:
+
+<div class="termy">
+
+```shell
+$ dstack apply -f examples/misc/fleets/.dstack.yml
+
+Provisioning...
+---> 100%
+
+ FLEET     INSTANCE  BACKEND              GPU             PRICE    STATUS  CREATED 
+```
+
+</div>
+
+Once the fleet is created, you can run distributed tasks on it.
+
+### NCCL tests example
+
+Below is an example of using distributed tasks to run NCCL tests.
+It also demonstrates how to use mpirun with `dstack`:
+
+<div editor-title="examples/clusters/nccl-tests/.dstack.yml">
+
+```yaml
+type: task
+name: nccl-tests
+
+nodes: 2
+
+# The `startup_order` and `stop_criteria` properties are required for `mpirun`
+startup_order: workers-first
+stop_criteria: master-done
+
+env:
+  - NCCL_DEBUG=INFO
+commands:
+  - |
+    if [ $DSTACK_NODE_RANK -eq 0 ]; then
+      mpirun \
+        --allow-run-as-root \
+        --hostfile $DSTACK_MPI_HOSTFILE \
+        -n $DSTACK_GPUS_NUM \
+        -N $DSTACK_GPUS_PER_NODE \
+        --bind-to none \
+        /opt/nccl-tests/build/all_reduce_perf -b 8 -e 8G -f 2 -g 1
+    else
+      sleep infinity
+    fi
+
+# The `kubernetes` backend requires it
+privileged: true
+
+resources:
+  gpu: nvidia:1..8
+  shm_size: 16GB
+```
+
+</div>
+
+To run the configuration, use the [`dstack apply`](../../docs/reference/cli/dstack/apply.md) command.
+
+<div class="termy">
+
+```shell
+$ dstack apply -f examples/clusters/nccl-tests/.dstack.yml --fleet my-k8s-fleet
+
+#  BACKEND         RESOURCES                                   INSTANCE TYPE                       PRICE
+1  kubernetes (-)  cpu=127 mem=1574GB disk=871GB H200:141GB:8  computeinstance-u00hwk32d0xemhxhvj  $0
+2  kubernetes (-)  cpu=127 mem=1574GB disk=871GB H200:141GB:8  computeinstance-u00n24fb4q85yavc9z  $0
+
+Submit the run nccl-tests? [y/n]: y
+```
+
+</div>
+
+### Distributed training example
+
+Below is a minimal example of a distributed training configuration:
+
+<div editor-title="examples/distributed-training/torchrun/.dstack.yml">
+
+```yaml
+type: task
+name: train-distrib
+
+nodes: 2
+
+python: 3.12
+env:
+  - NCCL_DEBUG=INFO
+commands:
+  - git clone https://github.com/pytorch/examples.git pytorch-examples
+  - cd pytorch-examples/distributed/ddp-tutorial-series
+  - uv pip install -r requirements.txt
+  - |
+    torchrun \
+      --nproc-per-node=$DSTACK_GPUS_PER_NODE \
+      --node-rank=$DSTACK_NODE_RANK \
+      --nnodes=$DSTACK_NODES_NUM \
+      --master-addr=$DSTACK_MASTER_NODE_IP \
+      --master-port=12345 \
+      multinode.py 50 10
+
+resources:
+  gpu: 1..8
+  shm_size: 16GB
+```
+
+</div>
+
+To run the configuration, use the [`dstack apply`](../../docs/reference/cli/dstack/apply.md) command.
+
+<div class="termy">
+
+```shell
+$ dstack apply -f examples/distributed-training/torchrun/.dstack.yml --fleet my-k8s-fleet
+
+#  BACKEND         RESOURCES                                   INSTANCE TYPE                       PRICE
+1  kubernetes (-)  cpu=127 mem=1574GB disk=871GB H200:141GB:8  computeinstance-u00hwk32d0xemhxhvj  $0
+2  kubernetes (-)  cpu=127 mem=1574GB disk=871GB H200:141GB:8  computeinstance-u00n24fb4q85yavc9z  $0
+
+Submit the run nccl-tests? [y/n]: y
+```
+
+</div>
+
+For more examples, explore the [distirbuted training](../../examples.md#distributed-training) section in the docs.
+
+## FAQ
+
+### VM-based backends vs Kubernetes backend
+    
+While the `kubernetes` backend is preferred if your team depends on the Kubernetes ecosystem,
+the [VM-based](../../docs/concepts/backends.md#vm-based) backends leverage native integration with top GPU clouds (including Nebius and others) and may be a better choice if Kubernetes isn’t required.
+
+VM-based backends also offer more granular control over cluster provisioning.  
+
+> Note that `dstack` doesn’t yet support Kubernetes clusters with auto-scaling enabled (coming soon), which can be another reason to use VM-based backends.
+
+### SSH fleets vs Kubernetes backend
+
+If you’re using on-prem servers and Kubernetes isn’t a requirement, [SSH fleets](../../docs/concepts/fleets.md#ssh) may be simpler.
+They provide a lightweight and flexible alternative.
+
+### AMD GPUs
+
+Support for AMD GPUs is coming soon — our team is actively working on it right now.
+
+!!! info "What's next"
+    1. Check [Quickstart](../../docs/quickstart.md)
+    2. Explore [dev environments](../../docs/concepts/dev-environments.md), 
+        [tasks](../../docs/concepts/tasks.md), [services](../../docs/concepts/services.md), 
+        and [fleets](../../docs/concepts/fleets.md)
+    3. Read the the [clusters](../../docs/guides/clusters.md) guide
+    4. Join [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd){:target="_blank"}

{dstack-0.19.31 → dstack-0.19.32}/docs/docs/concepts/backends.md

@@ -452,7 +452,7 @@ There are two ways to configure GCP: using a service account or using the defaul
     - name: main
       backends:
         - type: gcp
-          project_id: gcp-project-id
+          project_id: my-gcp-project
           creds:
             type: service_account
             filename: ~/.dstack/server/gcp-024ed630eab5.json
@@ -460,6 +460,35 @@ There are two ways to configure GCP: using a service account or using the defaul
 
     </div>
 
+    ??? info "User interface"
+        If you are configuring the `gcp` backend on the [project settigns page](projects.md#backends), 
+        specify the contents of the JSON file in `data`:
+
+        <div editor-title="~/.dstack/server/config.yml">
+
+        ```yaml
+        type: gcp
+        project_id: my-gcp-project
+        creds:
+          type: service_account
+          data: |
+            {
+              "type": "service_account",
+              "project_id": "my-gcp-project",
+              "private_key_id": "abcd1234efgh5678ijkl9012mnop3456qrst7890",
+              "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEv...rest_of_key...IDAQAB\n-----END PRIVATE KEY-----\n",
+              "client_email": "my-service-account@my-gcp-project.iam.gserviceaccount.com",
+              "client_id": "123456789012345678901",
+              "auth_uri": "https://accounts.google.com/o/oauth2/auth",
+              "token_uri": "https://oauth2.googleapis.com/token",
+              "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
+              "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/my-service-account%40my-gcp-project.iam.gserviceaccount.com",
+              "universe_domain": "googleapis.com"
+            }
+        ```
+
+        </div>
+
 If you don't know your GCP project ID, use [Google Cloud CLI :material-arrow-top-right-thin:{ .external }](https://cloud.google.com/sdk/docs/install-sdk):
 
 ```shell
@@ -638,8 +667,33 @@ projects:
 
 </div>
 
-??? info "Configuring in the UI"
-    If you are configuring the backend in the `dstack` UI, specify the contents of the private key file in `private_key_content`.
+??? info "Credentials file"
+    It's also possible to configure the `nebius` backend using a credentials file [generated :material-arrow-top-right-thin:{ .external }](https://docs.nebius.com/iam/service-accounts/authorized-keys#create){:target="_blank"} by the `nebius` CLI:
+
+    <div class="termy">
+
+    ```shell
+    $ nebius iam auth-public-key generate \
+        --service-account-id <service account ID> \
+        --output ~/.nebius/sa-credentials.json
+    ```
+    
+    </div>
+
+  
+    ```yaml
+    projects:
+    - name: main
+      backends:
+      - type: nebius
+        creds:
+          type: service_account
+          filename: ~/.nebius/sa-credentials.json
+    ```
+
+??? info "User interface"
+    If you are configuring the `nebius` backend on the [project settigns page](projects.md#backends), 
+    specify the contents of the private key file in `private_key_content`:
 
     <div editor-title="~/.dstack/server/config.yml">
 

{dstack-0.19.31 → dstack-0.19.32}/docs/docs/concepts/fleets.md

@@ -118,11 +118,12 @@ This ensures all instances are provisioned with optimal inter-node connectivity.
     Refer to the [EFA](../../examples/clusters/efa/index.md) example for more details.
 
 ??? info "GCP"
-    When you create a fleet with GCP, for the A3 Mega and A3 High instance types, [GPUDirect-TCPXO and GPUDirect-TCPX :material-arrow-top-right-thin:{ .external }](https://cloud.google.com/kubernetes-engine/docs/how-to/gpu-bandwidth-gpudirect-tcpx-autopilot){:target="_blank"} networking is automatically configured.
+    When you create a fleet with GCP, `dstack` automatically configures [GPUDirect-TCPXO and GPUDirect-TCPX :material-arrow-top-right-thin:{ .external }](https://cloud.google.com/kubernetes-engine/docs/how-to/gpu-bandwidth-gpudirect-tcpx-autopilot){:target="_blank"} networking for the A3 Mega and A3 High instance types, as well as RoCE networking for the A4 instance type.
 
     !!! info "Backend configuration"    
-        Note, GPUDirect-TCPXO and GPUDirect-TCPX require `extra_vpcs` to be configured  in the `gcp` backend configuration.
-        Refer to the [A3 Mega](../../examples/clusters/a3mega/index.md) and 
+        You may need to configure `extra_vpcs` and `roce_vpcs` in the `gcp` backend configuration.
+        Refer to the [A4](../../examples/clusters/a4/index.md),
+        [A3 Mega](../../examples/clusters/a3mega/index.md), and 
         [A3 High](../../examples/clusters/a3high/index.md) examples for more details.
 
 ??? info "Nebius"

{dstack-0.19.31 → dstack-0.19.32}/docs/docs/guides/clusters.md

@@ -25,18 +25,19 @@ For cloud fleets, fast interconnect is currently supported only on the `aws`, `g
         Refer to the [EFA](../../examples/clusters/efa/index.md) example for more details.
 
 === "GCP"
-    When you create a cloud fleet with GCP, for the A3 Mega and A3 High instance types, [GPUDirect-TCPXO and GPUDirect-TCPX :material-arrow-top-right-thin:{ .external }](https://cloud.google.com/kubernetes-engine/docs/how-to/gpu-bandwidth-gpudirect-tcpx-autopilot){:target="_blank"} networking is automatically configured.
+    When you create a cloud fleet with GCP, `dstack` automatically configures [GPUDirect-TCPXO and GPUDirect-TCPX :material-arrow-top-right-thin:{ .external }](https://cloud.google.com/kubernetes-engine/docs/how-to/gpu-bandwidth-gpudirect-tcpx-autopilot){:target="_blank"} networking for the A3 Mega and A3 High instance types, as well as RoCE networking for the A4 instance type.
 
     !!! info "Backend configuration"    
-        Note, GPUDirect-TCPXO and GPUDirect-TCPX require `extra_vpcs` to be configured  in the `gcp` backend configuration.
-        Refer to the [A3 Mega](../../examples/clusters/a3mega/index.md) and 
+        You may need to configure `extra_vpcs` and `roce_vpcs` in the `gcp` backend configuration.
+        Refer to the [A4](../../examples/clusters/a4/index.md),
+        [A3 Mega](../../examples/clusters/a3mega/index.md), and 
         [A3 High](../../examples/clusters/a3high/index.md) examples for more details.
 
 === "Nebius"
     When you create a cloud fleet with Nebius, [InfiniBand :material-arrow-top-right-thin:{ .external }](https://docs.nebius.com/compute/clusters/gpu){:target="_blank"} networking is automatically configured if it’s supported for the corresponding instance type.
 
-> To request fast interconnect support for a other backends,
-file an [issue :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/issues){:target="_ blank"}. 
+> To request fast interconnect support for other backends,
+file an [issue :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/issues){:target="_ blank"}.
 
 ## Distributed tasks
 

{dstack-0.19.31 → dstack-0.19.32}/docs/docs/guides/kubernetes.md

@@ -1,7 +1,10 @@
 # Kubernetes
 
-While `dstack` can orchestrate container-based runs natively on both cloud and on-prem without Kubernetes (see [backends](../concepts/backends.md)), it also supports running container-based workloads directly on Kubernetes clusters.  
-For that, you need to configure the `kubernetes` backend.
+The [kubernetes](../concepts/backends.md#kubernetes) backend enables `dstack` to run [dev environments](/docs/concepts/dev-environments), [tasks](/docs/concepts/tasks), and [services](/docs/concepts/services) directly on existing Kubernetes clusters.
+
+If your GPUs are already deployed on Kubernetes and your team relies on its ecosystem and tooling, use this backend to integrate `dstack` with your clusters.
+
+> If Kubernetes is not required, you can run `dstack` on clouds or on-prem clusters without Kubernetes by using [VM-based](../concepts/backends.md#vm-based), [container-based](../concepts/backends.md#container-based), or [on-prem](../concepts/backends.md#on-prem) backends.
 
 ## Setting up the backend
 
@@ -61,7 +64,7 @@ If you’d like to run [distributed tasks](../concepts/tasks.md#distributed-task
 
     placement: cluster
     
-    backends: [kuberenetes]
+    backends: [kubernetes]
     
     resources:
       # Specify requirements to filter nodes