ray-vast 0.1.0__tar.gz

ray_vast-0.1.0/.gitignore

@@ -0,0 +1,8 @@
+.DS_Store
+__pycache__/
+*.pyc
+.venv/
+dist/
+build/
+*.egg-info/
+site/

ray_vast-0.1.0/CHANGELOG.md

@@ -0,0 +1,50 @@
+# Changelog
+
+## 0.1.0 (2026-10-15)
+
+Changes since 0.1.0rc1:
+
+- `dashboard`: Ctrl-C closes the tunnel, prints `tunnel closed` and exits with code 0, with no traceback.
+- The commands write each output line at once, also when stdout is a pipe or a file.
+- The watchdog retries a heartbeat when a thread cannot start (`can't start new thread`). Only the heartbeat time limit ends the worker.
+- The head log keeps the last line (the error) of a long destroy reason.
+- A missing YAML file, bad YAML syntax or a bad YAML field value prints one line `ray-vast <command>: <message>` and exits with code 1, with no traceback.
+
+## 0.1.0rc1 (2026-10-14, TestPyPI)
+
+First version. A release candidate on TestPyPI.
+
+- Ray autoscaler v2 node provider (`VastNodeProvider`) for vast.ai workers. The head runs on AWS EC2 (`head_instance_type`, default `t3.large`).
+- Commands: `ray-vast up`, `down`, `attach` and `dashboard`. Options: `--profile`, `--key-file`, `--key-name`, `--wheel`, `--index-url`.
+- `up` makes an IAM role, 5 security groups, an instance profile and the head. It builds a derived image with ray-vast. By default, `pip` installs the CLI's own version (`ray-vast==<version>`) from PyPI. `--index-url URL` installs it from another index, for example TestPyPI. `--wheel FILE` installs a wheel file, and a source tree builds its own wheel (both for development). It stops if the cluster already exists. It measures the image size and sets `image_size_gb`.
+- `down` ends the head first, then destroys the vast instances, then deletes the AWS resources. It finds everything by tag and label, so it works after a failed `up` and when the head is gone. It reports what is left (exit code 1 if anything). Without a vast key it cleans AWS and warns that vast instances were not swept.
+- `dashboard` copies the token to `~/.ray-vast/<cluster>/token`, prints the `RAY_AUTH_MODE` and `RAY_AUTH_TOKEN_PATH` export line, and opens the SSH tunnel, so `ray job submit` works from the laptop.
+- State files in `~/.ray-vast/`: `<cluster>.json`, `<cluster>.known_hosts`, `<cluster>/token`.
+- Cluster names match `[A-Za-z0-9][A-Za-z0-9_-]{0,39}`.
+- Cluster YAML with node types. Each type has declared resources (`CPU` minimum, `GPU` exact), `node_config.disk_gb` (default 50) and an `offer_query` in the vast API JSON form (MB units). ray-vast adds the filters `num_gpus`, `cpu_cores_effective`, `direct_port_count`, `verified`, `rentable` and `datacenter`. A stricter user filter on `cpu_cores_effective` or `direct_port_count` is allowed.
+- Provider fields with defaults: `candidate_factor` 2, `cert_max_age_days` 365, `allow_non_datacenter` false, `order_by` `dph_total`, `head_instance_type` `t3.large`, `port_a` 7443, `port_b` 7444. Optional: `image_size_gb`, `key_name`, `key_file`.
+- Example YAMLs: `gpu-24gb`, `gpu-80gb`, `cpu-heavy`.
+- Images with an `ENTRYPOINT` that ends with `exec "$@"` work, for example `rayproject/ray:*-gpu`. `up` keeps the ENTRYPOINT and reads the image check from one marked line of its output.
+- Candidates: the provider rents `candidate_factor` times as many instances as slots. The first candidates that enroll fill the slots. The provider destroys the others. The pull time limit comes from `image_size_gb` and `inet_down` (5 to 15 minutes).
+- `max_dollars_per_hour` budget limit. It is required. The vast key is not a config field: it comes from `VAST_API_KEY` or `~/.config/vastai/vast_api_key`.
+- Worker-first enrollment: the container command holds no secrets. The worker checks the host, proves its identity with a one-time join token, and then gets its certificate, key and the cluster token.
+- Host checks: egress IP, no NET_RAW or NET_ADMIN (read from `CapBnd`), own network namespace, empty `/dev/shm`, and the 64 identity ports (checked from the head).
+- TLS and token authentication on all Ray nodes. A new CA and token for each cluster. Node certificates are bound to the instance (IP SAN and URI SAN `rayvast:instance/<id>`); the head checks both on the mTLS port. The worker pins the CA by its SHA-256, rebuilds the CA PEM from the pinned DER, and refuses a PEM with more than one certificate.
+- Two head server ports (`port_a`, `port_b`): TLS for download and enrollment, and mTLS for heartbeat and reports.
+- Refusals before a valid join token (unknown instance, second enrollment, wrong source IP, wrong token) give 403 and do not destroy the instance. The head makes no vast call during enrollment.
+- `LD_PRELOAD` networking shim: workers run behind NAT with no hairpin NAT and no overlay network.
+- Watchdog: the bootstrap is PID 1. It sends a heartbeat every 30 seconds. It destroys its own instance after 5 minutes without a heartbeat, after 10 minutes without enrollment, on TERM, INT or HUP, and when Ray ends. It sends the reason to the head first.
+- The bootloader (about 1.5 KB) never exits. It retries, and it destroys its own instance after a hash mismatch or after 10 minutes of failures.
+- Wall-clock limit on every HTTP call of the worker. Vast calls do not follow redirects.
+- Blocklist of bad hosts, in memory for each cluster.
+- Security group rules for each worker IP, with reference counts for shared IPs, in up to 5 groups (about 299 worker IPs).
+- `down` finds only the 5 group names `ray-vast-<cluster>-0` … `-4`. A cluster that an older development build made has other group names. Clean up such a cluster by hand.
+- Start-up sweep: a new provider destroys every instance with the cluster label and removes the cluster's security group rules. The orphan sweep destroys instances that the provider does not know.
+- Rent-loop breaker: after 3 failed slots in a row of one node type, the provider stops that type for 10 minutes.
+- The head container runs `ray start --head --block` with no restart policy. If Ray's monitor dies, the head stops, and the workers destroy themselves within 5 minutes.
+- Warnings: untested Ray version, `CPU` above the 58 worker ports, `allow_non_datacenter`, and certificate expiry in 14 days.
+- Documented risks: the head holds the full vast account key; Ray tasks can read `CONTAINER_API_KEY` from `/proc/1/environ`. See `docs/security.md`.
+- Tests: a fast default run (`pytest`, about 30 s) and a full run (`pytest -o addopts="" -m "not live and not local_e2e"`).
+- Documents: the user docs site (`docs/`, MkDocs Material, published to https://duc-ph.github.io/ray-vast/), README, security model (`docs/security.md`), `CONTRIBUTING.md`.
+- Tested with Ray 2.59.
+- Published to TestPyPI only.

ray_vast-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Duc Pham
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ray_vast-0.1.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.5
+Name: ray-vast
+Version: 0.1.0
+Summary: A Ray autoscaler node provider for vast.ai workers with an AWS EC2 head.
+Project-URL: Documentation, https://duc-ph.github.io/ray-vast/
+Project-URL: Source, https://github.com/duc-ph/ray-vast
+Project-URL: Changelog, https://duc-ph.github.io/ray-vast/reference/changelog/
+Project-URL: Issues, https://github.com/duc-ph/ray-vast/issues
+Author-email: Duc Pham <thiel.pham@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: boto3
+Requires-Dist: cryptography>=42
+Requires-Dist: pyyaml
+Requires-Dist: ray[default]>=2.59
+Requires-Dist: requests
+Provides-Extra: test
+Requires-Dist: build; extra == 'test'
+Requires-Dist: freezegun; extra == 'test'
+Requires-Dist: hatchling; extra == 'test'
+Requires-Dist: moto[server]; extra == 'test'
+Requires-Dist: pytest; extra == 'test'
+Requires-Dist: pytest-timeout; extra == 'test'
+Requires-Dist: responses; extra == 'test'
+Description-Content-Type: text/markdown
+
+# ray-vast
+
+(work in progress)
+
+Run a Ray cluster with the head on AWS EC2 and GPU workers on vast.ai.
+
+- The Ray head runs on one EC2 instance in your AWS account.
+- The Ray autoscaler rents GPU workers from vast.ai when your jobs need them, and removes them when they are idle.
+- The workers talk to the head over the public internet, across NAT, with TLS and a token.
+- You do not need a VPN or an overlay network.
+
+**Documentation: https://duc-ph.github.io/ray-vast/**
+
+## Why
+
+- vast.ai has cheap GPUs. But a vast instance can stop at any time, and its IP can change.
+- ray-vast keeps the head on EC2, so the cluster stays up. Only workers come and go.
+- You keep the normal Ray workflow. Only the cluster commands change.
+- A worker that loses the head destroys its own instance. You do not pay for lost workers.
+
+## Install
+
+```
+pip install ray-vast "ray[default]==2.59.*"
+```
+
+The Ray pin keeps Ray on your laptop at the same version as Ray in the cluster image. ray-vast is tested with Ray 2.59.
+
+You need Python 3.10 or later, an AWS profile, an EC2 key pair, and a vast.ai API key.
+Details: [Getting started](https://duc-ph.github.io/ray-vast/getting-started/).
+
+## Quick start
+
+Copy an example cluster YAML (`gpu-24gb.yaml`, `gpu-80gb.yaml` or `cpu-heavy.yaml` in `ray_vast/examples/`).
+Set the region, the budget (`max_dollars_per_hour`) and your EC2 key pair (`key_name`, `key_file`).
+
+```
+ray-vast up cluster.yaml --profile my-aws-profile
+ray-vast dashboard cluster.yaml --profile my-aws-profile   # keep this terminal open
+```
+
+In a second terminal, run the `export` line that `dashboard` printed, and submit a job:
+
+```
+export RAY_AUTH_MODE=token RAY_AUTH_TOKEN_PATH=...     # the line that dashboard printed
+ray job submit --address http://127.0.0.1:8265 --working-dir . -- python train.py
+```
+
+When you finish:
+
+```
+ray-vast down cluster.yaml --profile my-aws-profile
+```
+
+| Plain Ray | ray-vast |
+|---|---|
+| `ray up` | `ray-vast up` |
+| `ray dashboard` | `ray-vast dashboard` |
+| `ray down` | `ray-vast down` |
+
+Some `ray` cluster commands do not work. Read [limits](https://duc-ph.github.io/ray-vast/reference/limits/#some-ray-commands-do-not-work). Use `--working-dir` and `runtime_env` to send code to the cluster.
+
+## Documentation
+
+- [Getting started](https://duc-ph.github.io/ray-vast/getting-started/): your first cluster, step by step.
+- How-to guides: [run your Ray workload](https://duc-ph.github.io/ray-vast/how-to/run-your-workload/), [choose node types](https://duc-ph.github.io/ray-vast/how-to/node-types/), [use GPU images](https://duc-ph.github.io/ray-vast/how-to/gpu-images/), [control cost](https://duc-ph.github.io/ray-vast/how-to/control-cost/), [troubleshoot](https://duc-ph.github.io/ray-vast/how-to/troubleshoot/), [clean up](https://duc-ph.github.io/ray-vast/how-to/clean-up/).
+- Reference: [CLI](https://duc-ph.github.io/ray-vast/reference/cli/), [configuration](https://duc-ph.github.io/ray-vast/reference/configuration/), [Ray library support](https://duc-ph.github.io/ray-vast/reference/ray-libraries/), [limits](https://duc-ph.github.io/ray-vast/reference/limits/), [IAM permissions](https://duc-ph.github.io/ray-vast/reference/iam/), [changelog](https://duc-ph.github.io/ray-vast/reference/changelog/).
+- Explanation: [architecture](https://duc-ph.github.io/ray-vast/explanation/architecture/), [enrollment](https://duc-ph.github.io/ray-vast/explanation/enrollment/), [worker life cycle](https://duc-ph.github.io/ray-vast/explanation/worker-lifecycle/), [design decisions](https://duc-ph.github.io/ray-vast/explanation/design-decisions/).
+- [Security model](https://duc-ph.github.io/ray-vast/security/). Read it before you use the cluster with real data. A vast host has root access to the worker container.
+
+## Status
+
+Version 0.1.0. Tested with Ray 2.59.
+
+## Contributing
+
+Read [CONTRIBUTING.md](https://github.com/duc-ph/ray-vast/blob/main/ray-vast-pypi/CONTRIBUTING.md) for the tests and the docs build.
+
+## License
+
+MIT. See [LICENSE](https://github.com/duc-ph/ray-vast/blob/main/ray-vast-pypi/LICENSE).

ray_vast-0.1.0/README.md

@@ -0,0 +1,81 @@
+# ray-vast
+
+(work in progress)
+
+Run a Ray cluster with the head on AWS EC2 and GPU workers on vast.ai.
+
+- The Ray head runs on one EC2 instance in your AWS account.
+- The Ray autoscaler rents GPU workers from vast.ai when your jobs need them, and removes them when they are idle.
+- The workers talk to the head over the public internet, across NAT, with TLS and a token.
+- You do not need a VPN or an overlay network.
+
+**Documentation: https://duc-ph.github.io/ray-vast/**
+
+## Why
+
+- vast.ai has cheap GPUs. But a vast instance can stop at any time, and its IP can change.
+- ray-vast keeps the head on EC2, so the cluster stays up. Only workers come and go.
+- You keep the normal Ray workflow. Only the cluster commands change.
+- A worker that loses the head destroys its own instance. You do not pay for lost workers.
+
+## Install
+
+```
+pip install ray-vast "ray[default]==2.59.*"
+```
+
+The Ray pin keeps Ray on your laptop at the same version as Ray in the cluster image. ray-vast is tested with Ray 2.59.
+
+You need Python 3.10 or later, an AWS profile, an EC2 key pair, and a vast.ai API key.
+Details: [Getting started](https://duc-ph.github.io/ray-vast/getting-started/).
+
+## Quick start
+
+Copy an example cluster YAML (`gpu-24gb.yaml`, `gpu-80gb.yaml` or `cpu-heavy.yaml` in `ray_vast/examples/`).
+Set the region, the budget (`max_dollars_per_hour`) and your EC2 key pair (`key_name`, `key_file`).
+
+```
+ray-vast up cluster.yaml --profile my-aws-profile
+ray-vast dashboard cluster.yaml --profile my-aws-profile   # keep this terminal open
+```
+
+In a second terminal, run the `export` line that `dashboard` printed, and submit a job:
+
+```
+export RAY_AUTH_MODE=token RAY_AUTH_TOKEN_PATH=...     # the line that dashboard printed
+ray job submit --address http://127.0.0.1:8265 --working-dir . -- python train.py
+```
+
+When you finish:
+
+```
+ray-vast down cluster.yaml --profile my-aws-profile
+```
+
+| Plain Ray | ray-vast |
+|---|---|
+| `ray up` | `ray-vast up` |
+| `ray dashboard` | `ray-vast dashboard` |
+| `ray down` | `ray-vast down` |
+
+Some `ray` cluster commands do not work. Read [limits](https://duc-ph.github.io/ray-vast/reference/limits/#some-ray-commands-do-not-work). Use `--working-dir` and `runtime_env` to send code to the cluster.
+
+## Documentation
+
+- [Getting started](https://duc-ph.github.io/ray-vast/getting-started/): your first cluster, step by step.
+- How-to guides: [run your Ray workload](https://duc-ph.github.io/ray-vast/how-to/run-your-workload/), [choose node types](https://duc-ph.github.io/ray-vast/how-to/node-types/), [use GPU images](https://duc-ph.github.io/ray-vast/how-to/gpu-images/), [control cost](https://duc-ph.github.io/ray-vast/how-to/control-cost/), [troubleshoot](https://duc-ph.github.io/ray-vast/how-to/troubleshoot/), [clean up](https://duc-ph.github.io/ray-vast/how-to/clean-up/).
+- Reference: [CLI](https://duc-ph.github.io/ray-vast/reference/cli/), [configuration](https://duc-ph.github.io/ray-vast/reference/configuration/), [Ray library support](https://duc-ph.github.io/ray-vast/reference/ray-libraries/), [limits](https://duc-ph.github.io/ray-vast/reference/limits/), [IAM permissions](https://duc-ph.github.io/ray-vast/reference/iam/), [changelog](https://duc-ph.github.io/ray-vast/reference/changelog/).
+- Explanation: [architecture](https://duc-ph.github.io/ray-vast/explanation/architecture/), [enrollment](https://duc-ph.github.io/ray-vast/explanation/enrollment/), [worker life cycle](https://duc-ph.github.io/ray-vast/explanation/worker-lifecycle/), [design decisions](https://duc-ph.github.io/ray-vast/explanation/design-decisions/).
+- [Security model](https://duc-ph.github.io/ray-vast/security/). Read it before you use the cluster with real data. A vast host has root access to the worker container.
+
+## Status
+
+Version 0.1.0. Tested with Ray 2.59.
+
+## Contributing
+
+Read [CONTRIBUTING.md](https://github.com/duc-ph/ray-vast/blob/main/ray-vast-pypi/CONTRIBUTING.md) for the tests and the docs build.
+
+## License
+
+MIT. See [LICENSE](https://github.com/duc-ph/ray-vast/blob/main/ray-vast-pypi/LICENSE).

ray_vast-0.1.0/pyproject.toml

@@ -0,0 +1,50 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "ray-vast"
+dynamic = ["version"]
+description = "A Ray autoscaler node provider for vast.ai workers with an AWS EC2 head."
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.10"
+authors = [{ name = "Duc Pham", email = "thiel.pham@gmail.com" }]
+dependencies = ["ray[default]>=2.59", "boto3", "cryptography>=42", "pyyaml", "requests"]
+
+[project.urls]
+Documentation = "https://duc-ph.github.io/ray-vast/"
+Source = "https://github.com/duc-ph/ray-vast"
+Changelog = "https://duc-ph.github.io/ray-vast/reference/changelog/"
+Issues = "https://github.com/duc-ph/ray-vast/issues"
+
+[project.optional-dependencies]
+test = ["pytest", "pytest-timeout", "moto[server]", "responses", "freezegun", "build", "hatchling"]
+
+[project.scripts]
+ray-vast = "ray_vast.cli:main"
+
+[tool.hatch.version]
+path = "src/ray_vast/__init__.py"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/ray_vast"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src", "tests", "scripts", "README.md", "LICENSE", "CHANGELOG.md", "pyproject.toml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = [
+    "live: uses real AWS or vast.ai (skipped by default)",
+    "slow: starts processes, containers, TLS servers or wheel builds, or waits on real timers (skipped by default)",
+    "local_e2e: runs a Ray head and workers in Docker on the dev box (step 11; skipped by default)",
+]
+addopts = "-m 'not slow and not live and not local_e2e'"
+# A hung test (for example a `down` wait loop) fails after this many seconds instead of stopping the run.
+timeout = 300
+
+[tool.ruff]
+line-length = 120
+src = ["src", "tests"]

ray_vast-0.1.0/scripts/build_shim.sh

@@ -0,0 +1,18 @@
+#!/bin/sh
+# Build the shim in manylinux2014 (glibc 2.17) and check it. Run where Docker works.
+# The build goes to a temp dir; compare the printed sha256 with the committed .so.
+set -eu
+dir=$(cd "$(dirname "$0")/../src/ray_vast/shim" && pwd)
+out=$(mktemp -d)
+docker run --rm -v "$dir":/src:ro -v "$out":/out quay.io/pypa/manylinux2014_x86_64 sh -eu -c '
+  gcc -shared -fPIC -O2 -o /out/librayvast_shim.so /src/rayvast_shim.c -ldl
+  sha256sum /out/librayvast_shim.so
+  symbols=$(nm -D /out/librayvast_shim.so)
+  echo "$symbols"
+  echo "$symbols" | grep -q " T bind$" || { echo "missing T bind" >&2; exit 1; }
+  echo "$symbols" | grep -q " T connect$" || { echo "missing T connect" >&2; exit 1; }
+  newest=$(echo "$symbols" | grep -o "GLIBC_[0-9.]*" | sed "s/GLIBC_//" | sort -V | tail -1)
+  [ -z "$newest" ] || [ "$(printf "%s\n2.17\n" "$newest" | sort -V | tail -1)" = "2.17" ] || { echo "GLIBC_$newest is above 2.17" >&2; exit 1; }
+  echo "ok: symbols and glibc version"
+'
+echo "built: $out/librayvast_shim.so"

ray_vast-0.1.0/scripts/shim_mutants.py

@@ -0,0 +1,130 @@
+#!/usr/bin/env python3
+"""Mutation run for the shim tests (docs/implementation-plan.md step 10c).
+
+Makes one mutant of src/ray_vast/shim/rayvast_shim.c for each entry in MUTANTS (one small change each), builds the
+unchanged source and all mutants in quay.io/pypa/manylinux2014_x86_64 (as scripts/build_shim.sh does), runs
+tests/integration/test_shim.py against each .so, and prints killed or survived. The unchanged build must pass.
+
+Run from the package directory with the test venv, where Docker works:
+    .venv/bin/python scripts/shim_mutants.py
+Exit code 0 when the unchanged build passes; the survivors are printed (explain each one in docs/test-results.md).
+"""
+import hashlib
+import os
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+PACKAGE = Path(__file__).resolve().parents[1]
+SOURCE = PACKAGE / "src" / "ray_vast" / "shim" / "rayvast_shim.c"
+TEST = PACKAGE / "tests" / "integration" / "test_shim.py"
+
+# name: (text in the source, its replacement). Each text must occur exactly once.
+MUTANTS = {
+    "rule1-removed": ("if (is_bind && is_localhost_port) {", "if (0) {"),
+    "rule1-target-private": (
+        "target.s_addr = htonl(INADDR_LOOPBACK);\n        *loopback_bind = 1;",
+        "target = private_ip;\n        *loopback_bind = 1;",
+    ),
+    "rule3-removed": ("} else if (is_public && !is_bind && is_localhost_port) {", "} else if (0) {"),
+    "rules2-4-removed": ("} else if (is_public && have_private) {", "} else if (0) {"),
+    "rule2-removed": ("} else if (is_public && have_private) {", "} else if (is_public && have_private && !is_bind) {"),
+    "rule4-removed": ("} else if (is_public && have_private) {", "} else if (is_public && have_private && is_bind) {"),
+    "port-compare-flipped": ("if (localhost_ports[i] == port)", "if (localhost_ports[i] != port)"),
+    "public-compare-flipped": (
+        "is_public = !v6_any && have_public && ip4->s_addr == public_ip.s_addr;",
+        "is_public = !v6_any && have_public && ip4->s_addr != public_ip.s_addr;",
+    ),
+    "same-target-compare-flipped": (
+        "if (!v6_any && ip4->s_addr == target.s_addr)", "if (!v6_any && ip4->s_addr != target.s_addr)"
+    ),
+    "port-range-flipped": ("p > 0 && p < 65536", "p > 0 && p > 65536"),
+    "port-list-first-only": ("s = strchr(end, ',');", "s = NULL;"),
+    "inet-length-check-flipped": (
+        "addr->sa_family == AF_INET && len >= sizeof(struct sockaddr_in))",
+        "addr->sa_family == AF_INET && len < sizeof(struct sockaddr_in))",
+    ),
+    "inet6-family-check-wrong": (
+        "addr->sa_family == AF_INET6 && len >= sizeof(struct sockaddr_in6)",
+        "addr->sa_family == AF_INET && len >= sizeof(struct sockaddr_in6)",
+    ),
+    "inet6-v4mapped-check-removed": (
+        "if (!IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr) && !v6_any)\n            return 0;\n", ""
+    ),
+    "inet6-any-not-detected": ("v6_any = IN6_IS_ADDR_UNSPECIFIED(&sin6->sin6_addr);", "v6_any = 0;"),
+    "inet6-any-no-ffff-prefix": ("sin6->sin6_addr.s6_addr[10] = 0xff;\n", ""),
+    "inet6-length-wrong": ("*out_len = sizeof(struct sockaddr_in6);", "*out_len = sizeof(struct sockaddr_in);"),
+    "reuseport-removed": ("setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &one, sizeof one);", "(void)one;"),
+    "connect-uses-old-address": (
+        "return real_connect(fd, (const struct sockaddr *)&new_addr, new_len);", "return real_connect(fd, addr, len);"
+    ),
+    "errno-save-removed-in-log": ("    errno = saved_errno;\n}\n\nint bind", "}\n\nint bind"),
+    "errno-save-removed-in-bind": ("        errno = saved_errno;\n    }\n    if (changed)", "    }\n    if (changed)"),
+    "errno-save-removed-in-init": ("    errno = saved_errno;\n}\n\n__attribute__", "}\n\n__attribute__"),
+    "env-copy-removed": (
+        "static char log_path[512];",
+        "static const char *log_path = \"\";",
+    ),
+    "log-bind-removed": ('log_rewrite(loopback_bind ? "bind+reuseport" : "bind", addr, &new_addr);', ""),
+    "log-connect-removed": ('log_rewrite("connect", addr, &new_addr);', ""),
+}
+# The env-copy mutant also needs the copy code replaced by a pointer assignment.
+ENV_COPY_CODE = "    log_path[0] = 0;\n    if (s && strlen(s) < sizeof log_path)\n        strcpy(log_path, s);\n"
+
+
+def main():
+    source = SOURCE.read_text()
+    work = Path(tempfile.mkdtemp(prefix="shim-mutants-"))
+    # Each source keeps the name rayvast_shim.c (the name goes into the .so), so the unchanged build is the same file
+    # as the committed .so.
+    (work / "original").mkdir()
+    (work / "original" / "rayvast_shim.c").write_text(source)
+    for name, (old, new) in MUTANTS.items():
+        assert source.count(old) == 1, f"{name}: the text occurs {source.count(old)} times"
+        mutant = source.replace(old, new)
+        if name == "env-copy-removed":
+            assert mutant.count(ENV_COPY_CODE) == 1
+            mutant = mutant.replace(ENV_COPY_CODE, "    log_path = s ? s : \"\";\n")
+        (work / name).mkdir()
+        (work / name / "rayvast_shim.c").write_text(mutant)
+
+    build = subprocess.run(
+        ["docker", "run", "--rm", "--name", f"st10c-shim-mutants-{os.getpid()}", "-u", f"{os.getuid()}:{os.getgid()}",
+         "-v", f"{work}:/work", "quay.io/pypa/manylinux2014_x86_64", "sh", "-eu", "-c",
+         'for d in /work/*/; do cd "$d" && gcc -shared -fPIC -O2 -o librayvast_shim.so rayvast_shim.c -ldl; done'],
+        capture_output=True, text=True, check=False,
+    )
+    if build.returncode != 0:
+        sys.exit(f"build failed:\n{build.stdout}{build.stderr}")
+    print(f"mutants in {work}", flush=True)
+    committed = hashlib.sha256((SOURCE.parent / "librayvast_shim.so").read_bytes()).hexdigest()
+    rebuilt = hashlib.sha256((work / "original" / "librayvast_shim.so").read_bytes()).hexdigest()
+    if rebuilt != committed:
+        sys.exit(f"the unchanged source builds to sha256 {rebuilt}, not to the committed .so {committed}")
+    print("the unchanged source builds to the committed .so", flush=True)
+
+    def run_tests(so):
+        result = subprocess.run(
+            [sys.executable, "-m", "pytest", "-o", "addopts=", "-q", "-p", "no:cacheprovider", str(TEST)],
+            env={**os.environ, "RAYVAST_TEST_SHIM_SO": str(so)}, cwd=PACKAGE, capture_output=True, text=True, check=False,
+        )
+        if result.returncode not in (0, 1):  # 1 = some tests failed; anything else is a broken run
+            sys.exit(f"pytest error for {so}:\n{result.stdout}{result.stderr}")
+        failed = [line.split(" ")[1].split("::")[-1] for line in result.stdout.splitlines() if line.startswith("FAILED")]
+        return result.returncode == 0, failed
+
+    passed, failed = run_tests(work / "original" / "librayvast_shim.so")
+    if not passed:
+        sys.exit(f"the unchanged source fails the tests: {failed}")
+    survivors = []
+    for name in MUTANTS:
+        survived, failed = run_tests(work / name / "librayvast_shim.so")
+        print(f"{'SURVIVED' if survived else 'killed  '} {name}" + (f"  ({', '.join(failed)})" if failed else ""), flush=True)
+        if survived:
+            survivors.append(name)
+    print(f"{len(MUTANTS) - len(survivors)} killed, {len(survivors)} survived: {', '.join(survivors) or '-'}")
+
+
+if __name__ == "__main__":
+    main()

ray_vast-0.1.0/src/ray_vast/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

ray_vast-0.1.0/src/ray_vast/aws_sg.py

@@ -0,0 +1,70 @@
+"""Ingress rules on the cluster's security groups (up to 5, design 8.3): TCP all ports from one worker IP, with the
+description `ray-vast:<cluster>`. A rule goes into the first group with room."""
+import botocore.config
+import botocore.exceptions
+
+from .vast_api import label
+
+class SgQuotaError(Exception):
+    """No security group of the cluster has room for another worker IP."""
+
+
+def ec2_client_config():
+    """botocore Config for the ec2 client that the caller passes to SecurityGroupRules."""
+    return botocore.config.Config(connect_timeout=5, read_timeout=30, retries={"max_attempts": 5, "mode": "standard"})
+
+
+class SecurityGroupRules:
+    def __init__(self, ec2, group_ids, cluster):
+        self._ec2 = ec2
+        self._group_ids = list(group_ids)
+        self._description = label(cluster)
+
+    def _our_rules(self):
+        """Return {cidr: group id} for every rule of this cluster in its groups."""
+        groups = self._ec2.describe_security_groups(GroupIds=self._group_ids)["SecurityGroups"]
+        return {r["CidrIp"]: g["GroupId"] for g in groups for p in g["IpPermissions"] for r in p.get("IpRanges", [])
+                if r.get("Description") == self._description}
+
+    def allow_ip(self, ip):
+        """Add the rule to the first group with room. A rule that exists already (in any group) is success."""
+        cidr = f"{ip}/32"
+        if cidr in self._our_rules():
+            return
+        for group_id in self._group_ids:
+            try:
+                self._ec2.authorize_security_group_ingress(GroupId=group_id, IpPermissions=[{
+                    "IpProtocol": "tcp", "FromPort": 0, "ToPort": 65535,
+                    "IpRanges": [{"CidrIp": cidr, "Description": self._description}]}])
+                return
+            except botocore.exceptions.ClientError as e:
+                code = e.response["Error"]["Code"]
+                if code == "InvalidPermission.Duplicate":  # success: the groups belong to this cluster only
+                    return
+                if code != "RulesPerSecurityGroupLimitExceeded":
+                    raise
+        raise SgQuotaError(f"all {len(self._group_ids)} security groups of the cluster are full; raise the quota in AWS "
+                           "Service Quotas (VPC: inbound or outbound rules per security group)")
+
+    def _revoke(self, group_id, cidrs):
+        try:
+            self._ec2.revoke_security_group_ingress(GroupId=group_id, IpPermissions=[{
+                "IpProtocol": "tcp", "FromPort": 0, "ToPort": 65535, "IpRanges": [{"CidrIp": c} for c in cidrs]}])
+        except botocore.exceptions.ClientError as e:
+            if e.response["Error"]["Code"] != "InvalidPermission.NotFound":
+                raise
+
+    def revoke_ip(self, ip):
+        """Remove the rule from the group that holds it. A missing rule is success. Only a rule with our description."""
+        cidr = f"{ip}/32"
+        group_id = self._our_rules().get(cidr)
+        if group_id:
+            self._revoke(group_id, [cidr])
+
+    def revoke_all(self):
+        """Remove every rule of this cluster from all its groups; other rules (SSH, other clusters) stay."""
+        by_group = {}
+        for cidr, group_id in self._our_rules().items():
+            by_group.setdefault(group_id, []).append(cidr)
+        for group_id, cidrs in by_group.items():
+            self._revoke(group_id, cidrs)

ray_vast-0.1.0/src/ray_vast/cli.py

@@ -0,0 +1,32 @@
+import argparse
+import os
+import sys
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(prog="ray-vast", description="Ray clusters on vast.ai workers with an AWS EC2 head.")
+    subparsers = parser.add_subparsers(dest="command", required=True)
+    for name, help_text in [
+        ("up", "create the head and start the cluster"),
+        ("down", "destroy all workers and the head"),
+        ("attach", "open a shell on the head"),
+        ("dashboard", "open an SSH tunnel to the Ray dashboard"),
+    ]:
+        sub = subparsers.add_parser(name, help=help_text)
+        sub.add_argument("cluster_yaml")
+        sub.add_argument("--profile", default=os.environ.get("AWS_PROFILE"), help="AWS profile (default: AWS_PROFILE or the default credentials)")
+        if name in ("up", "attach", "dashboard"):
+            sub.add_argument("--key-file", help="SSH private key of the head (or provider.key_file)")
+        if name == "up":
+            sub.add_argument("--key-name", help="EC2 key pair name (or provider.key_name)")
+            sub.add_argument("--wheel", help="ray-vast wheel to install on the head, for development (default: in a source tree, build "
+                                             "the wheel; else install this CLI's own version from PyPI)")
+            sub.add_argument("--index-url", help="install ray-vast on the head from this package index, with PyPI as an extra index "
+                                                 "(to test a release on TestPyPI; not for normal use)")
+    args = parser.parse_args(argv)
+    # Progress lines show at once also when stdout is a pipe or a file (there Python buffers whole blocks).
+    sys.stdout.reconfigure(line_buffering=True)
+
+    from .commands import run
+
+    return run(args)