eray 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. eray-0.1.0/PKG-INFO +561 -0
  2. eray-0.1.0/README.md +530 -0
  3. eray-0.1.0/eray/__init__.py +192 -0
  4. eray-0.1.0/eray/cli/__init__.py +36 -0
  5. eray-0.1.0/eray/cli/autoscale.py +418 -0
  6. eray-0.1.0/eray/cli/dashboard.py +437 -0
  7. eray-0.1.0/eray/cli/fleet.py +422 -0
  8. eray-0.1.0/eray/cli/hostjob.py +506 -0
  9. eray-0.1.0/eray/cli/jobs.py +1350 -0
  10. eray-0.1.0/eray/cli/main.py +802 -0
  11. eray-0.1.0/eray/cli/qr.py +249 -0
  12. eray-0.1.0/eray/cli/tpu.py +763 -0
  13. eray-0.1.0/eray/cli/utils.py +707 -0
  14. eray-0.1.0/eray/core/__init__.py +63 -0
  15. eray-0.1.0/eray/core/cluster.py +106 -0
  16. eray-0.1.0/eray/core/exceptions.py +207 -0
  17. eray-0.1.0/eray/core/monitoring.py +383 -0
  18. eray-0.1.0/eray/core/sentinels.py +97 -0
  19. eray-0.1.0/eray/core/status.py +105 -0
  20. eray-0.1.0/eray/docker/__init__.py +48 -0
  21. eray-0.1.0/eray/docker/config.py +131 -0
  22. eray-0.1.0/eray/docker/runner.py +351 -0
  23. eray-0.1.0/eray/execution/__init__.py +43 -0
  24. eray-0.1.0/eray/execution/decorators.py +281 -0
  25. eray-0.1.0/eray/execution/executor.py +741 -0
  26. eray-0.1.0/eray/execution/remote.py +655 -0
  27. eray-0.1.0/eray/gcp.py +177 -0
  28. eray-0.1.0/eray/pool/__init__.py +40 -0
  29. eray-0.1.0/eray/pool/base.py +284 -0
  30. eray-0.1.0/eray/pool/device_host.py +717 -0
  31. eray-0.1.0/eray/pool/slice.py +1195 -0
  32. eray-0.1.0/eray/provision/__init__.py +66 -0
  33. eray-0.1.0/eray/provision/fleet.py +332 -0
  34. eray-0.1.0/eray/provision/launcher.py +497 -0
  35. eray-0.1.0/eray/provision/qr.py +442 -0
  36. eray-0.1.0/eray/provision/registry.py +510 -0
  37. eray-0.1.0/eray/provision/templates/cluster-template.yaml +63 -0
  38. eray-0.1.0/eray/provision/tunnel.py +424 -0
  39. eray-0.1.0/eray/provision/watcher.py +777 -0
  40. eray-0.1.0/eray/py.typed +0 -0
  41. eray-0.1.0/eray/resources/__init__.py +50 -0
  42. eray-0.1.0/eray/resources/configs.py +676 -0
  43. eray-0.1.0/eray/resources/hardware.py +82 -0
  44. eray-0.1.0/eray/resources/ray_resources.py +387 -0
  45. eray-0.1.0/eray/resources/topology.py +344 -0
  46. eray-0.1.0/eray/swarm.py +845 -0
  47. eray-0.1.0/pyproject.toml +115 -0
eray-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,561 @@
1
+ Metadata-Version: 2.3
2
+ Name: eray
3
+ Version: 0.1.0
4
+ Summary: Ray-based distributed execution, scaling, and resource management for ML workloads on CPUs, GPUs, and TPUs
5
+ Keywords: Ray,Distributed Computing,TPU,GPU,Scaling,EasyDeL,eFormer,JAX
6
+ Author: Erfan Zare Chavoshi
7
+ Author-email: Erfan Zare Chavoshi <Erfanzare810@gmail.com>
8
+ License: Apache-2.0
9
+ Classifier: Development Status :: 3 - Alpha
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: Topic :: System :: Distributed Computing
13
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
14
+ Classifier: License :: OSI Approved :: Apache Software License
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Requires-Dist: ray[default]>=2.54.0
20
+ Requires-Dist: mergedeep~=1.3.4
21
+ Requires-Dist: tblib~=3.0.0
22
+ Requires-Dist: requests>=2.31.0
23
+ Requires-Dist: click>=8.1.0
24
+ Requires-Dist: pytest ; extra == 'dev'
25
+ Requires-Python: >=3.11, <3.14
26
+ Project-URL: Documentation, https://eray.readthedocs.io/
27
+ Project-URL: Homepage, https://github.com/erfanzar/eray
28
+ Project-URL: Repository, https://github.com/erfanzar/eray
29
+ Provides-Extra: dev
30
+ Description-Content-Type: text/markdown
31
+
32
+ # eray ⚡
33
+
34
+ [![PyPI version](https://img.shields.io/pypi/v/eray?logo=pypi&color=3776ab)](https://pypi.org/project/eray/)
35
+ [![Python](https://img.shields.io/badge/python-3.11--3.13-blue)](https://www.python.org/)
36
+ [![Ray](https://img.shields.io/badge/Ray-2.54%2B-informational)](https://github.com/ray-project/ray)
37
+ [![License](https://img.shields.io/badge/license-Apache%202.0-green)](LICENSE)
38
+ [![Status](https://img.shields.io/badge/status-alpha-orange)](#)
39
+
40
+ > Ray-based distributed execution, scaling, and resource management for ML workloads on CPUs, GPUs, and TPUs.
41
+
42
+ ## What is eray?
43
+
44
+ **eray** is a standalone extraction of the Ray executor stack from [eFormer](https://github.com/erfanzar/eFormer) / [EasyDeL](https://github.com/erfanzar/EasyDeL). It provides everything you need to run distributed machine learning workloads on Ray-managed clusters:
45
+
46
+ - **Fault-tolerant execution** — automatic retry and resumption for preempted/failed jobs
47
+ - **Multi-slice TPU support** — coordinate workloads across TPU pod slices with MegaScale
48
+ - **GPU & CPU execution** — run on NVIDIA GPUs, Intel/AMD accelerators, or plain CPU
49
+ - **Docker integration** — build, push, and run containers on remote nodes
50
+ - **Resource pooling** — actor pool management with health checks and auto-scaling
51
+ - **Rich type system** — job status tracking, exception serialization across processes
52
+
53
+ ## Installation
54
+
55
+ ```bash
56
+ pip install eray
57
+ ```
58
+
59
+ For TPU environments, you may also want `jax` and `jaxlib` installed separately.
60
+
61
+ ## Quickstart
62
+
63
+ ### Single-pod execution
64
+
65
+ ```python
66
+ import eray
67
+
68
+ # Run a function on a GPU pod with automatic retries
69
+ @eray.execute(eray.GpuAcceleratorConfig(count=1, gpu_model="A100"))
70
+ def train_step(batch):
71
+ ...
72
+ return loss
73
+ ```
74
+
75
+ ### Resumable execution with preemption recovery
76
+
77
+ ```python
78
+ result = eray.execute_resumable(
79
+ remote_fn=train_fn,
80
+ accelerator_config=eray.TpuAcceleratorConfig(tpu_version="v4-8", pod_count=1),
81
+ max_preemption_retries=1_000_000,
82
+ )
83
+ ```
84
+
85
+ ### Multi-slice TPU autoscaling
86
+
87
+ ```python
88
+ results = eray.autoscale_execute_resumable(
89
+ remote_fn=distributed_train,
90
+ accelerator_config=eray.TpuAcceleratorConfig(
91
+ tpu_version="v4-32",
92
+ pod_count=4,
93
+ ),
94
+ )
95
+ ```
96
+
97
+ ### Docker on distributed nodes
98
+
99
+ ```python
100
+ from eray import DockerConfig, run_docker_multislice
101
+
102
+ docker_config = DockerConfig(
103
+ image="my-ml-image:latest",
104
+ command="python train.py",
105
+ volumes={"/data": "/data"},
106
+ )
107
+
108
+ outputs = run_docker_multislice(
109
+ docker_config,
110
+ accelerator_config=eray.TpuAcceleratorConfig(tpu_version="v4-32", pod_count=4),
111
+ )
112
+ ```
113
+
114
+ ### `device_remote` decorator
115
+
116
+ ```python
117
+ @eray.device_remote(accelerator_config=eray.TpuAcceleratorConfig(tpu_version="v4-8"))
118
+ class Trainer:
119
+ def train(self, data):
120
+ ...
121
+
122
+ trainer = Trainer()
123
+ trainer.train.remote(data)
124
+ ```
125
+
126
+ ### Host chip splitting
127
+
128
+ A TPU host is normally driven by one process owning every chip. eray can carve
129
+ a host into multiple processes with disjoint chip ownership (the same libtpu
130
+ environment mechanism torch_xla uses):
131
+
132
+ ```python
133
+ import eray
134
+
135
+ # Plan only (pure, no Ray): inspect the per-split environment.
136
+ plan = eray.plan_host_split(num_chips=4, num_splits=4, mode="isolated")
137
+
138
+ # On a slice: run fn once per split on every host.
139
+ # SliceActor.run_split_remote_fn / DeviceHostActor.run_split_remote_fn
140
+ refs = ray.get(host_actor.run_split_remote_fn.remote(fn, 4, mode="isolated"))
141
+ results = ray.get(refs) # one result per split; ERAY_SPLIT_ID identifies each
142
+ ```
143
+
144
+ - `isolated` (default): each split is an independent runtime with 1 or 2 chips
145
+ assigned disjointly by Ray — e.g. four single-chip serving replicas on a
146
+ v4/v5p host.
147
+ - `cooperative`: one process per chip forming a single multi-process runtime
148
+ (communicating over ICI via `jax.distributed`); chip-dependent variables
149
+ (`TPU_VISIBLE_CHIPS`, `CLOUD_TPU_TASK_ID`, `TPU_PROCESS_PORT`) are bound
150
+ inside each task from Ray's chip assignment.
151
+
152
+ ### Swarms — heterogeneous runs with a chip plan
153
+
154
+ `eray.swarm` packs several independent workloads onto the same TPU hosts.
155
+ `SwarmConfig` mirrors `TpuAcceleratorConfig` (it takes the chip type) plus the
156
+ number of runs and an optional plan for how to split each host's chips:
157
+
158
+ ```python
159
+ import eray
160
+
161
+ # Even split: four replicas, one chip each per host.
162
+ status = eray.swarm_execute(serve_replica, eray.SwarmConfig(tpu_version="v5p-8", num_runs=4))
163
+
164
+ # A plan: three different workloads, chips split (2, 1, 1).
165
+ status = eray.swarm_execute(
166
+ [
167
+ eray.SwarmRun(train_probe, name="train"),
168
+ eray.SwarmRun(serve_small, name="serve"),
169
+ eray.SwarmRun(eval_loop, name="eval"),
170
+ ],
171
+ eray.SwarmConfig(tpu_version="v5p-8", chip_split=(2, 1, 1)),
172
+ )
173
+ ```
174
+
175
+ **GPU swarms** use `GpuSwarmConfig` — same layout knobs, but no slice pool:
176
+ Ray natively assigns disjoint GPUs (`CUDA_VISIBLE_DEVICES`) per run, any GPU
177
+ count is allowed (fractions of one GPU too), and `colocate=True` packs all
178
+ runs onto one node via a placement group:
179
+
180
+ ```python
181
+ status = eray.swarm_execute(
182
+ [
183
+ eray.SwarmRun(train_probe, name="train", num_cores=8), # 2 GPUs, 8 CPU cores
184
+ eray.SwarmRun(serve_small, name="serve"), # 1 GPU
185
+ eray.SwarmRun(eval_loop, name="eval"), # 1 GPU
186
+ ],
187
+ eray.GpuSwarmConfig(gpu_model="A100", gpu_split=(2, 1, 1), colocate=True),
188
+ )
189
+ ```
190
+
191
+ `SwarmRun.num_cores` reserves CPU cores per run on both TPU and GPU swarms
192
+ (default: the config's `cpu_count` on GPU, the launcher default on TPU).
193
+ GPU runs see `ERAY_RUN_GPUS` instead of `ERAY_RUN_CHIPS`.
194
+
195
+ There is also a decorator form — bare `SwarmRun(...)` specs (no `fn`) bind to
196
+ the decorated function, and calling it launches the swarm:
197
+
198
+ ```python
199
+ @eray.swarmed(
200
+ eray.SwarmConfig(tpu_version="v5p-8"),
201
+ runs=[
202
+ eray.SwarmRun(name="lr3e4", f_kwargs={"lr": 3e-4}),
203
+ eray.SwarmRun(name="lr1e4", f_kwargs={"lr": 1e-4}),
204
+ eray.SwarmRun(name="lr5e5", f_kwargs={"lr": 5e-5}),
205
+ eray.SwarmRun(name="lr1e5", f_kwargs={"lr": 1e-5}),
206
+ ],
207
+ )
208
+ def train(lr, warmup):
209
+ ...
210
+
211
+ status = train(warmup=100) # call-time kwargs broadcast to every run
212
+ ```
213
+
214
+ A run may also be a plain **class**: it is instantiated as a long-lived,
215
+ detached named Ray actor on its chip share, addressable from any driver:
216
+
217
+ ```python
218
+ runs = [
219
+ eray.SwarmRun(ServeReplica, name="serve-a"), # class -> named actor
220
+ eray.SwarmRun(ServeReplica, name="serve-b"),
221
+ eray.SwarmRun(train_probe, name="train"), # function -> task
222
+ ]
223
+ status = eray.swarm_execute(
224
+ runs,
225
+ eray.SwarmConfig(tpu_version="v5p-8", chip_split=(1, 1, 2), namespace="my-swarm"),
226
+ )
227
+
228
+ # later, from any driver:
229
+ serve = ray.get_actor("serve-a", namespace="my-swarm")
230
+ ray.get(serve.generate.remote(prompt))
231
+
232
+ eray.shutdown_swarm("my-swarm") # kill all of the swarm's named actors
233
+ ```
234
+
235
+ Each run is an isolated runtime with disjoint chips (assigned by Ray) and sees
236
+ `ERAY_RUN_ID` / `ERAY_NUM_RUNS` / `ERAY_RUN_CHIPS` / `ERAY_RUN_NAME` in its
237
+ environment. Chip counts per run must be 1, 2, or the whole host, and the plan
238
+ must not oversubscribe the host. Hardware note (validated on v5p-8): 2-chip
239
+ runs only form a sub-slice on aligned chip pairs ({0,1}, {2,3}); a misaligned
240
+ Ray assignment raises a clear error in-task instead of crashing libtpu. When
241
+ launching a Ray driver on a TPU VM, invoke the venv python directly — Ray's
242
+ `uv run` hook repackages the working dir and workers lose libtpu. With `pod_count > 1` or multi-host slices the
243
+ same layout repeats on every host (actor names get a `-s{slice}h{host}`
244
+ suffix). Class-run actors are detached: they survive the swarm call and hold
245
+ their chips until `shutdown_swarm(namespace)` or `ray.kill`.
246
+
247
+ ## Module Map
248
+
249
+ | Module | Description |
250
+ |--------|-------------|
251
+ | `eray.types` | Job status types, exception serialization, sentinels, coordination primitives |
252
+ | `eray.resource_manager` | `ComputeResourceConfig`, `RayResources`, hardware constants |
253
+ | `eray.pool_manager` | Actor pool management, `SlicePoolManager`, `DeviceHostActor` |
254
+ | `eray.executor` | `RayExecutor`, `execute`, `autoscale_execute`, `device_remote` |
255
+ | `eray.docker_executor` | Docker config, build/push, container execution |
256
+
257
+ ## Key Types
258
+
259
+ ```python
260
+ from eray import (
261
+ # Execution
262
+ execute, execute_resumable,
263
+ autoscale_execute, autoscale_execute_resumable,
264
+ device_remote, RayExecutor,
265
+
266
+ # Resource configs
267
+ CpuAcceleratorConfig, GpuAcceleratorConfig, TpuAcceleratorConfig,
268
+ ComputeResourceConfig, RayResources, AcceleratorConfigType,
269
+
270
+ # Job status
271
+ JobSucceeded, JobFailed, JobPreempted, JobError, JobStatus, JobInfo,
272
+
273
+ # Docker
274
+ DockerConfig, run_docker_on_pod, run_docker_multislice,
275
+
276
+ # Pool management
277
+ ResourcePoolManager, SlicePoolManager, DeviceHostActor,
278
+
279
+ # Host chip splitting
280
+ HostSplitPlan, plan_host_split,
281
+
282
+ # Swarms
283
+ SwarmConfig, GpuSwarmConfig, SwarmRun, swarm_execute, swarmed, shutdown_swarm, HostPartitionPlan, plan_host_partition,
284
+
285
+ # Utilities
286
+ handle_ray_error, ExceptionInfo, StopwatchActor, RefBox, DONE,
287
+ )
288
+ ```
289
+
290
+ ## Environment Variables
291
+
292
+ | Variable | Default | Description |
293
+ |----------|---------|-------------|
294
+ | `ERAY_SCALE_RETRY_SLEEP` | `60` | Sleep between retry attempts when slices are unavailable |
295
+ | `ERAY_SUBPROCESS_TIMEOUT` | — | Timeout for forked subprocess execution |
296
+ | `ERAY_SAFE_GATHER` | — | Use safe gather (prune dead actors) in slice preparation |
297
+ | `ERAY_MODERATE` | — | Adjust num_hosts/num_devices discovery |
298
+ | `ERAY_KILL_VFIO` | — | Kill VFIO holders on TPU hosts |
299
+ | `ERAY_INSTALL_LSOF` | — | Attempt quiet lsof install for VFIO holder detection |
300
+ | `ERAY_HOST_HEALTH_WAIT` | `60` | Timeout for waiting on all hosts to become healthy |
301
+ | `ERAY_SCALE_POLL` | `30` | Poll interval for actor pool scaling |
302
+ | `ERAY_SCALE_ADD_TIMEOUT` | `604800` | Timeout for adding actors to pool (7 days) |
303
+
304
+ ## CLI — TPU Cluster Management
305
+
306
+ `eray` ships with a CLI for connecting TPU hosts into a Ray cluster. It ports the Ray connection logic from `scripts/tpu_setup.sh` into a clean Python tool.
307
+
308
+ Run on the TPU VM itself, no flags are needed at all — the TPU's name, type,
309
+ zone, and every worker's IP are auto-detected from the instance metadata
310
+ server, and commands targeting this machine execute locally (no SSH):
311
+
312
+ ```bash
313
+ eray tpu connect # auto-detects the current TPU
314
+ eray tpu status # auto-detects the head address
315
+ eray tpu health
316
+ eray resources # cluster resources: used / total / free / util
317
+ eray tpu disconnect
318
+ ```
319
+
320
+ ### Prerequisites
321
+
322
+ - `gcloud` CLI installed and authenticated
323
+ - TPU VMs in `READY` state
324
+ - `ray` installed on each TPU host
325
+
326
+ ### Commands
327
+
328
+ ```bash
329
+ # Connect a TPU slice into a Ray cluster
330
+ eray tpu connect -n my-tpu -p my-project -z us-central2-b
331
+
332
+ # Check cluster status (nodes, TPU resources)
333
+ eray tpu status -a 10.0.0.1:6379
334
+
335
+ # Run health check (JAX devices per host)
336
+ eray tpu health -a 10.0.0.1:6379
337
+
338
+ # Show resource usage (CPU, TPU, memory, custom resources); --per-node for a host table
339
+ eray resources -a 10.0.0.1:6379 --per-node
340
+
341
+ # Stop Ray on all hosts
342
+ eray tpu disconnect -n my-tpu -p my-project -z us-central2-b
343
+
344
+ # List TPUs in a zone
345
+ eray tpu list -p my-project -z us-central2-b
346
+ ```
347
+
348
+ ### How `connect` works
349
+
350
+ 1. **Discovers TPU** via `gcloud compute tpus tpu-vm describe` → gets internal IPs, accelerator type, worker count
351
+ 2. **Cleans existing Ray** on all hosts (kills stale processes, frees ports)
352
+ 3. **Starts Ray head** on worker 0 with TPU-specific custom resources
353
+ 4. **Starts Ray workers** on all other hosts, pointing at the head
354
+ 5. **Waits for readiness** — polls `ray.nodes()` until all hosts register
355
+
356
+ ### Resource allocation
357
+
358
+ The cluster is configured with custom Ray resources matching the TPU topology:
359
+
360
+ | Resource | Head | Workers |
361
+ |----------|------|---------|
362
+ | `TPU` | `chips_per_host` | `chips_per_host` |
363
+ | `TPU-{version}` | `chips_per_host` | `chips_per_host` |
364
+ | `TPU-{version}-{slice}-head` | `1` | — |
365
+ | `accelerator_type:TPU-{VERSION}` | `1` | `1` |
366
+ | `head-node` | `1` | — |
367
+
368
+ ### JSON output
369
+
370
+ All commands support `--json` for machine-readable output:
371
+
372
+ ```bash
373
+ eray tpu connect -n my-tpu -p proj -z zone --json | jq .ray_address
374
+ ```
375
+
376
+ ## Spot fleet management
377
+
378
+ eray manages spot TPU capacity end to end: **queued resources** are the
379
+ provisioning primitive, a **fleet registry** remembers every cluster, and a
380
+ **watcher** re-queues preempted capacity, reconnects Ray, and resubmits jobs.
381
+
382
+ ```bash
383
+ # capacity (stateless): spot by default, waits in queue instead of failing
384
+ eray qr create trainer1 --type v5p-64 --wait
385
+ eray qr list
386
+
387
+ # fleet (stateful): registry + reconciliation
388
+ eray fleet init --state gs://my-bucket/eray/clusters.json # shared registry (optional)
389
+ eray fleet add trainer1 --type v5p-64 --setup-easydel # EasyDeL bootstrap (--branch vnext pins a ref)
390
+ eray fleet add gpubox --type v5p-8 --bootstrap-cmd '...' # or any custom per-host bootstrap command
391
+ eray fleet add n_server_spot_m # adopt an existing QR by name
392
+ eray fleet ensure trainer1 --wait 3600 # request capacity + connect Ray, idempotent
393
+ eray fleet status
394
+
395
+ # jobs against a registered cluster; --restartable opts into auto-resubmission
396
+ eray run -c trainer1 --restartable -- python train.py
397
+
398
+ # the watcher: detect preemption -> re-queue -> reconnect -> resubmit
399
+ eray fleet watch --resubmit # foreground; --once for cron; --dry-run to preview
400
+ eray fleet pause / eray fleet resume # kill switch; resume clears HALTED_* park states
401
+ ```
402
+
403
+ How recovery works: the watcher (run it OFF the TPU — the head dies with the
404
+ slice) fuses three signals per tick (queued-resource state, node state, head
405
+ TCP reachability). Definitive preemption (node `PREEMPTED`, QR `SUSPENDED`)
406
+ triggers: mark degraded → delete the owned QR → create `{name}-r{gen+1}` (the
407
+ node id never changes, so `eray tpu connect -n NAME` always works) → wait
408
+ ACTIVE → bootstrap once per generation → reconnect → resubmit `--restartable`
409
+ jobs as `{id}-p{N}` with `ERAY_RESTART_COUNT`/`ERAY_PREEMPTED_FROM` env (your
410
+ trainer resumes from its own checkpoints). Budgets (4 recreates/hour, 12/day)
411
+ and quota-flavored failures park the cluster in `HALTED_*` instead of looping.
412
+ Jobs are snapshotted while healthy, and resubmission re-packages the recorded
413
+ cwd, so run the watcher on the machine you submit from.
414
+
415
+ Systemd unit example:
416
+
417
+ ```ini
418
+ [Unit]
419
+ Description=eray fleet watcher
420
+ After=network-online.target
421
+
422
+ [Service]
423
+ ExecStart=/path/to/venv/bin/eray fleet watch --resubmit
424
+ Restart=always
425
+ RestartSec=10
426
+
427
+ [Install]
428
+ WantedBy=default.target
429
+ ```
430
+
431
+ ### From a laptop (macOS)
432
+
433
+ Ray heads listen on internal VPC IPs, so a laptop outside the VPC splits the
434
+ work in two: the laptop is the control plane (everything gcloud-backed), and
435
+ one always-on box inside the VPC — any existing VM works — runs the watcher
436
+ and submits restartable jobs (resubmission re-packages that box's cwd).
437
+
438
+ Laptop, once (Python 3.11–3.13):
439
+
440
+ ```bash
441
+ brew install google-cloud-sdk uv && gcloud auth login
442
+ uv tool install "eray @ git+https://github.com/erfanzar/eray"
443
+ eray fleet init --state gs://my-bucket/eray/clusters.json # shared with the watcher box
444
+ ```
445
+
446
+ Watcher box, once (same install + `fleet init`, then keep alive via systemd
447
+ or tmux):
448
+
449
+ ```bash
450
+ eray fleet watch --resubmit
451
+ ```
452
+
453
+ Day to day, from the laptop:
454
+
455
+ ```bash
456
+ eray fleet add trainer1 --type v5p-64 --setup-easydel && eray fleet up trainer1
457
+ eray fleet status --no-probe # registry view; probes need VPC access
458
+ gcloud compute ssh <watcher-box> -- 'cd repo && eray run -c trainer1 --restartable -- python train.py'
459
+ eray fleet tunnel trainer1 & # forward the head's dashboard to 127.0.0.1:8265
460
+ eray logs <job-id> -a http://127.0.0.1:8265 -f # live logs / browser dashboard through the tunnel
461
+ ```
462
+
463
+ Preemption needs nothing from you: the watcher re-queues, reconnects, and
464
+ resubmits; `eray fleet status` shows the new generation. Non-restartable
465
+ one-off jobs can be submitted straight from the laptop through the tunnel
466
+ (`eray run -a http://127.0.0.1:8265 -- ...`).
467
+
468
+ ## Elastic autoscale clusters
469
+
470
+ `eray autoscale` wraps Ray's own cluster-launcher (`ray up/down/dashboard`)
471
+ for the elastic many-small-slices case (prefer `eray fleet` above for
472
+ reliable big slices). `up`/`down`/`status` never need a YAML path: with no
473
+ argument they pick from whatever `generate` already wrote — auto-selecting
474
+ the only profile, or prompting when there's more than one.
475
+
476
+ ```bash
477
+ eray autoscale generate --zones us-east5-a # writes ~/.eray/autoscale/easydel-us-east5-a.yaml
478
+ eray autoscale up # no path needed: picks/prompts among generated profiles
479
+ eray autoscale up easydel-us-east5-a # or name one directly (also accepts a YAML path)
480
+ eray autoscale status # every profile: zone, live status, head, uptime/died
481
+ eray autoscale down # picker again; stamps when it went down
482
+ ```
483
+
484
+ Every `up`/`down` also registers the cluster into the same fleet registry
485
+ `eray fleet` uses (`kind=launcher`), which is what makes `eray dashboard`
486
+ below see autoscale clusters right alongside `eray fleet` ones.
487
+
488
+ ## Dashboard
489
+
490
+ One place to see everything eray manages — `eray fleet` (QR-based TPU pods)
491
+ and `eray autoscale` (cluster-launcher) clusters together — and to open a
492
+ dashboard without hand-building a `ray dashboard`/gcloud SSH command:
493
+
494
+ ```bash
495
+ eray dashboard # table of every registered cluster; prompts + opens one if interactive
496
+ eray dashboard ls # same table, no opening (scriptable; --json too)
497
+ eray dashboard open trainer1 -o # open (or reuse) trainer1's dashboard; -o also launches the browser
498
+ eray dashboard stop --all # close every tracked tunnel
499
+ ```
500
+
501
+ Opening a dashboard spawns a tracked **background** port-forward instead of
502
+ blocking your terminal — `eray dashboard ls` shows the local port and PID of
503
+ anything already open, and asking to open the same cluster again reuses that
504
+ forward instead of colliding on the port (the failure mode a bare
505
+ `ray dashboard` has no memory to avoid on its own).
506
+
507
+ `eray dashboard` also sees generated autoscale profiles that were never
508
+ registered — brought up by hand, or by an eray version before `autoscale up`
509
+ started writing to the registry — listed as `UNREGISTERED`; you can `open`
510
+ one directly with no extra step, or run `eray autoscale up <name>` once
511
+ (idempotent against an already-running cluster) to make it a first-class
512
+ registered entry.
513
+
514
+ Opening a dashboard forwards **both** Ray's dashboard port (8265) and its GCS
515
+ port (6379) — over one SSH connection — because `ray status`, `eray
516
+ resources`, and `eray tpu status` read cluster state over GCS, not the
517
+ dashboard's HTTP port. (`eray` reads that state the way `ray status` does — a
518
+ pure GCS read via the autoscaler SDK — so it works over a tunnel;
519
+ `ray.init(address=...)` deliberately isn't used, as its driver join can't
520
+ complete over a single tunneled port.) You don't type the address either:
521
+ these commands **auto-detect** the tunnel's local port from eray's own tunnel
522
+ registry, so no `-a` is needed — correct even when the tunnel landed on a
523
+ non-default port because 8265/6379 were busy:
524
+
525
+ ```bash
526
+ eray dashboard open trainer1 # one SSH forwarding dashboard (8265) + GCS (6379)
527
+ eray resources # auto-detects the GCS tunnel — no -a
528
+ eray tpu status # ditto
529
+ eray status # auto-detects the dashboard tunnel — no -a
530
+ eray dashboard open trainer1 --no-gcs # dashboard port only
531
+ ```
532
+
533
+ Tunnels take ~15-20s to establish (SSH negotiation) before the ports answer.
534
+ `eray fleet tunnel` (the foreground, QR-only variant) forwards both ports the
535
+ same way; `--no-gcs` opts out. (`eray tpu health` runs a device probe on every
536
+ worker, so it needs a real driver join and only works on the cluster itself,
537
+ not from a laptop tunnel.)
538
+
539
+ ## Relationship to eFormer
540
+
541
+ `eray` was extracted from [`eformer.executor.ray`](https://github.com/erfanzar/eFormer) to provide a focused, standalone Ray execution toolkit. It has **zero dependencies on eFormer or JAX** — you can use it for any Ray-based distributed workload, ML or otherwise.
542
+
543
+ ## License
544
+
545
+ Apache License 2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE).
546
+
547
+ ```
548
+ Copyright 2026 The EasyDeL/eray Author Erfan Zare Chavoshi
549
+
550
+ Licensed under the Apache License, Version 2.0 (the "License");
551
+ you may not use this file except in compliance with the License.
552
+ You may obtain a copy of the License at
553
+
554
+ https://www.apache.org/licenses/LICENSE-2.0
555
+
556
+ Unless required by applicable law or agreed to in writing, software
557
+ distributed under the License is distributed on an "AS IS" BASIS,
558
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
559
+ See the License for the specific language governing permissions and
560
+ limitations under the License.
561
+ ```