dockhand-cli 0.3.1__tar.gz → 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- dockhand_cli-0.4.0/.github/workflows/publish.yml +16 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/AGENTS.md +33 -18
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/PKG-INFO +22 -6
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/README.md +21 -5
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/__init__.py +1 -1
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/pyproject.toml +1 -1
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/uv.lock +2 -2
- dockhand_cli-0.3.1/.claude/settings.local.json +0 -9
- dockhand_cli-0.3.1/CLAUDE.md +0 -309
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/.dockhand.json +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/.gitignore +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/LICENSE +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/build.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/client/__init__.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/client/base.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/client/local.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/client/ssh.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/config.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/constants.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/download.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/error.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/history.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/manage.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/queue.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/resubmit.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/submit.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/sync.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/tagging.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/transport.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/tunnel.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/dockhand/volumes.py +0 -0
- {dockhand_cli-0.3.1 → dockhand_cli-0.4.0}/media/Gemini_Generated_Image_i700fvi700fvi700.png +0 -0
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
name: Publish to PyPI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
release:
|
|
5
|
+
types: [published]
|
|
6
|
+
|
|
7
|
+
jobs:
|
|
8
|
+
publish:
|
|
9
|
+
runs-on: ubuntu-latest
|
|
10
|
+
permissions:
|
|
11
|
+
id-token: write # required for PyPI trusted publishing (OIDC) - no token needed
|
|
12
|
+
steps:
|
|
13
|
+
- uses: actions/checkout@v4
|
|
14
|
+
- uses: astral-sh/setup-uv@v3
|
|
15
|
+
- run: uv build
|
|
16
|
+
- run: uv publish
|
|
@@ -42,6 +42,12 @@ uv run ruff format .
|
|
|
42
42
|
uv run ruff check --fix .
|
|
43
43
|
```
|
|
44
44
|
|
|
45
|
+
**Build / release:**
|
|
46
|
+
The PyPI distribution is named `dockhand-cli` (the `dockhand` name is taken); the installed command and import package are still `dockhand`. Version lives in `pyproject.toml`. Publishing a GitHub release triggers `.github/workflows/publish.yml`, which runs `uv build && uv publish` via PyPI trusted publishing (OIDC, no API token). Pushing to `main` alone publishes nothing.
|
|
47
|
+
```bash
|
|
48
|
+
uv build # local sanity check; artifacts land in dist/
|
|
49
|
+
```
|
|
50
|
+
|
|
45
51
|
## Architecture
|
|
46
52
|
|
|
47
53
|
### Entry Point
|
|
@@ -52,6 +58,7 @@ uv run ruff check --fix .
|
|
|
52
58
|
- `dockhand run` — queue/run from an already-built image, no sync
|
|
53
59
|
- `dockhand install` — build the image only
|
|
54
60
|
- `dockhand logs`, `stop`, `remove`, `jobs`, `urgent`, `prune` — job/queue lifecycle (`manage.py` + `queue.py`)
|
|
61
|
+
- `jobs` shows this project's last 30 active jobs; `--all` lifts the cap and includes finished/failed/stopped; `--queue` shows the host's whole task-spooler queue, including other projects' jobs
|
|
55
62
|
- `dockhand history` — show past runs
|
|
56
63
|
- `dockhand volumes`, `download` — inspect and pull files from mounted volumes
|
|
57
64
|
- `dockhand resubmit` — rerun a past job with optional overrides
|
|
@@ -76,10 +83,10 @@ Config file: `.dockhand.json` in the project root (must also contain a `.git`
|
|
|
76
83
|
### Key Modules
|
|
77
84
|
- `submit.py` — Builds the `docker run` command (code mount vs. data volumes vs. GPU/port flags), resolves code delivery, and hands off to a transport to start the job.
|
|
78
85
|
- `build.py` — Runs `docker build` on the client (optionally syncing first).
|
|
79
|
-
- `manage.py` — Job lifecycle: `logs`, `stop`, `remove`, `jobs` (`execute_stats`), and `prune` (removes baked images no longer referenced by an active job).
|
|
86
|
+
- `manage.py` — Job lifecycle: `logs`, `stop`, `remove`, `jobs` (`execute_stats`), `jobs --queue` (`execute_queue`), and `prune` (removes baked images no longer referenced by an active job). Also owns job timing (`_observe_job_time`, `_docker_started_at`, `_host_now`).
|
|
80
87
|
- `resubmit.py` — Looks up a history entry and re-invokes `execute_submit` with overrides; pins to the original baked image when applicable.
|
|
81
|
-
- `queue.py` — Task spooler (`tsp`) integration: submit/list/promote/remove/kill,
|
|
82
|
-
- `transport.py` — Abstracts "how a job runs": `TaskSpoolerTransport` (queue enabled) vs. `DockerTransport` (direct `docker run -d`). Both expose the same interface (`submit`, `list_jobs`, `logs`, `stop`, `remove`) so job-management commands don't care which backend created a job; the transport used is recorded per job in history.
|
|
88
|
+
- `queue.py` — Task spooler (`tsp`) integration: submit/list/promote/remove/kill, `tsp -l` output parsing, max slot count (`ts_max_slots`), and start times from `tsp -i` (`ts_start_times`).
|
|
89
|
+
- `transport.py` — Abstracts "how a job runs": `TaskSpoolerTransport` (queue enabled) vs. `DockerTransport` (direct `docker run -d`). Both expose the same interface (`run_flags`, `submit`, `list_jobs`, `container_name`, `logs`, `stop`, `remove`) so job-management commands don't care which backend created a job; the transport used is recorded per job in history. Containers are named `dockhand-<local_id>` in both modes (queued: `--rm --name`; direct: `-d --name`, no `--rm` so logs survive exit).
|
|
83
90
|
- `tagging.py` — Resolves the image tag/ref for baked code delivery (content-addressed from git commit + dirty-state hash; unique per queued submit, reused for direct runs).
|
|
84
91
|
- `history.py` — Reads/writes `.dockhand_history.json`, reserves/looks up local job IDs.
|
|
85
92
|
- `volumes.py` — Lists the container filesystem as a tree (code mount + data volumes) and resolves a workdir-relative path back to its host path.
|
|
@@ -111,28 +118,36 @@ Stores container runs in `.dockhand_history.json` as JSON. Each entry contains:
|
|
|
111
118
|
"handle": 12, // tsp job id, or container name for direct runs
|
|
112
119
|
"ts_job_id": 12, // task_spooler transport only
|
|
113
120
|
"host": "remote.example.com", // or "localhost"
|
|
114
|
-
"started_at": 1234567891.0, // optional, set
|
|
115
|
-
"ended_at": 1234567895.0
|
|
121
|
+
"started_at": 1234567891.0, // optional, set when `jobs`/`logs` first sees it running
|
|
122
|
+
"ended_at": 1234567895.0, // optional, set when `jobs`/`logs` first sees it finished/failed/stopped
|
|
123
|
+
"stopped": true // optional, set by `dockhand stop`; shows finished/failed as "stopped"
|
|
116
124
|
}
|
|
117
125
|
```
|
|
118
126
|
|
|
119
127
|
Used by `resubmit` (look up a previous run and re-run with overrides, pinning the original baked image when unchanged), `logs`, `stop`, `remove`, `jobs`, `urgent`, `prune` (default to latest if no ID given). Job-management commands dispatch to the transport recorded on the entry (`transport_for_entry`), so `logs`/`stop`/`remove` work the same regardless of whether a job went through the queue or ran directly.
|
|
120
128
|
|
|
121
|
-
`started_at`/`ended_at`
|
|
129
|
+
`started_at`/`ended_at` are written the first time `jobs` or `logs` observes a job in the running/terminal state (`_observe_job_time`), but the values come from docker/ts, not the observation time:
|
|
130
|
+
- `started_at` — the container's real `docker inspect .State.StartedAt` (found via the `dockhand-<local_id>` name). Falls back to "now" only if the container can't be inspected (pre-naming jobs, already removed).
|
|
131
|
+
- `ended_at` — `started_at + duration_seconds` from ts's own elapsed-time field when available, else "now".
|
|
132
|
+
- Running durations are measured against the host clock (`_host_now` runs `date +%s` on the host) so local/host clock skew doesn't distort them.
|
|
133
|
+
|
|
134
|
+
`dockhand jobs` shows Started/Ended (absolute, `%Y-%m-%d %H:%M:%S`) and Duration columns, grouped running → queued → finished/failed/stopped, newest ID first within each group. `dockhand logs` prints a one-line "running for Xm" / "finished in Xm" header before the log output.
|
|
135
|
+
|
|
136
|
+
`dockhand jobs --queue` bypasses history entirely: it prints the raw host-wide `tsp -l` queue (ts is one shared queue per host), with each job's dockhand ID parsed from its `dockhand-<n>` container name, a project column from the image name, start times from `tsp -i`, and a "N running / M slots, K queued" footer.
|
|
122
137
|
|
|
123
138
|
### Design Patterns
|
|
124
139
|
|
|
125
140
|
**DockerDefault factory:**
|
|
126
|
-
In `__init__.py`, `DockerDefault` is a callable factory that pulls defaults from `cli_config.docker` at call time
|
|
141
|
+
In `__init__.py`, `DockerDefault` is a callable factory that pulls defaults from `cli_config.docker` at call time (after the `--profile` callback has run), for use with Typer's `default_factory`. `SyncDefault` does the same for `cli_config.sync`.
|
|
127
142
|
|
|
128
143
|
**Transport abstraction:**
|
|
129
|
-
`get_transport()` in `transport.py` picks `TaskSpoolerTransport` or `DockerTransport` based on `cli_config.queue.enabled` at submit time.
|
|
144
|
+
`get_transport()` in `transport.py` picks `TaskSpoolerTransport` or `DockerTransport` based on `cli_config.queue.enabled` at submit time. Per-job commands (`logs`, `stop`, `remove`, `urgent`) re-derive the transport from the history entry (`transport_for_entry`; entries without a `transport` field are treated as task spooler), and `logs`/`stop`/`remove` also connect to the entry's recorded `host`. The exception is `jobs`, which lists via the *current* config's transport and host.
|
|
130
145
|
|
|
131
146
|
**Code delivery resolution:**
|
|
132
147
|
`DockerConfig.resolve_code_delivery(queue_enabled)` picks `mount` or `bake` when `code_delivery` isn't explicitly set: `bake` when the queue is enabled (avoids code drift while a job waits in queue), `mount` otherwise (zero-rebuild iteration). See README's [Code delivery](README.md#code-delivery-mount-vs-bake) section for the full rationale.
|
|
133
148
|
|
|
134
149
|
**Profile support:**
|
|
135
|
-
`cli_config.load_profile(name)` loads overrides from `.dockhand.json` profiles section
|
|
150
|
+
`cli_config.load_profile(name)` loads overrides from `.dockhand.json` profiles section. It only merges `history_path`, `remote_path` and `ssh` — `docker`, `queue` and `sync` keys in a profile are silently ignored. `SSHConfig.validate()` is used for safe merging of partial ssh overrides.
|
|
136
151
|
|
|
137
152
|
**Command structure:**
|
|
138
153
|
Each command:
|
|
@@ -155,7 +170,7 @@ Each command:
|
|
|
155
170
|
| `__init__.py` | CLI app definition, command routing, config defaults |
|
|
156
171
|
| `submit.py` | Builds the `docker run` command, resolves code delivery, submits via transport |
|
|
157
172
|
| `build.py` | `docker build` execution |
|
|
158
|
-
| `manage.py` | `logs`, `stop`, `remove`, `jobs`, `prune
|
|
173
|
+
| `manage.py` | `logs`, `stop`, `remove`, `jobs`, `jobs --queue`, `prune`, job timing |
|
|
159
174
|
| `resubmit.py` | Rerun a past job with overrides |
|
|
160
175
|
| `queue.py` | Task spooler (`tsp`) integration |
|
|
161
176
|
| `transport.py` | Task-spooler vs. direct-docker execution backends |
|
|
@@ -172,6 +187,7 @@ Each command:
|
|
|
172
187
|
| `sync.py` | rsync-based code synchronization |
|
|
173
188
|
| `error.py` | Error reporting |
|
|
174
189
|
| `constants.py` | Config/history file names |
|
|
190
|
+
| `.github/workflows/publish.yml` | Publish `dockhand-cli` to PyPI on GitHub release |
|
|
175
191
|
|
|
176
192
|
## Configuration Format
|
|
177
193
|
|
|
@@ -207,11 +223,9 @@ Each command:
|
|
|
207
223
|
},
|
|
208
224
|
"remote_path": "~/my-project",
|
|
209
225
|
"profiles": {
|
|
210
|
-
"
|
|
211
|
-
"
|
|
212
|
-
|
|
213
|
-
"dockerfile": "Dockerfile.dev"
|
|
214
|
-
}
|
|
226
|
+
"gpu2": {
|
|
227
|
+
"ssh": { "hostname": "gpu2.example.com" },
|
|
228
|
+
"remote_path": "~/my-project-gpu2"
|
|
215
229
|
}
|
|
216
230
|
}
|
|
217
231
|
}
|
|
@@ -242,13 +256,14 @@ uv run dockhand run 'bash' # Run interactively
|
|
|
242
256
|
|
|
243
257
|
**Using profiles:**
|
|
244
258
|
```bash
|
|
245
|
-
uv run dockhand --profile
|
|
246
|
-
uv run dockhand --profile
|
|
259
|
+
uv run dockhand --profile gpu2 install # Build on the gpu2 host
|
|
260
|
+
uv run dockhand --profile gpu2 submit 'python train.py' # Run there
|
|
247
261
|
```
|
|
248
262
|
|
|
249
263
|
**Queue management:**
|
|
250
264
|
```bash
|
|
251
265
|
uv run dockhand jobs # List active jobs
|
|
266
|
+
uv run dockhand jobs --queue # Whole host queue, all projects, with slot usage
|
|
252
267
|
uv run dockhand urgent 5 # Promote job #5 to front of queue
|
|
253
268
|
uv run dockhand prune --dry-run # Preview unused baked images before removing
|
|
254
269
|
```
|
|
@@ -269,7 +284,7 @@ uv run ruff format --check . # Format check
|
|
|
269
284
|
2. **SSH by default** — Assumes docker host is remote. Auto-detects local by resolving the configured hostname to a loopback address.
|
|
270
285
|
3. **History in JSON** — Simple, human-readable, easily editable.
|
|
271
286
|
4. **Profile support** — Allows per-project config variants.
|
|
272
|
-
5. **Minimal dependencies** — typer, fabric, paramiko, gitpython
|
|
287
|
+
5. **Minimal dependencies** — typer, fabric, paramiko, gitpython (rich comes in via typer).
|
|
273
288
|
6. **Transport abstraction over queue state** — job-management commands read the transport from each job's own history entry rather than current `queue.enabled`, so toggling the queue mid-flight doesn't orphan existing jobs.
|
|
274
289
|
7. **Code-delivery default follows queue mode** — `bake` when queued (avoids code drift while waiting), `mount` when not (zero-rebuild iteration); explicit `code_delivery` overrides either way.
|
|
275
290
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: dockhand-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.4.0
|
|
4
4
|
Summary: CLI for managing Docker containers on remote machines
|
|
5
5
|
Project-URL: Repository, https://github.com/NicholasPHansen/dockhand
|
|
6
6
|
Author: Nicholas P. Hansen
|
|
@@ -108,7 +108,7 @@ All commands work with the same `.dockhand.json` configuration file. See [Config
|
|
|
108
108
|
| `submit` | Build the image and queue a container run |
|
|
109
109
|
| `run` | Queue a container run from an already-built image |
|
|
110
110
|
| `install` | Build the Docker image without running it |
|
|
111
|
-
| `jobs` | List active (running/queued) jobs, with start/end/duration — use `--all` for finished jobs too |
|
|
111
|
+
| `jobs` | List active (running/queued) jobs, with start/end/duration — use `--all` for finished jobs too, or `--queue` for the host's whole task-spooler queue across all projects |
|
|
112
112
|
| `logs` | Show logs from a job, preceded by a running/finished duration line — use `--follow`/`-f` to stream live |
|
|
113
113
|
| `stop` | Stop a **running** job |
|
|
114
114
|
| `remove` | Remove a **queued** job before it starts |
|
|
@@ -164,6 +164,9 @@ dockhand jobs
|
|
|
164
164
|
# Check all jobs including finished
|
|
165
165
|
dockhand jobs --all
|
|
166
166
|
|
|
167
|
+
# Show the host's whole task-spooler queue, including other projects' jobs
|
|
168
|
+
dockhand jobs --queue
|
|
169
|
+
|
|
167
170
|
# Stream logs from the last job
|
|
168
171
|
dockhand logs --follow
|
|
169
172
|
|
|
@@ -198,10 +201,23 @@ dockhand resubmit --gpus 2
|
|
|
198
201
|
**Note:** If you've installed dockhand globally, you can omit `uv run`.
|
|
199
202
|
|
|
200
203
|
`jobs` shows Started/Ended/Duration for each job, and `logs` prints a one-line
|
|
201
|
-
`running for 5m30s` / `finished in 12m45s` header before the log output.
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
204
|
+
`running for 5m30s` / `finished in 12m45s` header before the log output. Start times come
|
|
205
|
+
from the container's `docker inspect` and finished durations from task spooler itself, so
|
|
206
|
+
they're exact no matter when you check (older jobs from before container naming fall back
|
|
207
|
+
to the time they were first observed). A running job's elapsed time is measured against the
|
|
208
|
+
host's clock, so clock skew between your machine and the host doesn't distort it (keep the
|
|
209
|
+
host's clock NTP-synced, though — Started/Ended are shown in the host's time).
|
|
210
|
+
|
|
211
|
+
`jobs --queue` lists every job in the host's task-spooler queue, not just this project's:
|
|
212
|
+
|
|
213
|
+
| Column | Meaning |
|
|
214
|
+
|--------|---------|
|
|
215
|
+
| `ID` | dockhand job ID in the submitting project — what `logs`/`stop`/`urgent` take there (`-` for jobs submitted before dockhand named its containers) |
|
|
216
|
+
| `Project` | Image the job runs (the project's `imagename`) |
|
|
217
|
+
| `Status` / `Started` / `Duration` | From task spooler (`tsp -l`, `tsp -i`), measured on the host's clock |
|
|
218
|
+
| `TS ID` | Task spooler's own job ID, for use with `tsp` directly |
|
|
219
|
+
|
|
220
|
+
Running and queued jobs come first (queued in queue order), then the rest newest-started first.
|
|
205
221
|
|
|
206
222
|
Resubmitting a job that ran in [`bake`](#code-delivery-mount-vs-bake) mode reruns the exact
|
|
207
223
|
image it originally built — recorded per job — rather than rebuilding from current code, so a
|
|
@@ -91,7 +91,7 @@ All commands work with the same `.dockhand.json` configuration file. See [Config
|
|
|
91
91
|
| `submit` | Build the image and queue a container run |
|
|
92
92
|
| `run` | Queue a container run from an already-built image |
|
|
93
93
|
| `install` | Build the Docker image without running it |
|
|
94
|
-
| `jobs` | List active (running/queued) jobs, with start/end/duration — use `--all` for finished jobs too |
|
|
94
|
+
| `jobs` | List active (running/queued) jobs, with start/end/duration — use `--all` for finished jobs too, or `--queue` for the host's whole task-spooler queue across all projects |
|
|
95
95
|
| `logs` | Show logs from a job, preceded by a running/finished duration line — use `--follow`/`-f` to stream live |
|
|
96
96
|
| `stop` | Stop a **running** job |
|
|
97
97
|
| `remove` | Remove a **queued** job before it starts |
|
|
@@ -147,6 +147,9 @@ dockhand jobs
|
|
|
147
147
|
# Check all jobs including finished
|
|
148
148
|
dockhand jobs --all
|
|
149
149
|
|
|
150
|
+
# Show the host's whole task-spooler queue, including other projects' jobs
|
|
151
|
+
dockhand jobs --queue
|
|
152
|
+
|
|
150
153
|
# Stream logs from the last job
|
|
151
154
|
dockhand logs --follow
|
|
152
155
|
|
|
@@ -181,10 +184,23 @@ dockhand resubmit --gpus 2
|
|
|
181
184
|
**Note:** If you've installed dockhand globally, you can omit `uv run`.
|
|
182
185
|
|
|
183
186
|
`jobs` shows Started/Ended/Duration for each job, and `logs` prints a one-line
|
|
184
|
-
`running for 5m30s` / `finished in 12m45s` header before the log output.
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
187
|
+
`running for 5m30s` / `finished in 12m45s` header before the log output. Start times come
|
|
188
|
+
from the container's `docker inspect` and finished durations from task spooler itself, so
|
|
189
|
+
they're exact no matter when you check (older jobs from before container naming fall back
|
|
190
|
+
to the time they were first observed). A running job's elapsed time is measured against the
|
|
191
|
+
host's clock, so clock skew between your machine and the host doesn't distort it (keep the
|
|
192
|
+
host's clock NTP-synced, though — Started/Ended are shown in the host's time).
|
|
193
|
+
|
|
194
|
+
`jobs --queue` lists every job in the host's task-spooler queue, not just this project's:
|
|
195
|
+
|
|
196
|
+
| Column | Meaning |
|
|
197
|
+
|--------|---------|
|
|
198
|
+
| `ID` | dockhand job ID in the submitting project — what `logs`/`stop`/`urgent` take there (`-` for jobs submitted before dockhand named its containers) |
|
|
199
|
+
| `Project` | Image the job runs (the project's `imagename`) |
|
|
200
|
+
| `Status` / `Started` / `Duration` | From task spooler (`tsp -l`, `tsp -i`), measured on the host's clock |
|
|
201
|
+
| `TS ID` | Task spooler's own job ID, for use with `tsp` directly |
|
|
202
|
+
|
|
203
|
+
Running and queued jobs come first (queued in queue order), then the rest newest-started first.
|
|
188
204
|
|
|
189
205
|
Resubmitting a job that ran in [`bake`](#code-delivery-mount-vs-bake) mode reruns the exact
|
|
190
206
|
image it originally built — recorded per job — rather than rebuilding from current code, so a
|
dockhand_cli-0.3.1/CLAUDE.md
DELETED
|
@@ -1,309 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md — dockhand Development Guide
|
|
2
|
-
|
|
3
|
-
## Project Overview
|
|
4
|
-
|
|
5
|
-
**dockhand** is a standalone CLI tool, originally extracted from DTU-HPC-CLI, for managing Docker containers on remote machines (or locally). It provides a unified interface to build, run, queue, manage, and monitor Docker containers via SSH or locally.
|
|
6
|
-
|
|
7
|
-
**Key philosophy:** Simple, flat command structure with sensible SSH defaults. One `.dockhand.json` config file works everywhere.
|
|
8
|
-
|
|
9
|
-
Since extraction, dockhand has grown features DTU-HPC-CLI never had: a task-spooler-based job queue, a mount-vs-bake code delivery mode, a direct (non-queued) run transport, and baked-image pruning. See [Relationship to DTU-HPC-CLI](#relationship-to-dtu-hpc-cli).
|
|
10
|
-
|
|
11
|
-
## Commands
|
|
12
|
-
|
|
13
|
-
**Prerequisites:**
|
|
14
|
-
- Install [uv](https://docs.astral.sh/uv/): `curl -LsSf https://astral.sh/uv/install.sh | sh`
|
|
15
|
-
|
|
16
|
-
**Installation & Setup:**
|
|
17
|
-
```bash
|
|
18
|
-
# Install dependencies and create lock file
|
|
19
|
-
uv sync
|
|
20
|
-
|
|
21
|
-
# Update lock file with latest versions (if needed)
|
|
22
|
-
uv lock --upgrade
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
**Run the CLI locally:**
|
|
26
|
-
```bash
|
|
27
|
-
uv run dockhand --help
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
**Lint:**
|
|
31
|
-
```bash
|
|
32
|
-
uv run ruff check .
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
**Format:**
|
|
36
|
-
```bash
|
|
37
|
-
uv run ruff format .
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
**Auto-fix lint issues:**
|
|
41
|
-
```bash
|
|
42
|
-
uv run ruff check --fix .
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
## Architecture
|
|
46
|
-
|
|
47
|
-
### Entry Point
|
|
48
|
-
`dockhand/__init__.py` — Defines the Typer CLI app. All commands are at the top level (flat structure, not nested under a `docker` sub-group). Each command validates config with `cli_config.check_docker()` and delegates to an `execute_*` function in the relevant module below.
|
|
49
|
-
|
|
50
|
-
**Commands at top level:**
|
|
51
|
-
- `dockhand submit` — sync code, then queue/run a container
|
|
52
|
-
- `dockhand run` — queue/run from an already-built image, no sync
|
|
53
|
-
- `dockhand install` — build the image only
|
|
54
|
-
- `dockhand logs`, `stop`, `remove`, `jobs`, `urgent`, `prune` — job/queue lifecycle (`manage.py` + `queue.py`)
|
|
55
|
-
- `dockhand history` — show past runs
|
|
56
|
-
- `dockhand volumes`, `download` — inspect and pull files from mounted volumes
|
|
57
|
-
- `dockhand resubmit` — rerun a past job with optional overrides
|
|
58
|
-
- `dockhand tunnel` — SSH-forward container ports to localhost
|
|
59
|
-
|
|
60
|
-
### Configuration System
|
|
61
|
-
`dockhand/config.py` — The `CLIConfig` class is loaded at module import time (`cli_config = CLIConfig.load()`) by walking up the directory tree to find `.dockhand.json`. Contains:
|
|
62
|
-
- `SSHConfig` — hostname, user, identity file for SSH connections
|
|
63
|
-
- `DockerConfig` — dockerfile, imagename, volumes, ports, gpus, containerworkdir, preserve_paths, code_delivery
|
|
64
|
-
- `QueueConfig` — enabled, tool, slots (task-spooler queue settings)
|
|
65
|
-
- `CLIConfig` — top-level config holder (ssh, docker, queue, sync, remote_path, profiles)
|
|
66
|
-
|
|
67
|
-
Config file: `.dockhand.json` in the project root (must also contain a `.git` — `CLIConfig.load()` errors if no git repo is found). History file: `.dockhand_history.json` (same location by default, overridable via `history_path`).
|
|
68
|
-
|
|
69
|
-
### Client Abstraction
|
|
70
|
-
`dockhand/client/` — `Client` (abstract base) has two implementations:
|
|
71
|
-
- `SSHClient` — connects via Fabric/Paramiko over SSH
|
|
72
|
-
- `LocalClient` — runs commands locally
|
|
73
|
-
|
|
74
|
-
`get_client()` picks `LocalClient` when `cli_config.ssh` is unset or the configured hostname resolves to loopback, otherwise `SSHClient`. `get_client_for_host(hostname)` does the same but for an explicit hostname (used by `logs`/`stop`/`remove`, which read the host from the job's history entry rather than current config).
|
|
75
|
-
|
|
76
|
-
### Key Modules
|
|
77
|
-
- `submit.py` — Builds the `docker run` command (code mount vs. data volumes vs. GPU/port flags), resolves code delivery, and hands off to a transport to start the job.
|
|
78
|
-
- `build.py` — Runs `docker build` on the client (optionally syncing first).
|
|
79
|
-
- `manage.py` — Job lifecycle: `logs`, `stop`, `remove`, `jobs` (`execute_stats`), and `prune` (removes baked images no longer referenced by an active job).
|
|
80
|
-
- `resubmit.py` — Looks up a history entry and re-invokes `execute_submit` with overrides; pins to the original baked image when applicable.
|
|
81
|
-
- `queue.py` — Task spooler (`tsp`) integration: submit/list/promote/remove/kill, plus `ts -l` output parsing.
|
|
82
|
-
- `transport.py` — Abstracts "how a job runs": `TaskSpoolerTransport` (queue enabled) vs. `DockerTransport` (direct `docker run -d`). Both expose the same interface (`submit`, `list_jobs`, `logs`, `stop`, `remove`) so job-management commands don't care which backend created a job; the transport used is recorded per job in history.
|
|
83
|
-
- `tagging.py` — Resolves the image tag/ref for baked code delivery (content-addressed from git commit + dirty-state hash; unique per queued submit, reused for direct runs).
|
|
84
|
-
- `history.py` — Reads/writes `.dockhand_history.json`, reserves/looks up local job IDs.
|
|
85
|
-
- `volumes.py` — Lists the container filesystem as a tree (code mount + data volumes) and resolves a workdir-relative path back to its host path.
|
|
86
|
-
- `download.py` — Uses `volumes._resolve_to_host` + rsync to pull a file/directory from a volume.
|
|
87
|
-
- `tunnel.py` — SSH local port forwarding to container ports (via Fabric).
|
|
88
|
-
- `sync.py` — Uses `rsync` over SSH to copy local files to `remote_path`, respecting `.gitignore`; prompts to confirm when the worktree is dirty.
|
|
89
|
-
- `config.py` — Configuration loading and validation.
|
|
90
|
-
- `error.py` — Centralized error reporting with rich panels.
|
|
91
|
-
- `constants.py` — Config/history filenames.
|
|
92
|
-
|
|
93
|
-
> **Known inconsistency:** `tunnel.py` still looks up history entries by a `container_id` field (`entry["container_id"]`), but `history.py` no longer writes that field — entries are keyed by `local_id` with a transport `handle`. Passing an explicit `container_id` to `dockhand tunnel` will not resolve; only the no-argument (last job) path works reliably.
|
|
94
|
-
|
|
95
|
-
### Docker History
|
|
96
|
-
Stores container runs in `.dockhand_history.json` as JSON. Each entry contains:
|
|
97
|
-
```json
|
|
98
|
-
{
|
|
99
|
-
"local_id": 7,
|
|
100
|
-
"timestamp": 1234567890.123,
|
|
101
|
-
"config": {
|
|
102
|
-
"gpus": "all",
|
|
103
|
-
"volumes": [...],
|
|
104
|
-
"imagename": "my-image",
|
|
105
|
-
"commands": ["python", "train.py"],
|
|
106
|
-
"ports": ["6006:6006"],
|
|
107
|
-
"image_ref": "my-image:abc123def456", // present when built (baked delivery)
|
|
108
|
-
"branch": "main" // optional, detected from git
|
|
109
|
-
},
|
|
110
|
-
"transport": "task_spooler", // or "docker"
|
|
111
|
-
"handle": 12, // tsp job id, or container name for direct runs
|
|
112
|
-
"ts_job_id": 12, // task_spooler transport only
|
|
113
|
-
"host": "remote.example.com", // or "localhost"
|
|
114
|
-
"started_at": 1234567891.0, // optional, set the first time `jobs` observes it running
|
|
115
|
-
"ended_at": 1234567895.0 // optional, set the first time `jobs` observes it finished/failed/stopped
|
|
116
|
-
}
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
Used by `resubmit` (look up a previous run and re-run with overrides, pinning the original baked image when unchanged), `logs`, `stop`, `remove`, `jobs`, `urgent`, `prune` (default to latest if no ID given). Job-management commands dispatch to the transport recorded on the entry (`transport_for_entry`), so `logs`/`stop`/`remove` work the same regardless of whether a job went through the queue or ran directly.
|
|
120
|
-
|
|
121
|
-
`started_at`/`ended_at` aren't queried from tsp/docker (neither exposes exact start/end timestamps cheaply for both transports) — `jobs` and `logs` each stamp them lazily the first time they happen to observe a job in the running/terminal state, so a job never checked on while running will show no start time once it finishes. `dockhand jobs` displays these as Started/Ended (absolute, `%Y-%m-%d %H:%M:%S`) and Duration (elapsed while running, total once finished) columns; `dockhand logs` prints a one-line "running for Xm" / "finished in Xm" header before the log output.
|
|
122
|
-
|
|
123
|
-
### Design Patterns
|
|
124
|
-
|
|
125
|
-
**DockerDefault factory:**
|
|
126
|
-
In `__init__.py`, `DockerDefault` is a callable factory that pulls defaults from `cli_config.docker` at call time. Enables Typer's `default_factory` to respect active profiles.
|
|
127
|
-
|
|
128
|
-
**Transport abstraction:**
|
|
129
|
-
`get_transport()` in `transport.py` picks `TaskSpoolerTransport` or `DockerTransport` based on `cli_config.queue.enabled` at submit time. Every job-management command re-derives the correct transport per-job from the history entry (`transport_for_entry`) rather than from current config, so switching `queue.enabled` doesn't strand in-flight jobs from the other mode.
|
|
130
|
-
|
|
131
|
-
**Code delivery resolution:**
|
|
132
|
-
`DockerConfig.resolve_code_delivery(queue_enabled)` picks `mount` or `bake` when `code_delivery` isn't explicitly set: `bake` when the queue is enabled (avoids code drift while a job waits in queue), `mount` otherwise (zero-rebuild iteration). See README's [Code delivery](README.md#code-delivery-mount-vs-bake) section for the full rationale.
|
|
133
|
-
|
|
134
|
-
**Profile support:**
|
|
135
|
-
`cli_config.load_profile(name)` loads overrides from `.dockhand.json` profiles section (currently merges `history_path`, `remote_path`, `ssh`). All config classes support `validate()` for safe merging.
|
|
136
|
-
|
|
137
|
-
**Command structure:**
|
|
138
|
-
Each command:
|
|
139
|
-
1. Calls `cli_config.check_docker()` to validate config
|
|
140
|
-
2. Calls an `execute_*` function from the relevant module
|
|
141
|
-
3. Passes `cli_config.docker` (and often `cli_config.queue`) as config arguments
|
|
142
|
-
|
|
143
|
-
## Relationship to DTU-HPC-CLI
|
|
144
|
-
|
|
145
|
-
- **Flat commands** — no `docker` sub-group. Commands are top-level.
|
|
146
|
-
- **No HPC support** — SubmitConfig, InstallConfig, and all LSF/job submission code removed
|
|
147
|
-
- **Docker-focused** — all UI/UX optimized for docker workflows
|
|
148
|
-
- **Config-compatible in spirit, not in filename** — dockhand originally reused DTU-HPC-CLI's `.dtu_hpc.json`/`.dtu_docker_history.json` names; both have since been renamed to `.dockhand.json`/`.dockhand_history.json` (see `constants.py`). Old DTU-HPC-CLI config files need renaming (or a `history_path` override) to work with dockhand.
|
|
149
|
-
- **Diverged, not just trimmed** — dockhand has added its own features DTU-HPC-CLI doesn't have: a task-spooler queue (`queue.py`/`transport.py`), slot reservations, `mount`/`bake` code delivery (`tagging.py`), a direct-run transport for when the queue is off, `urgent`/`prune`/`tunnel` commands. Maintained independently; version/release cadence is separate from DTU-HPC-CLI.
|
|
150
|
-
|
|
151
|
-
## Key Files and Their Responsibilities
|
|
152
|
-
|
|
153
|
-
| File | Purpose |
|
|
154
|
-
|------|---------|
|
|
155
|
-
| `__init__.py` | CLI app definition, command routing, config defaults |
|
|
156
|
-
| `submit.py` | Builds the `docker run` command, resolves code delivery, submits via transport |
|
|
157
|
-
| `build.py` | `docker build` execution |
|
|
158
|
-
| `manage.py` | `logs`, `stop`, `remove`, `jobs`, `prune` |
|
|
159
|
-
| `resubmit.py` | Rerun a past job with overrides |
|
|
160
|
-
| `queue.py` | Task spooler (`tsp`) integration |
|
|
161
|
-
| `transport.py` | Task-spooler vs. direct-docker execution backends |
|
|
162
|
-
| `tagging.py` | Image tag resolution for baked code delivery |
|
|
163
|
-
| `history.py` | Read/write `.dockhand_history.json`, job ID allocation |
|
|
164
|
-
| `volumes.py` | Container filesystem tree, path resolution |
|
|
165
|
-
| `download.py` | rsync-based file download from volumes |
|
|
166
|
-
| `tunnel.py` | SSH port forwarding to container ports |
|
|
167
|
-
| `config.py` | Config loading, validation, profile support |
|
|
168
|
-
| `client/__init__.py` | Auto-detect and return appropriate client |
|
|
169
|
-
| `client/base.py` | Abstract Client interface |
|
|
170
|
-
| `client/local.py` | Local command execution |
|
|
171
|
-
| `client/ssh.py` | Remote execution via SSH (Fabric/Paramiko) |
|
|
172
|
-
| `sync.py` | rsync-based code synchronization |
|
|
173
|
-
| `error.py` | Error reporting |
|
|
174
|
-
| `constants.py` | Config/history file names |
|
|
175
|
-
|
|
176
|
-
## Configuration Format
|
|
177
|
-
|
|
178
|
-
`.dockhand.json` in project root (project root is also where `CLIConfig.load()` expects to find a `.git` directory):
|
|
179
|
-
|
|
180
|
-
```json
|
|
181
|
-
{
|
|
182
|
-
"sync": true,
|
|
183
|
-
"ssh": {
|
|
184
|
-
"user": "your_username",
|
|
185
|
-
"identityfile": "~/.ssh/id_rsa",
|
|
186
|
-
"hostname": "remote.example.com"
|
|
187
|
-
},
|
|
188
|
-
"queue": {
|
|
189
|
-
"enabled": true,
|
|
190
|
-
"slots": 1
|
|
191
|
-
},
|
|
192
|
-
"docker": {
|
|
193
|
-
"dockerfile": "Dockerfile",
|
|
194
|
-
"imagename": "my-image",
|
|
195
|
-
"volumes": [
|
|
196
|
-
{
|
|
197
|
-
"hostpath": "/local/data",
|
|
198
|
-
"containerpath": "/data",
|
|
199
|
-
"permissions": "rw"
|
|
200
|
-
}
|
|
201
|
-
],
|
|
202
|
-
"ports": ["8080:80"],
|
|
203
|
-
"gpus": "all",
|
|
204
|
-
"containerworkdir": "/",
|
|
205
|
-
"preserve_paths": [".venv"],
|
|
206
|
-
"code_delivery": null
|
|
207
|
-
},
|
|
208
|
-
"remote_path": "~/my-project",
|
|
209
|
-
"profiles": {
|
|
210
|
-
"dev": {
|
|
211
|
-
"docker": {
|
|
212
|
-
"gpus": "1",
|
|
213
|
-
"dockerfile": "Dockerfile.dev"
|
|
214
|
-
}
|
|
215
|
-
}
|
|
216
|
-
}
|
|
217
|
-
}
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
All options are optional except `dockerfile`, `imagename`, and `volumes` within docker config. `queue.slots` may also be set as `docker.slots` for back-compat (deprecation warning printed). Full option tables and defaults are documented in README.md's [Configuration](README.md#configuration) section — keep both in sync when config shape changes.
|
|
221
|
-
|
|
222
|
-
## Common Workflows
|
|
223
|
-
|
|
224
|
-
**First time setting up:**
|
|
225
|
-
1. Create `.dockhand.json` with `docker` and `ssh` sections
|
|
226
|
-
2. Run `dockhand install` to build the image
|
|
227
|
-
3. Run `dockhand submit 'command'` to run a container
|
|
228
|
-
|
|
229
|
-
**Development iteration:**
|
|
230
|
-
```bash
|
|
231
|
-
uv run dockhand submit --gpus 1 'python train.py' # Sync + queue/run with 1 GPU
|
|
232
|
-
uv run dockhand logs --n 50 # Check last 50 log lines
|
|
233
|
-
uv run dockhand resubmit --gpus 2 # Re-run with 2 GPUs
|
|
234
|
-
uv run dockhand download results/model.pth # Get results back
|
|
235
|
-
```
|
|
236
|
-
|
|
237
|
-
**Quick rebuild:**
|
|
238
|
-
```bash
|
|
239
|
-
uv run dockhand install --dockerfile Dockerfile.dev # Rebuild only
|
|
240
|
-
uv run dockhand run 'bash' # Run interactively
|
|
241
|
-
```
|
|
242
|
-
|
|
243
|
-
**Using profiles:**
|
|
244
|
-
```bash
|
|
245
|
-
uv run dockhand --profile dev submit 'python train.py' # Use dev profile
|
|
246
|
-
uv run dockhand --profile prod install # Build prod image
|
|
247
|
-
```
|
|
248
|
-
|
|
249
|
-
**Queue management:**
|
|
250
|
-
```bash
|
|
251
|
-
uv run dockhand jobs # List active jobs
|
|
252
|
-
uv run dockhand urgent 5 # Promote job #5 to front of queue
|
|
253
|
-
uv run dockhand prune --dry-run # Preview unused baked images before removing
|
|
254
|
-
```
|
|
255
|
-
|
|
256
|
-
## Testing & Validation
|
|
257
|
-
|
|
258
|
-
No automated tests yet. Manual validation:
|
|
259
|
-
```bash
|
|
260
|
-
uv run dockhand --help # Check CLI structure
|
|
261
|
-
uv run dockhand submit --help # Check submit options
|
|
262
|
-
uv run ruff check . # Lint
|
|
263
|
-
uv run ruff format --check . # Format check
|
|
264
|
-
```
|
|
265
|
-
|
|
266
|
-
## Important Design Decisions
|
|
267
|
-
|
|
268
|
-
1. **Flat commands** — No `docker` sub-group. Simpler for a docker-only tool.
|
|
269
|
-
2. **SSH by default** — Assumes docker host is remote. Auto-detects local by resolving the configured hostname to a loopback address.
|
|
270
|
-
3. **History in JSON** — Simple, human-readable, easily editable.
|
|
271
|
-
4. **Profile support** — Allows per-project config variants.
|
|
272
|
-
5. **Minimal dependencies** — typer, fabric, paramiko, gitpython, rich.
|
|
273
|
-
6. **Transport abstraction over queue state** — job-management commands read the transport from each job's own history entry rather than current `queue.enabled`, so toggling the queue mid-flight doesn't orphan existing jobs.
|
|
274
|
-
7. **Code-delivery default follows queue mode** — `bake` when queued (avoids code drift while waiting), `mount` when not (zero-rebuild iteration); explicit `code_delivery` overrides either way.
|
|
275
|
-
|
|
276
|
-
## Extraction History
|
|
277
|
-
|
|
278
|
-
- **Source:** DTU-HPC-CLI (github.com/ChrisFugl/DTU-HPC-CLI)
|
|
279
|
-
- **Extracted:** 2026-04-09
|
|
280
|
-
- **Initial changes:**
|
|
281
|
-
- Removed: SubmitConfig, InstallConfig, types.py, all HPC modules
|
|
282
|
-
- Modified: config.py (trimmed), __init__.py (flat commands)
|
|
283
|
-
- Unchanged at the time: docker.py, sync.py, error.py, client/*, constants.py
|
|
284
|
-
- **Since extraction:** the original monolithic `docker.py` was split into `build.py`, `submit.py`, `manage.py`, `resubmit.py`, `history.py`, `download.py`, `volumes.py`, `tunnel.py`; `queue.py`, `transport.py`, and `tagging.py` were added for the task-spooler queue, transport abstraction, and baked code delivery. Config/history filenames moved from `.dtu_hpc.json`/`.dtu_docker_history.json` to `.dockhand.json`/`.dockhand_history.json`.
|
|
285
|
-
|
|
286
|
-
## Future Enhancements
|
|
287
|
-
|
|
288
|
-
Potential improvements specific to dockhand (volume inspection is already implemented — see `volumes.py`):
|
|
289
|
-
- Docker Compose support (`dockhand compose up`)
|
|
290
|
-
- Kubernetes pod management
|
|
291
|
-
- Local-only mode (drop SSH dependency for pure local docker)
|
|
292
|
-
- Config auto-generation wizard
|
|
293
|
-
- Container registry integration (Docker Hub, ECR, GCR)
|
|
294
|
-
- Multi-container orchestration helpers
|
|
295
|
-
- Fix `tunnel.py`'s stale `container_id` history lookup (see [Known inconsistency](#key-modules) above)
|
|
296
|
-
|
|
297
|
-
## Contributing
|
|
298
|
-
|
|
299
|
-
When working on dockhand:
|
|
300
|
-
1. Keep the flat command structure — don't re-introduce `docker` sub-groups
|
|
301
|
-
2. Maintain `.dockhand.json` compatibility
|
|
302
|
-
3. Test manually with `uv run dockhand <command> --help` and actual docker operations
|
|
303
|
-
4. Follow ruff lint/format rules (use `uv run ruff check . && uv run ruff format .`)
|
|
304
|
-
5. Document new features in README.md and CLAUDE.md
|
|
305
|
-
6. Update this file's Architecture section when modules are added, split, or renamed — it has drifted from actual code structure before
|
|
306
|
-
|
|
307
|
-
## Maintenance
|
|
308
|
-
|
|
309
|
-
dockhand is maintained independently from DTU-HPC-CLI. Version bumps and releases are separate. If you need HPC support, use the original DTU-HPC-CLI package.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|