dsh-ssh-compute 0.0.0-stage → 1.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +262 -2
- package/client.js +1058 -0
- package/cordis.patch.yml +17 -0
- package/icon.svg +7 -0
- package/index.js +578 -0
- package/lib/config.js +307 -0
- package/lib/glob.js +189 -0
- package/lib/guard.js +169 -0
- package/lib/keys.js +101 -0
- package/lib/remote.js +642 -0
- package/lib/schema.js +108 -0
- package/lib/shell.js +69 -0
- package/lib/ssh.js +312 -0
- package/lib/sshconfig.js +256 -0
- package/lib/transfer.js +336 -0
- package/locale/en.json +6 -0
- package/locale/zh.json +6 -0
- package/package.json +67 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 arcxya09
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,263 @@
|
|
|
1
|
-
#
|
|
1
|
+
# dsh-ssh-compute
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A DeepSeek Harness (DSH) plugin that hands the agent one remote machine over
|
|
4
|
+
SSH, and nothing else.
|
|
5
|
+
|
|
6
|
+
A job is a directory on the server. The plugin starts it, leaves it running, and
|
|
7
|
+
reports what is there when asked. It never decides what "compute-heavy" means,
|
|
8
|
+
never installs software, and never trusts its own memory over the machine's
|
|
9
|
+
files: the agent probes the machine (`nproc`, `free`, `nvidia-smi`) and decides
|
|
10
|
+
for itself, exactly as it would on a local shell.
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
you: build the toolchain on the big box and run the benchmark suite there
|
|
14
|
+
agent: [ssh-compute run] started box-20261115-031846-d197 on box (pid 21147)
|
|
15
|
+
[ssh-compute status] state: running
|
|
16
|
+
[ssh-compute logs] ... 99% Built target benchmark
|
|
17
|
+
[ssh-compute status] state: exited exit: 0
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Install
|
|
21
|
+
|
|
22
|
+
In the DSH GUI: **Plugins → Add plugin**, and enter a git spec.
|
|
23
|
+
|
|
24
|
+
```
|
|
25
|
+
github:arcxya09/dsh-ssh-compute
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
or the equivalent, if you prefer an explicit URL:
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
git+https://github.com/arcxya09/dsh-ssh-compute.git
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
The same spec works from the agent side:
|
|
35
|
+
|
|
36
|
+
```
|
|
37
|
+
plugin_manager action=install_bundle target="github:arcxya09/dsh-ssh-compute"
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Then open **Plugins → SSH compute**, add a server, and press **Test** on its row.
|
|
41
|
+
The tool is called `ssh-compute` and is registered process-wide by this bundle's
|
|
42
|
+
`cordis.patch.yml`, so no agent preset needs to mention it.
|
|
43
|
+
|
|
44
|
+
Nothing is built or installed to use it: the package is plain ESM, has no runtime
|
|
45
|
+
dependencies, and needs `ssh` on `PATH`.
|
|
46
|
+
|
|
47
|
+
> **Network:** a git install fetches from github.com, so it needs a route there.
|
|
48
|
+
> Behind a proxy, set `HTTPS_PROXY` for the DSH process or add
|
|
49
|
+
> `https-proxy=…` to the profile's `.npmrc`; otherwise clone the repository and
|
|
50
|
+
> install from the checkout path instead — that is exactly what a local
|
|
51
|
+
> development install does.
|
|
52
|
+
|
|
53
|
+
## The tool
|
|
54
|
+
|
|
55
|
+
One tool, `ssh-compute`, with an `action` that selects the verb:
|
|
56
|
+
|
|
57
|
+
| action | what it does |
|
|
58
|
+
| --- | --- |
|
|
59
|
+
| `run` | starts a command in the background on a server and returns a job id immediately |
|
|
60
|
+
| `wait` | blocks until the job ends or the timeout, then returns the exit code and the log tail |
|
|
61
|
+
| `status` | state, pid, start time, exit code, duration, peak memory and the last log lines of one job |
|
|
62
|
+
| `logs` | the job's output, either from a byte `offset` (incremental) or as the last N lines |
|
|
63
|
+
| `kill` | terminates the job's process group |
|
|
64
|
+
| `list` | the configured servers, or the jobs on one server |
|
|
65
|
+
| `upload` | copies local files or directories to the server |
|
|
66
|
+
| `download` | copies files from the server back |
|
|
67
|
+
|
|
68
|
+
`run` never waits for the command. A training run or a long build would otherwise
|
|
69
|
+
hold one tool call open for hours, and a cancelled call would lose the result.
|
|
70
|
+
|
|
71
|
+
**Prefer `wait` over polling.** It holds one SSH connection on the remote side
|
|
72
|
+
until the job writes its exit marker, so a five-minute job costs one round trip
|
|
73
|
+
instead of a hundred:
|
|
74
|
+
|
|
75
|
+
```
|
|
76
|
+
[ssh-compute run] started gpu01-20261115-054127-1fe1 on gpu01 (pid 292654)
|
|
77
|
+
[ssh-compute wait] state: exited exit: 0 waited: 96.4 s … last 40 lines
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The timeout is a ceiling, not a kill: when it elapses you get `state: still
|
|
81
|
+
running` and the tail so far, and the job keeps going. Use `status`/`logs` for a
|
|
82
|
+
cheap glance, and pass the `nextOffset` a `logs` answer returns to read only what
|
|
83
|
+
is new.
|
|
84
|
+
|
|
85
|
+
## Configure
|
|
86
|
+
|
|
87
|
+
Everything is on the plugin's own page: **Plugins → SSH compute**.
|
|
88
|
+
|
|
89
|
+
Each server has:
|
|
90
|
+
|
|
91
|
+
- `name`, the short name the tool's `server` argument uses, e.g. `box`. It becomes
|
|
92
|
+
a directory name and part of every job id, so it may only use letters, digits,
|
|
93
|
+
`.`, `-` and `_`; a name with a space or a shell character is refused rather than
|
|
94
|
+
silently mangled.
|
|
95
|
+
- `host`, `port`, `user`
|
|
96
|
+
- `keyPath`, a private key file. Leave it empty and the plugin tries the keys it
|
|
97
|
+
finds in `~/.ssh` (`id_ed25519`, `id_ecdsa`, `id_rsa`, …) in order of
|
|
98
|
+
preference; if there are none it passes no `-i` at all, so `~/.ssh/config` and
|
|
99
|
+
ssh-agent decide.
|
|
100
|
+
- `keyText`, paste key material instead. It is written to a `0600` file in the
|
|
101
|
+
system temp directory and removed when the plugin unloads.
|
|
102
|
+
- `extraArgs`, space-separated ssh arguments passed straight through, e.g.
|
|
103
|
+
`-J bastion` for a jump host or `-o UseKeychain=no`. They travel as argv, never
|
|
104
|
+
through a shell, so a value that needs its own spaces cannot be expressed here
|
|
105
|
+
(rsync transfers split the same string on spaces again).
|
|
106
|
+
|
|
107
|
+
Three settings bound what a job can do to the server:
|
|
108
|
+
|
|
109
|
+
- `maxFileSizeMb` (default 10240, `0` disables), applied by the job wrapper as
|
|
110
|
+
`ulimit -f`, so a run-away output dies on SIGXFSZ (exit 153) instead of filling
|
|
111
|
+
the disk. Core dumps are disabled unconditionally (`ulimit -c 0`).
|
|
112
|
+
- `minFreeSpaceMb` (default 1024): `run` refuses to start when the volume holding
|
|
113
|
+
the jobs is below it, naming the numbers.
|
|
114
|
+
- `waitTimeoutMs` (default 300000): the default ceiling for `wait`.
|
|
115
|
+
- `maxRunningJobs` (default 64, `0` disables): `run` refuses to start a job when
|
|
116
|
+
that many are already running under the same root, and names the count and the
|
|
117
|
+
limit. Counting happens inside the launcher, so the check costs no extra round
|
|
118
|
+
trip.
|
|
119
|
+
|
|
120
|
+
Connection reuse and job metadata:
|
|
121
|
+
|
|
122
|
+
- `reuseConnection` (default on) multiplexes calls over one ssh master; see below.
|
|
123
|
+
- `reuseIdleSeconds` (default 300): how long an idle master lingers.
|
|
124
|
+
- `recordResources` (default on): write `env.txt` and sample peak memory.
|
|
125
|
+
|
|
126
|
+
With exactly one server, `server` may be omitted. With several it is required,
|
|
127
|
+
except that `status`, `logs` and `kill` read it from the job id.
|
|
128
|
+
|
|
129
|
+
### Import from `~/.ssh/config`
|
|
130
|
+
|
|
131
|
+
**Import from ~/.ssh/config** reads this machine's ssh config and lists every
|
|
132
|
+
concrete `Host` alias with the address, port, user and identity file that ssh
|
|
133
|
+
itself would resolve. Tick the ones you want and press **Import selected**: they
|
|
134
|
+
are appended to the form, and nothing is written until you press **Save**.
|
|
135
|
+
|
|
136
|
+
Resolution follows ssh's own rules rather than appearance: the first value met
|
|
137
|
+
while walking the file wins, so a `Host *` block above a host supplies that
|
|
138
|
+
host's value too. Only plain aliases are offered; `Host *` and `Host *.corp`
|
|
139
|
+
patterns are skipped. `ProxyJump` is imported as `-J <host>` in the entry's extra
|
|
140
|
+
arguments. An entry whose block uses `ProxyCommand`, `LocalCommand` or
|
|
141
|
+
`RemoteCommand` is listed but not importable: a `ProxyCommand` is a shell command
|
|
142
|
+
line, and importing it would connect somewhere other than the user's own ssh
|
|
143
|
+
does. `Include` and `Match` are reported, not followed.
|
|
144
|
+
|
|
145
|
+
## Connection reuse
|
|
146
|
+
|
|
147
|
+
Every call opens its own ssh connection, which costs a full handshake. With
|
|
148
|
+
`reuseConnection` on, the first call creates an `ssh` master under the system
|
|
149
|
+
temp directory and later calls ride it. Measured against one Ubuntu 24.04 host:
|
|
150
|
+
**969 ms per call without, 199 ms with** (4.9×), because only the handshake
|
|
151
|
+
disappears — the remote command still runs.
|
|
152
|
+
|
|
153
|
+
This is off on Windows and cannot be turned on there: Windows OpenSSH accepts the
|
|
154
|
+
`ControlMaster` options and prints them in `ssh -G`, but setting a `ControlPath`
|
|
155
|
+
makes it fail with `getsockname failed: Not a socket` and drop the connection it
|
|
156
|
+
was opening (measured on `OpenSSH_for_Windows_9.5p2`). The plugin therefore never
|
|
157
|
+
emits those options on `win32`, rather than trusting what ssh says about itself.
|
|
158
|
+
|
|
159
|
+
## On the server
|
|
160
|
+
|
|
161
|
+
Every job lives in `<remoteRoot>/<jobId>/` (default `~/dsh-compute`):
|
|
162
|
+
|
|
163
|
+
```
|
|
164
|
+
cmd.sh the wrapper that ran; its #DSHCMD comment holds the original command
|
|
165
|
+
started epoch seconds, written before the process starts
|
|
166
|
+
pid the process-group leader
|
|
167
|
+
output.log combined stdout and stderr
|
|
168
|
+
exit the exit code, written when the command finishes
|
|
169
|
+
ended epoch seconds when it finished
|
|
170
|
+
duration seconds it ran, measured on the server
|
|
171
|
+
rss peak memory of the whole job tree, in KiB (Linux only)
|
|
172
|
+
env.txt hostname, user, cwd, kernel, start time and the python/gcc versions found
|
|
173
|
+
killed present when this plugin stopped the job
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
`env.txt` exists so a result can be attributed to a compiler and an OS later; turn
|
|
177
|
+
it off with `recordResources` if the extra probes bother you.
|
|
178
|
+
|
|
179
|
+
Nothing is cleaned up automatically. Delete old directories when you no longer
|
|
180
|
+
want them:
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
ssh box 'rm -rf ~/dsh-compute/box-20261115-031846-d197'
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
## Transfers
|
|
187
|
+
|
|
188
|
+
`upload` and `download` use `rsync` when both this machine and the server have
|
|
189
|
+
it, and otherwise stream a `tar` archive through the same SSH connection. The
|
|
190
|
+
fallback is the common case on Windows, where rsync is usually absent; it needs
|
|
191
|
+
only `ssh` and a `tar` (`tar.exe` ships with Windows).
|
|
192
|
+
|
|
193
|
+
Local patterns (`*`, `?`, `**`) are expanded by the plugin, because ssh is
|
|
194
|
+
spawned without a local shell. Remote patterns are expanded by the server.
|
|
195
|
+
Downloaded members are named relative to their own directory, so
|
|
196
|
+
`download sources=["~/work/run.out"]` produces `run.out`, not the whole path.
|
|
197
|
+
|
|
198
|
+
## Guard
|
|
199
|
+
|
|
200
|
+
Two refusals are on by default, and both are escapable with
|
|
201
|
+
`allowDangerous: true`, so the guard is a speed bump on accidents rather than a
|
|
202
|
+
permission system:
|
|
203
|
+
|
|
204
|
+
- `run` refuses commands that would destroy the machine — `rm -rf /`, `mkfs`,
|
|
205
|
+
`dd` onto a block device, a fork bomb, `shutdown`/`reboot`
|
|
206
|
+
- `upload` refuses to write into `/etc`, `/boot`, `/usr`, `/bin`, `/sbin`,
|
|
207
|
+
`/lib*`, `/dev`, `/proc`, `/sys`, `/var/lib`. `.` and `..` are resolved first,
|
|
208
|
+
so `/tmp/../etc/passwd` is refused like `/etc/passwd`.
|
|
209
|
+
|
|
210
|
+
Turn it off with the `guard` setting.
|
|
211
|
+
|
|
212
|
+
## Limitations
|
|
213
|
+
|
|
214
|
+
- **Key authentication only.** There is no password path: ssh runs with
|
|
215
|
+
`BatchMode=yes` so it can never stop on a password prompt. Use a key, or
|
|
216
|
+
`~/.ssh/config` with an agent.
|
|
217
|
+
- **No `ProxyCommand`.** `ProxyJump` and any other plain ssh option work through
|
|
218
|
+
`extraArgs`, but a `ProxyCommand` is a shell command line and this plugin has
|
|
219
|
+
no field for it; the import lists such an entry and refuses it instead of
|
|
220
|
+
connecting somewhere else.
|
|
221
|
+
- **Host keys are recorded, not verified.** ssh runs with
|
|
222
|
+
`StrictHostKeyChecking=accept-new`: an unknown host key is written to
|
|
223
|
+
`known_hosts` once, and a *changed* one is still refused.
|
|
224
|
+
- **No local job state.** Everything lives on the server, so a restarted DSH, a
|
|
225
|
+
second machine, or a colleague with the same key sees the same jobs.
|
|
226
|
+
- **Jobs outlive DSH.** That is the point, but it also means nothing stops a job
|
|
227
|
+
when you close the app; `kill` is the way to stop one.
|
|
228
|
+
|
|
229
|
+
## Where the time goes
|
|
230
|
+
|
|
231
|
+
Measured against one Ubuntu 24.04 host, medians of repeated calls. The numbers
|
|
232
|
+
that matter are the ones that were fixed, but the profile is here so nobody has
|
|
233
|
+
to guess where a slow call comes from.
|
|
234
|
+
|
|
235
|
+
| step | cost |
|
|
236
|
+
| --- | --- |
|
|
237
|
+
| one ssh round trip, direct | 870–940 ms |
|
|
238
|
+
| the same, with `reuseConnection` | 119 ms |
|
|
239
|
+
| the remote launcher (staging one job) | 45 ms, was 67 |
|
|
240
|
+
| `recordResources` on the critical path | ~0 ms, was 43 |
|
|
241
|
+
| `status`, remote shell work | 9 ms, was 23 |
|
|
242
|
+
| `wait`, overshoot past the job's own end | ~0.1 s, was up to 1 s |
|
|
243
|
+
| `list` walking 1000 job directories | 23 ms |
|
|
244
|
+
| the settings form rendering 50 servers | 0.7 ms |
|
|
245
|
+
| local work per call (config, guard, wrapper text) | 1–3 µs |
|
|
246
|
+
|
|
247
|
+
What that says: on Windows, where connections cannot be reused, the round trip
|
|
248
|
+
*is* the cost and the remote shell work is noise. On Linux or macOS with reuse on,
|
|
249
|
+
the remote shell work is a tenth of a call, which is why the launcher was worth
|
|
250
|
+
rewriting — one `printf` instead of a `cat` per field, the environment probes moved
|
|
251
|
+
off the critical path, and the memory sampler started before them so a short job is
|
|
252
|
+
still sampled.
|
|
253
|
+
|
|
254
|
+
## Compatibility
|
|
255
|
+
|
|
256
|
+
Requires DSH `>=0.2.2-rc.3` (`engines.dsh` refuses older hosts, because the
|
|
257
|
+
settings form uses the `plugins.row.config` slot and the `configForms` client
|
|
258
|
+
service). Verified against 0.2.2-rc.3 on Windows 11 (Windows OpenSSH,
|
|
259
|
+
`tar.exe`) with Ubuntu 24.04 remotes.
|
|
260
|
+
|
|
261
|
+
## License
|
|
262
|
+
|
|
263
|
+
MIT. See [LICENSE](LICENSE).
|