remote-exec-grpc 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- remote_exec_grpc-0.2.1/.gitignore +8 -0
- remote_exec_grpc-0.2.1/Makefile +13 -0
- remote_exec_grpc-0.2.1/PKG-INFO +539 -0
- remote_exec_grpc-0.2.1/README.md +528 -0
- remote_exec_grpc-0.2.1/README_CN.md +461 -0
- remote_exec_grpc-0.2.1/docs/requirements.md +1021 -0
- remote_exec_grpc-0.2.1/examples/config.yaml +36 -0
- remote_exec_grpc-0.2.1/examples/remote-exec-server.service +58 -0
- remote_exec_grpc-0.2.1/examples/server.yaml +33 -0
- remote_exec_grpc-0.2.1/examples/tools.yaml +34 -0
- remote_exec_grpc-0.2.1/proto/remote_exec/v1/remote_exec.proto +101 -0
- remote_exec_grpc-0.2.1/pyproject.toml +35 -0
- remote_exec_grpc-0.2.1/scripts/gen_proto.sh +13 -0
- remote_exec_grpc-0.2.1/src/remote_exec/__init__.py +10 -0
- remote_exec_grpc-0.2.1/src/remote_exec/claude_skill.md +118 -0
- remote_exec_grpc-0.2.1/src/remote_exec/client.py +242 -0
- remote_exec_grpc-0.2.1/src/remote_exec/client_cli.py +226 -0
- remote_exec_grpc-0.2.1/src/remote_exec/cmdlog.py +159 -0
- remote_exec_grpc-0.2.1/src/remote_exec/config.py +202 -0
- remote_exec_grpc-0.2.1/src/remote_exec/executor.py +287 -0
- remote_exec_grpc-0.2.1/src/remote_exec/policy.py +420 -0
- remote_exec_grpc-0.2.1/src/remote_exec/security.py +178 -0
- remote_exec_grpc-0.2.1/src/remote_exec/server.py +292 -0
- remote_exec_grpc-0.2.1/src/remote_exec/server_cli.py +219 -0
- remote_exec_grpc-0.2.1/src/remote_exec/skill.py +39 -0
- remote_exec_grpc-0.2.1/src/remote_exec/v1/__init__.py +1 -0
- remote_exec_grpc-0.2.1/src/remote_exec/v1/remote_exec_pb2.py +57 -0
- remote_exec_grpc-0.2.1/src/remote_exec/v1/remote_exec_pb2.pyi +108 -0
- remote_exec_grpc-0.2.1/src/remote_exec/v1/remote_exec_pb2_grpc.py +142 -0
- remote_exec_grpc-0.2.1/tests/conftest.py +105 -0
- remote_exec_grpc-0.2.1/tests/test_cli.py +231 -0
- remote_exec_grpc-0.2.1/tests/test_cmdlog.py +185 -0
- remote_exec_grpc-0.2.1/tests/test_config.py +221 -0
- remote_exec_grpc-0.2.1/tests/test_executor.py +135 -0
- remote_exec_grpc-0.2.1/tests/test_grpc.py +219 -0
- remote_exec_grpc-0.2.1/tests/test_policy.py +198 -0
- remote_exec_grpc-0.2.1/tests/test_skill.py +57 -0
- remote_exec_grpc-0.2.1/tests/test_token.py +80 -0
- remote_exec_grpc-0.2.1/uv.lock +320 -0
|
@@ -0,0 +1,539 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: remote-exec-grpc
|
|
3
|
+
Version: 0.2.1
|
|
4
|
+
Summary: Lightweight gRPC remote command execution with real-time stdout/stderr streaming
|
|
5
|
+
License: MIT
|
|
6
|
+
Requires-Python: >=3.11
|
|
7
|
+
Requires-Dist: grpcio>=1.84.0
|
|
8
|
+
Requires-Dist: protobuf<8,>=7.35.1
|
|
9
|
+
Requires-Dist: pyyaml>=6.0
|
|
10
|
+
Description-Content-Type: text/markdown
|
|
11
|
+
|
|
12
|
+
# remote-exec
|
|
13
|
+
|
|
14
|
+
Lightweight remote command execution over gRPC: run a command on another Linux
|
|
15
|
+
server and get its **stdout / stderr streamed in real time** plus its **exit
|
|
16
|
+
code**, with an `ssh host cmd`-like experience.
|
|
17
|
+
|
|
18
|
+
## 1. Purpose
|
|
19
|
+
|
|
20
|
+
AI coding agents (Claude Code, Codex, ...) run on a local machine, but some
|
|
21
|
+
work needs tools or hardware that only exist on specific servers: Vivado and
|
|
22
|
+
Questa on an FPGA box, CUDA GPUs, licensed tools, large datasets.
|
|
23
|
+
`remote-exec` extends the agent's *tool execution* to those servers:
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
remote-exec-client fpga01 --cwd /home/user/project "vivado -mode batch -source build.tcl"
|
|
27
|
+
echo $? # exit code of the remote vivado run
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
The remote execution layer is **agent-independent**. It contains no AI/LLM logic
|
|
31
|
+
and works the same for agents, shell scripts, CI and humans. Claude Code is just
|
|
32
|
+
one consumer (`remote-exec-client --install-skill`, see [Claude Code integration](#14-claude-code-integration)).
|
|
33
|
+
|
|
34
|
+
## 2. Architecture
|
|
35
|
+
|
|
36
|
+
```
|
|
37
|
+
Claude Code / script / human
|
|
38
|
+
│ (Skill → Bash)
|
|
39
|
+
▼
|
|
40
|
+
remote-exec-client ── client_cli.py → client.py (grpc.aio)
|
|
41
|
+
│ gRPC server-streaming, optional TLS/mTLS + bearer token
|
|
42
|
+
▼
|
|
43
|
+
remote-exec-server ── server_cli.py → server.py (grpc.aio servicer)
|
|
44
|
+
│ │
|
|
45
|
+
│ executor.py (asyncio subprocess,
|
|
46
|
+
▼ process groups, timeouts)
|
|
47
|
+
bash -c "<command>" (own session / process group)
|
|
48
|
+
├── vivado ── child processes
|
|
49
|
+
├── python / nvcc / git ...
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
One Python package provides **both** commands. Every machine that installs it
|
|
53
|
+
gets `remote-exec-server` and `remote-exec-client`; a machine can run either or
|
|
54
|
+
both (server A can call server B).
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
proto/remote_exec/v1/remote_exec.proto API definition
|
|
58
|
+
src/remote_exec/
|
|
59
|
+
v1/ generated protobuf/gRPC code (committed)
|
|
60
|
+
executor.py subprocess management (no gRPC dependency)
|
|
61
|
+
server.py gRPC servicer, validation, audit log, lifecycle
|
|
62
|
+
server_cli.py remote-exec-server entry point
|
|
63
|
+
client.py async client library
|
|
64
|
+
client_cli.py remote-exec-client entry point
|
|
65
|
+
config.py client/server config loading
|
|
66
|
+
security.py token auth interceptor, TLS credentials
|
|
67
|
+
policy.py tools.yaml allow/deny policy and command analysis
|
|
68
|
+
cmdlog.py command log (commands + output, size-based rotation)
|
|
69
|
+
skill.py --install-skill implementation
|
|
70
|
+
claude_skill.md Claude Code skill text (installed by --install-skill)
|
|
71
|
+
tests/ unit + localhost integration tests
|
|
72
|
+
examples/ config.yaml, server.yaml, systemd unit
|
|
73
|
+
scripts/gen_proto.sh protobuf code generation
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### API (`remote_exec.v1.RemoteExecutor`)
|
|
77
|
+
|
|
78
|
+
```proto
|
|
79
|
+
rpc Execute(ExecuteRequest) returns (stream ExecuteEvent);
|
|
80
|
+
rpc GetServerInfo(GetServerInfoRequest) returns (ServerInfo);
|
|
81
|
+
|
|
82
|
+
ExecuteRequest { command, working_directory, map environment, timeout_seconds }
|
|
83
|
+
ExecuteEvent { oneof: started | stdout | stderr | finished }
|
|
84
|
+
Finished { status: EXITED|TIMEOUT|ERROR, exit_code, signal,
|
|
85
|
+
start_time, end_time, message, execution_id }
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Key design choices:
|
|
89
|
+
|
|
90
|
+
- **`oneof` events**: `Started` → any number of `stdout`/`stderr` chunks →
|
|
91
|
+
exactly one `Finished`, always last. The type system enforces that each event
|
|
92
|
+
carries only the fields that make sense for it.
|
|
93
|
+
- **Raw bytes, not lines**: progress bars without newlines appear immediately
|
|
94
|
+
and non-UTF-8 output survives unchanged.
|
|
95
|
+
- **Two error channels**: problems with the *request* (empty command, invalid
|
|
96
|
+
env name, auth, server busy) are gRPC status codes. What happens *after* the
|
|
97
|
+
request is accepted (spawn failure, timeout, exit status) is reported in
|
|
98
|
+
`Finished`.
|
|
99
|
+
- **Shell-style `exit_code`**: `128 + N` when killed by signal N, plus the raw
|
|
100
|
+
`signal` field.
|
|
101
|
+
- **Versioned package** (`remote_exec.v1`), so `Submit/GetStatus/GetResult/
|
|
102
|
+
Cancel` or `Upload/Download/Stat/List` can be added later as new RPCs or
|
|
103
|
+
services without breaking existing clients.
|
|
104
|
+
|
|
105
|
+
## 3. Installing uv
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## 4. Installing the tools (`uv tool install`)
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
uv tool install remote-exec-grpc # from PyPI
|
|
115
|
+
uv tool install . # from a checkout
|
|
116
|
+
uv tool install git+https://example.com/remote-exec.git # from a git repo
|
|
117
|
+
uv tool install --force . # upgrade / reinstall
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
This puts `remote-exec-server` and `remote-exec-client` in `~/.local/bin`.
|
|
121
|
+
The PyPI distribution is named `remote-exec-grpc` (`remote-exec` is taken by an
|
|
122
|
+
unrelated project); the import package and command names stay `remote_exec` /
|
|
123
|
+
`remote-exec-*`.
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
remote-exec-server --port 50051 # on the remote server
|
|
127
|
+
remote-exec-client fpga01 "hostname" # on the local machine
|
|
128
|
+
remote-exec-client --install-skill # optional: let Claude Code use it (see §14)
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
`fpga01` is a server name defined in the client config file
|
|
132
|
+
(`~/.config/remote-exec/config.yaml`), which maps it to a host and port; see
|
|
133
|
+
[§9 Client configuration](#9-client-configuration). `remote-exec-client --list`
|
|
134
|
+
shows all configured names and their addresses.
|
|
135
|
+
|
|
136
|
+
## 5. Development environment
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
uv sync # creates .venv with runtime + dev deps (from uv.lock)
|
|
140
|
+
uv run pytest # run tests
|
|
141
|
+
uv run remote-exec-server --help
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Runtime dependencies: `grpcio`, `protobuf`, `pyyaml`. Dev: `grpcio-tools`,
|
|
145
|
+
`pytest`, `pytest-asyncio`.
|
|
146
|
+
|
|
147
|
+
## 6. Protobuf generation
|
|
148
|
+
|
|
149
|
+
The generated code in `src/remote_exec/v1/` is committed, so installing never
|
|
150
|
+
needs `protoc`. After editing the `.proto`, regenerate it with:
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
make proto # or ./scripts/gen_proto.sh, which runs:
|
|
154
|
+
uv run python -m grpc_tools.protoc -I proto --python_out=src --pyi_out=src \
|
|
155
|
+
--grpc_python_out=src proto/remote_exec/v1/remote_exec.proto
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
The proto lives at `proto/remote_exec/v1/` (rather than `proto/remote_exec.proto`)
|
|
159
|
+
so the generated `import` statements resolve to the `remote_exec.v1` package.
|
|
160
|
+
If you upgrade `grpcio-tools`, keep the `grpcio`/`protobuf` lower bounds in
|
|
161
|
+
`pyproject.toml` at least at the generator versions.
|
|
162
|
+
|
|
163
|
+
## 7. Server startup
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
remote-exec-server # 127.0.0.1:50051, local only
|
|
167
|
+
remote-exec-server --generate-token # create ~/.config/remote-exec/token (mode 600)
|
|
168
|
+
remote-exec-server --host 0.0.0.0 --port 50051 --token-file ~/.config/remote-exec/token
|
|
169
|
+
remote-exec-server --config /etc/remote-exec/server.yaml
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Safety rules enforced at startup:
|
|
173
|
+
|
|
174
|
+
- **Default bind address is `127.0.0.1`**. Because the service is remote code
|
|
175
|
+
execution, it must never be exposed by accident. To serve other machines you
|
|
176
|
+
set `--host` explicitly.
|
|
177
|
+
- **Non-loopback addresses require authentication** (`--token-file` and/or mTLS).
|
|
178
|
+
Otherwise the server refuses to start unless you pass `--insecure-no-auth`.
|
|
179
|
+
- **Refuses to run as root** unless you pass `--allow-root`.
|
|
180
|
+
- Warns when TLS is off on a non-loopback address.
|
|
181
|
+
|
|
182
|
+
Other options: `--shell` (default `/bin/bash`), `--max-concurrent` (32),
|
|
183
|
+
`--max-timeout`, `--kill-grace` (5s), `--log-level`, `--log-file`. See
|
|
184
|
+
`remote-exec-server --help` and [examples/server.yaml](examples/server.yaml).
|
|
185
|
+
|
|
186
|
+
### Allowed programs (`tools.yaml`)
|
|
187
|
+
|
|
188
|
+
`~/.config/remote-exec/tools.yaml` on the server lists the programs that
|
|
189
|
+
commands may and may not run. Use `--tools FILE` or `tools:` in the server
|
|
190
|
+
config to load it from elsewhere. See [examples/tools.yaml](examples/tools.yaml).
|
|
191
|
+
|
|
192
|
+
```yaml
|
|
193
|
+
allow: ["*"] # "*" = every program; globs like "python*" work
|
|
194
|
+
deny: [ssh] # deny wins if a program matches both lists
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
- **Default:** without the file, or for a missing key, the server uses
|
|
198
|
+
`allow: ["*"]` and `deny: ["ssh"]`, so everything except `ssh` is allowed.
|
|
199
|
+
`deny: []` denies nothing.
|
|
200
|
+
- **What is checked:** every program a command line would start. That covers
|
|
201
|
+
each part of `|`, `&&`, `;` and newlines, `$(...)` and backticks, wrappers
|
|
202
|
+
such as `sudo`, `env`, `timeout` and `nohup`, and scripts given to
|
|
203
|
+
`bash -c` / `eval`. So `ssh h`, `/usr/bin/ssh h`, `make && ssh h`,
|
|
204
|
+
`echo $(ssh h)` and `bash -c "ssh h"` are all rejected. `echo ssh` and
|
|
205
|
+
`grep ssh log` are not, because there `ssh` is only an argument.
|
|
206
|
+
- **How patterns match:** a pattern without `/` matches the program's base
|
|
207
|
+
name; a pattern with `/` matches the path as written. Shell builtins (`cd`,
|
|
208
|
+
`echo`, `export`, `test`, ...) never need to be in `allow`.
|
|
209
|
+
- **Rejections:** a rejected command is not started. The client exits with
|
|
210
|
+
**126** and prints the reason, and the audit log records it.
|
|
211
|
+
`remote-exec-client SERVER --ping` shows the active lists.
|
|
212
|
+
- **Reloading:** edits take effect on the next command, with no restart. An
|
|
213
|
+
invalid file is rejected at startup; if a later edit breaks the file, the
|
|
214
|
+
previous policy stays active and an error is logged.
|
|
215
|
+
- **Limits:** this is a guardrail against mistakes (e.g. an agent hopping to
|
|
216
|
+
other machines), **not a sandbox**. Programs started indirectly, such as by
|
|
217
|
+
a script, `python -c`, or `X=ssh; $X`, are invisible to it. Use OS-level
|
|
218
|
+
controls (the service user's permissions, firewall rules) for hard limits.
|
|
219
|
+
|
|
220
|
+
### Command log
|
|
221
|
+
|
|
222
|
+
The server writes every command it runs, together with its output, to
|
|
223
|
+
`remote-exec-commands.log` in **the directory it was started from**. Under
|
|
224
|
+
systemd that is the unit's `WorkingDirectory=`. Each line carries the
|
|
225
|
+
execution id, so concurrent commands can be told apart:
|
|
226
|
+
|
|
227
|
+
```
|
|
228
|
+
2026-10-04 10:00:00.120 [3f2a9c1d0b7e] START client=ipv4:10.0.0.7:51234 cwd="/proj" timeout=0 env_keys="" command="make"
|
|
229
|
+
2026-10-04 10:00:00.250 [3f2a9c1d0b7e] OUT| gcc -c main.c
|
|
230
|
+
2026-10-04 10:00:00.260 [3f2a9c1d0b7e] ERR| warning: unused variable
|
|
231
|
+
2026-10-04 10:00:01.900 [3f2a9c1d0b7e] END status=exited exit_code=0 duration=1.780s
|
|
232
|
+
2026-10-04 10:00:02.010 [------------] REJECTED client=... reason="tools policy: ..." command="ssh other"
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
- **Rotation:** when the file would exceed **10 MB**, it is renamed to
|
|
236
|
+
`remote-exec-commands.<YYYYmmdd-HHMMSS>.log` and a new file is started.
|
|
237
|
+
- **Retention:** rotated files are **all kept** by default; use
|
|
238
|
+
`--command-log-keep N` to keep only the newest N, since long tool output can
|
|
239
|
+
add up quickly.
|
|
240
|
+
- **Options:** `--command-log FILE` (another location), `--command-log-max-mb MB`
|
|
241
|
+
and `--no-command-log`. The config keys are `command_log`,
|
|
242
|
+
`command_log_max_mb`, `command_log_keep` and `no_command_log`.
|
|
243
|
+
- **Privacy:** files are created with mode 600. Environment variable values
|
|
244
|
+
are not logged, but command output is, so treat the files as sensitive.
|
|
245
|
+
- **Reliability:** a full disk or other write error is reported once in the
|
|
246
|
+
server log and never interrupts commands.
|
|
247
|
+
- This is separate from the audit log (`remote_exec.audit`, stderr/journald
|
|
248
|
+
or `--log-file`), which records only command metadata.
|
|
249
|
+
|
|
250
|
+
## 8. Server port configuration
|
|
251
|
+
|
|
252
|
+
The default port is **50051**. Change it with any of:
|
|
253
|
+
|
|
254
|
+
```bash
|
|
255
|
+
remote-exec-server --port 50052
|
|
256
|
+
remote-exec-server --host 0.0.0.0 --port 50052
|
|
257
|
+
remote-exec-server --listen 0.0.0.0:50052 # same thing, one option
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
or `port:` / `listen:` in the server config file. CLI options override the file.
|
|
261
|
+
|
|
262
|
+
## 9. Client configuration
|
|
263
|
+
|
|
264
|
+
The default path is `~/.config/remote-exec/config.yaml`. It honours
|
|
265
|
+
`$XDG_CONFIG_HOME`, and `$REMOTE_EXEC_CONFIG` overrides it. You can also pass
|
|
266
|
+
`--config FILE`.
|
|
267
|
+
|
|
268
|
+
```yaml
|
|
269
|
+
defaults: # merged into every server
|
|
270
|
+
token_file: ~/.config/remote-exec/token
|
|
271
|
+
servers:
|
|
272
|
+
acme2:
|
|
273
|
+
host: acme2.example.local
|
|
274
|
+
port: 50051
|
|
275
|
+
gpu01:
|
|
276
|
+
host: gpu01.example.local # no port -> 50051
|
|
277
|
+
description: GPU computation
|
|
278
|
+
fpga01:
|
|
279
|
+
host: fpga01.example.local
|
|
280
|
+
port: 50052
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
`remote-exec-client fpga01 ...` connects to `fpga01.example.local:50052` and
|
|
284
|
+
`gpu01` to `gpu01.example.local:50051`. A target that is not in the config is
|
|
285
|
+
treated as `HOST[:PORT]`, for example `remote-exec-client 10.0.0.5 hostname`.
|
|
286
|
+
Per-server keys: `host`, `port`, `description`, `token`, `token_file`, `tls`,
|
|
287
|
+
`ca_cert`, `client_cert`, `client_key`, `tls_server_name`, `connect_timeout`.
|
|
288
|
+
Unknown keys are rejected so a typo can't silently disable a setting.
|
|
289
|
+
`$REMOTE_EXEC_TOKEN` is used when no token is configured.
|
|
290
|
+
|
|
291
|
+
`remote-exec-client --list` shows the configured servers and
|
|
292
|
+
`remote-exec-client fpga01 --ping` checks connectivity.
|
|
293
|
+
|
|
294
|
+
### One token per server
|
|
295
|
+
|
|
296
|
+
Each server generates its own token (`remote-exec-server --generate-token`).
|
|
297
|
+
On the client, keep one token file per server and point each server entry at
|
|
298
|
+
its own file with `token_file`:
|
|
299
|
+
|
|
300
|
+
```yaml
|
|
301
|
+
servers:
|
|
302
|
+
fpga01:
|
|
303
|
+
host: fpga01.example.local
|
|
304
|
+
token_file: ~/.config/remote-exec/tokens/fpga01.token
|
|
305
|
+
gpu01:
|
|
306
|
+
host: gpu01.example.local
|
|
307
|
+
token_file: ~/.config/remote-exec/tokens/gpu01.token
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
`remote-exec-client fpga01 ...` then sends only `fpga01.token`, and only to
|
|
311
|
+
fpga01. Separate tokens limit the damage if one leaks: it can be rotated on
|
|
312
|
+
that server without touching the others.
|
|
313
|
+
|
|
314
|
+
Fetch each server's token into its own file, then check every server:
|
|
315
|
+
|
|
316
|
+
```bash
|
|
317
|
+
mkdir -p ~/.config/remote-exec/tokens && chmod 700 ~/.config/remote-exec/tokens
|
|
318
|
+
( umask 077 # files are created with mode 600
|
|
319
|
+
ssh fpga01 cat ~/.config/remote-exec/token > ~/.config/remote-exec/tokens/fpga01.token
|
|
320
|
+
ssh gpu01 cat ~/.config/remote-exec/token > ~/.config/remote-exec/tokens/gpu01.token
|
|
321
|
+
)
|
|
322
|
+
remote-exec-client fpga01 --ping # "authentication failed" = token mismatch
|
|
323
|
+
remote-exec-client gpu01 --ping
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
For each server the client takes the first token it finds, in this order:
|
|
327
|
+
|
|
328
|
+
1. `token:` in that server's entry (the token string itself)
|
|
329
|
+
2. `token_file:` in that server's entry
|
|
330
|
+
3. `token` / `token_file` from `defaults:`. A server's own keys override
|
|
331
|
+
`defaults`.
|
|
332
|
+
4. the `$REMOTE_EXEC_TOKEN` environment variable
|
|
333
|
+
|
|
334
|
+
So a shared token in `defaults` and per-server exceptions can be mixed:
|
|
335
|
+
|
|
336
|
+
```yaml
|
|
337
|
+
defaults:
|
|
338
|
+
token_file: ~/.config/remote-exec/tokens/shared.token # used by gpu01
|
|
339
|
+
servers:
|
|
340
|
+
gpu01:
|
|
341
|
+
host: gpu01.example.local
|
|
342
|
+
fpga01:
|
|
343
|
+
host: fpga01.example.local
|
|
344
|
+
token_file: ~/.config/remote-exec/tokens/fpga01.token # overrides defaults
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
A configured `token_file` that does not exist makes the client exit with 255.
|
|
348
|
+
Keep token files at mode 600; the client warns about group/world-readable
|
|
349
|
+
token files.
|
|
350
|
+
|
|
351
|
+
## 10. Command execution
|
|
352
|
+
|
|
353
|
+
```bash
|
|
354
|
+
remote-exec-client acme2 "hostname"
|
|
355
|
+
remote-exec-client gpu01 "nvidia-smi"
|
|
356
|
+
remote-exec-client acme2 --cwd /home/user/project "python test.py"
|
|
357
|
+
remote-exec-client fpga01 --env FPGA_PART=xcvu37p --env BUILD_MODE=release "./build.sh"
|
|
358
|
+
remote-exec-client gpu01 --timeout 60 -- ls -la /data # unquoted words joined like ssh
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
- The command runs as `bash -c "<command>"` on the server, with stdin set to
|
|
362
|
+
`/dev/null`. It runs as the server's user, with the server's environment plus
|
|
363
|
+
the `--env` values.
|
|
364
|
+
- `--cwd` must be absolute or start with `~` (expanded on the server). It is
|
|
365
|
+
equivalent to `cd DIR && command`.
|
|
366
|
+
- `--timeout SECONDS` kills the whole remote process tree when it expires.
|
|
367
|
+
|
|
368
|
+
**Exit status** of `remote-exec-client`:
|
|
369
|
+
|
|
370
|
+
| status | meaning |
|
|
371
|
+
|--------|---------|
|
|
372
|
+
| N | the remote command's exit status (`false` → 1, command not found → 127) |
|
|
373
|
+
| 128+N | remote command killed by signal N |
|
|
374
|
+
| 124 | timeout (like coreutils `timeout`) |
|
|
375
|
+
| 126 | rejected by the server's tools policy (`tools.yaml`) |
|
|
376
|
+
| 130 / 143 | client interrupted by SIGINT / SIGTERM; remote command was cancelled |
|
|
377
|
+
| 255 | remote-exec error (like ssh): unreachable, auth, config, invalid cwd |
|
|
378
|
+
|
|
379
|
+
## 11. Streaming behaviour
|
|
380
|
+
|
|
381
|
+
Output is forwarded **as it is produced**, in chunks of up to 64 KiB, with no
|
|
382
|
+
line buffering on the server side. Remote stdout goes to local stdout and
|
|
383
|
+
remote stderr goes to local stderr. The relative order *within* each stream is
|
|
384
|
+
preserved; ordering *between* stdout and stderr is best-effort, as with ssh.
|
|
385
|
+
|
|
386
|
+
Programs that buffer their own output when not on a TTY (e.g. Python) may need
|
|
387
|
+
`python -u` or `PYTHONUNBUFFERED=1` (`-e PYTHONUNBUFFERED=1`).
|
|
388
|
+
|
|
389
|
+
Lifecycle guarantees:
|
|
390
|
+
|
|
391
|
+
- Each command runs in its own session/process group. On **timeout**, **client
|
|
392
|
+
disconnect** (Ctrl-C, SIGTERM, killed client, lost network detected by gRPC
|
|
393
|
+
keepalive) or **server shutdown**, the server sends SIGTERM to the *whole
|
|
394
|
+
group*, then SIGKILL after `--kill-grace` seconds. Child processes (e.g.
|
|
395
|
+
Vivado workers) are not left behind.
|
|
396
|
+
- Background processes the command leaves in its group are also terminated
|
|
397
|
+
once the main command exits, so nothing is orphaned.
|
|
398
|
+
- A slow client applies back-pressure: the remote process blocks on a full
|
|
399
|
+
pipe instead of the server buffering unbounded output.
|
|
400
|
+
- Keepalive pings keep silent multi-hour jobs alive through NAT/firewalls.
|
|
401
|
+
|
|
402
|
+
## 12. systemd setup
|
|
403
|
+
|
|
404
|
+
See [examples/remote-exec-server.service](examples/remote-exec-server.service).
|
|
405
|
+
|
|
406
|
+
```bash
|
|
407
|
+
sudo -u builder uv tool install /path/to/remote-exec
|
|
408
|
+
sudo install -m 644 examples/remote-exec-server.service /etc/systemd/system/
|
|
409
|
+
sudo install -d -o builder -m 700 /etc/remote-exec
|
|
410
|
+
sudo -u builder /home/builder/.local/bin/remote-exec-server --generate-token /etc/remote-exec/token
|
|
411
|
+
# edit User=/ExecStart= paths in the unit
|
|
412
|
+
sudo systemctl daemon-reload
|
|
413
|
+
sudo systemctl enable remote-exec-server
|
|
414
|
+
sudo systemctl start remote-exec-server
|
|
415
|
+
journalctl -u remote-exec-server -f
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
The unit runs as an unprivileged user (`User=`). It shows how to set host,
|
|
419
|
+
port, token, config file and log level/file. On stop, the cgroup-based
|
|
420
|
+
`KillMode` also removes any process that escaped its process group.
|
|
421
|
+
|
|
422
|
+
## 13. Security considerations
|
|
423
|
+
|
|
424
|
+
This service **is remote code execution** by design. Anyone who can
|
|
425
|
+
authenticate can run any command as the service user.
|
|
426
|
+
|
|
427
|
+
- Bind to loopback by default. For remote use, prefer binding to a private
|
|
428
|
+
interface (Tailscale/WireGuard IP) over `0.0.0.0`. Never expose the port to
|
|
429
|
+
the internet; firewall it.
|
|
430
|
+
- **Authenticate even on private networks**:
|
|
431
|
+
- *Bearer token*: `--token-file` on the server; `token_file` in the client
|
|
432
|
+
config. The token is compared in constant time. Keep the file at mode 600
|
|
433
|
+
(the server warns otherwise).
|
|
434
|
+
`remote-exec-server --generate-token [FILE]` creates a random 256-bit token.
|
|
435
|
+
The default FILE is `~/.config/remote-exec/token`. The file gets mode 600
|
|
436
|
+
and its directory 700, and the token itself is never printed. An existing
|
|
437
|
+
file is only replaced with `--force`, because that locks out clients still
|
|
438
|
+
using the old token. Copy the file to each client (e.g. `scp`, then
|
|
439
|
+
`chmod 600`) and point `token_file` at it. Without TLS the token travels in clear text,
|
|
440
|
+
which is fine inside WireGuard/Tailscale but not on a shared LAN.
|
|
441
|
+
- *TLS*: `--tls-cert/--tls-key` on the server; `tls: true` + `ca_cert` on the
|
|
442
|
+
client.
|
|
443
|
+
- *mTLS* (recommended for LANs): add `--tls-client-ca` on the server, and
|
|
444
|
+
`client_cert`/`client_key` on the client. The client certificate's CN
|
|
445
|
+
appears in the audit log.
|
|
446
|
+
- Run as a dedicated unprivileged user; root is refused by default. Restrict
|
|
447
|
+
what that user can access (groups, file permissions, sudo).
|
|
448
|
+
- Limit which programs commands may run with `tools.yaml` (see §7). By
|
|
449
|
+
default `ssh` is denied so commands cannot hop to other machines. It is a
|
|
450
|
+
guardrail, not a sandbox.
|
|
451
|
+
- **Command log** (`remote-exec-commands.log`, see §7) contains full command
|
|
452
|
+
output. It is created with mode 600; restrict who can read the server's
|
|
453
|
+
working directory.
|
|
454
|
+
- **Audit log** (`remote_exec.audit` logger): one `start` record (id, client,
|
|
455
|
+
cwd, timeout, env variable *names*, command) and one `end` record (status,
|
|
456
|
+
exit code, duration) per command. Environment variable **values are never
|
|
457
|
+
logged**. Command lines are logged, so don't put secrets in them.
|
|
458
|
+
- Input validation: non-empty command without NUL bytes, size limits, absolute
|
|
459
|
+
cwd, POSIX env names, finite non-negative timeout, optional server-wide
|
|
460
|
+
`--max-timeout` and `--max-concurrent` limits.
|
|
461
|
+
|
|
462
|
+
## 14. Claude Code integration
|
|
463
|
+
|
|
464
|
+
On the machine where Claude Code runs:
|
|
465
|
+
|
|
466
|
+
```bash
|
|
467
|
+
remote-exec-client --install-skill # -> ~/.claude/commands/remote-exec.md
|
|
468
|
+
remote-exec-client --install-skill DIR # -> DIR/remote-exec.md
|
|
469
|
+
remote-exec-client --install-skill --force # overwrite a locally modified copy
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
This installs the `remote-exec` skill (a Claude Code custom command; source:
|
|
473
|
+
[src/remote_exec/claude_skill.md](src/remote_exec/claude_skill.md)). It covers
|
|
474
|
+
`remote-exec-client` only, since the servers are assumed to be running. It tells
|
|
475
|
+
Claude to:
|
|
476
|
+
|
|
477
|
+
- discover servers and their tools with `remote-exec-client --list`, and check
|
|
478
|
+
them with `--ping`
|
|
479
|
+
- decide when remote execution is appropriate
|
|
480
|
+
- pass `--cwd` / `-e` / `--timeout`, interpret exit codes, and handle long jobs
|
|
481
|
+
and large output
|
|
482
|
+
- avoid destructive commands
|
|
483
|
+
|
|
484
|
+
Describe each server's tools in the client config's `description` field, because
|
|
485
|
+
that is what `--list` shows Claude:
|
|
486
|
+
|
|
487
|
+
```yaml
|
|
488
|
+
servers:
|
|
489
|
+
fpga01: {host: fpga01.example.local, description: "FPGA development: Vivado, Questa"}
|
|
490
|
+
gpu01: {host: gpu01.example.local, description: "GPU computation: CUDA, nvidia-smi, Python"}
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
Claude then uses the skill on its own (or when you type `/remote-exec <task>`) and calls, e.g.,
|
|
494
|
+
`remote-exec-client fpga01 --cwd /home/user/project --timeout 7200 "vivado -mode batch -source build.tcl"`.
|
|
495
|
+
It reads the streamed output and the exit code. Re-run `--install-skill` after
|
|
496
|
+
upgrading the package to pick up a newer version of the skill.
|
|
497
|
+
|
|
498
|
+
## 15. Testing
|
|
499
|
+
|
|
500
|
+
```bash
|
|
501
|
+
uv run pytest # ~20 s
|
|
502
|
+
```
|
|
503
|
+
|
|
504
|
+
- `tests/test_executor.py`: subprocess layer (streaming, exit codes, signals,
|
|
505
|
+
cwd/env, timeout + SIGKILL escalation, process-group cleanup).
|
|
506
|
+
- `tests/test_grpc.py`: in-process server + client library (success,
|
|
507
|
+
non-zero exit, stdout, stderr, mixed, cwd, env, timeout, command not found,
|
|
508
|
+
real-time streaming, concurrency, client disconnect/cancel, server
|
|
509
|
+
unavailable, invalid requests, max concurrency, token auth, mTLS).
|
|
510
|
+
- `tests/test_cli.py`: the actual console scripts as subprocesses. Covers the
|
|
511
|
+
**default port 50051** (`remote-exec-server --host 127.0.0.1` +
|
|
512
|
+
`remote-exec-client local "echo hello"`), the **custom port 50052**, exit
|
|
513
|
+
codes, timeout → 124, unavailable → 255, SIGINT/SIGTERM/SIGKILL of the client
|
|
514
|
+
killing the remote process tree, server shutdown, and refusal of
|
|
515
|
+
unauthenticated public binds. The fixed-port tests are skipped if the port is
|
|
516
|
+
already in use.
|
|
517
|
+
- `tests/test_config.py`: config parsing, default/custom ports, server option
|
|
518
|
+
precedence, safety checks.
|
|
519
|
+
- `tests/test_skill.py`: `--install-skill` (default/custom dir, update, `--force`).
|
|
520
|
+
- `tests/test_token.py`: `--generate-token` (permissions, randomness, `--force`,
|
|
521
|
+
generated token authenticates).
|
|
522
|
+
- `tests/test_policy.py`: `tools.yaml` (command analysis, deny-wins, globs,
|
|
523
|
+
defaults, reload, enforcement, exit 126).
|
|
524
|
+
- `tests/test_cmdlog.py`: command log (format, partial/long lines, rotation,
|
|
525
|
+
retention, permissions, default location, disabling).
|
|
526
|
+
|
|
527
|
+
## 16. Future extensions
|
|
528
|
+
|
|
529
|
+
Kept out of v1 on purpose, but the structure is ready for them:
|
|
530
|
+
|
|
531
|
+
- **Asynchronous jobs** (`Submit`, `GetStatus`, `GetResult`, `Cancel`): the
|
|
532
|
+
`Executor` already produces a gRPC-independent event stream with an
|
|
533
|
+
`execution_id`. A small in-memory task registry could consume it in a
|
|
534
|
+
background task, and the new RPCs could read from it.
|
|
535
|
+
- **File RPCs** (`Upload`, `Download`, `Stat`, `List`) as a sibling service in
|
|
536
|
+
`remote_exec.v1`, sharing the same auth/TLS setup.
|
|
537
|
+
- Per-client authorization (map mTLS CN or token → allowed commands/dirs).
|
|
538
|
+
- Optional PTY mode and stdin forwarding (bidirectional streaming).
|
|
539
|
+
- Agent-to-agent communication is explicitly **not** a goal of this layer.
|