@shardflux/cli 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +73 -0
- package/README.md +79 -7
- package/dist/args.d.ts +1 -1
- package/dist/args.js +50 -5
- package/dist/commands.js +219 -9
- package/dist/errors.d.ts +10 -1
- package/dist/errors.js +15 -1
- package/dist/format.d.ts +11 -1
- package/dist/format.js +64 -1
- package/dist/main.js +34 -0
- package/dist/template-yaml.js +3 -0
- package/dist/timing.js +2 -1
- package/package.json +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,79 @@
|
|
|
3
3
|
Every command and option the README shows is available from the version named here. Breaking changes ship in minor
|
|
4
4
|
releases and are marked **Breaking**.
|
|
5
5
|
|
|
6
|
+
## 0.8.0 (not yet published)
|
|
7
|
+
|
|
8
|
+
Needs `@shardflux/sdk` 0.13.0 or later.
|
|
9
|
+
|
|
10
|
+
### Elastic memory
|
|
11
|
+
|
|
12
|
+
- `shard ws open` and `shard ws fork`: `--allocation-mode fixed|elastic` and `--memory-held <mib>` (elastic only; a
|
|
13
|
+
usage error otherwise), sent with the caps. Caps flags replace the stored caps; no caps flag keeps the stored layout.
|
|
14
|
+
- `shard ws get` (and a ready `ws open`) prints a `memory` line: `fixed 4096 MiB`, or `elastic: promised 8192 MiB,
|
|
15
|
+
held 1024 MiB, plugged +2048 MiB` (`- (not running)` without a live allocation). `caps` shows `allocation_mode` and
|
|
16
|
+
`memory_mib_held`.
|
|
17
|
+
- `shard ws exec -v` prints `memory grow partial 1024 -> 3072 of 4096 MiB in 175 ms` on stderr when the command's
|
|
18
|
+
start waited for a grow; `--json` adds `memory_grow`. Without a grow nothing changes.
|
|
19
|
+
|
|
20
|
+
### Burst execution
|
|
21
|
+
|
|
22
|
+
- `shard ws exec --burst [--burst-vcpus N] [--burst-memory MIB]` runs the command on a larger, short-lived burst VM
|
|
23
|
+
and applies its file changes back.
|
|
24
|
+
`-v` prints a `burst` line (size, method, applied changes, leftover processes killed, overhead) on stderr; `--json`
|
|
25
|
+
adds `burst`. The sizes without `--burst`, out of range, or `--burst` with `--stdin`, are usage errors (exit 2);
|
|
26
|
+
`--burst` on a file-first workspace too.
|
|
27
|
+
- Ctrl-C does not cancel a burst (it cannot be canceled). A `burst_apply_failed` error names the exec session on
|
|
28
|
+
stderr.
|
|
29
|
+
|
|
30
|
+
### Background commands
|
|
31
|
+
|
|
32
|
+
- `shard ws exec-start --timeout <duration>`: kill the command after this long (parsed like `shard ws exec --timeout`;
|
|
33
|
+
none unless set). Without `--json` it names the commands that read and stop the session on stderr.
|
|
34
|
+
- `shard ws exec-read <id|key> <session-id> [--follow]`: the session's stdout and stderr from the start. An ended
|
|
35
|
+
session exits with the command's exit code (124 timed out, 128+N killed by signal N), one that still runs exits 5
|
|
36
|
+
with the command that follows it. `--follow` streams until the command exits, resuming from byte offsets after a
|
|
37
|
+
dropped connection; Ctrl-C stops following and the command keeps running. `--json` prints one object (`session_id`,
|
|
38
|
+
`state`, `exit_code`, `term_signal`, `timed_out`, `canceled`, `stdout`, `stderr`, `truncated`, `reconnects`).
|
|
39
|
+
A session that could not start exits 6 with the workspace's reason, as `shard ws exec` does.
|
|
40
|
+
- `shard ws exec-cancel <id|key> <session-id> [--grace <duration>]`: SIGTERM, then SIGKILL after the grace (default
|
|
41
|
+
5s, at most 60s). Prints `exec session <id>: canceled (signal 15)` (`--json`: the session).
|
|
42
|
+
|
|
43
|
+
### Suspend by id
|
|
44
|
+
|
|
45
|
+
- `shard ws suspend <id>` sends the suspend straight to the workspace, without first fetching it: one request fewer.
|
|
46
|
+
The line it prints names the id you gave (`Suspending <id>: operation ...`); with a key it is unchanged
|
|
47
|
+
(`Suspending <key> (<id>): ...`). A UUID that is no workspace id is still tried as a key.
|
|
48
|
+
|
|
49
|
+
### Held fork
|
|
50
|
+
|
|
51
|
+
- `shard ws fork --wait` gets the running copy and its tool token in the fork request itself when the API holds the
|
|
52
|
+
fork (SDK 0.13.0), without the follow-up operation poll and copy refresh. Against an API that does not, it polls as
|
|
53
|
+
before. `shard ws fork` without `--wait` is unchanged.
|
|
54
|
+
|
|
55
|
+
### Exec start and output in one request
|
|
56
|
+
|
|
57
|
+
- `shard ws exec` starts the command and streams its output in one request on cells that support it (SDK 0.13.0);
|
|
58
|
+
output still resumes from byte offsets after a dropped connection, and the command never runs twice.
|
|
59
|
+
|
|
60
|
+
### Immutable paths
|
|
61
|
+
|
|
62
|
+
A template may declare directories as `immutable`: every workspace of it mounts them read-only from the template's
|
|
63
|
+
newest published version.
|
|
64
|
+
|
|
65
|
+
- `shard templates get`: an `IMMUTABLE PATHS` column with each version's paths and image size
|
|
66
|
+
(`/opt/acme, /srv/tools (128.0 MiB)`), shown when a version declares any.
|
|
67
|
+
- `shard ws get` (and a ready `ws open`): `immutable paths v2 (mounted read-only)`, the template version whose
|
|
68
|
+
immutable paths the workspace has mounted.
|
|
69
|
+
- `shard templates build` sends template.yaml's `immutable` list; `shard templates builds get` (and a build's output)
|
|
70
|
+
prints `immutable paths`, and for a build that failed on them a `fix` line (`immutable_path_missing`: make the path a
|
|
71
|
+
directory in the build; `immutable_image_too_large`). `shard templates export` writes the list back.
|
|
72
|
+
`shard templates init` describes `immutable` in a comment.
|
|
73
|
+
- Hints for the refusals: `immutable_path_removed` names the paths to add back, `immutable_paths_unsupported_base` the
|
|
74
|
+
base to build on, `invalid_path` the `immutable[<i>]` entry to fix (exit 2), and a write under an immutable path
|
|
75
|
+
(`read_only_path`, 409 from the workspace, exit 1) says where to change it.
|
|
76
|
+
- **Breaking:** `--json` output no longer carries `update_policy`: the API removed the update policy (it was always
|
|
77
|
+
`pinned`).
|
|
78
|
+
|
|
6
79
|
## 0.6.2
|
|
7
80
|
|
|
8
81
|
Needs `@shardflux/sdk` 0.11.1 or later.
|
package/README.md
CHANGED
|
@@ -374,11 +374,12 @@ shows the status; `shard version --check` asks now (`--json`: `{"cli", "sdk", "u
|
|
|
374
374
|
| `version [--check]` | Print the CLI and SDK versions and the update status (`--check` asks the API now). |
|
|
375
375
|
| `usage [--org <id>] [--estimate \| --series \| --workspace <id\|key>] [--from T] [--to T] [--granularity hour\|day] [--meter M]` | Usage summary for the current period: plan, meters and allowances. `--estimate`: the bill so far and projected allowance use; `--series`: usage over time; `--workspace`: one workspace's usage over time (0.5.0+). The summary includes the [overage](#usage-and-overage) block (0.5.1+). |
|
|
376
376
|
| `feedback [--category C] [--workspace W] [--request-id ID] [--error-code CODE] [--command TEXT] [--from-file F\|-] <message...>` | Send feedback straight to the Shardflux team (0.5.0+). See [Feedback](#feedback-agents-use-this-while-you-work). |
|
|
377
|
-
| `workspaces open <key> --template <slug> [--secret NAME]... [--input NAME=value]... [--input-file F] [--lifetime persistent\|session] [--mode processful\|file-first] [--cpu-millis N --memory-mib N --disk-gib N] [--no-wait] [--timeout 5m] [--timing]` | Open by key. Creates the workspace on first use and reconnects or resumes it afterwards, never resetting it. Waits until it is ready (the template's start commands and services included) unless you pass `--no-wait`. `--secret` binds secret names (replacing the binding of an existing workspace). `--input` (0.4.0+) passes the template's text inputs (replacing those of an existing workspace); `--input-file` reads `NAME=value` lines or a JSON object. `--lifetime session`: the workspace is discarded when the session ends (`workspaces close` or the idle timeout), and the key then opens a new workspace. `--mode file-first` (0.5.0+) opens a [file-first workspace](#file-first-workspaces); the default is processful for a new key and the stored mode for an existing one. |
|
|
377
|
+
| `workspaces open <key> --template <slug> [--secret NAME]... [--input NAME=value]... [--input-file F] [--lifetime persistent\|session] [--mode processful\|file-first] [--cpu-millis N --memory-mib N --disk-gib N] [--allocation-mode fixed\|elastic] [--memory-held N] [--no-wait] [--timeout 5m] [--timing]` | Open by key. Creates the workspace on first use and reconnects or resumes it afterwards, never resetting it. Waits until it is ready (the template's start commands and services included) unless you pass `--no-wait`. `--secret` binds secret names (replacing the binding of an existing workspace). `--input` (0.4.0+) passes the template's text inputs (replacing those of an existing workspace); `--input-file` reads `NAME=value` lines or a JSON object. `--lifetime session`: the workspace is discarded when the session ends (`workspaces close` or the idle timeout), and the key then opens a new workspace. `--mode file-first` (0.5.0+) opens a [file-first workspace](#file-first-workspaces); the default is processful for a new key and the stored mode for an existing one. `--allocation-mode elastic` (0.8.0+): [elastic memory](#elastic-memory-080). |
|
|
378
378
|
| `workspaces list [--state S] [--prefix P] [--lifetime persistent\|session\|any] [--purpose standard\|template_draft\|template_test\|any] [--include-deleted] [--all] [--limit N] [--cursor C]` | One page, or every page with `--all`. The next cursor is printed on stderr. By default only persistent standard workspaces are listed. A file-first workspace's state reads `running (file-first)` (0.5.0+). |
|
|
379
|
-
| `workspaces get <id\|key>` | One workspace (deleted ones included), with the startup of its template's start commands and services (0.4.0+): a failed step shows its exit code, reason and output tail. Its mode, and a file-first workspace's tree revision (0.5.0+). |
|
|
379
|
+
| `workspaces get <id\|key>` | One workspace (deleted ones included), with the startup of its template's start commands and services (0.4.0+): a failed step shows its exit code, reason and output tail. Its mode, and a file-first workspace's tree revision (0.5.0+). Its memory (0.8.0+): `fixed 4096 MiB`, or `elastic: promised …, held …, plugged +… MiB`. The template version whose [immutable paths](#immutable-paths-080) it has mounted (0.8.0+): `immutable paths v3 (mounted read-only)`. |
|
|
380
380
|
| `workspaces inputs <id\|key>` | The workspace's text inputs (0.4.0+). |
|
|
381
|
-
| `workspaces exec <id\|key> [--cwd D] [--env K=V]... [--timeout T] [--stdin F\|-] [-v] [--execution-id ID] [--output-limit N] -- <cmd> [args...]` | Runs argv with no shell. Output is streamed and resumes from byte offsets after dropped connections. `shard` exits with the command's exit code. Ctrl-C cancels the command. `-v` (0.5.0+) then prints the session id and exit status on stderr. `--cwd` is an absolute path (default `/home/user`); a relative one is refused (`invalid_cwd`, exit 2). A command that could not start (a `--cwd` that is not a directory, a program not on `PATH`) exits 6 with the workspace's reason and a hint (0.5.1+; before, it exited 1 with no output). On a file-first workspace (0.5.0+) the command runs as an execution instead: see [File-first workspaces](#file-first-workspaces). |
|
|
381
|
+
| `workspaces exec <id\|key> [--cwd D] [--env K=V]... [--timeout T] [--stdin F\|-] [-v] [--execution-id ID] [--output-limit N] [--burst [--burst-vcpus N] [--burst-memory MIB]] -- <cmd> [args...]` | Runs argv with no shell. Output is streamed (in the start request itself, 0.8.0+) and resumes from byte offsets after dropped connections. `shard` exits with the command's exit code. Ctrl-C cancels the command. `-v` (0.5.0+) then prints the session id and exit status on stderr, and the memory grow the command waited for on an elastic workspace (0.8.0+; `--json`: `memory_grow`). `--cwd` is an absolute path (default `/home/user`); a relative one is refused (`invalid_cwd`, exit 2). A command that could not start (a `--cwd` that is not a directory, a program not on `PATH`) exits 6 with the workspace's reason and a hint (0.5.1+; before, it exited 1 with no output). On a file-first workspace (0.5.0+) the command runs as an execution instead: see [File-first workspaces](#file-first-workspaces). `--burst` (0.8.0+): [burst execution](#burst-execution-080). |
|
|
382
|
+
| `workspaces exec-start <id\|key> [--session-id ID] [--cwd D] [--timeout T] [--stdin-open] -- <cmd> [args...]`, `workspaces exec-read <id\|key> <session-id> [--follow]`, `workspaces exec-cancel <id\|key> <session-id> [--grace D]` | Background commands (0.8.0+): start a command and return at once, read its output and state (`--follow` streams it to its exit and exits with its code; while it runs `exec-read` exits 5), stop it. See [Background commands](#background-commands-080). |
|
|
382
383
|
| `executions get <id\|key> <execution-id> [--wait D]` | The result of a file-first workspace's execution (0.5.0+): its output, then what ran. Exits with the command's code, 6 if the execution failed or was lost, 5 while it still runs. |
|
|
383
384
|
| `files read <id\|key> <path> [--out F]` | Raw bytes to stdout, or to a local file. |
|
|
384
385
|
| `files write <id\|key> <path> [--from F\|-] [--append] [--no-parents] [--if-revision N]` | Atomic write. The default source is stdin. `--if-revision` (0.5.0+): file-first workspaces only, see [Tree revisions](#tree-revisions). |
|
|
@@ -389,7 +390,7 @@ shows the status; `shard version --check` asks now (`--json`: `{"cli", "sdk", "u
|
|
|
389
390
|
| `workspaces suspend\|resume <id\|key> [--wait] [--timeout T] [--timing]` | Lifecycle operation. Returns immediately unless you pass `--wait`. |
|
|
390
391
|
| `workspaces suspend <id\|key> --durable [--timeout T]` | (0.7.0+) Suspend and wait until the copy is in durable storage. See "Durable storage". |
|
|
391
392
|
| `workspaces suspend <id\|key> --when-idle D` / `--cancel-when-idle` | (0.5.1+) Suspend once the workspace has been idle for `D` (0 to 1h; `0` = as soon as it is idle, 0.6.1+), or cancel that. See "Suspend when idle". |
|
|
392
|
-
| `workspaces fork <id\|key> <new-key> [caps] [--wait]` | Fork into a new key. |
|
|
393
|
+
| `workspaces fork <id\|key> <new-key> [caps] [--allocation-mode fixed\|elastic] [--memory-held N] [--wait]` | Fork into a new key. Without caps the fork keeps the source's caps and memory allocation (0.8.0+). With `--wait` the API holds the fork until the copy runs and returns it ready in the same request (0.8.0+; `--timing` shows one `request` phase marked `(held)`). |
|
|
393
394
|
| `workspaces delete <id\|key> --yes [--wait]` | Delete the workspace. Tool access ends at once. After deletion finishes, the key can open a new workspace ID. |
|
|
394
395
|
| `workspaces close <id\|key> [--wait]` | End a session workspace now: it is deleted, and the key then opens a new workspace. A persistent workspace is refused (`not_session`). |
|
|
395
396
|
| `workspaces reset <id\|key> --yes [--wait]` | Layered workspaces: wipe every change and restart on the template. The previous state stays restorable for 7 days. |
|
|
@@ -404,7 +405,7 @@ shows the status; `shard version --check` asks now (`--json`: `{"cli", "sdk", "u
|
|
|
404
405
|
| `secrets versions <id\|NAME>`, `secrets delete <id\|NAME> --yes`, `secrets access-log <id\|NAME>` | Versions, deletion (also removes the name from every workspace binding), and the access log. |
|
|
405
406
|
| `operations list <id\|key> [--state S] [--kind K]`, `operations get <op>`, `operations wait <op> [--timeout T] [--timing]` | Operations. |
|
|
406
407
|
| `templates list [--owner platform\|organization] [--include-archived] [--all]` | Templates the project can open, with their open version, layouts, file list state and draft. |
|
|
407
|
-
| `templates get <slug> [--owner O] [--include-archived]` | One template (0.4.0+): its versions, what open resolves to, and the open version's settings (env, inputs, start commands, services, defaults). |
|
|
408
|
+
| `templates get <slug> [--owner O] [--include-archived]` | One template (0.4.0+): its versions, what open resolves to, and the open version's settings (env, inputs, start commands, services, defaults). Each version's [immutable paths](#immutable-paths-080) and their image size (0.8.0+). |
|
|
408
409
|
| `templates init --base <slug[@version]> [--out F] [--force]` | Write a starter `template.yaml` on a base (0.4.0+). |
|
|
409
410
|
| `templates build <file> [--slug S] [--display-name N] [--description T] [--publish] [--acknowledge PATH]... [--wait] [--timeout 30m] [--timing]` | Build a template from `template.yaml` (0.4.0+): local files and folders are uploaded, then the build is created. See [Build a template](#build-a-template-from-templateyaml). |
|
|
410
411
|
| `templates export <slug>@<version> [--out F] [--force]` | The `template.yaml` a version was built from, ready to build again (0.4.0+). |
|
|
@@ -475,6 +476,30 @@ settings:
|
|
|
475
476
|
- A file `shard` cannot read or pack (a symlink that leaves the folder, a device) is refused before any request (exit
|
|
476
477
|
2). The API's refusals (`upload_missing`, `language_unavailable`, `invalid_settings`, ...) exit 2 with the reason.
|
|
477
478
|
|
|
479
|
+
### Immutable paths (0.8.0+)
|
|
480
|
+
|
|
481
|
+
Directories listed under `immutable` follow the template: every workspace of it mounts them read-only from the
|
|
482
|
+
template's newest published version, picked up at its next cold boot or resume. Ship tools, models or data there and
|
|
483
|
+
existing workspaces get each new version; everything else in a workspace stays its own.
|
|
484
|
+
|
|
485
|
+
```yaml
|
|
486
|
+
immutable: [/opt/acme]
|
|
487
|
+
```
|
|
488
|
+
|
|
489
|
+
```sh
|
|
490
|
+
shard templates get acme-dev
|
|
491
|
+
# VERSION STATE SOURCE LAYOUTS IMMUTABLE PATHS CREATED DESCRIPTION
|
|
492
|
+
# v2 * published recipe layered /opt/acme (96.0 MiB) 2026-10-01 03:14:52Z tools 2.4
|
|
493
|
+
shard ws get acme/main
|
|
494
|
+
# template acme-dev v1 (<version id>)
|
|
495
|
+
# immutable paths v2 (mounted read-only)
|
|
496
|
+
```
|
|
497
|
+
|
|
498
|
+
A template.yaml without `immutable` keeps the list of the template's open version, and a new version keeps every path
|
|
499
|
+
of it (`immutable_path_removed` names the ones to add back). `shard templates builds get` lists a build's immutable
|
|
500
|
+
paths; `shard templates export` writes them back into template.yaml. A write under an immutable path
|
|
501
|
+
(`shard files write`) exits 1 with `read_only_path` and a hint.
|
|
502
|
+
|
|
478
503
|
## File-first workspaces
|
|
479
504
|
|
|
480
505
|
(0.5.0+) A file-first workspace has no VM between commands. Its state is a versioned file tree under `/home/user`.
|
|
@@ -617,6 +642,53 @@ shard ws suspend acme/demo --cancel-when-idle
|
|
|
617
642
|
workspace that is not running is refused with `conflict` (`not_running`, exit 1), and a file-first workspace (never
|
|
618
643
|
suspended) with `not_supported_for_mode` (exit 1).
|
|
619
644
|
|
|
645
|
+
## Elastic memory (0.8.0+)
|
|
646
|
+
|
|
647
|
+
Give a workspace a large memory promise while it holds only what it uses. It idles at `--memory-held` (default
|
|
648
|
+
1024 MiB) and grows before each command starts, so builds and test runners see the memory they need from their first
|
|
649
|
+
instruction.
|
|
650
|
+
|
|
651
|
+
```sh
|
|
652
|
+
shard ws open acme/demo --template python-node-browser --memory-mib 8192 --allocation-mode elastic
|
|
653
|
+
shard ws get acme/demo # memory elastic: promised 8192 MiB, held 1024 MiB, plugged +3072 MiB
|
|
654
|
+
shard ws exec acme/demo -v -- npm test # memory grow delivered 1024 -> 4096 MiB in 175 ms
|
|
655
|
+
```
|
|
656
|
+
|
|
657
|
+
The layout takes effect at the workspace's next start. `--json` output carries `memory` on the workspace and
|
|
658
|
+
`memory_grow` on a command that grew it.
|
|
659
|
+
|
|
660
|
+
## Burst execution (0.8.0+)
|
|
661
|
+
|
|
662
|
+
Run one heavy command, such as a cold build or a full test suite, on a larger VM without resizing the workspace:
|
|
663
|
+
`--burst` runs it on a short-lived burst VM over a copy of the workspace and applies its file changes back when it
|
|
664
|
+
exits. Output streams as usual and `shard` exits with the command's exit code.
|
|
665
|
+
|
|
666
|
+
```sh
|
|
667
|
+
shard ws exec acme/demo --burst --burst-vcpus 16 -v -- bash -lc "cd app && go build ./..."
|
|
668
|
+
# burst local 16 vCPU / 8192 MiB (layer); applied 3 files, 1 dirs, 1 removed, 40960 bytes; overhead 641 ms
|
|
669
|
+
```
|
|
670
|
+
|
|
671
|
+
`--burst-vcpus` and `--burst-memory` size the burst VM (default 8 vCPU and 8192 MiB within the plan's
|
|
672
|
+
per-workspace limits); `--json` adds `burst` to the result.
|
|
673
|
+
|
|
674
|
+
## Background commands (0.8.0+)
|
|
675
|
+
|
|
676
|
+
A long command (a build, a test suite, a training run, a server) can run in the background of a processful workspace
|
|
677
|
+
while you or your agent do other work:
|
|
678
|
+
|
|
679
|
+
```sh
|
|
680
|
+
shard ws exec-start acme/demo --session-id build-1 --timeout 2h -- bash -lc "make -C /home/user/app"
|
|
681
|
+
shard ws exec-read acme/demo build-1 # the output so far; exit 5 while it runs
|
|
682
|
+
shard ws exec-read acme/demo build-1 --follow # stream to the end; exits with the command's exit code
|
|
683
|
+
shard ws exec-cancel acme/demo build-1 --grace 10s
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
- `--timeout` is the longest the command may run; the workspace stays awake until it ends or that time passes.
|
|
687
|
+
- `exec-read` prints stdout and stderr from the start. `--follow` resumes from byte offsets after a dropped
|
|
688
|
+
connection; Ctrl-C stops following and the command keeps running. `--json` prints one object with the state, exit
|
|
689
|
+
code and output.
|
|
690
|
+
- `exec-cancel` sends SIGTERM, then SIGKILL after `--grace` (default 5s, at most 60s).
|
|
691
|
+
|
|
620
692
|
## Start deadlines
|
|
621
693
|
|
|
622
694
|
A queued open, resume or fork (`capacity_pending`) has a deadline 15 minutes after it was created.
|
|
@@ -788,10 +860,10 @@ printf %s "$NEW_KEY" | shard secrets rotate OPENAI_API_KEY
|
|
|
788
860
|
| 2 | usage error: bad arguments, or input the API rejected as `validation_failed` |
|
|
789
861
|
| 3 | not found: workspace, operation or route |
|
|
790
862
|
| 4 | authentication or authorization: missing, malformed, revoked or insufficient key or session; `step_up_required` (run `shard auth step-up`), `mfa_required` (a sign-in waiting for its code), `email_unverified`, `invalid_credentials` |
|
|
791
|
-
| 5 | timeout: waiting gave up (`--timeout`, or `--wake-timeout` while waking a workspace); the operation continues server side (`shard operations wait <id>`), up to a queued start's 15-minute deadline. `executions get` (0.5.0+): the execution is still queued or running. `billing upgrade --wait` (0.5.0+): nobody paid in time (the checkout stays open) |
|
|
863
|
+
| 5 | timeout: waiting gave up (`--timeout`, or `--wake-timeout` while waking a workspace); the operation continues server side (`shard operations wait <id>`), up to a queued start's 15-minute deadline. `executions get` (0.5.0+): the execution is still queued or running. `ws exec-read` (0.8.0+): the background command still runs. `billing upgrade --wait` (0.5.0+): nobody paid in time (the checkout stays open) |
|
|
792
864
|
| 6 | the awaited operation ended `failed` or `canceled` (also a resume or open that failed while waking a workspace). `retryable=true` in the error means a retry can succeed (`capacity_unavailable`). Also a file-first execution that ended `failed` or `lost` (0.5.0+): nothing was published. `exec` (0.5.1+): the command could not start (`exec_failed_to_start`). `billing upgrade --wait` (0.5.0+): the checkout expired or was canceled (`checkout_ended`) |
|
|
793
865
|
| 130 | interrupted (Ctrl-C). Waits stop at once, and exec sends a cancel to the session. A file-first execution runs to completion, and the error names the `shard executions get` command for its result. |
|
|
794
|
-
| exec, executions get | the command's own code: 124 if it timed out, 128+N if killed by signal N. Failures before the command ran use the codes above; use `--json` to tell them apart. |
|
|
866
|
+
| exec, executions get, ws exec-read | the command's own code: 124 if it timed out, 128+N if killed by signal N. Failures before the command ran use the codes above; use `--json` to tell them apart. |
|
|
795
867
|
|
|
796
868
|
## Durable storage (0.7.0+)
|
|
797
869
|
|
package/dist/args.d.ts
CHANGED
package/dist/args.js
CHANGED
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
import { parseArgs } from 'node:util';
|
|
13
13
|
import { ACCOUNT_COMMANDS } from "./account-args.js";
|
|
14
14
|
import { UsageError } from "./errors.js";
|
|
15
|
-
export const CLI_VERSION = '0.
|
|
15
|
+
export const CLI_VERSION = '0.8.0';
|
|
16
16
|
export const GLOBAL_OPTIONS = {
|
|
17
17
|
'api-url': { type: 'string', value: '<url>', description: 'API base URL (default $SHARDFLUX_API_URL, else https://api.shardflux.dev).' },
|
|
18
18
|
json: { type: 'boolean', description: 'Print machine-readable JSON on stdout (errors as JSON on stderr).' },
|
|
@@ -27,6 +27,15 @@ const CAPS = {
|
|
|
27
27
|
'memory-mib': { type: 'string', value: '<n>', description: 'Memory cap in MiB.' },
|
|
28
28
|
'disk-gib': { type: 'string', value: '<n>', description: 'Disk cap in GiB.' },
|
|
29
29
|
};
|
|
30
|
+
/** Elastic memory (0.8.0+): the memory allocation of a workspace's caps (open and fork only). */
|
|
31
|
+
const MEMORY_ALLOCATION = {
|
|
32
|
+
'allocation-mode': {
|
|
33
|
+
type: 'string',
|
|
34
|
+
value: '<fixed|elastic>',
|
|
35
|
+
description: 'elastic: --memory-mib is a promise; the workspace holds --memory-held while idle and the host grows it when a command needs it (needs the elastic memory entitlement). Takes effect at the next VM start. Caps given without it make the workspace fixed; a command without caps keeps the stored layout.',
|
|
36
|
+
},
|
|
37
|
+
'memory-held': { type: 'string', value: '<mib>', description: 'With --allocation-mode elastic: memory held while idle, in MiB (at least 512, default 1024, at most --memory-mib).' },
|
|
38
|
+
};
|
|
30
39
|
/** `--timing` (0.3.0+): the SDK's lifecycle timing of the command's open, wait or wake (README "Timing"). */
|
|
31
40
|
const TIMING = {
|
|
32
41
|
timing: {
|
|
@@ -209,6 +218,7 @@ const KEY_COMMANDS = [
|
|
|
209
218
|
},
|
|
210
219
|
...INPUTS,
|
|
211
220
|
...CAPS,
|
|
221
|
+
...MEMORY_ALLOCATION,
|
|
212
222
|
wait: { type: 'boolean', description: 'Wait until ready (default; --no-wait returns at once). Ready includes the template’s start commands and services.' },
|
|
213
223
|
timeout: { type: 'string', value: '<duration>', description: 'Give up waiting after this long (default 5m). The start continues server side; one waiting for capacity gives up 15 minutes after it began (capacity_unavailable, retryable).' },
|
|
214
224
|
...TIMING,
|
|
@@ -221,6 +231,7 @@ const KEY_COMMANDS = [
|
|
|
221
231
|
'shard ws open acme/demo --template python-node-browser --no-wait --json',
|
|
222
232
|
'shard ws open acme/demo --template python-node-browser --timing',
|
|
223
233
|
'shard ws open acme/agent-7 --template python-node-browser --mode file-first',
|
|
234
|
+
'shard ws open acme/demo --template python-node-browser --memory-mib 8192 --allocation-mode elastic --memory-held 1024',
|
|
224
235
|
],
|
|
225
236
|
},
|
|
226
237
|
{
|
|
@@ -245,11 +256,38 @@ const KEY_COMMANDS = [
|
|
|
245
256
|
policy: { type: 'string', value: '<policy>', description: 'adaptive, never, fixed:<seconds>, or default to clear the override.' },
|
|
246
257
|
keepalive: { type: 'string', value: '<seconds>', description: 'Declare ongoing work for this many seconds.' },
|
|
247
258
|
} },
|
|
248
|
-
{
|
|
259
|
+
{
|
|
260
|
+
path: ['workspaces', 'exec-start'],
|
|
261
|
+
summary: 'Start a background exec session with optional pipe stdin',
|
|
262
|
+
description: 'Runs argv directly (no shell; use `-- bash -lc "..."` for shell syntax) and returns the session at once (JSON on stdout). Read it with "shard ws exec-read", stop it with "shard ws exec-cancel".',
|
|
263
|
+
positionals: [REF, { name: 'command', required: true, variadic: true }],
|
|
264
|
+
options: {
|
|
249
265
|
'stdin-open': { type: 'boolean', description: 'Keep pipe stdin open for exec-input.' },
|
|
250
266
|
'session-id': { type: 'string', value: '<id>', description: 'Caller-chosen id to deduplicate starts.' },
|
|
251
267
|
cwd: { type: 'string', value: '<dir>', description: 'Absolute working directory.' },
|
|
252
|
-
|
|
268
|
+
timeout: { type: 'string', value: '<duration>', description: 'Kill the command after this long (e.g. 30m, 2h; none unless set). The workspace stays awake until the command ends or this passes (1 hour when unset).' },
|
|
269
|
+
},
|
|
270
|
+
examples: ['shard ws exec-start acme/demo --session-id build-1 --timeout 2h -- bash -lc "make -C /home/user/app"', 'shard ws exec-read acme/demo build-1 --follow'],
|
|
271
|
+
},
|
|
272
|
+
{
|
|
273
|
+
path: ['workspaces', 'exec-read'],
|
|
274
|
+
summary: 'Print a background exec session’s output and state; --follow streams it until the command exits',
|
|
275
|
+
description: 'Prints the session’s stdout and stderr from the start. When the command has ended, shard exits with its exit code (124 if it timed out, 128+N if killed by signal N); while it still runs, exit 5. --follow keeps streaming until it ends and resumes from byte offsets after dropped connections; Ctrl-C stops following, the command keeps running. --json prints one object at the end.',
|
|
276
|
+
positionals: [REF, { name: 'session-id', required: true }],
|
|
277
|
+
options: {
|
|
278
|
+
follow: { type: 'boolean', short: 'f', description: 'Stream the output until the command exits, then exit with its exit code.' },
|
|
279
|
+
},
|
|
280
|
+
examples: ['shard ws exec-read acme/demo build-1', 'shard ws exec-read acme/demo build-1 --follow', 'shard ws exec-read acme/demo build-1 --json'],
|
|
281
|
+
},
|
|
282
|
+
{
|
|
283
|
+
path: ['workspaces', 'exec-cancel'],
|
|
284
|
+
summary: 'Stop a background exec session: SIGTERM, then SIGKILL after the grace',
|
|
285
|
+
positionals: [REF, { name: 'session-id', required: true }],
|
|
286
|
+
options: {
|
|
287
|
+
grace: { type: 'string', value: '<duration>', description: 'Time between SIGTERM and SIGKILL (e.g. 10s; at most 60s; default 5s).' },
|
|
288
|
+
},
|
|
289
|
+
examples: ['shard ws exec-cancel acme/demo build-1', 'shard ws exec-cancel acme/demo build-1 --grace 30s'],
|
|
290
|
+
},
|
|
253
291
|
{ path: ['workspaces', 'exec-input'], summary: 'Write pipe stdin at its acknowledged byte offset', positionals: [REF, { name: 'session-id', required: true }, { name: 'data' }], options: {
|
|
254
292
|
offset: { type: 'string', value: '<bytes>', description: 'Required acknowledged offset (0 initially).' },
|
|
255
293
|
close: { type: 'boolean', description: 'Send EOF after these bytes are accepted.' },
|
|
@@ -272,10 +310,16 @@ const KEY_COMMANDS = [
|
|
|
272
310
|
description: 'File-first: the execution’s id (8-128 of A-Z a-z 0-9 . _ : -, starting with a letter or digit; default a fresh ex-<uuid>). Running the same command again with the same id returns its recorded result instead of running it twice, so scripts can repeat the call safely.',
|
|
273
311
|
},
|
|
274
312
|
'output-limit': { type: 'string', value: '<bytes>', description: 'File-first: keep at most this many bytes of stdout and of stderr each (1-16777216; default 1 MiB).' },
|
|
313
|
+
burst: {
|
|
314
|
+
type: 'boolean',
|
|
315
|
+
description: 'Processful: run this command on a larger, short-lived burst VM on the workspace’s host and apply its file changes back when it exits (needs the burst execution entitlement). Its processes and tmpfs content do not come back; the workspace’s own processes pause meanwhile. Not with --stdin; cannot be canceled.',
|
|
316
|
+
},
|
|
317
|
+
'burst-vcpus': { type: 'string', value: '<n>', description: 'With --burst: the burst VM’s vCPUs (1-32; default the host’s, 8, or the plan’s ceiling when lower).' },
|
|
318
|
+
'burst-memory': { type: 'string', value: '<mib>', description: 'With --burst: the burst VM’s memory in MiB (512-65536; default the host’s, 8192, or the plan’s ceiling when lower).' },
|
|
275
319
|
verbose: {
|
|
276
320
|
type: 'boolean',
|
|
277
321
|
short: 'v',
|
|
278
|
-
description: 'After the output, print on stderr what ran: file-first: execution id, state, base -> tree revision, replayed, timings and the changed paths (A|M|D path); processful: the session id and exit status.',
|
|
322
|
+
description: 'After the output, print on stderr what ran: file-first: execution id, state, base -> tree revision, replayed, timings and the changed paths (A|M|D path); processful: the session id and exit status (and with --burst the burst: size, method, applied changes, overhead).',
|
|
279
323
|
},
|
|
280
324
|
...WAKE_TIMING,
|
|
281
325
|
},
|
|
@@ -284,6 +328,7 @@ const KEY_COMMANDS = [
|
|
|
284
328
|
'shard ws exec acme/demo --cwd /home/user/app --env CI=1 -- bash -lc "npm test"',
|
|
285
329
|
'shard ws exec acme/agent-7 -v --execution-id build-42 --timeout 10m -- bash -lc "make -C /home/user/app"',
|
|
286
330
|
'shard ws exec acme/agent-7 --json -- python3 /home/user/report.py > result.json',
|
|
331
|
+
'shard ws exec acme/demo --burst --burst-vcpus 16 -v -- bash -lc "cd /home/user/app && go build ./..."',
|
|
287
332
|
],
|
|
288
333
|
},
|
|
289
334
|
{
|
|
@@ -304,7 +349,7 @@ const KEY_COMMANDS = [
|
|
|
304
349
|
path: ['workspaces', 'fork'],
|
|
305
350
|
summary: 'Fork a workspace into a new key',
|
|
306
351
|
positionals: [REF, { name: 'new-key', required: true }],
|
|
307
|
-
options: { ...CAPS, ...WAIT },
|
|
352
|
+
options: { ...CAPS, ...MEMORY_ALLOCATION, ...WAIT },
|
|
308
353
|
},
|
|
309
354
|
{
|
|
310
355
|
path: ['workspaces', 'delete'],
|
package/dist/commands.js
CHANGED
|
@@ -8,15 +8,15 @@
|
|
|
8
8
|
import { randomUUID } from 'node:crypto';
|
|
9
9
|
import { access, readFile, writeFile } from 'node:fs/promises';
|
|
10
10
|
import { basename, dirname, resolve as resolvePath } from 'node:path';
|
|
11
|
-
import { EXECUTION_ID, FEEDBACK_CATEGORIES, FEEDBACK_MESSAGE_MAX_LENGTH, NotSupportedForModeError, SDK_VERSION, ShardfluxApiError, newExecutionId } from '@shardflux/sdk';
|
|
11
|
+
import { EXECUTION_ID, ExecStartError, FEEDBACK_CATEGORIES, FEEDBACK_MESSAGE_MAX_LENGTH, NotSupportedForModeError, SDK_VERSION, ShardfluxApiError, ShardfluxProtocolError, newExecutionId } from '@shardflux/sdk';
|
|
12
12
|
import { parse as parseYaml } from 'yaml';
|
|
13
13
|
import { CLI_VERSION, SECRET_NAME, UUID, parseDuration, parseEnvPairs, parsePositiveInt } from "./args.js";
|
|
14
|
-
import { ExecutionFailedError, ExecutionPendingError, InterruptedError, NotFoundError, UsageError } from "./errors.js";
|
|
14
|
+
import { ExecSessionRunningError, ExecutionFailedError, ExecutionPendingError, InterruptedError, NotFoundError, UsageError } from "./errors.js";
|
|
15
15
|
import { updateCheckDisabled } from "./update-check.js";
|
|
16
16
|
import { initTemplateYaml, recipeYaml } from "./template-yaml.js";
|
|
17
17
|
import { BuildTimer, buildTimingJson, buildTimingText, progressLine } from "./timing.js";
|
|
18
18
|
import { timingJson } from "./timing.js";
|
|
19
|
-
import { bindingsText, buildTable, buildText, changesTable, diffSummaryText, diffTable, draftText, draftStateTable, executionJson, executionSummary, fileInfoText, fileTable, languageTable, json, kv, modeText, operationDetail, pairs, operationTable, packageTable, searchText, secretAccessTable, secretDetail, secretTable, secretVersionTable, sessionTable, durableText, suspendRequestText, suspendWhenIdleText, templateDetailText, templateEntryText, templateFileTable, templateTable, usageEstimateText, usageSeriesText, usageText, workspaceDetail, workspaceTable, } from "./format.js";
|
|
19
|
+
import { bindingsText, buildTable, buildText, changesTable, diffSummaryText, diffTable, draftText, draftStateTable, executionJson, executionSummary, fileInfoText, fileTable, languageTable, json, kv, memoryGrowText, burstText, modeText, operationDetail, pairs, operationTable, packageTable, searchText, secretAccessTable, secretDetail, secretTable, secretVersionTable, sessionTable, durableText, suspendRequestText, suspendWhenIdleText, templateDetailText, templateEntryText, templateFileTable, templateTable, usageEstimateText, usageSeriesText, usageText, workspaceDetail, workspaceTable, } from "./format.js";
|
|
20
20
|
const DEFAULT_WAIT_MS = 300_000;
|
|
21
21
|
const MAX_JSON_OUTPUT = 16 * 1024 * 1024;
|
|
22
22
|
const MAX_STDIN = 1024 * 1024;
|
|
@@ -66,6 +66,47 @@ function caps(values) {
|
|
|
66
66
|
c.disk_gib = parsePositiveInt(disk, 'disk-gib', 1_000_000);
|
|
67
67
|
return Object.keys(c).length ? c : undefined;
|
|
68
68
|
}
|
|
69
|
+
const ALLOCATION_MODES = ['fixed', 'elastic'];
|
|
70
|
+
/**
|
|
71
|
+
* The caps of `workspaces open` and `workspaces fork` (0.8.0): the resource caps plus the memory allocation. Given caps
|
|
72
|
+
* replace the stored ones, so they are sent only when a flag was given.
|
|
73
|
+
*/
|
|
74
|
+
function workspaceCaps(values) {
|
|
75
|
+
const c = caps(values) ?? {};
|
|
76
|
+
const mode = oneOf(values, 'allocation-mode', ALLOCATION_MODES);
|
|
77
|
+
const held = str(values['memory-held']);
|
|
78
|
+
if (held !== undefined && mode !== 'elastic')
|
|
79
|
+
throw new UsageError('--memory-held needs --allocation-mode elastic (a fixed workspace holds all of --memory-mib)');
|
|
80
|
+
if (mode !== undefined)
|
|
81
|
+
c.allocation_mode = mode;
|
|
82
|
+
if (held !== undefined)
|
|
83
|
+
c.memory_mib_held = parsePositiveInt(held, 'memory-held', 16_777_216);
|
|
84
|
+
return Object.keys(c).length ? c : undefined;
|
|
85
|
+
}
|
|
86
|
+
/**
|
|
87
|
+
* `--burst [--burst-vcpus N] [--burst-memory MIB]` of `workspaces exec` (0.8.0): the exec.run options,
|
|
88
|
+
* or undefined without --burst.
|
|
89
|
+
*/
|
|
90
|
+
function burstOptions(values, withStdin) {
|
|
91
|
+
const vcpus = str(values['burst-vcpus']);
|
|
92
|
+
const mem = str(values['burst-memory']);
|
|
93
|
+
if (values.burst !== true) {
|
|
94
|
+
if (vcpus !== undefined || mem !== undefined)
|
|
95
|
+
throw new UsageError('--burst-vcpus and --burst-memory need --burst');
|
|
96
|
+
return undefined;
|
|
97
|
+
}
|
|
98
|
+
if (withStdin)
|
|
99
|
+
throw new UsageError('--burst runs the command to completion without stdin; drop --stdin');
|
|
100
|
+
const o = { burst: 'always' };
|
|
101
|
+
if (vcpus !== undefined)
|
|
102
|
+
o.burstVcpus = parsePositiveInt(vcpus, 'burst-vcpus', 32);
|
|
103
|
+
if (mem !== undefined) {
|
|
104
|
+
o.burstMemoryMib = parsePositiveInt(mem, 'burst-memory', 65_536);
|
|
105
|
+
if (o.burstMemoryMib < 512)
|
|
106
|
+
throw new UsageError('--burst-memory must be between 512 and 65536');
|
|
107
|
+
}
|
|
108
|
+
return o;
|
|
109
|
+
}
|
|
69
110
|
/** `--when-idle <duration>`: whole seconds from 0 (as soon as it is idle; 0.6.1) to 3600 (the API's after_seconds bounds). */
|
|
70
111
|
export function whenIdleSeconds(text) {
|
|
71
112
|
if (/^0+(?:\.0+)?(?:ms|s|m|h)?$/.test(text.trim()))
|
|
@@ -414,6 +455,75 @@ export function execExitCode(r) {
|
|
|
414
455
|
return 128 + r.termSignal;
|
|
415
456
|
return 1;
|
|
416
457
|
}
|
|
458
|
+
/** The longest cancel grace the workspace applies (it waits 5 s without one). */
|
|
459
|
+
const MAX_CANCEL_GRACE_MS = 60_000;
|
|
460
|
+
/** Error codes of a burst's own failure (an output stream's final error event; the session carries it too). */
|
|
461
|
+
const BURST_FAILURES = new Set(['burst_unavailable', 'burst_apply_failed']);
|
|
462
|
+
/**
|
|
463
|
+
* `shard ws exec-read` (0.8.0): an exec session's output from offset 0, and the session. Without `follow`, the output
|
|
464
|
+
* that exists now; with it, until the session ends, resuming from the byte offsets read after a dropped stream.
|
|
465
|
+
* `maxBytes` of each stream are kept for the result (the rest is counted); `onOutput` gets every new chunk.
|
|
466
|
+
*/
|
|
467
|
+
async function readSession(cell, sessionId, opts) {
|
|
468
|
+
let session = await cell.exec.get(sessionId);
|
|
469
|
+
const kept = { stdout: [], stderr: [] };
|
|
470
|
+
const size = { stdout: 0, stderr: 0 };
|
|
471
|
+
let total = { stdout: 0, stderr: 0 };
|
|
472
|
+
let reconnects = 0;
|
|
473
|
+
const live = (s) => s.state === 'starting' || s.state === 'running';
|
|
474
|
+
for (;;) {
|
|
475
|
+
let ended;
|
|
476
|
+
try {
|
|
477
|
+
const events = await cell.exec.output(sessionId, { stdoutOffset: total.stdout, stderrOffset: total.stderr, follow: opts.follow && live(session), signal: opts.signal });
|
|
478
|
+
for await (const ev of events) {
|
|
479
|
+
if (ev.type === 'output' && ev.data !== undefined) {
|
|
480
|
+
const stream = ev.stream === 'stderr' ? 'stderr' : 'stdout';
|
|
481
|
+
const bytes = Buffer.from(ev.data, 'base64');
|
|
482
|
+
const start = ev.offset ?? total[stream];
|
|
483
|
+
// A reconnect may resend bytes already printed.
|
|
484
|
+
const fresh = start < total[stream] ? bytes.subarray(Math.min(total[stream] - start, bytes.length)) : bytes;
|
|
485
|
+
if (fresh.length === 0)
|
|
486
|
+
continue;
|
|
487
|
+
total = { ...total, [stream]: Math.max(total[stream], start + bytes.length) };
|
|
488
|
+
const room = opts.maxBytes - size[stream];
|
|
489
|
+
if (room > 0) {
|
|
490
|
+
kept[stream].push(fresh.subarray(0, room));
|
|
491
|
+
size[stream] += Math.min(room, fresh.length);
|
|
492
|
+
}
|
|
493
|
+
opts.onOutput?.(stream, fresh);
|
|
494
|
+
}
|
|
495
|
+
else if (ev.type === 'exit') {
|
|
496
|
+
ended = ev.session ?? (await cell.exec.get(sessionId));
|
|
497
|
+
break;
|
|
498
|
+
}
|
|
499
|
+
else if (ev.type === 'error' && ev.error && !BURST_FAILURES.has(ev.error.error.code)) {
|
|
500
|
+
throw new ShardfluxApiError(502, ev.error, 'cell');
|
|
501
|
+
}
|
|
502
|
+
}
|
|
503
|
+
}
|
|
504
|
+
catch (err) {
|
|
505
|
+
if (opts.signal.aborted)
|
|
506
|
+
throw err;
|
|
507
|
+
const transient = !(err instanceof ShardfluxApiError) || err.retryable;
|
|
508
|
+
if (!opts.follow || !transient || reconnects >= 10)
|
|
509
|
+
throw err;
|
|
510
|
+
}
|
|
511
|
+
if (ended)
|
|
512
|
+
session = ended;
|
|
513
|
+
else if (opts.follow) {
|
|
514
|
+
// The stream ended without the exit (a dropped connection, a restarted gateway): continue from the offsets.
|
|
515
|
+
reconnects += 1;
|
|
516
|
+
if (reconnects > 10)
|
|
517
|
+
throw new ShardfluxProtocolError(`exec ${sessionId}: output stream kept dropping`, 0, 'cell');
|
|
518
|
+
await new Promise((r) => setTimeout(r, Math.min(2_000, 100 * 2 ** reconnects)).unref());
|
|
519
|
+
session = await cell.exec.get(sessionId);
|
|
520
|
+
if (live(session) || total.stdout < session.stdout_size || total.stderr < session.stderr_size)
|
|
521
|
+
continue;
|
|
522
|
+
}
|
|
523
|
+
const text = (b) => new TextDecoder().decode(Buffer.concat(b));
|
|
524
|
+
return { session, stdout: text(kept.stdout), stderr: text(kept.stderr), truncated: total.stdout > size.stdout || total.stderr > size.stderr, reconnects };
|
|
525
|
+
}
|
|
526
|
+
}
|
|
417
527
|
async function bestEffortCancel(cell, sessionId) {
|
|
418
528
|
await Promise.race([cell.exec.cancel(sessionId).catch(() => undefined), new Promise((r) => setTimeout(r, 3_000).unref())]);
|
|
419
529
|
}
|
|
@@ -496,7 +606,7 @@ function reportExecution(ctx, ref, r, opts) {
|
|
|
496
606
|
}
|
|
497
607
|
function lifecycleText(verb, w, op) {
|
|
498
608
|
const finished = op.state === 'succeeded' || op.state === 'failed' || op.state === 'canceled';
|
|
499
|
-
return `${verb} ${w.key} (${w.id}): operation ${op.id} ${op.kind} ${op.state}${op.state_reason ? ` (${op.state_reason})` : ''}\n${finished ? durableText(op) : `wait with: shard operations wait ${op.id}\n`}`;
|
|
609
|
+
return `${verb} ${w.key !== undefined ? `${w.key} (${w.id})` : w.id}: operation ${op.id} ${op.kind} ${op.state}${op.state_reason ? ` (${op.state_reason})` : ''}\n${finished ? durableText(op) : `wait with: shard operations wait ${op.id}\n`}`;
|
|
500
610
|
}
|
|
501
611
|
/** The project-key commands; handlers.ts adds the account plane's. */
|
|
502
612
|
export const KEY_HANDLERS = {
|
|
@@ -590,10 +700,78 @@ export const KEY_HANDLERS = {
|
|
|
590
700
|
return 0;
|
|
591
701
|
},
|
|
592
702
|
async 'workspaces exec-start'(ctx, values, [ref, ...argv]) {
|
|
703
|
+
const timeout = str(values.timeout);
|
|
704
|
+
const timeoutMs = timeout === undefined ? undefined : parseDuration(timeout, 'timeout');
|
|
593
705
|
const ws = await resolveWorkspace(ctx, ref);
|
|
594
706
|
const session = await cellOf(ctx, ws).exec.start({ argv, stdin_open: values['stdin-open'] === true,
|
|
595
|
-
...(str(values['session-id']) ? { session_id: str(values['session-id']) } : {}), ...(str(values.cwd) ? { cwd: str(values.cwd) } : {})
|
|
707
|
+
...(str(values['session-id']) ? { session_id: str(values['session-id']) } : {}), ...(str(values.cwd) ? { cwd: str(values.cwd) } : {}),
|
|
708
|
+
...(timeoutMs !== undefined ? { timeout_ms: timeoutMs } : {}) }, ctx.signal);
|
|
596
709
|
out(ctx, JSON.stringify(session, null, 2) + '\n', session);
|
|
710
|
+
// 0.8.0: what to do next (stderr, so stdout stays the session's JSON).
|
|
711
|
+
if (!ctx.json && session.state !== 'failed_to_start')
|
|
712
|
+
ctx.io.stderr.write(`read it with: shard ws exec-read ${ref} ${session.session_id} --follow; stop it with: shard ws exec-cancel ${ref} ${session.session_id}\n`);
|
|
713
|
+
return 0;
|
|
714
|
+
},
|
|
715
|
+
async 'workspaces exec-read'(ctx, values, [ref, sessionId]) {
|
|
716
|
+
const ws = await resolveWorkspace(ctx, ref);
|
|
717
|
+
const cell = cellOf(ctx, ws);
|
|
718
|
+
const follow = values.follow === true;
|
|
719
|
+
let r;
|
|
720
|
+
try {
|
|
721
|
+
r = await readSession(cell, sessionId, {
|
|
722
|
+
follow,
|
|
723
|
+
signal: ctx.signal,
|
|
724
|
+
maxBytes: ctx.json ? MAX_JSON_OUTPUT : 0,
|
|
725
|
+
...(ctx.json ? {} : { onOutput: (stream, chunk) => void (stream === 'stderr' ? ctx.io.stderr : ctx.io.stdout).write(chunk) }),
|
|
726
|
+
});
|
|
727
|
+
}
|
|
728
|
+
catch (err) {
|
|
729
|
+
if (ctx.signal.aborted && follow) {
|
|
730
|
+
throw new InterruptedError(`Stopped following exec session ${sessionId}; the command keeps running.`, undefined, { hint: `stop it with: shard ws exec-cancel ${ref} ${sessionId}` });
|
|
731
|
+
}
|
|
732
|
+
throw err;
|
|
733
|
+
}
|
|
734
|
+
const s = r.session;
|
|
735
|
+
// Nothing ran (a cwd that is not a directory, a program not on PATH): exit 6 with the workspace's reason, as ws exec.
|
|
736
|
+
if (s.burst?.error)
|
|
737
|
+
throw new ShardfluxApiError(409, { error: { code: s.burst.error.code, message: s.burst.error.message, request_id: '', retryable: s.burst.error.retryable, ...(s.burst.error.details ? { details: s.burst.error.details } : {}) } }, 'cell');
|
|
738
|
+
if (s.state === 'failed_to_start')
|
|
739
|
+
throw new ExecStartError(s);
|
|
740
|
+
if (ctx.json) {
|
|
741
|
+
ctx.io.stdout.write(json(withTiming(ctx, {
|
|
742
|
+
session_id: s.session_id,
|
|
743
|
+
state: s.state,
|
|
744
|
+
exit_code: s.exit_code ?? null,
|
|
745
|
+
term_signal: s.term_signal ?? null,
|
|
746
|
+
timed_out: s.timed_out ?? false,
|
|
747
|
+
canceled: s.canceled ?? false,
|
|
748
|
+
stdout: r.stdout,
|
|
749
|
+
stderr: r.stderr,
|
|
750
|
+
truncated: r.truncated,
|
|
751
|
+
reconnects: r.reconnects,
|
|
752
|
+
...(s.error ? { error: s.error } : {}),
|
|
753
|
+
...(s.burst ? { burst: s.burst } : {}),
|
|
754
|
+
})));
|
|
755
|
+
}
|
|
756
|
+
if (s.state === 'starting' || s.state === 'running') {
|
|
757
|
+
throw new ExecSessionRunningError(`Exec session ${s.session_id} is still ${s.state}.`, s, `follow it to its end with: shard ws exec-read ${ref} ${s.session_id} --follow`);
|
|
758
|
+
}
|
|
759
|
+
const result = { exitCode: s.exit_code ?? null, termSignal: s.term_signal ?? null, timedOut: s.timed_out ?? false };
|
|
760
|
+
if (!ctx.json && result.exitCode === null && !result.timedOut && !(result.termSignal !== null && result.termSignal > 0)) {
|
|
761
|
+
ctx.io.stderr.write(`exec session ${s.session_id} ended ${s.state} without an exit code${s.error ? `: ${s.error}` : ''}\n`);
|
|
762
|
+
}
|
|
763
|
+
return execExitCode(result);
|
|
764
|
+
},
|
|
765
|
+
async 'workspaces exec-cancel'(ctx, values, [ref, sessionId]) {
|
|
766
|
+
const grace = str(values.grace);
|
|
767
|
+
const graceMs = grace === undefined ? undefined : parseDuration(grace, 'grace');
|
|
768
|
+
if (graceMs !== undefined && graceMs > MAX_CANCEL_GRACE_MS)
|
|
769
|
+
throw new UsageError(`--grace must be at most 60s (got "${grace}")`);
|
|
770
|
+
const ws = await resolveWorkspace(ctx, ref);
|
|
771
|
+
const s = await cellOf(ctx, ws).exec.cancel(sessionId, graceMs);
|
|
772
|
+
const ended = s.state !== 'starting' && s.state !== 'running';
|
|
773
|
+
const status = !ended ? `${s.state} (cancel sent)` : s.canceled ? `canceled${s.term_signal ? ` (signal ${s.term_signal})` : ''}` : `already ${s.state}${typeof s.exit_code === 'number' ? ` (exit code ${s.exit_code})` : ''}`;
|
|
774
|
+
out(ctx, `exec session ${s.session_id}: ${status}\n`, s);
|
|
597
775
|
return 0;
|
|
598
776
|
},
|
|
599
777
|
async 'workspaces exec-input'(ctx, values, [ref, sessionId, data]) {
|
|
@@ -609,7 +787,7 @@ export const KEY_HANDLERS = {
|
|
|
609
787
|
const template = str(values.template);
|
|
610
788
|
if (!template)
|
|
611
789
|
throw new UsageError('--template <slug> is required for "shard workspaces open"');
|
|
612
|
-
const c =
|
|
790
|
+
const c = workspaceCaps(values);
|
|
613
791
|
const bind = secretNames(strs(values.secret), '--secret');
|
|
614
792
|
const lifetime = oneOf(values, 'lifetime', LIFETIMES);
|
|
615
793
|
const mode = modeFlag(values);
|
|
@@ -699,11 +877,14 @@ export const KEY_HANDLERS = {
|
|
|
699
877
|
const limitText = str(values['output-limit']);
|
|
700
878
|
const outputLimit = limitText === undefined ? undefined : parsePositiveInt(limitText, 'output-limit', MAX_OUTPUT_LIMIT);
|
|
701
879
|
const stdinSource = str(values.stdin);
|
|
880
|
+
const burst = burstOptions(values, stdinSource !== undefined);
|
|
702
881
|
const stdin = stdinSource === undefined ? undefined : await readInput(ctx, stdinSource, MAX_STDIN, '--stdin');
|
|
703
882
|
const ws = await resolveWorkspace(ctx, ref);
|
|
704
883
|
const cell = cellOf(ctx, ws);
|
|
705
884
|
const cwd = str(values.cwd);
|
|
706
885
|
if (ws.mode === 'file_first') {
|
|
886
|
+
if (burst)
|
|
887
|
+
throw new UsageError(`--burst applies to processful workspaces only; ${ws.key} is ${modeText(ws.mode)}`);
|
|
707
888
|
// An execution: answered when the command ended. The id is ours, so Ctrl-C can name it.
|
|
708
889
|
const id = executionId ?? newExecutionId();
|
|
709
890
|
let r;
|
|
@@ -739,6 +920,7 @@ export const KEY_HANDLERS = {
|
|
|
739
920
|
...(Object.keys(env).length ? { env } : {}),
|
|
740
921
|
...(timeoutMs !== undefined ? { timeoutMs } : {}),
|
|
741
922
|
...(stdin !== undefined ? { stdin } : {}),
|
|
923
|
+
...(burst ?? {}),
|
|
742
924
|
maxOutputBytes: ctx.json ? MAX_JSON_OUTPUT : 0,
|
|
743
925
|
...(ctx.json ? {} : { onOutput: (stream, chunk) => void (stream === 'stderr' ? ctx.io.stderr : ctx.io.stdout).write(chunk) }),
|
|
744
926
|
signal: ctx.signal,
|
|
@@ -746,9 +928,14 @@ export const KEY_HANDLERS = {
|
|
|
746
928
|
}
|
|
747
929
|
catch (err) {
|
|
748
930
|
if (ctx.signal.aborted) {
|
|
931
|
+
if (burst)
|
|
932
|
+
throw new InterruptedError(`Interrupted; burst exec session ${sessionId} keeps running server side (a burst cannot be canceled).`);
|
|
749
933
|
await bestEffortCancel(cell, sessionId);
|
|
750
934
|
throw new InterruptedError(`Interrupted; sent cancel to exec session ${sessionId}.`);
|
|
751
935
|
}
|
|
936
|
+
// A burst's apply that did not complete is finished by a retry of the same session (SDK exec.run / the API).
|
|
937
|
+
if (burst && err instanceof ShardfluxApiError && err.code === 'burst_apply_failed')
|
|
938
|
+
ctx.io.stderr.write(`burst exec session ${sessionId}\n`);
|
|
752
939
|
throw err;
|
|
753
940
|
}
|
|
754
941
|
const code = execExitCode(r);
|
|
@@ -757,7 +944,14 @@ export const KEY_HANDLERS = {
|
|
|
757
944
|
const unexplained = r.exitCode === null && !r.timedOut && !(r.termSignal !== null && r.termSignal > 0);
|
|
758
945
|
if (values.verbose === true && !ctx.json) {
|
|
759
946
|
const status = r.timedOut ? 'timed out' : r.canceled ? 'canceled' : r.termSignal ? `killed by signal ${r.termSignal}` : unexplained ? r.session.state : `exit code ${r.exitCode ?? '-'}`;
|
|
760
|
-
ctx.io.stderr.write(kv([
|
|
947
|
+
ctx.io.stderr.write(kv([
|
|
948
|
+
['session', r.sessionId],
|
|
949
|
+
['status', status],
|
|
950
|
+
['output', `${r.stdoutBytes} bytes stdout, ${r.stderrBytes} bytes stderr`],
|
|
951
|
+
['reconnects', String(r.reconnects)],
|
|
952
|
+
...(r.memoryGrow ? [['memory grow', memoryGrowText(r.memoryGrow)]] : []),
|
|
953
|
+
...(r.burst ? [['burst', burstText(r.burst)]] : []),
|
|
954
|
+
]));
|
|
761
955
|
}
|
|
762
956
|
if (ctx.json) {
|
|
763
957
|
ctx.io.stdout.write(json(withTiming(ctx, {
|
|
@@ -772,6 +966,8 @@ export const KEY_HANDLERS = {
|
|
|
772
966
|
truncated: r.truncated,
|
|
773
967
|
reconnects: r.reconnects,
|
|
774
968
|
...(r.session.error ? { error: r.session.error } : {}),
|
|
969
|
+
...(r.memoryGrow ? { memory_grow: r.memoryGrow } : {}),
|
|
970
|
+
...(r.burst ? { burst: r.burst } : {}),
|
|
775
971
|
})));
|
|
776
972
|
}
|
|
777
973
|
else if (unexplained) {
|
|
@@ -899,7 +1095,21 @@ export const KEY_HANDLERS = {
|
|
|
899
1095
|
const cancel = values['cancel-when-idle'] === true;
|
|
900
1096
|
const durable = values.durable === true;
|
|
901
1097
|
if (whenIdle === undefined && !cancel) {
|
|
902
|
-
|
|
1098
|
+
const suspend = (id, o) => ctx.cloud().workspaces.suspend(id, durable ? { ...o, durable: true, wait: o.wait ?? { timeoutMs: waitTimeout(values), signal: ctx.signal } } : o);
|
|
1099
|
+
if (UUID.test(ref)) {
|
|
1100
|
+
// An id needs no resolve request: the suspend goes straight to it (the API refuses a file-first workspace
|
|
1101
|
+
// itself). Only a UUID that is no workspace id here (404, nothing started) is looked up as a key.
|
|
1102
|
+
try {
|
|
1103
|
+
const op = await waited(ctx, 'the operation continues server side.', () => suspend(ref, lifecycleOptions(ctx, values, waitTimeout(values))));
|
|
1104
|
+
out(ctx, lifecycleText('Suspending', { id: ref }, op), op);
|
|
1105
|
+
return 0;
|
|
1106
|
+
}
|
|
1107
|
+
catch (err) {
|
|
1108
|
+
if (!(err instanceof ShardfluxApiError && err.status === 404 && err.operationId === undefined))
|
|
1109
|
+
throw err;
|
|
1110
|
+
}
|
|
1111
|
+
}
|
|
1112
|
+
return lifecycle(ctx, values, ref, 'Suspending', (ws, o) => suspend(ws.id, o), ['suspend', 'Suspend']);
|
|
903
1113
|
}
|
|
904
1114
|
if (durable)
|
|
905
1115
|
throw new UsageError(`${whenIdle !== undefined ? '--when-idle' : '--cancel-when-idle'} cannot be combined with --durable (nothing is suspended now)`);
|
|
@@ -1423,7 +1633,7 @@ export const KEY_HANDLERS = {
|
|
|
1423
1633
|
return 0;
|
|
1424
1634
|
},
|
|
1425
1635
|
async 'workspaces fork'(ctx, values, [ref, newKey]) {
|
|
1426
|
-
const c =
|
|
1636
|
+
const c = workspaceCaps(values);
|
|
1427
1637
|
const timeoutMs = waitTimeout(values);
|
|
1428
1638
|
const source = await resolveWorkspace(ctx, ref);
|
|
1429
1639
|
requireProcessful(source, 'fork', 'A fork');
|
package/dist/errors.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import type { CheckoutStatus, ExecutionResult } from '@shardflux/sdk';
|
|
1
|
+
import type { CheckoutStatus, ExecSession, ExecutionResult } from '@shardflux/sdk';
|
|
2
2
|
export declare const EXIT: {
|
|
3
3
|
readonly ok: 0;
|
|
4
4
|
/** Any other failure: API refusal (conflict, quota, ...), network, protocol. */
|
|
@@ -76,6 +76,15 @@ export declare class ExecutionPendingError extends Error {
|
|
|
76
76
|
readonly hint: string;
|
|
77
77
|
constructor(message: string, result: ExecutionResult, hint: string);
|
|
78
78
|
}
|
|
79
|
+
/**
|
|
80
|
+
* `shard ws exec-read` without --follow (0.8.0): the background command still runs (exit 5, as a file-first execution
|
|
81
|
+
* that still runs); `hint` names the command that follows it to its end.
|
|
82
|
+
*/
|
|
83
|
+
export declare class ExecSessionRunningError extends HintedError {
|
|
84
|
+
name: string;
|
|
85
|
+
readonly session: ExecSession;
|
|
86
|
+
constructor(message: string, session: ExecSession, hint: string);
|
|
87
|
+
}
|
|
79
88
|
export interface ErrorInfo {
|
|
80
89
|
code: string;
|
|
81
90
|
message: string;
|
package/dist/errors.js
CHANGED
|
@@ -97,6 +97,18 @@ export class ExecutionPendingError extends Error {
|
|
|
97
97
|
this.hint = hint;
|
|
98
98
|
}
|
|
99
99
|
}
|
|
100
|
+
/**
|
|
101
|
+
* `shard ws exec-read` without --follow (0.8.0): the background command still runs (exit 5, as a file-first execution
|
|
102
|
+
* that still runs); `hint` names the command that follows it to its end.
|
|
103
|
+
*/
|
|
104
|
+
export class ExecSessionRunningError extends HintedError {
|
|
105
|
+
name = 'ExecSessionRunningError';
|
|
106
|
+
session;
|
|
107
|
+
constructor(message, session, hint) {
|
|
108
|
+
super(message, hint);
|
|
109
|
+
this.session = session;
|
|
110
|
+
}
|
|
111
|
+
}
|
|
100
112
|
const AUTH_CODES = new Set(['unauthenticated', 'invalid_credentials', 'token_invalid', 'forbidden', 'mfa_required', 'email_unverified', 'step_up_required', 'csrf_failed']);
|
|
101
113
|
const KEY_PATTERN = /sfk_[A-Za-z0-9]{8,}_[A-Za-z0-9]+/g;
|
|
102
114
|
/** User session tokens (`sfu_` + 43 base64url characters), also cut short. */
|
|
@@ -163,6 +175,8 @@ export function describeError(err) {
|
|
|
163
175
|
}
|
|
164
176
|
if (err instanceof ExecutionPendingError)
|
|
165
177
|
return { code: 'timeout', message: err.message, execution_id: err.result.executionId, last_state: err.result.state };
|
|
178
|
+
if (err instanceof ExecSessionRunningError)
|
|
179
|
+
return { code: 'timeout', message: err.message, last_state: err.session.state, details: { session_id: err.session.session_id } };
|
|
166
180
|
if (err instanceof UsageError)
|
|
167
181
|
return { code: 'usage', message: err.message };
|
|
168
182
|
if (err instanceof CheckoutTimeoutError) {
|
|
@@ -214,7 +228,7 @@ export function exitCodeFor(err, interrupted = false) {
|
|
|
214
228
|
return EXIT.timeout;
|
|
215
229
|
if (err instanceof OperationFailedError || err instanceof DurabilityLostError || err instanceof ExecutionFailedError || err instanceof CheckoutEndedError)
|
|
216
230
|
return EXIT.operationFailed;
|
|
217
|
-
if (err instanceof ExecutionPendingError)
|
|
231
|
+
if (err instanceof ExecutionPendingError || err instanceof ExecSessionRunningError)
|
|
218
232
|
return EXIT.timeout;
|
|
219
233
|
// A command that could not start exits 6 in both modes, as a failed execution does (0.5.1).
|
|
220
234
|
if (err instanceof ExecStartError)
|
package/dist/format.d.ts
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
|
-
import type { AgentSession, ApiKey, AuditEvent, AuthSessionInfo, BillingCatalog, BillingInvoicePage, BillingSubscription, DataExport, DraftState, ExecutionChange, ExecutionResult, FileInfo, EgressPolicyVersion, Invitation, Member, Organization, Project, ProjectEgressPolicy, Volume, VolumeAttachment, WorkspaceEgressPolicy, FileList, FileSearchResponse, Operation, Secret, SecretAccessEvent, SecretVersion, SpendPolicy, SuspendRequest, TemplateBuild, TemplateDetail, TemplateDiffEntry, TemplateDraft, TemplateFileEntry, TemplateLanguages, TemplatePackagePage, TemplateSettings, TemplateSummary, WorkspaceChange, WorkspaceSecretBindings, WorkspaceView } from '@shardflux/sdk';
|
|
1
|
+
import type { AgentSession, ApiKey, AuditEvent, AuthSessionInfo, BillingCatalog, BillingInvoicePage, BillingSubscription, DataExport, DraftState, ExecutionChange, ExecutionResult, FileInfo, EgressPolicyVersion, Invitation, Member, Organization, Project, ProjectEgressPolicy, Volume, VolumeAttachment, WorkspaceEgressPolicy, FileList, FileSearchResponse, Operation, Secret, SecretAccessEvent, SecretVersion, SpendPolicy, SuspendRequest, TemplateBuild, TemplateDetail, TemplateDiffEntry, TemplateDraft, TemplateFileEntry, TemplateLanguages, TemplatePackagePage, TemplateSettings, TemplateSummary, TemplateVersion, WorkspaceChange, WorkspaceSecretBindings, WorkspaceMemory, WorkspaceView, MemoryGrow, BurstSummary } from '@shardflux/sdk';
|
|
2
2
|
import type { ErrorInfo } from './errors.js';
|
|
3
|
+
/** A size for people: `512 B`, `10.0 KiB`, `128.0 MiB`, `1.50 GiB`. */
|
|
4
|
+
export declare function byteSize(n: number): string;
|
|
3
5
|
export declare function json(value: unknown): string;
|
|
4
6
|
export declare function table(headers: readonly string[], rows: readonly (readonly string[])[]): string;
|
|
5
7
|
export declare function kv(pairs: readonly (readonly [string, string])[]): string;
|
|
@@ -33,6 +35,12 @@ export declare function suspendWhenIdleText(w: {
|
|
|
33
35
|
suspend_request: SuspendRequest | null;
|
|
34
36
|
operation: Operation | null;
|
|
35
37
|
}): string;
|
|
38
|
+
/** `fixed 4096 MiB`, or `elastic: promised 8192 MiB, held 1024 MiB, plugged +2048 MiB` (plugged: above the held floor now). */
|
|
39
|
+
export declare function memoryText(m: WorkspaceMemory): string;
|
|
40
|
+
/** The grow an exec's start waited for: `delivered 1024 -> 4096 MiB in 175 ms`, `partial 1024 -> 3072 of 4096 MiB in 175 ms`. */
|
|
41
|
+
/** The burst line of `ws exec -v` (0.8.0). */
|
|
42
|
+
export declare function burstText(b: BurstSummary): string;
|
|
43
|
+
export declare function memoryGrowText(g: MemoryGrow): string;
|
|
36
44
|
export declare function workspaceDetail(w: WorkspaceView): string;
|
|
37
45
|
export declare const OPERATION_HEADERS: readonly ["ID", "KIND", "STATE", "REASON", "CREATED", "COMPLETED"];
|
|
38
46
|
export declare function operationTable(ops: readonly Operation[]): string;
|
|
@@ -90,6 +98,8 @@ export declare function languageTable(l: TemplateLanguages): string;
|
|
|
90
98
|
export declare function packageTable(page: TemplatePackagePage): string;
|
|
91
99
|
/** A template: identity, what open resolves to, its versions, and the open version's settings. */
|
|
92
100
|
export declare function templateDetailText(t: TemplateDetail): string;
|
|
101
|
+
/** A version's immutable paths and the size of their image: `/opt/acme, /srv/tools (128.0 MiB)`, or `-`. */
|
|
102
|
+
export declare function immutablePathsText(i: TemplateVersion['immutable'] | undefined): string;
|
|
93
103
|
/** A version's settings, one line per part. */
|
|
94
104
|
export declare function settingsText(s: TemplateSettings): string;
|
|
95
105
|
/** How many changed paths the human output lists before "... N more" (--json lists every one the result has). */
|
package/dist/format.js
CHANGED
|
@@ -4,6 +4,10 @@
|
|
|
4
4
|
* these functions only shape what people read.
|
|
5
5
|
*/
|
|
6
6
|
import { durabilityOf, isDurable } from '@shardflux/sdk';
|
|
7
|
+
/** A size for people: `512 B`, `10.0 KiB`, `128.0 MiB`, `1.50 GiB`. */
|
|
8
|
+
export function byteSize(n) {
|
|
9
|
+
return n < 1024 ? `${n} B` : n < 1024 ** 2 ? `${(n / 1024).toFixed(1)} KiB` : n < 1024 ** 3 ? `${(n / 1024 ** 2).toFixed(1)} MiB` : `${(n / 1024 ** 3).toFixed(2)} GiB`;
|
|
10
|
+
}
|
|
7
11
|
export function json(value) {
|
|
8
12
|
return `${JSON.stringify(value, null, 2)}\n`;
|
|
9
13
|
}
|
|
@@ -128,6 +132,32 @@ export function suspendWhenIdleText(w, r) {
|
|
|
128
132
|
const op = r.operation;
|
|
129
133
|
return `A suspend of ${w.key} (${w.id}) is already in progress (operation ${op.id} ${op.kind} ${op.state}); nothing was scheduled.\nwait with: shard operations wait ${op.id}\n`;
|
|
130
134
|
}
|
|
135
|
+
const mib = (n) => (n === null ? '-' : `${n} MiB`);
|
|
136
|
+
/** `fixed 4096 MiB`, or `elastic: promised 8192 MiB, held 1024 MiB, plugged +2048 MiB` (plugged: above the held floor now). */
|
|
137
|
+
export function memoryText(m) {
|
|
138
|
+
if (m.allocation_mode !== 'elastic')
|
|
139
|
+
return `fixed ${mib(m.promised_mib)}`;
|
|
140
|
+
return `elastic: promised ${mib(m.promised_mib)}, held ${mib(m.held_mib)}, plugged ${m.plugged_mib === null ? '- (not running)' : `+${m.plugged_mib} MiB`}`;
|
|
141
|
+
}
|
|
142
|
+
/** The grow an exec's start waited for: `delivered 1024 -> 4096 MiB in 175 ms`, `partial 1024 -> 3072 of 4096 MiB in 175 ms`. */
|
|
143
|
+
/** The burst line of `ws exec -v` (0.8.0). */
|
|
144
|
+
export function burstText(b) {
|
|
145
|
+
const size = [b.vcpus !== undefined ? `${b.vcpus} vCPU` : null, b.memory_mib !== undefined ? `${b.memory_mib} MiB` : null].filter(Boolean).join(' / ');
|
|
146
|
+
const head = `${b.host}${size ? ` ${size}` : ''}${b.method ? ` (${b.method})` : ''}`;
|
|
147
|
+
const changes = b.applied
|
|
148
|
+
? `applied ${b.written_files ?? 0} files, ${b.written_dirs ?? 0} dirs, ${b.removed ?? 0} removed, ${b.written_bytes ?? 0} bytes`
|
|
149
|
+
: 'changes not applied';
|
|
150
|
+
const extra = [
|
|
151
|
+
b.leftover_killed ? `${b.leftover_killed} leftover processes killed` : null,
|
|
152
|
+
b.overhead_ms !== undefined ? `overhead ${b.overhead_ms} ms` : null,
|
|
153
|
+
b.replayed ? 'replayed' : null,
|
|
154
|
+
].filter(Boolean);
|
|
155
|
+
return [head, changes, ...extra].join('; ');
|
|
156
|
+
}
|
|
157
|
+
export function memoryGrowText(g) {
|
|
158
|
+
const to = g.got_mib === g.want_mib ? `${g.got_mib} MiB` : `${g.got_mib} of ${g.want_mib} MiB`;
|
|
159
|
+
return `${g.outcome} ${g.from_mib} -> ${to} in ${g.deliver_ms} ms`;
|
|
160
|
+
}
|
|
131
161
|
export function workspaceDetail(w) {
|
|
132
162
|
const c = w.ceilings;
|
|
133
163
|
const startup = startupText(w.startup);
|
|
@@ -138,6 +168,8 @@ export function workspaceDetail(w) {
|
|
|
138
168
|
['state', `${w.observed_state} (desired ${w.desired_state})`],
|
|
139
169
|
['ready', isReady(w) ? 'yes' : 'no'],
|
|
140
170
|
['template', `${w.template.slug} v${w.template.version} (${w.template.version_id})`],
|
|
171
|
+
// Immutable paths (0.8.0): the template version whose immutable paths the workspace has mounted, when it has one.
|
|
172
|
+
...(typeof w.template.immutable_version === 'number' ? [['immutable paths', `v${w.template.immutable_version} (mounted read-only)`]] : []),
|
|
141
173
|
// The mode (0.5.0) when the API reports it; a file-first workspace's state is its tree revision.
|
|
142
174
|
...(w.mode !== undefined ? [['mode', modeText(w.mode)]] : []),
|
|
143
175
|
...(w.mode === 'file_first' ? [['tree revision', String(w.tree_revision)]] : []),
|
|
@@ -145,6 +177,8 @@ export function workspaceDetail(w) {
|
|
|
145
177
|
['ceilings', c ? `cpu ${c.cpu_millis}m (${c.sources.cpu_millis}), memory ${c.memory_mib} MiB (${c.sources.memory_mib}), disk ${c.disk_gib} GiB (${c.sources.disk_gib})` : '-'],
|
|
146
178
|
['grants', pairs(w.grants)],
|
|
147
179
|
['caps', pairs(w.caps)],
|
|
180
|
+
// Elastic memory (0.8.0) when the API reports it.
|
|
181
|
+
...(w.memory ? [['memory', memoryText(w.memory)]] : []),
|
|
148
182
|
['operation', w.active_operation ? `${operationSummary(w.active_operation)} ${w.active_operation.id}` : '-'],
|
|
149
183
|
['pending', w.pending_reason ?? '-'],
|
|
150
184
|
...(suspendRequest ? [['suspend when idle', suspendRequestText(suspendRequest)]] : []),
|
|
@@ -509,15 +543,30 @@ function buildSource(b) {
|
|
|
509
543
|
const v2 = b.provenance.recipe_schema === 'shardflux.template-recipe.v2';
|
|
510
544
|
return `recipe${v2 ? ' v2' : ''}${b.provenance.recipe_sha256 ? ` (sha256 ${b.provenance.recipe_sha256.slice(0, 12)})` : ''}`;
|
|
511
545
|
}
|
|
546
|
+
/** What to do about a build that failed on its immutable paths (0.8.0; contracts §34.2), from failure.code and details. */
|
|
547
|
+
function immutableFailureFix(f) {
|
|
548
|
+
if (f.code === 'immutable_path_missing') {
|
|
549
|
+
const path = typeof f.details?.path === 'string' ? f.details.path : 'the path';
|
|
550
|
+
return `make ${path} a directory in the build (a step: mkdir -p ${path}), then build again`;
|
|
551
|
+
}
|
|
552
|
+
if (f.code === 'immutable_image_too_large')
|
|
553
|
+
return 'keep large data outside the immutable paths, then build again';
|
|
554
|
+
return null;
|
|
555
|
+
}
|
|
512
556
|
export function buildText(b) {
|
|
513
557
|
const settled = b.state === 'published' || b.state === 'failed' || b.state === 'canceled' || b.state === 'succeeded';
|
|
514
558
|
const denied = b.denied_hosts ?? [];
|
|
559
|
+
const immutable = b.immutable_paths ?? [];
|
|
560
|
+
const fix = b.failure ? immutableFailureFix(b.failure) : null;
|
|
515
561
|
return (kv([
|
|
516
562
|
['build', b.id],
|
|
517
563
|
['template', `${b.template.slug} v${b.target_version ?? '?'}`],
|
|
518
564
|
['state', `${b.state}${b.failure ? ` (${b.failure.code}: ${b.failure.message})` : ''}`],
|
|
565
|
+
...(fix ? [['fix', fix]] : []),
|
|
519
566
|
['registration', `${b.registration.state}${b.registration.error ? ` (${b.registration.error.code}: ${b.registration.error.message})` : ''}`],
|
|
520
567
|
['source', buildSource(b)],
|
|
568
|
+
// Immutable paths (0.8.0): the list the produced version declares.
|
|
569
|
+
...(immutable.length ? [['immutable paths', immutable.join(', ')]] : []),
|
|
521
570
|
['network', b.network.mode === 'none' ? 'none' : `allowlist ${b.network.allow_hosts.join(', ') || '(empty)'}`],
|
|
522
571
|
...(denied.length ? [['denied hosts', `${denied.join(', ')} (allow one: add it to build.network.extra_hosts and build again)`]] : []),
|
|
523
572
|
['version', b.template_version ? `${b.template_version.id}${b.template_version.published ? ' (published)' : ' (unpublished)'}` : '-'],
|
|
@@ -543,11 +592,25 @@ export function templateDetailText(t) {
|
|
|
543
592
|
['draft', t.draft ? `${t.draft.workspace_key} (${t.draft.observed_state})` : '-'],
|
|
544
593
|
['archived', time(t.archived_at)],
|
|
545
594
|
]);
|
|
595
|
+
// Immutable paths (0.8.0): the column appears when a version declares any.
|
|
596
|
+
const immutable = t.versions.some((v) => v.immutable);
|
|
546
597
|
const versions = t.versions.length
|
|
547
|
-
? table(['VERSION', 'STATE', 'SOURCE', 'LAYOUTS',
|
|
598
|
+
? table(['VERSION', 'STATE', 'SOURCE', 'LAYOUTS', ...(immutable ? ['IMMUTABLE PATHS'] : []), 'CREATED', 'DESCRIPTION'], t.versions.map((v) => [
|
|
599
|
+
`v${v.version}${v.is_open_version ? ' *' : ''}`,
|
|
600
|
+
v.state,
|
|
601
|
+
v.source?.kind ?? '-',
|
|
602
|
+
v.disk_layouts.join(',') || '-',
|
|
603
|
+
...(immutable ? [immutablePathsText(v.immutable)] : []),
|
|
604
|
+
time(v.created_at),
|
|
605
|
+
v.description ?? '-',
|
|
606
|
+
]))
|
|
548
607
|
: 'No versions.\n';
|
|
549
608
|
return `${head}\n${versions}${open?.settings ? `\nsettings of v${open.version}:\n${settingsText(open.settings)}` : ''}`;
|
|
550
609
|
}
|
|
610
|
+
/** A version's immutable paths and the size of their image: `/opt/acme, /srv/tools (128.0 MiB)`, or `-`. */
|
|
611
|
+
export function immutablePathsText(i) {
|
|
612
|
+
return i ? `${i.paths.join(', ')} (${byteSize(i.bytes)})` : '-';
|
|
613
|
+
}
|
|
551
614
|
/** A version's settings, one line per part. */
|
|
552
615
|
export function settingsText(s) {
|
|
553
616
|
const inputs = Object.entries(s.inputs).map(([n, i]) => `${n} (${i.kind}${i.required ? ', required' : ''}${i.default !== null ? `, default ${JSON.stringify(i.default)}` : ''})`);
|
package/dist/main.js
CHANGED
|
@@ -79,6 +79,9 @@ function hintFor(info, cred, file) {
|
|
|
79
79
|
const policyHint = spendPolicyHint(info);
|
|
80
80
|
if (policyHint)
|
|
81
81
|
return policyHint;
|
|
82
|
+
const immutable = immutableHint(info);
|
|
83
|
+
if (immutable)
|
|
84
|
+
return immutable;
|
|
82
85
|
// File patches: what to do about each refusal. details.index is the 0-based edit (--old) index.
|
|
83
86
|
const reason = info.details?.reason;
|
|
84
87
|
const index = typeof info.details?.index === 'number' ? ` #${info.details.index + 1}` : '';
|
|
@@ -166,6 +169,35 @@ function spendPolicyHint(info) {
|
|
|
166
169
|
return undefined;
|
|
167
170
|
}
|
|
168
171
|
}
|
|
172
|
+
/**
|
|
173
|
+
* Immutable paths (0.8.0; contracts §34): the build refusals (422 validation_failed per details.reason, with the paths
|
|
174
|
+
* and base the API names) and a files write under an immutable path (409 conflict read_only_path from the cell).
|
|
175
|
+
*/
|
|
176
|
+
function immutableHint(info) {
|
|
177
|
+
const d = info.details ?? {};
|
|
178
|
+
const reason = reasonOf(info);
|
|
179
|
+
const list = (v) => (Array.isArray(v) ? v.filter((p) => typeof p === 'string').join(', ') : '');
|
|
180
|
+
if (info.code === 'conflict' && reason === 'read_only_path') {
|
|
181
|
+
return 'this path is an immutable path of the workspace’s template: read-only, it follows the template’s newest version. Write elsewhere, or change it in the template and build a new version (shard templates get <slug> lists the immutable paths)';
|
|
182
|
+
}
|
|
183
|
+
if (info.code !== 'validation_failed')
|
|
184
|
+
return undefined;
|
|
185
|
+
if (reason === 'immutable_path_removed') {
|
|
186
|
+
const removed = list(d.removed);
|
|
187
|
+
const open = typeof d.open_version === 'number' ? `v${d.open_version}` : 'the open version';
|
|
188
|
+
return `a new version keeps every immutable path of ${open}: add ${removed || 'them'} back to immutable in template.yaml (adding paths is fine)`;
|
|
189
|
+
}
|
|
190
|
+
if (reason === 'immutable_paths_unsupported_base') {
|
|
191
|
+
if (d.field === 'recipe')
|
|
192
|
+
return 'this template has immutable paths: build it from a template.yaml (shard templates export <slug>@<version> writes the current one, shard templates build builds it)';
|
|
193
|
+
const base = typeof d.base === 'object' && d.base !== null ? d.base.name : undefined;
|
|
194
|
+
return typeof base === 'string' ? `build on a newer version of ${base} (shard templates get ${base} lists its versions)` : 'build on a newer version of the base';
|
|
195
|
+
}
|
|
196
|
+
if (reason === 'invalid_path' && typeof d.field === 'string' && d.field.startsWith('recipe.immutable[')) {
|
|
197
|
+
return `fix ${d.field.slice('recipe.'.length)} in template.yaml (the list counts from 0)`;
|
|
198
|
+
}
|
|
199
|
+
return undefined;
|
|
200
|
+
}
|
|
169
201
|
/**
|
|
170
202
|
* 409 conflict host_feature_unavailable (0.5.0): the call is not available for this workspace. Not retryable, so the
|
|
171
203
|
* hint names the fallback that works now.
|
|
@@ -258,6 +290,8 @@ const NO_NUDGE_REASONS = new Set([
|
|
|
258
290
|
'subscription_exists',
|
|
259
291
|
// A stale spend policy (If-Match): the hint names the version to retry with (0.5.1).
|
|
260
292
|
'version_mismatch',
|
|
293
|
+
// A write under an immutable path (0.8.0): the hint says where to change it.
|
|
294
|
+
'read_only_path',
|
|
261
295
|
]);
|
|
262
296
|
/**
|
|
263
297
|
* Whether a failure ends with the `feedback:` line (0.5.0+): every failure except the expected flow above (feedback
|
package/dist/template-yaml.js
CHANGED
|
@@ -20,6 +20,9 @@ export function initTemplateYaml(base) {
|
|
|
20
20
|
# build: what goes into the template (languages, packages, files, steps). settings: what a workspace gets when it opens.
|
|
21
21
|
schema: shardflux.template-recipe.v2
|
|
22
22
|
base: ${base}
|
|
23
|
+
# Immutable paths: directories every workspace of this template mounts read-only from its newest published version
|
|
24
|
+
# (workspaces pick up a new version at their next cold boot or resume), e.g. tools or data you ship with each version.
|
|
25
|
+
# immutable: [/opt/acme]
|
|
23
26
|
build:
|
|
24
27
|
# Languages from the platform's table: python, node, go, rust, java (version omitted: the base's default).
|
|
25
28
|
languages: []
|
package/dist/timing.js
CHANGED
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
* carry no `timing` (network errors, Ctrl-C).
|
|
7
7
|
*/
|
|
8
8
|
import { formatTiming } from '@shardflux/sdk';
|
|
9
|
+
import { byteSize } from "./format.js";
|
|
9
10
|
export class TimingRecorder {
|
|
10
11
|
/** The timing of the last traced call that ended (token fetches are not lifecycle calls and are skipped). */
|
|
11
12
|
last = null;
|
|
@@ -196,7 +197,7 @@ export class BuildTimer {
|
|
|
196
197
|
}
|
|
197
198
|
const fmtMs = (ms) => (ms === null ? '-' : ms < 1000 ? `${Math.round(ms)} ms` : `${(ms / 1000).toFixed(2)} s`);
|
|
198
199
|
const short = (sha) => sha.slice(0, 12);
|
|
199
|
-
const bytes =
|
|
200
|
+
const bytes = byteSize;
|
|
200
201
|
/** One stderr line per build-from-file progress event. */
|
|
201
202
|
export function progressLine(e) {
|
|
202
203
|
if (e.type === 'pack')
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@shardflux/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "shard: the Shardflux command line. Create an account, sign in, set up keys and billing, then open, run commands in, move files to, suspend, resume and fork persistent agent workspaces.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -44,7 +44,7 @@
|
|
|
44
44
|
},
|
|
45
45
|
"dependencies": {
|
|
46
46
|
"yaml": "2.9.1",
|
|
47
|
-
"@shardflux/sdk": "^0.
|
|
47
|
+
"@shardflux/sdk": "^0.13.0"
|
|
48
48
|
},
|
|
49
49
|
"devDependencies": {
|
|
50
50
|
"@eslint/js": "10.0.1",
|