@thermai/flightdeck-cli 0.1.0-pilot.8 → 0.1.0-pilot.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +206 -16
  2. package/dist/index.js +7901 -2608
  3. package/package.json +6 -2
package/README.md CHANGED
@@ -62,12 +62,22 @@ admin view. `flightdeck login` now defaults the credential name to
62
62
  `flightdeck-cli@<hostname>` so a second machine no longer collides (#254), and
63
63
  `flightdeck sbatch` prints a runnable status hint with the submitted flight ID
64
64
  and labels the job ID on the ATC hint (#257, #259). The required backend is
65
- already deployed; there are no new runtime dependencies. Publication requires the
66
- approved `cli-v0.1.0-pilot.8` GitHub prerelease and a successful release workflow.
67
- After publication, use the generated `@next` command below, or pin
65
+ already deployed; there are no new runtime dependencies. It was published to
66
+ npm's `next` channel on 8 October 2026. Use the generated `@next` command below, or pin
68
67
  `@thermai/flightdeck-cli@0.1.0-pilot.8`, and confirm `flightdeck --version`
69
68
  reports `0.1.0-pilot.8`.
70
69
 
70
+ **`0.1.0-pilot.9`** adds managed images and datasets. `flightdeck image check`,
71
+ `push`, `list` and `inspect` check and publish workload images to your
72
+ organization's managed registry. `flightdeck image alias ls|set|rm` and
73
+ `image push --alias` manage channels, and `managed://<name>:<alias>` (including
74
+ the computed `latest`) can be submitted as an image. `flightdeck dataset push`,
75
+ `list` and `rm` manage datasets, and `submit --dataset` mounts them. `sbatch`
76
+ refuses an array `%N` limit Flightdeck cannot enforce unless
77
+ `--ignore-array-limit` is passed. Alias commands need a backend with image
78
+ aliases; the other commands work against the current backend. There are no new
79
+ runtime dependencies.
80
+
71
81
  Members and organization administrators receive whole-credit balances,
72
82
  estimates and actual customer charges, without provider-cost information.
73
83
  Platform administrators retain their authorized monetary API fields. Credits
@@ -162,18 +172,21 @@ warnings. Keep `--dry-run`: this is a translation exercise, not the approved
162
172
  pilot workload or permission to launch it. A dry run does not check provider
163
173
  capacity, account policy/credit, bundle contents, image access or GPU execution.
164
174
 
165
- For the real workload, obtain the reviewed private bundle, exact runner-image
175
+ For the real workload, obtain the reviewed private bundle, exact image
166
176
  digest, provider selection and runtime from the operator. Review its dry run,
167
177
  then use the supplied submission command only after execution approval. Do not
168
178
  copy internal test data into public packages or partner bundles. Use explicit
169
179
  `--files` selections from a clean directory; the submission default includes
170
- the current directory. Never include credentials or private workstation files.
180
+ the current directory, less the default excludes listed under
181
+ [What goes into a bundle](#what-goes-into-a-bundle). Never include credentials
182
+ or private workstation files.
171
183
 
172
184
  ## Workload commands
173
185
 
174
186
  After your account's policy/credit setup and agreed first-run approval:
175
187
 
176
188
  ```sh
189
+ flightdeck image check <image> # before using --base-image
177
190
  flightdeck sbatch workload.sbatch
178
191
  flightdeck status <flight-id>
179
192
  flightdeck logs <flight-id> --follow
@@ -183,17 +196,44 @@ flightdeck cancel <flight-id>
183
196
  ```
184
197
 
185
198
  Submission, including `flightdeck slurm-test`, can provision paid resources.
199
+ `submit` and `sbatch` print the deployment's output size limit first. Everything
200
+ in `/outputs` is stored as one compressed archive, and an archive over the limit
201
+ is not stored at all. `sbatch --dry-run` stays offline and states the limit
202
+ without a figure.
186
203
  `sbatch --dry-run` only translates locally and does not submit. Flightdeck
187
204
  supports a subset of single-node Slurm directives, not an arbitrary HPC cluster.
188
- Workload images must satisfy the Flightdeck runner contract.
205
+ Without `--base-image`, a flight runs on the official Flightdeck base
206
+ (`ghcr.io/thermai-group/flightdeck-base`): Ubuntu 24.04 with Python 3 and pip,
207
+ and no PyTorch or CUDA, so supply Python packages with `--requirements` or use
208
+ your own image. A `--base-image` can be any linux/amd64 image that meets the
209
+ workload runtime contract (see `/docs/images`); it does not need the
210
+ Flightdeck runner. Before charging, submission refuses an image whose
211
+ `ENTRYPOINT` is shell form
212
+ (`shell_form_entrypoint`) or whose `uk.thermai.flightdeck.contract` label names
213
+ a contract version Flightdeck does not run (`image_unsupported_contract`).
214
+ Check an image locally first with `flightdeck image check`; see
215
+ [Checking an image before you submit](#checking-an-image-before-you-submit).
216
+ Environment variables (`--env`, `#SBATCH --export`) are checked before upload:
217
+ names are at most 64 characters, and names starting with `FLIGHTDECK_` belong
218
+ to the platform and are refused.
189
219
 
190
220
  An array prints a separate Flightdeck ID for each accepted task. Record them
191
221
  all: inspect, download and cancel each flight individually (`flightdeck atc
192
222
  <job-id>` shows them together). Partial submission
193
223
  can leave accepted flights running even if the command exits unsuccessfully;
194
- do not blindly resubmit the whole array. Choose a fresh download directory for
195
- each flight: downloads use filename basenames and refuse to overwrite existing
196
- files. Check terminal status and exit code as well as the expected artifacts;
224
+ do not blindly resubmit the whole array. Downloads go to
225
+ `<output-dir>/<flight-id>/attempt-<n>/`, or `legacy/` for results from before
226
+ attempts were recorded, keeping each output's relative path. `--attempt <n>`
227
+ selects an earlier attempt. An interrupted download resumes when you run the same
228
+ command again, and a file only appears once its size and SHA-256 match the stored
229
+ output. Partial data is kept in `<file>.part` until then. A `.part` file that
230
+ the CLI did not leave for that output is refused, never deleted. A matching file is
231
+ skipped and a different one is never overwritten. A second download of the same
232
+ file stops with an error while the first is running. Stored results have no expiry
233
+ today. `outputs` and `download` print a warning on stderr when an output was
234
+ over the size limit or collection ended early, including a cancelled flight
235
+ whose outputs may not have been collected, so a short list isn't mistaken for
236
+ everything. With `--attempt`, only that attempt's size refusals are reported. Check terminal status and exit code as well as the expected artifacts;
197
237
  successful submission or download alone does not mean the workload succeeded.
198
238
 
199
239
  For an incident, stop submitting, cancel the known active flights, and contact
@@ -202,26 +242,176 @@ must confirm provider teardown and reconcile any outstanding charges before a
202
242
  retry. Support contact, supported hours and credit increases come from your
203
243
  agreed pilot handoff; this client does not promise unattended support.
204
244
 
245
+ ### What goes into a bundle
246
+
247
+ `submit` and `sbatch` pack the current directory, or the `--files` paths, into
248
+ the bundle that is unpacked on the node. These are left out by default:
249
+
250
+ - version control: `.git`, `.hg`, `.svn`, `.bzr` (directories or files)
251
+ - environment and credential files: `.env*`, `.netrc`, `.pypirc`, `.npmrc`,
252
+ `.git-credentials`, `.ssh/`, `.aws/`, `.gnupg/`, `.docker/`, `.kube/`,
253
+ `.flightdeck/`
254
+ - key files: `*.pem`, `*.key`, `*.p12`, `*.pfx`, `*.ppk`, `id_rsa`, `id_dsa`,
255
+ `id_ecdsa`, `id_ed25519`, `id_ecdsa_sk`, `id_ed25519_sk`
256
+ - virtualenvs: `venv/`, `.venv/` and any directory containing `pyvenv.cfg`
257
+
258
+ These match regardless of case, so `.ENV` and `ID_RSA` are left out too.
259
+
260
+ A `.flightdeckignore` in the current directory adds rules in `.gitignore`
261
+ syntax, and `!<path>` includes something a default would leave out. Only a `!`
262
+ rule without wildcards lifts a default; one such as `!*/` re-includes only what
263
+ your own rules excluded:
264
+
265
+ ```gitignore
266
+ checkpoints/*
267
+ !checkpoints/final.pt
268
+ !.env.example
269
+ ```
270
+
271
+ As with `.gitignore`, a path under an excluded directory cannot be re-included
272
+ on its own; re-include the directory, or exclude its contents with `dir/*`
273
+ instead. Naming an excluded path with `--files` does not include it (a shell
274
+ glob such as `--files *` expands to every name); use `!<path>` in
275
+ `.flightdeckignore`. A `--files` path that goes through a symlinked directory
276
+ (`alias/credentials` with `alias -> .aws`) is checked under the path it resolves
277
+ to as well, so the `!<path>` must name the resolved path; one that resolves
278
+ outside the current directory is refused. Symlinks that dangle, whose target is not in the bundle
279
+ (such as `data -> /scratch/...`), or whose target is an absolute path are
280
+ skipped with a warning because they cannot resolve on the node; use relative
281
+ links within the directory. To use a dataset that lives outside it, copy it
282
+ in or stage it on the node another way. The file count and size print before upload, and
283
+ `sbatch --dry-run` shows the same summary.
284
+
285
+ ### Checking an image before you submit
286
+
287
+ `flightdeck image check <image>` checks an image with your local Docker, so you
288
+ learn before you submit whether Flightdeck is likely to admit it and how it
289
+ behaves. It
290
+ needs no login. When you are logged in, it reads the platform's current runtime
291
+ facts (the supervisor release and the host drivers) from the API; otherwise it
292
+ uses the ones built into the CLI.
293
+
294
+ ```sh
295
+ flightdeck image check ghcr.io/your-org/your-image:1.0
296
+ flightdeck image check ghcr.io/your-org/your-image:1.0 --json
297
+ ```
298
+
299
+ It reports each check as PASS, WARN, FAIL, INFO, SKIP or ERROR (the check
300
+ could not run), with a one-line reason, and ends with a verdict:
301
+
302
+ - **Submission rules**, the same code the API uses: `linux/amd64`, a docker.io,
303
+ ghcr.io or nvcr.io reference within 255 characters once pinned to its digest,
304
+ and the `uk.thermai.flightdeck.contract` label. If Docker's copy is for
305
+ another platform (Docker on Apple silicon pulls `arm64`), the check asks the
306
+ registry whether it has the `linux/amd64` image Flightdeck runs: if it does,
307
+ that is a warning and the dry run is skipped; if not, it fails. The check
308
+ never re-pulls or retags your copy. A shell-form `ENTRYPOINT` is a
309
+ warning: Flightdeck refuses it for flights with a command, and runs it as it
310
+ is for image-only flights. `tini` or `dumb-init` as the `ENTRYPOINT` is a
311
+ warning too, because the supervisor already does their job.
312
+ - **Registry**: Flightdeck reads the image from its registry, without
313
+ credentials. A local image that is not in its registry yet is a warning, and
314
+ so is one the registry will not serve without credentials (the check asks it
315
+ anonymously, with `docker manifest inspect` and an empty Docker
316
+ configuration). New GitHub packages are private until you make them public.
317
+ - **Dry run**: the image starts the way a flight starts, under the Flightdeck
318
+ supervisor with `/workspace` and `/outputs` mounted, CPU only unless you pass
319
+ `--gpu`. `--gpu` attaches GPUs with Docker's `--gpus`; Flightdeck's hosts use
320
+ CDI, so the dry run does not test a provider's GPUs or driver. It runs a command as the image's `USER`; an `ENTRYPOINT` that ignores
321
+ the command fails. It installs a requirements file with no packages in it; a
322
+ missing or unusable `pip` is a warning, since only a bundle with a
323
+ requirements file needs one. It stops a running command with `SIGTERM` and
324
+ expects it to exit within the 30-second grace. A missing `bash` is a warning,
325
+ because commands then run under `sh` and sbatch scripts need `bash`.
326
+ - **CUDA**, from `CUDA_VERSION`, `libcudart`, pip's CUDA runtime wheel and
327
+ `torch.version.cuda`: a CUDA newer than every published host driver runs is a
328
+ warning. An image without CUDA is fine.
329
+ - **Secrets**: the image's file names are listed through `docker export`, without
330
+ extracting anything, and matched against the credential and key patterns in
331
+ [What goes into a bundle](#what-goes-into-a-bundle). Empty files, CA
332
+ certificates and the test keys that packages ship are skipped.
333
+
334
+ The exit status is 0 when nothing failed (warnings allowed), 1 when a check
335
+ failed, and 2 when the check itself could not run, for example without Docker
336
+ or when the pull failed. `--static-only` runs nothing from the image, and
337
+ `--supervisor <image>` runs the dry run under another supervisor image.
338
+
339
+ The verdict says what the result means for submission:
340
+
341
+ - **PASS**, with every check run: Flightdeck will admit the image, and it ran
342
+ commands as the workload runtime contract describes.
343
+ - **PASS for the checks that ran**: nothing failed, but the verdict lists what
344
+ was not checked, such as the dry run under `--static-only`.
345
+ - **WARN, "Flightdeck may refuse this image"**: a warning that admission acts
346
+ on, named in the verdict. The image is not in its registry yet, the registry
347
+ will not serve it without credentials, or its `ENTRYPOINT` is shell form,
348
+ which is refused for every flight with a command.
349
+ - **WARN** otherwise: Flightdeck will admit the image; read the warnings.
350
+ - **FAIL**: Flightdeck would refuse the image, or its flights would not run.
351
+
352
+ If Docker does not have the image, the check pulls it for `linux/amd64` and
353
+ leaves it for you; it never removes an image you already had. Everything else
354
+ it creates is removed when it finishes, fails or is interrupted with Ctrl-C:
355
+ its containers (named `flightdeck-image-check-<id>-…` and labelled
356
+ `uk.thermai.flightdeck.image-check`), its temporary directory, and the
357
+ supervisor image if the check pulled it (unless another image check is
358
+ running, which may still need it). After Ctrl-C the check waits for Docker to
359
+ remove its containers, up to a minute each, and a second Ctrl-C does not cut
360
+ that short. If the check is killed outright, list what it left with
361
+ `docker ps -a --filter label=uk.thermai.flightdeck.image-check`, remove each with
362
+ `docker rm -f <name>`, and delete any `flightdeck-image-check-*` directory in
363
+ your temporary directory. On Apple silicon and other machines that are not
364
+ amd64, the image runs under emulation, which is slow.
365
+
366
+ The check does not test your code or the provider's GPUs, so run a short flight
367
+ before real work.
368
+
369
+ ### Publishing a private image
370
+
371
+ `flightdeck image push <local-image> --name <name> --version <label>` publishes
372
+ a local image to your organization's private registry. `image list` and
373
+ `image inspect <version-id> --format reference` show it and print the reference
374
+ to pass to `--base-image`. Log in with `flightdeck login --images` first (run
375
+ `logout` before it if you are already logged in). Rerun the same push to resume.
376
+ `--base-image managed://<name>:latest` runs the newest ready version, and
377
+ `image push --alias ci` or `image alias set <name> ci --version <label>` moves a channel
378
+ you can run as `managed://<name>:ci`. The full guide, including errors and limits, is in
379
+ [`docs/managed-images.md`](../docs/managed-images.md).
380
+
205
381
  ### Slurm differences
206
382
 
207
383
  - Single-node jobs only; this is not a full Slurm scheduler or an HPC login node.
208
- - Array `%N` concurrency throttles are not enforced: tasks are submitted
384
+ - Array `%N` concurrency throttles are not enforced, so `sbatch` refuses one
385
+ smaller than the array. `--ignore-array-limit` submits every task
209
386
  immediately. Admission ceilings may reject excess tasks rather than queue
210
387
  them for later admission. Review every warning before real submission.
211
- - Use explicit `%A_%a` output names for arrays, and write results under
388
+ - In an array task, `%j` is the array job id, not a per-task id; use `%a` in
389
+ `--output`/`--error` names so tasks don't share a file.
390
+ - Array tasks default to `slurm-%A_%a.out`, as in Slurm; write results under
212
391
  `$OUTPUTS_DIR`. There is no shared university filesystem or implicit `module`,
213
392
  Singularity, nested Docker, or cluster software environment.
214
393
  - Hardware, runtime, image and credit must fit the operator-approved policy.
215
- A compatible runner image is required, not just any CUDA image.
394
+ The image must meet the workload runtime contract: linux/amd64, and an
395
+ exec-form `ENTRYPOINT` if it sets one. Submission refuses an image that does
396
+ not.
216
397
 
217
398
  ### Reading logs completely
218
399
 
219
400
  From `0.1.0-pilot.3` (containing the #99 pagination fix), `flightdeck logs`
220
401
  drains all pages in the current stored-log snapshot, then returns. For a running
221
- job it may append one current live tail after the stored pages. `--follow`
222
- drains those pages immediately, then polls every two seconds until the job is
223
- terminal and its stored backlog is exhausted. Every new invocation starts at
224
- the beginning; stdout/stderr retain their respective output streams.
402
+ job it then reads the live tail, which the backend serves a bounded page at a
403
+ time. `--follow` drains those pages immediately, then polls every two seconds
404
+ until the job is terminal and its stored backlog is exhausted. Every new
405
+ invocation starts at the beginning; stdout/stderr retain their respective
406
+ output streams.
407
+
408
+ Each log stream is capped at 5 MiB. On Lambda, a finished flight's stream that
409
+ passes the cap keeps its first 5 MiB and its last 64 KiB (each cut between
410
+ characters), with a `flightdeck: log truncated` marker line between them when
411
+ anything was left out, so the end of the run and any
412
+ `flightdeck: failure reason` line survive. When a log is truncated,
413
+ `flightdeck logs` prints a one-line warning on stderr after the text; stdout
414
+ carries only the flight's own output.
225
415
 
226
416
  **Rollout order:** deploy and verify the pagination-capable backend before
227
417
  releasing/upgrading this CLI. A backend without the `has_more` response field