@shardflux/cli 0.6.2 → 0.7.0

package/CHANGELOG.md

@@ -38,6 +38,29 @@ New organizations can start on a trial: 60 days at Developer limits, no card. Th
 - An interrupted file-first `ws exec` says `(an execution runs to completion)`; `mode_not_available` drops "yet";
   `--open` no longer says "best effort"; `templates init --help` no longer names an internal document.
 
+## 0.7.0 (release candidate)
+
+Completed exec output streams are drained before releasing their HTTP connections, with a bounded cleanup if a peer does not close.
+
+Uses SDK 0.12.0 and its pooled transport. Adds --labels/--idle-policy to open, --labels to list, ws labels, ws idle, ws exec-start --stdin-open and ws exec-input --offset. Protocol-error JSON includes source. Resume recovers failed workspaces; completed deletion permits key reuse.
+
+Requires the QM integration backend release for labels, failed recovery, key reuse and pipe stdin. No production deployment has occurred from this branch.
+
+### Instant suspend: durable storage
+
+- `shard ws suspend --wait` still returns when the suspend succeeds (sealed on its host); a finished suspend or fork
+  now prints `durable: yes (<n> ms after the suspend)` or `durable: in progress ...` when the result says so.
+- `shard ws suspend --durable`: waits until `result.durable` is true (implies `--wait`, shares `--timeout`). A copy
+  that cannot be made exits 6 with code `durability_lost` (operation id, `details.state` `lost`, `details.reason`); a
+  timeout exits 5 with `details.durable` false and `details.state` `pending`. `--durable` with `--when-idle` or
+  `--cancel-when-idle` is a usage error.
+- `shard operations wait <id> --durable`, and a `durable` line in `operations get|wait` for suspends and forks.
+- `--timing --json`: `server` gains `durable`, `suspend_path`, `durability` and `lost_suspend`.
+- A resume (or a command's wake) whose result carries `lost_suspend` prints a note on stderr naming the restored
+  checkpoint and its time.
+- A suspend-when-idle canceled `workspace_active` says the workspace was in use and keeps running.
+- The `ws suspend` summary no longer says "durable checkpoint".
+
 ## 0.6.1
 
 Needs `@shardflux/sdk` 0.11.0 or later.

package/README.md

@@ -387,9 +387,10 @@ shows the status; `shard version --check` asks now (`--json`: `{"cli", "sdk", "u
 | `files search <id\|key> <path> <pattern> [--regex] [-i\|--ignore-case] [--include G]... [--exclude G]... [--max N]` | Search file contents under a directory (or in one file; gitignore-style globs): `path:line:column: text` per match (0.5.0+). |
 | `files patch <id\|key> <path> (--old T --new T)... [--replace-all] \| --content-file F\|- [--expected-revision R] [--if-revision N]` | Replace exact text (each `--old` must occur once unless `--replace-all`), or the whole content, atomically; `R` is a revision from `files stat --revision` or `absent` (0.5.0+). `--if-revision` (0.5.0+): file-first workspaces only. |
 | `workspaces suspend\|resume <id\|key> [--wait] [--timeout T] [--timing]` | Lifecycle operation. Returns immediately unless you pass `--wait`. |
+| `workspaces suspend <id\|key> --durable [--timeout T]` | (0.7.0+) Suspend and wait until the copy is in durable storage. See "Durable storage". |
 | `workspaces suspend <id\|key> --when-idle D` / `--cancel-when-idle` | (0.5.1+) Suspend once the workspace has been idle for `D` (0 to 1h; `0` = as soon as it is idle, 0.6.1+), or cancel that. See "Suspend when idle". |
 | `workspaces fork <id\|key> <new-key> [caps] [--wait]` | Fork into a new key. |
-| `workspaces delete <id\|key> --yes [--wait]` | Delete the workspace. Tool access ends at once. The key of a persistent workspace is never reused. |
+| `workspaces delete <id\|key> --yes [--wait]` | Delete the workspace. Tool access ends at once. After deletion finishes, the key can open a new workspace ID. |
 | `workspaces close <id\|key> [--wait]` | End a session workspace now: it is deleted, and the key then opens a new workspace. A persistent workspace is refused (`not_session`). |
 | `workspaces reset <id\|key> --yes [--wait]` | Layered workspaces: wipe every change and restart on the template. The previous state stays restorable for 7 days. |
 | `workspaces save-as-template <id\|key> --template <slug> [--display-name N] [--description T] [--checkpoint ID] [--default-lifetime L] [--default-idle-timeout D] [--acknowledge PATH]... [--no-auto-publish] [--wait]` | Save a layered workspace as the next version of an organization template. `--wait` follows the build. |
@@ -792,11 +793,49 @@ printf %s "$NEW_KEY" | shard secrets rotate OPENAI_API_KEY
 | 130 | interrupted (Ctrl-C). Waits stop at once, and exec sends a cancel to the session. A file-first execution runs to completion, and the error names the `shard executions get` command for its result. |
 | exec, executions get | the command's own code: 124 if it timed out, 128+N if killed by signal N. Failures before the command ran use the codes above; use `--json` to tell them apart. |
 
+## Durable storage (0.7.0+)
+
+`shard ws suspend --wait` returns as soon as the workspace is sealed on its host, typically in a few hundred ms, and
+its RAM and CPU are released then. The copy lands in durable storage right after, typically within a second:
+
+```sh
+shard ws suspend acme/demo --wait
+# Suspending acme/demo (<id>): operation <op> suspend succeeded
+# durable: in progress, typically within a second (wait with: shard operations wait <op> --durable)
+shard ws suspend acme/demo --durable      # returns once it is durable: "durable: yes (435 ms after the suspend)"
+shard operations wait <op> --durable      # the same for a suspend or fork you already started
+```
+
+- `--durable` implies `--wait` and shares its `--timeout`.
+- `--json` prints the operation as the API returns it: `result.durable` and `result.durability` (`state`
+  `pending` or `durable`, `local_commit_at`, `durable_at`, ...). `--timing` adds `durable`, `suspend_path` and
+  `durability` to `server`.
+- `operations get|wait` show a `durable` line for suspends and forks. A fork of a running workspace reports its copy the
+  same way.
+- The exit codes and errors of `--durable` are in the CLI reference on docs.shardflux.dev.
+
 ## HTTP connections
 
-By default every request uses a fresh connection (`Connection: close`); `SHARDFLUX_HTTP_KEEPALIVE=1` reuses
-connections.
+`shard` 0.7.0 uses the SDK’s pooled HTTP/1.1 transport on Node 26. Custom fetch is preserved. `SHARDFLUX_HTTP_KEEPALIVE=0` forces connection closure for diagnosis; `=1` opts into native pooling.
 
 ## License
 
 Apache-2.0
+
+
+## Integration controls (0.7.0+)
+
+```sh
+shard ws open qm/project-42 --template python-node-browser --labels '{"scope":"project-42"}' --idle-policy never
+shard ws list --labels '{"scope":"project-42"}' --json
+shard ws labels qm/project-42 --set '{"scope":"project-42","owner":"qm"}'
+shard ws idle qm/project-42 --json
+shard ws idle qm/project-42 --keepalive 60
+shard ws idle qm/project-42 --policy adaptive
+shard ws suspend qm/project-42 --when-idle 2s
+shard ws exec-start qm/project-42 --stdin-open --session-id qm-worker-1 -- cat
+shard ws exec-input qm/project-42 qm-worker-1 hello --offset 0
+shard ws exec-input qm/project-42 qm-worker-1 --offset 5 --close
+```
+
+Use the byte offset in each acknowledgement, including partial writes. `ws resume` can recover a failed workspace with its existing ID and disk. A completed delete frees the key; old IDs remain deleted. Protocol-error JSON includes `source`; HTTP 404 alone does not mean a workspace was deleted.

package/dist/args.d.ts

@@ -1,4 +1,4 @@
-export declare const CLI_VERSION = "0.6.2";
+export declare const CLI_VERSION = "0.7.0";
 export interface OptionSpec {
     type: 'string' | 'boolean';
     short?: string;

package/dist/args.js

@@ -12,7 +12,7 @@
 import { parseArgs } from 'node:util';
 import { ACCOUNT_COMMANDS } from "./account-args.js";
 import { UsageError } from "./errors.js";
-export const CLI_VERSION = '0.6.2';
+export const CLI_VERSION = '0.7.0';
 export const GLOBAL_OPTIONS = {
     'api-url': { type: 'string', value: '<url>', description: 'API base URL (default $SHARDFLUX_API_URL, else https://api.shardflux.dev).' },
     json: { type: 'boolean', description: 'Print machine-readable JSON on stdout (errors as JSON on stderr).' },
@@ -189,6 +189,8 @@ const KEY_COMMANDS = [
         positionals: [{ name: 'key', required: true }],
         options: {
             template: { type: 'string', value: '<slug>', description: 'Template slug (required), e.g. python-node-browser.' },
+            labels: { type: 'string', value: '<json>', description: 'Replace workspace labels with a JSON object of strings.' },
+            'idle-policy': { type: 'string', value: '<policy>', description: 'adaptive, never or fixed:<seconds>.' },
             secret: {
                 type: 'string',
                 multiple: true,
@@ -228,6 +230,7 @@ const KEY_COMMANDS = [
         options: {
             state: { type: 'string', value: '<state>', description: 'Observed state filter (running, suspended, creating, ...).' },
             prefix: { type: 'string', value: '<key-prefix>', description: 'Only keys starting with this prefix.' },
+            labels: { type: 'string', value: '<json>', description: 'Exact label matches (JSON object; all pairs must match).' },
             ...LIFETIME_FILTER,
             'include-deleted': { type: 'boolean', description: 'Include deleted workspaces and ended sessions (tombstones).' },
             all: { type: 'boolean', description: 'Fetch every page.' },
@@ -235,6 +238,22 @@ const KEY_COMMANDS = [
             cursor: { type: 'string', value: '<cursor>', description: 'Continue from a previous page.' },
         },
     },
+    { path: ['workspaces', 'labels'], summary: 'Read or replace workspace labels', positionals: [REF], options: {
+            set: { type: 'string', value: '<json>', description: 'Replace all labels; {} clears them.' },
+        } },
+    { path: ['workspaces', 'idle'], summary: 'Inspect idle status, set policy or declare a keepalive', positionals: [REF], options: {
+            policy: { type: 'string', value: '<policy>', description: 'adaptive, never, fixed:<seconds>, or default to clear the override.' },
+            keepalive: { type: 'string', value: '<seconds>', description: 'Declare ongoing work for this many seconds.' },
+        } },
+    { path: ['workspaces', 'exec-start'], summary: 'Start a background exec session with optional pipe stdin', positionals: [REF, { name: 'command', required: true, variadic: true }], options: {
+            'stdin-open': { type: 'boolean', description: 'Keep pipe stdin open for exec-input.' },
+            'session-id': { type: 'string', value: '<id>', description: 'Caller-chosen id to deduplicate starts.' },
+            cwd: { type: 'string', value: '<dir>', description: 'Absolute working directory.' },
+        } },
+    { path: ['workspaces', 'exec-input'], summary: 'Write pipe stdin at its acknowledged byte offset', positionals: [REF, { name: 'session-id', required: true }, { name: 'data' }], options: {
+            offset: { type: 'string', value: '<bytes>', description: 'Required acknowledged offset (0 initially).' },
+            close: { type: 'boolean', description: 'Send EOF after these bytes are accepted.' },
+        } },
     { path: ['workspaces', 'get'], summary: 'Show one workspace (state, template, startup)', positionals: [REF], options: {} },
     { path: ['workspaces', 'inputs'], summary: 'Show the text inputs of a workspace (secret inputs are bound secrets: shard ws secrets)', positionals: [REF], options: {} },
     {
@@ -269,15 +288,16 @@ const KEY_COMMANDS = [
     },
     {
         path: ['workspaces', 'suspend'],
-        summary: 'Suspend a running workspace (durable checkpoint), now or once it has been idle',
-        description: 'Without options the suspend is requested now. --when-idle (0.5.1+) defers it instead: the workspace is suspended once it has been idle for that long, counted from the later of its last work and the request, so call it when an agent turn ends. A command still running, an attached exec or terminal stream, or a keepalive postpones it; the next tool call (the next turn) or a resume cancels it. Repeating replaces the pending request. --cancel-when-idle cancels it.',
+        summary: 'Suspend a running workspace (memory and processes checkpointed), now or once it has been idle',
+        description: 'Without options the suspend is requested now. With --wait it returns as soon as the workspace is sealed on its host, typically in a few hundred ms; the result\'s durable turns true when the copy lands in durable storage, typically within a second. --durable (0.7.0+) waits for that too. --when-idle (0.5.1+) defers it instead: the workspace is suspended once it has been idle for that long, counted from the later of its last work and the request, so call it when an agent turn ends. A command still running, an attached exec or terminal stream, or a keepalive postpones it; the next tool call (the next turn) or a resume cancels it. Repeating replaces the pending request. --cancel-when-idle cancels it.',
         positionals: [REF],
         options: {
             'when-idle': { type: 'string', value: '<duration>', description: 'Suspend once the workspace has been idle this long (0.5.1+; 0 to 1h, e.g. 0, 30s, 2m; 0 = as soon as it is idle, 0.6.1+). Cannot be combined with --wait.' },
             'cancel-when-idle': { type: 'boolean', description: 'Cancel a pending --when-idle request (0.5.1+; idempotent).' },
+            durable: { type: 'boolean', description: 'Wait until the suspend is in durable storage (result durable: true; 0.7.0+). Implies --wait and shares its --timeout.' },
             ...WAIT,
         },
-        examples: ['shard ws suspend acme/demo --wait', 'shard ws suspend acme/demo --when-idle 2m', 'shard ws suspend acme/demo --cancel-when-idle'],
+        examples: ['shard ws suspend acme/demo --wait', 'shard ws suspend acme/demo --durable', 'shard ws suspend acme/demo --when-idle 2m', 'shard ws suspend acme/demo --cancel-when-idle'],
     },
     { path: ['workspaces', 'resume'], summary: 'Resume a suspended workspace', positionals: [REF], options: { ...WAIT } },
     {
@@ -695,7 +715,11 @@ const KEY_COMMANDS = [
         path: ['operations', 'wait'],
         summary: 'Wait for an operation to finish (exit 0 succeeded, 6 failed/canceled, 5 timeout)',
         positionals: [{ name: 'operation-id', required: true }],
-        options: { timeout: { type: 'string', value: '<duration>', description: 'Give up after this long (default 5m).' }, ...TIMING },
+        options: {
+            timeout: { type: 'string', value: '<duration>', description: 'Give up after this long (default 5m).' },
+            durable: { type: 'boolean', description: 'A suspend or fork: also wait until its copy is in durable storage (result durable: true; 0.7.0+).' },
+            ...TIMING,
+        },
     },
 ];
 /** Every command: the account plane first (sign in, set up), then the project-key commands. */

package/dist/commands.js

@@ -16,7 +16,7 @@ import { updateCheckDisabled } from "./update-check.js";
 import { initTemplateYaml, recipeYaml } from "./template-yaml.js";
 import { BuildTimer, buildTimingJson, buildTimingText, progressLine } from "./timing.js";
 import { timingJson } from "./timing.js";
-import { bindingsText, buildTable, buildText, changesTable, diffSummaryText, diffTable, draftText, draftStateTable, executionJson, executionSummary, fileInfoText, fileTable, languageTable, json, kv, modeText, operationDetail, pairs, operationTable, packageTable, searchText, secretAccessTable, secretDetail, secretTable, secretVersionTable, sessionTable, suspendRequestText, suspendWhenIdleText, templateDetailText, templateEntryText, templateFileTable, templateTable, usageEstimateText, usageSeriesText, usageText, workspaceDetail, workspaceTable, } from "./format.js";
+import { bindingsText, buildTable, buildText, changesTable, diffSummaryText, diffTable, draftText, draftStateTable, executionJson, executionSummary, fileInfoText, fileTable, languageTable, json, kv, modeText, operationDetail, pairs, operationTable, packageTable, searchText, secretAccessTable, secretDetail, secretTable, secretVersionTable, sessionTable, durableText, suspendRequestText, suspendWhenIdleText, templateDetailText, templateEntryText, templateFileTable, templateTable, usageEstimateText, usageSeriesText, usageText, workspaceDetail, workspaceTable, } from "./format.js";
 const DEFAULT_WAIT_MS = 300_000;
 const MAX_JSON_OUTPUT = 16 * 1024 * 1024;
 const MAX_STDIN = 1024 * 1024;
@@ -496,7 +496,7 @@ function reportExecution(ctx, ref, r, opts) {
 }
 function lifecycleText(verb, w, op) {
     const finished = op.state === 'succeeded' || op.state === 'failed' || op.state === 'canceled';
-    return `${verb} ${w.key} (${w.id}): operation ${op.id} ${op.kind} ${op.state}${op.state_reason ? ` (${op.state_reason})` : ''}\n${finished ? '' : `wait with: shard operations wait ${op.id}\n`}`;
+    return `${verb} ${w.key} (${w.id}): operation ${op.id} ${op.kind} ${op.state}${op.state_reason ? ` (${op.state_reason})` : ''}\n${finished ? durableText(op) : `wait with: shard operations wait ${op.id}\n`}`;
 }
 /** The project-key commands; handlers.ts adds the account plane's. */
 export const KEY_HANDLERS = {
@@ -565,6 +565,46 @@ export const KEY_HANDLERS = {
         out(ctx, usageText(summary), summary);
         return 0;
     },
+    async 'workspaces labels'(ctx, values, [ref]) {
+        const ws = await resolveWorkspace(ctx, ref);
+        const labels = labelMap(str(values.set));
+        if (labels !== undefined)
+            await ws.setLabels(labels);
+        out(ctx, JSON.stringify(ws.labels, null, 2) + '\n', ws.labels);
+        return 0;
+    },
+    async 'workspaces idle'(ctx, values, [ref]) {
+        const ws = await resolveWorkspace(ctx, ref);
+        const policy = str(values.policy);
+        const keepalive = str(values.keepalive);
+        if (policy !== undefined && keepalive !== undefined)
+            throw new UsageError('--policy and --keepalive cannot be combined');
+        let result;
+        if (policy !== undefined)
+            result = (await ws.setIdlePolicy(policy === 'default' ? null : policy)).data;
+        else if (keepalive !== undefined)
+            result = await ws.keepalive(parsePositiveInt(keepalive, 'keepalive', 86400));
+        else
+            result = await ws.idle(ctx.signal);
+        out(ctx, JSON.stringify(result, null, 2) + '\n', result);
+        return 0;
+    },
+    async 'workspaces exec-start'(ctx, values, [ref, ...argv]) {
+        const ws = await resolveWorkspace(ctx, ref);
+        const session = await cellOf(ctx, ws).exec.start({ argv, stdin_open: values['stdin-open'] === true,
+            ...(str(values['session-id']) ? { session_id: str(values['session-id']) } : {}), ...(str(values.cwd) ? { cwd: str(values.cwd) } : {}) }, ctx.signal);
+        out(ctx, JSON.stringify(session, null, 2) + '\n', session);
+        return 0;
+    },
+    async 'workspaces exec-input'(ctx, values, [ref, sessionId, data]) {
+        const offset = Number(str(values.offset));
+        if (str(values.offset) === undefined || !Number.isSafeInteger(offset) || offset < 0)
+            throw new UsageError('--offset must be a non-negative integer');
+        const ws = await resolveWorkspace(ctx, ref);
+        const ack = await cellOf(ctx, ws).exec.input(sessionId, data ?? '', { offset, close: values.close === true, signal: ctx.signal });
+        out(ctx, JSON.stringify(ack) + '\n', ack);
+        return 0;
+    },
     async 'workspaces open'(ctx, values, [key]) {
         const template = str(values.template);
         if (!template)
@@ -583,6 +623,8 @@ export const KEY_HANDLERS = {
         const ws = await waited(ctx, 'the open continues server side (open the key again to keep waiting).', () => cloud.workspaces.open({
             key: key,
             template,
+            ...(str(values.labels) !== undefined ? { labels: labelMap(str(values.labels)) } : {}),
+            ...(str(values['idle-policy']) ? { idlePolicy: str(values['idle-policy']) } : {}),
             ...(c ? { caps: c } : {}),
             ...(bind.length ? { secrets: bind } : {}),
             ...(inputs ? { inputs } : {}),
@@ -609,6 +651,7 @@ export const KEY_HANDLERS = {
         const params = {
             ...(state ? { state: state } : {}),
             ...(prefix ? { keyPrefix: prefix } : {}),
+            ...(str(values.labels) !== undefined ? { labels: labelMap(str(values.labels)) } : {}),
             ...(lifetime ? { lifetime } : {}),
             ...(purpose ? { purpose } : {}),
             ...(values['include-deleted'] === true ? { includeDeleted: true } : {}),
@@ -854,8 +897,12 @@ export const KEY_HANDLERS = {
     async 'workspaces suspend'(ctx, values, [ref]) {
         const whenIdle = str(values['when-idle']);
         const cancel = values['cancel-when-idle'] === true;
-        if (whenIdle === undefined && !cancel)
-            return lifecycle(ctx, values, ref, 'Suspending', (ws, o) => ctx.cloud().workspaces.suspend(ws.id, o), ['suspend', 'Suspend']);
+        const durable = values.durable === true;
+        if (whenIdle === undefined && !cancel) {
+            return lifecycle(ctx, values, ref, 'Suspending', (ws, o) => ctx.cloud().workspaces.suspend(ws.id, durable ? { ...o, durable: true, wait: o.wait ?? { timeoutMs: waitTimeout(values), signal: ctx.signal } } : o), ['suspend', 'Suspend']);
+        }
+        if (durable)
+            throw new UsageError(`${whenIdle !== undefined ? '--when-idle' : '--cancel-when-idle'} cannot be combined with --durable (nothing is suspended now)`);
         if (whenIdle !== undefined && cancel)
             throw new UsageError('--when-idle and --cancel-when-idle cannot be combined');
         const flag = cancel ? '--cancel-when-idle' : '--when-idle';
@@ -1526,7 +1573,10 @@ export const KEY_HANDLERS = {
     async 'operations wait'(ctx, values, [id]) {
         if (!UUID.test(id))
             throw new UsageError(`<operation-id> must be a UUID (got "${id}")`);
-        const op = await waitOperation(ctx, id, waitTimeout(values));
+        const timeoutMs = waitTimeout(values);
+        const op = values.durable === true
+            ? await waited(ctx, `operation ${id} continues server side.`, () => ctx.cloud().workspaces.waitForDurable(id, { timeoutMs, signal: ctx.signal }))
+            : await waitOperation(ctx, id, timeoutMs);
         out(ctx, operationDetail(op), op);
         return 0;
     },
@@ -1681,3 +1731,17 @@ async function lifecycle(ctx, values, ref, verb, start, processfulOnly) {
     out(ctx, lifecycleText(verb, ws, op), op);
     return 0;
 }
+function labelMap(value) {
+    if (value === undefined)
+        return undefined;
+    let parsed;
+    try {
+        parsed = JSON.parse(value);
+    }
+    catch {
+        throw new UsageError('labels must be a JSON object of strings');
+    }
+    if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed) || Object.values(parsed).some((v) => typeof v !== 'string'))
+        throw new UsageError('labels must be a JSON object of strings');
+    return parsed;
+}

package/dist/errors.d.ts

@@ -84,7 +84,7 @@ export interface ErrorInfo {
     retryable?: boolean;
     operation_id?: string;
     details?: Record<string, unknown>;
-    source?: 'api' | 'cell';
+    source?: 'api' | 'cell' | 'unknown';
     last_state?: string;
     operation?: unknown;
     /** A file-first execution the error is about (failed, lost, still running, or left running by Ctrl-C). */

package/dist/errors.js

@@ -3,7 +3,7 @@
  * stderr with the API's error code, message and request id; the process exit
  * code tells scripts what kind of failure it was (README "Exit codes").
  */
-import { CheckoutTimeoutError, ExecStartError, OperationFailedError, OperationTimeoutError, ShardfluxApiError, ShardfluxProtocolError, TemplateBuildTimeoutError, TemplateFileError, TemplateUploadError } from '@shardflux/sdk';
+import { CheckoutTimeoutError, DurabilityLostError, ExecStartError, OperationFailedError, OperationTimeoutError, ShardfluxApiError, ShardfluxProtocolError, TemplateBuildTimeoutError, TemplateFileError, TemplateUploadError } from '@shardflux/sdk';
 export const EXIT = {
     ok: 0,
     /** Any other failure: API refusal (conflict, quota, ...), network, protocol. */
@@ -137,15 +137,22 @@ export function describeError(err) {
             source: err.source,
         };
     }
-    if (err instanceof OperationTimeoutError)
-        return { code: 'timeout', message: err.message, operation_id: err.operationId, last_state: err.lastState };
+    if (err instanceof OperationTimeoutError) {
+        return { code: 'timeout', message: err.message, operation_id: err.operationId, last_state: err.lastState, ...(err.durable ? { details: { durable: false, state: 'pending' } } : {}) };
+    }
+    // --durable (0.7.0+): the suspend succeeded; its durable copy could not be made (see the lifecycle reference).
+    if (err instanceof DurabilityLostError) {
+        return { code: 'durability_lost', message: err.message, operation_id: err.operationId, retryable: false, details: { state: 'lost', ...(err.durability.reason ? { reason: err.durability.reason } : {}), checkpoint_id: err.durability.checkpointId }, operation: err.operation };
+    }
     if (err instanceof OperationFailedError) {
         // retryable: the operation error's own flag (true for capacity_unavailable: nothing was started, retry later).
         // details.reason: the operation error's reason (e.g. the scheduler's no_ready_hosts), printed after the message.
         const opDetails = err.operation.error?.details;
         const reason = typeof opDetails === 'object' && opDetails !== null ? opDetails.reason : undefined;
         const details = { ...(err.errorCode ? { error_code: err.errorCode } : {}), ...(typeof reason === 'string' ? { reason } : {}) };
-        return { code: 'operation_failed', message: err.message, operation_id: err.operationId, retryable: err.retryable, ...(Object.keys(details).length ? { details } : {}), operation: err.operation };
+        // A suspend-when-idle canceled workspace_active (0.7.0+): the workspace was in use and keeps running.
+        const message = err.workspaceActive ? `${err.message}: the workspace was in use, so the suspend-when-idle was canceled; it keeps running` : err.message;
+        return { code: 'operation_failed', message, operation_id: err.operationId, retryable: err.retryable, ...(Object.keys(details).length ? { details } : {}), operation: err.operation };
     }
     // File-first executions (0.5.0): failed or lost like a failed operation (details.reason says why;
     // retryable: a new execution id may succeed), still running like a wait that gave up.
@@ -180,7 +187,7 @@ export function describeError(err) {
         return { code: 'interrupted', message: err.message, ...(err.operationId ? { operation_id: err.operationId } : {}), ...(err.executionId ? { execution_id: err.executionId } : {}) };
     }
     if (err instanceof ShardfluxProtocolError)
-        return { code: 'protocol_error', message: err.message, status: err.status };
+        return { code: 'protocol_error', message: err.message, status: err.status, source: err.source };
     if (isNamed(err, 'TimeoutError'))
         return { code: 'timeout', message: 'The request timed out.' };
     if (isNamed(err, 'AbortError'))
@@ -205,7 +212,7 @@ export function exitCodeFor(err, interrupted = false) {
         return EXIT.notFound;
     if (err instanceof OperationTimeoutError || err instanceof CheckoutTimeoutError || isNamed(err, 'TimeoutError'))
         return EXIT.timeout;
-    if (err instanceof OperationFailedError || err instanceof ExecutionFailedError || err instanceof CheckoutEndedError)
+    if (err instanceof OperationFailedError || err instanceof DurabilityLostError || err instanceof ExecutionFailedError || err instanceof CheckoutEndedError)
         return EXIT.operationFailed;
     if (err instanceof ExecutionPendingError)
         return EXIT.timeout;

package/dist/format.d.ts

@@ -1,8 +1,3 @@
-/**
- * Human-readable output (aligned tables, key/value blocks) and the error line.
- * `--json` output is the API's own objects (see README "JSON output"), so
- * these functions only shape what people read.
- */
 import type { AgentSession, ApiKey, AuditEvent, AuthSessionInfo, BillingCatalog, BillingInvoicePage, BillingSubscription, DataExport, DraftState, ExecutionChange, ExecutionResult, FileInfo, EgressPolicyVersion, Invitation, Member, Organization, Project, ProjectEgressPolicy, Volume, VolumeAttachment, WorkspaceEgressPolicy, FileList, FileSearchResponse, Operation, Secret, SecretAccessEvent, SecretVersion, SpendPolicy, SuspendRequest, TemplateBuild, TemplateDetail, TemplateDiffEntry, TemplateDraft, TemplateFileEntry, TemplateLanguages, TemplatePackagePage, TemplateSettings, TemplateSummary, WorkspaceChange, WorkspaceSecretBindings, WorkspaceView } from '@shardflux/sdk';
 import type { ErrorInfo } from './errors.js';
 export declare function json(value: unknown): string;
@@ -41,6 +36,11 @@ export declare function suspendWhenIdleText(w: {
 export declare function workspaceDetail(w: WorkspaceView): string;
 export declare const OPERATION_HEADERS: readonly ["ID", "KIND", "STATE", "REASON", "CREATED", "COMPLETED"];
 export declare function operationTable(ops: readonly Operation[]): string;
+/**
+ * A finished suspend or fork (0.7.0+): whether its copy is in durable storage, when the result says (`durable`,
+ * `durability`). Empty when the result has no durability block (stored durably before the operation completed).
+ */
+export declare function durableText(op: Operation): string;
 export declare function operationDetail(op: Operation): string;
 export declare function sessionTable(sessions: readonly AgentSession[]): string;
 /** `files stat`: one path's metadata; the revision only when requested and returned. */

package/dist/format.js

@@ -1,3 +1,9 @@
+/**
+ * Human-readable output (aligned tables, key/value blocks) and the error line.
+ * `--json` output is the API's own objects (see README "JSON output"), so
+ * these functions only shape what people read.
+ */
+import { durabilityOf, isDurable } from '@shardflux/sdk';
 export function json(value) {
     return `${JSON.stringify(value, null, 2)}\n`;
 }
@@ -161,6 +167,28 @@ export const OPERATION_HEADERS = ['ID', 'KIND', 'STATE', 'REASON', 'CREATED', 'C
 export function operationTable(ops) {
     return table(OPERATION_HEADERS, ops.map((o) => [o.id, o.kind, o.state, o.state_reason ?? '-', time(o.created_at), time(o.completed_at)]));
 }
+/**
+ * A finished suspend or fork (0.7.0+): whether its copy is in durable storage, when the result says (`durable`,
+ * `durability`). Empty when the result has no durability block (stored durably before the operation completed).
+ */
+export function durableText(op) {
+    const d = durabilityOf(op);
+    if (op.state !== 'succeeded' || !d)
+        return '';
+    if (d.state === 'durable')
+        return `durable: yes${d.localCommitToDurableMs !== null ? ` (${d.localCommitToDurableMs} ms after the suspend)` : ''}\n`;
+    if (d.state === 'pending')
+        return `durable: in progress, typically within a second (wait with: shard operations wait ${op.id} --durable)\n`;
+    return `durable: no (${d.reason ?? 'lost'}); see docs.shardflux.dev/concepts/lifecycle\n`;
+}
+/** `durable` of a finished suspend or fork for `operations get`/`wait`: yes, in progress, no, or '-' for other kinds. */
+function durableField(op) {
+    const d = durabilityOf(op);
+    if (d)
+        return d.state === 'durable' ? 'yes' : d.state === 'pending' ? 'in progress' : `no (${d.reason ?? 'lost'})`;
+    const v = isDurable(op);
+    return v === null ? '-' : v ? 'yes' : 'no';
+}
 export function operationDetail(op) {
     const err = op.error;
     return kv([
@@ -168,6 +196,7 @@ export function operationDetail(op) {
         ['kind', op.kind],
         ['state', op.state],
         ['reason', op.state_reason ?? '-'],
+        ...(op.kind === 'suspend' || op.kind === 'fork' ? [['durable', durableField(op)]] : []),
         ['workspace', op.workspace_id ?? '-'],
         ['created', time(op.created_at)],
         ['updated', time(op.updated_at)],

package/dist/http.d.ts

@@ -1,11 +1,2 @@
-/**
- * The fetch given to the SDK: the runtime's fetch (API and cell gateway). By
- * default every request uses a fresh connection (`Connection: close`);
- * SHARDFLUX_HTTP_KEEPALIVE=1 reuses connections.
- *
- * `signal` (the CLI's Ctrl-C) is attached to held requests (`Prefer: wait=N`,
- * held by the server for up to 20 s). The SDK passes the caller's signal to its
- * held polls itself, but not to the held open (`workspaces.open()` with `wait`
- * sends its POST without one), so Ctrl-C would otherwise wait the hold out.
- */
+/** Share the SDK pool and attach Ctrl-C to held API requests. */
 export declare function makeFetch(env: Record<string, string | undefined>, base?: typeof fetch, signal?: AbortSignal): typeof fetch;

package/dist/http.js

@@ -1,24 +1,12 @@
-/**
- * The fetch given to the SDK: the runtime's fetch (API and cell gateway). By
- * default every request uses a fresh connection (`Connection: close`);
- * SHARDFLUX_HTTP_KEEPALIVE=1 reuses connections.
- *
- * `signal` (the CLI's Ctrl-C) is attached to held requests (`Prefer: wait=N`,
- * held by the server for up to 20 s). The SDK passes the caller's signal to its
- * held polls itself, but not to the held open (`workspaces.open()` with `wait`
- * sends its POST without one), so Ctrl-C would otherwise wait the hold out.
- */
-export function makeFetch(env, base = fetch, signal) {
-    // Why a fresh connection by default (Node 26 keep-alive measurements): docs/progress/startup-latency.md.
-    const keepAlive = env.SHARDFLUX_HTTP_KEEPALIVE === '1';
+import { defaultFetch } from '@shardflux/sdk';
+/** Share the SDK pool and attach Ctrl-C to held API requests. */
+export function makeFetch(env, base = defaultFetch(env), signal) {
     return (input, init) => {
         const headers = new Headers(init?.headers);
         const held = signal !== undefined && /\bwait\s*=/i.test(headers.get('prefer') ?? '');
-        if (keepAlive && !held)
+        if (!held)
             return base(input, init);
-        if (!keepAlive)
-            headers.set('connection', 'close');
-        const s = held && signal ? (init?.signal ? AbortSignal.any([init.signal, signal]) : signal) : init?.signal;
-        return base(input, { ...init, headers, ...(s ? { signal: s } : {}) });
+        const s = init?.signal && signal ? AbortSignal.any([init.signal, signal]) : signal;
+        return base(input, { ...init, ...(s ? { signal: s } : {}) });
     };
 }

package/dist/index.d.ts

@@ -22,4 +22,4 @@ export { openUrl, openerFor } from './browser.js';
 export { EXIT, AuthConfigError, CheckoutEndedError, ExecutionFailedError, ExecutionPendingError, InterruptedError, MfaPendingError, NotFoundError, UsageError, describeError, exitCodeFor, redact } from './errors.js';
 export type { ErrorInfo } from './errors.js';
 export { changedText, errorText, executionJson, executionSummary, feedbackCommand, json, suspendRequestText, suspendWhenIdleText, table, time, usageText, workspaceDetail, workspaceTable } from './format.js';
-export { TimingRecorder, coldBootNotice, timingJson, timingText } from './timing.js';
+export { TimingRecorder, coldBootNotice, lostSuspendNotice, notices, timingJson, timingText } from './timing.js';

package/dist/index.js

@@ -14,4 +14,4 @@ export { CHECK_INTERVAL_MS, CLI_DISTRIBUTION, NOTICE_WAIT_MS, UPDATE_CHECK_FILE,
 export { openUrl, openerFor } from "./browser.js";
 export { EXIT, AuthConfigError, CheckoutEndedError, ExecutionFailedError, ExecutionPendingError, InterruptedError, MfaPendingError, NotFoundError, UsageError, describeError, exitCodeFor, redact } from "./errors.js";
 export { changedText, errorText, executionJson, executionSummary, feedbackCommand, json, suspendRequestText, suspendWhenIdleText, table, time, usageText, workspaceDetail, workspaceTable } from "./format.js";
-export { TimingRecorder, coldBootNotice, timingJson, timingText } from "./timing.js";
+export { TimingRecorder, coldBootNotice, lostSuspendNotice, notices, timingJson, timingText } from "./timing.js";

package/dist/main.js

@@ -19,7 +19,7 @@ import { errorText, feedbackCommand, json, time } from "./format.js";
 import { HANDLERS } from "./handlers.js";
 import { makeFetch } from "./http.js";
 import { Input } from "./input.js";
-import { TimingRecorder, coldBootNotice, timingJson, timingText } from "./timing.js";
+import { TimingRecorder, notices, timingJson, timingText } from "./timing.js";
 import { CLI_DISTRIBUTION, NOTICE_WAIT_MS, UpdateCheck, noticeOf, updateCheckDisabled, updateView } from "./update-check.js";
 const KEY_SHAPE = /^sfk_[a-z2-7]{16}_[A-Za-z0-9]+$/;
 // Printable (no C0 controls or DEL), as the API's AgentLabel pattern.
@@ -327,7 +327,7 @@ export async function run(argv, io, opts = {}) {
         const feedback = nudges(info, command) ? feedbackCommand(info) : undefined;
         const text = wantsJson
             ? json({ error: info, ...(showTiming ? { timing: timingJson(timing) } : {}), ...(feedback ? { feedback } : {}) })
-            : `${timing ? timingText(timing) : ''}${traces.coldBoot ? coldBootNotice(traces.coldBoot) : ''}${errorText(info, hint, feedback)}`;
+            : `${timing ? timingText(timing) : ''}${notices(traces)}${errorText(info, hint, feedback)}`;
         io.stderr.write(redact(text, ...secrets));
         return code;
     };
@@ -359,7 +359,7 @@ export async function run(argv, io, opts = {}) {
         const wakeTimeoutMs = resolveWakeTimeout(parsed.global.wakeTimeout, io.env.SHARDFLUX_WAKE_TIMEOUT_MS);
         showTiming = parsed.values.timing === true;
         const userAgent = `shard-cli/${CLI_VERSION} shardflux-sdk-ts/${SDK_VERSION}${distribution.package === CLI_DISTRIBUTION.package ? '' : ` ${distribution.package}/${distribution.version}`}`;
-        const base = makeFetch(io.env, fetch, signal);
+        const base = makeFetch(io.env, undefined, signal);
         // Notes which credential a 401 unauthenticated answered (the hint, and removing a dead saved session).
         const watchedFetch = async (input, init) => {
             const res = await base(input, init);
@@ -560,9 +560,11 @@ export async function run(argv, io, opts = {}) {
         // With --json the handler put the timing into its output; otherwise it follows the output, on stderr.
         if (showTiming && !ctx.json && traces.last)
             io.stderr.write(timingText(traces.last));
-        // A resume that booted the saved disk (0.6.0+): said on stderr with or without --timing, also with --json.
-        if (traces.coldBoot)
-            io.stderr.write(coldBootNotice(traces.coldBoot));
+        // A resume that booted the saved disk (0.6.0+), or restored an earlier checkpoint (0.7.0+): said on stderr with or
+        // without --timing, also with --json.
+        const notice = notices(traces);
+        if (notice)
+            io.stderr.write(notice);
         code = outcome.code;
     }
     // The update notice: after the output, never after Ctrl-C or a --json failure (stderr then holds only the error).

package/dist/timing.d.ts

@@ -10,6 +10,11 @@ export declare class TimingRecorder {
      * the wake of a command that found the workspace suspended. `coldBootNotice()` says it.
      */
     coldBoot: LifecycleTiming | null;
+    /**
+     * (0.7.0+) The last traced resume whose result names a suspend it could not restore (`lost_suspend`): a `ws resume
+     * --wait`, an `operations wait` of that resume, or a command's wake. `lostSuspendNotice()` says it.
+     */
+    lostSuspend: LifecycleTiming | null;
     readonly listener: ProgressListener;
     /**
      * The failed call's own timing (`err.timing`) when the SDK attached one, else the last recorded. A refused tool token
@@ -22,6 +27,13 @@ export declare class TimingRecorder {
  * `--timing`: the workspace runs and its files are as of the suspend, but running processes were restarted.
  */
 export declare function coldBootNotice(t: LifecycleTiming): string;
+/**
+ * The one-line stderr notice after a command whose resume restored an earlier checkpoint because the latest suspend
+ * could not be kept (0.7.0+; `lost_suspend`): the state the workspace runs with is as of `state_as_of`.
+ */
+export declare function lostSuspendNotice(t: LifecycleTiming): string;
+/** The notices a command's traced calls leave on stderr (cold boot, lost suspend), in that order. */
+export declare function notices(r: TimingRecorder): string;
 /** The human form (stderr): formatTiming() of the SDK, one block. */
 export declare function timingText(t: LifecycleTiming): string;
 /** The `--json` form: the SDK's LifecycleTiming with the CLI's snake_case keys (the rest of its JSON is the API's). */

package/dist/timing.js

@@ -17,6 +17,11 @@ export class TimingRecorder {
      * the wake of a command that found the workspace suspended. `coldBootNotice()` says it.
      */
     coldBoot = null;
+    /**
+     * (0.7.0+) The last traced resume whose result names a suspend it could not restore (`lost_suspend`): a `ws resume
+     * --wait`, an `operations wait` of that resume, or a command's wake. `lostSuspendNotice()` says it.
+     */
+    lostSuspend = null;
     listener = (e) => {
         if (e.action === 'token' || e.action === 'tool')
             return;
@@ -27,6 +32,8 @@ export class TimingRecorder {
         this.last = e.timing;
         if (restartedProcesses(e.timing))
             this.coldBoot = e.timing;
+        if (e.timing.server?.lostSuspend)
+            this.lostSuspend = e.timing;
     };
     /**
      * The failed call's own timing (`err.timing`) when the SDK attached one, else the last recorded. A refused tool token
@@ -49,6 +56,20 @@ export function coldBootNotice(t) {
     const reason = t.server?.coldBootReason;
     return `note: ${t.workspaceId ? `workspace ${t.workspaceId} ` : ''}resumed from its saved disk; running processes were restarted${reason ? ` (${reason})` : ''}\n`;
 }
+/**
+ * The one-line stderr notice after a command whose resume restored an earlier checkpoint because the latest suspend
+ * could not be kept (0.7.0+; `lost_suspend`): the state the workspace runs with is as of `state_as_of`.
+ */
+export function lostSuspendNotice(t) {
+    const l = t.server?.lostSuspend;
+    if (!l)
+        return '';
+    return `note: ${t.workspaceId ? `workspace ${t.workspaceId} ` : ''}resumed from checkpoint ${l.restoredCheckpointId ?? '-'}${l.stateAsOf ? ` (state as of ${l.stateAsOf})` : ''}; the suspend of ${l.suspendedAt ?? l.checkpointId} was not kept${l.reason ? ` (${l.reason})` : ''}\n`;
+}
+/** The notices a command's traced calls leave on stderr (cold boot, lost suspend), in that order. */
+export function notices(r) {
+    return `${r.coldBoot ? coldBootNotice(r.coldBoot) : ''}${r.lostSuspend ? lostSuspendNotice(r.lostSuspend) : ''}`;
+}
 /** The human form (stderr): formatTiming() of the SDK, one block. */
 export function timingText(t) {
     return `${formatTiming(t)}\n`;
@@ -79,6 +100,31 @@ export function timingJson(t) {
                 resume_path: s.resumePath,
                 memory_restored: s.memoryRestored ?? null,
                 cold_boot_reason: s.coldBootReason ?? null,
+                durable: s.durable ?? null,
+                suspend_path: s.suspendPath ?? null,
+                durability: s.durability
+                    ? {
+                        state: s.durability.state,
+                        checkpoint_id: s.durability.checkpointId,
+                        generation_id: s.durability.generationId,
+                        local_commit_at: s.durability.localCommitAt,
+                        durable_by: s.durability.durableBy,
+                        durable_at: s.durability.durableAt,
+                        local_commit_to_durable_ms: s.durability.localCommitToDurableMs,
+                        overdue_at: s.durability.overdueAt,
+                        reason: s.durability.reason,
+                    }
+                    : null,
+                lost_suspend: s.lostSuspend
+                    ? {
+                        checkpoint_id: s.lostSuspend.checkpointId,
+                        generation_id: s.lostSuspend.generationId,
+                        reason: s.lostSuspend.reason,
+                        suspended_at: s.lostSuspend.suspendedAt,
+                        restored_checkpoint_id: s.lostSuspend.restoredCheckpointId,
+                        state_as_of: s.lostSuspend.stateAsOf,
+                    }
+                    : null,
                 boot_to_ready_ms: s.bootToReadyMs,
                 host_timings_ms: s.hostTimingsMs,
             }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@shardflux/cli",
-  "version": "0.6.2",
+  "version": "0.7.0",
   "type": "module",
   "description": "shard: the Shardflux command line. Create an account, sign in, set up keys and billing, then open, run commands in, move files to, suspend, resume and fork persistent agent workspaces.",
   "license": "Apache-2.0",
@@ -44,7 +44,7 @@
   },
   "dependencies": {
     "yaml": "2.9.1",
-    "@shardflux/sdk": "^0.11.1"
+    "@shardflux/sdk": "^0.12.0"
   },
   "devDependencies": {
     "@eslint/js": "10.0.1",