runcloud 0.1.119 → 0.1.121

package/dist/version.js

@@ -1 +1 @@
-export const CLI_VERSION = '0.1.119';
+export const CLI_VERSION = '0.1.121';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "runcloud",
-  "version": "0.1.119",
+  "version": "0.1.121",
   "description": "Create and control run.cloud remote mobile simulators and cloud sandboxes",
   "license": "Apache-2.0",
   "keywords": [

package/skills/run-cloud-sandboxes/SKILL.md

@@ -286,20 +286,51 @@ printed, and lifecycle markers, interleaved in the order they happened.
 the file rather than deleting it, so a post-mortem read works on a sandbox that
 no longer exists.
 
-Two limits that change what you can conclude from it:
+Three limits that change what you can conclude from it:
 
+- A destroyed sandbox's console gets removed after 14 days (2 weeks), so a
+  post-mortem has a deadline. A running or paused sandbox's log is never removed,
+  however long it has sat quiet.
 - It is capped at 10 MiB and trimmed to the newest 5 MiB, so on a chatty workload
   early output is gone, not merely paged out. An empty-looking start is a trim,
   not proof the process printed nothing.
 - `systemd-run` output goes to the journal, not the console, so those units never
-  appear here at all. Read `journalctl -u <unit>` inside the sandbox instead.
+  appear here at all. `journalctl -u <unit>` reaches it, but only from inside a
+  running sandbox, so that output is unrecoverable once the sandbox is destroyed
+  and needs a resume on a paused one. A workload whose output has to survive a
+  post-mortem should write to stdout rather than a unit.
+
+Two details about reading the file: the `created` marker is appended to a shell
+prompt line rather than starting one, so match `--- sandbox` anywhere in the line
+rather than anchoring to the start, and boot output dwarfs everything else, so
+reach for the tail before the head.
+
+### What the metrics actually settle
+
+`runcloud sandbox metrics <id> --json` carries typed counters, and they are how
+you refute a theory instead of arguing about it. The natural wrong answer on a
+128 MiB sandbox is always "it OOMed":
+
+| field | what a zero proves |
+| --- | --- |
+| `memory_oom_kills`, `memory_oom_events` | nothing was OOM-killed, so a dead process exited on its own |
+| `cpu_throttled_periods`, `cpu_throttled_millicores` | the CPU reservation was not starving it |
+| lifetime `network` totals | at ~0 bytes in, no request ever arrived, so the fault is upstream of the guest |
+
+Each has a `*_valid` companion; when that is false the counter is unavailable
+rather than zero, and a zero you cannot trust proves nothing.
+
+`memory_bytes` is the exception and it misleads. It is measured host-side and
+includes page cache, so it routinely reads several times the sandbox's `mem_mb`
+reservation. That is not a leak and not an impending OOM. Believe
+`memory_oom_kills` over it, every time.
 
 ### Lifecycle markers
 
 The host writes a marker into the console on every transition:
 
 ```
---- sandbox destroyed: timeout-sweep at 2026-08-05T11:02:11Z ---
+--- sandbox paused: timeout-sweep at 2026-08-05T11:02:11Z ---
 ```
 
 The events are `created`, `paused`, `resumed`, `stopped`, and `destroyed`. The
@@ -307,22 +338,52 @@ token after the colon is the control plane's actor, written verbatim so it can b
 matched rather than parsed; the sentence around it is not a contract. Only stops
 carry a reason, so `created` and `resumed` appear without one.
 
-| actor | what it means |
-| --- | --- |
-| `timeout-sweep` | hit its `--timeout` lifetime cap. Not a crash |
-| `idle-sweep` | idle-paused after `--idle-pause` seconds. Resumable, and not a failure |
-| `retention-sweep` | destroyed after its 48h parked window expired |
-| `pause` `resume` `stop` `archive` `restore` | a caller requested exactly this |
-| `api` `api-force` | a caller destroyed it. `api-force` bypassed a wedged guest |
-| `health-sweep` `host-reconcile` `reap-sweep` | the platform reclaimed it |
-| `boot-failure` `placement-timeout` `request-timeout` `scheduler` | it never started. Capacity or image, not the workload |
-| `image-built` `image-build-failure` | an async image build finished or failed |
-| `startup-recover` `volume-lease-invalid` `replica-fork-resume-failure` | platform recovery paths |
+Read the **event** before the actor. A paused sandbox still exists and its disk is
+intact; a destroyed one is gone. Telling someone their work is lost when it is
+warm-parked and resumable is the most expensive mistake available here.
+
+| actor | ends up | what it means |
+| --- | --- | --- |
+| `timeout-sweep` | paused | hit its `--timeout` lifetime cap. Not a crash, and the work is still there |
+| `idle-sweep` | paused | idle-paused after `--idle-pause` seconds. Not a failure at all |
+| `retention-sweep` | destroyed | the 48h parked window expired and it was reaped for real |
+| `pause` `resume` `stop` `archive` `restore` | as asked | a caller requested exactly this |
+| `api` `api-force` | destroyed | a caller destroyed it. `api-force` bypassed a wedged guest |
+| `health-sweep` | interrupted | the platform judged it unhealthy |
+| `host-reconcile` `reap-sweep` | destroyed | the platform reclaimed it |
+| `boot-failure` `placement-timeout` `request-timeout` `scheduler` | never ran | capacity or image, not the workload |
+| `image-built` `image-build-failure` | varies | an async image build finished or failed |
+| `startup-recover` `volume-lease-invalid` `replica-fork-resume-failure` | varies | platform recovery paths |
+
+**Getting a parked sandbox back.** `timeout-sweep` and `idle-sweep` leave a full
+snapshot, so `runcloud sandbox resume <id>` restores it warm. You have 48h before
+`retention-sweep` destroys it for real. Resuming restarts the lifetime window from
+now rather than clearing it, so a sandbox parked by `timeout-sweep` will park again
+after the same interval; raise the cap first with the SDK's
+`cloud.sandboxes.setTimeout(id, seconds)`, which has no CLI equivalent.
+
+**The `--timeout` trap.** `--timeout` is a wall-clock lifetime cap, wholly separate
+from `--idle-pause`. It fires on a fully busy sandbox, and `--persistent` /
+`--idle-pause 0` do **not** hold it off, despite `--persistent` reading as "never
+pause when idle". Default is 300s, ceiling 24h.
 
 A sandbox that failed before it ever reached a host has no console, because the
 file is created at boot. Those carry a `Last error` row on
 `runcloud sandbox get` instead, which is then the only record of the cause.
 
+### "Not reachable" is usually not a network problem
+
+Exposure is a separate record, not a field on the sandbox, so `sandbox get` omits
+it entirely when none exists rather than showing it as empty. Absence of a
+`Hostname` row therefore means **no public route exists**, not that the CLI
+declined to print one. `runcloud sandbox domain list <id>` states it outright and
+names the fix. A sandbox created without `--expose` is unreachable from outside
+however healthy the workload is.
+
+When a route does exist and the port still refuses, the guest is usually bound to
+`127.0.0.1` rather than `0.0.0.0`, which is invisible from outside and looks
+identical to a dead app.
+
 ## Guardrails
 
 - Destroy every sandbox created during a task unless the user explicitly asks