@vzn/vx 0.0.640 → 0.0.642

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/launcher.sh +9 -2
  2. package/package.json +5 -12
  3. package/skills/vx/SKILL.md +4 -4
  4. package/src/bin.ts +8 -6
  5. package/src/cache/archive.ts +735 -107
  6. package/src/cache/cache.ts +133 -29
  7. package/src/cache/fold-memo.ts +58 -0
  8. package/src/cache/git-inputs.ts +213 -31
  9. package/src/cache/index.ts +9 -2
  10. package/src/cache/inputs.ts +244 -108
  11. package/src/cache/key-fold.ts +90 -69
  12. package/src/cache/layer.ts +9 -7
  13. package/src/cache/layered-cache.ts +74 -9
  14. package/src/cache/output-index.ts +25 -4
  15. package/src/cache/remote-lane.ts +27 -10
  16. package/src/cache/schema.ts +13 -3
  17. package/src/cache/scoped-cache.ts +11 -3
  18. package/src/cache/umask.ts +29 -0
  19. package/src/cli/docs.ts +71 -7
  20. package/src/cli/help.ts +11 -5
  21. package/src/cli/plan-format.ts +4 -1
  22. package/src/cli/run.ts +10 -3
  23. package/src/cli/select.ts +2 -0
  24. package/src/cli/upgrade.ts +37 -2
  25. package/src/cli/watch-filter.ts +1 -6
  26. package/src/cli/watch-fs.ts +2 -1
  27. package/src/exec/runner.ts +5 -0
  28. package/src/exec/sandbox-runtime.ts +81 -1
  29. package/src/graph/priorities.ts +14 -1
  30. package/src/handoff.ts +5 -0
  31. package/src/history/history.ts +43 -33
  32. package/src/index.ts +11 -1
  33. package/src/orchestrator/deferred-outputs.ts +3 -2
  34. package/src/orchestrator/execute-task.ts +20 -14
  35. package/src/orchestrator/hit-restore.ts +111 -29
  36. package/src/orchestrator/index.ts +1 -0
  37. package/src/orchestrator/local-shortcircuit.ts +8 -0
  38. package/src/orchestrator/lockfile-claim.ts +54 -8
  39. package/src/orchestrator/miss-save.ts +27 -17
  40. package/src/orchestrator/output-host.ts +9 -1
  41. package/src/orchestrator/plan.ts +4 -0
  42. package/src/orchestrator/plugin-host.ts +35 -5
  43. package/src/orchestrator/plugin.ts +29 -3
  44. package/src/orchestrator/prepare.ts +74 -5
  45. package/src/orchestrator/projects.ts +6 -1
  46. package/src/orchestrator/run-report.ts +30 -9
  47. package/src/orchestrator/run.ts +62 -38
  48. package/src/orchestrator/sandbox-request.ts +1 -48
  49. package/src/orchestrator/save-lane.ts +17 -10
  50. package/src/orchestrator/signals.ts +7 -10
  51. package/src/orchestrator/stable-keys.ts +83 -25
  52. package/src/orchestrator/task-hash.ts +46 -12
  53. package/src/report/events.ts +12 -0
  54. package/src/report/framed-output.ts +2 -1
  55. package/src/report/logger.ts +58 -15
  56. package/src/report/status-line.ts +45 -4
  57. package/src/report/summary.ts +11 -6
  58. package/src/util/errors.ts +1 -1
  59. package/src/util/hidden-prompt.ts +74 -0
  60. package/src/util/index.ts +13 -0
  61. package/src/util/paths.ts +58 -5
  62. package/src/util/secrets.ts +150 -0
  63. package/src/workspace/affected.ts +63 -13
  64. package/src/workspace/config-eval.ts +2 -2
  65. package/src/workspace/config-schema.ts +37 -3
  66. package/src/workspace/project-loader.ts +6 -4
  67. package/src/workspace/workspace.ts +17 -11
  68. package/docs/caching.md +0 -2352
  69. package/docs/cli.md +0 -3280
  70. package/docs/execution.md +0 -675
  71. package/docs/features.md +0 -178
  72. package/docs/patterns.md +0 -219
  73. package/docs/schema.md +0 -2461
  74. package/docs/security.md +0 -252
  75. package/src/cli/docs-corpus.ts +0 -21
  76. package/src/cli/docs-text.d.ts +0 -5
package/docs/execution.md DELETED
@@ -1,675 +0,0 @@
1
- # Task execution lifecycle
2
-
3
- This document traces what happens between `vx run build` typed at the
4
- terminal and a task succeeding or failing. Read it alongside
5
- [`architecture.md`](./architecture.md) (module map) and
6
- [`caching.md`](./caching.md) (cache mechanics).
7
-
8
- ## End-to-end timeline
9
-
10
- ```
11
- ┌─ CLI dispatch (src/bin.ts → src/cli/index.ts → src/cli/run.ts)
12
- │ 1. bin.ts spawns; forwards process.argv to cli.run().
13
- │ 2. cli/index.ts dispatches by subcommand (run / watch / cache /
14
- │ lock / init / upgrade / show / info / why / last / docs /
15
- │ completions / help / version); any other verb is asked of the
16
- │ workspace's plugins (`commands` seam).
17
- │ 3. cli/run.ts:parseRunArgs(argv) → RunArgs (validated; resolves
18
- │ the 4-axis cache policy from --cache-policy / --no-cache / --force).
19
- │ 4. cli/run.ts:runCmd resolves the project scope:
20
- │ - bare positionals respect --all / --filter / --affected / cwd
21
- │ - anchored positionals (pkg#task) target directly
22
- │ - no positionals + TTY → interactive picker → pkg#task
23
- │ 5. The options map to RunOptions and the run executes IN THIS
24
- │ PROCESS — always. (A whole-run `backend` seam existed until
25
- │ 2026-08; it moved the scheduler server-side, which is why it
26
- │ went. Per-task placement is the `executor` capability.)
27
- │ --dry short-circuits into planRun instead.
28
- │
29
- ├─ Prepare (src/orchestrator/prepare.ts:prepareRun — shared with planRun)
30
- │ 1. findWorkspaceRoot(cwd) — the directory of the nearest
31
- │ .vx/workspace.* up to the git top-level. An UNSCOPED run (no
32
- │ explicit project scope, at least one bare task name) then
33
- │ starts step 12's git enumeration over the whole tree,
34
- │ overlapping steps 2–11; a scoped run waits, since its
35
- │ pathspecs depend on which projects the task graph holds.
36
- │ 2. Read .vx/projects.lock.json: the project directories `vx lock`
37
- │ found (every directory holding a vx.config.*, git's ignore
38
- │ rules applied) and their configs. Each locked config file is
39
- │ stat'ed; one gone is refused ("run `vx lock`").
40
- │ 3. loadWorkspacePlugins — loadWorkspaceConfig reads the
41
- │ .vx/workspace.{ts,mts,js,mjs,cts,cjs} the root lookup found (run / cache /
42
- │ watch / plugins / lock),
43
- │ then the plugin `config` stage runs on it.
44
- │ 4. The plugin `discover` stage: more directories, names and
45
- │ edges. A project is named by its config's `name`, else a
46
- │ plugin's, else its path (`//` for the root); two of one name
47
- │ are refused naming both directories.
48
- │ 5. buildPackageGraph — project edges the `discover` plugins gave.
49
- │ 6. Local cache open: new Cache(cacheDir, { read, write }, root)
50
- │ with the policy's local slice. Then
51
- │ readClaimedFiles: one read of each root file a `fingerprint`
52
- │ plugin claims (`@vzn/vx-lockfile`: its lockfile and install
53
- │ settings; nothing without one), the bytes for the mid-run
54
- │ check. Core folds none of them into a key.
55
- │ 7. SCOPED config loading — only in-scope projects plus their
56
- │ transitive dependency closure load ('^task' frontier
57
- │ expansion never escapes the closure); one staged load,
58
- │ loadProjects, shared with every reading verb. A run's configs
59
- │ load from .vx/projects.lock.json — no evaluation, no staleness check
60
- │ of its own (vx lock --check is the audit), frozen-env
61
- │ semantics; a missing lock or entry is a hard UserError. The
62
- │ verbs that evaluate (vx lock, the readers) use
63
- │ loadProjectConfigs in one batch: native Bun await import()
64
- │ with a content-hash query-string bust, every config
65
- │ evaluated; the loader validates each
66
- │ TaskConfig shape. Either way the plugin `project` stage runs
67
- │ on each config with a re-validation after every plugin.
68
- │ 8. computeNestedProjectDirs — set of projects rooted inside each
69
- │ project, computed over EVERY config-bearing project (loaded
70
- │ or not) for boundary enforcement.
71
- │ 9. expandRequested (see Task selection below).
72
- │ 10. Cache layer: an injected RunOptions.remoteCache is composed
73
- │ with the local cache into a LayeredCache (it wins); else
74
- │ resolveCache layers each plugin's remote over the local
75
- │ cache, in declared order; else bare local.
76
- │ 11. buildTaskGraph (see below).
77
- │ 12. Bulk git populate — the enumeration step 1 started is
78
- │ awaited, or a scoped run starts it here over the projects
79
- │ that own a task (not the dependency closure step 7 loaded,
80
- │ which a `lint` of one package does not key). THREE spawns at
81
- │ the root, concurrent, and the repository's facts asked while
82
- │ they run (`ls-files -s -v -z` for the index: every tracked
83
- │ path's OID and its cache-state flag;
84
- │ `status --porcelain -z -uall` for
85
- │ the dirty AND untracked sets, the one worktree walk; `var -l`
86
- │ for core.autocrlf and the attributes files git reads outside
87
- │ the tree; the facts — prefix, common dir, object format, index
88
- │ file — read off a plain `.git` directory, else one
89
- │ `rev-parse --show-prefix --git-common-dir --show-object-format
90
- │ --git-path index`,
91
- │ memoized per process and shared with the file hasher)
92
- │ fill the per-project GitFilesCache with file lists + index
93
- │ OIDs. `ls-files --others` is NOT among them — status's
94
- │ `-uall` already answers untracked, and asking git twice
95
- │ walked the same tree again. A fifth, `check-attr`, runs only
96
- │ when an attributes file could rewrite bytes. An index the
97
- │ cache holds no blob-size verdict for (keyed by the index
98
- │ file's hash, A-60) adds `ls-files --debug` for the recorded
99
- │ sizes and `cat-file --batch-check` for the blob sizes
100
- │ `blob_sizes` lacks: an OID whose blob is not the recorded size
101
- │ is not trusted. The run's HashCache is created
102
- │ after it.
103
- ├─ Task selection (graph/task-graph.ts:expandRequested)
104
- │ Bare task names fan out across the resolved candidate projects
105
- │ (every project that declares the task). Anchored entries
106
- │ (`pkg#task`) resolve directly. Duplicates are deduped.
107
- │ Empty result → run returns `{ ok: false, outcomes: [] }` —
108
- │ no task is treated as a CI footgun. The exception: a diff-scoped
109
- │ run (--affected) whose task names are all declared by projects
110
- │ outside the selection is `none-affected` and returns
111
- │ `{ ok: true, outcomes: [] }`.
112
- │
113
- ├─ Task graph (src/graph/task-graph.ts:buildTaskGraph)
114
- │ Starting from the resolved {project, task}[] pairs, walk
115
- │ dependsOn:
116
- │ - 'name' → same-project task
117
- │ - '^name' → task in the nearest deps declaring it
118
- │ (frontier walk; non-holders passed through)
119
- │ - 'pkg#name' → specific package's task
120
- │ Detect cycles — throws with the path. After the graph and key
121
- │ stages, excluded edges (per --exclude-dependencies[=names]) leave
122
- │ the SCHEDULE; each dropped task is still keyed and folded into
123
- │ its dependant (orchestrator/excluded-keys.ts).
124
- │ Each node carries: id (`${project}#${task}`), projectName,
125
- │ projectDir, taskName, config, sorted deps, `requested: boolean`.
126
- │ markSurfacedDeps then flags the display-only `surfaced` tasks a
127
- │ requested GROUP stands for (transparent folders).
128
- │
129
- ├─ Plugins + telemetry (src/orchestrator/run.ts)
130
- │ 1. installPlugins — always called; each plugin's optional
131
- │ setup() hook runs; a throw aborts the run with a clean
132
- │ UserError naming it. With no plugins it runs nothing.
133
- │ 2. Run context capture — always, for the `invocations` row:
134
- │ commit + branch read from `.git` directly, `git rev-parse`
135
- │ only when that reader does not understand the layout; dirty
136
- │ reuses the GitFilesCache's status; CI-env detection,
137
- │ host/os/arch. Never fails a run.
138
- │ 3. Telemetry — only when a plugin has a `telemetry` hook or
139
- │ RunOptions.telemetrySinks is passed: the run-context record
140
- │ (adding captureWorkspaceIdentity) is built and
141
- │ subscribeTelemetry collects the sinks; with ZERO sinks it
142
- │ returns undefined and NOTHING subscribes (the no-telemetry
143
- │ hot path is byte-identical).
144
- │
145
- ├─ Run-level state
146
- │ • runId — UUIDv7 stamped once per `vx run` invocation; every
147
- │ task in the resulting graph carries it.
148
- │ • runStartHrTimeNs — hrtime.bigint() anchor; per-task wallclock
149
- │ spans are stored relative to it.
150
- │ • persistentRegistry — Map<taskId, Subprocess> of long-running
151
- │ children.
152
- │ • liveChildren — Set<Subprocess> of in-flight children; the
153
- │ runner adds/removes each around its spawn.
154
- │ • SIGINT/SIGTERM/SIGHUP handlers (removed in a finally): on
155
- │ signal, forward it (SIGHUP as SIGTERM) to everything
156
- │ in liveChildren + persistentRegistry (and kill any
157
- │ running inputs.runtime probe, C-65), wait
158
- │ 2 s for their groups, SIGKILL
159
- │ what is still there, let the run finish its own
160
- │ end (flush, teardown, cache close), then
161
- │ die of the signal (SIGINT → 130, SIGTERM → 143,
162
- │ SIGHUP → 129), so a shell script running vx stops
163
- │ too; a run that handed a task the terminal exits
164
- │ with that code instead, so Bun restores it. SIGHUP is registered because a task
165
- │ runs in its own session, so a closing terminal
166
- │ reaches vx and nothing else. A second signal
167
- │ SIGKILLs and exits at once.
168
- │
169
- ├─ Cache acceleration (before scheduling)
170
- │ • REMOTE PREFETCH (a cache layer with a remote only) — derive every
171
- │ stable-key cacheable task's key up front (reusing the run's
172
- │ hashCache memo) and fire the remote GETs in the background
173
- │ under a bounded pool. Not awaited before scheduling (the
174
- │ overlap is the point); drained before cache.close(). At most
175
- │ one remote GET per key (shared in-flight map with the lazy
176
- │ read-through).
177
- │ • LOCAL SHORT-CIRCUIT (no remote layer, local reads on, ≥1
178
- │ task) — derive the same stable keys and probe them in ONE
179
- │ batched cache.getMany (a layer without getMany: cache.get once
180
- │ per task under a bounded pool) → a `preProbed` map (hits AND
181
- │ stable misses; execute reuses these probes) + a `restoreTier`
182
- │ set (confirmed hits). Awaited before scheduling; never throws
183
- │ (degrades to the normal schedule).
184
- │
185
- ├─ Scheduling (src/graph/scheduler.ts:runGraph — two-tier)
186
- │ Two ready queues, each on its own lane:
187
- │ - EXEC tier — dep-gated; ready when every dep completed; up
188
- │ to N (concurrency) at once.
189
- │ Priority: transitive-reverse-dependent count (bitset
190
- │ closure), optionally overridden by a `priorities` map.
191
- │ - RESTORE tier — confirmed stable local hits; ready
192
- │ IMMEDIATELY (no dep gate, no failed-dep→skip check — their
193
- │ key is dep-independent) at LOW priority, on a restore lane
194
- │ of 2×N. The drain rule: exec-tier first; a restore never
195
- │ takes an exec slot and the two lanes never wait on each
196
- │ other — except at N = 1, where they share the one slot.
197
- │ On failure: exec-tier dependents are marked `skipped` (exit 1,
198
- │ durationMs 0, no spawn); independent siblings keep running.
199
- │ The scheduler doesn't know about caching; the execute callback
200
- │ is the seam. (An embedder running concurrent runs in one process
201
- │ may pass a shared `inflight` map; admission.ts then dedupes
202
- │ identical-key tasks across them. A plain `vx run` passes none.)
203
- │
204
- ├─ Per-task execution (src/orchestrator/execute-task.ts:executeTask)
205
- │ Each task takes one of three paths:
206
- │
207
- │ A. GROUP — no `exec`.
208
- │ Return success with a derived hash rolled up from upstream
209
- │ outcomes. No spawn, no I/O. Wallclock = 0.
210
- │
211
- │ B. PERSISTENT — `exec.persistent` set.
212
- │ 1. Build isolated env (essentials + passThrough + define +
213
- │ <projectDir>/node_modules/.bin, then
214
- │ <workspaceRoot>/node_modules/.bin, prepended to PATH).
215
- │ 2. runPersistent — Bun.spawn the command; subscribe to
216
- │ stdout/stderr chunk-by-chunk.
217
- │ 3. Resolve `ready` when:
218
- │ - no readyWhen → immediately on spawn
219
- │ - readyWhen matches the output (complete lines OR the
220
- │ trailing partial line) → on that match
221
- │ - child exits before the match → reject with
222
- │ PersistentReadyError('persistent task exited before
223
- │ becoming ready (exit N)'); its output already streamed
224
- │ through the logger
225
- │ - readyWhen set and the timeout (exec.timeout, else the
226
- │ run default) elapses first → fail, SIGTERM, SIGKILL
227
- │ after the grace
228
- │ 4. On ready: stash child in persistentRegistry; return
229
- │ success. Downstream tasks unblock.
230
- │ Note: cache + persistent is a config error (rejected by the
231
- │ project loader). Persistent tasks never write to cache.
232
- │
233
- │ C. NORMAL — `exec.command` only.
234
- │ 1. resolveInputs(files, env, runtime)
235
- │ - glob inputs.files (git-backed, declared-outputs-
236
- │ excluded, nested-projects-excluded)
237
- │ - glob inputs.workspaceFiles from the WORKSPACE ROOT
238
- │ (git-aware, NO boundary rule — the documented escape
239
- │ hatch); resolved paths join the same input list
240
- │ - read host process.env values for inputs.env names
241
- │ - resolve inputs.runtime / workspaceRuntime command
242
- │ output (deduped per run; a non-zero exit throws a
243
- │ UserError that fails this task — its dependents skip,
244
- │ independent tasks continue)
245
- │ 2. hashTaskConfig (memoized per run via HashCache)
246
- │ 3. filterUpstreamHashes (apply inputs.tasks filter)
247
- │ 4. cache.key({ taskId, taskConfigHash,
248
- │ forwardArgs, envValues, runtimeValues,
249
- │ workspaceRuntimeValues, upstreamHashes,
250
- │ pluginParts, inputFiles }) → 16-hex xxh3
251
- │ (pluginParts is whatever the `key` stage contributed,
252
- │ folded between the upstream keys and the input files —
253
- │ see caching.md § Cache key derivation for the order.)
254
- │ (Skipped when the local short-circuit pre-derived it.)
255
- │ 5. If willRead (cache block + a read axis on):
256
- │ consume the up-front probe when present, else
257
- │ cache.get(hash)
258
- │ · hit → up-to-date short-circuit when the on-disk tree
259
- │ already matches; else cleanOutputs →
260
- │ restoreOutputs → replay stored stdout →
261
- │ return cache-hit / cache-hit-remote
262
- │ · miss → fall through; cleanOutputs first (when
263
- │ willWrite) so a stale prior build can't
264
- │ survive the fresh exec
265
- │ 6. Build isolated env. For a cacheable task,
266
- │ describeTaskInputs({ captureInto }) then describes the input
267
- │ set ONCE, before the spawn: the executor gets the input
268
- │ values and the save reuses the captured per-component input
269
- │ fingerprint (a re-fold over the memos, no I/O).
270
- │ 7. runCommand (or runSandboxed when `sandbox` is declared) →
271
- │ Bun.spawn shell with `command` + forwardArgs
272
- │ (shell-quoted). Buffer chunks via onStdout/onStderr.
273
- │ exec.timeout SIGTERMs an overrun → real `failed`, never
274
- │ cached. An attempt that ends while the run is STOPPING
275
- │ (Ctrl-C teardown) classifies as `aborted` instead, whatever
276
- │ its exit — not recorded, not retried.
277
- │ 8. On exit 0 + willSave (willWrite, no tainted upstream):
278
- │ keyStillTrue() — if a claimed root file or an
279
- │ input moved since the key was taken, the result stands
280
- │ but the save is withheld (said on the status line).
281
- │ saveMiss — in the slot: resolveOutputs(outputs) +
282
- │ resolveWorkspaceOutputs → file lists, and the
283
- │ project's git snapshot notes the written output paths
284
- │ so a same-project downstream task doesn't re-spawn git
285
- │ unless its globs overlap. Deferred onto the run's save
286
- │ lane (2×concurrency wide): cache.save(...) — pack
287
- │ <hash>.tar.zst with stdout + outputs/<rel>
288
- │ (+ workspace-outputs/<rel-to-root>), upsert entries +
289
- │ output_files + entry_inputs rows in one transaction.
290
- │ The slot frees at once; dependents wait on the save's
291
- │ `landed` promise (the scheduler's settledOf). Under a
292
- │ LayeredCache with remote writes on, the remote upload
293
- │ fires in the BACKGROUND (drained at end of run).
294
- │ 9. Return TaskOutcome { node, status, exitCode, durationMs,
295
- │ hash, cpuMs?, peakRssBytes?, restored?,
296
- │ wallclockStartNs, wallclockEndNs }.
297
- │
298
- └─ End-of-run
299
- 1. SIGTERM dependency-only persistent children; persistent tasks
300
- the user REQUESTED, those a requested group stands for (through
301
- nested groups, not through a one-shot), and the persistent tasks
302
- they depend on, are kept alive (see below).
303
- 2. Run summary footer (projects / tasks / cache meters + info +
304
- time + result) — counts only real tasks (with `exec`); group tasks
305
- don't pollute the totals; failure frames replay just above it.
306
- 3. Optional --profile Chrome-trace JSON (default profile.json).
307
- 4. recordRunBundle — one transaction writing a `runs` row per real
308
- task plus the `invocations` header row (command, git/CI/host
309
- context, counts). Group + aborted tasks skipped; a run
310
- a signal stopped records nothing.
311
- 5. Telemetry: emit the RunSummaryRecord to every active sink +
312
- await their flush (crash-isolated; skipped when no sink).
313
- 6. Drain background remote prefetches/uploads; cache.close();
314
- sandbox teardown when any task was sandboxed.
315
- 7. Return { ok, outcomes }; ok = every task ended success or a
316
- cache hit (any failed/skipped/aborted → ok = false → exit 1),
317
- and no persistent child exited on its own before the run
318
- stopped it.
319
- 8. FOREGROUND ONLY: if the user requested persistent tasks (dev
320
- servers) and nothing else failed (or under `--continue=always`),
321
- the process now blocks until ONE of them, or a persistent task they
322
- depend on, exits — the
323
- summary is already printed, `▸ <id> running` rows list what's
324
- alive, Ctrl-C reaps the process group. That first exit ends the
325
- session: the others are torn down (SIGTERM, grace, SIGKILL) and
326
- a non-zero exit makes the run exit 1, so a script's `vx run dev`
327
- fails when the server it started fell over. Under
328
- `holdPersistent` (the watch loop) run() instead returns them on
329
- `RunSummary.persistent`, still running, for the caller to stop;
330
- one that dies on its own after that is said, its `stop()` is not.
331
- ```
332
-
333
- ## One command per task
334
-
335
- `exec.command` is a single shell command — there's no multi-step
336
- sequence. Three ways to chain:
337
-
338
- - **Shell composition** — `&&`, `;`, pipes. The shell is the API.
339
- ```ts
340
- exec: {
341
- command: 'gen && tsc && cp -r assets dist/'
342
- }
343
- ```
344
- - **Separate tasks** linked by `dependsOn`:
345
- ```ts
346
- codegen: { exec: { command: 'gen' }, ... },
347
- build: { exec: { command: 'tsc' }, dependsOn: ['codegen'], ... },
348
- ```
349
- - **Group task** that fans out:
350
- ```ts
351
- release: {
352
- dependsOn: ['build', 'test', 'package']
353
- }
354
- ```
355
-
356
- Per-task caching is the right granularity for invalidation. Splitting
357
- gives you per-step caching naturally; combining with `&&` gives you
358
- one cache slot for the whole chain.
359
-
360
- A task is over when its command's own process exits. What a descendant
361
- writes after that — one the command detached with `setsid … &`, or
362
- daemonized — is not the task's output: the save has already run, and a
363
- later hit's restore removes it ([caching](./caching.md#cache-write)).
364
- `wait` for what the command backgrounds.
365
-
366
- ## Env isolation
367
-
368
- The child process gets, in priority order (lowest first):
369
-
370
- 1. **Essential allowlist** (`PATH`, `HOME`, `SHELL`, `USER`, `LOGNAME`,
371
- `TMPDIR`, `TEMP`, `TMP`, `LANG`, `LC_ALL`, `LC_CTYPE`, `TERM`,
372
- `COLORTERM`, `FORCE_COLOR`, `NO_COLOR`, `CI`, `NODE_OPTIONS`,
373
- `COREPACK_HOME`, `PNPM_HOME` — the list is
374
- `ESSENTIAL_ENV` in `src/util/essential-env.ts`).
375
- 2. **`exec.env.passThrough`** names → values from host `process.env`.
376
- 3. **`exec.env.define`** literal name/value pairs.
377
- 4. **PATH augmentation** — `<projectDir>/node_modules/.bin`, then
378
- `<workspaceRoot>/node_modules/.bin`, are prepended so installed
379
- tools (`oxlint`, `vite`, etc.) work without `npx`. Never a sibling
380
- project's bin; those stay invisible. A bin directory whose path holds
381
- PATH's delimiter (`:`) is left out: PATH cannot name it, and split it
382
- became two entries, one relative to the task's cwd. A task whose shell exits 127 or 126 gets one more frame line
383
- (`orchestrator/shell-verdict.ts`): for a bare word, that 127 is the
384
- shell's "command not found", the word (when the command is a plain
385
- `word args…`), the two bin directories vx puts first, and that a
386
- sibling project's bin is never visible (or, when a bin directory was
387
- left out for its `:`, that directory and the fix: move the
388
- workspace), and on 126 `chmod +x`; for
389
- a word with a slash, what the file says — missing (the resolved
390
- path), a directory, not executable by this user, a `#!` interpreter that does
391
- not exist (a CRLF line ending is named as such), or no `#!` line.
392
- An exit above 128 gets the same line for its signal: which one, and
393
- what sends it (the OOM killer, a crash in native code, an abort, a
394
- reader that left a pipe, a ulimit, a seccomp refusal); vx's own
395
- timeout and a shutdown's SIGTERM keep their own lines, and a task the
396
- run's stop killed (the SIGKILL past the grace too) gets none.
397
-
398
- Anything not in these four layers is absent from the child's environment, except
399
- the two vx sets itself — `VX_RUN_WORKSPACE` and `VX_RUN_TASK` — so a
400
- task that shells out to `vx run` in its own workspace is refused
401
- before it forks without bound, and `npm_execpath`, the workspace's
402
- package manager (unkeyed; `schema.md` § `env`). A sandboxed task also gets the sandbox's own
403
- proxy, CA and `TMPDIR` values
404
- ([`modules/sandbox-runtime.md`](./modules/sandbox-runtime.md#the-environment-srt-sets)). This prevents incidental env leakage
405
- between machines and gives reproducible runs. It is hygiene, not a
406
- boundary: an unsandboxed task runs as you and can still read your home
407
- (vx hides its own environment, `modules/util-hide-env.md`). `exec.sandbox` is the boundary ([`security.md`](./security.md#what-the-sandbox-stops)).
408
-
409
- The allowlist + isolation contract lives in
410
- [`modules/env.md`](./modules/env.md) and is the only field the
411
- contract assumes for "what every command needs to function" on
412
- Linux and macOS (Windows is WSL, which is Linux). Adding to the allowlist would be a deliberate
413
- schema-extending change (consumer code expects a particular set;
414
- broader access has cache-stability implications).
415
-
416
- ## Failure handling
417
-
418
- | Failure | Behavior |
419
- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
420
- | Exec exits non-zero | Task is `failed`; cache NOT written; output streamed live + the failure frame replays at run end |
421
- | `exec.timeout` overrun | SIGTERM to the task's group, SIGKILL for whoever is left after the grace; task is `failed` (timed out), exit 143 (137 when the SIGKILL took it, and the line says so), never cached |
422
- | Child killed by Ctrl-C teardown (SIGINT/SIGTERM/SIGHUP), or by an embedder's `RunOptions.signal` abort — which also completes every never-started task `aborted` | Task is `aborted` — not recorded in history. An attempt that ends while the run is stopping is `aborted` whatever its exit: a trap that exits 0 is not cached, and a failure is not retried (item 962). What it printed still shows in its frame. A child killed by a signal while the run is NOT stopping (a supervisor's SIGTERM, a `kill` from another shell) is `failed (exit 143, 128 + SIGTERM)`: retried, shown, and fail-fast trips (item 1100) |
423
- | `execute()` throws (internal error) | Task marked `failed`; stderr written `[vx] internal error in <id>` (a `UserError` reports plainly); a plugin executor's throw reports plainly as `plugin '<p>' (executor '<e>') failed in execute: <reason>` (C-63, C-85) |
424
- | Persistent task exits before ready | Task marked `failed` with `exited before becoming ready (exit N)`; its output already streamed live; `aborted` when the run's stop killed it (C-62) |
425
- | Upstream task fails | Under the default `--continue=deps-ok`, dependents marked `skipped` (exit 1, durationMs 0); no command runs — EXCEPT a restore-tier task, whose confirmed cache hit still restores (its key is dep-independent). `--continue=never` stops dispatch; `--continue=always` runs dependents, never saved |
426
- | Sandbox violation (macOS monitor / Linux structural) | Task is `failed`; violations render in the frame; nothing cached |
427
- | Remote-cache error (500, timeout, corrupt artifact) | Degrades to a cache miss; never fails the run |
428
- | No .vx/workspace.* in cwd or any parent up to the git top-level | `findWorkspaceRoot` throws (UserError); `vx run` exits 1 |
429
- | Same-project task referenced in `dependsOn` not declared | `buildTaskGraph` throws with the offending edge |
430
- | Two projects of one name | `checkProjectNames` throws with both paths |
431
- | Cycle in task graph | `detectCycle` throws with the cycle path |
432
- | Malformed config | `loadProjectConfigs` throws (UserError) with file + field |
433
- | `inputs.runtime` command exits non-zero | UserError naming the command + exit code; that task is `failed`, dependents skip |
434
-
435
- By default (`--continue=deps-ok`) failures don't kill the scheduler —
436
- independent tasks already in flight finish, and unrelated tasks not yet
437
- started still run. `--continue=never` stops dispatch at the first
438
- failure; `--continue=always` runs dependents too
439
- ([`cli.md` § Failure propagation](./cli.md#failure-propagation----continue)). The
440
- overall exit code is 1 if any task ended in `failed`, `skipped` or
441
- `aborted` status, or a persistent child exited before the run stopped
442
- it.
443
-
444
- ## Output capture and rendering
445
-
446
- The orchestrator emits `RunEvent`s through an in-process bus; the
447
- terminal renderer is the always-on subscriber (plugins, telemetry, and
448
- wire forwarders attach beside it). What renders:
449
-
450
- - **Flow-aware policy.** What gets rendered depends on the run's
451
- intent: FOCUSED (no selection flag) streams the requested task's
452
- output raw and live and silences successful dependencies; BROAD
453
- (`--all` / `--filter` / `--affected`) prints news only — one grid
454
- one-liner per executed task, failure frames replayed at run end,
455
- silence for cache hits; truthy `CI` env (and the programmatic
456
- default) keeps full grouped output. `--output-logs` overrides
457
- everything. Full table in [`cli.md`](./cli.md#output).
458
- - **The glyph grid.** Reported task lines share one column grid —
459
- `<glyph> <time> <status> <cache> <name>`. Glyph SHAPE = cache axis
460
- (`⏺` miss / `►` fresh / `⇢` local / `⇣` remote / `◼` failed / `⊘`
461
- skipped, plus `▸` for a pinned persistent row; a live worker row has
462
- no glyph — its ticking elapsed time leads instead); glyph COLOR + the
463
- status word = task axis
464
- (success / failed / skipped / running); the cache word (miss /
465
- fresh / local / remote) spells it out.
466
- - **Buffered, framed (non-focused paths).** `runCommand` listens to
467
- the child's stdout/stderr and calls `onStdout` / `onStderr` per
468
- chunk. Outside focused streaming, the default logger buffers the
469
- chunks per-task and dumps the full body as a framed block on task
470
- completion. No per-line prefix, no interleaving between concurrent
471
- tasks.
472
- - **Cache write.** The captured stdout — its first and last 8 MiB,
473
- the cut middle named — is stored in the entry; replay is stdout-only.
474
- - **Cache hit replay.** The stored stdout is fed through the same
475
- logger path, so it renders per the active flow (streamed raw for a
476
- focused requested task, framed in full mode, silent in broad).
477
- - **Live stream for failures.** Even though cached output is not
478
- written for failures, the live stream means the user sees the
479
- failure as it happens; the full frames replay together at run end,
480
- right above the summary.
481
- - **Status region.** On TTY stdout outside CI, a multi-row region
482
- tracks the run live: a blank separator, pinned `▸` persistent rows,
483
- one row per worker slot (the ticking elapsed time IS the motion —
484
- no spinner), and the live summary section (the same meters as the
485
- final footer). Redrawn in place; forced redraws coalesce under a
486
- 30 ms floor; erased before the summary prints. All default-logger
487
- writes serialize through one writer so content and the region never
488
- interleave.
489
- - **GitHub Actions.** With `GITHUB_ACTIONS` truthy in full mode, task
490
- blocks collapse under `::group::` commands; failures stay open and
491
- emit `::error` annotations.
492
-
493
- A task's stdout and stderr are pipes, so `echo x > /dev/stdout` and
494
- `cmd | tee /dev/stderr` work as in a terminal or under Turbo. Bun's own
495
- `'pipe'` is a socketpair, which Linux will not open by path, and they
496
- failed with "No such device or address" until X-113
497
- (`tests/runner.test.ts` › "a task's stdout and stderr are pipes it can
498
- open by path"; sandboxed in `tests/sandbox-runtime.unsafe.test.ts`).
499
-
500
- There is no special handling for binary output. Stdin is never the
501
- terminal unless the task declares `exec.interactive` and vx's stdin is
502
- a TTY (schema.md § `interactive`): then the task gets vx's stdin,
503
- stdout and stderr, runs alone, and nothing of it passes through vx.
504
- Otherwise the rule splits on `exec.persistent`:
505
-
506
- - **A one-shot task** gets `'ignore'`: it sees EOF at once, so a task
507
- that reads stdin finishes instead of hanging CI
508
- (`tests/runner.test.ts` › "a task that reads stdin sees EOF at once,
509
- never a hang"). A task that needs TTY input declares
510
- `exec.interactive`.
511
- - **A persistent task** gets a pipe vx holds and never writes: it stays
512
- open while vx lives and ends when vx does. A dev server that exits on
513
- stdin EOF — esbuild `--watch`, the Vite case in turborepo#8915 —
514
- became ready and exited 0 under `'ignore'`, and `vx run dev` ended
515
- green (fixed 2026-09-24, `tests/keep-alive.test.ts` › "a server that
516
- exits on stdin EOF stays up while vx runs"). Each task has its own
517
- pipe, so several persistent tasks never compete for keystrokes; none
518
- receives any, so a server's keyboard shortcuts (Vite's `h`, `r`) do
519
- not reach it unless it declares `exec.interactive`. Not the terminal, because
520
- several servers would steal each other's input, and a CI's
521
- `/dev/null` stdin is the same EOF. Turbo's stream mode draws the same
522
- line: its pin is named `nonpersistent_task_sees_eof_on_stdin`.
523
-
524
- A one-shot task ends when its shell exits, not when its pipes close. A
525
- process the task left running (`server & echo up`) keeps stdout and
526
- stderr open, so vx reads on for 250 ms after the shell exits and then
527
- stops. That bound is what keeps a leftover server from hanging the run.
528
- What the leftover writes after it is not captured — not in the frame,
529
- not in the cache entry — and the frame says so with one line on stderr:
530
- `[vx] output after the task's shell exited was cut: a process it left
531
- running still held its stdout/stderr 250 ms later`. Until 2026-09-24 the
532
- cut was silent (nx#35302 reproduced on vx; `tests/runner.test.ts` ›
533
- "output a backgrounded child writes after the drain bound is cut, and
534
- the frame says so"). A task whose output matters waits for what it
535
- starts (`wait`).
536
-
537
- Every surface uses one outcome vocabulary: task axis `success` /
538
- `failed` / `skipped` / `aborted` (+ `running` live), cache axis
539
- `miss` / `up-to-date` (fresh) / `local` / `remote`. The `--verbosity 1`
540
- table spells the combinations out as `success` / `restored-local` /
541
- `restored-remote` / `up-to-date`.
542
-
543
- The colors / framing modules:
544
-
545
- - `util/colors.ts` — ANSI truecolor (`ansi-16m`), gated by
546
- `NO_COLOR` / `FORCE_COLOR` / `isTTY`. Programmatic-logger callers
547
- always see plain text.
548
- - `report/framed-output.ts` — the `┌─` frames, the glyph grid
549
- (`formatTaskRow`), one-liners, persistent markers.
550
- - `report/status-line.ts` — the serialized writer + the worker
551
- region.
552
- - `report/logger.ts` — composes them; resolves the output view
553
- and applies the per-flow visibility policy.
554
- - `report/summary.ts` — the closing footer (wordmark rule,
555
- projects/tasks/cache meters, info and time rows).
556
-
557
- ## Concurrency
558
-
559
- - **Default** — the cores this process may use: `navigator.hardwareConcurrency`
560
- capped by the cgroup CPU quota a container runs under (`cpu.max` on v2,
561
- `cpu.cfs_quota_us` on v1, an ancestor's quota binding too; a 1.5-core
562
- quota is two workers, never below one — `util/cgroup.ts`), or
563
- `.vx/workspace.ts`'s `run.concurrency` field when set. Inside a container the
564
- raw count is the HOST's, and eight workers on a two-core quota is
565
- oversubscription by four; `--concurrency <n>%` is a percentage of the
566
- same capped count.
567
- - **Override** — `--concurrency N` (CLI). CLI wins over workspace
568
- config.
569
- - **`concurrency: 1`** serializes everything while still respecting
570
- topo order: execs and restores share the one slot.
571
- - The scheduler never exceeds the cap; tasks queue. Above 1,
572
- restore-tier tasks run on their own lane, `2 × concurrency` wide,
573
- and never take an exec slot.
574
- - Failure of a task doesn't pause the scheduler — independent
575
- siblings continue running and starting (unless `--continue=never`).
576
-
577
- ### Executor pools
578
-
579
- `concurrency` counts LOCAL worker slots. An executor that runs its tasks
580
- somewhere else declares its own `capacity`, and the tasks placed on it are
581
- admitted against that number instead — so a 64-wide worker pool is not
582
- throttled by a 10-core laptop, and a `--concurrency 1` run still keeps the
583
- pool full. A `capacity` that is not a positive integer is refused:
584
- `plugin '<p>' returned executor '<e>' with capacity <v>: it must be a
585
- positive integer`.
586
-
587
- - **Placement is decided once per task, before scheduling.** `run()` asks
588
- the declared executors, in order, which one takes each task (see
589
- [`schema.md` § `remote`](./schema.md#remote-optional) for what pins a
590
- task here); the scheduler then knows which pool every task will occupy.
591
- - **A pooled task holds nothing on this machine.** The `admit` stage is
592
- asked only for tasks about to run here, with the tasks running here; a
593
- task on an executor pool is never asked and never counted, so it can
594
- never park behind a local reservation.
595
- - **Restore-tier tasks always use the restore lane**, even when placed
596
- on an executor with a `capacity`: a restore is local disk work.
597
- - **No pooled executor declared = the legacy path.** With every task on
598
- the local pool the admission gate is byte-identical to before pools
599
- existed.
600
-
601
- ## Cache control flags
602
-
603
- `--no-cache` turns all four cache axes off: every task runs, nothing
604
- is read or written, and `cleanOutputs` is skipped — the user is
605
- debugging and managing the tree themselves; silently wiping `dist/`
606
- mid-debug would be hostile.
607
-
608
- `--force` turns only the READ axes off: every task re-executes, but
609
- fresh artifacts are still written (outputs ARE cleaned so the saved
610
- snapshot is clean). `--cache-policy=<spec>` gives per-layer control. Full
611
- semantics: [`cli.md` § Cache control](./cli.md#cache-control---cache-policy---no-cache---force)
612
- and [`caching.md` § Cache policy](./caching.md#cache-policy-readwrite-axes).
613
-
614
- ## Planning path (`--dry`)
615
-
616
- `--dry` short-circuits execution. The planner (`orchestrator/plan.ts`)
617
- runs the same setup steps — workspace discovery, config load, package
618
- graph, task graph — and the same per-task hash derivation, then
619
- probes the cache. The probe is read-only; against a remote layer it
620
- is an existence check only (no artifact download, no ingest). The one
621
- deliberate side effect: `inputs.runtime` / `workspaceRuntime`
622
- probe commands DO run, because predicting a key requires resolving
623
- them — keep runtime inputs side-effect-free.
624
-
625
- - `--dry[=text|json]` → text (default) or JSON list of predicted
626
- outcomes per task: `hit-local`, `hit-remote`, `miss`, `no-cache`,
627
- `group`. Formatter: `cli/plan-format.ts:formatPlanText` /
628
- `formatPlanJson`. When the workspace declares more than one executor,
629
- each line also names the executor the task would be PLACED on — the only
630
- surface that makes `exec.remote` checkable before the run. Resolving the
631
- executors is a plugin-factory call, the same class the cache capability
632
- already makes at plan time; a plugin that throws costs the label, never
633
- the plan.
634
- Combining it with `--profile` is a parse error (the profile needs a
635
- real run).
636
-
637
- ## Run artifacts
638
-
639
- - **`--profile[=<path>]`** — Chrome-trace JSON of every task's
640
- wallclock span. Default path: `profile.json` (cwd-relative). One
641
- `tid` per project so concurrent tasks render on distinct lanes.
642
- Open with `chrome://tracing` or https://ui.perfetto.dev.
643
-
644
- The writer is `orchestrator/run-artifacts.ts:writeRunProfile`. A write error is surfaced via `log.status` and does
645
- not change the run's exit code — the run already happened. A CI job
646
- summary is `@vzn/vx-ci`'s.
647
-
648
- ## Why each rule
649
-
650
- - **Cache failures are never cached.** Caching a failure prevents
651
- retry flows. The next run gets the same failure even after the
652
- user fixes the cause.
653
- - **Group tasks are silent in the summary.** Including them in the
654
- count is confusing — "3 of 4 cached" when one was a group that
655
- ran nothing isn't informative.
656
- - **Project scope defaults to cwd.** Most invocations are "build/
657
- test the thing I'm working on". `--all` / `--filter` exist for
658
- the workspace-wide case.
659
- - **The scheduler doesn't bail on first failure by default.** A flaky
660
- test failing shouldn't stop an unrelated build. Independent siblings
661
- continue; only dependents are skipped. `--continue=never` opts into
662
- fail-fast.
663
- - **Misses own the worker pool.** A miss is the critical path, so
664
- the exec lane is `concurrency` wide and only executions take it.
665
- Restores run on their own lane, so warm work never queues behind
666
- ordering it doesn't need and never takes a slot from real work.
667
- - **Forwarded args don't reach upstream tasks.** Otherwise
668
- `vx run build -- --watch` would set `--watch` on every upstream's
669
- build, and upstream cache keys would partition by CLI args that
670
- don't change their behavior.
671
- - **`recordRunBundle` skips group tasks.** They aren't real runs;
672
- analytics queries that sum duration would double-count without it.
673
- - **A requested persistent task keeps the run alive.** The dev server
674
- IS the point of the run; tearing it down the instant it became
675
- ready would make `vx run dev` useless.