@vzn/vx 0.0.641 → 0.0.643

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/package.json +5 -12
  2. package/skills/vx/SKILL.md +4 -4
  3. package/src/cache/archive.ts +525 -104
  4. package/src/cache/cache.ts +85 -11
  5. package/src/cache/fold-memo.ts +58 -0
  6. package/src/cache/git-inputs.ts +213 -31
  7. package/src/cache/index.ts +12 -3
  8. package/src/cache/inputs.ts +169 -79
  9. package/src/cache/key-fold.ts +90 -69
  10. package/src/cache/layer.ts +4 -6
  11. package/src/cache/layered-cache.ts +96 -10
  12. package/src/cache/output-index.ts +25 -4
  13. package/src/cache/plugin-cache.ts +23 -16
  14. package/src/cache/remote-lane.ts +25 -8
  15. package/src/cache/run-store.ts +4 -1
  16. package/src/cache/schema.ts +13 -3
  17. package/src/cache/scoped-cache.ts +74 -237
  18. package/src/cache/umask.ts +29 -0
  19. package/src/cli/docs.ts +71 -7
  20. package/src/cli/help.ts +6 -8
  21. package/src/cli/plan-format.ts +4 -1
  22. package/src/cli/plugin-commands.ts +11 -6
  23. package/src/cli/run.ts +14 -15
  24. package/src/cli/select.ts +9 -9
  25. package/src/cli/workspace-config.ts +8 -2
  26. package/src/config.ts +7 -16
  27. package/src/exec/runner.ts +5 -0
  28. package/src/exec/sandbox-runtime.ts +81 -1
  29. package/src/graph/priorities.ts +14 -1
  30. package/src/history/history.ts +43 -33
  31. package/src/index.ts +14 -0
  32. package/src/orchestrator/affected-tasks.ts +23 -2
  33. package/src/orchestrator/deferred-outputs.ts +3 -2
  34. package/src/orchestrator/execute-task.ts +20 -14
  35. package/src/orchestrator/hit-restore.ts +10 -2
  36. package/src/orchestrator/index.ts +5 -0
  37. package/src/orchestrator/lockfile-claim.ts +54 -8
  38. package/src/orchestrator/miss-save.ts +27 -17
  39. package/src/orchestrator/options.ts +9 -1
  40. package/src/orchestrator/output-host.ts +9 -1
  41. package/src/orchestrator/plan.ts +4 -0
  42. package/src/orchestrator/plugin-host.ts +38 -4
  43. package/src/orchestrator/plugin.ts +49 -5
  44. package/src/orchestrator/prepare.ts +141 -22
  45. package/src/orchestrator/projects.ts +25 -6
  46. package/src/orchestrator/run-context.ts +80 -0
  47. package/src/orchestrator/run.ts +38 -15
  48. package/src/orchestrator/sandbox-request.ts +1 -48
  49. package/src/orchestrator/save-lane.ts +17 -10
  50. package/src/orchestrator/signals.ts +7 -10
  51. package/src/orchestrator/stable-keys.ts +83 -25
  52. package/src/orchestrator/task-hash.ts +52 -12
  53. package/src/report/events.ts +12 -0
  54. package/src/report/framed-output.ts +2 -1
  55. package/src/report/logger.ts +53 -13
  56. package/src/report/status-line.ts +45 -4
  57. package/src/report/summary.ts +8 -2
  58. package/src/settings.ts +11 -11
  59. package/src/util/errors.ts +1 -1
  60. package/src/util/hidden-prompt.ts +74 -0
  61. package/src/util/index.ts +12 -2
  62. package/src/util/paths.ts +33 -14
  63. package/src/util/secrets.ts +150 -0
  64. package/src/workspace/affected.ts +182 -75
  65. package/src/workspace/config-schema.ts +81 -19
  66. package/src/workspace/index.ts +3 -0
  67. package/src/workspace/project-loader.ts +1 -0
  68. package/src/workspace/workspace.ts +17 -11
  69. package/docs/caching.md +0 -2376
  70. package/docs/cli.md +0 -3285
  71. package/docs/execution.md +0 -676
  72. package/docs/features.md +0 -178
  73. package/docs/patterns.md +0 -219
  74. package/docs/schema.md +0 -2468
  75. package/docs/security.md +0 -253
  76. package/src/cli/docs-corpus.ts +0 -21
  77. package/src/cli/docs-text.d.ts +0 -5
package/docs/caching.md DELETED
@@ -1,2376 +0,0 @@
1
- # Caching
2
-
3
- `@vzn/vx`'s cache is content-addressed, opt-in per task, and shaped to
4
- cascade through the dependency graph the same way Turborepo's does.
5
- This page explains _what's in the cache key_, _what triggers
6
- invalidation_, _what's actually stored_, and _why_.
7
-
8
- ## Why caching is opt-in
9
-
10
- A task is cached iff its `TaskConfig` provides a `cache` block, with
11
- `cache.outputs.files` and the task's `inputs.files`. Omit `cache` and
12
- the task always runs; no read, no write. `inputs` stands alone: a task
13
- is keyed on it whether it caches or not, and its dependents fold that
14
- key.
15
-
16
- The reasoning:
17
-
18
- - **Defaulting caching ON with implicit globs leads to silently stale
19
- builds.** The first time a user forgets to revisit their config to
20
- add an input, the cache returns a hit for an out-of-date snapshot.
21
- Stale hits are the worst failure mode of a task runner — they erode
22
- trust in the cache itself.
23
- - **Forcing declaration makes "what does this task read?" and "what
24
- does it produce?" conscious choices.** The user pays a one-time
25
- cost (write the globs) for a permanent gain (the cache key actually
26
- reflects reality).
27
- - **The cost of a forgotten cache miss is small.** A task re-runs.
28
- The cost of a stale cache hit is large. Asymmetric risk justifies
29
- asymmetric defaults.
30
-
31
- Turbo defaults caching ON for `outputs: []` tasks; Nx requires you to
32
- opt _out_ via `cache: false`. We chose the strictest of the three.
33
-
34
- ## Cache key derivation
35
-
36
- Core keys no package file: no `package.json`, lockfile or install
37
- setting is a part of its own. A `package.json` keys a task as an input
38
- it lists (step 10), a lockfile or install setting as a plugin's key
39
- material (step 9); `@vzn/vx-lockfile`'s `pnpm()`, `bun()`, `npm()` and
40
- `yarn()` add both. Without a lockfile plugin an install moves no key, so
41
- a JS workspace declares one (`vx init` and `@vzn/vx-migrate` write it).
42
-
43
- The cache key for one task is a **16-hex xxHash3 digest**, seed-chained
44
- over (in order):
45
-
46
- 1. **`CACHE_VERSION`** — the key-derivation sentinel
47
- (currently `'vx-cache-v43'`, in `src/cache/key-fold.ts`). Bumped when
48
- the key derivation or the artifact container changes, or stored bytes
49
- are wrong under an unchanged key. See
50
- [§ Bumping CACHE_VERSION](#bumping-cache_version).
51
- 2. **`taskId`** — `${projectName}#${taskName}`. Two tasks with
52
- identical everything else still produce distinct keys — protects
53
- against e.g. `pkg-a#build` accidentally cache-hitting on a
54
- `pkg-b#build` entry.
55
- 3. **Task config hash** — `xxh3(JSON.stringify(hashableConfig(node.config)))`
56
- of the _evaluated_ task config. The projection is one field wide:
57
- `exec.remote` is dropped, because it says WHERE a task runs and not
58
- what it does (§ What's NOT in the key). Captures:
59
- - `exec` block (command, env declarations, timeout, persistent).
60
- - `dependsOn` and `inputs.tasks` declarations.
61
- - `cache.outputs.files`, `inputs.files`, `inputs.env`,
62
- `inputs.runtime` / `workspaceRuntime` declarations (the
63
- strings themselves; their resolved file content / env values /
64
- command output contribute separately).
65
- - `description` (because it's part of the resolved object — even
66
- though it has no behavioural effect; a description change isn't a
67
- correctness change but the cost of a re-run is low).
68
- - **Imported / computed values** — anything a preset or
69
- `process.env`-read at config-load time injected. Bun's native
70
- `await import()` evaluates the module and bakes those values into
71
- the object before we serialize.
72
- 4. **`forwardArgs`** — CLI args passed after `--`. Folded into the
73
- key so `vx run test -- --watch` doesn't cache-hit a previous
74
- `vx run test`. Scoped to the user-requested tasks only — dependsOn-
75
- pulled deps don't see them (their cache identity stays clean) — and
76
- to a command: a requested default `build` runs none and folds none
77
- (X-119).
78
- 5. **`inputs.env` resolved values** — `[name, value]` pairs
79
- read from host `process.env` at hash time (delimited `name\0value`
80
- so boundaries are unambiguous). Listed names get their current
81
- values; an unset name folds its bare name, with no `\0`, so unset
82
- and set-to-empty are two keys (the child tells them apart:
83
- `passThrough` leaves an unset name out). A name holding a NUL is
84
- refused at load.
85
- 6. **`inputs.runtime` resolved output** — `[command, output]`
86
- pairs, where `output` is the trimmed stdout of each command run via
87
- `sh -c` in the **project dir** at hash time; a command with stderr
88
- (or a NUL in stdout) folds `\0<stdout length>\0<stdout><stderr>`,
89
- so bytes moved between the streams move the key.
90
- The runtime-output analog of step 5: the command _strings_ are in
91
- the resolved config (step 3), their _output_ is resolved live every
92
- run. Folded with the command count + each `command\0output` pair.
93
- 7. **`inputs.workspaceRuntime` resolved output** — same as step
94
- 8 but commands run at the **workspace root**, and the pairs fold
95
- into a **distinct namespace** (`ws-runtime-values:`) so an identical
96
- `(command, output)` never aliases the project-cwd `runtime` values.
97
- 8. **Filtered upstream task cache hashes** — every upstream task's
98
- own cache key, filtered by `inputs.tasks` (default: all of
99
- them). Sorted by hash before folding so the ordering of `dependsOn`
100
- doesn't change the key. This is the cascade mechanism: if anything
101
- beneath you changes, your hash changes too. The set is `dependsOn`'s,
102
- never the run's selection: a dependency `--exclude-dependencies`
103
- keeps from running is still folded, with the key a full run would
104
- derive for it (nx#35234), so the flag moves no key. Its outputs are
105
- whatever is on disk, which nothing proves current, so a task whose
106
- key folds one, and everything built on it, may hit but does not
107
- save (`orchestrator/excluded-keys.ts`); a task with
108
- `inputs.tasks: []` folds none and saves as usual.
109
- 9. **Plugin key material** — the `{ name: value }` pairs a plugin's
110
- `key(task, ctx)` stage returned for this task, stored on the node as
111
- sorted `plugin/name` pairs and folded after the upstream keys and BEFORE the input files,
112
- ONLY when non-empty — so a workspace with no `key` plugin derives
113
- byte-identical keys to one before the stage existed (no
114
- `CACHE_VERSION` bump when it shipped). `vx why` names a changed pair
115
- as `plugin <plugin>/<name>`. Two plugins of one package (a plugin's
116
- name is its package's) that return one name are told apart as
117
- `<name>`, `<name>#2`, … in value order, so declaration order still
118
- keys nothing (item 1028).
119
-
120
- A plugin that reads a root file claims it (`VxPlugin.fingerprint`).
121
- `@vzn/vx-lockfile`'s `pnpm()` folds the project's own resolved
122
- dependency closure from `pnpm-lock.yaml` and the root package's, so
123
- `pnpm update foo` re-keys only the projects that depend on `foo` —
124
- every project when the root does, since the root's tools run from the
125
- root `node_modules/.bin` on every task's PATH; `bun()`, `npm()` and
126
- `yarn()` are the same for their lockfiles (`npm()` reads
127
- `npm-shrinkwrap.json` over `package-lock.json`, as npm does; `bun()`
128
- refuses a `bun.lockb` with no `bun.lock`, which it cannot read), and
129
- this repo declares `bun()`. Each also claims its manager's install
130
- settings (`pnpm()`: `pnpm-workspace.yaml`, `.npmrc`; `bun()`:
131
- `bunfig.toml`, `.npmrc`; `npm()`: `.npmrc`; `yarn()`: `.yarnrc.yml`)
132
- and folds each whole into every task.
133
-
134
- The run reads each claimed root file once, before any task. A task
135
- in the root project may rewrite one — `pnpm install` without
136
- `--frozen-lockfile` updates the lockfile and the installed tree — and
137
- every key taken before it names the old one. So a task that may
138
- (unsandboxed in the root project, or granted a write over one) tells
139
- the run when its command ran, the run re-checks the claimed files
140
- then (one `stat` each, following a symlink as both reads do — an
141
- `lstat` missed a lockfile rewritten through a link, item 760 — the
142
- bytes compared for any written since the read), and once one moved
143
- nothing keyed on the old bytes is probed or saved for the rest of the
144
- run, with one status line naming the file. Every reader after such a
145
- task is keyed late, not up front. Without it the run after a lockfile
146
- rewrite restored a build made against the old install, and a save
147
- filed a build against the new install under the old lockfile's key,
148
- replayed whenever the tree went back (item 750,
149
- [`modules/fingerprint-watch.md`](./modules/fingerprint-watch.md)).
150
- With nothing claimed, no root file is read or watched.
151
-
152
- 10. **Input files' content hashes** — `inputs.files` resolved to
153
- a concrete list of project-relative paths (gitignore-aware,
154
- declared-outputs-excluded, nested-projects-excluded), each file
155
- contributing its **git blob OID** (v20). On a clean tree the OID
156
- comes straight from the index — the run's up-front enumeration is
157
- three concurrent spawns, `git ls-files -s -v -z` (every tracked
158
- path, its OID and its cache-state flag),
159
- `git status --porcelain -z -uall --ignored=matching --no-renames`
160
- (dirty tracked paths, the untracked files and the ignored ones) and
161
- `git var -l` (the clean-filter gate's config) — so deriving these hashes
162
- costs zero file reads and zero per-file stats. Each trusted OID's
163
- blob must be the size the index recorded for the file (A-60): `git
164
- add` under a clean filter (`core.autocrlf=true`, a `text` rule)
165
- stores the LF blob of a CRLF file, and once the filter is gone git
166
- holds that stat-clean entry clean without re-reading it, so status
167
- and the filter gate (today's config) both let the LF blob key the
168
- CRLF bytes. Which paths an index distrusts is a function of its
169
- entries, so the verdict is kept in `blob_verdicts` by a hash of
170
- the index file and the pathspecs: a warm run reads the file and
171
- one row (the `--debug` listing and a lookup per entry cost 550 ms
172
- at 100,000 files). The key stands only for an index written before
173
- the run began and still in place after its listing, so a `git add`
174
- between the two cannot pair one index's verdict with the other's
175
- entries; any other run checks every blob. A changed index spawns
176
- `git ls-files -s -v -z --debug` for the recorded sizes, takes each
177
- blob's size from `blob_sizes` (fixed for its OID), and asks one
178
- `git cat-file --batch-check` for the ones not yet known (65 ms
179
- over 3,000 loose objects, 10 ms packed). The check is by size,
180
- so a filter that keeps the size (a `filter` driver such as
181
- `tr a-z A-Z`), removed after an add, still leaves a blob that
182
- stands for other bytes: probed, the run hit the build of the
183
- filtered bytes. Only a read of every trusted file would catch it,
184
- the cost the index OIDs exist to avoid. After changing a filter,
185
- `git add --renormalize .` makes the index describe the worktree
186
- again. A
187
- re-listing mid-run, or a nested repository's project, spawns
188
- `git ls-files -s --others --exclude-standard -z .` in the project
189
- dir instead, and its OIDs are not trusted: those files hash by
190
- content. `--no-renames`, because status pairs a deleted file with a
191
- similar unmerged path (a conflict mid-resolve) as its rename source
192
- and prints only `UU <path>`: the deletion went unsaid, the file kept
193
- its index OID, and the run hit the output built with it (A-59).
194
-
195
- Ref storage is not a key input: every ref vx reads comes from a git
196
- command, so a repository in reftable storage (git 2.45) keys every
197
- task as its files-backend twin. A sha256 repository keys from its
198
- 64-hex index OIDs, and a dirty file is hashed in-process as sha256,
199
- so Git 3's defaults (reftable and sha256 together) run, hit and
200
- select `--affected` as a sha1 repository does
201
- (`tests/git-reftable.unsafe.test.ts`).
202
-
203
- A **submodule or an embedded repository** is enumerated by its own
204
- git: the workspace repository lists the nested one as a single entry
205
- (a gitlink, or `dir/` when untracked) and none of its files, so vx
206
- replaces that entry with the files `git ls-files` lists inside it
207
- (one spawn per nested repository per run, nested ones recursively),
208
- and those files hash by content rather than by index OID. That holds
209
- for a nested repository inside a project (`vendor/lib` under a
210
- `**` glob — until 2026-09-27 its files never reached the key, and an
211
- edit there was a hit on the old output), for a project inside one,
212
- and for a `workspaceFiles` glob. A gitlink whose directory has no
213
- `.git` (a submodule never initialised, or one whose `.git` was
214
- removed to vendor its files, the gitlink left in the index) has no
215
- repository to ask and `git status` says nothing of it: its files
216
- are listed by a walk and hash by content (A-61), so an empty one
217
- folds nothing. `--affected` follows the
218
- same shape — git reports the nested repository as one changed path,
219
- and every project under it is selected.
220
-
221
- A task with **no `cache` block** derives a key too, from its
222
- `inputs` — its dependents fold it (step 8), as Turbo and Nx hash
223
- every task. That key is for its dependents only: the task always
224
- runs, and no cache, local or remote, is asked about or handed
225
- anything under it; its log goes to the run's own store
226
- (`run/<id>/<task>/log`). A lookup follows `cache`, never a key
227
- (`tests/uncached-no-lookup.test.ts`). One that also declares no
228
- `inputs` is **keyless**: no
229
- key can stand for what it read, so a key that folds it is this
230
- run's alone, and every task behind it misses and saves nothing.
231
- A task reading `**` with no `cache` folds its own outputs too,
232
- since it declared none: its key moves whenever it writes a file
233
- git does not ignore, so a cached dependent misses once more after
234
- the upstream's first run. Ignore the outputs in git, narrow its
235
- `inputs`, or declare the block (`tests/uncached-upstream-key.test.ts`).
236
-
237
- Declaring no outputs, such a task may also have **written** anywhere
238
- in its project, so what the run learned about that project at its
239
- start — the git listing and the index OIDs — is dropped once its command exits, pass or fail, and a later key
240
- re-lists and hashes by content. For the same reason a task after it
241
- in the same project (`dependsOn`, even with `tasks: []`) is never
242
- keyed up front by the local short-circuit or the remote prefetch.
243
- Without both, an uncached `gen` that copies a seed into a `build`'s
244
- declared `config.json` replayed seed B's build under seed A
245
- (turborepo#13788, item 743). A sandbox narrows the reach to its
246
- write grants: none reaches nothing, and a grant elsewhere in the
247
- workspace reaches every project. An unsandboxed write into another
248
- project crosses a project boundary and is not tracked, as for a
249
- cached task's undeclared write (`tests/undeclared-writes.test.ts`).
250
-
251
- A task **with** a `cache` block may still rewrite its own inputs in
252
- place — a formatter declaring `outputs: []` — and a same-project
253
- reader after it whose key does not fold its key (`tasks: []`, or a
254
- filter that leaves it out, on every path) is not keyed up front
255
- either: its key would be taken over the bytes before the rewrite, and
256
- seeds A,B,B,A replayed B on the fourth run (item 750). A reader that
257
- folds the rewriter's key keeps its up-front probe — that key names
258
- the rewriter's inputs, which are what it may rewrite. A cached task
259
- writing a file that is neither a declared output nor its own input
260
- is out of contract: a hit restores only what it declared.
261
-
262
- A file whose name is **not valid UTF-8** (Linux allows any byte
263
- but `/` and NUL) cannot be opened from a string, so it cannot be
264
- hashed: a task whose globs select one is refused by name, and so
265
- is a task whose declared outputs hold one, rather than the file
266
- dropping out of the key or the artifact without a word
267
- (turborepo#9345). Rename it, or exclude it with a negated glob.
268
-
269
- Your globs are a **filter over the set git reports**, so a broad
270
- glob never reaches a gitignored file. An entry that names one
271
- outright does: a literal (`.env.local`, `gen`), or a glob whose fixed
272
- prefix git ignores (`dist/**`), walked on disk under that prefix
273
- only (owner, 2026-10-16). Such a file is hashed from disk with the
274
- blob formula an untracked file gets, and is never stored or
275
- restored. `--affected` reflects only what git sees, so an edit to
276
- one never selects a task on its own (as in Nx); `vx watch` re-runs
277
- it on an edit there.
278
- A **glob** matching nothing stays silent, and so does a literal
279
- naming a path that does not exist, until it does. An ignored
280
- directory named outright folds every file under it; an empty one
281
- folds nothing.
282
-
283
- A **bracket is literal** in a task glob, and there are no character
284
- classes (item 667): `app/[id]/**` folds the route directory
285
- `app/[id]`, and `app/\[id\]/**` is the same path. Read as a class
286
- — what `Bun.Glob`, Turbo and Nx do — it matched `app/i/…` and never
287
- the route, so the route's files keyed nothing (an edit replayed the
288
- old output, green) and an output declared under it cleaned an
289
- unrelated `app/i/page.js` before every run while the artifact
290
- saved nothing.
291
-
292
- An index OID is only trusted where git stores the worktree bytes
293
- **verbatim**, so the enumeration prunes it three ways:
294
- `git status --porcelain` drops paths whose working tree diverges;
295
- the `-v` flag on the same `ls-files` spawn drops `skip-worktree` /
296
- `assume-unchanged` entries, whose OID says nothing about what is
297
- (or isn't) on disk; and, once those spawns return, a clean-filter
298
- gate (a `git check-attr` spawn only when the gate needs one) drops paths where `text` / `eol` / `ident`,
299
- a `filter` driver, `working-tree-encoding`
300
- or `core.autocrlf` can rewrite bytes between index and worktree
301
- (the blob would be the LF-normalized form while the task reads the
302
- CRLF file). The gate costs nothing in a repo with no attributes
303
- and no `core.autocrlf` — see "Clean filters" below. And when the
304
- repository's config weakens the stat `git status` judges by —
305
- `core.trustctime=false`, or `core.checkStat=minimal` (whole-second
306
- mtime and size) — no index OID is trusted at all: a same-size
307
- rewrite that keeps its mtime (`cp -p`, `tar -x`) reads clean to
308
- git, and until 2026-09-27 (A-6) kept the old bytes' key. Every file
309
- is then hashed through the memo below, which keys on ctime and
310
- inode.
311
-
312
- Every pruned path, plus untracked files, falls back to an
313
- in-process `HASH("blob " + len + "\0" + content)` over the
314
- **worktree bytes** (sha1, or sha256 in `--object-format=sha256`
315
- repos), memoized in `file_hashes` on `(mtime, size, ctime, ino)`.
316
- A file changed within `FILE_HASH_RACY_MS` (50 ms) of its stat is
317
- hashed but not memoised; a ctime with no sub-second part, as a
318
- file system that keeps whole seconds writes it (ext3, HFS+,
319
- FAT/exFAT, some NFS), widens that window by a second, and an even
320
- second by two, since FAT32 keeps even seconds; a rewrite later in
321
- the same stamp keeps it (`racyWindowMs`; until 2026-09-27, A-2 and
322
- A-38, such a rewrite of the same size was a hit on the first bytes'
323
- output). Every such window compares a file's stamp, on its file
324
- system's clock, with a moment vx read from its own, so a run first
325
- measures how far the file system runs behind (one file written and
326
- removed in the cache dir) and widens every window by a lag past the
327
- kernel's tick (20 ms): a WSL2 VM drifted from the Windows host that
328
- stamps `/mnt/c`, a Docker Desktop bind mount or an NFS server dated an
329
- edit made while a command ran before the command, and the entry
330
- filed the edit's output under the old bytes' key (until 2026-10-16).
331
- That is the same value the index holds whenever no filter applies,
332
- so a file's contribution doesn't flip across dirty↔clean
333
- transitions. Folded as `(relPath, identity)` pairs, sorted for
334
- stability across OSes and walk orders. The identity is the OID
335
- prefixed by the file's git mode unless that mode is a plain
336
- file's: `100755:<oid>` for an executable, `120000:<oid>` for a
337
- symlink, the bare OID for 100644. The index gives the mode for a
338
- clean file, and the stat gives it for any other file (owner
339
- execute bit). A blob OID holds no mode, so until item 887 a
340
- `chmod +x`, or a symlink swapped for a file holding its target
341
- string, kept the key that `git status` and `--affected` saw move.
342
- Under `core.fileMode=false` (WSL's DrvFs) git reports no chmod, so
343
- there a clean file's mode comes from an lstat too, and its OID
344
- from the index (item 1076).
345
-
346
- A **symlink** folds as git folds it, the blob of its target
347
- _string_ (its mode-120000 index OID), and then the content behind
348
- it, as Nx reads a linked file where it points: a file's identity, or
349
- for a directory every file beneath it at any depth, tracked or not
350
- (`.git`, `node_modules`, `.vx/cache` and `.yarn/cache` aside, as Nx's walker skips them), a link
351
- among them followed in turn. The target may be anywhere: in the
352
- project, elsewhere in the workspace, outside it. Retargeting a link
353
- changes the key, to equal bytes too; an edit behind it changes the
354
- key. A dangling link, or a loop of links, keys its target string
355
- alone. `--affected` selects a link whose target changed (a file
356
- the diff names, or one under a linked directory) as a changed path of
357
- its project, and a target git cannot diff (outside the workspace,
358
- git-ignored) on every run; `vx watch` arms each link's target and
359
- names the link when it changes (`linked-inputs.ts`). Nx does not
360
- enter a linked directory; vx does, since the task reads it.
361
-
362
- The composition is seed-chained (`xxh3(part, prevDigest)`) with a
363
- label prefix per field, so two different field layouts can't collide.
364
- Bun's xxHash3 reads only the low 32 bits of a seed, so `xxh3` feeds the
365
- seed forward (`xxHash3(part, seed) ^ seed`) and the chain carries all 64
366
- bits of state from step to step (item 682; `tests/hash-chain.test.ts`).
367
- xxHash3 is non-cryptographic by design — cache keys need uniqueness
368
- across honest inputs, not adversarial collision resistance.
369
-
370
- ## Cache lookup → restore
371
-
372
- On a hit:
373
-
374
- 1. `cache.get(hash)` is **pure SQL**: the `entries` row carries the
375
- metadata and the captured stdout; the `output_files` rows carry the
376
- output fingerprints. The tar artifact is not touched by the probe
377
- (its existence is verified with one stat), so hit cost doesn't
378
- scale with artifact size.
379
- 2. If the on-disk output tree already matches the cached snapshot
380
- (size + mode + **millisecond**-mtime check against `output_files`),
381
- extraction is skipped entirely — the outcome reports **up-to-date**
382
- (`restored: false`). Mode and millisecond mtime ride the artifact's
383
- own `.vx-meta.json` sidecar — recorded from a stat while packing —
384
- so the index rows and the restored tree carry the identical values
385
- whether the entry was saved locally or ingested from a remote, and
386
- the comparison is exact in steady state. The check also requires the
387
- file's inode and ctime to equal the ones this machine recorded after
388
- the save or restore that last wrote it (`recordOutputStamps`). Size,
389
- mode and mtime alone cannot tell two entries apart when their
390
- outputs carry one fixed mtime (`tar -x`, `cp -p`, `SOURCE_DATE_EPOCH`)
391
- and one size: a v1 → v2 → v1 round trip reported up-to-date with v2's
392
- bytes on disk (item 886). No task can set a ctime, and a forged mtime
393
- (`touch -r`) moves the ctime too. The inode proves less than it
394
- seems: a restore unlinks the file and renames a new one in, and ext4
395
- hands the new one the freed inode (measured, item 941), so ctime is
396
- the guard. A row with no stamp (an ingest from a remote, or a file
397
- that had changed when the stamp was taken) is never current: the hit
398
- restores and stamps. The residual: ctime ticks on the kernel's coarse
399
- clock (4 ms at HZ=250), so a same-size rewrite inside the tick of
400
- vx's own write, in place or as a new file that took the same inode,
401
- with its mtime forged back, still matches.
402
- 3. Otherwise the `outputs/<rel>` (+ `workspace-outputs/<rel>`) entries
403
- are extracted from the artifact (its bytes inline in the index, read
404
- in one read transaction with the entry's rows, or
405
- `<cacheDir>/<hash>.tar.zst`) **in place**: every entry is staged in
406
- the workspace's `.vx/tmp/` (never an output folder), and only once the
407
- whole archive has staged is each renamed over the file it replaces. A
408
- file that already holds the entry's bytes (compared byte for byte, up
409
- to 4 MiB) is left in place, only its mode and mtime set. A
410
- target on another filesystem goes through a temp beside it; a file
411
- takes the group its directory gives new files (setgid on Linux, any
412
- directory on macOS), ACLs are not copied. The next run removes
413
- the staging a killed run left. A process reading the output tree during the
414
- restore never finds a declared output missing or half-written, and a
415
- restore that fails first (a full disk, a damaged entry) leaves the
416
- outputs as they were. Then the declared outputs the entry does not
417
- hold are removed — see
418
- [§ Strict output ownership](#strict-output-ownership). Only a found
419
- output that blocks a rename (a file where the entry needs a directory,
420
- or the reverse) goes before. Turbo also restores in place; Nx deletes
421
- and copies each file, and a reader meets a missing or short file.
422
- 4. Captured stdout is replayed to the live terminal via the logger —
423
- the framed block looks like a fresh run. (stderr is not cached —
424
- only successful runs are cached and their stderr is near-always
425
- empty; live runs still stream stderr normally.)
426
- 5. The task is marked `cache-hit` (or `cache-hit-remote` when the
427
- `LayeredCache` hydrated from the remote layer this run);
428
- `durationMs` is the wallclock for the restore op, not the original
429
- exec time.
430
-
431
- The cached `exitCode` is preserved. A cached non-zero exit is
432
- impossible by construction — see [§ Cache write](#cache-write) — and
433
- "by construction" is literal: neither `save` nor `ingest` accepts an
434
- `exitCode` at all, so the stored value is pinned to `0` rather than
435
- supplied. The restore path still checks it, because the column outlives
436
- this process: a row from a hand-edited or foreign `cache.db` with a
437
- non-zero exit classifies the hit `failed` instead of restoring a broken
438
- build's outputs under a green run.
439
-
440
- ### Local restore tier (two-tier scheduler)
441
-
442
- `dependsOn` is an ordering gate, so a dependent normally can't even
443
- probe the cache until its upstream finishes running. But a
444
- **stable-key** task's key is provably independent of any upstream's
445
- _outputs_ — its hit/miss status is knowable up front, and restoring
446
- it needs none of its deps' output. So on local-only runs, `run()`
447
- performs an up-front CLASSIFY (`orchestrator/local-shortcircuit.ts`):
448
-
449
- 1. Derive every stable, cacheable, local-read task's key (reusing the
450
- run's `hashCache` memo) and probe local `cache.get` **once**,
451
- building a `preProbed` map covering stable hits AND stable misses.
452
- 2. Confirmed hits become the **restore tier**: the scheduler makes
453
- them ready immediately (no dep gate, and no failed-dep→skip check —
454
- their key is dep-success-independent) but at LOW priority, so cache
455
- **misses own the worker pool and restores only backfill idle
456
- capacity**.
457
- 3. `execute-task` consumes `preProbed`, so the up-front probes ARE the
458
- probes it would have done, hoisted — no double work, and every task
459
- still flows through `execute()` so output is unchanged.
460
-
461
- Stability gate (shared with remote prefetch via
462
- `stable-keys.ts:deriveStableKeys`): a task whose input globs could
463
- match a same-project upstream's declared `outputs.files` (compared by
464
- literal prefix, less what its own `!` inputs take back whole; one with
465
- undeclared writes reaches every input), whose
466
- own `outputs.files` meet another same-project task's (it would restore
467
- into that tree beside the producer's restore), or whose
468
- `inputs.workspaceFiles` could reach any upstream's outputs, has a
469
- _preliminary_ key and stays exec-tier / dep-gated. **Any**
470
- `outputs.workspaceFiles` producer upstream also makes a dependent's key
471
- preliminary, whatever that dependent reads: a root-anchored output is
472
- boundary-ignoring by design, so it can land inside the dependent's own
473
- project dir, where an ordinary project-relative glob reads it. Such a
474
- dependent is therefore in NEITHER tier — it is excluded from probe
475
- reuse as well as from the restore tier, because `execute-task` reuses a
476
- `preProbed` hash verbatim, so probe reuse is itself a stale-hit path
477
- when the key is preliminary. On top of that, an `outputs.workspaceFiles`
478
- declaration keeps every task whose project directory its static prefix
479
- can reach out of the restore tier — edge or no edge, because a
480
- root-anchored output can land in any project's directory — and every
481
- transitive dependant of one with it (their up-front keys fold a
482
- preliminary key); a glob with no literal prefix reaches every project.
483
- Before item 584 one such declaration emptied the tier graph-wide.
484
- `vx lock` refuses a task whose input globs can match another task's
485
- declared outputs (X-54, no opt-out), unless git ignores those outputs
486
- whole (and the entry does not start inside them) or the producer is a
487
- dependency its key does not fold, whose bytes it then hashes after the
488
- producer ran, so the clauses on declared outputs fire only for such a
489
- dependency read or a pair the check cannot decide. An output git ignores whole is no reach either:
490
- a key reads only what git lists, so a `**/*` reader beside a gitignored
491
- `**/dist/**` producer keys up front, the same key before and after the
492
- build. The run asks git once, beside its walk and only when a reader
493
- reaches a same-project output: one `git check-ignore` over each such
494
- output's directory (`dist/`, and a fresh `__vx_probe__/dist/` for a
495
- glob that starts with `**`), and the walk's listing must hold no file
496
- the glob matches (a tracked output, a directory a `!` rule takes back);
497
- anything git cannot answer keeps the key late. Undeclared writes, an
498
- `outputs.workspaceFiles` producer upstream, a rewriter the key does not
499
- fold, and runtime probes after a writer still leave a key preliminary.
500
- With a
501
- remote declared the short-circuit probes the local store alone, so a
502
- local hit still restores ahead of its dependencies and no remote GET
503
- sits on the critical path (remote prefetch asks the remote for the
504
- rest). It never fires with local reads off or on a local cache
505
- that holds no artifact (`Cache.holdsNothing`: every probe would miss,
506
- and a cold 1,090-package run started its first task ~90 ms sooner
507
- without it), and never throws —
508
- any error degrades to the normal lazy schedule. Measured: −6.6% on a
509
- mixed slow-upstream/warm-downstream workload; parity on all-hit warm
510
- runs.
511
-
512
- ### Remote prefetch (async, remote-only)
513
-
514
- When a run is backed by a remote cache — a plugin's `cache` capability
515
- or an injected `RunOptions.remoteCache` layer — the network latency of
516
- every remote GET
517
- would otherwise sit on the critical path of the task that needs it.
518
- So before execution starts, `run()` kicks off **background prefetches**:
519
-
520
- 1. Every cacheable task's key is derived once, up front, in
521
- topological order (reusing the run's `hashCache` memo, so
522
- `execute-task`'s later `computeTaskHash` for the same task hits the
523
- memo — no double hashing). This derivation touches **no cache layer**
524
- — keys only.
525
- 2. Each **stable-key** task's remote GET is fired concurrently under a
526
- bounded pool (the run's concurrency). The prefetch ingests a hit
527
- into the _local_ cache; misses/errors degrade to `false`.
528
- 3. Execution starts immediately — the prefetches race alongside it, so
529
- remote latency overlaps real work instead of blocking it.
530
- 4. When `execute-task` later calls `cache.get(hash)`, the `LayeredCache`
531
- **awaits the already-in-flight (resolved-or-pending) prefetch** for
532
- that key rather than starting a fresh round-trip: **at most ONE
533
- remote GET per key**, whether it was served by the prefetch, the
534
- lazy read-through, or both.
535
-
536
- ## A current tree: when a warm hit restores nothing
537
-
538
- A hit whose outputs are already on disk should cost a few stats, not an
539
- extraction. Two checks decide, both against rows the cache recorded when
540
- the tree last matched the entry (`output_files`, and since 2026-09-03
541
- `output_dirs`):
542
-
543
- 1. **The set.** The files under the declared output globs must be exactly
544
- the entry's files — no strays, nothing missing — because a restore
545
- wipes and rewrites the declared outputs, and a stray left in place
546
- would be a stale file the hit silently kept. Until Wave 6 this was a
547
- glob walk on every hit (0.36 ms each; 365 ms of CPU on a warm
548
- 1000-project run). Now, for globs of the shape `<dir>/**` or a bare
549
- literal `<dir>` (a whole subtree — `wholeSubtreePrefixes`; a literal
550
- that names a file is recorded as a file, `mtime_ms` -2, and holds while
551
- a regular file stands there, its bytes being the per-file check's),
552
- the cache records every directory
553
- under `<dir>` with its mtime after each save and restore; on the next
554
- hit, unchanged mtimes on all of them prove the set unchanged, since a
555
- file added or removed anywhere the glob could see bumps its parent
556
- directory, and a new directory bumps the recorded directory that
557
- contains it. The rows are taken per task and written together — one
558
- transaction at the first read, at prune, at stats or at close, since
559
- item 622; a thousand commits at run end were the whole snapshot stage
560
- before — and a reader in the same process flushes them first. A
561
- save's or restore's snapshot is taken at run end, once the directories
562
- are past the racy window, and by then a later task or another process
563
- may have written into them: the walk collects the files it passes and
564
- records nothing unless they are the entry's rows (item 1087). Any other glob shape (`**/*.js`, `dist/*`), a missing
565
- row set (a remote ingest records none), a moved directory, or more
566
- than 8,192 directories (`OUTPUT_DIRS_CAP`) keeps the walk — and a walk that proves the tree
567
- current records the directories so the following hit can skip it.
568
- 2. **The files.** Every recorded file's `(size, mode, mtime-ms)` must
569
- match. This never left: the directory rule only replaces the
570
- enumeration, not the fingerprint.
571
-
572
- The trade is the one every mtime-based skip accepts, already documented
573
- for files: a deliberately forged directory mtime (`touch -r`) hides a
574
- stray. Both directions are pinned in `tests/output-dirs.test.ts`.
575
-
576
- Hard invariants of the remote prefetch:
577
-
578
- - **Remote-only.** This entire path is gated on a `LayeredCache` being
579
- configured. A local-only run never prefetches; its up-front keys and
580
- local probes are the short-circuit's (§ Local restore tier). The
581
- prefetch never adds an upfront _local_ `get` / `isOutputsCurrent` /
582
- stat pass.
583
- - **Stable keys only.** A task with a _preliminary_ key (§ Local
584
- restore tier's stability gate: undeclared writes, an
585
- `outputs.workspaceFiles` producer upstream, or input globs that match
586
- the outputs of a dependency the key does not fold) is not prefetched: its key would name the wrong
587
- artifact, and it resolves via the lazy read-through in `execute-task`. Instability propagates: a task that folds an
588
- unstable upstream is itself unstable. When in doubt, skip.
589
- - **At most once.** The `LayeredCache` keeps an in-flight map keyed by
590
- hash; `prefetch` and `get` share it, and a settled `false` (remote
591
- miss) prevents a second lazy probe of the same dead key.
592
- - **Provenance preserved.** A hash pulled from remote — even when a
593
- later `get` finds it as a now-local hit — still reports
594
- `source: 'remote'`, so the outcome is `cache-hit-remote`.
595
- - **A remote-read-off policy** (`--no-cache`, `--force`, or
596
- `--cache-policy=remote:`) fires no prefetch. With remote writes on
597
- (`--force`), the same up-front keys go to one `hasMany` per remote
598
- the run writes but does not read and that can batch, in the
599
- background: no task waits for it, only each upload does. An upload
600
- of a key that remote holds is skipped (an entry is immutable) and is
601
- counted nowhere. A key the probe did not cover, or a remote with no
602
- `hasMany`, asks `has` for that key inside its upload job; a probe
603
- that fails uploads, as before.
604
- - **No `has` before a `put`.** An entry is immutable: a remote's `put`
605
- of a key it holds keeps the stored bytes, and a wire that would
606
- overwrite checks inside (`turboCache()` asks HEAD unless a 404 or a
607
- batch answer this run said the key is absent; `reapi()` looks the
608
- action up beside its upload and keeps a stored result; Nx servers
609
- answer 409). `ctx.cache.put` and a run's own entries send the `put`
610
- alone.
611
- - **`--dry` batches too.** The plan derives every key first, then asks
612
- each remote with `hasMany` once; a remote without it is asked `has`
613
- per key.
614
- - **Never fail.** Every remote path (get / put / ingest / prefetch)
615
- catches all errors and degrades to a cache miss. A remote 500, a
616
- network drop, a corrupt artifact, or a failed integrity check can
617
- never fail a run.
618
- - **Said once, and said whole.** The warning names the operation
619
- (`probe`, `download`, `upload`), the artifact and the layer's
620
- `endpoint`, with a URL's credentials, query and fragment dropped:
621
- `upload <hash> to <endpoint> failed: HTTP 413`. One failure class
622
- (the error's `code` when it has one, else its message with the hash
623
- taken out) is said once per run, and the repeats are counted at
624
- close: `2 more requests failed the same way: <cause>`.
625
-
626
- ### Remote uploads (background, drained at run end)
627
-
628
- Writes go to the local cache synchronously; the **remote PUT is a
629
- fire-and-forget background upload**. The task's outcome (and its
630
- dependents) never wait on upload latency — the uploads race alongside
631
- the rest of the run and are drained before `cache.close()`, so a
632
- short run still ships every artifact before the process exits. Upload
633
- failures are warned once per failure class and are otherwise ignored (the task
634
- already succeeded; the only loss is the remote entry).
635
-
636
- An upload is the outputs as the task wrote them; secret masking does
637
- not reach file contents, so a task whose outputs embed a secret is one
638
- to leave uncached ([security](./security.md)).
639
-
640
- ### Planning probes (`--dry`)
641
-
642
- The planning path (`vx run --dry`) predicts hits without
643
- side effects: against a remote cache they use a **lightweight
644
- existence probe** — no artifact download, no local ingest. A predicted
645
- `hit-remote` means the artifact exists remotely; the bytes move only
646
- when a real run needs them. Locally the probe reads whether the entry's
647
- row is there and stats the artifact, never the row itself: the whole
648
- row, its stored stdout included, made a 200-task plan over 1 MB outputs
649
- 230 ms against 28 (min of 11, 2026-10-02).
650
-
651
- ## Cache policy (read/write axes)
652
-
653
- Caching is controlled by a four-axis `CachePolicy` — **localRead**,
654
- **localWrite**, **remoteRead**, **remoteWrite** — independent toggles,
655
- each enforced inside the matching cache layer at construction time. The
656
- local `Cache` gets a `{ read, write }` slice gating its task artifact
657
- get/save and the file-hash memo's writes (never `recordRun` / `stats` / `prune` / ingest
658
- / key derivation); the `LayeredCache` additionally gates its own remote
659
- read-through (`remoteRead`), upload (`remoteWrite`), and prefetch
660
- (`remoteRead`). The orchestrator derives two booleans per task:
661
-
662
- - `willRead = task has a cache block AND it is not remote-only AND it folds no keyless task AND (localRead || remoteRead)`
663
- - `willWrite = task has a cache block AND it is not remote-only AND (localWrite || remoteWrite)`
664
-
665
- A task reads the cache only when `willRead`, saves only when
666
- `willWrite` and it folds no keyless task, and cleans its declared outputs
667
- before exec only when `willWrite`. A keyless task runs a command and
668
- declares no `inputs`: its key is unknown, so a task that folds it
669
- (through `dependsOn`, directly or through a group) always misses. Remote-only is an `exec.remote: 'only'` task placed on a
670
- remote executor: it never touches this machine's disk (no probe, no
671
- restore, no output clean, no local save), and its result lives in the
672
- remote executor's own store.
673
-
674
- The CLI maps three flags to a policy (precedence: start all-on → apply
675
- `--cache-policy` → `--no-cache` forces all off → `--force` forces both reads
676
- off): `--no-cache` = everything off (no read, no write, no output
677
- clean); `--force` = reads off / writes on (re-execute, outputs cleaned;
678
- a key already stored, locally or in a remote, is kept as it is); `--cache-policy=<spec>` = explicit per-layer control.
679
- Off CI, unless a spec names the remote, `remoteWrite` starts off: a key
680
- holds only declared inputs, so a laptop's undeclared tool or file could
681
- replay on CI. See `docs/cli.md` § Cache control.
682
-
683
- The setup we recommend for a shared remote: the write token only in the
684
- default branch's CI (a protected environment), a read-only token on
685
- laptops and in pull request jobs, and `cache.scope` (`github()` sets it)
686
- so a PR's own writes, when its token allows them, land apart. The token
687
- is the boundary; the policy and the scope are the client's half
688
- (`docs/security.md` § Cache poisoning).
689
-
690
- One subtlety: when `localWrite` is off but `remoteWrite` is on
691
- (`--cache-policy=local:,remote:rw`), there's no on-disk artifact for the
692
- `LayeredCache` to read before uploading — so it packs the tar.zst bytes
693
- in memory (`Cache.packArtifactBytes`) and uploads those.
694
-
695
- ## Cache write
696
-
697
- A miss runs the task. If the final exit code is `0` and the task's
698
- `willWrite` is true (it has a `cache` block AND at least one write axis
699
- is on):
700
-
701
- 1. `cache.outputs.files` (and `outputs.workspaceFiles`) are resolved
702
- against the project dir / workspace root.
703
- 2. The per-component input fingerprint is the one captured before the
704
- command ran: on a miss `describeTaskInputs` fills `captureInto` once
705
- (the same call yields the key re-checked below), so no second
706
- `computeTaskHash` runs.
707
- 3. The artifact — one `stdout` entry (bounded: the first and last 8 Mi
708
- characters of the task's output, the dropped middle named where it was), the `outputs/<rel>` (+
709
- `workspace-outputs/<rel>`) entries, the `.vx-meta.json` sidecar and
710
- the `.vx-sum` checksum — is packed in-process (no staging dir, no subprocess) into a
711
- single `<hash>.tar.zst` and validated. One of at most 32 KiB
712
- compressed (`INLINE_MAX`, almost every artifact: a one-file `dist/`
713
- is a few hundred bytes) stays in memory; a larger one is written to a
714
- temp name.
715
- 4. One `BEGIN IMMEDIATE` SQLite transaction upserts a small artifact's
716
- bytes into `artifacts` (v33), or renames a large one into place and
717
- deletes the key's inline bytes, and upserts the `entries` row (taskId, command, exit code,
718
- duration, size, stdout, timestamps), the `output_files` fingerprint
719
- rows, and the entry's one `entry_inputs` row (`INSERT OR IGNORE`).
720
- Concurrent readers see either no entry or a complete entry — never a
721
- partial one — and bytes and rows go live together: the rename waits
722
- for the write lock, so no other writer's rows land beside it, and a
723
- commit that fails takes the artifact back out (the key then misses).
724
- Until 2026-09-27 (A-3) the rename came first, and a commit refused
725
- past the busy timeout left the new bytes beside the old rows: every
726
- later hit on the key failed the task as a corrupt artifact.
727
- A `--force` save of a key the index already holds is skipped, so
728
- `--force` rewrites nothing. A re-save (a key dropped as corrupt, or
729
- two writers that both found it absent) first moves the previous
730
- artifact aside under
731
- a temp name and unlinks it after the commit: ext4 flushes the
732
- incoming file when a rename replaces one, 0.55 ms a save on the main
733
- thread against 0.04 (a forced 1,000-task run 4.01 s → 3.46 s,
734
- 2026-10-02). A reader probing between the two renames misses.
735
- An inline save moves aside whatever file its key has, the same way
736
- (a rename that finds nothing on a first save, in place of a statement
737
- asking the index). As of every commit a key's bytes are in one place,
738
- and a reader takes the inline bytes first, so a file an older vx
739
- wrote beside them is never read and the sweep takes it. An inline save writes no file at all: no temp, no rename,
740
- and a crash leaves nothing behind (the bytes and the rows are one
741
- transaction in one database file).
742
-
743
- **The key is re-checked before the save** (item 743). It was taken
744
- before the command ran — at the task's start, or up front by the local
745
- short-circuit — and the save files the outputs under it, so the inputs
746
- must still be what it describes. Two checks, in order. The key
747
- re-derived just before the command (`describeTaskInputs`, which the
748
- executor seam needs anyway) must equal it. Then each input file is
749
- `lstat`ed once: a file whose ctime falls
750
- after, or within `FILE_HASH_RACY_MS` before, the moment its digest was
751
- learned — the enumeration's start for an index OID, the describe for a
752
- hashed file — is hashed again and compared, and a file that is gone has
753
- moved. A file written at or after the describe (just before the command)
754
- has moved whatever it holds (a whole-second stamp counts as any moment
755
- of its second, an even one of two, in both checks): an input changed and changed BACK while the
756
- command ran matched its digest again, and its output, built from the
757
- edit, was filed under the key and restored over the original (item
758
- 1015). When one moved, the task's result stands but no entry is saved,
759
- one status line names the file (``[vx] app#format: `packages/app/a.ts`
760
- changed after its key was taken — …``), and the run forgets what it
761
- knew about the project, as after an uncached task (§ Cache key
762
- derivation, step 10). This covers a formatter rewriting its own input
763
- (turborepo#10111) and a user's edit mid-run (turborepo#1146). A task
764
- left remote (`--cache-download=none`) is held to the same checks: a moved
765
- key still lets a local consumer fetch its outputs, but the fetch saves
766
- no entry; until X-123 it saved them under the old key. A task
767
- that rewrites its own input to the SAME bytes (`sed -i` always writes)
768
- is not saved either: its write cannot be told from an edit reverted
769
- mid-run, so it pays a re-run each time rather than risk a stale entry;
770
- declare what it writes as an output, which the status line says when it
771
- names a file other than `package.json` (TanStack Router's committed
772
- `routeTree.gen.ts`, rewritten by every build). That holds on CI too: a
773
- task of the same project running alongside can change an input and put
774
- it back. A task that rewrites its own input to identical bytes (tsdown's
775
- `package.json`) sets `cache.inputStability: { strategy: 'mutable' }`, and a write
776
- during its command is then judged by its bytes, as an earlier write is;
777
- different bytes still withhold the entry, and so does a write made while
778
- another command that may write the file ran (`command-windows.ts`: one
779
- in the same project, or one with a write grant elsewhere in the
780
- workspace), named on the line as `while <task> ran`. `VX_CACHE_INPUT_STABILITY_STRATEGY=mutable`
781
- makes that the default (owner, 2026-10-12); `'unchecked'` skips the
782
- check, and `severity: 'error'` fails the task instead of warning
783
- (`docs/schema.md`). A file ADDED under an
784
- input glob since the listing the key filtered (`addedInput`) withholds
785
- the save the same way, named on the same line: the listing's
786
- directories a glob reaches are `lstat`ed, one whose ctime moved since
787
- the listing is read, and a new name the declaration matches (or an
788
- unlisted directory) is asked of git, which alone knows what is ignored;
789
- a literal input absent then and present now is asked too. Not seen: a
790
- file added inside a directory that held no listed file before (only
791
- ignored ones, or none) and was not itself created mid-run. Cost: one
792
- `lstat` per input and per reached directory on a miss that saves, a
793
- `readdir` per directory that changed, and a `git ls-files` only when a
794
- candidate turned up; a hit runs no command and checks nothing.
795
- A file listed but gone before the key hashes it (an upstream that
796
- deletes a file its dependant's globs match) is keyed as absent, not
797
- read: the dependant failed as `internal error … ENOENT` on every run
798
- until A-55. Absent stays unmoved while it stays gone.
799
- Its dependants whose keys fold its key save nothing either,
800
- transitively: that key does not name the bytes they built from (`[vx] app#use: ran
801
- over app#gen's outputs, which its key no longer describes — …`). Until
802
- 2026-09-27 (A-12) they saved, and once the input was put back they hit
803
- the edit's output.
804
- A claimed root file a task rewrote since the run read it (§ Cache key
805
- derivation, step 9) withholds the save the same way, and no key
806
- taken before it is probed or restored: a hit the up-front probe found
807
- goes back to the scheduler and runs once its deps are done. Until X-124
808
- that hit restored the old install's bytes.
809
-
810
- A miss that ran here and saves **nothing** — it failed, the cache
811
- policy writes nothing (`--cache-policy=local:r,remote:r`), an upstream failed
812
- under `--continue` — runs the same input re-check and, when an input
813
- moved, forgets the project as above; otherwise its declared outputs are
814
- resolved and marked in the git snapshot exactly as a save marks them.
815
- Before item 750 only a save marked them, so a same-run reader keyed
816
- from the snapshot's index OID for an output (or for an input the task
817
- rewrote) restored the bytes from before the command.
818
-
819
- A declared set that resolves to **nothing** is said on the run's
820
- status line. `inputs matched no files (lib/**)` — the key would
821
- not change when the source does — is said on every miss of a cacheable
822
- task, right after `describeTaskInputs` and before the command, whether
823
- or not it saves. `cache.outputs matched no files (build/**)` — an empty
824
- artifact was saved and a later hit restores nothing — is said on the
825
- save path. Both are almost always a glob
826
- against the wrong directory; the output line names one other cause when
827
- it applies: a sandboxed task with no `exec.sandbox.allow.write`, whose
828
- writes never reached disk, or a `workspaceFiles` directory that is a
829
- symlink out of the workspace, whose files vx drops as outside (a project
830
- output directory linked out of the project refuses the task instead, X-88). `outputs.files: []` is a deliberate cached
831
- no-op and says nothing; a task with no `cache` block is never checked.
832
-
833
- **The outputs are what exists when the task's command exits.** The run
834
- waits for the command's own process, not for everything it started — a
835
- backgrounded grandchild that still holds the output pipe does not hold
836
- the run (`tests/runner.test.ts`) — and the save resolves the output
837
- globs at once. A descendant the command detached (`setsid … &`, a
838
- daemon) that writes after that is not in the artifact: the entry saves
839
- what was there, often nothing (with the `cache.outputs matched no
840
- files` line), and the next hit's restore removes what the
841
- descendant wrote and restores the saved entry in its place (upstream
842
- survey, turborepo#12786). vx does not wait for it and cannot see it: a
843
- process that left the task's session is outside the process group vx
844
- tracks and signals ([kill-tree](./modules/kill-tree.md)). So a command
845
- finishes writing its outputs before it exits — `wait` for what it
846
- backgrounds, or do not detach it.
847
-
848
- If the task exits non-zero, **nothing is cached.** This is deliberate:
849
-
850
- - Caching a failure prevents retry flows. The next run gets the same
851
- failure even after the user fixes the underlying cause (the inputs
852
- haven't changed, so the cache key matches).
853
- - Failures should be transient by default — flaky tests, network
854
- blips, transient resource exhaustion shouldn't bake into the cache.
855
-
856
- Failed-task stdout / stderr still reach the user via the live stream
857
- and the framed failure block replayed at run end. The `runs` table
858
- records the failure (status + exit code) for analytics.
859
-
860
- ## Strict output ownership
861
-
862
- Declared `cache.outputs.files` are wiped in two distinct places:
863
-
864
- - **Before exec on a cache miss.** A leftover `dist/old.js` from a
865
- prior build can't survive into a fresh build that doesn't rewrite
866
- it.
867
- - **After restore on a cache hit.** The post-restore tree is the
868
- cached snapshot byte-for-byte: the restore renames the entry's files
869
- over the ones it finds, then removes what the entry does not hold.
870
- Hand-edits to output files don't persist through a cache replay. When
871
- the hit's own check walked the output globs to find the tree stale,
872
- the removal uses that walk's list instead of walking again (X-161).
873
-
874
- Both use the same helpers (`cleanOutputs`, `cleanOutputPaths` in
875
- `src/cache/inputs.ts`) with the same boundary rules, and vx's own paths stay off the wipe
876
- whatever the glob: `.git`, `.vx` and each `vx.lock.json` (`OUTPUT_NEVER`;
877
- `.vx` holds the workspace config and the lock, which a clean deleted and a
878
- hit restored stale while `.vx/cache` alone was listed). `node_modules` is
879
- off it too unless a glob names it (`node_modules/**`, an install task's
880
- legitimate output); until 2026-09-27 (A-13) `**/*.js` cleaned every
881
- installed `.js`, and a `workspaceFiles` output glob reached even `.git`.
882
- No clean deletes a file git tracks (owner, 2026-10-14). Under an output
883
- glob it is still an output: the task may rewrite it (TanStack Router's
884
- committed `routeTree.gen.ts`), the artifact saves the bytes the run left,
885
- and a restore writes them. It also stays in the task's key, even under its
886
- own output glob, so an edit moves the key and the saved bytes follow from
887
- it; the task's rewrite is no input that moved mid-run. Until then the
888
- clean deleted a committed file under an output, and a reader with no edge
889
- to its producer failed naming only the file (A-48, vueuse's
890
- `metadata/index.json`). Untracked files under the glob are cleaned as
891
- before. Paths compare as
892
- git compares them: case-folded under `core.ignorecase` (macOS), and a
893
- file inside a submodule counts as committed.
894
- Workspace outputs take the same rules, and a path an output `!` entry
895
- takes back (A-44) is off the wipe, the artifact and the restore alike.
896
- Skipped when:
897
-
898
- - `cache.outputs.files` is empty (nothing declared as output).
899
- - The task's `willWrite` is false — no write axis is enabled (e.g.
900
- `--no-cache`, or a read-only `--cache-policy=local:r`). The user is debugging
901
- and managing the tree, so vx leaves it alone. `--force` keeps writes
902
- on, so it DOES clean (the saved snapshot must be clean).
903
- - `cache.outputs.clean` is `false` (on the task, else the workspace's
904
- `cache.outputs.clean`): the command sees last run's outputs (tsc's
905
- `.tsbuildinfo`). You opt out of the clean-tree guarantee: the save
906
- stores every file the outputs match after the run, so a leftover from
907
- an older build is cached and restored everywhere. A retry cleans as
908
- usual; a hit's restore still removes what its entry does not hold.
909
-
910
- A declared output the process cannot remove (a `dist/` another user
911
- wrote, a read-only checkout) fails the task with `cannot remove declared
912
- output <path>: EACCES — …`, and a restore that cannot write into such a
913
- directory with `restore of <hash> into <dir> could not write its outputs
914
- (EACCES: …)`: the environment's failure, reported plainly, never as an
915
- internal error or a corrupt artifact.
916
-
917
- Why so strict? Turbo and Nx restore additively — files from a prior
918
- state can survive. We've seen this cause:
919
-
920
- - Wrong test runs (a deleted-but-resurrected snapshot file from a
921
- cache miss survives a hit and now your test passes against the
922
- wrong baseline).
923
- - Wrong shipped artifacts (a deleted source-mapped file from a prior
924
- build sits in `dist/` alongside the new bundle).
925
-
926
- The strict-ownership behavior makes the project dir post-run a pure
927
- function of the cache key.
928
-
929
- ### Additive outputs: two tasks, one tree, an edge between them
930
-
931
- Two cached tasks whose declared outputs overlap are refused at graph
932
- build. By default that holds even when one depends on the other: the
933
- workspace rule `rules.exclusiveOutputs` (on unless set to `false` in
934
- `.vx/workspace.ts`, X-53) keeps one path to one task, and its refusal says
935
- how to turn it off. The shape below is correct, only slower: a stamp
936
- before every run, a diff after, and a clean by rows. Give each task its
937
- own output path where you can (`dist-individual` beside `dist`).
938
-
939
- With the rule off, an edge fixes the order, and the dependant is
940
- **additive** (item 588): twenty's
941
- `build` fills `dist` and `build:individual` depends on it and writes
942
- `dist/individual`; strapi's `build:types` runs `tsc` into the same
943
- `dist` as `build`. For the dependant:
944
-
945
- - its **own output set** is what its run added or changed under its
946
- declared outputs — the outputs are stamped (size, mtime, inode, ctime) before the
947
- run and diffed after, the same proof a hit's "already current" check
948
- trusts — and only that set is saved; `outputs.workspaceFiles` are
949
- stamped the same way (until A-43 its miss cleaned them by glob, deleting
950
- a same-tree upstream's root-anchored files before it read them);
951
- - a run that **removes** a file it found (a bundler deleting the
952
- upstream's intermediate) saves nothing: an artifact holds what a run
953
- wrote, never what it took away, and the upstream's restore puts the
954
- file back, so the dependant runs again on every warm run (X-32);
955
- - it **cleans by recorded rows**, never by glob, before a run (nothing:
956
- stale files of its own are its command's to clean, as under Turbo) and
957
- before a restore (its rows only, pruning emptied directories only
958
- inside its declared trees, so a sibling's fresh directory stays);
959
- - its "already current" check requires its rows present and current and
960
- ignores everything else under the glob;
961
- - it is **never restore-tier**: it restores or runs after its upstream,
962
- by the edge.
963
-
964
- The upstream keeps strict ownership of its glob — a miss or a restore
965
- wipes the whole tree, the dependant's additions included, and the
966
- dependant restores or runs after — and its "already current" check
967
- ignores strays a dependant's glob could have added, so a warm run stays
968
- a no-op for both. A file the dependant rewrites in place (refine's
969
- `types` regenerating `build`'s `.d.ts`) counts as the dependant's own
970
- (its ctime moved, even where a tool stamps the size and mtime back; X-33),
971
- which is correct and costs the upstream a restore on
972
- the next warm run; the design note keeps that shape out of scope.
973
-
974
- ## Invalidation paths
975
-
976
- A task's cache becomes invalid when any of these change:
977
-
978
- | Trigger | Mechanism |
979
- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
980
- | Edit a file in the task's `inputs.files` set | step 10 of key derivation |
981
- | Edit a file in the task's `inputs.workspaceFiles` set (root-anchored; may live in ANY project's dir — the documented boundary exception) | step 10 — resolved workspace files join the same input-file list |
982
- | Any package manager updates a lockfile (`pnpm`, `npm`, `yarn`, `bun`) | step 9 — the `key` material of the plugin that claims it (`@vzn/vx-lockfile`: only the projects whose dependency closure moved); **NOT a trigger** with no such plugin, since core keys no package file |
983
- | Edit an install setting (`.npmrc`, `bunfig.toml`, `pnpm-workspace.yaml`, `.yarnrc.yml`) | step 9 — the claiming lockfile plugin folds it whole into every task; **NOT a trigger** without one |
984
- | Edit `package.json`'s `workspaces` field | membership: a project that joins or leaves changes which projects exist and which nested dirs its parent's globs exclude (step 10) |
985
- | Edit the project's `package.json` (dep / version / scripts change) | step 10 — an input file of every cached task when `@vzn/vx-lockfile` is declared (it adds `package.json` to `inputs.files`), or when a task lists it; **NOT a trigger** otherwise |
986
- | Edit the task's `vx.config.ts` | step 3 (task config hash) |
987
- | Edit a config file that the task config imports | step 3 (configHash sees the resolved object after Bun evaluates imports) |
988
- | Change CLI `forwardArgs` (after `--`) | step 4 |
989
- | Change a `inputs.env` host value | step 5 |
990
- | Change the combined stdout+stderr of a `inputs.runtime` command (resolved at hash time) | step 6 |
991
- | Change the combined stdout+stderr of a `inputs.workspaceRuntime` command (resolved at hash time) | step 7 |
992
- | Upstream task's cache key changes (because its inputs changed) | step 8 |
993
- | Bump `CACHE_VERSION` | step 1 — orphans every entry |
994
- | Change `exec.env.passThrough` _values_ alone | **NOT a trigger** by design — passThrough values are host-specific |
995
- | Change a file not in `inputs.files` / `inputs.workspaceFiles` | **NOT a trigger** by design — declare it explicitly |
996
- | Edit `.vx/projects.lock.json` | **NOT a trigger** — globally excluded from inputs (v24); it's vx's own metadata, never a task input |
997
- | Change a file in a nested project's dir | **NOT a trigger** for the parent's `files` globs — project boundaries are hard (workspaceFiles is the explicit exception) |
998
-
999
- The cascade in step 8 is what makes monorepo caching work: edit a
1000
- file in `lib/`, and every package that depends on `lib`'s `build` task
1001
- re-runs automatically.
1002
-
1003
- ## Cross-project boundaries
1004
-
1005
- A project's `inputs.files` globs **never** reach into another
1006
- project's directory, even if a `**/*` pattern would otherwise match.
1007
-
1008
- `workspace/nested-dirs.ts` computes the set of nested project
1009
- directories (projects rooted inside this one) once per `vx run`. A
1010
- workspace package counts whether or not it has a vx config: it is the
1011
- project `--affected` gives its files to, and its default `build` keys
1012
- them (X-57). The same set fences the output clean, so a root
1013
- `outputs.files: ['**/*.js']` never removes a member's source. Every
1014
- glob pass drops a path under one of them (an ancestor lookup,
1015
- A-11). The only way
1016
- for project A to depend on project B's state via project-relative
1017
- globs is `dependsOn` + upstream-hash propagation (step 8).
1018
-
1019
- For a task with `exec.sandbox` and `cache`, the sandbox enforces that
1020
- "only". A workspace package reached through a `node_modules` link is
1021
- granted only when the task's key folds a task of that package, walking
1022
- the same selection `inputs.tasks` applies at hash time
1023
- (`orchestrator/keyed-projects.ts`, over `graph/folds.ts`'s
1024
- `selectFoldedDeps`); every other link target inside the root is
1025
- withheld, so an import of a sibling the key never sees fails and is
1026
- reported instead of caching a result an edit there would not re-run.
1027
- Any exec task counts, cached or not: its key folds its project's files
1028
- through its `inputs`, and one with no `inputs` is keyless, so a key that
1029
- folds it is never stored or read. A persistent task counts the same way,
1030
- so a cached e2e behind a dev server keyed on its sources re-runs when
1031
- they change.
1032
- The coverage is per package, not per file: an edge to `ui#source`
1033
- (inputs `src/**`) also admits a read of `ui/README.md`, whose edit moves
1034
- no key — the same limit a grant wider than a task's own inputs already
1035
- has. A task with no `cache` keeps every link, having no key to be stale.
1036
-
1037
- **Exception:** `inputs.workspaceFiles` /
1038
- `cache.outputs.workspaceFiles` are workspace-root-anchored and apply
1039
- NO boundary rule — a deliberate escape hatch (owner call: "they don't
1040
- care about boundaries; it is bad practice but is there"). Prefer
1041
- project-relative declarations; reach for workspaceFiles only for
1042
- genuinely root-anchored files.
1043
-
1044
- ## Clean filters (`text` / `eol` / `ident`, `core.autocrlf`)
1045
-
1046
- Git can store a file's blob in a different form than the bytes in your
1047
- working tree. Under a `text`, `eol` or `ident` attribute — or with
1048
- `core.autocrlf` set to `true`/`input` — the index holds the
1049
- LF-normalized (or ident-collapsed) blob while your editor and your
1050
- build see the CRLF (or expanded) file.
1051
-
1052
- That matters because `git status` compares **after** applying the
1053
- filter, so such a file reports _clean_: git considers it unmodified
1054
- even though the blob and the worktree file are different byte
1055
- sequences. Trusting the index OID as the file's content hash would
1056
- then let two genuinely different worktree contents fold the **same**
1057
- cache key, and a real change would be invisible.
1058
-
1059
- So vx does not trust an index OID where a filter can apply. The check
1060
- is gated in three steps, and the common case pays nothing:
1061
-
1062
- 1. `core.autocrlf` is `true`/`input` — conversion applies to every
1063
- auto-detected text file with no attribute needed, so no OID is
1064
- trusted.
1065
- 2. Otherwise, if no attributes source exists anywhere (no in-tree
1066
- `.gitattributes`, an ignored one included, which `git status
1067
- --ignored=matching` names from the walk it already does, no `$GIT_DIR/info/attributes`, and none of the
1068
- files git reads outside the tree: the global one, which is
1069
- `core.attributesFile` or by default `$XDG_CONFIG_HOME/git/attributes`
1070
- or `~/.config/git/attributes`, and the system one), no rule can name
1071
- a filter and vx does **no** extra work. This is the default
1072
- `git init` repo. The gate reads `git var -l`, one spawn, which from
1073
- git 2.42 names the global and system files itself
1074
- (`GIT_ATTR_GLOBAL`, `GIT_ATTR_SYSTEM`); for an older git the global
1075
- lookup is mirrored and the system file is looked for at
1076
- `/etc/gitattributes` and `<prefix>/etc/gitattributes` beside the
1077
- binary, where a standard build puts it. Before 2026-09-24 the gate
1078
- missed the default global file, so `* text` there left a CRLF file
1079
- keyed on its LF blob: a CRLF→LF edit was a stale hit.
1080
- 3. Otherwise `git check-attr` resolves `text`, `eol`, `ident`, `filter`
1081
- and `working-tree-encoding` from the index — without reading worktree
1082
- content — and only the paths actually carrying one lose their OID. A
1083
- `filter` driver was missing from the list until item 978: a clean
1084
- driver that drops comment lines (`sed '/^#/d'`, nbstripout's shape)
1085
- stores one blob for two worktree files, so an edit git calls clean
1086
- replayed the old output. Git LFS files are a `filter` too, and are
1087
- hashed from disk.
1088
-
1089
- `-text` (explicitly unset) and unspecified paths keep their OIDs:
1090
- both leave the blob byte-identical to the worktree file.
1091
-
1092
- Losing an OID is not over-invalidation. It routes that path to the
1093
- content hasher, which hashes the worktree bytes — the correct source
1094
- either way. The only cost is reading the file, which is exactly what
1095
- the gate exists to avoid paying needlessly.
1096
-
1097
- ## Runtime inputs and the lock (the env parallel)
1098
-
1099
- `inputs.runtime` / `inputs.workspaceRuntime` are modeled
1100
- exactly on `inputs.env`, and they share its asymmetry with the
1101
- lockfile:
1102
-
1103
- - **The command _strings_ live in the resolved config**, so `vx lock`
1104
- freezes them into `.vx/projects.lock.json` (just as it freezes the env _names_
1105
- a task declares).
1106
- - **The command _output_ is resolved live at hash time on every run**
1107
- — inside `resolveInputs`, the same place env _values_ are read from
1108
- the host `process.env`. The lock never stores it.
1109
-
1110
- A runtime command executes **once per run, per project** (memoized by
1111
- `projectDir + command`), at key-derivation time — which, with the
1112
- up-front classify/prefetch pass, is before any task runs. It is a
1113
- run-level reading of the ENVIRONMENT (toolchain versions, resolved
1114
- config), not a per-task probe: it is not asked again before a save, as
1115
- input files are, because that would be one spawn per command per miss
1116
- where a file costs one `lstat`. A command that reads another task's
1117
- OUTPUT reads whatever is on disk when it is asked. When the reader's key
1118
- folds that task's key, the upstream key covers it: an answer taken before
1119
- the upstream ran costs a spurious miss on the run after the output
1120
- changes, never a stale hit, and the answer stays shared. When it does
1121
- not (`tasks: []`, a filter that leaves it out, transitively), nothing
1122
- covered it until X-34: the save filed bytes built from the new state
1123
- under the old answer, and a later run that started from the old state
1124
- hit them. Now such a task (`probesAfterWrites`, stable-keys.ts) takes no
1125
- key up front and is never restore-tier, and its probes run for it alone,
1126
- after its upstream finished — one spawn per such task per run, not one
1127
- per project (`tests/in-run-writes.test.ts`). Declaring the producing
1128
- task's output as an input (`dependsOn` + files) is still the cheaper
1129
- spelling: files are re-checked, answers are not.
1130
-
1131
- A probe runs in its own process group. One still running when vx exits
1132
- — a Ctrl-C, or a refusal, while a hung `git` or `node -e …` answers — is
1133
- SIGKILLed with its whole tree; until 2026-09-27 (A-9) it ran on under
1134
- init. A `kill -9` of vx runs no exit hook, and leaves it.
1135
-
1136
- So a run loads the frozen command strings from the lock but still
1137
- spawns them and folds their current output into the key. A `node -v`
1138
- that goes from `v20` to `v22` after the lock was written busts the
1139
- cache, exactly as a changed `NODE_ENV` value would — the TypeScript
1140
- escape hatch (`define: { TSC_VERSION: execSync(...) }`) goes stale here
1141
- because its value was baked into the config object at lock time, whereas
1142
- a runtime input re-resolves.
1143
-
1144
- Consequently **`vx lock --check` does not — and need not — flag
1145
- runtime-output drift.** `lock --check` audits that the frozen config
1146
- object still matches a fresh evaluation; the command output was never
1147
- part of that object (only the strings are), so a changed probe output
1148
- is correct, expected, live behavior rather than lock drift. This is the
1149
- same reason `lock --check` ignores `inputs.env` value changes.
1150
-
1151
- ## Concurrent runs
1152
-
1153
- Two vx processes on one workspace (a `vx watch` beside a `vx run`, two
1154
- CI jobs on one checkout) take turns: a run takes the workspace's run
1155
- lock before it schedules and releases it with its cache handle — before
1156
- a persistent task's wait, so a dev server never holds it — and the
1157
- second run waits, saying after a second whom it waits for:
1158
-
1159
- ```
1160
- [vx] waiting for another vx run (pid 4821) on this workspace to finish…
1161
- ```
1162
-
1163
- The cache itself was always safe (SQLite waits on its lock, artifacts
1164
- land by rename; a transaction that reads before it writes takes the
1165
- lock at BEGIN, since SQLite answers its later write `database is locked`
1166
- at once: a run's history was lost so in 13 of 48 runs on one shared
1167
- cache dir, X-105); a task's OUTPUT TREE was not — both runs cleaned and
1168
- restored the same `dist/`, and a clean landing while the other run's
1169
- restore was staging took its files out from under it. The lock is an
1170
- atomic directory under the temp directory, keyed by the workspace root
1171
- (`--cache-dir` does not make two runs strangers) and holding the
1172
- holder's pid, so a lock a killed run left behind is reclaimed once its
1173
- pid is gone. The temp directory is the one each process sees, `TMPDIR`
1174
- included: two shells that set different ones (a `nix develop` shell
1175
- sets its own, as can direnv or `sudo`) hold two locks and do not take
1176
- turns (item 970). Run them under one `TMPDIR`. A lock under the
1177
- workspace would survive a reboot that `/tmp` does not, and where a
1178
- holder's start time is not readable (off Linux) a pid reused since
1179
- would read as a live holder forever. Where the directory cannot be made at all (another user's
1180
- lock, a temp directory this user cannot write) the run says so once and
1181
- proceeds unlocked — a courtesy between cooperating runs, never a
1182
- refusal — and a restore that then loses its staged files to the other
1183
- run's clean fails with `restore of <hash> into <dir> was interrupted: a
1184
- file it had just written vanished (ENOENT: …). Another vx run is using
1185
- this workspace — re-run once it is done.`; the artifact is intact
1186
- either way (a restore renames into place only once the whole archive
1187
- has staged). Machines sharing a workspace over a network file system
1188
- do not share a temp directory, so they do not share the lock.
1189
-
1190
- An eviction can still land between a hit's probe and its restore: a
1191
- `vx cache clean` in another shell, or the `cache.retention` of another
1192
- workspace that shares the `--cache-dir` (a different workspace, so a
1193
- different lock), which cannot see this run's pending `accessed_at`
1194
- bumps and reads its fresh hits as old. The restore then finds no
1195
- artifact, and that entry is a **miss**: the run says `[vx] <id>: its
1196
- cache artifact <hash> vanished before the restore … — running it`,
1197
- runs the task, and saves it again. It failed every such task as an
1198
- internal error until 2026-09-24 (upstream survey: nx#36688, nx#34032).
1199
- A hit the local short-circuit was restoring ahead of its dependencies
1200
- goes back to the schedule and runs once they are done, never in the
1201
- restore's slot: built before them, its bytes would be saved under the
1202
- healthy key (`tests/vanished-artifact.test.ts`).
1203
-
1204
- A local artifact whose bytes are wrong (a failed checksum, a torn
1205
- write, one past the artifact ceiling, an entry missing a recorded
1206
- output, a name that now reads unsafe, X-115) is a miss the same way: `[vx] <id>: cache: corrupt artifact
1207
- for <hash>: …; dropped it — running it`. The entry is dropped, so the
1208
- task's save stores the key again; it failed the task as an internal
1209
- error on every run until `--force` before A-52. A cache the run may
1210
- only read keeps the entry (`…; left it (this cache is read-only)`).
1211
-
1212
- ## Storage layout
1213
-
1214
- The run must be able to write here — it records its history at the
1215
- end of every run, hit or miss — so a directory this user cannot write
1216
- into (another user's `.vx/cache`, a read-only checkout) fails the run before
1217
- any task: `cache directory <path> is not writable (EACCES: …)`, with
1218
- `--cache-dir <path>` as the way out. `vx cache clean` is refused the
1219
- same way (its `--dry-run` only reads, and reads). The readers (`vx show`, `why`,
1220
- `last`, `info`) open such a directory read-only and go on.
1221
- Where the directory cannot be created at all (a read-only checkout with
1222
- no cache yet) a run says `cannot create cache directory <path>
1223
- (EACCES: …)`, naming the workspace `cache.dir` field and `--cache-dir`;
1224
- the readers and `vx cache clean` find no cache and go on.
1225
- A full disk is the same kind of failure and is reported the same way,
1226
- never as a corrupt artifact and never as a stack: a save that runs out
1227
- of room is `[vx] cache save failed: ENOSPC …` (the task's work ran, the
1228
- next run misses), a restore that does is the task's failure line
1229
- `could not write its outputs (ENOSPC: …). Free space on that disk and
1230
- re-run.`, and a run record that does is `[vx] run history not recorded:
1231
- … — the verdict above stands` (the run exits by its tasks; `vx last`
1232
- will not know that one). A full index (`SQLITE_FULL`) gives way where the
1233
- write is only a memo: the file-hash memo, the output stamps and the
1234
- access times are skipped and the run goes on; a prune unlinks the
1235
- artifacts first, so the rows it then deletes have room to go, and a
1236
- save's temp file is removed when its write fails.
1237
- A process out of file descriptors (`EMFILE`, `ENFILE`) is named the
1238
- same way: a save is `[vx] cache save failed: save of <hash> could not
1239
- open a file (EMFILE: …) — … raise the limit (ulimit -n 4096) and
1240
- re-run`, and a restore fails the task with that hint, never as a
1241
- corrupt artifact (A-39). Opening the index is named the same way: SQLite reports a
1242
- descriptor it could not get as `unable to open database file`, and vx
1243
- says `the cache index <path> could not be opened (…) — …raise the limit`
1244
- (A-56).
1245
- A restore that cannot READ its artifact (a cache directory another
1246
- user owns) names the cache, not the outputs: `restore of <hash> could
1247
- not read its artifact (EACCES: …). Make the cache directory readable
1248
- by this user, or point cacheDir / --cache-dir at one that is.` (A-40).
1249
- An input the key must hash that this user cannot read fails its task
1250
- naming the read: `<path> is not readable by this user (EACCES), and vx
1251
- reads it to derive a cache key. Make it readable, or, for a task input,
1252
- take it out of inputs.files.` (A-50).
1253
-
1254
- By default the entries and their artifacts live in a **shared store**
1255
- in `~/.vx/<id>/cache`, and each workspace keeps only its own index in
1256
- its `.vx/cache`: every checkout of the repository hits what another
1257
- saved (a second clone, a worktree), and none reads another's history.
1258
- Task timings are shared too: `task_durations`, what a duration forecast
1259
- reads, so a new worktree has one on its first run. So are plugins' own
1260
- entries (`plugin_blobs`, a plugin's `ctx.cache`).
1261
- The id is Nx 23's: 16 hex of a sha256 of the remote (`origin`, then
1262
- `upstream`, `base`, the first; `host/owner/repo` in lower case, so ssh
1263
- and https agree) and the workspace's path in the repository; with no
1264
- remote, the first commit. A repository with neither (no commit yet, a
1265
- shallow clone with no remote) shares nothing. Each level of `~/.vx` is
1266
- owner-only; vx closes one of yours that others can read (chmod 700), and one others can write, or owned by another user, is not used. Design: [`design/shared-store-2026-10.md`](./design/shared-store-2026-10.md).
1267
-
1268
- ```
1269
- ~/.vx/<id>/cache/ the shared store
1270
- ├── store.db entries, entry_stdout, output_files,
1271
- │ entry_inputs, task_durations,
1272
- │ plugin_blobs, store_meta; artifacts
1273
- │ of at most 32 KiB compressed (v33)
1274
- └── <hash>.tar.zst the larger artifacts (below)
1275
-
1276
- <workspaceRoot>/.vx/cache/ this workspace's own index
1277
- └── cache.db run history, memos, output stamps;
1278
- attaches store.db as `store`
1279
- ```
1280
-
1281
- The store's directory carries no version: every key is seeded with
1282
- `CACHE_VERSION`, which moves when hashing or the artifact layout does, so
1283
- two vx versions never read each other's artifacts. `store.db` is the
1284
- artifacts' inventory and records its schema (`store_meta.schema`): a vx
1285
- of another `SCHEMA_VERSION` drops its tables, prints nothing (the
1286
- cache is vx's to keep), and keeps every artifact, each indexed again
1287
- when its task next hits. The check, drop, re-create and stamp are one
1288
- write transaction, so another version's open waits rather than landing
1289
- between them. A home this user cannot write keeps the store
1290
- in `<workspaceRoot>/.vx/cache/` instead, silently. Name a
1291
- cache directory (`cache.dir` in .vx/workspace.ts, `--cache-dir`, or
1292
- `VX_CACHE_DIR`, in that order of precedence, relative to the workspace
1293
- root) and it holds everything, shared with no other workspace:
1294
-
1295
- ```
1296
- <cacheDir>/ (default .vx/cache when one is named)
1297
- ├── .gitignore `*` — written when the dir is created, or into an
1298
- │ existing dir that lacks one, so the cache is
1299
- │ never committed and never enumerated as an
1300
- │ input (a user's own file is left alone)
1301
- ├── cache.db SQLite metadata + run history, and the
1302
- │ artifacts of at most 32 KiB compressed,
1303
- │ the same bytes as a file below (v33)
1304
- ├── cache.db-wal write-ahead log
1305
- ├── cache.db-shm shared memory
1306
- ├── failures/<runId>.json a failed run's task output and the files it
1307
- │ names (`vx last --format json`, getFailures);
1308
- │ the newest 50 kept
1309
- └── <hash>.tar.zst one artifact per cache entry, the larger ones:
1310
- ├── stdout captured stdout (always present, may be empty)
1311
- ├── outputs/<rel> declared output files, project-relative (when any)
1312
- ├── workspace-outputs/<rel> declared outputs.workspaceFiles,
1313
- │ WORKSPACE-ROOT-relative (when any)
1314
- ├── .vx-meta.json per-output [mode, mtimeMs], the key it was
1315
- │ packed under (v35), the miss's CPU and RSS
1316
- └── .vx-sum CRC-32 of every entry above (v36)
1317
- ```
1318
-
1319
- `failures/` sits outside `cache.db` so the index layout, and every
1320
- shared store under it, stays as it was. Each file is
1321
- `{ runId, tasks: [{ taskId, project, task, exitCode, timedOut?, output, locations }] }`,
1322
- written at run end (temp file + rename) only when a task failed with
1323
- output; `output` is plain text (ANSI stripped, secrets masked, the first
1324
- 8 KiB and last 56 KiB). A refused write (`EACCES`, `ENOSPC`) costs only
1325
- that output, never the run's verdict.
1326
-
1327
- `<hash>` is the 16-hex xxh3 key. The `workspace-outputs/` namespace is
1328
- additive: tasks that don't declare `outputs.workspaceFiles` produce
1329
- byte-identical artifacts to the plain v17 format (which is why the
1330
- field needed no `CACHE_VERSION` bump). `output_files` rows mirror the
1331
- two namespaces — project rows store the bare rel, workspace rows store
1332
- the full `workspace-outputs/<rel>` name as the discriminator.
1333
-
1334
- The container is written and read by vx's own streaming tar code
1335
- (`src/cache/tar-stream.ts`): ustar with the name/prefix split and a pax
1336
- `path` record past it, header checksums, and nothing else — there is no
1337
- `tar` subprocess, no libarchive, and no staging copy. **Save streams
1338
- it**: each output is stat'd once, then read from disk as it is written
1339
- and compressed straight into the temp file (measured 2026-09-03 on a
1340
- 150 MiB output: peak +705 MiB → +241–269 MiB; the streamed compressor
1341
- is ~2.4× the one-call cost per byte, so a small artifact is packed in
1342
- memory and compressed in one call — tiny saves unchanged within
1343
- noise). Ingest lists entries through the reader without materialising
1344
- a byte, and **restore streams it**: the zstd
1345
- frame is decoded and the tar read as it arrives — ustar name/prefix
1346
- (the prefix under POSIX `ustar\0` magic only: old GNU's `ustar ` keeps
1347
- atime there, A-29),
1348
- pax `path`/`size`, GNU long names, header checksums, truncation; an
1349
- extended header (read whole) past 1 MiB or a pax `size` that is not a
1350
- whole number is refused as corrupt, since a 1 GiB pax header costs 2 GiB
1351
- of memory from a ~32 KB artifact — and
1352
- every regular entry is written to the workspace's `.vx/tmp/<pid>-0/`
1353
- (vx's own, ignored by git, never an output folder) and renamed over its
1354
- target only after the whole archive has ended cleanly and
1355
- the index's recorded outputs are all present. vx itself holds one
1356
- chunk of the tar at a time (measured 2026-09-03, incompressible
1357
- artifacts, fresh process: 150 MiB peaked at +644 MiB through
1358
- `Bun.Archive` and +49 MiB streamed at the same wall time; a single
1359
- 400 MiB entry restores at +30 MiB, the same as a 150 MiB one), while
1360
- the per-entry buffers of many mid-sized entries are garbage the
1361
- collector reclaims at its own pace (200 × 2 MiB measured +90–180 MiB,
1362
- never the artifact's size). An artifact up to 4 MiB compressed is decoded in one call
1363
- first — the stream setup costs ~35 µs each, 4% of the headline
1364
- restore row when every artifact is a one-file `dist/` — and then fed
1365
- to the same reader and extractor, so there is one extraction path.
1366
- The reader reads a header in place when it lies within one chunk and
1367
- its numeric fields off the bytes when they are plain octal (anything
1368
- else takes the full parse): a 4-entry artifact's read 50–57 µs → 22–23
1369
- (min of 15), and a 300-artifact, 20-file restore run's reader 224 → 92
1370
- ms of main thread (2026-10-02).
1371
- The 2 GiB decompression ceiling applies to both: declared size and
1372
- output length for the one-call decode, a running count for the stream.
1373
- An ingest bounds the compressed body first: a remote body past the
1374
- ceiling's zstd bound (a length header, a Blob's size, or a running count
1375
- on a chunked stream) is refused before it can fill the disk.
1376
- A frame that declares no content size (what a streaming compressor
1377
- writes — vx's own saves above 4 MiB) is always decoded as a stream, so
1378
- a sizeless bomb has nowhere to expand and vx's artifacts ingest
1379
- anywhere. So is a body that is not exactly one frame: the declaration is
1380
- the first frame's alone, while the one-call decoder decodes every frame,
1381
- so until 2026-09-27 (A-5) a 100-byte frame with a large one appended
1382
- expanded whole in memory (2 GiB from a 32 KB body) before its length was
1383
- checked. The frame's blocks are walked by their own sizes to tell. A save meets the same ceiling first: the pack plans the tar
1384
- from one stat per output, and a tar past 2 GiB is refused before a byte
1385
- is compressed — the run stays green, one status line names the task
1386
- (`[vx] cache save failed: <task> is not cached: its outputs pack to …,
1387
- past the 2.0 GB artifact ceiling a restore enforces`), and nothing is
1388
- stored for a later run to hit and fail to restore. Left to the scan, a
1389
- 2.2 GB output paid the whole compress and a decode (6 to 14 s) to learn
1390
- the same thing and called the task's outputs a corrupt artifact.
1391
-
1392
- The last entry, `.vx-sum`, is a CRC-32 over every entry before it (each
1393
- its name, a NUL, then its body) as 8 hex digits. Tar sums its headers
1394
- only and neither zstd writer asks for a frame checksum, so a byte flipped
1395
- in a raw zstd block (incompressible output: an image, a wasm, a tarball)
1396
- decoded clean and a hit replayed the wrong bytes: 1,141 of 1,141 flips in
1397
- a random 8 KB body went unseen (L-19). Scan and restore hash every entry
1398
- as they read it and refuse an artifact whose sum is absent, wrong, or
1399
- followed by another entry, before anything it holds lands; the refusal
1400
- is a corrupt artifact, so a restore degrades to a miss and an ingest
1401
- stores nothing. It catches damage in transit or at rest, not a store that
1402
- forges artifacts: nothing a key carries can tell those from honest ones.
1403
- Its cost is the CRC's (~10 GB/s): a 32 MB restore 11.3 → 14.4 ms and
1404
- 1,000 small files 16.2 → 18.9 ms on tmpfs, min of 15.
1405
-
1406
- Tar headers carry mode and second mtimes. vx needs both permission bits (a lost executable bit builds cold
1407
- and breaks warm) and millisecond mtimes (the skip-restore probe compares
1408
- them) exactly, so the pack stats each output once and writes
1409
- `.vx-meta.json` — `{ version, key, files: { <entry>: [mode, mtimeMs] }, exec? }` —
1410
- into the archive. Restore applies both, at their edges too: a mode of
1411
- 000, an mtime of 0 (`SOURCE_DATE_EPOCH=0`), one before 1970, whose
1412
- tar header carries 0 since ustar's field holds no sign, and one past
1413
- March 2242, whose header carries the field's 11-digit maximum (X-114:
1414
- the save failed `value … does not fit a 12-byte field`). Until 2026-09-27
1415
- (A-4) the first two were skipped, so the file came back 0644 and
1416
- stamped now and every later hit restored it again, and the third wrote
1417
- a header the reader refused, so its task never saved. `exec` (`{ cpuMs?, peakRssBytes? }`,
1418
- 2026-09-12) is what the PRODUCING execution used: it rides the artifact
1419
- so a machine that never ran the task — a fresh runner on a remote hit —
1420
- still learns what the task needs (`@vzn/vx-schedule-history` packs on
1421
- it), every wire ships the bytes verbatim, and an artifact without it
1422
- reads as before. The ingest side takes the numbers only as plain
1423
- non-negative numbers; anything else in a foreign sidecar is dropped. Entries that are not regular
1424
- files (symlinks, hardlinks, devices) are never materialised — the
1425
- reader reports them only to be skipped — so a
1426
- poisoned artifact cannot smuggle one onto disk. Nor can it name a file
1427
- the task did not declare: a remote artifact is ingested only when every
1428
- file it carries matches the task's `cache.outputs` (the lookup passes
1429
- them, `CacheGetContext.outputs`), and never when it holds one path as a
1430
- file and a directory at once. Either used to reach the tree, an input
1431
- overwritten or a `.git/hooks` file planted under a green
1432
- `cache-hit-remote`, or a restore that failed every later run from the
1433
- local copy (item 942); now the read is a miss and the task runs. Every
1434
- restore asks the same of the local copy, whatever route it reached the
1435
- store by: an entry outside the declared outputs, one a `!` entry takes
1436
- back, one under `.git`, `.vx` or a `vx.lock.json`, or one under a
1437
- `node_modules` no output glob names fails the whole restore before
1438
- anything is written; the entry is dropped and the task runs. Remote
1439
- execution (`@vzn/vx-reapi`) writes a remote action's outputs through the
1440
- same gate (`restorableOutput`). On the save side a
1441
- **symlinked output** is captured as its target's bytes and comes back
1442
- as a regular file (a hit that finds the task's own link still current
1443
- leaves it); a link to a directory, or a dangling one, has no bytes to
1444
- store, so the save refuses it by name rather than cache an entry that
1445
- restores to nothing. So does a link whose target is outside the
1446
- project: vx reads outputs outside the task's sandbox, and a planted
1447
- link packed a file the task could not read (L-23). Each refusal names
1448
- the path as the config spells it (`workspaceFiles output gen/latest`). The clean before exec and restore removes every
1449
- file AND symlink the output globs cover (a link is unlinked, never
1450
- followed). An output directory that is a symlink (`dist -> real-out`) is
1451
- followed by the clean as by the save and restore, so its target is the
1452
- output and an entry holds only what its run wrote (X-88); one that
1453
- resolves outside the project refuses the task, naming the link, and
1454
- nothing is deleted through it (X-5). The clean prunes the directories it emptied (before a miss it keeps
1455
- the directory a wildcard glob is rooted at, `dist` for `dist/**`, as the
1456
- task writes there), so a task whose
1457
- output changed shape — `dist/out` a directory one run and a file the
1458
- next — restores either entry over the other's tree; a stray the globs
1459
- do not cover, or an empty directory where the entry holds a file, that
1460
- stands in an entry's way fails the restore naming it, not as a corrupt
1461
- artifact (item 1094). A directory on the way that is a
1462
- symbolic link OUT of the project (a `dist` made a link after the entry
1463
- was saved) is never written through and never replaced — the link is
1464
- the user's, and replacing it is the bug nx#37061 reports — so the
1465
- restore refuses, naming it: `<dir>/dist is a symbolic link to
1466
- <target>, outside <dir> — a cache restore never writes through a link
1467
- that leaves its directory. Remove the link and re-run (the restore
1468
- puts a real directory there), or stop declaring outputs under it.` A
1469
- link that stays inside the project is written through, and so is one
1470
- whose target is gone: the restore creates the directory it names —
1471
- following a chain of links to its end — and writes through it, the
1472
- link kept. Whether a link stays inside is decided on where it resolves,
1473
- so a dangling link to `../elsewhere` or to an absolute path out of the
1474
- project is the same refusal, nothing created outside, and a cycle of
1475
- links is refused by name. (A link the output globs themselves cover,
1476
- `dist/sub` under `dist/**`, is an output: the clean unlinks it.) Entry NAMES are
1477
- validated by vx before anything decides where to write, and a bad
1478
- entry anywhere, even the last, rejects the WHOLE archive: the temps
1479
- are unlinked and the empty directories the extraction created are
1480
- pruned (`tests/archive-security.test.ts`). A backslash in a name is a
1481
- name character (vx runs on Linux and macOS; Windows through WSL), so an
1482
- output like `dist/back\slash` caches and restores as written.
1483
-
1484
- **Key properties:** one entry is one artifact — a row of `artifacts`
1485
- when small, else one file, so eviction is one delete in the rows'
1486
- transaction or a single unlink; no per-entry manifest, no separate
1487
- `logs/` tree; and local + remote layers transport the exact same
1488
- tar.zst bytes end-to-end, wherever the local copy lives.
1489
- The artifact is the record and the index its inventory (owner,
1490
- 2026-10-06): a lookup reads the row first, and a key with no row whose
1491
- `<hash>.tar.zst` is on disk (a `SCHEMA_VERSION` drop, a deleted
1492
- `cache.db` or `store.db`) has the artifact indexed again from its own
1493
- bytes, checked as a remote's are (its recorded key, its names against
1494
- the task's declared outputs), and hits; one that fails the check is a
1495
- miss, and the save that follows replaces it. An inline artifact with no
1496
- row is indexed the same way, from its bytes, in one transaction; a reset
1497
- keeps the `artifacts` table, but deleting `store.db` or `cache.db` by
1498
- hand takes the inline artifacts with it. A `.tmp-*` a crashed save
1499
- left is never a hit. `vx cache clean` sweeps row-less files, and so does a run whose workspace declares `cache.retention`, at
1500
- most once an hour (the sweep's clock is `schema_meta.orphans_swept_at`;
1501
- the policy sums index rows, so orphans alone never make it due). A
1502
- row-less artifact may be in use: two vx versions share one store, and
1503
- each open drops the other's rows. So the policy judges it as it judges a
1504
- row, its file time standing for `accessed_at`: past `olderThan` it goes,
1505
- and under `maxSize` it counts, oldest use first with the rows. A hit
1506
- renews a file time over an hour old, so the last use is read as the file
1507
- time plus an hour. An inline artifact with no row is judged the same
1508
- way, its `at` standing for the file time, and `vx info` counts it with
1509
- the files. A temp a crashed save left goes once it is an hour
1510
- old, and so does a file an inline artifact of its key shadows; nothing
1511
- younger than an hour is taken. Re-indexing touches the
1512
- artifact's mtime before it links its temp, so the sweep sees that one
1513
- fresh too.
1514
-
1515
- A row's `accessed_at` is renewed the same way: a hit renews it only once
1516
- it is over an hour old (owner, 2026-10-08). Renewing every hit rewrote
1517
- 2,180 rows at a warm 1,090-package run's close, ~30 ms. So the policy
1518
- reads a row's last use as `accessed_at` plus an hour too: an entry used
1519
- within `olderThan` is never pruned, and one may stay up to an hour past
1520
- it (`tests/cache.test.ts` › "an entry used within the age limit is never
1521
- pruned").
1522
- Captured stdout is stored twice on purpose: in the artifact (so it
1523
- survives the remote round-trip) and in the `entries` row (so a local
1524
- hit replays it with pure SQL, never decompressing the artifact).
1525
-
1526
- ### SQLite tables
1527
-
1528
- `schema_meta.version` is the gate: an index written by any other
1529
- `SCHEMA_VERSION`, earlier or newer, is reset by the first run that opens
1530
- it (pre-alpha: no migrations; the index is an inventory, owner
1531
- 2026-10-06): every table but `schema_meta` is dropped and recreated,
1532
- so each comes back in its current shape (A-54: `output_dirs` kept an
1533
- earlier vx's columns). The check, drop, re-create
1534
- and stamp are one write transaction, so another version's open waits
1535
- rather than landing between them; an index already current is opened
1536
- without the lock.
1537
- A reading verb (`vx why`, `vx last`, `vx info`) leaves it untouched and
1538
- says why (item 896; `vx cache clean --dry-run` previews the reset
1539
- instead, item 1083).
1540
- That open prints nothing (owner, 2026-10-06: the cache is vx's to
1541
- keep). The artifacts stay: each is indexed again from its own bytes
1542
- when its task next asks for its key (below).
1543
-
1544
- ```sql
1545
- -- src/cache/schema.ts (SCHEMA_VERSION = 'v40', in cache.ts)
1546
- -- With a shared store, entries, entry_stdout, output_files, entry_inputs,
1547
- -- store_meta, task_durations, task_outcomes, blob_sizes and plugin_blobs live in its store.db, attached as `store`; the rest is
1548
- -- the workspace's cache.db. A named cache dir holds all of them.
1549
-
1550
- CREATE TABLE schema_meta (
1551
- key TEXT PRIMARY KEY, -- 'version', 'cache_version', 'orphans_swept_at', 'file_hashes_swept_at', 'store_dir'
1552
- value TEXT NOT NULL
1553
- );
1554
-
1555
- CREATE TABLE entries (
1556
- hash TEXT PRIMARY KEY, -- the 16-hex xxh3 cache key
1557
- project TEXT NOT NULL,
1558
- task TEXT NOT NULL,
1559
- command TEXT NOT NULL,
1560
- exit_code INTEGER NOT NULL,
1561
- duration_ms INTEGER NOT NULL,
1562
- size_bytes INTEGER NOT NULL, -- artifact size
1563
- created_at INTEGER NOT NULL, -- ms-epoch
1564
- accessed_at INTEGER NOT NULL, -- ms-epoch; a hit renews it once over an hour old (LRU)
1565
- cpu_ms INTEGER, -- v26: the producing execution's usage, from
1566
- peak_rss_bytes INTEGER -- the artifact's sidecar (save + ingest)
1567
- );
1568
-
1569
- -- v29: stdout apart from its entry. An UPDATE rewrites a whole record, so
1570
- -- the accessed_at bump rewrote each hit's stdout (up to 16 MB): 200 hits
1571
- -- of 1 MB cost the run's close 125-150 ms.
1572
- CREATE TABLE entry_stdout (
1573
- hash TEXT PRIMARY KEY, -- FK entries(hash) ON DELETE CASCADE
1574
- stdout TEXT NOT NULL -- captured stdout (pure-SQL hit replay); no row = ''
1575
- );
1576
-
1577
- CREATE TABLE runs (
1578
- id INTEGER PRIMARY KEY AUTOINCREMENT,
1579
- hash TEXT NOT NULL, -- '' when the outcome derived no key (see below)
1580
- project TEXT NOT NULL,
1581
- task TEXT NOT NULL,
1582
- status TEXT NOT NULL, -- success | failed | cache-hit | cache-hit-remote | skipped
1583
- exit_code INTEGER NOT NULL,
1584
- duration_ms INTEGER NOT NULL,
1585
- forward_args TEXT, -- salted xxh3 of the JSON-encoded `--` args; null when none
1586
- started_at INTEGER NOT NULL, -- ms-epoch
1587
- ended_at INTEGER NOT NULL,
1588
- run_id TEXT, -- UUIDv7 shared across all tasks in one invocation
1589
- cpu_ms INTEGER,
1590
- peak_rss_bytes INTEGER,
1591
- wallclock_start_ns INTEGER, -- bigint; serialized as SQLite INTEGER (signed 64-bit)
1592
- wallclock_end_ns INTEGER,
1593
- cache_hit INTEGER, -- 0/1; convenience for flamegraph color
1594
- attempts INTEGER, -- v23: attempts a retried task took (>1)
1595
- cached INTEGER, -- v25: 1 = declared a cache block; 0 = runs every time
1596
- -- v27: why the task failed or was skipped, as the run's footer said it
1597
- -- (items 267–270); NULL where the reason does not apply
1598
- blocked_by TEXT, -- a skip's root blocker, or what a task ran behind (a task id)
1599
- timed_out INTEGER, -- 1 when vx's own timeout killed it
1600
- sandbox_violations INTEGER, -- the sandbox's violation count
1601
- not_ready TEXT, -- 'timeout' | 'exited' | 'spawn' (persistent task)
1602
- restored INTEGER, -- v31, on a hit: 1 = outputs restored, 0 = up to date
1603
- remote_error TEXT -- v40: each remote read or write of the key that failed
1604
- );
1605
-
1606
- -- Two indexes, both append-only under a run's inserts: every row of a run
1607
- -- carries the same run_id and a started_at newer than everything before it.
1608
- -- There is deliberately NO (project, task) index — it scattered every run's
1609
- -- rows over one B-tree leaf per pair (a 1,000-hit warm run's record stage
1610
- -- went 57–79 ms → 14–19 ms without it) while its readers gained nothing;
1611
- -- the history reader bounds its scan by rowid instead (see history.md).
1612
- CREATE INDEX runs_started_at ON runs(started_at);
1613
- CREATE INDEX runs_run_id ON runs(run_id);
1614
-
1615
- -- Every non-group, non-aborted outcome of a run gets a row, so
1616
- -- `invocations.task_count` always equals `COUNT(*)` here for that run_id
1617
- -- and the terminal summary's "N total". Two of those outcomes never derive
1618
- -- a cache key — a `skipped` task (its upstream failed, so it never probed)
1619
- -- and a `persistent` one (a dev server is not cacheable) — and they store
1620
- -- `hash = ''`. `''` is impossible for a real key (16 hex chars), so it reads
1621
- -- unambiguously as "no key recorded"; the key-diff surfaces (`vx why`'s
1622
- -- whyDidThisRerun, the cache-key diff) guard it rather than reporting
1623
- -- "inputs unchanged" from two rows that never had inputs to compare.
1624
- --
1625
- -- A `skipped` row is a task of the run but NOT an execution, so the rate and
1626
- -- average aggregates in metrics.ts exclude it: counting a zero-duration
1627
- -- non-event would dilute success rate, hit rate and mean duration. The
1628
- -- completeness reads (listRuns / getRun / the run-detail timeline) include it.
1629
-
1630
- -- The file-hash memo: a content hash per input file that git could not
1631
- -- answer (untracked or dirty), keyed by the stat identity that proves
1632
- -- the bytes unchanged. Machine-local; § Cache key derivation step 10.
1633
- -- A writing close drops rows unwritten for 30 days, at most once a day
1634
- -- (a memo miss is the whole cost of a dropped row; item 1082).
1635
- CREATE TABLE file_hashes (
1636
- path TEXT PRIMARY KEY,
1637
- mtime_ms INTEGER NOT NULL,
1638
- size_bytes INTEGER NOT NULL,
1639
- ctime_ms INTEGER NOT NULL,
1640
- ino INTEGER NOT NULL,
1641
- content_hash TEXT NOT NULL,
1642
- seen_at INTEGER NOT NULL
1643
- );
1644
-
1645
- -- The size of each index blob the enumeration checked against the
1646
- -- worktree size git recorded (§ Cache key derivation, A-60): fixed for
1647
- -- its OID, so a warm run asks git for none. Swept with file_hashes by
1648
- -- seen_at, the time the row was written. v38: in the store, so a worktree
1649
- -- reads what another learned.
1650
- CREATE TABLE blob_sizes (
1651
- oid TEXT PRIMARY KEY,
1652
- size INTEGER NOT NULL,
1653
- seen_at INTEGER NOT NULL
1654
- );
1655
-
1656
- -- v30: the paths an index distrusts, by a hash of the index file and the
1657
- -- pathspecs (A-60): a warm run reads one row, not one per blob. Swept
1658
- -- with file_hashes.
1659
- CREATE TABLE blob_verdicts (
1660
- digest TEXT PRIMARY KEY,
1661
- paths TEXT NOT NULL,
1662
- seen_at INTEGER NOT NULL
1663
- );
1664
-
1665
- -- v16: per-output-file fingerprints, scoped by the entry that produced
1666
- -- them — what a hit stats to skip the restore when the tree is already
1667
- -- current (§ A current tree). ON DELETE CASCADE follows a prune.
1668
- CREATE TABLE output_files (
1669
- entry_hash TEXT NOT NULL,
1670
- path TEXT NOT NULL,
1671
- size_bytes INTEGER NOT NULL,
1672
- mode INTEGER NOT NULL,
1673
- mtime_ms INTEGER NOT NULL,
1674
- PRIMARY KEY (entry_hash, path),
1675
- FOREIGN KEY (entry_hash) REFERENCES entries(hash) ON DELETE CASCADE
1676
- );
1677
-
1678
- -- v28, its own table since v32: inode + ctime THIS workspace saw after
1679
- -- the save/restore that last wrote the file (item 886); no row is never
1680
- -- current. Apart from the shared entry: two worktrees overwrote each
1681
- -- other's stamps and every switch restored. No foreign key (the entry
1682
- -- may be the store's); prune and re-save delete what they orphan.
1683
- CREATE TABLE output_stamps (
1684
- entry_hash TEXT NOT NULL,
1685
- path TEXT NOT NULL,
1686
- ino INTEGER NOT NULL,
1687
- ctime_ms INTEGER NOT NULL,
1688
- PRIMARY KEY (entry_hash, path)
1689
- );
1690
-
1691
- -- Every directory under a whole-subtree output glob, with its mtime as
1692
- -- of the last save or restore on THIS machine: unchanged mtimes prove
1693
- -- the output SET unchanged without a glob walk. Machine-local — a remote
1694
- -- ingest writes none, and the first hit after it walks and records.
1695
- CREATE TABLE output_dirs (
1696
- entry_hash TEXT NOT NULL,
1697
- path TEXT NOT NULL,
1698
- mtime_ms INTEGER NOT NULL,
1699
- PRIMARY KEY (entry_hash, path) -- no foreign key since v32, as output_stamps
1700
- );
1701
-
1702
- -- v22 (Tier 3): one header row per `vx run` invocation. The `runs`
1703
- -- table is per-task; this is the per-invocation record carrying the
1704
- -- command, git/CI/host context, and run-level counts. Recorded
1705
- -- atomically alongside `runs` via recordRunBundle (one transaction).
1706
- CREATE TABLE invocations (
1707
- run_id TEXT PRIMARY KEY, -- UUIDv7, == runs.run_id
1708
- command TEXT NOT NULL, -- full argv, e.g. "vx run build --all"
1709
- requested_tasks TEXT NOT NULL, -- JSON string[] of options.tasks
1710
- cache_policy TEXT NOT NULL, -- the --cache-policy spec, e.g. "local:rw,remote:r"
1711
- concurrency INTEGER NOT NULL,
1712
- flow TEXT, -- 'focused' | 'broad' | NULL (programmatic)
1713
- started_at INTEGER NOT NULL, -- ms-epoch
1714
- ended_at INTEGER NOT NULL,
1715
- total_duration_ms INTEGER NOT NULL, -- wall clock of the whole run
1716
- task_count INTEGER NOT NULL, -- non-group, non-aborted tasks recorded
1717
- failed_count INTEGER NOT NULL,
1718
- hit_count INTEGER NOT NULL, -- cache-hit + cache-hit-remote
1719
- hit_local_count INTEGER NOT NULL, -- cache-hit
1720
- hit_remote_count INTEGER NOT NULL, -- cache-hit-remote
1721
- up_to_date_count INTEGER NOT NULL DEFAULT 0, -- v31: hits that restored nothing
1722
- restored_local_count INTEGER NOT NULL DEFAULT 0, -- v31: hits that restored, local layer
1723
- restored_remote_count INTEGER NOT NULL DEFAULT 0, -- v31: hits that restored, remote layer
1724
- exit_ok INTEGER NOT NULL, -- 1 if the run's `ok`
1725
- commit_sha TEXT, -- nullable: not a git repo / probe failed
1726
- branch TEXT,
1727
- dirty INTEGER, -- 1 if worktree had uncommitted changes
1728
- ci INTEGER NOT NULL, -- 1 if a CI env was detected
1729
- ci_provider TEXT, -- 'github' | 'gitlab' | 'buildkite' | 'circleci' | 'generic'
1730
- host TEXT, -- os.hostname()
1731
- os TEXT, -- process.platform
1732
- arch TEXT, -- process.arch
1733
- vx_version TEXT NOT NULL
1734
- );
1735
- CREATE INDEX invocations_started ON invocations(started_at);
1736
- CREATE INDEX invocations_branch ON invocations(branch);
1737
- CREATE INDEX invocations_ci ON invocations(ci);
1738
-
1739
- -- v22 (Tier 3): the input-fingerprint moat. One row per cache-key
1740
- -- component, keyed by the cache-ENTRY hash it belongs to (NOT a run
1741
- -- id). Written INSIDE the entry-save transaction — only on a cache
1742
- -- MISS, never on a hit — via INSERT OR IGNORE. A warm all-cache-hit
1743
- -- run writes nothing here. The "why did this re-run?" diff resolves a
1744
- -- run to its task hash (runs.hash), then joins two entries' components
1745
- -- over (kind, name), no recompute. v34: one row per entry, the
1746
- -- components one JSON array (a row per component cost a b-tree insert
1747
- -- each: ~0.5 ms a save at 50 input files). ON DELETE CASCADE sweeps the
1748
- -- row when a prune drops the entry.
1749
- CREATE TABLE entry_inputs (
1750
- entry_hash TEXT PRIMARY KEY, -- == entries.hash / runs.hash
1751
- components TEXT NOT NULL, -- JSON [[kind, name, hash], …] in (kind, name) code-unit order
1752
- FOREIGN KEY (entry_hash) REFERENCES entries(hash) ON DELETE CASCADE
1753
- );
1754
- -- kind: file|env|runtime|ws-runtime|upstream|package|config|forward|workspace|plugin|format
1755
- -- name: file: workspace-rel path; env: var name; upstream: task id; …
1756
- -- hash: env|runtime|ws-runtime|forward|plugin: xxh3hex(salt + value); an unset env var: 'unset'
1757
-
1758
- -- v32: what belongs to the entries, not to one workspace: 'value_salt',
1759
- -- the salt entry_inputs digests are taken under, so they compare with
1760
- -- another workspace's run.
1761
- CREATE TABLE store_meta (
1762
- key TEXT PRIMARY KEY,
1763
- value TEXT NOT NULL
1764
- );
1765
-
1766
- -- v36: each executed success's duration, appended in the run's history
1767
- -- transaction, in the store so every worktree forecasts from it.
1768
- -- source: 'branch:<name>'; the forecast reads the newest five per task.
1769
- -- v37: 'remote' rows are a default-branch commit's timings a plugin
1770
- -- imported, one per (sha, task).
1771
- CREATE TABLE task_durations (
1772
- id INTEGER PRIMARY KEY AUTOINCREMENT,
1773
- project TEXT NOT NULL,
1774
- task TEXT NOT NULL,
1775
- source TEXT NOT NULL,
1776
- sha TEXT,
1777
- duration_ms INTEGER NOT NULL,
1778
- recorded_at INTEGER NOT NULL
1779
- );
1780
- CREATE INDEX task_durations_task ON task_durations(project, task, source, id);
1781
- CREATE UNIQUE INDEX task_durations_remote
1782
- ON task_durations(sha, project, task) WHERE source = 'remote';
1783
-
1784
- -- v38: the flaky record (failure-mode.ts), in the store so a key that
1785
- -- passed in one worktree and failed in another is flaky in both. Every
1786
- -- executed success or failure that names its key (keyed, cached, not run
1787
- -- behind a failure), and a key's first hit: a later hit replays a pass on
1788
- -- record, so a warm run on passed keys writes no row (one index probe,
1789
- -- ~1 ms per 1,000 hits). Pruned by age with the run history.
1790
- CREATE TABLE task_outcomes (
1791
- id INTEGER PRIMARY KEY AUTOINCREMENT,
1792
- project TEXT NOT NULL,
1793
- task TEXT NOT NULL,
1794
- hash TEXT NOT NULL,
1795
- status TEXT NOT NULL,
1796
- attempts INTEGER,
1797
- recorded_at INTEGER NOT NULL
1798
- );
1799
- CREATE INDEX task_outcomes_hash ON task_outcomes(hash, status);
1800
-
1801
- -- v37: a plugin's own entries (`ctx.cache`), by core's key for its
1802
- -- name; written once, pruned by age with the run history.
1803
- CREATE TABLE plugin_blobs (
1804
- key TEXT PRIMARY KEY,
1805
- bytes BLOB NOT NULL,
1806
- at INTEGER NOT NULL
1807
- );
1808
-
1809
- -- v33: an artifact of at most 32 KiB compressed, its exact <hash>.tar.zst
1810
- -- bytes, written in the transaction that writes its entry rows. In the
1811
- -- store (or a named cache dir's cache.db), and outside every drop a
1812
- -- SCHEMA_VERSION reset makes: the artifact is the record.
1813
- CREATE TABLE artifacts (
1814
- hash TEXT PRIMARY KEY,
1815
- at INTEGER NOT NULL, -- last use (ms): a row-less sweep's file time
1816
- bytes BLOB NOT NULL
1817
- ) WITHOUT ROWID;
1818
-
1819
- -- 'layout' = 'a1': the artifacts table's own version; another value
1820
- -- drops the table, silently. Nothing else drops it. Read only where
1821
- -- the schema was not this vx's, so a layout change bumps the schema.
1822
- CREATE TABLE artifacts_meta (
1823
- key TEXT PRIMARY KEY,
1824
- value TEXT NOT NULL
1825
- );
1826
- ```
1827
-
1828
- WAL mode is on; readers don't block writers. `PRAGMA busy_timeout =
1829
- 5000` makes concurrent `vx run` invocations queue instead of failing
1830
- with `SQLITE_BUSY`.
1831
-
1832
- > **Trust boundary (Tier 3):** `entry_inputs` stores a digest of each
1833
- > value-bearing component — `env`, `runtime`, `ws-runtime`, `forward`
1834
- > and `plugin` rows hold `xxh3hex(salt + value)`, an unset env var the
1835
- > literal `'unset'` — never the value, so a secret read as a cache input
1836
- > does not land in `cache.db` as plaintext. The salt is 128 random bits
1837
- > the store draws once (`store_meta` key `value_salt`): `vx why` prints
1838
- > these digests, and an unkeyed xxh3 in a public CI log let anyone
1839
- > confirm or brute-force a short secret. The "why did this re-run?" diff
1840
- > only needs to know a component changed, which the digest tells it.
1841
- > `cache.db` still records commands and captured stdout; it is a local,
1842
- > gitignored, single-user file.
1843
-
1844
- The Tier-3 tables persist components that were **already fed to
1845
- `Cache.key()`** — the cache key derivation is unchanged, so the
1846
- `CACHE_VERSION` is NOT bumped (only `SCHEMA_VERSION` rolls to `v22`).
1847
- Capture happens as a pure side-channel inside the existing `key()` fold
1848
- (`CacheKeyInput.captureInto`), **only on a cache miss** (the warm path
1849
- captures nothing), and the rows are persisted with the entry — so a
1850
- warm all-cache-hit run does no extra hashing, I/O, or DB writes for the
1851
- moat.
1852
-
1853
- ### Why SQLite + a single artifact per entry
1854
-
1855
- - **Index queries are fast.** Stats (`SELECT COUNT(*) FROM entries`),
1856
- TTL pruning (`WHERE accessed_at < ?`), per-task lookup
1857
- (`WHERE hash = ?`) all hit a B-tree.
1858
- - **A hit costs SQL, not decompression.** Metadata + stdout live in
1859
- the row; the artifact is only opened when outputs actually restore.
1860
- - **One artifact = one wire payload.** The same tar.zst bytes serve
1861
- local storage and the remote round-trip — no repacking.
1862
- - **One handle, one schema-meta sentinel.** An older schema drops
1863
- and recreates every table but `schema_meta` (pre-alpha) — there's no migration code to maintain.
1864
-
1865
- ## Performance characteristics
1866
-
1867
- - **Hashing cost** on a clean tree is near-zero per file: git index
1868
- OIDs come from the bulk enumeration spawn, so key derivation does no
1869
- file reads. Dirty/untracked files — and any path whose OID is not
1870
- trustworthy (see "Clean filters") — hash in-process (whole-file
1871
- read, behind a `(mtime, size, ctime, ino)` memo). Narrow
1872
- `inputs.files` still helps on heavily dirty trees.
1873
- - **Cache read** is three indexed `SELECT`s (the `entries` row, which
1874
- joins whether the artifact is inline, its `output_files` and its
1875
- `output_dirs`), plus a `stat` of the artifact when it is a file.
1876
- Restore is a tar.zst extract, skipped entirely when the on-disk
1877
- tree already matches. `accessed_at` bumps (only rows over an hour
1878
- old) are batched into one UPDATE at flush time.
1879
- - **Cache write** is one in-process tar.zst pack + one SQLite
1880
- transaction, which holds a small artifact's bytes itself; a large one
1881
- adds a temp write and an atomic rename. Hashing dominates the run; storage itself
1882
- is cheap. The remote upload (if any) is backgrounded.
1883
- - **Claimed root files** (a lockfile plugin's lockfile and settings)
1884
- are read once per `vx run` invocation; after a task that may rewrite
1885
- one ran, one `stat` per file re-checks it (item 750). With no plugin
1886
- claiming one, nothing is read.
1887
-
1888
- ## What's NOT in the key (and why)
1889
-
1890
- - **`exec.env.passThrough` _values_.** Would force cache misses across
1891
- machines with different CI flags, regions, or shell prompts. The
1892
- _names_ are in the config hash (step 3) so adding/removing a
1893
- passthrough still bumps the key for affected tasks.
1894
- - **Files outside the project directory that aren't declared.**
1895
- Workspace-root configs (`tsconfig.base.json`, etc.) are not
1896
- auto-included — declare them via `inputs.workspaceFiles`
1897
- (root-anchored globs).
1898
- - **Gitignored files no entry names.** Vite and Next load `.env.local`
1899
- on their own; a broad glob skips it, so editing it replays the old
1900
- build. Name it: `files: ['src/**', '.env.local']`. Its bytes are
1901
- hashed, never stored, and a missing file is simply not an input.
1902
- Machines with different files compute different keys, so they miss
1903
- each other's shared entries. `--affected` ignores it (only what git
1904
- sees selects a task), and `vx watch` re-runs on its edit. A probe
1905
- (`runtime: ['cat .env.local 2>/dev/null | cksum']`) still works where
1906
- a command, not a path, decides what to read.
1907
- - **Node / Bun / OS / build-tool versions — unless you declare them.**
1908
- The canonical mechanism is `inputs.runtime` /
1909
- `workspaceRuntime` (e.g. `workspaceRuntime: ['node -v']`): the
1910
- command output is resolved live at hash time, so it stays correct
1911
- under the lock. Avoid baking versions via
1912
- `define: { X: execSync(...) }` — that value freezes into the config
1913
- object at lock time and goes stale.
1914
- - **`.vx/projects.lock.json`** — globally excluded (v24); also filtered out of
1915
- `--affected` change sets. So are the files `vx lock` and `vx init`
1916
- write beside it: each project's `vx.lock.json` review copy,
1917
- `.vx/.gitattributes` and `.vx/.gitignore`.
1918
- - **`exec.remote`.** Pure PLACEMENT: it decides _where_ a task runs,
1919
- never what it produces, so pinning a task to this machine does not
1920
- bust its cache. It must not — the
1921
- whole contract of a remote executor is that the same command over the
1922
- same inputs yields the same outputs, so a key that moved with
1923
- placement would split your laptop from the worker pool over nothing.
1924
-
1925
- **`exec.timeout` and `exec.retries` are the deliberate exception** —
1926
- they stay in the key. That asymmetry is easy to misread as an
1927
- oversight, so: a timeout or a retry budget can change _whether the
1928
- task completes at all_, which is a different question from where it
1929
- ran. They were folded in before the placement fields existed, and
1930
- stripping them now would be a `CACHE_VERSION` bump for no correctness
1931
- gain.
1932
-
1933
- ## Bumping `CACHE_VERSION`
1934
-
1935
- The cache records the version it was written under
1936
- (`schema_meta.cache_version`). After an upgrade every cached task
1937
- misses once and re-saves, with nothing printed (owner, 2026-10-06: the
1938
- cache is vx's to keep). The index survives, so the old entries stay
1939
- until they age out under `vx cache clean --older-than` or
1940
- `cache.retention`; no key derives to them again.
1941
-
1942
- Required when:
1943
-
1944
- - A new field is added to the cache key derivation (step list above).
1945
- - The order or framing of existing key fields changes.
1946
- - The on-disk layout changes (artifact format, entry naming).
1947
- - The `CacheEntry` shape changes in a way that affects restore.
1948
- - The SQLite schema changes in a way that affects existing rows
1949
- (`SCHEMA_VERSION` also bumps in that case).
1950
-
1951
- Not required when:
1952
-
1953
- - Behavioural changes that adjust _which_ values flow into existing
1954
- key components — those naturally produce different keys for
1955
- affected tasks.
1956
- - Changes to WHEN reads/writes fire (policy, prefetch, restore tier,
1957
- background uploads) — key derivation and artifact bytes untouched.
1958
- - Doc-only updates.
1959
- - Refactors that don't change the bytes fed into the hash.
1960
- - A key part leaves the chain: every key moves, so no unchanged key
1961
- holds wrong bytes (`v43` stayed when core stopped folding the
1962
- `workspace:` and `pkg:` parts, 2026-10-14).
1963
-
1964
- `tests/contract-stored-format.test.ts` holds the layout rows: it
1965
- records the index's DDL beside `SCHEMA_VERSION` and a fixture
1966
- artifact's entries, sidecar and digest beside `CACHE_VERSION`
1967
- (`tests/contract/stored-format.json`), and fails when either layout
1968
- moves under its recorded version. Every bump of either version
1969
- regenerates the record
1970
- (`VX_UPDATE_CONTRACT=1 bun test tests/contract-stored-format.test.ts`),
1971
- which refuses a layout that moved without one.
1972
-
1973
- The bump procedure has a dedicated skill at
1974
- `.claude/skills/bump-cache-version/` (used as `/bump-cache-version`).
1975
- Files touched, in the skill's order: `src/cache/key-fold.ts` (the constant),
1976
- this doc (history), `docs/modules/cache.md` (the quoted version, and the
1977
- key/entry shape if it changed), `CLAUDE.md` § Live invariants (the quoted
1978
- version — the decision log it once named was retired 2026-09-02),
1979
- `docs/STATUS.md` (the entry that says why the bump was needed, or why it
1980
- was not), the cache tests, `tests/contract/stored-format.json`
1981
- (regenerated, above), and
1982
- `packages/vx-docs/src/content/docs/guides/upgrading.md` (the bump's
1983
- breaking footer).
1984
-
1985
- ### History
1986
-
1987
- - **v43 kept** (2026-10-14): core stopped folding the project's
1988
- `package.json` and the root fingerprint files; every key moved, so
1989
- nothing replays wrong bytes and no bump was needed.
1990
- - **v42 → v43**: stored bytes wrong under an unchanged key (Security
1991
- F4, 2026-10-14). A scoped run stored its local entries under the
1992
- default key, so a later default run on the same disk replayed a PR's
1993
- bytes. A scoped run now stores under the scope's key locally as it
1994
- does remotely; the fix cannot reach an entry already saved that way.
1995
- - **v41 → v42**: the stored output became both streams in the order
1996
- the run printed them (`orchestrator/output-log.ts`), so a hit replays
1997
- stderr too. A v41 entry holds stdout alone.
1998
- - **v40 → v41**: stored bytes wrong under an unchanged key (X-88). An
1999
- output directory linked inside its project was never cleaned, so an
2000
- entry could hold files a run of another key left there. The fix
2001
- cannot reach an entry already saved that way.
2002
- - **v39 → v40**: stored bytes wrong under an unchanged key (X-32, X-33,
2003
- X-34). An additive task's entry a hit replayed over a file the task
2004
- had removed, one that missed a same-size rewrite, and a runtime probe
2005
- answered before its upstream wrote.
2006
- - **v38 → v39**: stored bytes wrong under an unchanged key (A-61). A
2007
- gitlink whose directory had lost its `.git` but held files listed
2008
- none of them, so an entry built from them sits under the key the
2009
- same directory empty derives.
2010
- - **v37 → v38**: stored bytes wrong under an unchanged key (A-60). A
2011
- file added under a clean filter that was later removed kept its LF
2012
- index blob, git held it clean by its stat, and the key folded that
2013
- blob while the task read the CRLF bytes. The blob-size check cannot
2014
- reach an entry already saved that way.
2015
- - **v36 → v37**: stored bytes wrong under an unchanged key (A-59).
2016
- `git status` paired a deleted file with a similar unmerged path as its
2017
- rename source and printed only `UU <path>`, so the file kept its index
2018
- OID and the key folded it while the task ran without it. The fix
2019
- (`--no-renames`) cannot reach an entry already saved that way.
2020
- - **v35 → v36**: the container changes (L-19). Every artifact ends in a
2021
- `.vx-sum` entry, a CRC-32 over the entries before it, and scan and
2022
- restore refuse one whose sum is absent or wrong: a byte flipped in a
2023
- raw zstd block restored as a hit, undetected. A v35 artifact carries no
2024
- sum, so the bump retires them rather than refusing each on read.
2025
- - **v34 → v35**: the container changes (item 943). The sidecar records
2026
- the cache key the artifact was packed under, and ingest refuses bytes
2027
- whose key is not the one it asked for, or that record none. Nothing
2028
- tied an artifact to its key before: a remote layer that answered one
2029
- key with another's bytes (a truncated or colliding key mapping, two
2030
- namespaces mixed) replayed the other task's outputs under a green
2031
- `cache-hit-remote`, probed with project `b` restoring project `a`'s
2032
- `out.txt`. An artifact from v34 records no key, so the bump retires
2033
- them rather than refusing each one on read.
2034
- - **v33 → v34**: stored bytes wrong under a key a CORRECT derivation now
2035
- produces (item 750). A root task that rewrote the lockfile mid-run
2036
- (`pnpm install` without `--frozen-lockfile`) let a reader after it save
2037
- a build made against the new install under the key of the lockfile the
2038
- run started with, and that key is exactly what the fixed code derives
2039
- when the tree is back on that lockfile. The fix stops new entries, not
2040
- old ones. Probed, not argued: seeds A then B through the previous
2041
- commit, then A twice through the fixed code under v33 — the last run
2042
- started on lockfile A, installed A, and restored the build made against
2043
- B; under v34 it built A. The item's other two edges poisoned nothing:
2044
- their stale restores happened in runs that saved nothing (a read-only
2045
- policy, a tainted upstream) or saved under a key the 743 re-check
2046
- already guarded.
2047
- - **v32 → v33**: stored bytes wrong under a key a CORRECT derivation now
2048
- produces (item 743). The three stale-hit fixes of that item stop new
2049
- poisoned entries, but not the ones already saved: a formatter's entry
2050
- sits under the key of its unformatted input, an uncached `gen`'s
2051
- consumer's under the key of the seed before `gen` ran, an edit-mid-run's
2052
- under the key of the file before the edit — and each of those keys is
2053
- exactly what the fixed code derives when the tree is back in that
2054
- state. Not self-healing, unlike a fix whose old key no correct run
2055
- derives. Probed, not argued: an entry saved by the previous commit's
2056
- formatter, the input checked out unformatted, and the fixed code under
2057
- v32 reported `up-to-date` and left it unformatted; under v33 it runs.
2058
- - **v31 → v32**: stored bytes wrong under a key the fix does not change
2059
- (item 739). A declared output whose name is not UTF-8 (`x\xffy`) came
2060
- back from `Bun.Glob` decoded lossily as `x�y`, which names no file, so
2061
- the save packed the tree without it and every hit restored a tree
2062
- missing that file under an unchanged key. vx now refuses such a name
2063
- by name; an artifact saved before the fix is the wrong bytes, so v31
2064
- entries are not trusted.
2065
- - **v30 → v31**: stored bytes wrong under a key the fix does not change
2066
- (item 726), v30's shape for a sibling. A sandboxed task that declared
2067
- `cache` was granted every workspace package linked in its
2068
- `node_modules`, so it could import a sibling its key never folded and
2069
- save what it built. The fix withholds that link unless the key answers
2070
- for the package (§ Cross-project boundaries), so the next miss fails
2071
- on the read; but an entry saved before it still hits when only the
2072
- sibling changes. Probed, not argued: an entry saved by the previous
2073
- commit, `@x/ui` edited, then this commit's run under v30 reported
2074
- `up-to-date` and left the old `ui` in `dist/`; under v31 the same run
2075
- misses and fails on the read, with the hint.
2076
-
2077
- - **v29 → v30**: stored bytes wrong under a key the fix does not change
2078
- (item 720). npm and Yarn link every workspace package at the root, the
2079
- task's own included, and the sandbox granted every link target whole,
2080
- so a sandboxed task was handed its own project back whatever its
2081
- `allow.read` said. An undeclared read of its own file ran unreported
2082
- and the result was saved under a key that never saw that file. The fix
2083
- withholds the self-link, so the next miss fails on the read; but an
2084
- entry saved before it hits for as long as only that file changes.
2085
-
2086
- - **v28 → v29**: every key moves, and the bump is for the notice, not
2087
- for wrong bytes (item 682). The key is a seed-chained xxHash3 fold,
2088
- one step per field and per input file, and Bun's xxHash3 reads only
2089
- the low 32 bits of its seed: a bare chain carried 32 bits of state, so
2090
- two input sets whose running digests shared their low halves merged at
2091
- the next step, a stale hit at about 2^-32 per step rather than 2^-64.
2092
- A birthday search over 2^17 values of one env variable found two that
2093
- gave one `Cache.key` in under a second. `xxh3` now feeds the seed
2094
- forward (`xxHash3(part, seed) ^ seed`): states that share their low
2095
- half keep their high-half difference through every later step. The
2096
- old keys were wrong but their stored bytes are not, so the fix alone
2097
- is self-healing; without the bump every entry would miss silently.
2098
-
2099
- - **v27 → v28**: stored bytes wrong under a key the fix does not change —
2100
- the v25/v26 shape (item 667). A bracket in a task glob became a literal
2101
- character; before, `Bun.Glob` read it as a character class. An INPUT
2102
- glob over a route directory folded the wrong files, and that fix is
2103
- self-healing: the file set, and so the key, moves. An OUTPUT glob is
2104
- not: `outputs: ['app/[id]/page.js']` folds as its text, which the fix
2105
- leaves unchanged, and the entries under it hold nothing (or the class's
2106
- namesake `app/i/page.js`). Proven through a real run: an entry written
2107
- by v27 code, read by the fixed code, was a `cache-hit` that cleaned
2108
- `app/[id]/page.js` and restored nothing; under v28 it is a miss, and
2109
- the next hit restores the route.
2110
-
2111
- - **v26 → v27**: the artifact CONTAINER changed, so the stored bytes
2112
- under an unchanged key are no longer readable the same way — the
2113
- layout case, not the wrong-bytes case. Packing moved from a `tar`
2114
- subprocess (with a staged copy of every output and a per-host
2115
- `--format=gnu` / `--format=gnutar` probe) to `Bun.Archive`, and the
2116
- per-entry mode + millisecond mtime that tar headers carried natively
2117
- now ride a `.vx-meta.json` sidecar. Measured on the real `Cache.save`
2118
- path (300 outputs / 12 MB, min-of-5, interleaved arms against a
2119
- `git worktree` of the previous commit): **158 ms → 11 ms** to pack,
2120
- the artifact grows ~7 compressed bytes per output for the sidecar.
2121
- Restore is indistinguishable up to ~12 MB, but on a 150 MB
2122
- incompressible artifact it costs +28% time and +19% peak RSS
2123
- (74 → 95 ms, 575 → 683 MB peak, fresh process per arm): `files()`
2124
- returns entries that OWN their bytes, so they coexist with the
2125
- decompressed tar where the old reader returned views into it. Peak is
2126
- ~4.5× artifact size, up from ~3.8×. The bump is mandatory rather than
2127
- self-healing: a v26 artifact has no sidecar, so a v27 reader would
2128
- restore its outputs mode-0644 and mtime-now — silently wrong on disk
2129
- rather than a miss. Since 2026-09-03 the same layout is written and
2130
- read by vx's own streaming tar code (no bump: the bytes are readable
2131
- either way), and the peak above is history — save, ingest and restore
2132
- hold one chunk now; see § Storage layout.
2133
-
2134
- - **v25 → v26**: the same shape as v25 — stored bytes that are wrong
2135
- under a key nothing about the fix changes — reached by a different
2136
- producer. A task whose child was killed by a shutdown signal reports
2137
- `aborted`, but `aborted` did not propagate to dependents the way
2138
- `failed` and `skipped` do. So a dependent ran against the aborted
2139
- task's PARTIAL outputs, succeeded, and cached what it had built.
2140
- Because a dependent's key folds its upstream's INPUT key — and a
2141
- signal changes no input — that entry sits under exactly the key a
2142
- healthy run derives, and the next run replays it as a green hit with
2143
- exit 0. Reproduced end to end: run 1 killed mid-write leaves
2144
- `PARTIAL`, run 2 is fully healthy and still serves `PARTIAL` from
2145
- cache.
2146
-
2147
- Making `aborted` propagate stops new poison but cannot reach entries
2148
- already written, and a `LayeredCache` uploads them — so the reach is
2149
- a whole team's shared cache, not one developer's disk. That is what
2150
- makes the trade worth it: one cold rebuild against a class of
2151
- silently-wrong output. The interactive Ctrl-C path was never the
2152
- vector (vx's handler exits before a dependent can cache); the
2153
- reachable ones are an external `kill`, a supervisor, `docker stop`, a
2154
- self-terminating script, and every `handleSignals: false` embedder —
2155
- which includes `vx watch` and the distributed agent loop.
2156
-
2157
- - **v24 → v25**: the ARTIFACT BYTES in every existing entry are wrong
2158
- while the key addressing them is unchanged — the one situation a
2159
- version bump exists for, and the opposite of the recent self-healing
2160
- no-bump cases. Two defects on the pack/restore path, both silent data
2161
- loss on an ordinary cache hit with no attacker involved:
2162
- - **The executable bit was stripped from every cached output.**
2163
- `packArtifact` staged each output with `Bun.write`, which creates
2164
- the destination under the process umask and does NOT carry the
2165
- source's mode, so the tar header recorded 0644 and the restore
2166
- faithfully reproduced 0644. Any build emitting a CLI shim, a
2167
- compiled binary or a generated script worked cold and broke warm —
2168
- including this repo's own `build.bun.*` release binaries. Fixed by
2169
- chmod-ing each staged copy to the source's mode.
2170
- - **Outputs whose archive entry name exceeded 100 bytes were dropped
2171
- on every restore.** POSIX ustar splits such a name into
2172
- `prefix` + `name`; the reader read only `name`, which no longer
2173
- starts with `outputs/`, so the file was neither restored nor
2174
- indexed. Not self-healing: with no `output_files` row, the
2175
- skip-restore check compared a truncated expectation against a
2176
- truncated tree and agreed forever. Threshold is a
2177
- project-relative output path of ~93 chars — ordinary for a modern
2178
- bundler. Fixed by reading `prefix` (gated on the POSIX magic, since
2179
- GNU headers reuse those bytes for atime) AND by packing
2180
- `--format=gnu`, which carries long names in an `L` record.
2181
-
2182
- The format switch also fixes a working build being reported as
2183
- FAILED: ustar cannot split a single path COMPONENT over 100 bytes and
2184
- exits non-zero ("file name is too long (cannot be split)"), which
2185
- `packArtifact` raised _after_ the task had already succeeded. 120-char
2186
- filenames are legal everywhere and routine in snapshot/fixture trees.
2187
-
2188
- Shipped with three defence-in-depth fixes that needed no bump of their
2189
- own: `restoreOutputs` now throws instead of returning quietly when the
2190
- artifact is gone or cannot produce an output the index recorded (the
2191
- caller has already wiped the declared outputs by then, so a quiet
2192
- return reported a green hit over an emptied tree — reachable via a
2193
- concurrent `vx cache clean`); the tar reader rejects an entry whose
2194
- declared size runs past the end of the archive (`subarray` clamps, so
2195
- it used to install short, NUL-padded content as a cache hit instead of
2196
- degrading to a miss); and directory entries now get the same
2197
- containment + realpath checks as file entries (`mkdir` follows a
2198
- pre-existing symlink, so directories could be created outside the
2199
- destination). No `SCHEMA_VERSION` bump — no table changed.
2200
-
2201
- - **SCHEMA v23 → v24 (no `CACHE_VERSION` bump)**: `file_hashes` gains
2202
- `ctime_ms` + `ino`. The memo keyed on `(mtime, size)` alone, and its
2203
- row persists across runs, so any producer that preserves mtime —
2204
- `tar -x`, `unzip`, `cp -p`, `rsync --times`, a `SOURCE_DATE_EPOCH`
2205
- generator — got the previous run's digest for genuinely different
2206
- bytes: a stale cache hit. `utimes` cannot suppress ctime
2207
- unprivileged and an atomic write-then-rename changes the inode, so
2208
- the two together close it (git's index keys on ctime+ino+dev for the
2209
- same reason); both come free from the stat already taken. The key
2210
- DERIVATION is unchanged — the memo simply stops answering wrongly —
2211
- so an affected task's key moves from a wrong value to the right one:
2212
- it misses once, re-runs, re-caches. Self-healing, never a wrong hit.
2213
- Landed alongside two other stale-hit fixes that needed no schema
2214
- change: the cache-miss path now marks the outputs `cleanOutputs`
2215
- wiped (it was the only one of four sibling call sites that dropped
2216
- the return, so a deleted output kept a live index OID and stayed in
2217
- a consumer's input set), and `skip-worktree` / `assume-unchanged`
2218
- entries no longer keep a trusted OID (they sit at stage 0 and
2219
- `git status` reports nothing for them, so a sparse-checkout path
2220
- that was absent from disk still counted as an input). The schema
2221
- gate drops + recreates on the version mismatch (pre-alpha, no
2222
- migration), so this costs one cold rebuild.
2223
- - **SCHEMA v21 → v22 (no `CACHE_VERSION` bump)**: Tier-3 dashboard
2224
- tables — `invocations` (one header row per `vx run` with command,
2225
- git/CI/host context, and run-level counts, recorded with `runs`
2226
- via `Cache.recordRunBundle`) and `entry_inputs` (one row per cache-key
2227
- component, keyed by the cache-ENTRY hash — the input-fingerprint
2228
- moat). `entry_inputs` is written **inside the entry-save transaction,
2229
- only on a cache miss** (`INSERT OR IGNORE`), so a warm all-cache-hit
2230
- run writes nothing for the moat — Tier 3 has **zero warm-run cost**.
2231
- The cache KEY derivation is unchanged: these tables persist components
2232
- already fed to `Cache.key()` (captured via a pure side-channel,
2233
- `CacheKeyInput.captureInto`, at the same fold sites — only on a miss),
2234
- so existing artifacts stay valid and `CACHE_VERSION` stays `v24`. The
2235
- schema gate drops + recreates on the version mismatch (pre-alpha, no
2236
- migration).
2237
- - **v23 → v24**: exclude `.vx/projects.lock.json` from the input file set
2238
- globally (`ALWAYS_IGNORE` in `cache/inputs.ts`). The lockfile is
2239
- committed, so git enumerates it, but it's vx's own frozen-config
2240
- metadata — never a task input. Without this, a `vx lock` re-write
2241
- busts every key on a project that globs the root lockfile (a broad
2242
- `**/*` on the root project). Tasks whose `inputs.files` never
2243
- matched it derive byte-identical keys. No SCHEMA bump — only the
2244
- hashed file set changed, not the key layout or on-disk format.
2245
-
2246
- - **v22 → v23**: fold `inputs.runtime` / `workspaceRuntime`
2247
- command output into the key (two namespaced sections after
2248
- env-values). The command _strings_ stay in the config hash (step 3);
2249
- their combined trimmed stdout+stderr is resolved live at hash time
2250
- and folded as `runtime-values:` (project-cwd) and `ws-runtime-values:`
2251
- (root-cwd) sections, each `command\0output`. No SCHEMA bump — only
2252
- `Cache.key` derivation gained two sections; the on-disk format is
2253
- unchanged. Tasks declaring neither field fold a `:0` count for both
2254
- and derive byte-identical keys to before the bump.
2255
-
2256
- - **v21 → v22: pure-input transitive** (+ SCHEMA v21): reverted the v21
2257
- output-fold. Downstream keys fold the upstream's **input key** (its
2258
- own task hash) — a pure function of the filesystem, like Turbo/Nx.
2259
- No output content participates in any cache key. **Early cutoff is
2260
- gone**: an upstream that re-executes (comment edit, env change) but
2261
- reproduces byte-identical output now still re-runs its dependents.
2262
- This was a deliberate simplification — cutoff is rare in practice
2263
- and not worth the cascade complexity (it forced output content into
2264
- keys, which blocks any upfront/batched probe). **Multi-state is
2265
- preserved**: branch ping-pong A→B→A still re-hits, because the
2266
- upstream's _input_ differs per state and folds transitively into
2267
- every dependent key. SCHEMA v21 drops the now-unused `outputs_hash`
2268
- column; `CacheLayer.save` returns `void`.
2269
-
2270
- - **v20 → v21: early cutoff** (+ SCHEMA v19, **reverted in v22**):
2271
- downstream keys folded the upstream's output content identity
2272
- (`outputsHash`) instead of its task hash. Removed — see v22.
2273
-
2274
- - **v7 → v8** (PR #2): folded `forwardArgs` into the key for CLI
2275
- argument-forwarding alignment.
2276
- - **v8 → v9** (PR #3): `TaskConfig` shape changed — `exec` collapsed
2277
- from an array to a single command, `tasks` nested under `run`.
2278
- - **v9 → v10** (PR #7): on-disk layout switched from per-entry
2279
- `meta.json` + `outputs/` directory to a workspace-wide `cache.db`
2280
- (SQLite) plus output files directly at `<hash>/` and log files at
2281
- `logs/<hash>.{stdout,stderr}`. Adds run history for `vx info`.
2282
- Removes the per-entry manifest.
2283
- - **v10 → v11** (PR #19): analytics columns added to the `runs`
2284
- table: `run_id` (UUIDv7), `cpu_ms`, `peak_rss_bytes`,
2285
- `wallclock_start_ns` / `wallclock_end_ns`, `cache_hit`. All
2286
- nullable; directly queryable via `sqlite3 cache.db`. The on-disk
2287
- `<hash>/` layout itself was unchanged.
2288
- - **v11 → v12** (PR #42): project's `package.json` bytes folded into
2289
- every task's cache key implicitly. Matches Turbo / Nx "implicit
2290
- dependencies" behavior — a `package.json` dep change invalidates
2291
- the project's tasks even when `inputs.files` is narrow and
2292
- doesn't cover the file.
2293
- - **v12 → v13** (PR #65): per-entry on-disk layout unified. Outputs
2294
- moved from `<hash>/<rel>` (mixed with metadata) to
2295
- `<hash>/outputs/<rel>`; stdout / stderr moved from the sibling
2296
- `logs/<hash>.{stdout,stderr}` into `<hash>/stdout` and
2297
- `<hash>/stderr`. Eviction collapses to a single `rm -rf <hash>/`.
2298
- Also dropped the runner's `logs/<run_id>/<project>__<task>.{stdout,stderr}`
2299
- dump — output is already streamed live, surfaced on the outcome
2300
- object, and the cache entry covers successful runs; CI captures
2301
- parent stdout natively. The duplicate sibling dump was pure
2302
- redundancy.
2303
- - **v13 → v14**: file enumeration switched from a `Bun.Glob` walker
2304
- with our own `ignore`-library filter to `git ls-files --cached
2305
- --others --exclude-standard`. Matches what Turborepo and Nx both do
2306
- at the bottom of their hash pipelines. Side-effects user-visible:
2307
- (a) nested `.gitignore` patterns are anchored to the gitignore's own
2308
- directory, fixing the v13 footgun where `pkg/.gitignore: src/skip.ts`
2309
- was misinterpreted as `<workspaceRoot>/src/skip.ts`; (b)
2310
- `.git/info/exclude` and global excludes participate; (c)
2311
- untracked-but-not-ignored files enter inputs immediately (no
2312
- `git add` required). (The non-git fallback walker was later removed
2313
- entirely — vx hard-requires git; a non-repo workspace gets a clean
2314
- `UserError` telling the user to `git init`.)
2315
- - **v14 → v15**: cache-key hash swapped from SHA-256 (via
2316
- `Bun.CryptoHasher`) to xxHash3 (via `Bun.hash.xxHash3`). Key strings
2317
- shrink from 64 hex chars to 16, matching Turbo's xxh64 output width;
2318
- derivation is ~5× faster, dominating the cache-warm path that
2319
- hashes hundreds of input files. xxHash3 has no streaming Hasher
2320
- API, so `Cache.key()` chains via the seed parameter (each
2321
- `xxh3(part, prevDigest)` folds one field into the running digest)
2322
- and `hashFileFromDisk` reads the whole file before hashing — fine
2323
- for source files (typically < 1MB each); the throughput win
2324
- outweighs the memory hit. `SCHEMA_VERSION` bumps to `v15` at the
2325
- same time (PR #86 already took `v14` for the tar.zst artifact
2326
- layout): the `file_hashes.sha256` column is renamed to
2327
- `content_hash`, and the schema-mismatch path now `DROP`s the stale
2328
- tables before `CREATE TABLE IF NOT EXISTS` runs so the rename
2329
- actually takes effect on existing DBs. Non-cryptographic by design
2330
- — cache keys never need collision resistance against an adversary,
2331
- just uniqueness across honest inputs.
2332
- - **v15 → v16** (PR #86 series): artifact storage moved to a single
2333
- compressed `<hash>.tar.zst` per entry; the manifest.json entry was
2334
- dropped (file fingerprints live in the `output_files` table).
2335
- - **v16 → v17**: artifact narrowed to exactly `stdout` +
2336
- `outputs/<rel>` — no `meta.json`, no stderr (only successful runs
2337
- are cached and their stderr is near-always empty noise). Local and
2338
- remote layers transport the same bytes end-to-end; entry metadata
2339
- lives solely in SQLite.
2340
- - **v17 → v18**: env-value folding in `Cache.key()` switched its
2341
- name/value delimiter from `=` to `\0`. `${n}=${v}` was ambiguous —
2342
- `("A", "B=C")` and `("A=B", "C")` folded identical bytes. Env names
2343
- containing `=` are unreachable from a real POSIX environ, so this
2344
- is contract hardening rather than a field bug, but the key
2345
- derivation's stated invariant is unambiguous part boundaries —
2346
- now it holds everywhere (file inputs already used `\0`).
2347
- - **v18 → v19**: `'^task'` dependsOn expansion switched from
2348
- transitive-deps to nearest-holder frontier semantics (Turbo/Nx
2349
- direct-deps parity, plus vx's sparse bridging through deps that
2350
- don't declare the task). Task graphs lose the redundant deep edges,
2351
- so the filtered-upstream-hash set (step 8) shrinks for any task
2352
- whose deps chain `'^task'` themselves — same inputs now derive a
2353
- different key. Reachability/ordering is unchanged whenever holders
2354
- chain `'^task'` (the universal pattern); a holder that doesn't is
2355
- now the documented stopping point. No on-disk format change.
2356
- - **v19 → v20**: input-file content hashes switched from xxh3 to
2357
- **git blob OIDs** (Turbo's technique). The bulk enumeration spawn
2358
- became `git ls-files -s --others --exclude-standard` — `-s` lines
2359
- carry `<mode> <oid> <stage>\t<path>` for tracked files, so one
2360
- spawn yields the file list AND the index OIDs; a second
2361
- `git status --porcelain -z` spawn prunes OIDs for paths whose
2362
- working tree diverges from the index (renames drop both sides;
2363
- stage>0 conflict entries never get one; symlinks did not either
2364
- until 2026-09-09, when the fallback started hashing a link the way
2365
- the index does). A clean
2366
- tree's key derivation does zero reads / stats / SQLite per file.
2367
- Everything else falls back to `Cache.hashFile`, which now computes
2368
- the identical blob OID in-process (object format auto-detected via
2369
- `git rev-parse --show-object-format`, sha1 default) behind the
2370
- existing mtime+size memo. `SCHEMA_VERSION` bumps to `v18` in the
2371
- same change: pre-v20 `file_hashes.content_hash` rows store 16-hex
2372
- xxh3 digests that must not leak into the OID domain through the
2373
- memo. File-set visibility semantics are unchanged (verified: the
2374
- `-s --others` path set is identical to `--cached --others`,
2375
- including staged-but-deleted files and per-stage conflict
2376
- duplicates).