@vzn/vx 0.0.14 → 0.0.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -15,9 +15,6 @@ architecture, caching, and the full CLI / config reference.
15
15
  ```sh
16
16
  # From npm — ships the prebuilt standalone binary (no Bun required):
17
17
  npm install -g @vzn/vx # or: pnpm add -g @vzn/vx · bun add -g @vzn/vx
18
-
19
- # Or the zero-dependency install script (no Node/npm needed):
20
- curl -fsSL https://raw.githubusercontent.com/vznjs/vx/main/install.sh | sh
21
18
  ```
22
19
 
23
20
  ```ts
@@ -52,58 +49,6 @@ vx watch lint # re-run on file changes
52
49
  vx run build --dry # show the plan, don't execute
53
50
  ```
54
51
 
55
- ## Beyond a task runner — what shipped in the 2026-06 platform arc
56
-
57
- vx now ships an **open platform**, not just a CLI. Every surface
58
- below is built into the binary; no external services required.
59
-
60
- ```sh
61
- vx mcp # Model Context Protocol server (stdio)
62
- # — Claude Code / Cursor / Continue.dev talk to vx as a typed tool
63
-
64
- npm i -g @vzn/vx-cloud # the service CLI — also a standalone binary, no Bun
65
- vx-cloud serve --ui --open # team backend + embedded dashboard (token-authenticated)
66
- vx-cloud agent --url https://ci.acme.dev # join a distributed-execution pool (Nx-Agents-style DTE)
67
- vx-cloud connect https://ci.acme.dev # one-URL analytics + remote cache for every `vx run`
68
- ```
69
-
70
- ### Open platform highlights
71
-
72
- - **Provider-neutral core.** `vx` runs tasks and nothing else — the
73
- dashboard, remote cache, distribution, and telemetry all arrive via
74
- PLUGINS declared in `vx.workspace.ts`. `@vzn/vx-cloud` is the
75
- first-party one; anyone can write another against the same seams
76
- (backend / cache / telemetry).
77
- - **MCP for AI agents, twice.** `vx mcp` (stdio, reads your real
78
- `cache.db`: `getCacheStats`, `getRunHistory`, `explainCacheKey`,
79
- `whyDidThisRerun`) and `POST /mcp` on any `vx-cloud serve` — local
80
- or hosted, behind the bearer token.
81
- - **Provable cache correctness.** `vx run --verify` re-runs each
82
- cacheable task and byte-compares outputs; `--verify=inputs` runs it
83
- sandboxed against the declared inputs and names any undeclared read.
84
- The only runner that PROVES a cache entry safe instead of hoping.
85
- - **Distributed CI (DTE).** `vx-cloud agent` joins a session-keyed
86
- pool; a `vx run` fans tasks out across same-commit agents, outputs
87
- propagate through the shared artifact store, and a standing pool
88
- multiplexes concurrent runs with fair scheduling.
89
- - **Remote cache, one URL.** `vx-cloud connect <url>` wires analytics
90
- ingest AND a Turborepo-wire-compatible artifact store (HMAC signing,
91
- trusted/untrusted tiers derived from the token). Third-party Turbo
92
- cache servers work via two env vars.
93
- - **OTel CI/CD spans.** Declare `otel()` from `@vzn/vx-otel` in
94
- `vx.workspace.ts` — OTLP/HTTP traces + metrics with ZERO
95
- OpenTelemetry SDK dependency; runs flow to Grafana / Honeycomb /
96
- Datadog, including `--verify` hermeticity verdicts as span status.
97
- - **Dashboard embedded in the `vx-cloud` binary.** Runs, flamegraphs,
98
- live run cockpit with the task DAG, per-task logs + artifacts,
99
- cache-key diffs ("why did this re-run?"), flaky-task detection.
100
- Nothing to build or install; Docker image on ghcr for hosting.
101
- - **Predictive scheduling.** Opt in with `predictive: true` — the
102
- scheduler reads run history and starts the longest expected critical
103
- path first.
104
-
105
- Each surface lives behind a design doc under `docs/design/`.
106
-
107
52
  ## A cache that actually understands your build
108
53
 
109
54
  Every task runner caches. vx caches _correctly_ — and stops work
@@ -138,14 +83,10 @@ with the invariant that keeps it valid —
138
83
  `--verify=inputs` sandboxes it against the declared inputs and fails
139
84
  loud naming any undeclared read (completeness). No other runner can
140
85
  prove a cache entry safe.
141
- - **Signed artifacts.** HMAC signing on the remote-cache wire; with a
142
- key configured, unsigned or tampered artifacts are rejected and the
143
- task simply re-runs. A poisoned cache can't reach your machines.
144
- - **Cache trust tiers.** The server derives trusted/untrusted from the
145
- bearer token — a fork-PR artifact can never feed a trusted build.
146
- - **Corruption can't go live.** Artifacts are validated before they
147
- enter the store (zstd-bomb and oversize downloads refused); bad
148
- bytes degrade to a cache miss, never a crash.
86
+ - **Corruption can't go live.** A remote artifact is verified against
87
+ its content digest and validated before it enters the store
88
+ (zstd-bomb and oversize downloads refused); bad bytes degrade to a
89
+ cache miss, never a wrong hit and never a crash.
149
90
  - **Clean exits.** SIGINT/SIGTERM reap every child process — no
150
91
  orphaned dev servers in CI.
151
92
  - **Readiness you can bound.** Persistent tasks gate downstream work
@@ -183,37 +124,47 @@ TypeScript config with real imports · task graph with `^task`
183
124
  resolution that bridges packages without the task · multi-task runs
184
125
  with one shared graph · pnpm-style filters and `--affected` ·
185
126
  watch mode · `--dry` / `--graph` plans · persistent dev servers ·
186
- remote caching via one URL (or two env vars, wire-compatible with
187
- existing Turbo artifact servers) · retries, timeouts, `--continue`
188
- modes · per-layer cache control (`--cache=local:r,remote:`) ·
189
- `vx info`, `--summarize`, `--profile` Chrome traces, `--report` ·
190
- `vx cache prune` with TTL and size caps · `vx migrate` from
191
- turbo.json or an Nx graph.
127
+ retries, timeouts, `--continue` modes · per-layer cache control
128
+ (`--cache=local:r,remote:`) · `vx info`, `--summarize`, `--profile`
129
+ Chrome traces, `--report` · `vx cache prune` with TTL and size caps ·
130
+ `vx migrate` from turbo.json or an Nx graph.
131
+
132
+ ## Extensible by design — the core is provider-neutral
133
+
134
+ `vx` runs tasks and nothing else. A dashboard, a remote cache,
135
+ distributed execution, and telemetry export all arrive through
136
+ **plugins** declared in `vx.workspace.ts`, filling three seams the
137
+ core exposes — **backend** (where a task runs), **cache** (a remote
138
+ layer behind the local one), and **telemetry** (a read-only stream of
139
+ the versioned `TelemetryRecord` / `RunSummaryRecord` contract). Core
140
+ depends on none of them; the arrow only ever points plugin → core.
141
+ First-party plugins ship the OpenTelemetry exporter
142
+ ([`@vzn/vx-otel`](https://www.npmjs.com/package/@vzn/vx-otel), OTLP
143
+ traces + metrics with zero OTel-SDK deps) and **vx Cloud** (below) —
144
+ and the same seams are open to anyone.
192
145
 
193
146
  ## How it compares
194
147
 
195
- | | vx | Turborepo | Nx |
196
- | ------------------------- | --------------------------------------------------------------------- | ------------------------------ | ------------------- |
197
- | Fully cached, 100 pkgs¹ | **144 ms** | 279 ms | 583+ ms |
198
- | Config | TypeScript, evaluated into the cache key | JSON (static) | JSON (static) |
199
- | Output ownership | **Strict** — wiped before exec AND restore | Additive (stale files survive) | Additive |
200
- | Clean-tree hashing | **Zero reads** (git index OIDs) | git OIDs | re-hash / daemon |
201
- | Daemon required for speed | **No** | Optional | Yes |
202
- | Artifact signing | **Hard-fail** on unsigned | Soft | No |
203
- | Per-task sandbox | **Yes** — kernel-level, opt-in | No | No |
204
- | Provable cache safety | **Yes** — `--verify` (determinism) + `--verify=inputs` (completeness) | No | No |
205
- | MCP server for AI agents | **Yes** (`vx mcp` stdio + `POST /mcp` on the serve) | No | No |
206
- | Distributed CI execution | **Yes** — OSS, self-hostable (`vx-cloud agent`, Nx-DTE-style) | No | Paid (Nx Cloud DTE) |
207
- | Dashboard SPA | **Yes** — embedded in `vx-cloud serve --ui`: runs, logs, DAG, flaky | No | Paid |
208
- | Self-hosted cloud | **Yes** — `vx-cloud` binary or the ghcr Docker image; one stack | Vercel-only | No (proprietary) |
209
- | Plugin API | **Yes** — backend / cache / telemetry seams | No | Yes (TS-tied) |
210
- | Predictive scheduling | **Yes** (opt-in: `predictive: true`) | No | No |
211
- | OTel CI/CD spans | **Yes** — `otel()` plugin, zero OTel-SDK deps | No | Paid |
212
- | Install | **Single binary** — npm or 1 curl line, no Node/Bun needed | npm + Node | npm + Node |
148
+ | | vx | Turborepo | Nx |
149
+ | ------------------------- | --------------------------------------------------------------------- | ------------------------------ | ---------------- |
150
+ | Fully cached, 100 pkgs¹ | **144 ms** | 279 ms | 583+ ms |
151
+ | Config | TypeScript, evaluated into the cache key | JSON (static) | JSON (static) |
152
+ | Output ownership | **Strict** — wiped before exec AND restore | Additive (stale files survive) | Additive |
153
+ | Clean-tree hashing | **Zero reads** (git index OIDs) | git OIDs | re-hash / daemon |
154
+ | Daemon required for speed | **No** | Optional | Yes |
155
+ | Per-task sandbox | **Yes** — kernel-level, opt-in | No | No |
156
+ | Provable cache safety | **Yes** — `--verify` (determinism) + `--verify=inputs` (completeness) | No | No |
157
+ | MCP server for AI agents | **Yes** — `vx mcp` (stdio, reads your local cache) | No | No |
158
+ | Plugin API | **Yes** — backend / cache / telemetry seams | No | Yes (TS-tied) |
159
+ | Predictive scheduling | **Yes** (opt-in: `predictive: true`) | No | No |
160
+ | OTel CI/CD spans | **Yes** — `otel()` plugin, zero OTel-SDK deps | No | Paid |
161
+ | Self-hosted platform² | **Yes** — dashboard, distributed CI, remote cache; Docker compose | Vercel-only cache | Paid (Nx Cloud) |
162
+ | Install | **Single binary** — npm or 1 curl line, no Node/Bun needed | npm + Node | npm + Node |
213
163
 
214
164
  ¹ Wall-clock, direct binaries, same machine and workspace — full
215
165
  methodology and more scenarios in
216
- [`docs/benchmarks.md`](docs/benchmarks.md).
166
+ [`docs/benchmarks.md`](docs/benchmarks.md). ² Optional, self-hosted,
167
+ OSS — see [vx Cloud](#vx-cloud).
217
168
 
218
169
  ## Switching from another runner
219
170
 
@@ -259,25 +210,69 @@ Differences to know:
259
210
  - vx requires `cache.inputs.files` when caching is enabled (no default `$TURBO_DEFAULT$`).
260
211
  - vx defaults caching **off**; opt in per task by adding the `cache` block.
261
212
  - Persistent tasks: `persistent: { readyWhen: 'regex' }` (Turbo uses just `persistent: true`).
262
- - Remote cache: same wire format. Existing `VERCEL_*` / Turbo-cache-server tokens work via `VX_REMOTE_CACHE_TOKEN`.
213
+ - Remote caching is a plugin, not a built-in — connect one and every `vx run` reads through it.
263
214
 
264
215
  Side-by-side feature matrix + every known gap: [`docs/comparison.md`](./docs/comparison.md).
265
216
 
217
+ ## vx Cloud
218
+
219
+ An **optional, self-hosted CI platform** — the first-party service you
220
+ run yourself. It is a fully independent app (accounts, roles,
221
+ organizations, teams, multiple workspaces), with **Postgres** as the
222
+ system of record and an **S3-compatible bucket** for artifacts; the
223
+ controller keeps zero artifact bytes at rest. Core `vx` never depends
224
+ on it — it connects to a deployment through the plugin seams above.
225
+
226
+ What a connected platform adds:
227
+
228
+ - **Dashboard.** Runs, flamegraphs, a live run cockpit with the task
229
+ DAG, per-task logs + artifacts, cache-key diffs ("why did this
230
+ re-run?"), and flaky-task detection with the concrete fix.
231
+ - **Remote cache.** The vx-native `/v1/cache` wire, tenant-partitioned
232
+ per org/workspace, with **trusted / untrusted tiers derived from the
233
+ token** — a fork-PR artifact can never feed a trusted build — and an
234
+ always-on integrity digest the client verifies.
235
+ - **Distributed CI (DTE).** Agents form a session-keyed pool; a
236
+ `vx run` fans tasks out across same-commit agents, outputs propagate
237
+ through the shared artifact store, and a standing pool multiplexes
238
+ concurrent runs with fair scheduling.
239
+ - **MCP over the platform.** `POST /mcp` (Postgres-backed, behind the
240
+ token) exposes the analytics as typed tools for AI agents — alongside
241
+ the core `vx mcp` stdio server that reads your local cache.
242
+
243
+ Deploy the stack (app + Postgres + object storage) with Docker
244
+ Compose, then connect any workspace with one URL and a minted token:
245
+
246
+ ```sh
247
+ # Deploy — open the URL, register the first admin, mint a CI token under Admin → Tokens:
248
+ VX_CLOUD_SECRET=$(openssl rand -hex 32) \
249
+ docker compose -f packages/cloud/deploy/docker-compose.yml up
250
+
251
+ # Connect a workspace (remote cache + analytics + distribution):
252
+ vx-cloud connect https://ci.acme.dev --token vxc_…
253
+ # …or set VX_CLOUD_URL + VX_CLOUD_TOKEN in the environment.
254
+ ```
255
+
256
+ Install the platform CLI with `npm i -g @vzn/vx-cloud` (a standalone
257
+ binary, or the `ghcr.io/vznjs/vx-cloud` image). Full setup, RBAC,
258
+ distributed CI, and the wire protocol live in the **vx Cloud** section
259
+ of the [documentation site](https://vznjs.github.io/vx/).
260
+
266
261
  ## Architecture (one paragraph)
267
262
 
268
263
  `bin.ts → cli/index.ts` dispatches subcommands.
269
264
  `orchestrator/run.ts:run()` calls `prepareRun()` which discovers the
270
265
  workspace, loads configs, builds the package + task graph, opens the
271
- cache (local SQLite + optional remote layer), and installs plugins
266
+ cache (local SQLite + an optional remote layer), and installs plugins
272
267
  from `vx.workspace.ts`. The two-tier scheduler runs the graph in
273
268
  topological order with bounded concurrency (confirmed cache hits
274
269
  restore ahead of their deps); each task hits the cache (hash → get →
275
270
  restore on hit; spawn → save on miss) or short-circuits as a group /
276
271
  persistent. Every observation flows through one event bus — the
277
- terminal renderer subscribes directly, and PLUGINS receive the
272
+ terminal renderer subscribes directly, and plugins receive the
278
273
  versioned telemetry contract (`TelemetryRecord` / `RunSummaryRecord`)
279
- — that's how `otel()` and `cloud()` export without core knowing them.
280
- Core never imports a plugin; the arrow only points plugin → core.
274
+ — that's how telemetry and cache plugins export without core knowing
275
+ them. Core never imports a plugin; the arrow only points plugin → core.
281
276
  Every module has a docs page; every interface is a swappable seam.
282
277
 
283
278
  Read [`docs/architecture.md`](./docs/architecture.md) for the module
@@ -285,9 +280,8 @@ map; the design record lives under [`docs/design/`](./docs/design/).
285
280
 
286
281
  ## Documentation
287
282
 
288
- Full technical docs live under [`docs/`](./docs/):
289
-
290
- **Core**
283
+ Full technical docs live under [`docs/`](./docs/) and on the
284
+ [documentation site](https://vznjs.github.io/vx/):
291
285
 
292
286
  - [`docs/architecture.md`](./docs/architecture.md) — module map + data flow
293
287
  - [`docs/schema.md`](./docs/schema.md) — every config field
@@ -296,42 +290,32 @@ Full technical docs live under [`docs/`](./docs/):
296
290
  - [`docs/cli.md`](./docs/cli.md) — every flag
297
291
  - [`docs/comparison.md`](./docs/comparison.md) — Turbo / Nx / vite-task feature matrix
298
292
  - [`docs/modules/`](./docs/modules/) — one reference page per source module
293
+ - The **vx Cloud** section of the [docs site](https://vznjs.github.io/vx/) — deploy, RBAC, dashboard, distributed CI, wire protocol
299
294
 
300
- **Design + 2026-06 platform arc** (`docs/design/`)
301
-
302
- - [`architecture-north-star-2026-06.md`](./docs/design/architecture-north-star-2026-06.md) — the unified vision
303
- - [`architecture-review-2026-06.md`](./docs/design/architecture-review-2026-06.md) — review + applied checklist
304
- - [`wire-protocol-2026-06.md`](./docs/design/wire-protocol-2026-06.md) — JSON-RPC 2.0 + OTel envelope (shipped)
305
- - [`distributed-ci-2026-06.md`](./docs/design/distributed-ci-2026-06.md) — original coordinator/worker (superseded by `vx-cloud agent` DTE — `distributed-execution-2026-07.md`)
306
- - [`vx-cloud-2026-06.md`](./docs/design/vx-cloud-2026-06.md) — original CF cloud (superseded by the standalone `vx-cloud` service)
307
- - [`extension-protocol-2026-06.md`](./docs/design/extension-protocol-2026-06.md) — subscriber/inspector/driver/plugin (Phase 1 shipped)
308
- - [`predictive-execution-2026-06.md`](./docs/design/predictive-execution-2026-06.md) — history-aware scheduling (Phase A-B shipped)
309
- - [`docs/progress/implementation-log-2026-06.md`](./docs/progress/implementation-log-2026-06.md) — phase-by-phase narrative
295
+ The full design record (including the platform-pivot notes) lives
296
+ under [`docs/design/`](./docs/design/).
310
297
 
311
298
  ## Status
312
299
 
313
300
  **Pre-alpha.** The schema is settling; we bump `CACHE_VERSION` rather
314
- than maintain back-compat. **1,400+ tests (core + packages); CI green
301
+ than maintain back-compat. **1,600+ tests (core + packages); CI green
315
302
  on every commit**; the project dogfoods itself (`vx run ci`).
316
303
  Published on npm: [`@vzn/vx`](https://www.npmjs.com/package/@vzn/vx)
317
- and `@vzn/vx-cloud`, both as prebuilt standalone binaries.
304
+ (a prebuilt standalone binary) and `@vzn/vx-cloud`.
318
305
 
319
306
  Production readiness for the **core task runner**: the semantics are
320
- solid. The main operational rough edge is Windows (unsupported).
321
-
322
- Production readiness for the **platform layer**:
323
-
324
- | Surface | Maturity | Notes |
325
- | ---------------------------------------------- | -------------------------------- | ---------------------------------------------------------------- |
326
- | Core task runner + caching | **production-ready** | dogfooded continuously; 1,150+ core tests, all green |
327
- | `vx run --verify` (provable cache correctness) | **shippable** | determinism + input-completeness proofs; CI-gate recipe in docs |
328
- | `vx mcp` + serve `POST /mcp` | **shippable** | live cache.db tools (stdio); serve-hosted MCP behind the bearer |
329
- | `vx-cloud serve` (ingest + dashboard + /v8) | **shippable** | token-auth, multi-workspace, artifact store, ghcr image |
330
- | `vx-cloud agent` distributed execution | **shippable for self-hosted CI** | session-keyed DTE, shared pools, fair multi-run scheduling |
331
- | Plugin API (backend / cache / telemetry) | **shippable** | crash-isolated; `cloud()` + `otel()` are ordinary plugins |
332
- | Predictive scheduling | **shippable as opt-in** | gated on `predictive: true` + observed data |
333
- | Dashboard SPA (embedded in `vx-cloud`) | **iterating** | runs/logs/DAG/flaky live; data-first entity redesign in progress |
334
- | OTel export (`@vzn/vx-otel`) | **shippable** | declare `otel()` in `vx.workspace.ts`; OTLP traces + metrics |
307
+ solid; it is dogfooded continuously. The main operational rough edge
308
+ is Windows (unsupported).
309
+
310
+ | Surface | Maturity | Notes |
311
+ | ---------------------------------------------- | ------------------------------ | ----------------------------------------------------------------- |
312
+ | Core task runner + caching | **production-ready** | dogfooded continuously; 1,200+ core tests, all green |
313
+ | `vx run --verify` (provable cache correctness) | **shippable** | determinism + input-completeness proofs; CI-gate recipe in docs |
314
+ | `vx mcp` | **shippable** | live cache.db tools over stdio |
315
+ | Plugin API (backend / cache / telemetry) | **shippable** | crash-isolated; the cloud + OTel plugins are ordinary plugins |
316
+ | Predictive scheduling | **shippable as opt-in** | gated on `predictive: true` + observed data |
317
+ | OTel export (`@vzn/vx-otel`) | **shippable** | declare `otel()` in `vx.workspace.ts`; OTLP traces + metrics |
318
+ | vx Cloud platform (self-hosted) | **shippable for self-hosting** | accounts/RBAC/orgs on Postgres, S3 artifacts, dashboard, DTE, MCP |
335
319
 
336
320
  ## Development
337
321
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@vzn/vx",
3
- "version": "0.0.14",
3
+ "version": "0.0.16",
4
4
  "description": "An open, extensible monorepo task runner.",
5
5
  "type": "module",
6
6
  "exports": {
@@ -17,10 +17,10 @@
17
17
  "node": ">=18"
18
18
  },
19
19
  "optionalDependencies": {
20
- "@vzn/vx-linux-x64": "0.0.14",
21
- "@vzn/vx-linux-arm64": "0.0.14",
22
- "@vzn/vx-darwin-x64": "0.0.14",
23
- "@vzn/vx-darwin-arm64": "0.0.14"
20
+ "@vzn/vx-linux-x64": "0.0.16",
21
+ "@vzn/vx-linux-arm64": "0.0.16",
22
+ "@vzn/vx-darwin-x64": "0.0.16",
23
+ "@vzn/vx-darwin-arm64": "0.0.16"
24
24
  },
25
25
  "dependencies": {
26
26
  "@anthropic-ai/sandbox-runtime": "^0.0.56"
@@ -47,6 +47,15 @@ import { FULL_CACHE_POLICY } from './cache.js'
47
47
  export interface RemoteCacheLayer {
48
48
  /** Existence probe (drives the plan path's `--dry` remote prediction). */
49
49
  has(hash: string): Promise<boolean>
50
+ /**
51
+ * Optional batch existence probe: given N hashes, return the subset stored
52
+ * remotely in ONE round-trip. Lets the prefetch pass collapse N per-hash
53
+ * network probes into one, then fetch only the hits. A remote that can't
54
+ * batch omits this method (or returns `null`) and the layer falls back to
55
+ * the per-hash path. Never throws for control flow — `null` means "no batch
56
+ * info; use per-hash".
57
+ */
58
+ hasMany?(hashes: readonly string[]): Promise<Set<string> | null>
50
59
  /** Fetch an artifact's bytes; `null` = miss. `durationMs` is the
51
60
  * producing task's duration when the wire carries it. */
52
61
  get(hash: string): Promise<{ body: ArrayBuffer; durationMs: number | undefined } | null>
@@ -126,6 +135,38 @@ export class LayeredCache implements CacheLayer {
126
135
  return await this.pullFromRemote(hash, ctx)
127
136
  }
128
137
 
138
+ /**
139
+ * Batch existence probe over the remote layer — the subset of `hashes`
140
+ * present remotely, in one round-trip, or `null` when the remote can't
141
+ * batch (no `hasMany`, reads disabled, or an error). The prefetch pass uses
142
+ * this to fetch only the hits and to pre-mark the misses (`markRemoteAbsent`)
143
+ * so their lazy `get` skips the network. Never throws — a batch failure
144
+ * degrades to "no batch info" and the caller falls back to per-hash.
145
+ */
146
+ async remoteHasMany(hashes: readonly string[]): Promise<Set<string> | null> {
147
+ if (!this.policy.remoteRead || this.remote.hasMany === undefined) return null
148
+ try {
149
+ return await this.remote.hasMany(hashes)
150
+ } catch (err) {
151
+ this.reportRemoteError(err)
152
+ return null
153
+ }
154
+ }
155
+
156
+ /**
157
+ * Record that the remote layer has NO artifact for each of `hashes` (from a
158
+ * batch probe), so a later `get`/`prefetch` short-circuits to a miss WITHOUT
159
+ * a network round-trip. Byte-for-byte equivalent to a background prefetch GET
160
+ * having resolved `false` into `inflight` — same at-most-once semantics, same
161
+ * point-in-time staleness window — but without spending the GET. Never
162
+ * overwrites an entry already in flight.
163
+ */
164
+ markRemoteAbsent(hashes: Iterable<string>): void {
165
+ for (const hash of hashes) {
166
+ if (!this.inflight.has(hash)) this.inflight.set(hash, Promise.resolve(false))
167
+ }
168
+ }
169
+
129
170
  async get(hash: string, ctx?: CacheGetContext): Promise<CacheEntry | null> {
130
171
  const localHit = await this.local.get(hash, ctx)
131
172
  if (localHit) {
@@ -63,15 +63,30 @@ async function runPrefetch(args: PrefetchArgs): Promise<void> {
63
63
  const stableKeys = await deriveStableKeys(args)
64
64
  if (stableKeys.length === 0) return
65
65
 
66
- // Bounded worker pool over the stable keys. Each prefetch is
66
+ // Batch existence probe FIRST (when the remote supports it): one round-trip
67
+ // tells us which of the N stable hashes exist remotely, so we GET only the
68
+ // hits and pre-mark the misses — their lazy `get` then short-circuits with
69
+ // no network. This collapses N probe waves into 1 and skips every GET that
70
+ // would 404. When the remote can't batch (`null` — an older serve, or reads
71
+ // disabled), fall back to prefetching every stable key, exactly as before.
72
+ let toPrefetch = stableKeys
73
+ const uniqueHashes = [...new Set(stableKeys.map((k) => k.hash))]
74
+ const present = await args.cache.remoteHasMany(uniqueHashes)
75
+ if (present !== null) {
76
+ args.cache.markRemoteAbsent(uniqueHashes.filter((h) => !present.has(h)))
77
+ toPrefetch = stableKeys.filter((k) => present.has(k.hash))
78
+ if (toPrefetch.length === 0) return
79
+ }
80
+
81
+ // Bounded worker pool over the keys to fetch. Each prefetch is
67
82
  // self-contained (LayeredCache owns the in-flight map + ingest); we
68
83
  // cap concurrency so a 1000-task run doesn't open 1000 sockets at
69
84
  // once. The pumps race alongside execution.
70
85
  let next = 0
71
- const workers = Math.max(1, Math.min(args.concurrency, stableKeys.length))
86
+ const workers = Math.max(1, Math.min(args.concurrency, toPrefetch.length))
72
87
  const pump = async (): Promise<void> => {
73
- while (next < stableKeys.length) {
74
- const { hash, node } = stableKeys[next++]!
88
+ while (next < toPrefetch.length) {
89
+ const { hash, node } = toPrefetch[next++]!
75
90
  await args.cache
76
91
  .prefetch(hash, { taskId: node.id, command: node.config.exec?.command ?? '' })
77
92
  .catch(() => false)