@haystackeditor/cli 0.15.31 → 0.17.0

package/README.md

@@ -10,6 +10,13 @@ haystack login
 haystack setup
 ```
 
+Requires Node.js 22.18 or newer.
+
+After a command finishes in an interactive terminal, the CLI checks npm at most
+once a day and prints one line on stderr when a newer version exists. It never
+runs with `--json`, when stdout is not a TTY, or when `CI` is set. Set
+`HAYSTACK_NO_UPDATE_CHECK=1` to turn it off.
+
 The `setup` command walks you through an interactive wizard:
 
 1. **Select repositories** to configure
@@ -20,8 +27,9 @@ The `setup` command walks you through an interactive wizard:
 6. **Write `.haystack.json`** to your repos
 
 The CLI sends limited operational events to PostHog by default so Haystack can
-measure setup reliability. Events contain a pseudonymous machine identifier,
-CLI/version/platform fields, command-step and error categories, and numeric
+measure setup reliability. Events contain a pseudonymous install identifier
+(a random id generated once and stored in `~/.haystack/telemetry-id`; it is
+not derived from your username or hostname), CLI/version/platform fields, command-step and error categories, and numeric
 outcomes. They exclude source code, repository names, local paths, usernames,
 hostnames, and raw error text. IP enrichment is disabled. Set `DO_NOT_TRACK=1`
 or `HAYSTACK_TELEMETRY_DISABLED=1` to turn this telemetry off.
@@ -72,90 +80,89 @@ haystack triage <ref> --json    # poll findings later / after --no-wait
 `dismiss`, `mark_reviewed`, `undismiss`, `request_review`, `trigger_review`,
 and `schema`.
 
-**`haystack verify mcp`** runs the cloud-verifier MCP server. Retained
-application state is available through `verify_reopen`, `verify_refork`, and
-`verify_cleanup`. Agents select state by an exact Archil fork ID or a stable
-run/test/side coordinate; refork returns a durable child ID that can be
-reopened or nested again without receiving provider credentials. The matching
-CLI flow is:
+**`haystack verify`** verifies the current change using the authenticated
+Haystack fleet. Inside a git checkout, push your commit and run:
 
 ```bash
-haystack verify reopen latest 3
-# inspect or mutate the returned sandbox
-haystack verify refork latest 3 attempt:fix-2
-haystack verify reopen archil:<disk-id>:branch:<returned-child>
+haystack verify
+haystack verify --json
 ```
 
-**`haystack verify hosted`** is the authenticated production entry point for
-exact GitHub commit pairs. It uses the repository-default saved account,
-submits an idempotent run, and waits up to 35 minutes by default:
+The repository defaults to `owner/repo` from the `origin` remote URL. The head
+is the current `HEAD` commit. The base is `git merge-base HEAD
+origin/<default branch>`, using `origin/HEAD` to discover the default branch
+and falling back to `origin/main`. If the current commit is not reachable from
+any remote-tracking branch, the command asks you to push first.
+Explicit `--repo`, `--base`, and `--head` values override their defaults; supplying
+all three also works outside a checkout:
 
 ```bash
-haystack verify hosted start owner/repo \
-  --base <40-character-commit-sha> \
-  --head <40-character-commit-sha> \
-  --json
+haystack verify --repo owner/repo --base main --head feature --json
 ```
 
-Use `--no-wait` to return as soon as the workflow is queued. The result includes
-an account-bound `next_command`; use it verbatim on machines with multiple
-saved GitHub accounts. Repeating the same repository/base/head tuple reuses the
-same production run by default. Pass a new explicit `--idempotency-key` only
-when a fresh execution is intentional. A bounded wait that expires still emits
-the `cloud-verifier` schema with `timed_out: true`, the last observed status,
-and `next_command`, then exits 2. Read or resume a known run with:
+Intent is optional: `--intent-file <path>` accepts JSON with exactly `problem`,
+`goal`, and `intended_outcomes`. The CLI uses your saved account and waits up to
+35 minutes by default. Pass `--no-wait` to return after the fleet run is queued.
+Repeating the same repository, refs, and intent reuses the same run; choose a new
+`--idempotency-key` when a fresh execution is intentional. A wait that expires
+prints `timed_out: true` and `next_command`, then exits 2. Resume with:
 
 ```bash
-haystack verify hosted status cv_<48-lowercase-hex-characters> \
-  --account <login> \
-  --wait \
-  --json
+haystack verify hosted status cv_<48-lowercase-hex-characters> --wait --json
+haystack verify history owner/repo --limit 20
 ```
 
-To execute a completed run's exact validated risk plan again, keep the same
-repository and commit pair and pass:
+Use the returned account-bound `next_command` verbatim on machines with multiple
+saved accounts. `haystack verify hosted start owner/repo --base <sha> --head
+<sha>` starts the same fleet execution through an explicit subcommand.
+
+Retained fleet cases remain available through their exact case IDs:
 
 ```bash
-haystack verify hosted start owner/repo \
-  --base <40-character-commit-sha> \
-  --head <40-character-commit-sha> \
-  --replay-plan-from cv_<48-lowercase-hex-characters> \
-  --json
+haystack verify cases <run-id> --repo owner/repo
+haystack verify explore <run-id> <case-id> --repo owner/repo
 ```
 
-A successful replay emits a schema-9 terminal result whose
-`riskPlanReplayReceipt` binds the source run and canonical risk-plan hash. The
-replay does not mint or deliver a planning-model credential.
+Agents can use the same lifecycle with `haystack verify hosted mcp --account
+<login>`. Its tools are `verify_start`, `verify_status`, and `verify_wait`.
 
-Agents can use the same production lifecycle over stdio MCP:
+To repeat independent fleet runs for one exact commit pair:
 
 ```bash
-haystack verify hosted mcp --account <login>
+haystack verify hosted reproducibility owner/repo \
+  --base <40-character-commit-sha> \
+  --head <40-character-commit-sha> \
+  --runs 3 --json
 ```
 
-The hosted server exposes only `verify_start`, `verify_status`, and
-`verify_wait`. Tool results use the same `cloud-verifier` schema as the CLI;
-credentials, repository source, provider metadata, and sandbox mutation are
-not exposed.
+The report includes each run's fleet receipt. Each ordinal has a distinct
+idempotency key derived from the printed series key, repository, commits, and
+intent; `--series-key <key>` resumes the same series. Independent plans do not
+share an execution identity, so reproducibility remains unestablished and the
+command exits 2. A timeout stops the series and includes the active run's resume
+command.
 
-Freeze one production plan and repeat its exact execution to detect executor or
-assessment drift:
+**`haystack case-batch`** runs an already-generated product-fuzzing case
+batch on the hosted coordinator. The laptop only submits and polls: it holds no
+fleet credential, creates no VM, and kills nothing.
 
 ```bash
-haystack verify hosted reproducibility owner/repo \
-  --base <40-character-commit-sha> \
-  --head <40-character-commit-sha> \
-  --runs 3 \
-  --json
+haystack case-batch submit --repository owner/repo \
+  --base <40-character-commit-sha> --head <40-character-commit-sha> \
+  --cases cases.json --combinations space.json --max-concurrent 200 --json
+haystack case-batch status <run-id> --repository owner/repo --watch --json
+haystack case-batch cancel <run-id> --repository owner/repo
+haystack case-batch bundle <run-id> --repository owner/repo --out ./bundle
 ```
 
-The first run creates the source plan; every later run names it through
-`replayPlanFromRunId`. Each run gets a distinct idempotency key derived from one
-printed series key. Pass `--series-key <key>` to resume the same series without
-creating duplicate runs. The report verifies schema-9 replay provenance and
-compares stable risk-cell, capability, and comparison IDs. Missing replay
-provenance, assessment drift, timeouts, unsupported receipts, and inconclusive
-cells all prevent a reproducible verdict and exit 2.
+Cases and the combination space travel inline and the whole request is bounded
+at 8 MiB; no source patch is sent unless `--source-patch <file>` names the
+precompute-captured `{ cacheKey, patchSha256, patchGzBase64 }` record.
+`status` follows the case cursor to the end of the list and prints one row per
+case — id, position, status, unknown reason, wall time and whether cleanup was
+proven — including when the batch ends `incomplete`, which exits 2 with a
+summary rather than a single failure. `haystack schema case-batch` prints the
+`--json` schema.
 
 **`haystack setup --json`** speaks NDJSON: events (`question`, `permission`,
 `progress`, `result`) on stdout, replies on stdin keyed by
@@ -211,7 +218,7 @@ haystack logout
 
 ### `haystack submit`
 
-Create a PR from current changes. Runs pre-PR triage (code review, rules validation, instruction drift), pushes your branch, and opens the PR.
+Create a PR from current changes. Runs pre-PR triage (code review, rules validation), pushes your branch, and opens the PR.
 
 ```bash
 haystack submit                      # Triage -> create PR -> wait for analysis
@@ -222,6 +229,28 @@ haystack submit --force              # Skip triage checks
 haystack submit --no-wait            # Don't wait for analysis results
 ```
 
+**Pre-PR triage**: two checkers run in parallel, each one structured call to
+`gpt-6-astra` over the OpenAI Responses API at reasoning effort `max` (the
+highest the API accepts for that model). The key comes from `OPENAI_API_KEY`
+if set, else from the SSM SecureString `/haystack/secrets/shared/prod/OPENAI_API_KEY`
+(us-west-2) through the default AWS credential chain (e.g. `AWS_PROFILE=prod`).
+With neither, each checker reports `no OpenAI key (...)` and submit continues.
+
+- **code-review** reads `git diff -U10 origin/<base>...HEAD` and reports objective bugs.
+- **rules-validator** runs when `.haystack/pr-rules.yml` or an agent
+  instruction file (`CLAUDE.md`, `AGENTS.md`, `COPILOT.md`, `.cursorrules`,
+  `.github/copilot-instructions.md`) exists, and checks the diff against them.
+
+Each finding carries `file`, `line`, `severity` (`error`/`warning`/`info`),
+`category`, a one-sentence summary and a concrete failure scenario; results
+are saved to `.haystack/triage/<checker>.json`. A finding with severity
+`error` stops the submit (`--force` skips triage). A checker that cannot run
+(missing key, API error, stalled stream) prints `code review failed: <error>`
+and the submit continues. The reviewer sees only the diff, not the rest of the
+repository. `--triage-timeout <sec>` (or `.haystack.json` `triage.timeoutMs`)
+aborts a checker whose response stream has been silent that long (default
+300s); there is no total-time cap.
+
 The `--json` payload (`haystack schema submit`) reports the PR ref, the
 resolved title/body and where each came from, auto-merge/auto-fix state with
 its source (flag vs `.haystack.json`), and the analysis outcome in the shared
@@ -440,6 +469,57 @@ haystack hooks install-session --cli all    # All detected CLIs
 haystack hooks session-status               # Check session hook status
 ```
 
+### `haystack case-batch`
+
+Submit and poll product-fuzzing case batches on the hosted coordinator
+(`CASE-BATCH-V1`):
+
+```bash
+# Submit an ordered batch against one frozen world pair
+haystack case-batch submit --repository owner/repo \
+  --base <sha> --head <sha> \
+  --cases cases.json --combinations space.json \
+  --base-world frz_<16-hex> --head-world frz_<16-hex> \
+  --max-concurrent 200 --per-case-wall-ms 15000 --total-budget-ms 600000 --json
+
+# Read every per-case outcome; --watch polls until terminal or --max-wall
+haystack case-batch status <run-id> --repository owner/repo --watch --json
+
+# Request cancellation (a claimed batch drains until its owner proves cleanup)
+haystack case-batch cancel <run-id> --repository owner/repo
+
+# Download the replay bundle, verifying every artifact digest
+haystack case-batch bundle <run-id> --repository owner/repo --out ./bundle
+```
+
+`bundle` requires the service's sealed SHA-256 for `manifest.json` (served as
+its ETag), because the manifest is the one artifact with no parent digest to
+check it against; without it the command fails and writes nothing. Every other
+artifact is checked against the sha256 and byte length the manifest declares
+for it.
+
+Knobs and what turning them does:
+
+| Flag | Default | Effect |
+|------|---------|--------|
+| `--max-concurrent <n>` | the case count, capped at 500 | Ceiling only. Admitted concurrency is the smaller of it and the fleet's free capacity; the run's receipt reports both. |
+| `--per-case-wall-ms <ms>` | 120000 | A case that overruns is killed and recorded `UNKNOWN`/`wall-budget`, never dropped. Above the MVP's measured 103.8 s maximum, so it does not kill cases running at today's cost. |
+| `--total-budget-ms <ms>` | 1800000 (the contract ceiling) | Bounds launches from the first launch. Cases already launched run to their own wall budget. |
+| `--warm-settle-ms <ms>` | absent (0) | Each arm adopts only a warm clone that has been ready at least this long, 0 to the per-case wall. A clone adopted right after its restore attaches cold (quiet fleet: connectMs 5.8 s at 0 vs 1.1 s after 3.5-6 s of idle); each second of settle can delay a case's first create by up to a second. |
+| `--idempotency-key <key>` | derived from the whole submission | An exact retry returns the original run; a new key is a new execution. |
+| `--interval <seconds>` | 10 | Polling interval while `--watch` is set. |
+| `--max-wall <minutes>` | 45 | Total polling wall. Hitting it prints the snapshot, says `timed_out`, and exits 2. |
+
+`status` exits 0 for a `completed` batch with cleanup proven, and 2 for
+`incomplete`, `cancelled`, unproven cleanup, or a wait that hits `--max-wall`.
+Either way every per-case row is printed. A `--json` row from a v4 batch also
+carries `node_ids: { base1, head }` (the node each arm's VM was placed on) and
+`failure: { source, arm, error_class, error_code, node_id }` (the typed fact
+behind an `UNKNOWN` that is not an arrival or wall outcome, and behind an
+`arrival-failed` whose two arms stalled on the same step, `same-step-stall`); each
+is `null` when the outcome has none. A case in `--cases` may carry `repeatIndex` (1-16), which
+must be part of its `contentDigest`, to run as a repeat next to its original.
+
 ### `haystack policy`
 
 Manage review policies (`.haystack/review-policy.md`):

package/dist/assets/hooks/scripts/commit-msg.sh

@@ -1,2 +1,5 @@
 #!/bin/sh
 # Haystack commit-msg hook
+# Haystack's own hooks never need the push credential the CLI exports for
+# `git push`; drop it so nothing these hooks run can read it.
+unset HAYSTACK_GIT_TOKEN

package/dist/assets/hooks/scripts/post-commit.sh

@@ -1,2 +1,5 @@
 #!/bin/sh
 # Haystack post-commit hook
+# Haystack's own hooks never need the push credential the CLI exports for
+# `git push`; drop it so nothing these hooks run can read it.
+unset HAYSTACK_GIT_TOKEN

package/dist/assets/hooks/scripts/pre-commit.sh

@@ -1,4 +1,7 @@
 #!/usr/bin/env bash
+# Haystack's own hooks never need the push credential the CLI exports for
+# `git push`; drop it so nothing these hooks run can read it.
+unset HAYSTACK_GIT_TOKEN
 HOOKS_DIR="$(cd "$(dirname "$0")" && pwd)"
 REPO_ROOT="$(cd "$HOOKS_DIR/.." && pwd)"
 TSX_IMPORT="$HOOKS_DIR/node_modules/tsx/dist/loader.mjs"
@@ -72,14 +75,16 @@ run_tsx() {
     return
   fi
 
-  if [ -x "$TSX_BIN" ]; then
-    "$TSX_BIN" "$@"
-    return
-  fi
-
-  npx tsx "$@"
+  "$TSX_BIN" "$@"
 }
 
+# No network fallback: without a supported Node or the installed hook
+# dependencies, fail rather than fetch an unpinned tsx with npx.
+if ! node_supports_import && ! node_supports_loader && [ ! -x "$TSX_BIN" ]; then
+  echo "Haystack pre-commit: needs Node 18 or later, or hooks/node_modules (run npm ci in hooks/)." >&2
+  exit 1
+fi
+
 # Extract agent context if running under an AI agent
 # Capture stderr to detect whether an AI agent is present
 AGENT_OUTPUT=$(run_tsx "$HOOKS_DIR/agent-context/index.ts" "$REPO_ROOT" 2>&1) || true

package/dist/assets/hooks/scripts/pre-push.sh

@@ -1,5 +1,8 @@
 #!/bin/sh
 # Haystack pre-push hook
+# Haystack's own hooks never need the push credential the CLI exports for
+# `git push`; drop it so nothing these hooks run can read it.
+unset HAYSTACK_GIT_TOKEN
 # $1 is the remote name (e.g., "origin")
 
 # Check if pushing a non-haystack branch and remind about haystack submit

package/dist/assets/hooks/scripts/prepare-commit-msg.sh

@@ -1,2 +1,5 @@
 #!/bin/sh
 # Haystack prepare-commit-msg hook
+# Haystack's own hooks never need the push credential the CLI exports for
+# `git push`; drop it so nothing these hooks run can read it.
+unset HAYSTACK_GIT_TOKEN