jules-orchestrator-kit 0.68.0 → 0.69.0

package/CHANGELOG.md

@@ -5,6 +5,34 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.69.0] - 2026-09-04
+*A denominator is not evidence if the things counted in it were never read.*
+
+A third cold-start trial against v0.68.0. Twelve findings; six reproduced, and the two most serious were graded lower by the trial than they deserved. The pattern in both: the guard recognised a line as an assertion, counted it in `assertionsSeen`, and reported `PASS` without ever reading the value being asserted. `UNREADABLE` exists precisely so that "I could not read this" and "I read this and it is fine" look different — and a line could pass the readability test while its expectation stayed opaque.
+
+### Fixed
+- **Expected-Value-First Assertions Were Dismissed As Reworded Messages (`src/security.mjs`)**: `messageArgIndices` treated a string in argument 0 of any two-or-more argument assertion as prose for a human. JUnit and PHPUnit *document* the opposite order — `assertEquals(expected, actual)` — so no Java or PHP repository had any protection against a rewritten string expectation, and neither did Python's `assertIn(member, container)` or `assertNotIn`. Measured: `assertEquals("Hello World", out)` → `assertEquals("Hello Tampered", out)` returned `PASS` with `assertionsSeen: 2`. The one dialect that really does put prose first is JUnit 4's three-argument form, and it is distinguishable, because its trailing argument is the actual value rather than a message.
+- **A Rewritten Regex Was Neither A Change Nor A Loss (`src/security.mjs`)**: `blankLiterals` blanked strings, numbers and booleans, and left regex literals alone — so `toMatch(/Hello World/)` and `toMatch(/Hello Tampered/)` normalised to two different shapes, never met in a bucket, and balanced each other out in the count: one specific assertion removed, one added, silence. Jest's `toMatch` and `toThrow` and RSpec's `match` all take the pattern as their only argument, which is where a test states what it expects. The lookbehind separating a regex from a division is what makes this safe: an operand never precedes the opening `/` of a pattern.
+- **Renaming A Test Was Reported As Rewriting An Expectation (`src/security.mjs`)**: on the one-line form — `test("adds", () => { assert.strictEqual(add(2, 3), 5); });` — the name blanked to the same shape as its replacement, the two paired, and an author who renamed a test and nothing else was handed a `CRITICAL` finding quoting their whole line back at them. The multi-line form was never affected, because `NON_JOINING_CALL` already keeps a test name from joining to the assertion below it; this is the same rule for statements that fit on one line. A rename that also moves the expectation still differs after the name is blanked, so it is still reported.
+- **A Verification Command With An Environment Prefix Never Started (`src/git.mjs`)**: `PYTHONPATH=src python3 -m pytest` is how a large part of the Python world writes its test command. `execFileSync` took the whole assignment as the program name and failed with `spawnSync PYTHONPATH=src ENOENT`, which the gate reported as a failed verification — telling the user their tests broke when the command had never run. Leading assignments are peeled into the child's environment, which behaves the same on every platform where handing the string to a shell would not.
+- **`node -e ""` Was An Oracle (`src/stack-detector.mjs`)**: `true`, `echo ok` and `exit 0` were refused; the same no-op spelled as an interpreter with an empty program was accepted, and the gate returned `APPROVED (Exit 0)` on a change verified by nothing. Nothing can enumerate every way to write a command that runs nothing — that is what the collection floor is for, one-sidedly and by design — but the spellings that *look* like work are a small, decidable set.
+- **The Timeout Message Named Neither The Limit Nor The Knob (`src/git.mjs`, `src/engine.mjs`, `src/wizard-init.mjs`, `README.md`)**: the default was 60 seconds, which an ordinary mid-sized suite exceeds on cold caches, and the explanation was suppressed by a guard testing for a token that Node's own `spawnSync sh ETIMEDOUT` already contains — so it was skipped exactly when it was needed. `verify.timeout_ms` existed and worked, and appeared in no README, no generated config, and no error message. Default is now 300000, the key is written into the config `init` generates, and the message says the command was killed rather than failed.
+
+### Changed
+- **`init` Rejects An Oracle That Proves Nothing (`src/wizard-init.mjs`)**: `pnpm -r test` on a workspace whose packages declare no test script exits 0, prints nothing, and runs nothing — and the probe blessed it, leaving the repository configured to approve every future change against silence. Choosing a command is the right moment to be strict about this: at `init` the cost of rejecting a candidate is trying the next one, where at gate time it would be a hard red on a repository that is fine. Two passes now — prefer a command that proves it ran something, and only settle for one that merely exits 0 while saying so out loud.
+- **Seven More Runners State Their Count (`src/ops/test-collection.mjs`)**: Python `unittest`, GoogleTest, Catch2, `deno test`, `bun test`, AVA, and a bare TAP plan all landed in the same "I could not tell" bucket as a command that ran nothing. The floor stays one-sided — an unrecognised runner still passes, because failing on "I could not tell" would hard-red every correct repository not on the list — so widening it only ever converts silence into an answer.
+
+### Also Fixed
+- **The Documented Default Was Never The Loader's Default (`src/config.mjs`)**: the fallback was added to the engine, and `loadConfig` always supplied a value — so the fallback was unreachable and every repository without an explicit `timeout_ms` kept the one-minute limit while the changelog said five. A default belongs where the value is resolved, not where it is consumed. This is the third time in this project's history that a rule has been written in one place while another kept the old answer, so the regression test asserts the value a caller actually receives rather than any one site.
+- **Two Tests Gated This Repository Against Itself (`test/engine.test.mjs`, `test/kit.test.mjs`)**: both called the gate with `root: process.cwd()`, so the verify stage ran `npm test` — the whole suite, from inside the suite. Neither could finish; the stage timeout was their only stopping condition, and each asserted little more than that a boolean was a boolean. They cost one full timeout per run and were invisible while that timeout was 60 seconds. Raising it to 300 turned them into a CI failure on every matrix cell, which is how a pair of tests that verified nothing finally got noticed. Both now run against a fixture with a bounded oracle and assert the verdict. The suite went from 67s to 19s.
+- **The Egress Guard Could Not See `.js` (`test/egress-allowlist.test.mjs`)**: the scan collected `.mjs` only, so `bin/init.js` — published as the `jules-init` binary — sat outside the boundary entirely. Harmless as it stands, but the guard's whole purpose is that a reviewer can trust the boundary without reading every commit.
+
+### Not Reproduced
+Six of the twelve did not hold, and five quoted terminal output that does not exist anywhere in the shipped code. `--allow-test-change expectation` propagates correctly through both phases and returns `APPROVED`; `--allow-test-change removal` allows deleting a dead test alongside its dead function, also `APPROVED` — both reported failures were a test suite genuinely broken by the reporter's own edit, which is the same fixture artefact this project has hit repeatedly. `agentctl plan approve --dry-run` errors with *"Session ID is required"* rather than printing the quoted *"Plan Approved Successfully!"*. `task create -p` did not block on a prompt. A protected `package.json` is the design, and the gate already prints `To allow protected files in this run, pass: agentctl gate --allow-protected` — the finding stated there were no flag hints in the output. The `TEST_DIALECT_UNREADABLE` remediation copy was already correct: *"Exit 6 Test Integrity Violation — no secret was found"*, not the quoted *"Secret Leak Prevented"*. And `pnpm -r test` exits 0 on pnpm 10.33.4 rather than the reported `ERR_PNPM_RECURSIVE_RUN_NO_SCRIPT` — the real defect there was worse than the one reported, and is fixed above.
+
+### Added
+- **Eleven Cases In The Policy Contract (`src/guard-policy.mjs`)**: seven canaries for the expectation forms that were invisible — JUnit and PHPUnit expected-first, `assertIn`, `assertNotIn`, and regex patterns in `toMatch`, `toThrow` and RSpec `match` — and four innocent edits for the renames that were called tampering, plus JUnit 4's message-first form, which must stay silent. 42 canaries, 16 innocent edits.
+
 ## [0.68.0] - 2026-09-04
 *A check that examined nothing does not get to say APPROVED.*
 

package/README.md

@@ -208,7 +208,7 @@ To maximize PR merge rates, dispatch tasks according to deterministic boundaries
 * **Fail-Closed Security & Secret Redaction:** Evaluates explicit Deny rules before Allow rules against canonicalized, case-folded paths. Redacts high-entropy keys and base64-encoded credentials (such as Kubernetes `Secret` manifests).
 * **Complexity & Cost Router:** Zero-dependency heuristic classifier (`src/router.mjs`) routing mechanical tasks to lightweight models while reserving primary models for complex refactors, with a `node --check` syntax-verification gate that transparently escalates a FAST-tier result to the primary provider if it left broken JS on disk.
 * **Terminal UI & Diagnostic Matrix (`agentctl doctor`):** Interactive terminal dashboard, task sidecar manager, and automated transactional self-repair.
-* **Verified Test Suite:** Tested with **1163 unit tests across 166 suites passing in < 15.0s**.
+* **Verified Test Suite:** Tested with **1223 unit tests across 173 suites**, green on every supported platform.
 
 <br/>
 
@@ -286,6 +286,9 @@ branchPrefix: "agent/"   # Prefix for task branches
 verify:
   test: "npm test"
   build: "npm run build"
+  timeout_ms: 300000     # Per-stage kill time in ms (default 300000)
+  minTests: 1            # Floor for "the suite actually ran" (0 disables)
+  required: true         # false = this repo uses only the scope/secret phases
 
 # Scope protection rules (Deny-first evaluation)
 scope:

package/ROADMAP_V1.md

@@ -11,15 +11,23 @@ The **jules-orchestrator-kit** is the zero-dependency safety gatekeeper and self
 ## 📌 Release Milestones Overview
 
 ```
- v0.68.0 (Current Stable) ──► v0.69.0 (Distributed Swarms & Leases) ──► v1.0.0 (Production Hardened Kernel)
- (Can It Still Go Red?)      (Multi-Agent DAG & Resource Locks)        (Enterprise Telemetry & SLA)
+ v0.69.0 (Current Stable) ──► v0.70.0 (Distributed Swarms & Leases) ──► v1.0.0 (Production Hardened Kernel)
+ (Read, Not Just Counted)    (Multi-Agent DAG & Resource Locks)        (Enterprise Telemetry & SLA)
 ```
 
 ---
 
-## ✅ Shipped Milestones (v0.20.0 – v0.68.0)
+## ✅ Shipped Milestones (v0.20.0 – v0.69.0)
 
 
+### v0.69.0: Read, Not Just Counted
+- [x] **Expected-Value-First Assertions (`src/security.mjs`)** — JUnit and PHPUnit document the order the guard read as prose.
+- [x] **A Regex Is An Expected Value (`src/security.mjs`)** — a rewritten pattern was neither a change nor a loss.
+- [x] **Renaming A Test Is Not Tampering (`src/security.mjs`)** — on the one-line form, the name blanked into the expectation.
+- [x] **An Environment Prefix Runs (`src/git.mjs`)** — `PYTHONPATH=src pytest` never started, and was reported as a failure.
+- [x] **`node -e ""` Is A Placeholder (`src/stack-detector.mjs`)** — the same no-op, spelled to look like work.
+- [x] **`init` Rejects An Oracle That Proves Nothing (`src/wizard-init.mjs`)** — silence at setup is silence at every gate after it.
+
 ### v0.68.0: Not An Approval Either
 - [x] **An Unreadable Dialect Blocks (`src/security.mjs`)** — the guard said it had not checked, and approved anyway.
 - [x] **A Command That Cannot Fail Is Not An Oracle (`src/engine.mjs`)** — `task create` already refused what the gate accepted.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "jules-orchestrator-kit",
-  "version": "0.68.0",
+  "version": "0.69.0",
   "description": "Zero-dependency safety gatekeeper, test oracle generator, and multi-agent coordination protocol for autonomous coding agents — Google Jules, Claude Code, Codex and Gemini CLI.",
   "repository": {
     "type": "git",

package/src/config.mjs

@@ -626,7 +626,13 @@ export function loadConfig(root = resolveRoot(), explicitPath = null) {
   const rawTeardown = parsed.teardown_cmd ?? parsed.verify?.teardown;
   const rawBuild = parsed.build_cmd ?? parsed.verify?.build;
   const rawUnit = parsed.verify?.unit;
-  const verifyTimeoutMs = parsed.verify?.timeoutMs ?? parsed.verify?.timeout_ms ?? 60000;
+  // Five minutes, matching the value `init` writes and the README documents.
+  // The engine carries the same number as a fallback, but this loader always
+  // supplies one — so the fallback never fired, and every repository without
+  // an explicit `timeout_ms` kept the old one-minute limit while the
+  // changelog said otherwise. The default has to live where the value is
+  // resolved, not where it is consumed.
+  const verifyTimeoutMs = parsed.verify?.timeoutMs ?? parsed.verify?.timeout_ms ?? 300_000;
   const autoVerify = resolveVerify(root);
 
   const rawTier = String(process.env.JULES_TIER || parsed.tier || FALLBACK_TIER).toLowerCase();
@@ -697,7 +703,7 @@ export function loadConfig(root = resolveRoot(), explicitPath = null) {
       // Named so `doctor` can tell an operator that the gate they enabled is
       // running one fewer check than they think, and why.
       profileSkipped: profilePlan?.skipped ?? [],
-      timeoutMs: Number.isFinite(Number(verifyTimeoutMs)) ? Number(verifyTimeoutMs) : 60000,
+      timeoutMs: Number.isFinite(Number(verifyTimeoutMs)) ? Number(verifyTimeoutMs) : 300_000,
     },
     evidence: {
       enabled: parsed.evidence?.enabled ?? true,

package/src/engine.mjs

@@ -393,7 +393,11 @@ export async function gate(opts = {}) {
   }
 
   let flakyVerdictResult = null;
-  const verifyTimeout = trustedVerify.timeoutMs || 60000;
+  // Five minutes. A minute was too short for an ordinary suite — a mid-sized
+  // Node repository takes longer than that on cold caches — and the timeout
+  // surfaced as a plain non-zero verification, so the gate told the user their
+  // tests failed when it had killed them. Raise it with verify.timeout_ms.
+  const verifyTimeout = trustedVerify.timeoutMs || 300_000;
 
   // Monorepo scoping: run the suites the change can actually break.
   //

package/src/git.mjs

@@ -15,6 +15,10 @@ export class GateError extends Error {
   }
 }
 
+// `NAME=value` at the head of a command line: a POSIX environment prefix,
+// not the name of a program.
+const ENV_ASSIGNMENT = /^[A-Za-z_][A-Za-z0-9_]*=/;
+
 const DEFAULT_TIMEOUT = 10 * 60 * 1000; // 10 minutes
 const DEFAULT_MAX_BUFFER = 10 * 1024 * 1024; // 10 MB
 
@@ -139,6 +143,8 @@ export function runCmd(command, opts = {}) {
   let args = [];
   let useShell = false;
   let shellCmd = "";
+  /** @type {Record<string,string>|null} */
+  let envOverlay = null;
   // True when `args` came from splitting a whitespace-separated string, which
   // means no element can itself contain whitespace. See the Windows note below.
   let tokenized = false;
@@ -153,6 +159,24 @@ export function runCmd(command, opts = {}) {
       shellCmd = trimmed;
     } else {
       const tokens = trimmed.split(/\s+/).filter(Boolean);
+
+      // A POSIX command may be prefixed with environment assignments, and
+      // half the Python world writes its test command exactly that way:
+      // `PYTHONPATH=src python3 -m pytest`. execFileSync took the first token
+      // as the program and failed with `spawnSync PYTHONPATH=src ENOENT` —
+      // which the gate then reported as a failed verification, so the user
+      // was told their tests broke when the command had never started.
+      //
+      // Peeling the assignments into the child's environment behaves the same
+      // on every platform. Handing the string to a shell would not: cmd.exe
+      // has no such syntax, so the Windows matrix would keep failing.
+      while (tokens.length > 1 && ENV_ASSIGNMENT.test(tokens[0])) {
+        const token = tokens.shift();
+        const eq = token.indexOf("=");
+        if (!envOverlay) envOverlay = {};
+        envOverlay[token.slice(0, eq)] = token.slice(eq + 1);
+      }
+
       binary = tokens[0] || "";
       args = tokens.slice(1);
       tokenized = true;
@@ -177,10 +201,12 @@ export function runCmd(command, opts = {}) {
   // wrapped in cmd.exe with every element quoted per the C-runtime + cmd.exe
   // rules, which preserves argv exactly while still letting the shell resolve
   // the `.cmd` shim (P-07).
+  const childEnv = envOverlay ? { ...(opts.env || process.env), ...envOverlay } : opts.env || process.env;
+
   const winShim = tokenized && process.platform === "win32";
   const winSpawn =
     !useShell && !winShim && process.platform === "win32"
-      ? resolveWindowsSpawn(binary, args, opts.env || process.env)
+      ? resolveWindowsSpawn(binary, args, childEnv)
       : null;
 
   if (!binary && !useShell) {
@@ -194,7 +220,7 @@ export function runCmd(command, opts = {}) {
           cwd,
           encoding: "utf-8",
           stdio: ["ignore", "pipe", "pipe"],
-          env: opts.env || process.env,
+          env: childEnv,
           timeout,
           maxBuffer,
         })
@@ -204,7 +230,7 @@ export function runCmd(command, opts = {}) {
           shell: winShim,
           windowsVerbatimArguments: Boolean(winSpawn && winSpawn.verbatim),
           stdio: ["ignore", "pipe", "pipe"],
-          env: opts.env || process.env,
+          env: childEnv,
           timeout,
           maxBuffer,
         });
@@ -218,8 +244,16 @@ export function runCmd(command, opts = {}) {
     let stdout = (err.stdout || "").toString().trim();
     let stderr = (err.stderr || err.message || "").toString().trim();
 
-    if (isTimeout && !stderr.includes("ETIMEDOUT")) {
-      stderr = `Command execution timed out after ${timeout}ms (ETIMEDOUT)${stderr ? "\n" + stderr : ""}`;
+    if (isTimeout) {
+      // Node's own message for this is `spawnSync sh ETIMEDOUT`, which already
+      // contains the token the old guard tested for — so the explanation was
+      // skipped exactly when it was needed, and the user was left with five
+      // words that name neither the limit nor the way to raise it.
+      stderr =
+        `Command execution timed out after ${timeout}ms (ETIMEDOUT). ` +
+        `The command was killed, not failed: raise verify.timeout_ms in ` +
+        `.agent/config.yml if this suite legitimately takes longer.` +
+        (stderr ? "\n" + stderr : "");
     }
     if (isNobufs && !stderr.includes("ENOBUFS")) {
       stderr = `Command output buffer exceeded limit of ${maxBuffer} bytes (ENOBUFS)${stderr ? "\n" + stderr : ""}`;

package/src/guard-policy.mjs

@@ -68,6 +68,62 @@ export const TEST_PATH_CASES = [
  * the guard the layout the implementation was written from.
  */
 export const TAMPER_CANARIES = [
+  // Argument order is not a detail. JUnit and PHPUnit document
+  // `assertEquals(expected, actual)`, and Python's containment helpers take
+  // the expected member first — all of which the guard used to read as "a
+  // human message was reworded" and dismiss. Each of these was a silent PASS.
+  {
+    id: "expectation/junit-expected-first",
+    file: "src/test/java/CalcTest.java",
+    removed: ['        assertEquals("Hello World", out);'],
+    added: ['        assertEquals("Hello Tampered", out);'],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
+  {
+    id: "expectation/phpunit-expected-first",
+    file: "tests/CalcTest.php",
+    removed: ['        $this->assertSame("Hello World", $out);'],
+    added: ['        $this->assertSame("Hello Tampered", $out);'],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
+  {
+    id: "expectation/unittest-assert-in",
+    file: "tests/test_cli.py",
+    removed: ['        self.assertIn("Hello World", result.output)'],
+    added: ['        self.assertIn("Hello Tampered", result.output)'],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
+  {
+    id: "expectation/unittest-assert-not-in",
+    file: "tests/test_cli.py",
+    removed: ['        self.assertNotIn("error", result.output)'],
+    added: ['        self.assertNotIn("zzz", result.output)'],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
+  // A regex literal is an expected value. It never blanked, so the rewritten
+  // pattern never met its original in a shape bucket and the swap was
+  // reported as neither a change nor a loss.
+  {
+    id: "expectation/jest-to-match-regex",
+    file: "test/cli.test.js",
+    removed: ["  expect(out).toMatch(/Hello World/);"],
+    added: ["  expect(out).toMatch(/Hello Tampered/);"],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
+  {
+    id: "expectation/jest-to-throw-regex",
+    file: "test/cli.test.js",
+    removed: ["  expect(fn).toThrow(/permission denied/);"],
+    added: ["  expect(fn).toThrow(/x/);"],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
+  {
+    id: "expectation/rspec-match-regex",
+    file: "spec/cli_spec.rb",
+    removed: ["    expect(out).to match(/Hello World/)"],
+    added: ["    expect(out).to match(/Hello Tampered/)"],
+    expect: "ASSERTION_EXPECTATION_CHANGED",
+  },
   {
     id: "skip-injection/node",
     file: "test/calc.test.js",
@@ -398,6 +454,41 @@ export const SCOPE_CANARIES = [
  * from never showed it.
  */
 export const INNOCENT_EDITS = [
+  // Renaming a test is one of the most ordinary edits there is. On the
+  // one-line form the name blanked to the same shape as its replacement, the
+  // two paired, and the pair was reported as a rewritten expectation.
+  {
+    id: "rename-test-oneline/node",
+    file: "test/calc.test.js",
+    context: "// arithmetic",
+    removed: ['test("adds", () => { assert.strictEqual(add(2, 3), 5); });'],
+    added: ['test("adds positives", () => { assert.strictEqual(add(2, 3), 5); });'],
+    why: "the test name is prose about the test, not a value it asserts",
+  },
+  {
+    id: "rename-test-oneline/jest",
+    file: "test/calc.spec.js",
+    context: "// arithmetic",
+    removed: ['it("adds", () => { expect(add(2, 3)).toBe(5); });'],
+    added: ['it("adds two positives", () => { expect(add(2, 3)).toBe(5); });'],
+    why: "same rename, the dialect a new user is most likely to be in",
+  },
+  {
+    id: "rename-test-oneline/tap",
+    file: "test/calc.test.js",
+    context: "// arithmetic",
+    removed: ['t.test("adds", (ct) => { ct.equal(add(2, 3), 5); ct.end(); });'],
+    added: ['t.test("adds positives", (ct) => { ct.equal(add(2, 3), 5); ct.end(); });'],
+    why: "the receiver is a sub-test callback argument, not an assertion subject",
+  },
+  {
+    id: "reword-junit-message-first",
+    file: "src/test/java/CalcTest.java",
+    context: "// arithmetic",
+    removed: ['        assertEquals("why", expected, out);'],
+    added: ['        assertEquals("why this matters", expected, out);'],
+    why: "JUnit 4 puts the message first — rewording it changes no expectation",
+  },
   {
     id: "add-assertion-beside-comment/node",
     file: "test/calc.test.js",

package/src/ops/test-collection.mjs

@@ -50,6 +50,27 @@ const COUNT_PATTERNS = [
   { name: "dotnet", re: /\bTotal(?:\s+tests)?:\s*(\d+)/i },
   // swift test / XCTest — "Executed 12 tests"
   { name: "xctest", re: /\bExecuted\s+(\d+)\s+tests?/i },
+  // Every runner below states a count the gate used to read as silence, and
+  // silence is what makes `pnpm -r test` on a workspace with no package-level
+  // test script indistinguishable from a suite that ran. The floor stays
+  // one-sided — an unrecognised runner still passes — so widening the list
+  // only ever converts an "I could not tell" into an answer.
+  // Python unittest — "Ran 12 tests in 0.003s"
+  { name: "unittest", re: /^Ran\s+(\d+)\s+tests?\s+in\b/m },
+  // GoogleTest — "[==========] 12 tests from 3 test suites ran."
+  { name: "gtest", re: /\[=+\]\s+(\d+)\s+tests?\s+from\b/ },
+  // Catch2 — "test cases: 12 | 12 passed"
+  { name: "catch2", re: /\btest cases:\s*(\d+)/i },
+  // deno test — "ok | 12 passed | 0 failed"
+  { name: "deno", re: /\|\s*(\d+)\s+passed\s*\|/ },
+  // bun test — "12 pass"
+  { name: "bun", re: /^\s*(\d+)\s+pass\s*$/m },
+  // ava — "12 tests passed"
+  { name: "ava", re: /^\s*(\d+)\s+tests?\s+passed\s*$/m },
+  // A bare TAP plan, which is how tape, bats and the TAP producers that
+  // print no summary state their count. Last of the count patterns: node:test
+  // emits TAP too, and its own line above is the more specific reading.
+  { name: "tap", re: /^1\.\.(\d+)\s*$/m },
 ];
 
 /** Per-test lines, which `go test` only prints under -v. */

package/src/security.mjs

@@ -1219,6 +1219,21 @@ const isSpecificAssertion = (str) => SPECIFIC_ASSERTION.test(str);
 const blankLiterals = (str) =>
   str
     .replace(/(['"`])(?:\\.|(?!\1)[^\\])*\1/g, "\u0000S")
+    // A regex literal is an expected value like any other. Without this,
+    // `toMatch(/Hello World/)` and `toMatch(/Hello Tampered/)` normalized to
+    // two different shapes, never met in a bucket, and the rewrite was
+    // reported as neither a change nor a loss — one specific assertion out,
+    // one in, and silence. Runs after the string pass so a `/` inside a
+    // string is already gone, and before the number pass so a pattern
+    // containing digits collapses whole.
+    //
+    // The lookbehind is what separates a regex from a division: an operand
+    // never precedes `/` here, only an opening paren, a comma, or an
+    // operator, which is where a test's expected pattern actually sits.
+    .replace(
+      /(?<=[(,=:[!&|?{;]\s{0,8})\/(?![*/])(?:\\.|\[(?:\\.|[^\]\\\n])*\]|[^/\\\n])+\/[dgimsuvy]*/g,
+      "\u0000R"
+    )
     // The sign belongs to the literal: without it `3` and `-1` normalized to
     // different shapes and the rewritten expectation was never paired.
     .replace(
@@ -1802,8 +1817,22 @@ function isPureStringLiteral(arg) {
  */
 function messageArgIndices(args) {
   const idx = new Set();
-  if (args.length >= 3 && isPureStringLiteral(args[args.length - 1])) idx.add(args.length - 1);
-  if (args.length >= 2 && isPureStringLiteral(args[0])) idx.add(0);
+  const lastIsMessage = args.length >= 3 && isPureStringLiteral(args[args.length - 1]);
+  if (lastIsMessage) idx.add(args.length - 1);
+
+  // JUnit 4 is the one common dialect that puts the message *first*:
+  // `assertEquals("why this matters", expected, actual)`. It is also
+  // distinguishable, because its trailing argument is the actual value rather
+  // than prose — so a call that already carries a trailing message is not
+  // that shape, whatever its first argument looks like.
+  //
+  // Reading argument 0 as prose whenever it happened to be a string is what
+  // made a whole family of assertions invisible: `assertEquals(expected,
+  // actual)` — JUnit's and PHPUnit's own two-argument order — along with
+  // Python's `assertIn(member, container)` and `assertNotIn`. A rewritten
+  // expectation in any of them was dismissed as a reworded message, and the
+  // guard reported PASS on a check it had not performed.
+  if (!lastIsMessage && args.length >= 3 && isPureStringLiteral(args[0])) idx.add(0);
   return idx;
 }
 
@@ -1846,6 +1875,63 @@ function splitTrailingMessage(clean) {
   return { head: clean.slice(0, lastComma), msg: tail };
 }
 
+// A test declaration whose first argument is the test's name. The name is
+// prose about the test, not a value the test asserts — `test("adds", ...)`
+// renamed to `test("adds positives", ...)` is the rename the diff says it is.
+const TEST_DECL_CALL =
+  /\b(?:it|test|describe|context|suite|specify|bench|scenario)\s*\(|\b[a-z_$][\w$]{0,2}\.(?:test|Run|describe)\s*\(/;
+
+/**
+ * Replace the name argument of a test declaration with a placeholder.
+ *
+ * @param {string} clean - comment-stripped statement text
+ * @returns {string|null} the text with the name blanked, or null when the
+ *   statement is not a test declaration with a literal name.
+ */
+function blankTestName(clean) {
+  const m = TEST_DECL_CALL.exec(clean);
+  if (!m) return null;
+
+  let i = m.index + m[0].length;
+  while (i < clean.length && /\s/.test(clean[i])) i++;
+  const quote = clean[i];
+  if (quote !== '"' && quote !== "'" && quote !== "`") return null;
+
+  let j = i + 1;
+  while (j < clean.length) {
+    if (clean[j] === "\\") { j += 2; continue; }
+    if (clean[j] === quote) break;
+    j += 1;
+  }
+  if (j >= clean.length) return null;
+
+  return clean.slice(0, i) + "\u0000T" + clean.slice(j + 1);
+}
+
+/**
+ * True when the only thing that changed was the test's name.
+ *
+ * A one-line `test("adds", () => { assert.strictEqual(add(2, 3), 5); });`
+ * blanks to the same shape as its renamed copy, so the two pair — and the
+ * pair was then reported as a rewritten expectation, quoting the whole line
+ * back at an author who had renamed a test and nothing else. Renaming a test
+ * is one of the most ordinary edits there is, and a gate that calls it
+ * tampering is a gate that gets switched off.
+ *
+ * The multi-line form was never affected: NON_JOINING_CALL already keeps a
+ * test name from joining to the assertion below it. This is the same rule for
+ * the statements that fit on one line.
+ *
+ * A rename that also moves the expectation still differs after the name is
+ * blanked, so it is still reported.
+ */
+function differsOnlyInTestName(cleanRemoved, cleanAdded) {
+  const a = blankTestName(cleanRemoved);
+  const b = blankTestName(cleanAdded);
+  if (a === null || b === null) return false;
+  return a.replace(/\s+/g, "") === b.replace(/\s+/g, "");
+}
+
 function differsOnlyInMessage(cleanRemoved, cleanAdded, lang) {
   const ta = splitTrailingMessage(cleanRemoved);
   const tb = splitTrailingMessage(cleanAdded);
@@ -1874,6 +1960,17 @@ function differsOnlyInMessage(cleanRemoved, cleanAdded, lang) {
   return sawDifference;
 }
 
+/**
+ * True when a paired difference is something other than a rewritten
+ * expectation — a reworded message, or a renamed test.
+ */
+function isNonExpectationDifference(cleanRemoved, cleanAdded, lang) {
+  return (
+    differsOnlyInMessage(cleanRemoved, cleanAdded, lang) ||
+    differsOnlyInTestName(cleanRemoved, cleanAdded)
+  );
+}
+
 /**
  * Pair rewritten expectations across the removed and added images of every
  * hunk of one file, and report each pair.
@@ -1988,7 +2085,7 @@ function detectExpectationRewrites(file, hunks, stats, violations) {
         pairedOld.add(r);
         pairedNew.add(a);
         if (r.canon === a.canon) continue;
-        if (differsOnlyInMessage(r.clean, a.clean, lang)) continue;
+        if (isNonExpectationDifference(r.clean, a.clean, lang)) continue;
         pairs.push({ r: r.s, a: a.s });
       }
     }
@@ -2018,7 +2115,7 @@ function detectExpectationRewrites(file, hunks, stats, violations) {
         // different assertions that merely resemble each other.
         if (!same(0)) continue;
         if (ra.every((_, i) => same(i))) continue;
-        if (differsOnlyInMessage(r.clean, a.clean, lang)) continue;
+        if (isNonExpectationDifference(r.clean, a.clean, lang)) continue;
         pairs.push({ r: r.s, a: a.s });
         pairedOld.add(r);
         pairedNew.add(a);
@@ -2039,7 +2136,7 @@ function detectExpectationRewrites(file, hunks, stats, violations) {
         if (sr === sa && hasLiteralPlaceholder(sr)) {
           const clr = stripComments(r.text, lang);
           const cla = stripComments(a.text, lang);
-          if (clr.replace(/\s+/g, "") !== cla.replace(/\s+/g, "") && !differsOnlyInMessage(clr, cla, lang)) {
+          if (clr.replace(/\s+/g, "") !== cla.replace(/\s+/g, "") && !isNonExpectationDifference(clr, cla, lang)) {
             pairs.push({ r, a });
           }
         }

package/src/stack-detector.mjs

@@ -66,10 +66,20 @@ function makefileHasTestTarget(root) {
  * placeholder by this definition, and correctly so: it exits non-zero, which
  * fails loudly rather than certifying nothing.
  */
+// An interpreter handed an empty program: `node -e ""`, `python3 -c ''`,
+// `sh -c ""`. It reads as a test command at a glance and runs nothing.
+//
+// This does not enumerate every way to write a no-op — nothing can, and the
+// collection floor is what reports the general case, one-sidedly and by
+// design. It closes the spellings that look like work.
+const EMPTY_PROGRAM =
+  /^(?:\S*[/\\])?(?:node|nodejs|deno|bun|python[\d.]*|ruby|perl|php|sh|bash|zsh|dash)(?:\s+-{1,2}[\w-]+)*\s+(?:-e|-c|--eval|--execute)(?:\s*(?:""|''|``))?\s*$/;
+
 export function isPlaceholderTestScript(cmd) {
   if (typeof cmd !== "string") return false;
   const trimmed = cmd.trim();
   if (!trimmed) return true;
+  if (EMPTY_PROGRAM.test(trimmed)) return true;
   // Drop the announcements; what matters is what the shell is left doing.
   const remainder = trimmed
     .split(/&&|;/)

package/src/wizard-init.mjs

@@ -6,6 +6,7 @@ import { detectDefaultBranch } from "./git.mjs";
 import { resolveWorkspaceBoundary, oracleCandidates } from "./stack-detector.mjs";
 import { PROFILE_NAMES, PROFILE_DESCRIPTIONS } from "./profiles.mjs";
 import { detectStackOracles, runVerificationProbe } from "./wizard-oracle.mjs";
+import { parseCollectedTests } from "./ops/test-collection.mjs";
 import { select, multiSelect, input, confirm, spinner, isTTY } from "./tui.mjs";
 import { KIT_VERSION } from "./version.mjs";
 
@@ -215,6 +216,9 @@ verify:
   # to their sub-projects and runs only those suites (monorepos)
   scope: ${verifyScope}
   test: "${verify.test}"
+  # How long a verification stage may run before the gate kills it (default
+  # 300000). Raise it for a suite that legitimately takes longer.
+  timeout_ms: 300000
   build: "${verify.build}"
   lint: "${verify.lint}"
   typecheck: "${verify.typecheck}"
@@ -314,25 +318,77 @@ export function loadPresets(root = process.cwd()) {
  *
  * @returns {Promise<string>} the command to save
  */
+/**
+ * How much a probe actually proved.
+ *
+ * Exit 0 is the weakest of the three answers. `pnpm -r test` on a workspace
+ * whose packages declare no test script exits 0, prints nothing, and runs
+ * nothing — and it was accepted here as a verified oracle, which left the
+ * repository configured to approve every future change against silence.
+ *
+ * Choosing a command is the right moment to be strict about this: at init the
+ * cost of rejecting a candidate is trying the next one, where at gate time it
+ * would be a hard red on a repository that is fine.
+ */
+function probeVerdict(probeRes) {
+  if (!probeRes.ok) return "failed";
+  const { count } = parseCollectedTests(probeRes.stdout, probeRes.stderr);
+  if (count === null) return "silent";
+  return count > 0 ? "ran" : "empty";
+}
+
 async function resolveRunnableOracle(root, testCmd, options = {}) {
   if (!testCmd) return testCmd;
   const probeSp = spinner(`Probing oracle: ${testCmd}`, options);
   const probeRes = await runVerificationProbe(testCmd, root);
-  if (probeRes.ok) {
+  const verdict = probeVerdict(probeRes);
+  if (verdict === "ran") {
     probeSp.stop(`Oracle verified successfully (${probeRes.durationMs}ms)`);
     return testCmd;
   }
-  probeSp.fail(`Oracle verification probe failed (Exit ${probeRes.code})`);
+  if (verdict === "failed") {
+    probeSp.fail(`Oracle verification probe failed (Exit ${probeRes.code})`);
+  } else {
+    probeSp.fail(
+      verdict === "empty"
+        ? `"${testCmd}" exited 0 but ran no tests — looking for a command that does`
+        : `"${testCmd}" exited 0 without stating how many tests it ran — looking for a better one`
+    );
+  }
 
   const alternates = oracleCandidates(root, testCmd).filter((c) => c !== testCmd).slice(0, 3);
+  // Two passes: prefer a command that proves it ran something, and only then
+  // settle for one that merely exits 0. Falling back on the first exit-0
+  // candidate would reintroduce exactly the silence this rejects.
+  const settled = [];
   for (const cand of alternates) {
     const altSp = spinner(`Trying ${cand}`, options);
     const altRes = await runVerificationProbe(cand, root);
-    if (altRes.ok) {
+    const altVerdict = probeVerdict(altRes);
+    if (altVerdict === "ran") {
       altSp.stop(`${cand} runs here (${altRes.durationMs}ms) — using it instead`);
       return cand;
     }
-    altSp.fail(`${cand} also failed (Exit ${altRes.code})`);
+    if (altVerdict === "silent") {
+      altSp.fail(`${cand} exited 0 without stating a test count`);
+      settled.push(cand);
+      continue;
+    }
+    altSp.fail(altVerdict === "empty" ? `${cand} ran no tests` : `${cand} also failed (Exit ${altRes.code})`);
+  }
+
+  // Nothing proved it ran a suite. A command that at least starts and exits 0
+  // still beats one that does not run at all, so it is used — and said out
+  // loud, because the gate will only be able to check it by exit code.
+  if (settled.length > 0 || verdict === "silent") {
+    const chosen = verdict === "silent" ? testCmd : settled[0];
+    const out = options.stdout || process.stdout;
+    out.write("\n");
+    out.write(`   \u26a0\ufe0f  "${chosen}" exits 0 but states no test count.\n`);
+    out.write("      The gate can verify it by exit code alone, which cannot tell a\n");
+    out.write("      full suite from a command that ran nothing. If this repository\n");
+    out.write("      has a suite, point verify.test at it in .agent/config.yml.\n\n");
+    return chosen;
   }
 
   // Nothing runs. Say so in terms the user can act on, rather than leaving a