jules-orchestrator-kit 0.69.0 → 0.70.0

package/CHANGELOG.md

@@ -5,6 +5,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.70.0] - 2026-09-05
+*A session that has not finished is not a session that passed.*
+
+An audit of the Jules session layer against the API it talks to. Twelve findings, each traced to a file and line in `docs/jules-quality-plan.md`; the three below are the ones that let the kit believe something about a session that was not true.
+
+### Fixed
+- **The Retry Was Dispatched Without The Failure It Existed To Fix (`src/session-ops.mjs`)**: `retrySession` collected its diagnostics from `act.error`, `act.executionOutput`, `act.exitCode` and `act.status`. None of those four fields exists in the documented `Activity` type — what the API returns is `artifacts[].bashOutput.{command,output,exitCode}` and `sessionFailed.reason`. Measured against a response shaped exactly like the documentation, a `FAILED` session whose bash artifact carries `not ok 1 - invoice totals must round to cents … AssertionError: expected 10.01 to equal 10.00` with `exitCode: 1`, the retry went out with `[PREVIOUS_ATTEMPT_FAILURE_DIAGNOSTIC]` set to *"Previous session did not complete cleanly."* It was handed a sentence when it needed the assertion, the file and the line. `extractFailureDiagnostics` reads the documented fields, returns them highest-signal first so the 4000-character cut drops the least useful evidence rather than the one that failed, deduplicates repeats, and still reads the legacy spellings for provider shapes that have not been observed.
+- **An Unfinished Session Was Reported As COMPLETED (`src/engine.mjs`)**: `pollSessionState` recognised two of the nine documented `SessionState` values and returned `String(session.status || "COMPLETED")` for every other exit — and `session.status` is never set on a fresh dispatch. Measured: a session in `AWAITING_USER_FEEDBACK` and one still `IN_PROGRESS` when the poll budget expired both came back `COMPLETED`, so `QUEUED`, `PLANNING`, `PAUSED` and a session waiting on a human were indistinguishable from success. Terminal, blocked and timed-out are now three distinct verdicts, a provider that stops answering is `unreachable` rather than a timeout describing the wrong thing, and nothing synthesises `COMPLETED` any more. The one path that still returns it is the dry-run simulation, and it is now flagged `simulated`.
+
+### Added
+- **A Contract For The Poll (`test/session-poll.test.mjs`)**: 22 cases across all nine `SessionState` values, the timeout and wall-clock paths, the `approvePlan` side effect granted and refused, the unreachable paths, and the dry-run short-circuit. `pollSessionState` decides whether an agent session is believed to have finished writing — every later gate builds on that — and `grep -rn "pollSession" test/` returned nothing. That is the same class of hole `scripts/guard-reach-check.mjs` exists to close, in the one function that most needed it.
+- **A Green Run Is Not Evidence (`test/session-ops.test.mjs`)**: the opposite failure of a diagnostic collector is one that flags everything, which buries the command that failed under a hundred that passed. `pytest` printing `13 passed`, node:test printing `# fail 0` and `go test` printing `ok` on exit 0 each yield no diagnostics.
+
+### Changed
+- **`agentctl retry` Says When The Trace Is The Fallback (`bin/agentctl.mjs`, `src/session-ops.mjs`)**: `failureReason` alone cannot tell a real trace from the generic sentence — both are non-empty strings. `retrySession` now returns `diagnosticsFound` and `diagnosticSources`, and the CLI prints the count and where the evidence came from, or says plainly that the retry is going out with nothing but the generic line.
+- **A Non-Terminal Session Is Announced Before The Gate Runs (`src/engine.mjs`)**: the repair loop polled the session and discarded the answer, then ran re-verification against a tree the agent might not have finished writing. The verdict is now read: a non-terminal session prints `[SESSION_NOT_TERMINAL]` naming what it is waiting on and appends a `session_not_terminal` telemetry event. The gate still runs either way — it is the authority on whether the change works — but it no longer runs silently on a half-applied patch.
+
 ## [0.69.0] - 2026-09-04
 *A denominator is not evidence if the things counted in it were never read.*
 

package/README.md

@@ -208,7 +208,7 @@ To maximize PR merge rates, dispatch tasks according to deterministic boundaries
 * **Fail-Closed Security & Secret Redaction:** Evaluates explicit Deny rules before Allow rules against canonicalized, case-folded paths. Redacts high-entropy keys and base64-encoded credentials (such as Kubernetes `Secret` manifests).
 * **Complexity & Cost Router:** Zero-dependency heuristic classifier (`src/router.mjs`) routing mechanical tasks to lightweight models while reserving primary models for complex refactors, with a `node --check` syntax-verification gate that transparently escalates a FAST-tier result to the primary provider if it left broken JS on disk.
 * **Terminal UI & Diagnostic Matrix (`agentctl doctor`):** Interactive terminal dashboard, task sidecar manager, and automated transactional self-repair.
-* **Verified Test Suite:** Tested with **1223 unit tests across 173 suites**, green on every supported platform.
+* **Verified Test Suite:** Tested with **1255 unit tests across 173 suites**, green on every supported platform.
 
 <br/>
 

package/ROADMAP_V1.md

@@ -11,15 +11,21 @@ The **jules-orchestrator-kit** is the zero-dependency safety gatekeeper and self
 ## 📌 Release Milestones Overview
 
 ```
- v0.69.0 (Current Stable) ──► v0.70.0 (Distributed Swarms & Leases) ──► v1.0.0 (Production Hardened Kernel)
- (Read, Not Just Counted)    (Multi-Agent DAG & Resource Locks)        (Enterprise Telemetry & SLA)
+ v0.70.0 (Current Stable) ──► v0.71.0 (Distributed Swarms & Leases) ──► v1.0.0 (Production Hardened Kernel)
+ (Not Finished Is Not Passed) (Multi-Agent DAG & Resource Locks)       (Enterprise Telemetry & SLA)
 ```
 
 ---
 
-## ✅ Shipped Milestones (v0.20.0 – v0.69.0)
+## ✅ Shipped Milestones (v0.20.0 – v0.70.0)
 
 
+### v0.70.0: Not Finished Is Not Passed
+- [x] **An Unfinished Session Is Not COMPLETED (`src/engine.mjs`)** — terminal, blocked and timed-out are three verdicts, not one.
+- [x] **The Retry Carries The Failure (`src/session-ops.mjs`)** — it was reading four fields the API does not return.
+- [x] **`agentctl retry` Says When The Trace Is The Fallback (`bin/agentctl.mjs`)** — a generic sentence must not look like evidence.
+- [x] **A Contract For The Poll (`test/session-poll.test.mjs`)** — 22 cases over all nine documented session states.
+
 ### v0.69.0: Read, Not Just Counted
 - [x] **Expected-Value-First Assertions (`src/security.mjs`)** — JUnit and PHPUnit document the order the guard read as prose.
 - [x] **A Regex Is An Expected Value (`src/security.mjs`)** — a rewritten pattern was neither a change nor a loss.

package/bin/agentctl.mjs

@@ -959,7 +959,13 @@ async function main() {
         console.log(`------------------------------------------------------------------`);
         console.log(`  Original Session    : ${res.originalSessionId}`);
         console.log(`  New Session ID      : ${res.newSession?.id || "N/A"}`);
-        console.log(`  Failure Trace Added : ${res.failureReason ? "YES" : "NO"}`);
+        console.log(
+          `  Failure Trace Added : ${
+            res.diagnosticsFound > 0
+              ? `YES (${res.diagnosticsFound} block${res.diagnosticsFound === 1 ? "" : "s"} from ${[...new Set(res.diagnosticSources || [])].join(", ")})`
+              : "NO — the session carried no readable diagnostics; the retry got the generic sentence"
+          }`
+        );
         console.log(`------------------------------------------------------------------\n`);
       }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "jules-orchestrator-kit",
-  "version": "0.69.0",
+  "version": "0.70.0",
   "description": "Zero-dependency safety gatekeeper, test oracle generator, and multi-agent coordination protocol for autonomous coding agents — Google Jules, Claude Code, Codex and Gemini CLI.",
   "repository": {
     "type": "git",

package/src/engine.mjs

@@ -894,19 +894,49 @@ export async function repair(failure, opts = {}) {
       break;
     }
 
-    attempts.push({ n, session, ok: true });
-    appendTelemetry(root, "ooda_repair_attempt", { attempt: n, ok: true });
-
     // Poll async provider for terminal session state before executing re-verification gates
+    let pollVerdict = null;
     if (provider && session) {
-      await pollSessionState(provider, session, {
+      pollVerdict = await pollSessionState(provider, session, {
         root,
         dryRun: opts.dryRun,
         pollIntervalMs: opts.pollIntervalMs,
         maxPollAttempts: opts.maxPollAttempts,
       });
+
+      // The gate below is the authority on whether the change works, so it runs
+      // either way. But a non-terminal session means it is about to judge a
+      // tree the agent may not have finished writing, and saying nothing here
+      // turns that into a confusing cascade of repair attempts against a
+      // half-applied patch.
+      if (pollVerdict && pollVerdict.terminal !== true) {
+        const why = pollVerdict.blockedOn
+          ? `waiting on an actor (${pollVerdict.blockedOn})`
+          : pollVerdict.unreachable
+            ? "the provider stopped answering"
+            : `still ${pollVerdict.status} when the poll budget ran out`;
+        console.warn(
+          `[SESSION_NOT_TERMINAL] Session ${session.id} is ${why}. Re-verification is running against a tree the agent may not have finished writing.`
+        );
+        appendTelemetry(root, "session_not_terminal", {
+          attempt: n,
+          sessionId: session.id,
+          status: pollVerdict.status,
+          blockedOn: pollVerdict.blockedOn || null,
+          timedOut: Boolean(pollVerdict.timedOut),
+          unreachable: Boolean(pollVerdict.unreachable),
+        });
+      }
     }
 
+    attempts.push({
+      n,
+      session,
+      ok: true,
+      poll: pollVerdict ? { status: pollVerdict.status, terminal: pollVerdict.terminal } : null,
+    });
+    appendTelemetry(root, "ooda_repair_attempt", { attempt: n, ok: true });
+
     // Re-verify after repair attempt
     const gateRes = await gate({ root, config, fix: false, progressBus, progressToken });
     if (gateRes.ok) {
@@ -1000,10 +1030,38 @@ export async function checkTaskPremise(task = {}, opts = {}) {
 }
 
 /**
- * Polls the provider until the session terminates in COMPLETED, FAILED, or reaches timeout.
+ * Session states the API documents as final. The full `SessionState` enum is
+ * transcribed in `docs/jules-quality-plan.md`.
+ */
+export const TERMINAL_SESSION_STATES = new Set(["COMPLETED", "FAILED"]);
+
+/**
+ * Session states that cannot advance without an actor — a human approving a
+ * plan, a human answering a question, or whoever paused it resuming it.
+ *
+ * Polling these is not waiting, it is spending the budget: the state cannot
+ * change on its own no matter how long the loop runs.
+ */
+export const BLOCKING_SESSION_STATES = new Set(["AWAITING_PLAN_APPROVAL", "AWAITING_USER_FEEDBACK", "PAUSED"]);
+
+/**
+ * Polls the provider until the session reaches a terminal state, blocks on an
+ * actor, or the poll budget runs out.
+ *
+ * The verdict is explicit about which of those three happened, because the
+ * caller runs the verification gate on the assumption that the agent has
+ * finished writing. A previous version returned `COMPLETED` for every
+ * non-terminal exit — a session sitting in `AWAITING_USER_FEEDBACK` and one
+ * still `IN_PROGRESS` when the budget expired were both reported as success,
+ * which is the same shape this project exists to refuse.
+ *
+ * @returns {Promise<object>} Always carries `status` and `terminal`. Non-terminal
+ *   exits additionally carry one of `blockedOn`, `timedOut` or `unreachable`.
+ *   `status` is never synthesised: it is the last state the provider reported,
+ *   or `UNKNOWN` when it never answered.
  */
 export async function pollSessionState(provider, session, opts = {}) {
-  if (!session || !session.id) return { status: "COMPLETED" };
+  if (!session || !session.id) return { status: "UNKNOWN", terminal: false, unpolled: true };
   const initialStatus = String(session.status || session.state || "").toUpperCase();
   if (
     initialStatus === "COMPLETED" ||
@@ -1013,13 +1071,20 @@ export async function pollSessionState(provider, session, opts = {}) {
     session.id.startsWith("mock-") ||
     session.id.startsWith("dry-run-")
   ) {
-    return { status: initialStatus || "COMPLETED" };
+    // A dry run has no session to watch, so it reports the outcome a real
+    // dispatch would have had to earn. Flagged as simulated so a caller can
+    // tell the two apart; a terminal state already reported by the provider is
+    // a fact, not a simulation, and is returned as itself.
+    const status = initialStatus || "COMPLETED";
+    return { status, terminal: TERMINAL_SESSION_STATES.has(status), simulated: true, polls: 0 };
   }
 
   const maxAttempts = opts.maxPollAttempts || 30;
   const pollIntervalMs = opts.pollIntervalMs || 1000;
   const timeoutMs = opts.pollTimeoutMs || 300000;
   const startTime = Date.now();
+  let lastStatus = "";
+  let polls = 0;
 
   for (let attempt = 0; attempt < maxAttempts; attempt++) {
     if (Date.now() - startTime > timeoutMs) break;
@@ -1035,30 +1100,59 @@ export async function pollSessionState(provider, session, opts = {}) {
       } catch (_) {}
     }
 
-    if (currentSession) {
-      const status = String(currentSession.status || currentSession.state || "").toUpperCase();
-      if (
-        (status === "AWAITING_PLAN_APPROVAL" || status === "PENDING_APPROVAL") &&
-        (opts.autoApprovePlan || opts.autoApprove || session.autoApprovePlan)
-      ) {
-        if (provider && typeof provider.approvePlan === "function") {
-          try {
-            await provider.approvePlan(session.id, opts);
-          } catch (_) {}
+    if (!currentSession) {
+      return {
+        ...session,
+        status: lastStatus || "UNKNOWN",
+        terminal: false,
+        unreachable: true,
+        polls,
+      };
+    }
+
+    polls += 1;
+    const status = String(currentSession.status || currentSession.state || "").toUpperCase();
+    if (status) lastStatus = status;
+
+    if (TERMINAL_SESSION_STATES.has(status)) {
+      return { ...currentSession, status, terminal: true, polls };
+    }
+
+    if (BLOCKING_SESSION_STATES.has(status)) {
+      // A pending plan approval is the one blocking state this loop is allowed
+      // to resolve itself, and only when the caller said so.
+      const isPlanApproval = status === "AWAITING_PLAN_APPROVAL";
+      const wantsAutoApprove = Boolean(opts.autoApprovePlan || opts.autoApprove || session.autoApprovePlan);
+      if (isPlanApproval && wantsAutoApprove && provider && typeof provider.approvePlan === "function") {
+        try {
+          await provider.approvePlan(session.id, opts);
+          await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
+          continue;
+        } catch (err) {
+          return {
+            ...currentSession,
+            status,
+            terminal: false,
+            blockedOn: status,
+            approvePlanError: err && err.message ? err.message : String(err),
+            polls,
+          };
         }
       }
 
-      if (status === "COMPLETED" || status === "FAILED") {
-        return { ...currentSession, status };
-      }
-    } else {
-      break;
+      return { ...currentSession, status, terminal: false, blockedOn: status, polls };
     }
 
     await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
   }
 
-  return { ...session, status: String(session.status || "COMPLETED").toUpperCase() };
+  return {
+    ...session,
+    status: lastStatus || "UNKNOWN",
+    terminal: false,
+    timedOut: true,
+    polls,
+  };
 }
 
 function buildRepairPrompt(failure, attempt, _config, extraPromptDirective = null) {

package/src/session-ops.mjs

@@ -31,6 +31,106 @@ export function parseAgeDuration(duration) {
   }
 }
 
+/**
+ * Output shapes that mean "this command failed" even when the runner exited 0.
+ *
+ * Kept deliberately narrow: this list is only consulted for a `bashOutput`
+ * whose `exitCode` is `0` or absent, where the alternative is reporting
+ * nothing at all. A false positive here costs a few hundred characters of
+ * retry prompt; a false negative costs the retry its only evidence.
+ */
+const BASH_FAILURE_HINTS = [
+  /(^|\n)\s*not ok\b/i,
+  /(^|\n)# fail [1-9]/i,
+  /\b[1-9]\d* failed\b/i,
+  /\b[1-9]\d* failing\b/i,
+  /\bAssertionError\b/,
+  /\bFAILED\b/,
+  /\bTraceback \(most recent call last\)/,
+  /\bpanic: /,
+  /\berror:/i,
+];
+
+/**
+ * Collects the diagnostics a session actually carries.
+ *
+ * The documented Activity type — the Jules API types reference, transcribed
+ * in `docs/jules-quality-plan.md` — puts command output under
+ * `artifacts[].bashOutput.{command,output,exitCode}` and the failure reason
+ * under `sessionFailed.reason`. Neither `act.error` nor `act.executionOutput`
+ * — the only two fields this file used to read — exists in that schema, so a
+ * real failure came back as the generic fallback sentence and the retry session
+ * was dispatched without the assertion it existed to fix.
+ *
+ * Blocks are returned highest-signal first, because `retrySession` truncates
+ * the joined result to 4000 characters from the front: the ordering decides
+ * which evidence survives the cut.
+ *
+ * The legacy spellings are still read. They cost nothing, and an unrecognised
+ * provider shape that does carry an error is better served by it than by the
+ * fallback sentence.
+ *
+ * @param {Array<object>} activities - Activities as returned by `listActivities`.
+ * @returns {Array<{ source: string, text: string }>} Highest-signal first.
+ */
+export function extractFailureDiagnostics(activities) {
+  const list = Array.isArray(activities) ? activities : [];
+  const failingBash = [];
+  const failureReasons = [];
+  const suspiciousBash = [];
+  const legacy = [];
+  const agentNotes = [];
+  const seen = new Set();
+
+  const push = (bucket, source, text) => {
+    const clean = String(text ?? "").trim();
+    if (!clean) return;
+    const key = `${source}\u0000${clean}`;
+    if (seen.has(key)) return;
+    seen.add(key);
+    bucket.push({ source, text: clean });
+  };
+
+  for (const act of list) {
+    if (!act || typeof act !== "object") continue;
+
+    const artifacts = Array.isArray(act.artifacts) ? act.artifacts : [];
+    for (const art of artifacts) {
+      const bash = art && typeof art === "object" ? art.bashOutput : null;
+      if (!bash || typeof bash !== "object") continue;
+      const command = typeof bash.command === "string" ? bash.command : "";
+      const output = typeof bash.output === "string" ? bash.output : "";
+      if (!command && !output) continue;
+      const rendered = `$ ${command || "(unknown command)"}\n${output || "(no output)"}`;
+      const exitCode = bash.exitCode;
+      if (typeof exitCode === "number" && exitCode !== 0) {
+        push(failingBash, "bashOutput", `${rendered}\n(exit code ${exitCode})`);
+      } else if (BASH_FAILURE_HINTS.some((re) => re.test(output))) {
+        const codeNote = typeof exitCode === "number" ? String(exitCode) : "not reported";
+        push(suspiciousBash, "bashOutput", `${rendered}\n(exit code ${codeNote})`);
+      }
+    }
+
+    const failed = act.sessionFailed && typeof act.sessionFailed === "object" ? act.sessionFailed : null;
+    if (failed && typeof failed.reason === "string") {
+      push(failureReasons, "sessionFailed", failed.reason);
+    }
+
+    if (act.error) {
+      push(legacy, "legacy.error", typeof act.error === "string" ? act.error : JSON.stringify(act.error));
+    }
+    if (act.executionOutput && (act.exitCode !== 0 || act.status === "FAILED")) {
+      push(legacy, "legacy.executionOutput", act.executionOutput);
+    }
+
+    if (act.originator === "agent" && typeof act.description === "string") {
+      push(agentNotes, "agentMessage", act.description);
+    }
+  }
+
+  return [...failingBash, ...failureReasons, ...suspiciousBash, ...legacy, ...agentNotes];
+}
+
 /**
  * Extracts git diff patch, PR details, and modified files from a Jules session.
  *
@@ -213,7 +313,7 @@ export async function applySessionPatch(sessionId, opts = {}) {
  *
  * @param {string} sessionId
  * @param {object} opts
- * @returns {Promise<{ ok: boolean, originalSessionId: string, newSession: object, failureReason: string }>}
+ * @returns {Promise<{ ok: boolean, originalSessionId: string, newSession: object, failureReason: string, diagnosticsFound: number, diagnosticSources: string[] }>}
  */
 export async function retrySession(sessionId, opts = {}) {
   if (!sessionId || typeof sessionId !== "string") {
@@ -227,18 +327,19 @@ export async function retrySession(sessionId, opts = {}) {
   const activitiesRes = await provider.listActivities(sessionId, opts).catch(() => ({ activities: [] }));
   const activities = activitiesRes.activities || [];
 
-  // Extract failure diagnostics
-  const errors = [];
-  for (const act of activities) {
-    if (act.error) errors.push(typeof act.error === "string" ? act.error : JSON.stringify(act.error));
-    if (act.executionOutput && (act.exitCode !== 0 || act.status === "FAILED")) {
-      errors.push(act.executionOutput);
-    }
-  }
+  // Extract failure diagnostics. `extractFailureDiagnostics` reads the fields
+  // the API actually documents (artifacts[].bashOutput, sessionFailed.reason)
+  // and returns them highest-signal first, so the 4000-character cut below
+  // drops the least useful evidence rather than the assertion that failed.
+  const diagnostics = extractFailureDiagnostics(activities);
 
   const rawPrompt = session?.raw?.prompt || session?.prompt || "";
   const title = session?.raw?.title || `Retry of Session ${sessionId}`;
-  const failureReason = errors.join("\n").slice(0, 4000) || "Previous session did not complete cleanly.";
+  const failureReason =
+    diagnostics
+      .map((d) => d.text)
+      .join("\n")
+      .slice(0, 4000) || "Previous session did not complete cleanly.";
 
   let synthesizedPrompt = rawPrompt;
   if (opts.withFailure !== false && failureReason) {
@@ -264,6 +365,12 @@ export async function retrySession(sessionId, opts = {}) {
     originalSessionId: sessionId,
     newSession: dispatchRes,
     failureReason,
+    // Non-zero only when the session carried evidence the retry could act on.
+    // A zero here with a non-empty `failureReason` means the fallback sentence
+    // was sent — the distinction the CLI and any telemetry need, because the
+    // two look identical in `failureReason` alone.
+    diagnosticsFound: diagnostics.length,
+    diagnosticSources: diagnostics.map((d) => d.source),
   };
 }