taskchef 5.1.0 → 5.2.0

package/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "taskchef",
-  "version": "5.1.0",
+  "version": "5.2.0",
   "description": "Dispatch work from a data-only workspace to visible Codex project tasks.",
   "author": {
     "name": "Favo Yang",

package/README.md

@@ -241,6 +241,17 @@ loads and validates configured projects, and returns a generated task UUID,
 preparation timestamp, and exact correlation marker. `task record` accepts one
 JSON value only from closed, non-interactive standard input.
 
+For repeatable live delegation measurements, run
+`npm run benchmark:e2e -- write`. It reads one non-interactive JSON value,
+validates a stable schema, derives durations and summary totals, and writes a
+timestamped result under the supplied output directory (default:
+`reports/e2e-benchmarks`). Use `validate <file>` to verify a saved result and
+`clean [directory]` to remove only prior TaskChef end-to-end result JSON files.
+Start from `assets/e2e-benchmark-example.json`; the writer accepts strict
+schema fields only and supports stopped workflows after preparation, creation,
+or recording failures. The writer stamps `taskchefVersion` from its own package;
+saved files retain that version for historical validation.
+
 ### One-time upgrade from an older workspace
 
 TaskChef 5 does not include a general migration command. For a one-time upgrade,

package/SPEC.md

@@ -196,10 +196,12 @@ For each assignment, `$taskchef-delegate`:
    marked entry with `threadId: null`, then prefers one native client-ID wait or
    resolution call with a 30-second timeout when Codex exposes one
 7. when no native operation is available, takes at most two recent-thread
-   snapshots near 10 and 30 seconds after the provisional result, filters
-   candidates by available host/project/time/worktree metadata, uses title only
-   as an advisory ordering hint, and accepts only one thread whose structured
-   delegated input starts with the exact marker
+   snapshots on a nominal 10- and 30-second schedule after the provisional
+   result; a late first checkpoint runs immediately, and the second starts at
+   least 20 seconds after the first actually started. It filters candidates by
+   available host/project/time/worktree metadata, uses title only as an advisory
+   ordering hint, and accepts only one thread whose structured delegated input
+   starts with the exact marker
 8. atomically fills the nullable thread ID after an exact match
 9. returns after recording or after reporting that bounded resolution was
    unresolved, without waiting for executor work completion.
@@ -209,11 +211,13 @@ Creation-time filtering allows five seconds of clock skew. Candidate reads run
 concurrently where the host permits and inspect only structured
 `codexDelegation.input`, never untrusted title, summary, preview, or plain-text
 marker echoes. A native resolver result is verified against the same structured
-marker before persistence. Zero exact matches time out unresolved; multiple
-exact matches are ambiguous. Snapshot, candidate-read, native-resolution, or
-task-resolution errors leave the already-recorded nullable entry intact. No
-snapshot, candidate read, marker verification, or task-resolution write starts
-after the 30-second deadline, so tool latency can reduce the number of attempts.
+marker before persistence. Zero exact matches remain unresolved; multiple exact
+matches are ambiguous. Snapshot, candidate-read, native-resolution, or
+task-resolution errors leave the already-recorded nullable entry intact. The
+fallback is bounded by two snapshots rather than an absolute wall-clock cutoff:
+mandatory recording or tool latency can shift both attempts later, but cannot
+erase an attempt or reduce the minimum 20-second interval between their start
+times.
 A `clientThreadId`, `pendingWorktreeId`, or ID in the documented provisional
 `local:` namespace remains diagnostic context and is rejected from every path
 that could persist the canonical `threadId` field.
@@ -224,6 +228,51 @@ helpers with injected thread-tool callbacks, while the skill owns the actual
 desktop-tool calls and the CLI remains responsible only for validated data
 operations.
 
+### End-to-end benchmark results
+
+An explicitly requested live benchmark writes one ignored, timestamped JSON
+artifact through `scripts/e2e-benchmark.js`. Schema version 1 contains:
+
+- benchmark name, TaskChef version, run ID, and wall-clock start/end
+- project, title, and exact unmarked workload prompt
+- TaskChef task ID, nullable durable/client IDs, recording state, resolution,
+  and attempt count
+- ISO start/end, derived duration, and outcome for preparation/project listing,
+  task creation, recording, and optional provisional resolution
+- boolean record, marker, executor-output, and candidate-filter-effectiveness
+  validation results
+- derived total wall time, measured stage time, orchestration overhead,
+  resolution state, and attempt count
+
+Stage outcomes are `success|failed` for preparation,
+`durable|provisional|failed` for creation, `recorded|failed` for recording, and
+`native|discovered|unresolved|failed` for provisional resolution. `failed`
+distinguishes a terminal resolver error from a completed no-match checkpoint,
+which must proceed to the second fallback attempt. Task resolution is
+`immediate|native|discovered|unresolved`. Preparation failure has only the
+preparation stage and a null task ID; creation failure stops after creation;
+recording failure stops before provisional resolution. The committed example
+shows the durable success shape, while these transition rules define the
+shorter failure and longer provisional shapes.
+The writer derives `taskchefVersion` from the running package instead of
+trusting template input. Raw executor output and transcripts are never stored;
+`outputVerified` records only whether the requested output was verified.
+
+The writer rejects missing required stages, duplicate or unknown stages,
+invalid timestamps, contradictory workflow outcomes, overlapping stages,
+unknown fields, and overwriting an existing run. Later stages are omitted when
+preparation or creation fails, and provisional resolution is omitted when task
+recording fails. Fallback snapshot observations include the complete recent
+task count plus the filtered candidate and exact-match counts, from which
+candidate-filter effectiveness is checked. Each snapshot records
+`resolveWriteOutcome` as `not-attempted`, `succeeded`, or `failed`, so a unique
+match followed by an atomic write failure remains an honest unresolved result.
+A compact valid starting document is committed as
+`assets/e2e-benchmark-example.json`. Filenames use
+`<startedAt-with-colons-replaced>-taskchef-delegate-e2e.json`. Its cleanup
+operation removes only files matching the full generated filename grammar in
+the selected results directory.
+
 `task record` rejects an interactive TTY before reading because its protocol is
 exactly one JSON value followed by EOF. Workspace-lock contention is retried for
 up to seven seconds, while permanent permission failures such as `EPERM` or

package/assets/e2e-benchmark-example.json

@@ -0,0 +1,46 @@
+{
+  "schemaVersion": 1,
+  "benchmark": "taskchef-delegate-e2e",
+  "runId": "replace-with-run-id",
+  "startedAt": "2026-08-13T06:30:00.000Z",
+  "completedAt": "2026-08-13T06:30:05.000Z",
+  "workload": {
+    "project": "example-project",
+    "title": "Example workload",
+    "prompt": "Return the requested example output."
+  },
+  "task": {
+    "taskId": "00000000-0000-4000-8000-000000000000",
+    "threadId": "00000000-0000-7000-8000-000000000000",
+    "clientThreadId": null,
+    "recorded": true,
+    "resolution": "immediate",
+    "resolutionAttempts": 0
+  },
+  "stages": [
+    {
+      "name": "prepare-and-list-projects",
+      "startedAt": "2026-08-13T06:30:00.000Z",
+      "completedAt": "2026-08-13T06:30:00.200Z",
+      "outcome": "success"
+    },
+    {
+      "name": "create-thread",
+      "startedAt": "2026-08-13T06:30:01.000Z",
+      "completedAt": "2026-08-13T06:30:01.100Z",
+      "outcome": "durable"
+    },
+    {
+      "name": "record-task",
+      "startedAt": "2026-08-13T06:30:02.000Z",
+      "completedAt": "2026-08-13T06:30:04.000Z",
+      "outcome": "recorded"
+    }
+  ],
+  "validation": {
+    "recordVerified": true,
+    "markerVerified": true,
+    "outputVerified": false,
+    "candidateFilterEffective": false
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "taskchef",
-  "version": "5.1.0",
+  "version": "5.2.0",
   "description": "A non-blocking interactive dispatcher for visible Codex tasks.",
   "license": "MIT",
   "author": "Favo Yang",
@@ -25,6 +25,7 @@
     "index.js",
     "SPEC.md",
     "scripts/benchmark-dispatch-prepare.js",
+    "scripts/e2e-benchmark.js",
     "src",
     "skills/taskchef-bootstrap",
     "skills/taskchef-delegate",
@@ -39,6 +40,7 @@
   },
   "scripts": {
     "benchmark:dispatch": "node scripts/benchmark-dispatch-prepare.js",
+    "benchmark:e2e": "node scripts/e2e-benchmark.js",
     "test": "node --test tests/taskchef.test.js"
   },
   "dependencies": {

package/scripts/e2e-benchmark.js

@@ -0,0 +1,517 @@
+#!/usr/bin/env node
+
+import { mkdir, readFile, readdir, unlink, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const BENCHMARK_NAME = "taskchef-delegate-e2e";
+const STAGE_NAMES = [
+  "prepare-and-list-projects",
+  "create-thread",
+  "record-task",
+  "resolve-provisional",
+];
+const STAGE_OUTCOMES = Object.freeze({
+  "prepare-and-list-projects": new Set(["success", "failed"]),
+  "create-thread": new Set(["durable", "provisional", "failed"]),
+  "record-task": new Set(["recorded", "failed"]),
+  "resolve-provisional": new Set(["native", "discovered", "unresolved", "failed"]),
+});
+const RESOLUTIONS = new Set(["immediate", "native", "discovered", "unresolved"]);
+const RESULT_FILENAME_PATTERN = /^\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}\.\d{3}Z-taskchef-delegate-e2e\.json$/;
+const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
+const SEMVER_PATTERN = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[A-Za-z-][0-9A-Za-z-]*)(?:\.(?:0|[1-9]\d*|\d*[A-Za-z-][0-9A-Za-z-]*))*))?(?:\+([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?$/;
+
+function requireExactKeys(value, allowed, name) {
+  const unknown = Object.keys(value).filter((key) => !allowed.includes(key));
+  if (unknown.length > 0) throw new Error(`${name} has unknown field ${unknown[0]}`);
+}
+
+function requireObject(value, name) {
+  if (!value || typeof value !== "object" || Array.isArray(value)) {
+    throw new Error(`${name} must be an object`);
+  }
+  return value;
+}
+
+function requireString(value, name) {
+  if (typeof value !== "string" || value.trim().length === 0) {
+    throw new Error(`${name} must be a non-empty string`);
+  }
+  return value;
+}
+
+function timestamp(value, name) {
+  requireString(value, name);
+  if (!/^\d{4}-\d{2}-\d{2}T/.test(value)) {
+    throw new Error(`${name} must use a four-digit UTC year`);
+  }
+  const milliseconds = Date.parse(value);
+  if (!Number.isFinite(milliseconds) || new Date(milliseconds).toISOString() !== value) {
+    throw new Error(`${name} must be a canonical UTC ISO timestamp`);
+  }
+  return milliseconds;
+}
+
+function nullableString(value, name) {
+  if (value === null) return null;
+  return requireString(value, name);
+}
+
+function managedResultFilename(name) {
+  if (!RESULT_FILENAME_PATTERN.test(name)) return false;
+  const timestampPart = name.slice(0, -`-${BENCHMARK_NAME}.json`.length);
+  const iso = `${timestampPart.slice(0, 13)}:${timestampPart.slice(14, 16)}:${timestampPart.slice(17)}`;
+  try {
+    timestamp(iso, "result filename timestamp");
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+function requireNonnegativeNumber(value, name) {
+  if (typeof value !== "number" || !Number.isFinite(value) || value < 0) {
+    throw new Error(`${name} must be a nonnegative number`);
+  }
+  return value;
+}
+
+function normalizeObservations(
+  input,
+  startedMs,
+  completedMs,
+  preparationStage,
+  creationStage,
+  recordingStage,
+  resolutionStage,
+  resolutionAttempts,
+) {
+  if (input === undefined) return undefined;
+  const observations = requireObject(input, "observations");
+  requireExactKeys(observations, [
+    "preparation",
+    "resolutionSnapshots",
+    "validationDurationMs",
+    "candidateMetadata",
+  ], "observations");
+  const result = {};
+  if (observations.preparation !== undefined) {
+    const preparation = requireObject(observations.preparation, "observations.preparation");
+    requireExactKeys(preparation, ["dispatchPrepareMs", "nativeProjectListMs"], "observations.preparation");
+    result.preparation = {
+      dispatchPrepareMs: requireNonnegativeNumber(preparation.dispatchPrepareMs, "observations.preparation.dispatchPrepareMs"),
+      nativeProjectListMs: requireNonnegativeNumber(preparation.nativeProjectListMs, "observations.preparation.nativeProjectListMs"),
+    };
+    const preparationDurationMs = timestamp(preparationStage.completedAt, "prepare-and-list-projects.completedAt") -
+      timestamp(preparationStage.startedAt, "prepare-and-list-projects.startedAt");
+    if (Object.values(result.preparation).some((durationMs) => durationMs > preparationDurationMs)) {
+      throw new Error("observations.preparation durations must fit within the preparation stage");
+    }
+  }
+  if (observations.resolutionSnapshots !== undefined) {
+    if (!Array.isArray(observations.resolutionSnapshots)) {
+      throw new Error("observations.resolutionSnapshots must be an array");
+    }
+    let previousSnapshotStartedMs = null;
+    let previousSnapshotCompletedMs = null;
+    result.resolutionSnapshots = observations.resolutionSnapshots.map((snapshot, index) => {
+      requireObject(snapshot, `observations.resolutionSnapshots[${index}]`);
+      requireExactKeys(snapshot, [
+        "attempt", "startedAt", "completedAt", "recentTaskCount", "candidateCount", "exactMatchCount",
+        "resolveWriteMs", "resolveWriteOutcome",
+      ], `observations.resolutionSnapshots[${index}]`);
+      if (!Number.isInteger(snapshot.attempt) || snapshot.attempt !== index + 1 || snapshot.attempt > resolutionAttempts) {
+        throw new Error(`observations.resolutionSnapshots[${index}].attempt must be sequential and within resolutionAttempts`);
+      }
+      const snapshotStartedMs = timestamp(snapshot.startedAt, `observations.resolutionSnapshots[${index}].startedAt`);
+      const snapshotCompletedMs = timestamp(snapshot.completedAt, `observations.resolutionSnapshots[${index}].completedAt`);
+      const resolutionStartedMs = resolutionStage ? timestamp(resolutionStage.startedAt, "resolve-provisional.startedAt") : null;
+      const resolutionCompletedMs = resolutionStage ? timestamp(resolutionStage.completedAt, "resolve-provisional.completedAt") : null;
+      if (
+        resolutionStartedMs === null || snapshotStartedMs < resolutionStartedMs ||
+        snapshotCompletedMs < snapshotStartedMs || snapshotCompletedMs > resolutionCompletedMs ||
+        snapshotStartedMs < startedMs || snapshotCompletedMs > completedMs
+      ) {
+        throw new Error(`observations.resolutionSnapshots[${index}] must fall within the resolution stage`);
+      }
+      const firstCheckpointMs = Math.max(
+        timestamp(creationStage.completedAt, "create-thread.completedAt") + 10_000,
+        timestamp(recordingStage.completedAt, "record-task.completedAt"),
+      );
+      if (index === 0 && snapshotStartedMs < firstCheckpointMs) {
+        throw new Error("the first resolution snapshot must respect the 10-second catch-up checkpoint");
+      }
+      if (previousSnapshotCompletedMs !== null && snapshotStartedMs < previousSnapshotCompletedMs) {
+        throw new Error("resolutionSnapshots must be ordered and non-overlapping");
+      }
+      if (previousSnapshotStartedMs !== null && snapshotStartedMs - previousSnapshotStartedMs < 20_000) {
+        throw new Error("resolutionSnapshots must start at least 20 seconds apart");
+      }
+      for (const name of ["recentTaskCount", "candidateCount", "exactMatchCount", "resolveWriteMs"]) {
+        requireNonnegativeNumber(snapshot[name], `observations.resolutionSnapshots[${index}].${name}`);
+      }
+      if (!["not-attempted", "succeeded", "failed"].includes(snapshot.resolveWriteOutcome)) {
+        throw new Error(`observations.resolutionSnapshots[${index}].resolveWriteOutcome is invalid`);
+      }
+      if ((snapshot.exactMatchCount === 1) !== (snapshot.resolveWriteOutcome !== "not-attempted")) {
+        throw new Error(`observations.resolutionSnapshots[${index}] resolve write must agree with exact matches`);
+      }
+      if (snapshot.resolveWriteOutcome === "not-attempted" && snapshot.resolveWriteMs !== 0) {
+        throw new Error(`observations.resolutionSnapshots[${index}].resolveWriteMs must be zero when no write was attempted`);
+      }
+      if (!Number.isInteger(snapshot.recentTaskCount) || !Number.isInteger(snapshot.candidateCount) || !Number.isInteger(snapshot.exactMatchCount)) {
+        throw new Error(`observations.resolutionSnapshots[${index}] counts must be integers`);
+      }
+      if (snapshot.recentTaskCount > 50) {
+        throw new Error(`observations.resolutionSnapshots[${index}].recentTaskCount must not exceed 50`);
+      }
+      if (snapshot.candidateCount > snapshot.recentTaskCount || snapshot.exactMatchCount > snapshot.candidateCount) {
+        throw new Error(`observations.resolutionSnapshots[${index}] counts are inconsistent`);
+      }
+      if (snapshot.resolveWriteMs > snapshotCompletedMs - snapshotStartedMs) {
+        throw new Error(`observations.resolutionSnapshots[${index}].resolveWriteMs exceeds snapshot duration`);
+      }
+      previousSnapshotStartedMs = snapshotStartedMs;
+      previousSnapshotCompletedMs = snapshotCompletedMs;
+      return { ...snapshot };
+    });
+  }
+  if (observations.validationDurationMs !== undefined) {
+    result.validationDurationMs = requireNonnegativeNumber(observations.validationDurationMs, "observations.validationDurationMs");
+    const finalStage = resolutionStage ?? recordingStage ?? creationStage ?? preparationStage;
+    const postWorkflowMs = completedMs - timestamp(finalStage.completedAt, `${finalStage.name}.completedAt`);
+    if (result.validationDurationMs > postWorkflowMs) {
+      throw new Error("observations.validationDurationMs must fit after the workflow stages");
+    }
+  }
+  if (observations.candidateMetadata !== undefined) {
+    const metadata = requireObject(observations.candidateMetadata, "observations.candidateMetadata");
+    const names = ["targetProjectIdPresent", "targetCreatedAtPresent", "targetEnvironmentPresent", "targetCwdPresent"];
+    requireExactKeys(metadata, names, "observations.candidateMetadata");
+    for (const name of names) {
+      if (typeof metadata[name] !== "boolean") throw new Error(`observations.candidateMetadata.${name} must be boolean`);
+    }
+    result.candidateMetadata = { ...metadata };
+  }
+  return result;
+}
+
+export function normalizeBenchmarkResult(input, { requireDerived = false } = {}) {
+  const value = structuredClone(requireObject(input, "benchmark result"));
+  requireExactKeys(value, [
+    "schemaVersion", "benchmark", "taskchefVersion", "runId", "startedAt", "completedAt",
+    "workload", "task", "stages", "validation", "observations", "summary",
+  ], "benchmark result");
+  if (value.schemaVersion !== 1) throw new Error("schemaVersion must be 1");
+  if (value.benchmark !== BENCHMARK_NAME) {
+    throw new Error(`benchmark must be ${BENCHMARK_NAME}`);
+  }
+  requireString(value.taskchefVersion, "taskchefVersion");
+  if (!SEMVER_PATTERN.test(value.taskchefVersion)) throw new Error("taskchefVersion must be a semantic version");
+  requireString(value.runId, "runId");
+  const startedMs = timestamp(value.startedAt, "startedAt");
+  const completedMs = timestamp(value.completedAt, "completedAt");
+  if (completedMs < startedMs) throw new Error("completedAt must not precede startedAt");
+
+  const workload = requireObject(value.workload, "workload");
+  requireExactKeys(workload, ["project", "title", "prompt"], "workload");
+  requireString(workload.project, "workload.project");
+  requireString(workload.title, "workload.title");
+  requireString(workload.prompt, "workload.prompt");
+  if (/<!--\s*taskchef_id=/i.test(workload.prompt)) {
+    throw new Error("workload.prompt must not contain a TaskChef marker");
+  }
+
+  const task = requireObject(value.task, "task");
+  requireExactKeys(task, [
+    "taskId", "threadId", "clientThreadId", "recorded", "resolution", "resolutionAttempts",
+  ], "task");
+  task.taskId = nullableString(task.taskId, "task.taskId");
+  if (task.taskId !== null && !UUID_PATTERN.test(task.taskId)) {
+    throw new Error("task.taskId must be a lowercase UUID");
+  }
+  task.threadId = nullableString(task.threadId, "task.threadId");
+  task.clientThreadId = nullableString(task.clientThreadId, "task.clientThreadId");
+  if (task.threadId !== null && task.threadId.trim().toLowerCase().startsWith("local:")) {
+    throw new Error("task.threadId must not use the provisional local: namespace");
+  }
+  if (task.threadId !== null && task.threadId === task.clientThreadId) {
+    throw new Error("task.threadId must differ from task.clientThreadId");
+  }
+  if (typeof task.recorded !== "boolean") throw new Error("task.recorded must be boolean");
+  requireString(task.resolution, "task.resolution");
+  if (!RESOLUTIONS.has(task.resolution)) throw new Error("task.resolution is invalid");
+  if (!Number.isInteger(task.resolutionAttempts) || task.resolutionAttempts < 0) {
+    throw new Error("task.resolutionAttempts must be a nonnegative integer");
+  }
+  if (task.resolutionAttempts > 2) throw new Error("task.resolutionAttempts must not exceed 2");
+
+  if (!Array.isArray(value.stages)) throw new Error("stages must be an array");
+  const stagesByName = new Map();
+  let previousCompletedMs = startedMs;
+  for (const [index, stage] of value.stages.entries()) {
+    requireObject(stage, `stages[${index}]`);
+    requireExactKeys(stage, ["name", "startedAt", "completedAt", "outcome", "durationMs"], `stages[${index}]`);
+    requireString(stage.name, `stages[${index}].name`);
+    if (!STAGE_NAMES.includes(stage.name)) throw new Error(`unknown stage ${stage.name}`);
+    if (stagesByName.has(stage.name)) throw new Error(`duplicate stage ${stage.name}`);
+    const stageStartedMs = timestamp(stage.startedAt, `${stage.name}.startedAt`);
+    const stageCompletedMs = timestamp(stage.completedAt, `${stage.name}.completedAt`);
+    if (stageCompletedMs < stageStartedMs) {
+      throw new Error(`${stage.name}.completedAt must not precede startedAt`);
+    }
+    if (stageStartedMs < startedMs || stageCompletedMs > completedMs) {
+      throw new Error(`${stage.name} must fall within the benchmark run`);
+    }
+    if (stageStartedMs < previousCompletedMs) throw new Error("stages must be ordered and non-overlapping");
+    requireString(stage.outcome, `${stage.name}.outcome`);
+    if (!STAGE_OUTCOMES[stage.name].has(stage.outcome)) {
+      throw new Error(`${stage.name}.outcome is invalid`);
+    }
+    const durationMs = stageCompletedMs - stageStartedMs;
+    if (requireDerived && stage.durationMs === undefined) {
+      throw new Error(`${stage.name}.durationMs is required in a saved result`);
+    }
+    if (stage.durationMs !== undefined && stage.durationMs !== durationMs) {
+      throw new Error(`${stage.name}.durationMs does not match its timestamps`);
+    }
+    stage.durationMs = durationMs;
+    stagesByName.set(stage.name, stage);
+    previousCompletedMs = stageCompletedMs;
+  }
+  if (!stagesByName.has(STAGE_NAMES[0])) throw new Error(`missing required stage ${STAGE_NAMES[0]}`);
+  const resolutionStagePresent = stagesByName.has("resolve-provisional");
+  if (task.resolutionAttempts > 0 && !resolutionStagePresent) {
+    throw new Error("resolutionAttempts must agree with the resolve-provisional stage");
+  }
+  if (task.resolutionAttempts === 0 && resolutionStagePresent) {
+    throw new Error("zero resolutionAttempts must omit the resolve-provisional stage");
+  }
+
+  if (!value.stages.every((stage, index) => stage.name === STAGE_NAMES[index])) {
+    throw new Error("stages must use the documented workflow order");
+  }
+
+  const preparationOutcome = stagesByName.get("prepare-and-list-projects").outcome;
+  const creationOutcome = stagesByName.get("create-thread")?.outcome;
+  const recordingOutcome = stagesByName.get("record-task")?.outcome;
+  const resolutionOutcome = stagesByName.get("resolve-provisional")?.outcome;
+  if (preparationOutcome === "failed") {
+    if (
+      value.stages.length !== 1 || task.taskId !== null || task.threadId !== null || task.clientThreadId !== null ||
+      task.recorded || task.resolution !== "unresolved" || task.resolutionAttempts !== 0
+    ) throw new Error("failed preparation must stop the workflow unresolved");
+  } else if (task.taskId === null || creationOutcome === undefined) {
+    throw new Error("successful preparation requires a task ID and creation stage");
+  }
+  if (creationOutcome !== undefined && creationOutcome !== "failed" && recordingOutcome === undefined) {
+    throw new Error("successful creation requires a record-task stage");
+  }
+  if (recordingOutcome !== undefined && (recordingOutcome === "recorded") !== task.recorded) {
+    throw new Error("record-task outcome must agree with task.recorded");
+  }
+  if (creationOutcome === "durable") {
+    if (task.resolution !== "immediate" || task.threadId === null || resolutionStagePresent) {
+      throw new Error("durable creation must be an immediate durable resolution");
+    }
+  } else if (creationOutcome === "provisional") {
+    if (task.clientThreadId === null) throw new Error("provisional creation requires a clientThreadId");
+    if (task.recorded) {
+      if (task.resolutionAttempts === 0) {
+        if (resolutionStagePresent || task.resolution !== "unresolved" || task.threadId !== null) {
+          throw new Error("zero-attempt provisional creation must remain unresolved");
+        }
+      } else if (
+        !resolutionStagePresent ||
+        (resolutionOutcome !== task.resolution && !(task.resolution === "unresolved" && resolutionOutcome === "failed"))
+      ) {
+        throw new Error("recorded provisional creation must agree with its resolution stage");
+      }
+      if ((task.resolution === "unresolved") !== (task.threadId === null)) {
+        throw new Error("provisional resolution must agree with task.threadId");
+      }
+    } else if (
+      resolutionStagePresent || task.resolution !== "unresolved" || task.threadId !== null || task.resolutionAttempts !== 0
+    ) {
+      throw new Error("unrecorded provisional creation must remain unresolved");
+    }
+  } else if (
+    creationOutcome === "failed" && (
+      value.stages.length !== 2 || task.resolution !== "unresolved" || task.threadId !== null ||
+      task.clientThreadId !== null || task.recorded || resolutionStagePresent
+    )
+  ) {
+    throw new Error("failed creation must remain unrecorded and unresolved");
+  }
+
+  const validation = requireObject(value.validation, "validation");
+  const validationNames = [
+    "recordVerified",
+    "markerVerified",
+    "outputVerified",
+    "candidateFilterEffective",
+  ];
+  requireExactKeys(validation, validationNames, "validation");
+  for (const name of validationNames) {
+    if (typeof validation[name] !== "boolean") throw new Error(`validation.${name} must be boolean`);
+  }
+  if (validation.recordVerified && !task.recorded) throw new Error("recordVerified requires a recorded task");
+  if (validation.markerVerified && task.threadId === null) throw new Error("markerVerified requires a durable threadId");
+  if (validation.outputVerified && task.threadId === null) throw new Error("outputVerified requires a durable threadId");
+
+  value.observations = normalizeObservations(
+    value.observations,
+    startedMs,
+    completedMs,
+    stagesByName.get("prepare-and-list-projects"),
+    stagesByName.get("create-thread"),
+    stagesByName.get("record-task"),
+    stagesByName.get("resolve-provisional"),
+    task.resolutionAttempts,
+  );
+  if (value.observations === undefined) delete value.observations;
+  const snapshots = value.observations?.resolutionSnapshots ?? [];
+  if (
+    snapshots.length > 0 && snapshots.length !== task.resolutionAttempts &&
+    !(resolutionOutcome === "failed" && snapshots.length === task.resolutionAttempts - 1)
+  ) {
+    throw new Error("resolutionSnapshots must account for every fallback attempt");
+  }
+  if (
+    task.resolutionAttempts === 2 && snapshots.length !== 2 &&
+    !(resolutionOutcome === "failed" && snapshots.length === 1)
+  ) {
+    throw new Error("two resolution attempts require two fallback snapshots");
+  }
+  if (task.resolution === "native" && (task.resolutionAttempts !== 1 || snapshots.length !== 0)) {
+    throw new Error("native resolution requires exactly one non-snapshot attempt");
+  }
+  if (task.resolution === "discovered") {
+    if (
+      snapshots.length === 0 || snapshots.at(-1).exactMatchCount !== 1 ||
+      snapshots.at(-1).resolveWriteOutcome !== "succeeded"
+    ) {
+      throw new Error("discovered resolution requires one successfully persisted final snapshot match");
+    }
+  }
+  if (snapshots.some(
+    (snapshot) => snapshot.attempt < task.resolutionAttempts && snapshot.exactMatchCount > 0,
+  )) {
+    throw new Error("any nonzero snapshot match count must terminate fallback resolution");
+  }
+  if (
+    task.resolution === "unresolved" && snapshots.some(
+      (snapshot) => snapshot.resolveWriteOutcome === "succeeded",
+    )
+  ) {
+    throw new Error("unresolved resolution cannot have a successful resolve write");
+  }
+  if (
+    snapshots.some((snapshot) => snapshot.resolveWriteOutcome === "failed") &&
+    resolutionOutcome !== "failed"
+  ) {
+    throw new Error("a failed resolve write requires a failed resolution stage");
+  }
+  if (
+    resolutionStagePresent && snapshots.length === 0 &&
+    !["native", "unresolved"].includes(task.resolution)
+  ) {
+    throw new Error("non-snapshot resolution must be native or unresolved");
+  }
+  if (snapshots.length === 1 && snapshots[0].exactMatchCount === 0 && task.resolutionAttempts !== 2) {
+    throw new Error("a completed first snapshot with zero matches requires the second fallback attempt");
+  }
+  const filterEffective = snapshots.length > 0 && snapshots.every(
+    (snapshot) => snapshot.candidateCount < snapshot.recentTaskCount,
+  );
+  if (validation.candidateFilterEffective !== filterEffective) {
+    throw new Error("candidateFilterEffective must match resolution snapshot counts");
+  }
+
+  const measuredStageMs = value.stages.reduce((total, stage) => total + stage.durationMs, 0);
+  const totalWallMs = completedMs - startedMs;
+  const summary = {
+    totalWallMs,
+    measuredStageMs,
+    orchestrationOverheadMs: totalWallMs - measuredStageMs,
+    resolved: task.threadId !== null,
+    resolutionAttempts: task.resolutionAttempts,
+  };
+  if (requireDerived && value.summary === undefined) {
+    throw new Error("summary is required in a saved result");
+  }
+  if (value.summary !== undefined) {
+    const suppliedSummary = requireObject(value.summary, "summary");
+    requireExactKeys(suppliedSummary, Object.keys(summary), "summary");
+    if (Object.entries(summary).some(([name, expected]) => suppliedSummary[name] !== expected)) {
+      throw new Error("summary does not match the derived benchmark summary");
+    }
+  }
+  value.summary = summary;
+  return value;
+}
+
+function timestampFilename(iso) {
+  return iso.replaceAll(":", "-");
+}
+
+export async function writeBenchmarkResult(input, outputDirectory) {
+  const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8"));
+  const result = normalizeBenchmarkResult({ ...input, taskchefVersion: packageJson.version });
+  await mkdir(outputDirectory, { recursive: true });
+  const filename = `${timestampFilename(result.startedAt)}-${BENCHMARK_NAME}.json`;
+  const outputPath = path.join(outputDirectory, filename);
+  await writeFile(outputPath, `${JSON.stringify(result, null, 2)}\n`, { flag: "wx" });
+  return { outputPath, result };
+}
+
+export async function cleanBenchmarkResults(outputDirectory) {
+  const entries = await readdir(outputDirectory, { withFileTypes: true }).catch((error) => {
+    if (error.code === "ENOENT") return [];
+    throw error;
+  });
+  const removed = [];
+  for (const entry of entries) {
+    if (!entry.isFile() || !managedResultFilename(entry.name)) continue;
+    await unlink(path.join(outputDirectory, entry.name));
+    removed.push(entry.name);
+  }
+  return removed.sort();
+}
+
+async function readJsonStdin() {
+  if (process.stdin.isTTY) throw new Error("write requires non-interactive JSON on stdin");
+  let input = "";
+  process.stdin.setEncoding("utf8");
+  for await (const chunk of process.stdin) input += chunk;
+  if (input.trim().length === 0) throw new Error("expected benchmark JSON on stdin");
+  return JSON.parse(input);
+}
+
+async function main(args) {
+  const command = args[0];
+  const target = path.resolve(args[1] ?? "reports/e2e-benchmarks");
+  if (command === "write") {
+    const { outputPath, result } = await writeBenchmarkResult(await readJsonStdin(), target);
+    process.stdout.write(`${JSON.stringify({ outputPath, summary: result.summary })}\n`);
+    return;
+  }
+  if (command === "validate") {
+    const result = normalizeBenchmarkResult(JSON.parse(await readFile(target, "utf8")), { requireDerived: true });
+    process.stdout.write(`${JSON.stringify({ inputPath: target, summary: result.summary })}\n`);
+    return;
+  }
+  if (command === "clean") {
+    process.stdout.write(`${JSON.stringify({ outputDirectory: target, removed: await cleanBenchmarkResults(target) })}\n`);
+    return;
+  }
+  throw new Error("usage: e2e-benchmark.js <write|validate|clean> [path]");
+}
+
+if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+  main(process.argv.slice(2)).catch((error) => {
+    process.stderr.write(`${error.message}\n`);
+    process.exitCode = 1;
+  });
+}

package/skills/taskchef-delegate/SKILL.md

@@ -82,10 +82,14 @@ for all deterministic workspace and task-record operations.
      ID, call it exactly once with a timeout of at most 30 seconds. Do not invent
      an operation or pass the provisional ID to tools that require `threadId`.
    - When no native operation is available, take at most two `list_threads`
-     snapshots with limit 50, near 10 and 30 seconds after the provisional
-     result. Count tool latency against the 30-second deadline. Do not start a
-     snapshot, candidate read, marker verification, or task-resolution write
-     after it.
+     snapshots with limit 50. The nominal schedule is 10 and 30 seconds after
+     the provisional result. Treat the first checkpoint as due, not expired: if
+     nullable recording or other required work finishes after 10 seconds, take
+     the first snapshot immediately. Start the second snapshot no sooner than
+     20 seconds after the first snapshot actually started. Useful candidate
+     work counts toward that interval; when it finishes sooner, wait only the
+     remainder. Never suppress either snapshot merely because an earlier step
+     ran late, and never take a third snapshot.
    - Filter Codex candidates by the expected host, project, creation time
      (allow five seconds of clock skew), and worktree environment whenever
      those fields are present. Use the title only to prioritize reads; Codex
@@ -135,3 +139,28 @@ delegated input contains an unresolved task's exact marker, run
 `<plugin-root>/bin/taskchef.js task resolve <task-id> --thread-id <thread-id> --json`.
 Never edit `tasks.jsonl` directly. The CLI permits only an idempotent one-way
 transition from `threadId: null` to one unique durable thread ID.
+
+## End-to-end evaluation
+
+When explicitly asked to benchmark delegation, measure one real task without
+waiting for executor completion. Start from
+`<plugin-root>/assets/e2e-benchmark-example.json`. Capture one ISO start/end
+interval for the parallel preparation/project-list operation, followed by
+sequential non-overlapping creation, recording, and optional provisional
+resolution stages. Follow the complete schema and workflow transitions in
+`<plugin-root>/SPEC.md`; the example is the durable-success starting shape.
+Omit later stages when an earlier operation stops the workflow.
+Fallback snapshot observations additionally require `recentTaskCount`,
+`candidateCount`, `exactMatchCount`, and `resolveWriteMs`. For each snapshot,
+also record `resolveWriteOutcome` as `not-attempted`, `succeeded`, or `failed`;
+a unique match whose atomic write fails remains unresolved. After the bounded
+workflow ends, verify the canonical task record once and, when resolved, read
+the task once to verify the requested output. A failed verification remains a
+false validation flag even when the corresponding operation succeeded. Feed
+one exact JSON value to
+`node <plugin-root>/scripts/e2e-benchmark.js write <output-directory>` on closed,
+non-interactive stdin. Keep timestamped results in the TaskChef source
+repository's ignored `reports/e2e-benchmarks/` directory. Never include hidden
+reasoning or transcripts. Use the script's `clean` command before establishing
+a replacement baseline. Mark candidate filtering effective only when the
+fallback snapshot narrows reads below the complete recent-task window.

package/src/delegation.js

@@ -1,6 +1,6 @@
 import { randomUUID } from "node:crypto";
 
-export const THREAD_RESOLUTION_CHECKPOINTS_MS = Object.freeze([10_000, 29_000]);
+export const THREAD_RESOLUTION_CHECKPOINTS_MS = Object.freeze([10_000, 30_000]);
 export const THREAD_RESOLUTION_TIMEOUT_MS = 30_000;
 export const THREAD_RESOLUTION_RECENT_LIMIT = 50;
 export const THREAD_RESOLUTION_CLOCK_SKEW_MS = 5_000;
@@ -179,6 +179,9 @@ function validateResolutionSchedule(checkpointsMs, timeoutMs) {
   if (!Array.isArray(checkpointsMs) || checkpointsMs.length === 0) {
     throw new Error("checkpointsMs must contain at least one checkpoint");
   }
+  if (checkpointsMs.length > 2) {
+    throw new Error("checkpointsMs must contain at most two checkpoints");
+  }
   if (!Number.isInteger(timeoutMs) || timeoutMs < 1) {
     throw new Error("timeoutMs must be positive");
   }
@@ -189,6 +192,9 @@ function validateResolutionSchedule(checkpointsMs, timeoutMs) {
     }
     previous = checkpoint;
   }
+  if (checkpointsMs.length === 2 && checkpointsMs[1] - checkpointsMs[0] < 20_000) {
+    throw new Error("checkpointsMs must keep two checkpoints at least 20000ms apart");
+  }
 }
 
 async function inspectCandidates(candidates, {
@@ -365,7 +371,11 @@ export async function createAndRecordDelegation({
     return false;
   };
 
-  const acceptResolvedThread = async (thread, resolution, attempt, { verified = false } = {}) => {
+  const acceptResolvedThread = async (thread, resolution, attempt, {
+    verified = false,
+    enforceDeadline = true,
+  } = {}) => {
+    const canContinue = enforceDeadline ? hasResolutionTime : () => true;
     const threadId = toolIdentifier(thread.id);
     if (
       threadId === null
@@ -381,18 +391,18 @@ export async function createAndRecordDelegation({
       return null;
     }
     const durableThread = { ...thread, id: threadId };
-    if (!hasResolutionTime()) return null;
+    if (!canContinue()) return null;
     if (!verified) {
       const inspected = await inspectCandidates([durableThread], {
         readThread,
         taskId: prepared.id,
         attempt,
-        canVerify: hasResolutionTime,
+        canVerify: canContinue,
       });
       discoveryErrors.push(...inspected.errors);
       if (inspected.exactMatches.length !== 1 || inspected.errors.length > 0) return null;
     }
-    if (!hasResolutionTime()) return null;
+    if (!canContinue()) return null;
     try {
       await resolveRecordedTask({ id: prepared.id, threadId });
     } catch (error) {
@@ -470,12 +480,14 @@ export async function createAndRecordDelegation({
   let exactMatches = [];
   let ambiguousMatches = [];
   let attemptsMade = 0;
+  let previousAttemptStartedAt = resolutionStartedAt;
   for (let index = 0; index < checkpointsMs.length; index += 1) {
     const attempt = index + 1;
-    const checkpointMs = checkpointsMs[index];
-    const elapsedBeforeAttempt = now() - resolutionStartedAt;
-    if (elapsedBeforeAttempt > timeoutMs) break;
-    const remainingDelayMs = checkpointMs - elapsedBeforeAttempt;
+    const intervalMs = index === 0
+      ? checkpointsMs[0]
+      : checkpointsMs[index] - checkpointsMs[index - 1];
+    const dueAt = previousAttemptStartedAt + intervalMs;
+    const remainingDelayMs = dueAt - now();
     if (remainingDelayMs > 0) {
       try {
         await waitImpl(remainingDelayMs);
@@ -488,43 +500,36 @@ export async function createAndRecordDelegation({
         break;
       }
     }
-    if (!hasResolutionTime()) break;
+    previousAttemptStartedAt = now();
     attemptsMade = attempt;
     let candidates = [];
     let attemptFailed = false;
     try {
       const snapshot = await listThreads({ limit: recentLimit });
-      if (hasResolutionTime()) {
-        candidates = filterThreadCandidates(snapshot, {
-          excludedThreadIds: provisionalIds,
-          hostId: expected.hostId ?? null,
-          projectId: expected.projectId ?? target.projectId ?? null,
-          title,
-          createdAfter,
-          environmentType: expected.environmentType ?? target.environment?.type ?? null,
-        });
-      }
+      candidates = filterThreadCandidates(snapshot, {
+        excludedThreadIds: provisionalIds,
+        hostId: expected.hostId ?? null,
+        projectId: expected.projectId ?? target.projectId ?? null,
+        title,
+        createdAfter,
+        environmentType: expected.environmentType ?? target.environment?.type ?? null,
+      });
     } catch (error) {
-      if (hasResolutionTime()) {
-        attemptFailed = true;
-        discoveryErrors.push({
-          attempt,
-          operation: "listThreads",
-          message: error instanceof Error ? error.message : String(error),
-        });
-      }
+      attemptFailed = true;
+      discoveryErrors.push({
+        attempt,
+        operation: "listThreads",
+        message: error instanceof Error ? error.message : String(error),
+      });
     }
-    if (!hasResolutionTime()) break;
     const inspected = await inspectCandidates(candidates, {
       readThread,
       taskId: prepared.id,
       attempt,
-      canVerify: hasResolutionTime,
     });
     exactMatches = inspected.exactMatches;
     discoveryErrors.push(...inspected.errors);
     if (inspected.errors.length > 0) attemptFailed = true;
-    if (!hasResolutionTime()) break;
     if (exactMatches.length > 1) ambiguousMatches = exactMatches;
     if (
       exactMatches.length === 1
@@ -536,7 +541,7 @@ export async function createAndRecordDelegation({
         exactMatches[0],
         "discovered",
         attempt,
-        { verified: true },
+        { verified: true, enforceDeadline: false },
       );
       if (resolved !== null) return resolved;
     }