@bastani/atomic 0.9.14-alpha.3 → 0.9.14-alpha.5

package/docs/workflows.md

@@ -731,9 +731,9 @@ Atomic bundles nine workflows: six reusable control-flow patterns, two autonomou
 |---|---|---|
 | `classify-and-act` | Structured classifier → deterministic category action; low confidence can fall back to human selection. | Route mixed requests to isolated category-specific work. |
 | `fan-out-and-synthesize` | Structured partition → bounded parallel artifact branches → synthesis barrier. | Split independent slices, including repository research, and merge evidence. |
-| `adversarial-verification` | Worker → fresh rubric verifiers → reducer → bounded repair loop. | Independently prove or reject a candidate. |
+| `adversarial-verification` | Worker → per-criterion fresh verifier fan-out → deterministic mean+veto gate → findings consolidation / bounded repair; consolidator cannot approve. | Independently prove or reject a candidate with auditable graded scores. |
 | `generate-and-filter` | Candidate fan-out → rubric dedupe/filter → optional judge → shortlist. | Explore more options than needed and keep the strongest distinct few. |
-| `tournament` | Whole-task attempts → balanced pairwise judges → bracket reducer. | Compare subjective or approach-sensitive solutions. |
+| `tournament` | Whole-task attempts → seeded ring and pivot-round soft scoring → full ranking reducer. | Compare subjective or approach-sensitive solutions. |
 | `loop-until-done` | Durable ledger → iteration/evaluator loop → success or inspectable bound exhaustion. | Continue until explicit evidence proves completion. |
 | `goal` | Durable goal ledger → bounded sub-agent orchestration → parallel review → deterministic reducer. | Autonomous implementation that needs receipts and reviewer-gated completion. |
 | `ralph` | Prompt refinement → codebase research → delegated implementation → multi-model review loop. | Research-first autonomous implementation with bounded review and repair. |
@@ -749,10 +749,10 @@ The six common patterns are full definitions exported from `@bastani/workflows/b
 |---|---|---|---|
 | `classify-and-act` | `prompt` | `categories` (1–8), `confidence_threshold` (0.5–0.99) | `result`, category, confidence, classification/action paths |
 | `fan-out-and-synthesize` | `prompt` | `max_branches` (1–12), `max_concurrency` (1–12) | `result`, partitions, branch paths, synthesis/manifest paths |
-| `adversarial-verification` | `task` | `verifier_count` (1–5), `max_repairs` (0–5) | `result`, approval, repairs, candidate/review/verifier paths |
+| `adversarial-verification` | `task` | `criteria` (record or criteria.md markdown; defaults to task_fit/evidence/completeness), `verifier_count=3` (1–5), `max_repairs=2` (0–5), `accept_mean=14`, `reask_limit=1`; normal calls per round: criteria.length × verifier_count | `approved`, `mean_score`, `score_table_path`, `repairs_completed`, `candidate_path`, `review_report_path`, `remaining_work` |
 | `generate-and-filter` | `prompt` | `num_candidates` (2–20), `shortlist_size` (1–10), `use_judge`, `max_concurrency` | `result`, shortlist, candidate/filter/judge/final/manifest paths |
-| `tournament` | `prompt` | `num_attempts` (2–8), `max_concurrency` (1–8) | `result`, winner, attempt/judge/bracket paths |
-| `loop-until-done` | `prompt` | `max_iterations` (1–20) | `result`, `status`, ledger, iteration/evaluation paths, remaining work |
+| `tournament` | `prompt` | `num_attempts` (2–8), `max_concurrency` (1–8), `n_evaluations=2`, `pivots=1`, `seed=0`, optional `criteria`/`models` | `result`, `winner`, `attempt_artifact_paths`, `judge_artifact_paths`, `comparisons_path`, `ranking`, `seed` |
+| `loop-until-done` | `prompt` | `max_iterations` (1–20), `progress_scoring`, `progress_repeats` (≥1) | `result`, `status`, ledger, iteration/evaluation paths, remaining work, `progress_curve`, `final_trend`, `progress_disclaimer` |
 
 ```ts
 import {
@@ -782,6 +782,7 @@ All six can run by name or as nested definitions. Prefer composition over copyin
 Goal persists the literal objective and immutable acceptance criteria in a run ledger, delegates implementation through bounded orchestrator turns, records receipts, and asks independent reviewers to inspect the current delta. A TypeScript reducer returns `complete`, `blocked`, or `needs_human` rather than trusting free-form completion claims.
 
 Goal reviewers derive checks from the literal objective before consulting implementation receipts, inspect the actual checkout delta, and report commands, observed output, and file:line evidence rather than internal reasoning. Shared contracts cover acceptance-matrix traceability, contract-fidelity risks, end-to-end and QA-video evidence, and independent verification. `stop_review_loop` is the authoritative convergence signal: it remains `false` for P0–P2 findings, any `required_by_objective` finding, or unproven implementation/validation requirements; it becomes `true` only when independent evidence proves the objective and only non-blocking or authorized post-approval work remains. The deterministic reducer consumes that signal without reinterpreting free-form prose.
+Goal and Ralph share the same low-confidence finding re-verification and per-round convergence evidence, documented under [`ralph`](#ralph).
 
 | Input | Type | Required | Default | Description |
 |---|---|---|---|---|
@@ -805,6 +806,12 @@ Ralph starts from the raw task, refines it into a research question, runs codeba
 
 Ralph uses the same canonical reviewer evidence and convergence contracts as Goal. Its reviewer prompt receives artifacts first and the review objective last, requires independently derived probes before implementation-authored evidence, and preserves unresolved findings when the bounded loop ends. Forked continuation prompts send only changed state and artifact paths instead of repeating the full established contract.
 
+Goal and Ralph re-verify only an eligible consolidated finding: it must still be blocking, have exactly one reviewer, carry a finite `confidence_score` strictly below `DEFAULT_REVERIFY_THRESHOLD=0.7`, and not be aligned `beyond_objective` or `contradicts_objective`; missing confidence is not eligible. Eligible findings are rescored in fresh contexts with the primitive's default `DEFAULT_REPEATS=3`, and an invalid repeat is re-asked once before its audit entry records a null score.
+
+Re-verification has two demotion bars. For an ordinary in-scope finding, demotion requires at least `ceil(repeatCount / 2)` valid scores and a mean below `STANDARD_CONFIRM_THRESHOLD=10`; for `required_by_objective`, every repeat must be valid and the mean must be below `REQUIRED_CONFIRM_THRESHOLD=6`. The original finding remains in the review record while the durable `reverification` audit records the verdict, mean, per-repeat scores, and evidence.
+
+Each parsed Goal review round and Ralph review round appends convergence evidence with `unresolvedBlockingCount`, `meanFindingConfidence`, `fractionProven`, `demotions`, and folded `usage`. Goal persists it in the goal ledger's `convergence` array beside `reverification`; Ralph exposes the same per-round series in `review-round-latest.json`. The convergence classifier reports blocker and proven trends, and its escalation text is evidence only: `stop_review_loop` remains the authoritative closure signal, while review scores and convergence evidence are audit/advisory data that never approve or terminate a loop.
+
 | Input | Type | Required | Default | Description |
 |---|---|---|---|---|
 | `prompt` | text | yes | — | Task, issue, or spec to research, implement, and review. Keep PR/MR creation out of this text. |
@@ -1072,7 +1079,7 @@ Workflow outputs are runtime contracts for completed workflow runs and for paren
 
 **Return convention:** outputs are return-object keys. Atomic never infers child workflow outputs from stage names, stage order, or the final assistant message. If a parent should read `child.outputs.foo`, the child workflow's `run` must both declare `outputs: { foo: schema }` and return `{ foo: value }`. `result` is not special, and Atomic never adds it: to expose `result`, declare it in `outputs` and return `{ result }` exactly like any other output. Returning a key that is not declared in `outputs` fails the run with `atomic-workflows: workflow "<name>" returned undeclared output "<key>"; declare it in outputs or remove it from the run return`.
 
-**Reserved `status` output convention and structured failures:** if a workflow declares and returns a top-level `status` output with the string value `"failed"`, Atomic treats the run as failed instead of recording a successful completion. Returned `"blocked"`, `"needs_human"`, `"incomplete"`, `"active"`, and `"auth_blocked"` statuses are treated as blocked/incomplete terminal states rather than successful completions.
+**Reserved `status` output convention and structured failures:** if a workflow declares and returns a top-level `status` output with the string value `"failed"`, Atomic treats the run as failed instead of recording a successful completion. Returned `"blocked"`, `"needs_human"`, `"incomplete"`, `"active"`, and `"auth_blocked"` statuses are treated as blocked/incomplete terminal states rather than successful completions. The engine's own system-owned `budget_exceeded` stop is likewise blocked; a workflow-returned `budget_exceeded` value cannot forge that stop and is treated as a normal completion.
 
 Independently of that convention, Atomic uses structured failure metadata captured from the run's blocking stage (`failedStageId`) or run-level failure metadata to keep recoverable auth, rate-limit, and provider fallback exhaustion blocked/resumable even when the workflow did not declare a `status` output. Atomic does not infer failure state by scanning arbitrary output text or by scanning every failed stage in an otherwise completed non-fail-fast branch.
 
@@ -3242,7 +3249,7 @@ Deliberate control actions on a top-level run report themselves too. `/workflow
 
 **One notice per request.** A whole-run pause or resume reports at run scope. A stage-scoped `/workflow pause <run> <stage>` that leaves other stages running reports at stage scope, and one that stops the last active stage reports the run instead — never a stage card and a run card for the same request. A quit reports only the quit, never the pause it publishes on the way. Because control actions are reversible, these notices are deduplicated by run id *and* the occurrence timestamp, so pause → resume → pause → resume emits four notices while repeated snapshot invalidations at one unchanged state emit one. Resuming reports a resume and never a start, whoever asked for it — a resumed run re-enters the dispatch path, so keying that on the resume rather than on the requester is what stops an agent-requested resume of a user-started run from being announced as a fresh launch. Resuming a failed or blocked run launches a continuation under a fresh run id, and its notice names both ("run 4d7e, continuing run 8c31"); resuming a quit run reuses the original workflow id so durable checkpoints replay, so that notice names the one id. A run that is already started, paused, or quit when notifications install — restore, replay, `/reload`, or a session-preserving reinstall — is seeded as delivered and stays silent, and nested `ctx.workflow(...)` child runs never notify at top level.
 
-Configure lifecycle behavior with `workflowNotifications.enabled` (default `true`) and `workflowNotifications.notifyOn` (default `["started", "completed", "failed", "blocked", "awaiting_input", "paused", "quit", "resumed"]`). A config that pins `notifyOn` explicitly keeps exactly the kinds it lists, so `notifyOn: ["failed"]` suppresses every control notice.
+Configure lifecycle behavior with `workflowNotifications.enabled` (default `true`) and `workflowNotifications.notifyOn` (default `["started", "completed", "failed", "blocked", "budget_warning", "awaiting_input", "paused", "quit", "resumed"]`). A config that pins `notifyOn` explicitly keeps exactly the kinds it lists, so `notifyOn: ["failed"]` suppresses every control notice. `budget_warning` is delivered once per run and dimension through the same lifecycle-notice renderer.
 
 **Heartbeats are separate from lifecycle notices.** A lifecycle notice reports a transition; a heartbeat reports that nothing has transitioned yet. While a top-level run is active, Atomic raises one `workflows:workflow-heartbeat` card per `startedAt + n × heartbeatIntervalMinutes` boundary, on the same queued-steer delivery (`triggerTurn`, `deliverAs: "steer"`, `persistWhenStreaming`) and the same notice-card renderer, under its own custom type. The cadence is per workflow definition — `15` minutes by default, `0` to disable — and is documented under [`heartbeatIntervalMinutes`](#heartbeatintervalminutes). `workflowNotifications.notifyOn` selects lifecycle kinds only; it does not list or filter heartbeats. Heartbeats stop when the run reaches a terminal state: one idempotent cleanup pass drops its timer, its schedule, and any heartbeat still queued inside the scheduler, a later process discards those records rather than replaying them, and a card the parent's queue had already accepted is excluded from the model's context when it is read ([#1975](https://github.com/bastani-inc/atomic/issues/1975)).
 
@@ -3314,7 +3321,7 @@ Repeated, sibling, sequential, parallel, and multi-level child calls keep indepe
 
 ### `ctx.tool` — durable cached tool execution
 
-The `ctx.tool(name, args, fn, options?)` primitive runs arbitrary TypeScript code as a first-class durable graph node and caches the result durably. The node is non-attachable and has no stage chat controls, and its graph card body is the constant `durable tool` in every state — status, timing, and dependency rows keep their own rows, and the card does not preview the result or error. In the graph viewer, focusing the node and pressing Enter, clicking it, or choosing it from the switcher opens a read-only host-style operator card from the snapshot: a status-tinted shaded rectangle with a `$ <tool-name>` call header, an optional short argument summary, and the result or error as its body. It is collapsed by default and wraps the fully bounded result or error before showing its last visual rows, with `... (N earlier lines, ctrl+o Expand)` above the tail when the action is bound; the configured `app.tools.expand` action (`ctrl+o` by default) toggles the full bounded result or error and then a muted callback-source block when source exists. The graph statusline advertises the resolved expand key with `expand` or `collapse` alongside return-to-graph and scroll hints, including remapped keys, and omits that segment entirely when the action is unbound. The footer says `Took` for settled calls or `Elapsed` for running calls, using milliseconds below one second and the existing human duration above it, with cached/replayed markers kept as a quiet suffix. The operator surface has no ARGS/RESULT/SOURCE/TIMING/MARKERS debug table and does not expose raw clock fields. Source capture uses `fn.toString()` at registration without re-executing the callback or reading a file. `↑`/`↓`, `PageUp`/`PageDown`, `Home`/`End`, the wheel, and the scrollbar all scroll the block, so a long payload stays readable on a keyboard-only session or a terminal without mouse reporting; Escape or `ctrl+x` returns to the graph. The message block is read-only and never offers chat attachment, steering, interrupt, or resume. Bounded payloads remain width-safe and mark truncation explicitly with `… [truncated]`; source tabs expand and control bytes become `\xNN`, while cyclic payloads, throwing `toJSON`, or throwing property getters render `<cycle>`, `<unserializable>`, or `<unreadable>` instead of crashing the view. The same cap applies to what the live run snapshot retains for a tool node, while durable checkpoints keep the exact output, raw-args `argsHash`, and replay behavior unchanged.
+The `ctx.tool(name, args, fn, options?)` primitive runs arbitrary TypeScript code as a first-class durable graph node and caches the result durably. The node is non-attachable and has no stage chat controls, and its graph card body is the constant `durable tool` in every state — status, timing, and dependency rows keep their own rows, and the card does not preview the result or error. In the graph viewer, focusing the node and pressing Enter, clicking it, or choosing it from the switcher opens a read-only host-style operator card from the snapshot: a status-tinted shaded rectangle with the same inner padding and header/body gap as the main-chat tool block, inset from the orchestrator header and footer bars, a `$ <tool-name>` call header, an optional short argument summary, and the result or error as its body. It is collapsed by default and wraps the fully bounded result or error before showing its last visual rows, with `... (N earlier lines, ctrl+o Expand)` above the tail when the action is bound; the configured `app.tools.expand` action (`ctrl+o` by default) toggles the full bounded result or error and then a muted callback-source block when source exists. The graph statusline advertises the resolved expand key with `expand` or `collapse` alongside return-to-graph and scroll hints, including remapped keys, and omits that segment entirely when the action is unbound. The footer says `Took` for settled calls or `Elapsed` for running calls, using the same second-resolution duration as the main-chat tool block, with cached/replayed markers kept as a quiet suffix. The operator surface has no ARGS/RESULT/SOURCE/TIMING/MARKERS debug table and does not expose raw clock fields. Source capture uses `fn.toString()` at registration without re-executing the callback or reading a file. `↑`/`↓`, `PageUp`/`PageDown`, `Home`/`End`, the wheel, and the scrollbar all scroll the block, so a long payload stays readable on a keyboard-only session or a terminal without mouse reporting; Escape or `ctrl+x` returns to the graph. The message block is read-only and never offers chat attachment, steering, interrupt, or resume. Bounded payloads remain width-safe and mark truncation explicitly with `… [truncated]`; source tabs expand and control bytes become `\xNN`, while cyclic payloads, throwing `toJSON`, or throwing property getters render `<cycle>`, `<unserializable>`, or `<unreadable>` instead of crashing the view. The same cap applies to what the live run snapshot retains for a tool node, while durable checkpoints keep the exact output, raw-args `argsHash`, and replay behavior unchanged.
 
 When the workflow body fulfills but one or more admitted tool calls failed, Atomic promotes the first observed failure to the terminal run failure, regardless of admission order, and persists that selected tool-node identity for status inspection and lifecycle output. A direct uncaught `await ctx.tool(...)` rejection keeps the original error and persists its failed-node link through session and durable restore. First-event arbitration also preserves the selected node when concurrent failures throw the same object or primitive; unrelated later stage or body errors do not inherit a caught tool's origin. Tool admission remains open while author code can catch a failure and continue. Once the body settles and failure has won before any real cancellation, Atomic closes admission, cancels remaining non-failed tool nodes, waits for observed failed nodes to finish publication, and publishes the failed root without waiting for callbacks that ignore cancellation.
 
@@ -3512,6 +3519,27 @@ The `/workflow` argument-completion popup reads that same live registry. Project
 
 A successful rescan may still contain per-resource diagnostics. Both reload surfaces show `CONFIG_INVALID`, `IMPORT_FAILED`, `INVALID_DEFINITION`, `PATH_NOT_FOUND`, and duplicate-name diagnostics instead of reporting bare success while silently skipping a resource. Valid sibling workflows remain available. Fix the reported source/path and reload again; no process restart is required.
 
+## Run budgets
+
+Set an optional `budget` on workflow extension config, an authored `workflow({...})` definition, a `workflow({ action: "run" })` tool call, or a `workflow({ action: "resume" })` continuation to raise or narrow the ceiling. Each field resolves independently: run override, then definition, then config default. An omitted field falls through; a present `0` disables that dimension.
+
+```ts
+export default workflow({
+  name: "bounded-review",
+  description: "Review a change within an operator-selected budget.",
+  budget: { maxDurationMs: 900_000, maxTokens: 50_000, maxCost: 5, warnAtPercent: 80 },
+  outputs: {},
+  run: async (ctx) => {
+    // ...
+    return {};
+  },
+});
+```
+
+`maxDurationMs` and `maxTokens` must be non-negative finite integers. `maxCost` and `warnAtPercent` must be non-negative finite numbers. Invalid config produces `CONFIG_INVALID`; invalid authored or direct-run declarations throw a `TypeError` before the workflow body runs. Nested `ctx.workflow(child)` calls use the child's own declared budget and remain subject to the root run's duration scope; a root exhaustion wins simultaneous child exhaustion, while a child-only exhaustion soft-lands that child run and returns to the parent.
+
+`maxDurationMs` is enforced at stage and durable-tool boundaries using elapsed run time (paused time is excluded and resumed runs carry prior elapsed time). `maxTokens` counts uncached input plus output tokens across the complete run tree, including nested children and stage retries; cache reads and writes remain reported counters and `maxCost` charges the summed `usage.cost`. A `budget_warning` lifecycle notice is emitted once per run and dimension at `warnAtPercent` (default `80`); exhaustion gives an already-live frontier stage one current-turn wrap-up, then records a resumable `budget_exceeded` blocked result with its reading, ceiling, frontier, wrap-up summary, and the wrap-up turn's own `wrapUpUsage` when model usage is available. No new stage is created just to host a wrap-up; when no stage turn is live at the exhausting boundary, the run stops with no wrap-up summary and leaves the once-per-run delivery allowance unused. A resumed run carries prior duration, token, and cost spend without double-charging replayed completions; pass a raised resume budget to continue with the prior spend. Nested child budgets meter only their subtree while the root meter still includes child spend, and a child-only exhaustion returns to the parent as a blocked child result while the parent continues.
+
 ## Workflow Configuration
 
 Configured workflow paths live in workflow extension config. Project config paths are relative to the project root. Global config paths are relative to `~/.atomic/agent`.
@@ -3538,12 +3566,13 @@ Example config:
   },
   "defaultConcurrency": 4,
   "maxDepth": 4,
+  "budget": { "maxDurationMs": 0, "maxTokens": 0, "maxCost": 0, "warnAtPercent": 80 },
   "persistRuns": true,
   "statusFile": false,
   "resumeInFlight": "ask",
   "workflowNotifications": {
     "enabled": true,
-    "notifyOn": ["started", "completed", "failed", "blocked", "awaiting_input", "paused", "quit", "resumed"]
+    "notifyOn": ["started", "completed", "failed", "blocked", "budget_warning", "awaiting_input", "paused", "quit", "resumed"]
   },
   "worktree": {
     "symlinkDirectories": ["node_modules"]
@@ -3557,11 +3586,12 @@ Runtime config defaults:
 |-----|---------|---------|
 | `defaultConcurrency` | `4` | Default concurrency for authored `ctx.parallel(...)` execution |
 | `maxDepth` | `4` | Maximum workflow nesting depth |
+| `budget` | `{ maxDurationMs: 0, maxTokens: 0, maxCost: 0, warnAtPercent: 80 }` | Default per-run budget declaration; `0` disables a dimension; warnings default to `80` percent |
 | `persistRuns` | `true` | Persist run metadata for status/resume/history |
 | `statusFile` | `false` | Write a derived status file; defaults under `.atomic/workflows/status.json` when enabled |
 | `resumeInFlight` | `"ask"` | Behavior when discovering resumable in-flight work |
 | `workflowNotifications.enabled` | `true` | Emit workflow lifecycle notices into the active main chat |
-| `workflowNotifications.notifyOn` | `["started", "completed", "failed", "blocked", "awaiting_input", "paused", "quit", "resumed"]` | Lifecycle states to track; terminal `completed`/`failed`/`blocked` outcomes, active recoverable blocks, and the user-initiated `started`/`paused`/`quit`/`resumed` control actions on a top-level run create main-chat notices, while `awaiting_input` is tracked for dedupe/restore without waking the main agent |
+| `workflowNotifications.notifyOn` | `["started", "completed", "failed", "blocked", "budget_warning", "awaiting_input", "paused", "quit", "resumed"]` | Lifecycle states to track; terminal `completed`/`failed`/`blocked` outcomes, active recoverable blocks, duration budget warnings, and the user-initiated `started`/`paused`/`quit`/`resumed` control actions on a top-level run create main-chat notices, while `awaiting_input` is tracked for dedupe/restore without waking the main agent |
 | `worktree.symlinkDirectories` | `["node_modules"]` | Main-root directories symlinked into each runner-managed temporary worktree during post-creation setup |
 
 Invalid JSON or invalid shapes produce `CONFIG_INVALID` diagnostics. Missing config files are ignored.
@@ -4547,7 +4577,7 @@ These patterns organize work **inside one root lifecycle**. They do not replace
 | **Fan-out-and-synthesize** | The task can be split into many independent slices that benefit from clean context windows. | `ctx.parallel([...])` with separate artifacts → synthesis barrier that reads the artifacts and merges the answer. |
 | **Adversarial verification** | Outputs need independent checking against a rubric, security rule, factual source, or acceptance contract. | Worker stage(s) → fresh-context verifier stage(s) → reducer that accepts, rejects, or asks for repair. |
 | **Generate-and-filter** | You need many candidate ideas, plans, names, fixes, or hypotheses before selecting the best few. | Generator fan-out → dedupe/filter stage → optional verifier/judge → final shortlist. |
-| **Tournament** | The whole task is subjective or approach-sensitive, and comparative judgment is more reliable than absolute scoring. | Several agents attempt the same task → pairwise judges compare results → bracket reducer returns winners. |
+| **Tournament** | The whole task is subjective or approach-sensitive, and comparative judgment is more reliable than absolute scoring. | Several agents attempt the same task → seeded ring and pivot rounds score each candidate pair by criterion → reducer reports the winner and full ranking. |
 | **Loop until done** | The amount of work is unknown up front, such as finding all failures, mining repeated issues, or iterating until checks pass. | Bounded loop with an explicit stop condition, progress ledger, per-iteration artifacts, and a max-iteration escape hatch. |
 | **Constructive quorum** | Several fresh-context verifiers judge the same artifact and a tallied vote could mask a defect one verifier found or block on one verifier's misreading. | Parallel verifiers form independent preliminary verdicts → exactly one bounded Intercom evidence-exchange round (share and challenge evidence) → each emits its own final structured verdict → deterministic reducer counts votes. |
 | **Scope guard** | A worker or repair stage may turn valid adjacent findings into unplanned work. | Immutable contract artifact → fresh boundary or live scope checker → bounded decision artifact → forked worker continuation; correctness review stays separate. |
@@ -4636,6 +4666,17 @@ Best practices:
 - Separate adversarial probe design from authoritative execution. Require a structured verifier plan with each exact probe, inputs, command/assertion, expected success condition, and covered requirement/risk; then run selected compile, test, schema generation/validation, runtime, or artifact checks through durable workflow-owned `ctx.tool(...)` calls. Actual tool results—not model self-report—feed judgment and consolidated repair.
 - Known contracts may use direct task-specific `ctx.tool(...)` gates designed before launch; uncertain risks may use model-selected probes executed by those deterministic tools. Rerun the tools after repair until the declared pass condition or iteration limit.
 - Ask verifiers to find blockers and not rewrite the candidate unless you explicitly assign them to repair it. Keep pure transformations as ordinary TypeScript rather than wrapping every model-stage action in `ctx.tool`.
+- Decompose the rubric into named criteria and score each in its own call. Compound rubrics can latch onto one salient factor; the reference scan reports 76.4% for the best single criterion versus 78.3% for a three-criterion ensemble (§4.3).
+- Aggregate by mean plus an explicit veto for genuinely disqualifying findings, never a unanimity AND across verifiers: unanimity makes false-reject grow as 1−(1−p)^K while the false-accept it buys only decays as (1−p)^K. See [Verification scaling](#verification-scaling).
+- The shipped `adversarial-verification` builtin accepts `criteria` as a record of criterion names to descriptions or as a `criteria.md` Markdown string; the shared `verification-criteria` module also canonicalizes string lists and `CriterionInput` lists. Its public doors are `parse_rubric`, `normalize_criteria`, `select_criteria`, and `decide_verification`, using the `Criterion`, `CriterionInput`, `CriterionScore`, and `Finding` shapes; `NoCriteria` and `EmptyCriterion` are explicit rubric errors.
+- A `criteria.md` rubric may have a `#` title, an optional `##` section whose heading contains `ground truth` (normally `## Ground Truth Note`; the first such section wins), and must include a `##` section whose heading contains `criteri` (normally `## Criteria`) whose `### Name {#id}` headings own non-empty criterion bodies. HTML comments are ignored; an omitted `{#id}` is slugged to lowercase alphanumeric/underscore text (up to 40 characters), with a fallback `criterion` id and encounter-order `_2`/`_3` deduplication. `parse_rubric` rejects a rubric with no criterion headings or an empty criterion body.
+- `VERIFICATION_SCALE` anchors integer scores from 1 (certainly fails) through 20 (verified correct). `select_criteria` preserves the requested id order and rejects unknown ids; `decide_verification` accepts only with quorum, a mean at or above the policy threshold, and no `veto` finding, while an invalid report remains metadata rather than a score.
+- Keep a scoring family in the `SHARED HEAD ‖ VARYING TAIL` layout from `verification-prompts`: the byte-identical head contains the task, ground-truth note, candidate bodies (or caller-provided read paths), and scale anchors in that order; the tail contains only the criterion name and description plus the output-format instruction. Candidate-specific bodies stay in the shared head, not the varying tail, so sibling criteria can reuse the cached prefix.
+- Inline the whole candidate family only while every body is at most `32 * 1024` UTF-8 bytes (`MAX_INLINE_CANDIDATE_BYTES`). If any body is larger, switch the whole family to caller-bound paths, preserving path order and duplicates; an oversized pathless family is rejected rather than guessed.
+- `warm_first_fan_out` schedules the first-seen step for each prefix before releasing the remaining steps, establishing the provider's warm prefix before sibling criteria or pair slots vary. Warm failures are observed without fail-fast, the remaining phase is still attempted before the error is rethrown, and successful results return in input order.
+- The builtin input defaults are `verifier_count=3`, `max_repairs=2`, `accept_mean=14` on the 1–20 scale, and `reask_limit=1`; omitted `criteria` uses the `task_fit`, `evidence`, and `completeness` record. A round expects one schema-valid score for every criterion/verifier cell, and the normal call shape is criteria length multiplied by verifier count.
+- Invalid criterion reports are written as invalid artifacts and re-asked in bounded waves up to `reask_limit`; an invalid or missing report is counted in `invalidCount` only and is never converted into a fail vote or included in the mean. If the required quorum is still missing after the re-asks, the round is `indeterminate` rather than silently narrowing the decision.
+- `score_table_path` names the durable `verification-summary-<round>.json` for the final round. Its object contains `scores` (`criterion_id`, integer `score`, `evidence`, and `findings` with `finding` plus `severity`), `mean`, `invalidCount`, the `decision` (`accept`, `repair`, or `indeterminate` with its corresponding mean/findings or missing count), and folded `usage`; `review_report_path` carries repair guidance or quorum evidence.
 
 ##### 4. Generate-and-filter
 
@@ -4662,6 +4703,8 @@ Best practices:
 - Generate more candidates than you need, then filter hard by an explicit rubric.
 - Dedupe before judging so near-identical candidates do not dominate the shortlist.
 - Use this for exploration, naming, design options, hypotheses, and lightweight eval ideas.
+- When the filter ranks candidates rather than applying a threshold, use the same judge guidance as Tournament: graded per-criterion integer scores rather than binary keep/drop, a Bradley–Terry preference from the score gap so near-ties stay near-ties, and K repeats with candidates swapped between the A and B slots. See [Verification scaling](#verification-scaling).
+- For a custom ranking filter, reuse the shared `verification-criteria` module and its `criteria.md` parser rather than inventing a binary keep/drop rubric; stable criterion ids let the judge select the same criteria in each comparison. See [Adversarial verification](#3-adversarial-verification) for the accepted shapes and score decision.
 
 ##### 5. Tournament
 
@@ -4690,6 +4733,11 @@ Best practices:
 - Use pairwise comparison when absolute scores are noisy or subjective.
 - Randomize or balance presentation order where possible to reduce order bias.
 - Keep the judge rubric short and require rationale tied to observable criteria.
+- Have judges emit graded per-criterion integer scores rather than a binary winner, then derive a Bradley–Terry preference from the score gap so near-ties stay near-ties.
+- Repeat each pair K times with the candidates swapped between the A and B slots; the swap cancels positional bias within the pair and variance falls as O(1/K). In the reference scan's discrete-judge study, 26.7% of pairs tied at K=1; with slot swaps, the reported K=1→16 result moved from 74.7% to 77.5%.
+- See [Verification scaling](#verification-scaling) for score granularity and call-budget trade-offs.
+- The shipped tournament inputs use `num_attempts=4` and `max_concurrency=4`; `n_evaluations=2` repeats each criterion/directed pair, `pivots=1` selects the second comparison phase's pivot candidates, and `seed=0` drives the deterministic schedule. `criteria` is optional and accepts a markdown rubric, a string-to-description record, a string list, or a `CriterionInput` list; omission uses the shipped three-criterion Correctness, Completeness, and Evidence and task fit rubric. Optional ordered `models` ids are assigned round-robin to attempt slots.
+- `comparisons_path` points to `comparisons.json`, whose ledger records the task and seed, `params` (`n`, `pivots`, `n_evaluations`, and normalized `criteria`), per-job `comparisons` rows (`a`, `b`, phase, criterion id, repeat, slot-swap flag, scores or an `invalid` marker, preference, and judge artifact path), aggregate `pairs`, weights/counts, the complete `ranking`, and optional model assignment. Its `budget` records planned versus executed judge stages, including re-asks; invalid reports remain auditable rows and an all-invalid pair remains marked invalid rather than becoming a score.
 
 ##### 6. Loop until done
 
@@ -4715,6 +4763,11 @@ Best practices:
 - Keep a durable ledger of attempted work, findings, failures, and validation evidence.
 - Bound loops by iterations, budget, or convergence criteria so exhausting a bound produces an inspectable failure instead of letting the loop continue indefinitely.
 - Materialize every iteration as distinct tracked work with stable iteration identity and call order. Never represent repetition by a self-edge, a back-edge to an ancestor, or reopening an ancestor below its downstream work.
+- Record a progress magnitude in the ledger beside the boolean stop bit; a flat or decreasing series is the stall signal that the loop is burning iterations without moving.
+- Treat the trend as a monitoring and escalate-to-human signal, never a kill switch: the explicit stop condition remains authoritative. See [Verification scaling](#verification-scaling).
+- The builtin defaults `max_iterations=5`, `progress_scoring=true`, and `progress_repeats=1`; set `progress_scoring` false to omit advisory scoring, while `progress_repeats` is the repeat count passed to the scoring primitive. Each scored iteration adds a `progress` entry to `progress-ledger.json` with `score`, `perRepeat` (null for an invalid repeat), `trend`, and the classifier `window`; the ledger also emits `progress_curve`, `final_trend`, and `progress_disclaimer`.
+- Progress scores use the anchored 1–20 scale and average valid repeat scores per checkpoint. `classify_trend` uses `window=3`, `riseDelta=1.5`, and `fallDelta=-1.5`; it compares equal leading/trailing halves of the trailing two windows, drops an odd middle sample, and classifies inclusive threshold crossings as `rising`, `flat`, or `regressing`. A short series is `flat` evidence.
+- The trend is monitoring and escalation evidence only: it never kills, terminates, or approves a loop, and the explicit evaluator stop condition remains authoritative. `progress_curve`, `final_trend`, and `progress_disclaimer` are advisory outputs, not alternate closure signals.
 
 ##### 7. Constructive quorum
 
@@ -4891,6 +4944,17 @@ The `prepareSliceWorktree` tools run before their child boundaries and use `git
 
 Use `ralph` or a task-specific child in the same positions when its input contract fits better. For a longer stack, keep the same explicit downstream shape: create each next named branch from the previous verified branch, pass that previous branch as the next child's `base_branch`, and use a distinct worktree. Do not replace the chain with a loop that points back to an ancestor. A final handoff can report `slice → branch → worktree → verified/failed` from the explicit inputs and preparation records without reopening completed child work.
 
+#### Verification scaling
+
+This is authoring guidance for custom workflows, not a description of shipped builtin inputs:
+
+- Use an anchored 1–20 integer scale as the default score granularity.
+- Providers expose no token logprobs, so a K-sample average is the substitute; K=16 parity costs roughly 16× the call cost, making K a budget decision.
+- Treat pool diversity as a bet on the selector's oracle ceiling. In the reference scan's pivot tournament, best-of-3 selection reached 86.5% ±1.1 against 79.4% pass@1 with a 92.1% oracle ceiling, while best-of-5 reached 88.0% ±0.6 against 78.7% pass@1 with a 96.6% oracle ceiling. A chance-level selector can make a more diverse pool worse, so widen the pool only once the judge beats chance.
+- Self-verification—having the same model judge its own rollouts—still gained +7.1 over pass@1 in the best-of-3 comparison (86.5% versus 79.4%) and +9.3 in the best-of-5 comparison (88.0% versus 78.7%).
+- For a cheap operating point, an author can use one pivot and K=2 repeats for a best-of-3-shaped comparison budget; this is an authoring recipe, not a shipped default.
+- For the shipped primitive references, see [Adversarial verification](#3-adversarial-verification) for criteria parsing, warm-first scoring, re-asks, and score summaries; [Tournament](#5-tournament) for inputs and `comparisons.json`; [Loop until done](#6-loop-until-done) for progress ledger/trend outputs; and [Goal](#goal)/[Ralph](#ralph) for re-verification and convergence evidence.
+
 #### Choosing a common workflow pattern
 
 - Pick **classify-and-act** when routing correctness matters more than breadth.

package/npm-shrinkwrap.json

@@ -1,15 +1,15 @@
 {
 	"name": "@bastani/atomic",
-	"version": "0.9.14-alpha.3",
+	"version": "0.9.14-alpha.5",
 	"lockfileVersion": 3,
 	"requires": true,
 	"packages": {
 		"": {
 			"name": "@bastani/atomic",
-			"version": "0.9.14-alpha.3",
+			"version": "0.9.14-alpha.5",
 			"license": "MIT",
 			"dependencies": {
-				"@bastani/atomic-natives": "0.9.14-alpha.3",
+				"@bastani/atomic-natives": "0.9.14-alpha.5",
 				"@dbos-inc/dbos-sdk": "4.25.14",
 				"@earendil-works/pi-agent-core": "^0.84.2",
 				"@earendil-works/pi-ai": "^0.84.2",
@@ -517,18 +517,18 @@
 			}
 		},
 		"node_modules/@bastani/atomic-natives": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives/-/atomic-natives-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives/-/atomic-natives-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"optionalDependencies": {
-				"@bastani/atomic-natives-darwin-arm64": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-darwin-x64": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-linux-arm64-gnu": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-linux-arm64-musl": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-linux-x64-gnu": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-linux-x64-musl": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-win32-arm64-msvc": "0.9.14-alpha.3",
-				"@bastani/atomic-natives-win32-x64-msvc": "0.9.14-alpha.3"
+				"@bastani/atomic-natives-darwin-arm64": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-darwin-x64": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-linux-arm64-gnu": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-linux-arm64-musl": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-linux-x64-gnu": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-linux-x64-musl": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-win32-arm64-msvc": "0.9.14-alpha.5",
+				"@bastani/atomic-natives-win32-x64-msvc": "0.9.14-alpha.5"
 			},
 			"engines": {
 				"bun": ">=1.3.14",
@@ -536,8 +536,8 @@
 			}
 		},
 		"node_modules/@bastani/atomic-natives-darwin-arm64": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-darwin-arm64/-/atomic-natives-darwin-arm64-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-darwin-arm64/-/atomic-natives-darwin-arm64-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"darwin"
@@ -548,8 +548,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-darwin-x64": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-darwin-x64/-/atomic-natives-darwin-x64-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-darwin-x64/-/atomic-natives-darwin-x64-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"darwin"
@@ -560,8 +560,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-linux-arm64-gnu": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-arm64-gnu/-/atomic-natives-linux-arm64-gnu-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-arm64-gnu/-/atomic-natives-linux-arm64-gnu-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"linux"
@@ -575,8 +575,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-linux-arm64-musl": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-arm64-musl/-/atomic-natives-linux-arm64-musl-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-arm64-musl/-/atomic-natives-linux-arm64-musl-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"linux"
@@ -590,8 +590,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-linux-x64-gnu": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-x64-gnu/-/atomic-natives-linux-x64-gnu-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-x64-gnu/-/atomic-natives-linux-x64-gnu-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"linux"
@@ -605,8 +605,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-linux-x64-musl": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-x64-musl/-/atomic-natives-linux-x64-musl-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-linux-x64-musl/-/atomic-natives-linux-x64-musl-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"linux"
@@ -620,8 +620,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-win32-arm64-msvc": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-win32-arm64-msvc/-/atomic-natives-win32-arm64-msvc-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-win32-arm64-msvc/-/atomic-natives-win32-arm64-msvc-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"win32"
@@ -632,8 +632,8 @@
 			"optional": true
 		},
 		"node_modules/@bastani/atomic-natives-win32-x64-msvc": {
-			"version": "0.9.14-alpha.3",
-			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-win32-x64-msvc/-/atomic-natives-win32-x64-msvc-0.9.14-alpha.3.tgz",
+			"version": "0.9.14-alpha.5",
+			"resolved": "https://registry.npmjs.org/@bastani/atomic-natives-win32-x64-msvc/-/atomic-natives-win32-x64-msvc-0.9.14-alpha.5.tgz",
 			"license": "MIT",
 			"os": [
 				"win32"

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bastani/atomic",
-  "version": "0.9.14-alpha.3",
+  "version": "0.9.14-alpha.5",
   "description": "Atomic coding agent CLI with read, bash, edit, write tools and session management",
   "type": "module",
   "atomicConfig": {
@@ -83,7 +83,7 @@
     "prepublishOnly": "bun run clean && bun run build && bun run shrinkwrap"
   },
   "dependencies": {
-    "@bastani/atomic-natives": "0.9.14-alpha.3",
+    "@bastani/atomic-natives": "0.9.14-alpha.5",
     "@dbos-inc/dbos-sdk": "4.25.14",
     "@earendil-works/pi-agent-core": "^0.84.2",
     "@earendil-works/pi-ai": "^0.84.2",