@hassangameryt/freeflow 0.7.3 → 0.9.0

package/skills/{discuss → discuss-work}/references/discussion-continuity.md

@@ -1,8 +1,10 @@
 # Discussion Continuity
 
-Read this before preserving discussion across context loss, creating or updating durable task memory from discussion, or carrying a proposed or selected checkpoint beyond the current conversational context.
+Read this when discussion state, an emerging route or its Checkpoints must outlast the conversation: before preserving discussion across context loss, creating or updating a Working Record or another durable source from discussion, or carrying a proposed or selected Checkpoint beyond the current context.
 
-Discussion state is not automatically a Working Record, Spec, Plan, checkpoint, or authority. Preserve the required outcome separately from the current approach, and distinguish the user's execution agreement from proposals that have not been selected.
+This reference is judgment: adapt it to the discussion. Its binding statements restate [Discuss Work](../SKILL.md) or Track Work Rules.
+
+Discussion state is not automatically a Working Record, Spec, Plan, checkpoint, or authority. Preserve the required outcome separately from the current approach, and distinguish the user's work agreement from proposals that have not been selected.
 
 ## Keep It In Conversation When Possible
 
@@ -25,7 +27,7 @@ Prefer natural prose. Use structured fields only when they prevent ambiguity or
 
 Do not preserve a transcript, repeated rationale, rejected alternatives that no longer matter, or every discussion turn. Retain a rejected approach when losing its reason would cause the next context to repeat it.
 
-## Choose Durable Memory By Purpose
+## Choose A Durable Destination By Purpose
 
 Use a durable destination only when losing the state could cause later misalignment:
 
@@ -40,15 +42,7 @@ Do not create an artifact merely because discussion occurred. Prefer an existing
 
 ## Preserve Through Track Work
 
-Discuss owns exploration and meaning. [Track Work](../../track-work/SKILL.md) owns the Working Record and its lifecycle.
-
-```text
-Discuss
--> supported state worth preserving
--> Track Work performs one atomic update
--> Discuss confirms the preserved meaning
--> continue discussion or return to Workflow
-```
+Discuss Work owns exploration and meaning. [Track Work](../../track-work/SKILL.md) owns the Working Record and its lifecycle: preserve supported state in one atomic update through it, then confirm the preserved meaning.
 
 A Working Record is appropriate when:
 
@@ -60,41 +54,19 @@ A Working Record is appropriate when:
 
 Preserve only supported state that could affect later interpretation or action. Give each detailed fact one canonical owner. Track Work should store current meaning rather than append discussion chronology.
 
-When the initial record need arises from discussion, preserve the state that justified the record in the first authorized atomic update. Do not create an empty shell and lose the decisions, proposals, or uncertainty that made continuity necessary.
+When the initial record need arises from discussion, preserve the state that justified the record in the first authorized atomic update. Do not create an empty shell and lose the decisions, proposals, or uncertainty that made continuity necessary. Once a record exists, use it as working memory during the discussion, updating it at meaningful boundaries (accepted choices, open questions, options still considered, material the user supplies, the route as it emerges) as Track Work describes.
 
-When a rough route is useful, preserve provisional Slices and their actual dependencies without pretending the later implementation is settled. Keep the current unit concrete and later work directional. Use the existing Current Context, Current Work, Future Work, and History fields; do not add a second phase or context-cycle ledger.
+When a rough route is useful, preserve provisional Slices, the Checkpoints between them, and their actual dependencies without pretending the later implementation is settled. Keep the current Slice concrete and later work directional. Use the existing Current Context, Current Work, Future Work, and History fields; do not add a second phase or context-cycle ledger.
 
 Preserve the agreed outcome, scope, and user-facing return condition when losing them could interrupt covered work or permit drift. Authorization to investigate or propose a design is not acceptance of its production consequences. When learning changes the approach, retain the question, strongest evidence and limits, remaining uncertainty, artifact disposition, and affected later work—not the prototype's construction history.
 
-After preservation, return to Discuss unless Workflow established another route. Record mutation does not transfer ownership or authorize execution.
-
-## Carry Proposed And Selected Checkpoints
-
-A possible checkpoint remains proposed in the current discussion until Workflow selects it.
-
-For a proposed checkpoint, preserve only:
+After preservation, return to the discussion unless the user chose another route. Record mutation does not transfer ownership or authorize execution.
 
-```text
-Status: Proposed
-Type:
-Purpose:
-Boundary it could protect:
-Authority or selection still needed:
-```
+## Carry Checkpoints On The Route
 
-For a selected checkpoint, preserve:
+A commit, publication, review, user decision, or planned handoff that the discussion anticipates is a Checkpoint on the route, not a Slice. With a record, propose it in Future Work using Track Work's Checkpoint types and states: `proposed` while only anticipated, `pending` once the user selects it. Without a record, keep it in the conversation.
 
-```text
-Status: Selected
-Type:
-Purpose:
-Due boundary:
-Condition:
-Selection source:
-Authority scope:
-```
-
-Persist checkpoint state through Track Work only when durable memory is needed. Keep local commit authority separate from push or integration, and keep migration, release, and launch separately controlled.
+Keep local commit authority separate from push or publication, and keep migration, release, and launch separately controlled.
 
 ## Preserve Re-entry State
 
@@ -111,17 +83,8 @@ Authority or user decision needed:
 
 Do not restart from zero, rewrite unaffected decisions, or narrate completed work.
 
-After context loss, recover the complete record through Track Work's `full` view before relying on it. Compare current sections and relevant History with current user direction, accepted artifacts, and live evidence. Reconcile affected meaning and dependent future work; do not repeat a disproved approach or preserve a stale plan merely because the record contains it. With intact context, use `resume` or targeted reads only when needed.
+After context loss, recover the record as Track Work's "Recover After Context Loss" says before relying on it. Compare current sections and relevant History with current user direction, accepted artifacts, and live evidence. Reconcile affected meaning and dependent future work; do not repeat a disproved approach or preserve a stale plan merely because the record contains it. With intact context, use `resume` or targeted reads only when needed.
 
 ## Stop
 
 Stop preserving when the current meaning and one next useful action can be reconstructed safely.
-
-Do not turn continuity into:
-
-- a transcript;
-- a mandatory artifact;
-- a second source of authority;
-- a frozen architecture;
-- an automatic checkpoint;
-- a reason to begin execution.

package/skills/discuss-work/references/learning-action.md

@@ -0,0 +1,36 @@
+# Learning Action
+
+Read this when existing evidence cannot settle a material question and running something could: before proposing an experiment or prototype, while running one, and when its evidence comes back.
+
+This reference is judgment: adapt it to the question. Its binding statements restate [Discuss Work](../SKILL.md) or [Execute Work](../../execute-work/SKILL.md) Rules.
+
+## Decide Whether To Run One
+
+Use an experiment or prototype when existing evidence cannot settle a material question and exercising behavior can distinguish the remaining answers. A separate prototype should isolate the uncertainty more safely or economically than changing production; unfamiliarity alone does not require one.
+
+If the work is an accepted useful outcome regardless of what it teaches, treat it as delivery work, not an experiment; do not disguise implementation as learning. Incidental learning does not change its authority or acceptance boundary.
+
+## Frame It
+
+Frame the action before proposing it:
+
+- the question and its consequence for the accepted outcome or approach;
+- plausible answers and the observation that would distinguish them;
+- the smallest adequate prototype or observer;
+- permitted effects, environment, and production exclusion;
+- expected evidence and return conditions: question answered, observing limit reached, or further work requiring a different question or expanded scope;
+- what would justify discarding, revising, or proposing promotion.
+
+Add a time or cost boundary when repetition or setup could grow; do not demand an exact token prediction. If no adequate observer is available within scope, state the limitation rather than inventing a proof.
+
+Check that the agreement covers the framed action, preserve it through [Track Work](../../track-work/SKILL.md) when needed, then run it through Execute Work. Existing authority may already cover the experiment; its name does not create another approval gate.
+
+## Run It
+
+Answer the one declared question with the least building that can, within the limits the framing set. Report a negative or inconclusive result as plainly as a positive one, and do not polish a prototype toward a favorable answer. A successful demonstration is not production acceptance: never promote exploratory code into production unless that promotion is deliberately selected and its effects authorized.
+
+## Assess What It Showed
+
+Assess returned evidence against the original question. Negative or inconclusive evidence is a valid result; one failed technique does not prove every alternative impossible. When it reports a new prerequisite, tell a limitation of the observer or chosen approach from a requirement of the intended outcome before recommending more work.
+
+Revise only the affected understanding. When continuity matters, preserve the consequence for the task, the remaining uncertainty, and what happens to the exploratory artifacts, as [discussion continuity](discussion-continuity.md) describes. Stop assessing when the supported next direction or the unresolved decision is clear.

package/skills/discuss-work/scripts/choices.mjs

@@ -0,0 +1,101 @@
+#!/usr/bin/env node
+// Checks and renders Discuss Work's decision list (SKILL.md, "Build The Decision List"), so its format is a tool's job
+// and the agent's attention stays on the choices:
+//   validate   IDs, states, T -> P links, a default on every plain item, the user's words on agreed items, a reason on
+//              changed ones;
+//   agreement  the reply sent before building: What I'll do / Where I stop / Your decisions / Technical: N;
+//   handover   the items the hand-over shows: new, changed and default;
+//   catch-up   after an unattended run: defaults taken, choices waiting, changes, and what ran, from the Working
+//              Record's current Slice Material updates (read through Track Work's `working-record.mjs view resume`).
+// Usage: choices.mjs <validate|agreement|handover|catch-up> (--list <file|-> | --record <record.md>)
+//                    [--do <outcome> --stop <where you stop>] [--show-technical] [--json]
+// --list reads a file or stdin (no Working Record: the list lives in the conversation); --record reads the task's
+// sources/decisions.md beside the record. Each item is one line, "- P1 [state] choice -> default ..." (a technical
+// item "-> serves P1"); other lines are ignored. Every view validates first. Exit code 0 when the list is valid
+// (warnings do not fail it), 1 when it has errors, 2 on a usage error.
+import { spawnSync } from "node:child_process";
+import { existsSync, readFileSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+import { parseArgs } from "node:util";
+import { parseChoices, renderChoices } from "./lib/choices.mjs";
+
+const USAGE = [
+  "Usage: choices.mjs <validate|agreement|handover|catch-up> (--list <file|-> | --record <record.md>)",
+  "                   [--do <outcome> --stop <where you stop>] [--show-technical] [--json]",
+].join("\n");
+const VIEWS = new Set(["validate", "agreement", "handover", "catch-up"]);
+const WORKING_RECORD = resolve(dirname(fileURLToPath(import.meta.url)), "../../track-work/scripts/working-record.mjs");
+
+function fail(message) {
+  console.error(message);
+  process.exit(2);
+}
+
+let parsed;
+try {
+  parsed = parseArgs({
+    allowPositionals: true,
+    options: {
+      list: { type: "string" },
+      record: { type: "string" },
+      do: { type: "string" },
+      stop: { type: "string" },
+      "show-technical": { type: "boolean", default: false },
+      json: { type: "boolean", default: false },
+    },
+  });
+} catch (error) {
+  fail(`${error.message}\n${USAGE}`);
+}
+const { values, positionals } = parsed;
+const [view] = positionals;
+if (!VIEWS.has(view) || positionals.length !== 1) fail(USAGE);
+if (Boolean(values.list) === Boolean(values.record)) fail(`give --list <file|-> or --record <record.md>\n${USAGE}`);
+if (view === "agreement" && (!values.do || !values.stop)) fail(`agreement needs --do and --stop\n${USAGE}`);
+
+let listText;
+let recordView = null;
+if (values.record) {
+  const record = resolve(values.record);
+  if (!existsSync(record)) fail(`no such record: ${values.record}`);
+  const listPath = join(dirname(record), "sources", "decisions.md");
+  if (!existsSync(listPath)) fail(`no decision list at ${listPath}; Track Work keeps it there`);
+  listText = readFileSync(listPath, "utf8");
+  if (view === "catch-up") {
+    const run = spawnSync(process.execPath, [WORKING_RECORD, "view", "resume", "--record", record], {
+      encoding: "utf8",
+      maxBuffer: 64 * 1024 * 1024,
+    });
+    if (run.status !== 0) fail(`working-record.mjs view resume failed: ${(run.stderr || run.stdout).trim()}`);
+    recordView = run.stdout;
+  }
+} else if (values.list === "-") listText = readFileSync(0, "utf8");
+else if (!existsSync(resolve(values.list))) fail(`no such list: ${values.list}`);
+else listText = readFileSync(resolve(values.list), "utf8");
+
+const list = parseChoices(listText);
+const notes = [...list.errors.map((each) => `error: ${each}`), ...list.warnings.map((each) => `warning: ${each}`)];
+if (!list.ok) {
+  if (values.json)
+    console.log(
+      JSON.stringify({ command: `choices ${view}`, ok: false, errors: list.errors, warnings: list.warnings }, null, 2),
+    );
+  else console.log(`Decision list has ${list.errors.length} error(s):\n${notes.map((each) => `- ${each}`).join("\n")}`);
+  process.exit(1);
+}
+const rendered = renderChoices(view, list, {
+  outcome: values.do,
+  stop: values.stop,
+  showTechnical: values["show-technical"],
+  recordText: recordView,
+});
+if (values.json)
+  console.log(
+    JSON.stringify(
+      { command: `choices ${view}`, ok: true, warnings: list.warnings, ...rendered.data, text: rendered.text },
+      null,
+      2,
+    ),
+  );
+else console.log(`${notes.length ? `${notes.map((each) => `- ${each}`).join("\n")}\n\n` : ""}${rendered.text}`);

package/skills/discuss-work/scripts/lib/choices.mjs

@@ -0,0 +1,159 @@
+// Discuss Work's decision list as a record file: the task's sources/decisions.md, or a list given on stdin or as a file
+// when there is no Working Record. One line per item, "- P1 [state] choice -> default ...", other lines ignored
+// (Discuss Work, "Build The Decision List").
+//   validate   IDs, states, T -> P links, a default on every plain item, the user's words on agreed items, a reason on
+//              changed ones;
+//   agreement  the reply sent before building: What I'll do / Where I stop / Your decisions / Technical: N;
+//   handover   the items a hand-over shows: new, changed and default;
+//   catch-up   after an unattended run: defaults taken, choices waiting, changes, and what ran (the record's current
+//              Slice Material updates).
+
+export const CHOICE_STATES = new Set(["proposed", "agreed", "default", "changed", "new", "dropped"]);
+const ITEM = /^\s*[-*]\s+([A-Za-z]+\d+)\s+\[([^\]]*)\]\s+(.*)$/;
+
+export function parseChoices(text) {
+  const items = [];
+  const errors = [];
+  const warnings = [];
+  text.split(/\r?\n/).forEach((raw, index) => {
+    const match = ITEM.exec(raw);
+    if (!match) return;
+    const [, id, state, rest] = match;
+    const arrow = rest.indexOf("->");
+    const choice = (arrow === -1 ? rest : rest.slice(0, arrow)).trim();
+    const tail = arrow === -1 ? "" : rest.slice(arrow + 2).trim();
+    // A technical item states its choice and the plain item it serves: "T1 [default] FIFO array -> serves P1".
+    const serves = /^(?:serves\s+)?(P\d+)\b/i.exec(tail);
+    items.push({
+      line: index + 1,
+      id,
+      kind: id[0] === "P" ? "plain" : id[0] === "T" ? "technical" : "unknown",
+      state,
+      choice,
+      default: serves ? choice : tail,
+      serves: serves?.[1] ?? null,
+      quote: /\bYou:\s*["“]/.test(rest),
+      reason: /\b(because|why|was|since|instead|reason)\b/i.test(tail),
+      text: rest.trim(),
+    });
+  });
+
+  const seen = new Map();
+  for (const item of items) {
+    const where = `line ${item.line} ${item.id}`;
+    if (!/^[PT][1-9]\d*$/.test(item.id)) errors.push(`${where}: ID must be P<n> or T<n>`);
+    if (seen.has(item.id)) errors.push(`${where}: duplicate ID (first at line ${seen.get(item.id)})`);
+    else seen.set(item.id, item.line);
+    if (!CHOICE_STATES.has(item.state))
+      errors.push(`${where}: state "${item.state}" is not one of ${[...CHOICE_STATES].join(", ")}`);
+    if (!item.choice) errors.push(`${where}: names no choice`);
+    if (item.kind === "plain" && !item.default)
+      errors.push(`${where}: has no default after "->" (what matches existing behavior and is easiest to reverse)`);
+    if (item.kind === "technical" && !item.serves)
+      errors.push(`${where}: a technical item names the plain item it serves ("-> serves P1")`);
+    if (item.state === "agreed" && !item.quote) warnings.push(`${where}: agreed without the user's words (You: "...")`);
+    if (item.state === "changed" && !item.reason) warnings.push(`${where}: changed without saying why`);
+  }
+  for (const item of items.filter((each) => each.serves)) {
+    const target = items.find((each) => each.id === item.serves);
+    if (!target) errors.push(`line ${item.line} ${item.id}: serves ${item.serves}, which is not in the list`);
+    else if (target.state === "dropped" && item.state !== "dropped")
+      warnings.push(`line ${item.line} ${item.id}: serves ${item.serves}, which is dropped`);
+  }
+  if (!items.length) errors.push('no items found: each item is a line "- P1 [state] choice -> default"');
+  const firstTechnical = items.findIndex((item) => item.kind === "technical");
+  if (firstTechnical !== -1 && items.slice(firstTechnical).some((item) => item.kind === "plain"))
+    warnings.push("plain items come first; a P item follows a T item");
+  return { items, errors, warnings, ok: errors.length === 0 };
+}
+
+const itemLine = (item) => `- ${item.id} [${item.state}] ${item.text}`;
+const byState = (items, states) => items.filter((item) => states.includes(item.state));
+
+// The current Slice's Material updates: what ran or changed while no one was answering.
+function currentSliceUpdates(recordText) {
+  const lines = recordText.split(/\r?\n/);
+  const start = lines.findIndex((each) => /^### Current Slice\s*$/.test(each));
+  if (start === -1) return { slice: "no Current Slice", updates: [] };
+  const end = lines.findIndex((each, index) => index > start && /^#{2,3} /.test(each));
+  const slice = lines.slice(start, end === -1 ? undefined : end);
+  const title = slice.find((each) => /^#### S-\d+/.test(each));
+  const at = slice.findIndex((each) => /^##### Material updates\s*$/.test(each));
+  const updates = at === -1 ? [] : slice.slice(at + 1).filter((each) => /^\s*-\s+\S/.test(each));
+  return {
+    slice: title ? title.replace(/^####\s+/, "") : "no Current Slice",
+    updates: updates.map((each) => each.trim().replace(/^-\s+/, "")),
+  };
+}
+
+// Renders one view; returns { text, data }. The caller validates first.
+export function renderChoices(view, parsed, { outcome, stop, showTechnical = false, recordText = null } = {}) {
+  const { items } = parsed;
+  const plain = items.filter((item) => item.kind === "plain" && item.state !== "dropped");
+  const technical = items.filter((item) => item.kind === "technical" && item.state !== "dropped");
+  if (view === "validate")
+    return { text: `Decision list valid: ${items.length} item(s).`, data: { items: items.length } };
+  if (view === "agreement") {
+    const open = plain.filter((item) => item.state !== "agreed");
+    const pad = "                ";
+    const decisions = open.length
+      ? open.map(
+          (item, index) =>
+            `${index ? pad : ""}${index + 1}. ${item.choice}${/[?.!]$/.test(item.choice) ? "" : "?"} I'd go with ${item.default}`,
+        )
+      : ["none: the request and the repository settle every choice the user would see."];
+    const technicalLine = showTechnical
+      ? [
+          `Technical:      ${technical.map((item, index) => `${index ? pad : ""}${item.id} ${item.choice} (serves ${item.serves})`).join("\n") || "none"}`,
+        ]
+      : technical.length
+        ? [
+            `Technical:      ${technical.length} more choice${technical.length === 1 ? "" : "s"}; say "show technical" to see them.`,
+          ]
+        : [];
+    const text = [
+      `What I'll do:   ${outcome}`,
+      `Where I stop:   ${stop}`,
+      `Your decisions: ${decisions.join("\n")}`,
+      ...technicalLine,
+    ].join("\n");
+    return { text, data: { decisions: open.map((item) => item.id), technical: technical.length } };
+  }
+  if (view === "handover") {
+    const shown = byState(items, ["new", "changed", "default"]);
+    return {
+      text: shown.length
+        ? `Choices you have not seen (${shown.length}):\n${shown.map(itemLine).join("\n")}`
+        : "No new, changed or default choices: every item was agreed or dropped.",
+      data: { shown: shown.map((item) => item.id) },
+    };
+  }
+  // catch-up
+  const defaults = byState(items, ["default"]);
+  const waiting = byState(items, ["proposed", "new"]);
+  const changed = byState(items, ["changed"]);
+  const out = [`Defaults taken (${defaults.length}):`, ...(defaults.length ? defaults.map(itemLine) : ["- none"])];
+  out.push(
+    "",
+    `Choices waiting for you (${waiting.length}):`,
+    ...(waiting.length ? waiting.map(itemLine) : ["- none"]),
+  );
+  if (changed.length) out.push("", `Changed while you were away (${changed.length}):`, ...changed.map(itemLine));
+  const ran = recordText === null ? null : currentSliceUpdates(recordText);
+  if (ran)
+    out.push(
+      "",
+      `What ran (${ran.slice}, Material updates: ${ran.updates.length}):`,
+      ...(ran.updates.length ? ran.updates.map((each) => `- ${each}`) : ["- nothing recorded"]),
+    );
+  else out.push("", "What ran: no Working Record; account for it from the conversation.");
+  return {
+    text: out.join("\n"),
+    data: {
+      defaults: defaults.map((item) => item.id),
+      waiting: waiting.map((item) => item.id),
+      changed: changed.map((item) => item.id),
+      ...(ran ? { slice: ran.slice, ran: ran.updates } : {}),
+    },
+  };
+}

package/skills/evaluate-behavior/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: evaluate-behavior
+description: "Use when testing, evaluating or comparing how an agent behaves: whether a skill triggers, whether a change to a prompt, skill, model, setting or extension changes outcomes, measuring a reviewer or checker, or benchmarking agents on coding tasks, conversations or pipelines. Also use when writing or checking an eval's JSON files, running an eval and reading its results, and deciding what a result can claim, how many cases and trials it needs and what it costs."
+---
+
+# Evaluate Behavior
+
+Determine whether a declared change to an agent's setup changes its behavior as intended under declared pressure. Evidence outranks confident prose.
+
+A **configuration** is everything that can shape the agent's behavior: host, model, skills, instruction files, and host extensions and settings. A **group** asks one behavioral question by running named **arms** of one configuration on the same task, usually `baseline` (the **reference**: the setup without the change, such as no target skill, the previous commit, or the setting off) and `candidate` (the same setup with exactly the declared change, its **component**); a single arm measures one setup alone. The **subject** is the agent performing the task in one arm. The **evaluator** isolates subjects and preserves canonical evidence; the **deterministic grader** derives fixed facts after the run is saved; a **judge** is a separate model whose answers are **judged evidence**. A **suite** runs ordered groups and may declare a **decision** rule that turns their results into a verdict.
+
+Evaluate Behavior does not change what it evaluates, claim readiness, or choose adoption. The component's owner revises it: [Write Skill](../write-skill/SKILL.md) for a skill or other instruction text; extension code goes to its owner, and a model, setting or extension choice to the user.
+
+The **Rules** below are binding; if one cannot be met, stop and say why. Everything else is **judgment**: follow its stated goal, and adapt when a different choice serves that goal better.
+
+## Routes
+
+| Situation | Go to |
+| --- | --- |
+| The question, evidence class, arms, pressure or component; what the result can claim, the decision rule, cases and trials, noise, judge counts, cost | [design and measurement](references/design-and-measurement.md) |
+| Measuring a reviewer, checker, or other detector | its [Defect Suites](references/design-and-measurement.md#defect-suites) |
+| Writing or checking group, suite, or configuration JSON | [authoring](references/authoring.md) |
+| A coding task (hidden tests, declared commands, a shell, a container, network), or the Claude Code host | [coding tasks](references/coding-tasks.md) |
+| A conversation, a pipeline, a rubric and judges, or a runtime profile | [conversations, pipelines and judges](references/conversations-pipelines-judges.md) |
+| Running the evaluator and reading its results; evidence that conflicts, the failed boundary, a revision target, or rerun scope | [running and evidence](references/running-and-evidence.md), and `skill-eval --help` |
+| Reviewing a definition before paying for its runs | Review Work's [Eval Definition Review](../review-work/references/evals-and-judges.md#eval-definition-review) |
+| Choosing a judge or relying on its answers | Review Work's [Judge Contract](../review-work/references/evals-and-judges.md#judge-contract) |
+
+## Rules
+
+- Never mutate canonical evidence or erase an unfavorable result. Preserve partial and infrastructure evidence without converting unavailable behavior into pass or fail.
+- Never fabricate behavioral evidence. Missing evidence stays unavailable.
+- Compare a change only against an exact immutable baseline, such as a commit. If none exists, preserve the current state first (for a skill or instruction text, through Write Skill); never create a baseline by editing what is being evaluated.
+- Keep deterministic criteria, grader files, review questions and rubric references out of subject prompts, the fixture and declared subject context.
+- Declare every part of the configuration that can change behavior. Ambient installation, settings, or context are not declared composition.
+- Claim only what the evidence class observed; one class never proves another.
+- Never write that one arm beats another without paired runs on the same cases and a suite `decision` declared before the run; the evaluator reports "beats" only from that rule. Below that, report counts.
+- Never cite a dry run as evidence. When a run is refused for uncommitted inputs, commit the candidate and the definition, or keep the run as a dry run for your own checking only.
+- Never change the component under test while evaluating it. Return one measured revision target to its owner.
+- After a behavioral change, rerun the complete fixed group with every arm. Never hand-pick an earlier run as the fresh counterpart or rerun only the preferred arm; an earlier run may stand in only through `--reuse`, which matches the arm's exact inputs, and the report names it as reused.
+- Judged evidence is its own class. Judge answers are stored apart from run and deterministic grade files, with each judge's identity and calibration status, and count toward a claim only through a decision rule declared before the run. They never change run or grade files and never establish readiness.
+- Installing anything an eval needs, such as the VM runtime behind a confined shell, is the user's decision. Relay the evaluator's exact refusal and ask before installing; never install silently.
+- Never invoke archived evaluators.
+- Never clear a run's transcript-scan flag on your own judgment. Show the user the flag and its evidence, and run `skill-eval clear` only with their decision and their reason.
+
+## Design The Smallest Valid Comparison
+
+Goal: the arms differ only in the component, under pressure strong enough to show it, in a run no larger than the claim needs.
+
+Answer from saved evidence or an existing group when they still fit (the same question, pressure, arms and fixed criteria); for a draft-only request, use Write Skill. Do not rerun merely to demonstrate process. Name one behavioral question, its evidence class, and the strongest claim the result must support: can do X, a rate on this suite, or A beats B; readiness is beyond Evaluate Behavior. Activation, first-read body and nearby behavior are questions about skills; for other components, check delivery, behavior and artifact outcome first.
+
+Choose the component:
+
+| Change under test | Component | Baseline |
+| --- | --- | --- |
+| A new or revised skill | `subject.skills`, `subject.target`, `subject.source` | no target, or the previous commit |
+| An instruction or context file | `subject.context` | without the file, or its previous commit |
+| A model or thinking level | `model.model`, `model.thinking` | the model in use |
+| An extension or plugin, or a host setting | `host.extensions`, `host.settings` | without it, or its current version or value |
+
+The task's fixture and the runtime are shared by every arm and cannot be the component.
+
+Choose the mode by where the behavior happens; modes combine:
+
+- **Compare:** arms run the same task once per trial; the default.
+- **Noise check:** `component: []` with identical arms; their gap is the floor a real difference must clear.
+- **Measure:** one arm; a rate on this suite, never a comparison.
+- **Conversation:** a simulated user who knows only its card answers back.
+- **Pipeline:** ordered stages, each a subject that sees only what it declares.
+- **Coding task:** hidden tests or other commands grade the changed repository.
+- **Judged evaluation:** judges answer a rubric for each run alone, blind to its arm.
+
+Before subject output exists:
+
+1. fix one behavioral question;
+2. fix deterministic expectations, rubric and review questions;
+3. preserve the earliest natural prompt or strongest pressure capable of distinguishing behavior;
+4. declare each arm's exact identity, and the suite's decision rule when the claim is a comparison;
+5. keep prompt or turns, fixture, tools, and every configuration part other than the component fixed, along with the criteria;
+6. run one group or an ordered suite;
+7. inspect canonical evidence and deterministic facts before judging meaning;
+8. classify the failed boundary before proposing revision.
+
+If every arm passes, the pressure may be weak or the baseline already sufficient. If every arm fails, distinguish component, fixture, dependency, environment, host, isolation, and criterion failures before revising anything.
+
+## Run And Read The Evidence
+
+Run the canonical [skill evaluator](scripts/skill-eval.mjs) from the definition root: `node <evaluate-behavior-directory>/scripts/skill-eval.mjs <run|check-tasks|view|review|clear> ...`. Run `check-tasks`, which calls no model, before spending on a coding task; use `--trials <n>` before claiming a difference, and `--max-cost` when a batch could spend more than intended. Canonical results live under `<definition-root>/.skill-eval/runs/<result-id>`; a view is a projection of them. A failed check is behavioral evidence; a transition such as `fail-to-pass` is a fact, not a verdict.
+
+## Report And Stop
+
+Report:
+
+- what ran;
+- each arm's exact identity, and which is the reference;
+- selected groups and arms, and any run reused from an earlier result;
+- deterministic results, with case and trial counts and pass^k;
+- metrics that bear on the question, such as cost, turns or duration per arm;
+- the decision's verdict, with its split, p and rule, when the suite declares one, and otherwise the claim class reached;
+- judged evidence apart from deterministic results, with each judge's identity and calibration status;
+- infrastructure failures, invalid runs with their reasons, runs held for uncleared transcript flags, cancellation, or unavailable evidence;
+- viewed scope and canonical result paths;
+- what the evidence proves;
+- what it does not prove;
+- any supported failed boundary or revision target.
+
+Stop rather than overclaim when required delivery, activation, first-read, composition, retained-use, isolation, artifact, or host evidence is unavailable. The user decides whether to revise, adopt, or reject the change.